Model types

What a model's architecture needs to run correctly, and how to add or change it.

Some architectures need more than their model card says. gpt-oss needs vocab files on disk and parsers for its harmony format, and Qwen3.5's chat template raises an error on a system message after the first turn. llm-helper keeps this as data, one file per HuggingFace model_type in catalogue/model-types/. new and the panel read the repo's config.json, find the file for its model_type, and apply what it says for the engine the profile runs on. A GGUF repo is matched by the model_type of the model it was quantized from.

A model type applies to every repo of that architecture, including finetunes and re-uploads. A launch configuration for one specific repo, run and measured on named hardware, is a recipe.

llm-helper model-types list shows the model types in effect, and llm-helper model-types show <id> shows what one does.

Fields

{
  "schema_version": 1,
  "id": "gpt_oss",
  "display_name": "gpt-oss",
  "why": "Why this architecture needs anything at all.",
  "engines": {
    "vllm": {
      "files": [
        {
          "dir": "tiktoken-encodings",
          "env": "TIKTOKEN_ENCODINGS_BASE",
          "base_url": "https://openaipublic.blob.core.windows.net/encodings/",
          "names": ["o200k_base.tiktoken", "cl100k_base.tiktoken"],
          "why": "What fails without them."
        }
      ],
      "args": [
        {
          "flag": "--tool-call-parser",
          "values": ["openai", "openai_gptoss"],
          "with": ["--enable-auto-tool-choice"],
          "why": "What fails without it."
        }
      ],
      "chat_template_fixes": [
        {
          "contains": "text that marks the broken template",
          "match": "a regular expression",
          "replace": "its replacement, with ${1} for groups",
          "why": "What fails without it."
        }
      ]
    }
  }
}

• id is the model_type value, and the file is named after it.
• engines is keyed by engine id. An engine with no entry gets nothing added. files apply on any engine. args and chat_template_fixes are in vLLM's flag syntax, so they are only accepted for engines with the vllm_args capability.
• files are fetched into dir under the data directory when a profile is created, unless already there. With env set, the profile gets that variable pointing at the directory.
• args are added to the profile's EXTRA_ARGS unless the card already named the flag. values are tried in order and the first the installed engine registers is used; if none is registered, the flag is left out. with flags are added alongside.
• chat_template_fixes apply when the repo's chat_template.jinja contains contains. The patched copy is saved under the data directory and passed with --chat-template. If the text is there but match does not match, nothing is patched and new says so.

Each file is checked when it loads: an unknown field, an unknown engine, args or chat_template_fixes for an engine without vllm_args, a bad regular expression, a dir outside the data directory or a base_url that is not https is refused.

Changing one locally

Put a file in ~/.config/vllm-helper/model-types.d/, named after its model_type. A new id adds a model type, and a shipped id replaces the shipped file, so you can change what llm-helper does for an architecture without rebuilding. A file that fails to load is skipped, and doctor and model-types list say why.