Smart Router

The Smart Router is a virtual model name (smart_router) that automatically classifies incoming requests by complexity and forwards them to the appropriate target model. You do not need to select a model in your application; instead, send all requests to the same model name.

How It Works

The Smart Router analyzes the latest user message across five dimensions:

Dimension	Weight	Measures
Code	30%	Code, debugging, programming terms
Reasoning	25%	Analytical and multi-step reasoning
Technical terms	25%	Architecture, infrastructure, operations
Word count	10%	Prompt length
Simple indicators	-5%	Greetings, definitions, straightforward asks

The weighted score maps to a complexity tier:

Tier	Score range	Typical requests
Simple	< 0.15	Greetings, simple lookups
Medium	0.15 to 0.35	General questions, short explanations
Complex	0.35 to 0.60	Technical questions, code help, multi-step tasks
Reasoning	>= 0.60	Analysis, architecture decisions, root-cause investigation

A reasoning override applies when two or more reasoning keywords are detected in the latest user message: the tier is forced to Reasoning regardless of the numeric score.

Target Models

Each tier is mapped to a fixed target model:

Tier	Target model
Simple	Gemma 4 31B
Medium	Gemma 4 31B
Complex	GPT-OSS 120B
Reasoning	GLM 5.2

Simple and medium-complexity requests are served by the fast Gemma 4 model. Complex requests with code and technical signals go to the larger GPT-OSS 120B. For deep reasoning or when reasoning keywords trigger an override, GLM 5.2 is used, which generates chain-of-thought intermediate steps.

Usage

Send smart_router as the model name in your API request:

{
  "model": "smart_router",
  "messages": [
    { "role": "user", "content": "Hello, how are you?" }
  ]
}

The response includes extra_fields.routing_info showing which model was actually used, so you can trace where the request was routed:

{
  "extra_fields": {
    "routing_info": {
      "provider": "vllm",
      "model": "release/gemma-4-31b-it",
      "key": "gemma-4-31b-it [0]"
    },
    "resolved_model_used": "release/gemma-4-31b-it"
  }
}

Multilingual Classification

The Smart Router keyword lists are maintained in both German and English. German prompts such as “Schritt fuer Schritt beweisen” or “Ursachenanalyse” are recognized just like their English counterparts (“step by step”, “root cause analysis”). Classification runs deterministically in-process with sub-millisecond latency and no external API calls.

Tips

• Simple prompts (greetings, short questions) are automatically routed to the affordable Gemma 4 model.
• Code and architecture questions with sufficient technical signals reach the GPT-OSS 120B model.
• Explicit reasoning phrases like “step by step”, “prove”, “root cause analysis”, or “explain your reasoning” trigger the Reasoning tier and thus GLM 5.2.
• Short prompts without code, technical, or reasoning signals stay on the Simple or Medium tier, even if the task is conceptually demanding. Add relevant keywords to the prompt if needed.