Token Types and Billing
A token is a text building block, roughly a word fragment. As a rule of thumb, about 1,000 tokens ≈ 750 words.
Four Token Types
Section titled “Four Token Types”| Token Type | Description | Billing |
|---|---|---|
| Input Token | Text the customer sends to the API (prompts, system instructions, RAG context) | Fully billable |
| Cached Input Token | Input tokens with an existing KV cache (prefix caching) | Billed at a reduced rate |
| Reasoning Token | Token for the thinking process (chain-of-thought), delivered to the customer | Fully billable (same rate as output) |
| Output Token | Response generated by the model | Fully billable (higher rate) |
KV Cache Prefix Caching
Section titled “KV Cache Prefix Caching”The KV cache resides volatilely in VRAM/RAM. It is not a persistent cache, the states are released as soon as processing completes. See Prefix Caching for details.
AI Points
Section titled “AI Points”Billing is usage-based (pay-as-you-grow) in AI points. A precommit is possible and economically attractive.
The specific AI points per model and an interactive consumption calculator are available in the AI points calculator.
Embedding and reranker models (Harrier, BGE) are billed exclusively via input tokens; cached input and output do not apply. Details on the models are available in the model overview.