Skip to content
HALOWERK

Denkwerk

token count multi

Exact counts, not estimates: OpenAI encodings via js-tiktoken, other families via their published tokenizer.json (Hugging Face, pinned revisions, SHA-256 checked, open licenses only: Apache-2.0 or MIT) run with transformers.js. Every Hugging Face tokenizer was checked token by token against the Rust reference implementation on a 29-text corpus. Llama and Gemma tokenizers are excluded for licensing reasons. Special-token strings in the text are counted as each tokenizer treats them by default.

POST https://denk.halowerk.com/v1/token-count-multi

Inputs

Fields of the published input schema
FieldTypeRequiredDescription
textstringYesUTF-8 text.
tokenizersarrayNoSubset of tokenizers; default all.
Show full input schema
{
  "type": "object",
  "additionalProperties": false,
  "required": [
    "text"
  ],
  "properties": {
    "text": {
      "type": "string",
      "minLength": 1,
      "maxLength": 200000,
      "description": "UTF-8 text."
    },
    "tokenizers": {
      "type": "array",
      "minItems": 1,
      "maxItems": 12,
      "uniqueItems": true,
      "items": {
        "type": "string",
        "enum": [
          "o200k_base",
          "cl100k_base",
          "p50k_base",
          "r50k_base",
          "qwen2.5",
          "deepseek-v3",
          "mistral-tekken",
          "falcon",
          "gpt-neox",
          "yi-1.5",
          "granite-3.3",
          "glm-4.5"
        ],
        "description": "Tokenizer id; see supported_tokenizers in the token-count-multi result."
      },
      "description": "Subset of tokenizers; default all."
    }
  }
}

Open original schema

Published input example

{
  "text": "Die Rechnung ist innerhalb von 14 Tagen ohne Abzug zu bezahlen.",
  "tokenizers": [
    "o200k_base",
    "qwen2.5",
    "mistral-tekken"
  ]
}

Result & documentation

Machine-readable documentation of this hub

Source of this page: the published HALOWERK catalog. Updated: 2026-11-17 UTC.

More endpoints in this hub

All endpoints in this hub โ†’