Skip to content
HALOWERK

Denkwerk

rag metrics

All metrics are defined in the response. With relevant_ids: precision@k and recall@k over the first k contexts, MRR, average precision and nDCG@k with binary relevance and log2 discount. Without relevant_ids, a context counts as relevant when it supports the reference (or the question if no reference) at or above the threshold; context precision = mean of precision@i over relevant positions (rank-weighted, as in RAGAS but without an LLM judge). Context recall: share of reference sentences.

POST https://denk.halowerk.com/v1/rag-metrics

Inputs

Fields of the published input schema
FieldTypeRequiredDescription
questionstringYesUser question.
answerstringYesGenerated answer.
contextsarrayYesRetrieved contexts in rank order: strings or objects {id, text}.
referencestringNoGround-truth answer (enables context recall and correctness).
relevant_idsarrayNoIds (or 0-based positions as strings) of the truly relevant contexts.
kintegerNo
methodstringNo
languagestringNo
lexical_thresholdnumberNo
semantic_thresholdnumberNoCosine threshold for intfloat/multilingual-e5-small (heuristic; related pairs typically score 0.80-0.90, unrelated 0.65-0.75).
Show full input schema
{
  "type": "object",
  "additionalProperties": false,
  "required": [
    "question",
    "answer",
    "contexts"
  ],
  "properties": {
    "question": {
      "type": "string",
      "minLength": 1,
      "maxLength": 5000,
      "description": "User question."
    },
    "answer": {
      "type": "string",
      "minLength": 1,
      "maxLength": 20000,
      "description": "Generated answer."
    },
    "contexts": {
      "type": "array",
      "minItems": 1,
      "maxItems": 50,
      "description": "Retrieved contexts in rank order: strings or objects {id, text}.",
      "items": {
        "anyOf": [
          {
            "type": "string",
            "minLength": 1,
            "maxLength": 20000
          },
          {
            "type": "object",
            "additionalProperties": false,
            "required": [
              "text"
            ],
            "properties": {
              "id": {
                "type": "string",
                "maxLength": 128
              },
              "text": {
                "type": "string",
                "minLength": 1,
                "maxLength": 20000
              }
            }
          }
        ]
      }
    },
    "reference": {
      "type": "string",
      "minLength": 1,
      "maxLength": 20000,
      "description": "Ground-truth answer (enables context recall and correctness)."
    },
    "relevant_ids": {
      "type": "array",
      "maxItems": 50,
      "uniqueItems": true,
      "items": {
        "type": "string",
        "maxLength": 128
      },
      "description": "Ids (or 0-based positions as strings) of the truly relevant contexts."
    },
    "k": {
      "type": "integer",
      "minimum": 1,
      "maximum": 50,
      "default": 5
    },
    "method": {
      "type": "string",
      "enum": [
        "lexical",
        "embedding",
        "both"
      ],
      "default": "lexical"
    },
    "language": {
      "type": "string",
      "enum": [
        "auto",
        "de",
        "en"
      ],
      "default": "auto"
    },
    "lexical_threshold": {
      "type": "number",
      "minimum": 0.1,
      "maximum": 1,
      "default": 0.6
    },
    "semantic_threshold": {
      "type": "number",
      "minimum": 0.5,
      "maximum": 1,
      "default": 0.8,
      "description": "Cosine threshold for intfloat/multilingual-e5-small (heuristic; related pairs typically score 0.80-0.90, unrelated 0.65-0.75)."
    }
  }
}

Open original schema

Published input example

{
  "question": "Where is the Eiffel Tower?",
  "answer": "The Eiffel Tower is in Paris.",
  "contexts": [
    "The Eiffel Tower is a landmark in Paris, completed in 1889.",
    "Berlin is the capital of Germany."
  ]
}

Result & documentation

Machine-readable documentation of this hub

Source of this page: the published HALOWERK catalog. Updated: 2026-11-17 UTC.

More endpoints in this hub

All endpoints in this hub โ†’