Denkwerk
near dup remove
Each document becomes a set of shingles (word n-grams after Unicode word segmentation and lower casing, or character n-grams). MinHash signatures use one 32-bit FNV-1a hash per shingle and num_perm deterministic MurmurHash3-finalizer permutations; locality-sensitive hashing with b bands of r rows (the largest r for which a pair exactly at the threshold is missed with at most 5 % probability) proposes candidate pairs, which are then verified with the exact Jaccard similarity of the shingle.
POST https://denk.halowerk.com/v1/near-dup-removeInputs
| Field | Type | Required | Description |
|---|---|---|---|
| documents | array | Yes | |
| threshold | number | No | Jaccard similarity at or above which documents count as duplicates. |
| shingle | string | No | |
| n | integer | No | |
| num_perm | integer | No | |
| keep | string | No |
Show full input schema
{
"type": "object",
"additionalProperties": false,
"required": [
"documents"
],
"properties": {
"documents": {
"type": "array",
"minItems": 2,
"maxItems": 3000,
"items": {
"type": "object",
"additionalProperties": false,
"required": [
"text"
],
"properties": {
"id": {
"type": "string",
"maxLength": 128
},
"text": {
"type": "string",
"minLength": 1,
"maxLength": 200000
}
}
}
},
"threshold": {
"type": "number",
"minimum": 0.1,
"maximum": 1,
"default": 0.8,
"description": "Jaccard similarity at or above which documents count as duplicates."
},
"shingle": {
"type": "string",
"enum": [
"word",
"char"
],
"default": "word"
},
"n": {
"type": "integer",
"minimum": 1,
"maximum": 20,
"default": 3
},
"num_perm": {
"type": "integer",
"minimum": 16,
"maximum": 512,
"default": 128
},
"keep": {
"type": "string",
"enum": [
"first",
"longest"
],
"default": "first"
}
}
}Published input example
{
"documents": [
{
"id": "a",
"text": "Die Lieferung erfolgt innerhalb von drei Werktagen nach Zahlungseingang."
},
{
"id": "b",
"text": "Die Lieferung erfolgt innerhalb von drei Werktagen nach Zahlungseingang!"
}
],
"threshold": 0.8
}Result & documentation
Machine-readable documentation of this hub
Source of this page: the published HALOWERK catalog. Updated: 2026-11-17 UTC.
More endpoints in this hub
- Denkwerk0.003 USDC
agent card generate
POSThttps://denk.halowerk.com/v1/agent-card-generateBuilds an Agent2Agent protocol Agent Card with protocolVersion, name, description, url, version, capabilities, default input/output modes and skills. Skills come from the supplied list or are derived from OpenAPI operations (operationId, su
- Denkwerk0.003 USDC
agent permission diff
POSThttps://denk.halowerk.com/v1/agent-permission-diffTakes the tool names (with optional descriptions) and permission scopes of an agent before and after a change and reports added and removed tools and scopes, classifies new capabilities by risk (payments, sending messages, deleting or writi
- Denkwerk0.003 USDC
annotation guide draft
POSThttps://denk.halowerk.com/v1/annotation-guide-draftTakes example texts per label and derives, without a language model, the most distinguishing terms per label (TF-IDF contrast against the other labels), representative examples, cross-label example pairs with high similarity that need a wri