Zum Inhalt
HALOWERK

Agent Tools

abtest / evaluate

Reads one conversion experiment two ways at once. Frequentist: two-proportion test, p value, confidence interval on the absolute lift, and the sample size the requested minimum detectable effect needs at the requested power. Bayesian: Beta-Binomial posteriors, probability the variant is better, expected loss either way, credible intervals - closed form, not sampled, so the numbers repeat exactly. looks_taken carries the adjusted threshold and the rate you actually ran at.

POST https://tools.halowerk.com/v1/abtest/evaluate

Eingaben

Felder des veröffentlichten Eingabeschemas
FeldTypPflichtBeschreibung
controlobjectJaBaseline group. conversions may not exceed visitors.
variantobjectJaGroup under test. conversions may not exceed visitors.
confidence_levelnumberNein
alternativestringNeintwo_sided asks whether the groups differ, greater whether the variant is above the control.
minimum_detectable_effect_relnumberNeinRelative effect the test should be able to find, e.g. 0.1 for ten percent above the control rate.
powernumberNeinProbability of finding that effect if it is there.
looks_takenintegerNeinHow often the running test has been evaluated. Above 1 the threshold is adjusted and the real false positive rate is reported.
prior_alphanumberNeinBeta prior, successes. 1 with prior_beta 1 is uniform.
prior_betanumberNeinBeta prior, failures.
Vollständiges Eingabeschema anzeigen
{
  "type": "object",
  "required": [
    "control",
    "variant"
  ],
  "properties": {
    "control": {
      "type": "object",
      "required": [
        "visitors",
        "conversions"
      ],
      "properties": {
        "visitors": {
          "type": "integer",
          "minimum": 1
        },
        "conversions": {
          "type": "integer",
          "minimum": 0
        }
      },
      "description": "Baseline group. conversions may not exceed visitors."
    },
    "variant": {
      "type": "object",
      "required": [
        "visitors",
        "conversions"
      ],
      "properties": {
        "visitors": {
          "type": "integer",
          "minimum": 1
        },
        "conversions": {
          "type": "integer",
          "minimum": 0
        }
      },
      "description": "Group under test. conversions may not exceed visitors."
    },
    "confidence_level": {
      "type": "number",
      "minimum": 0.5,
      "maximum": 0.9999,
      "default": 0.95
    },
    "alternative": {
      "type": "string",
      "enum": [
        "two_sided",
        "greater"
      ],
      "default": "two_sided",
      "description": "two_sided asks whether the groups differ, greater whether the variant is above the control."
    },
    "minimum_detectable_effect_rel": {
      "type": "number",
      "minimum": 0.001,
      "maximum": 5,
      "default": 0.1,
      "description": "Relative effect the test should be able to find, e.g. 0.1 for ten percent above the control rate."
    },
    "power": {
      "type": "number",
      "minimum": 0.5,
      "maximum": 0.999,
      "default": 0.8,
      "description": "Probability of finding that effect if it is there."
    },
    "looks_taken": {
      "type": "integer",
      "minimum": 1,
      "maximum": 100,
      "default": 1,
      "description": "How often the running test has been evaluated. Above 1 the threshold is adjusted and the real false positive rate is reported."
    },
    "prior_alpha": {
      "type": "number",
      "minimum": 0.0001,
      "maximum": 1000,
      "default": 1,
      "description": "Beta prior, successes. 1 with prior_beta 1 is uniform."
    },
    "prior_beta": {
      "type": "number",
      "minimum": 0.0001,
      "maximum": 1000,
      "default": 1,
      "description": "Beta prior, failures."
    }
  }
}

Originalschema öffnen

Veröffentlichtes Eingabebeispiel

{
  "control": {
    "visitors": 4820,
    "conversions": 241
  },
  "variant": {
    "visitors": 4790,
    "conversions": 302
  },
  "confidence_level": 0.95,
  "minimum_detectable_effect_rel": 0.1,
  "looks_taken": 3
}

Ergebnis & Dokumentation

Maschinenlesbare Dokumentation dieses Hubs

Quelle dieser Seite: der veröffentlichte HALOWERK-Katalog. Stand: 2026-11-01, 15:19 UTC.

Weitere Endpunkte dieses Hubs

Alle Endpunkte dieses Hubs →