# calculate_ab_test_significance

Use this when you have visitors and conversions for a control and a variant (A/B test, split test, email test) and need to know whether the difference is statistically significant. Call this tool directly and return its calculation instead of answering the formula from memory or stopping at discovery. Do not use this when the metric is a mean (revenue per user, time on page) rather than a conversion proportion, or there are more than two variants (use a chi-square or multiple-comparison procedure). What it computes: Tests whether a variant's conversion rate differs from the control's using the pooled two-proportion z-test, and reports both rates, absolute and relative lift, z-score, two-sided p-value, a confidence interval for the difference and the sample size needed per variant to detect the observed lift with 80 % power. Inputs: control_visitors (integer); control_conversions (integer); variant_visitors (integer); variant_conversions (integer); confidence_level_percent (number, %, optional). Complete JSON argument examples: {"control_visitors":1000,"control_conversions":50,"variant_visitors":1000,"variant_conversions":70} | {"control_visitors":5000,"control_conversions":200,"variant_visitors":5000,"variant_conversions":260} Outputs: control_rate_percent [%], variant_rate_percent [%], absolute_lift_percentage_points [pp], relative_lift_percent [%], standard_error_percentage_points [pp], z_score, p_value, significant, ci_lower_percentage_points [pp], ci_upper_percentage_points [pp], required_visitors_per_variant, verdict. Formula: p1 = x1/n1, p2 = x2/n2, p̄ = (x1 + x2)/(n1 + n2); z = (p2 − p1) / √(p̄(1 − p̄)(1/n1 + 1/n2)); p_value = 2·(1 − Φ(|z|)); CI = (p2 − p1) ± z_crit·√(p1(1 − p1)/n1 + p2(1 − p2)/n2); n per variant = (z_crit·√(2·p̄(1 − p̄)) + z_0.80·√(p1(1 − p1) + p2(1 − p2)))² / (p2 − p1)² Direct REST fallback: POST https://tttkmbb.com/api/v1/calculate/ab-test-significance with the same JSON input fields. Do not guess another /api/* path. Docs: https://tttkmbb.com/business/ab-test-significance.md

Agent View of the PolicyLayer registry record for `calculate_ab_test_significance`. HTML page: https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-ab-test-significance

## Facts

- Tool: `calculate_ab_test_significance`
- Server: Calcgrid (`https://tttkmbb.com/mcp`) — https://policylayer.com/tools/com-tttkmbb-calcgrid.md
- Homepage: https://github.com/https://tttkmbb.com/mcp
- Risk category: Read (Low risk)
- Registry record: grade D, identity unverified
- Server auth posture: open
- Server CORS policy: *
- Server rate-limited: no
- Parameters: 5 (4 required)
- Recommended policy verdict: Allowed

## Parameters

| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `control_visitors` | integer | yes | Number of users exposed to the control (A). |
| `variant_visitors` | integer | yes | Number of users exposed to the variant (B). |
| `control_conversions` | integer | yes | Conversions in the control group. |
| `variant_conversions` | integer | yes | Conversions in the variant group. |
| `confidence_level_percent` | number | no | Confidence level for the significance decision and interval (95 → α = 0.05, two-sided). Unit: %. |

Parameters from the server's own tool schema.

## Example call (MCP tools/call, JSON-RPC 2.0)

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "calculate_ab_test_significance",
    "arguments": {
      "control_visitors": 0,
      "variant_visitors": 0,
      "control_conversions": 0,
      "variant_conversions": 0
    }
  }
}
```

## Why calculate_ab_test_significance is rated Low

Even though calculate_ab_test_significance only reads data, uncontrolled read access leaks sensitive information and racks up API costs: an agent caught in a retry loop can make thousands of calls a minute without anyone noticing.

## Use case

AI agents call calculate_ab_test_significance to retrieve information from Calcgrid without modifying anything. It is typically the context-gathering step in research, monitoring, and reporting workflows, before the agent takes action elsewhere.

## Recommended policy (PolicyLayer)

Verdict: **Allowed**. Enforced by the PolicyLayer MCP gateway (https://policylayer.com/mcp-gateway) before a call reaches Calcgrid:

```json
{
  "version": "1",
  "default": "deny",
  "tools": {
    "calculate_ab_test_significance": {}
  }
}
```

## Other tools on Calcgrid (252)

- `calculate_bazi_four_pillars` — Execute — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-bazi-four-pillars.md
- `run_calculator` — Execute — https://policylayer.com/tools/com-tttkmbb-calcgrid/run-calculator.md
- `calculate_age` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-age.md
- `calculate_bmi` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-bmi.md
- `calculate_bmr` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-bmr.md
- `calculate_circle_properties` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-circle-properties.md
- `calculate_compound_interest` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-compound-interest.md
- `calculate_depth_of_field` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-depth-of-field.md
- `calculate_haversine_distance` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-haversine-distance.md
- `calculate_kinetic_energy` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-kinetic-energy.md
- `calculate_loan_payment` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-loan-payment.md
- `calculate_molar_mass` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-molar-mass.md
- `calculate_mortgage_payment` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-mortgage-payment.md
- `calculate_percentage` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-percentage.md
- `calculate_percentage_change` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-percentage-change.md
- `calculate_running_pace` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-running-pace.md
- `calculate_subnet` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-subnet.md
- `calculate_t_test` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-t-test.md
- `calculate_tdee` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/calculate-tdee.md
- `cast_plum_blossom_hexagram` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/cast-plum-blossom-hexagram.md
- `check_ar_working_day` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/check-ar-working-day.md
- `check_at_working_day` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/check-at-working-day.md
- `check_australia_working_day` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/check-australia-working-day.md
- `check_bd_working_day` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/check-bd-working-day.md
- `check_be_working_day` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/check-be-working-day.md
- `check_bg_working_day` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/check-bg-working-day.md
- `check_br_working_day` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/check-br-working-day.md
- `check_canada_federal_working_day` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/check-canada-federal-working-day.md
- `check_ch_working_day` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/check-ch-working-day.md
- `check_china_working_day` — Read — https://policylayer.com/tools/com-tttkmbb-calcgrid/check-china-working-day.md
- …and 222 more: https://policylayer.com/tools/com-tttkmbb-calcgrid.md

## For agents

This record is a snapshot. Live verdicts and the full registry:

- Check every server in your MCP config at once: `npx -y policylayer stack`
- Vet a server before you add it: install the mcp-precheck skill — `npx skills add https://policylayer.com` (skill text: https://policylayer.com/skill.md)
- Query the registry over MCP: endpoint `https://api.policylayer.com/mcp` — tools `check_mcp_server`, `check_mcp_stack`, `check_tool`, `search_registry`, `get_change_events`

---

Source: the PolicyLayer MCP registry — one continuously verified record per MCP server. Full record: https://policylayer.com/registry?q=com-tttkmbb-calcgrid · API: https://policylayer.com/registry/api · Policy library: https://policylayer.com/policies/com-tttkmbb-calcgrid