Connect A2ABench and its 3 tool definitions are loaded into the model's context on every request — 0.1% of a 200k window spent before your agent does anything.
QUICK ANSWER The A2ABench MCP server's tool definitions consume 259 tokens — below the median MCP server (1,905 tokens). A scoped grant exposing only the tools you use cuts that roughly in proportion.
Tool definitions are overhead: they occupy context on every request and compete with your code, documents and conversation history for the same window.
Corpus context: A2ABench ranks #3117 of 3,213 measured MCP servers by definition cost. The median is 1,905 tokens, p90 is 7,952, and the heaviest (Fusionauth) is 183,337 — 92% of a 200k window on its own.
Each row is one tool definition as a tools/list entry — name, description and
input schema — counted with o200k_base. Average: 86 tokens per tool.
| Tool | Category | Tokens | % of server |
|---|---|---|---|
| submit_benchmark_run | Write | 139 | 53.7% |
| get_leaderboard | Read | 61 | 23.6% |
| list_benchmark_questions | Read | 59 | 22.8% |
A PolicyLayer grant exposes only the tools you allow — ungranted definitions are filtered out of the tool list, so they never enter the context window. Estimates below assume typical-weight tools (86 tokens each).
At 3 tools, A2ABench is already a small surface — the win here is policy control and audit rather than token savings.
Model your own stack in the token-cost calculator, or see the A2ABench policy for what a sensible grant looks like.
Its 3 tool definitions total 259 tokens — 0.1% of a 200k context window — measured with tiktoken o200k_base over the serialised tools/list payload. Exact counts vary slightly by client and model.
MCP clients load every connected server's tool definitions — name, description, and input schema — into the model's context so it knows what it can call. That payload is charged against your context window on every request, whether or not a tool is used.
Expose fewer tools. A PolicyLayer grant scopes A2ABench to only the tools you allow — ungranted definitions are filtered out of the tool list, so they never enter the context window.
Partially, in some clients. Claude Code defers MCP tool schemas behind a tool-search step by default, and VS Code has experimental grouping — but you still pay tokens per search and reload, and Cursor, Windsurf and Gemini CLI load definitions upfront. Reducing the exposed tool set cuts the cost in every client.
Each tool is serialised as a tools/list entry — name, description, input schema — from the schemas in the PolicyLayer scan database. Clients differ slightly in framing, so treat counts as close estimates.
tiktoken o200k_base (GPT-4o/o-series). Anthropic's current tokeniser isn't published, so Claude's exact counts will differ; for English text and JSON schemas the totals are close enough to treat these as estimates.
Some clients now defer schema loading (Claude Code's tool search; VS Code experimental grouping). You still pay per search and reload — and Cursor, Windsurf and Gemini CLI load everything upfront.
Computed 07-06-2026 from the PolicyLayer scan database over all 3 catalogued A2ABench tools. Counts refresh with every site build.
A PolicyLayer grant scopes A2ABench to the tools you actually allow. Ungranted definitions never load, and every call that does run is checked against policy first.
Free to start. No card required.
4,600+ MCP servers and 31,000+ tools scanned and risk-classified.