// GLOSSARY -- AGENTIC AI

What is Context Window?

1 min read Updated

A context window is the maximum number of tokens an LLM can process in a single interaction, encompassing system prompt, conversation history, retrieved documents, and generated output.

WHY IT MATTERS

The context window is the LLM's working memory. Everything the model considers must fit within this window. Modern models range from 8K tokens to 200K+ tokens (Claude, Gemini).

For agents, context window management is critical. A long-running agent accumulates history that gradually fills the window. When it overflows, the oldest context is dropped — potentially losing critical instructions.

Larger windows help but don't fully solve the problem. Research shows LLMs struggle with information in the middle of long contexts ('lost in the middle'), making placement of critical instructions matter.

Running agents against MCP servers? Route them through PolicyLayer and every tool call is checked against policy first.

PUT POLICY ON YOUR TOOL CALLS →

Enforced before the call runs. Nothing to install.

IN THE CATALOGUE

Measured across 3,165 MCP servers (59,345 tools): connecting a server loads its full tool definitions into the context window on every request.

1,900 tokens — median server
7,952 tokens — 90th percentile
183,337 tokens — largest measured (Fusionauth)
ServerTool definitionsTokens per request
GitHub8614,406
Linear667,149
Supabase292,561
Filesystem141,642

Look up any server’s token cost →

FREQUENTLY ASKED QUESTIONS

What happens when the context window fills up?
Older content is truncated or summarized. This can cause the agent to lose earlier instructions or safety constraints — a real risk for long-running financial agents.
Does a larger context window mean better performance?
Not necessarily. Models can struggle to attend to all information equally. The 'lost in the middle' effect means information placement matters as much as window size.
How does context window affect costs?
LLM pricing is per-token. Larger contexts mean more input tokens per call, directly increasing costs. Efficient context management reduces expenses.

FURTHER READING

Take your agents live. Without losing control.

Route your MCP traffic through PolicyLayer. Every tool call is checked against your policy before it runs: allow, deny, or require approval. Per-identity grants. Full audit log. Live in minutes.

PUT POLICY ON YOUR TOOL CALLS →

Instant setup, no code required.

43,000+ MCP servers and 220,000+ tools scanned and risk-classified.

// GET IN TOUCH

Have a question or want to learn more? Send us a message.

Message sent.

We'll get back to you soon.