// GLOSSARY -- AGENTIC AI

What is Hallucination?

1 min read Updated

In AI, hallucination refers to when a language model generates confident, plausible-sounding output that is factually incorrect or fabricated — a fundamental challenge for agent reliability.

WHY IT MATTERS

Hallucination is the Achilles' heel of LLM-powered systems. A model can state incorrect facts with complete confidence, invent citations that don't exist, or generate code that looks correct but contains subtle bugs.

For AI agents, hallucination risks compound. An agent might hallucinate a wallet address, fabricate a token price, or invent a protocol that doesn't exist — and then act on that hallucinated information. In financial contexts, this means sending funds to wrong addresses or executing trades based on phantom data.

Mitigations include RAG, chain-of-thought reasoning, output verification, and external validation layers. No single technique eliminates hallucination; defense in depth is required.

HOW POLICYLAYER USES THIS

PolicyLayer acts as a hallucination safety net for financial agents. Even if an agent hallucinates a transaction target or amount, PolicyLayer validates every action against whitelists, spending limits, and allowed recipients — catching invalid transactions before they reach the blockchain.

FREQUENTLY ASKED QUESTIONS

Why do LLMs hallucinate?
LLMs predict probable token sequences, not facts. When the query is outside the model's confident knowledge, it generates plausible completions that may be factually wrong.
Can hallucination be fully eliminated?
Not with current architectures. It can be reduced through better training, RAG, and verification — but probabilistic models will always have some error rate.
How dangerous are hallucinations in financial agents?
Extremely. A hallucinated wallet address means lost funds. A hallucinated price means bad trades. External validation layers are essential.

FURTHER READING

Enforce policies on every tool call

Intercept is the open-source MCP proxy that enforces YAML policies on AI agent tool calls. No code changes needed.

npx -y @policylayer/intercept
github.com/policylayer/intercept →
// GET IN TOUCH

Have a question or want to learn more? Send us a message.

Message sent.

We'll get back to you soon.