Fetch content from a URL, convert to markdown, and add to the knowledge base. Mutating — makes an outbound HTTP request (requires internet access), strips HTML, converts to markdown, saves to disk, and indexes immediately. Args: url: Full URL to fetch (https:// required). The page must be publicl...
AI agents use add_from_url to create or update resources in Knowledge Rag — usually the action step of a workflow, after the agent has gathered context. Every call changes real data in your Knowledge Rag environment.
This tool creates and modifies data in the knowledge base by fetching external content and persisting it. While it makes an outbound HTTP request, the primary action is Write (creating indexed documents). Severity is medium because the blast radius is limited to the local knowledge base; misuse could pollute the index with unwanted content but cannot delete data or execute arbitrary code.
From the tool's definition Mutating — makes an outbound HTTP request, strips HTML, converts to markdown, saves to disk, and indexes immediately. The tool creates new documents in the knowledge base ('saves to disk, and indexes immediately').
Documented attack patterns abuse exactly the kind of access add_from_url gives an agent:
PolicyLayer is an MCP gateway — it sits between your AI agents and Knowledge Rag, and nothing reaches the server without passing your rules. This is the rule we recommend for add_from_url:
{
"version": "1",
"default": "deny",
"tools": {
"add_from_url": {
"limits": [
{
"counter": "add_from_url_rate",
"window": "minute",
"max": 30,
"scope": "grant"
}
]
}
}
}add_from_url stays usable, but capped — an agent stuck in a loop can't make hundreds of changes a minute. Everything else on the server is denied unless you say otherwise.
Free to start. No card required.
Fetch content from a URL, convert to markdown, and add to the knowledge base. Mutating — makes an outbound HTTP request (requires internet access), strips HTML, converts to markdown, saves to disk, and indexes immediately. Args: url: Full URL to fetch (https:// required). The page must be publicly accessible. category: Document category — one of: security, ctf, logscale, development, general, redteam, blueteam (default: general) title: Optional document title. Auto-detected from the page's <title> tag if omitted. Returns: JSON string with indexing results (detected title, filepath, chunks created, status). Usage: Use to ingest web content (writeups, blog posts, documentation pages) directly by URL. Use add_document() instead when you already have the text content. The document is immediately searchable after this call — no manual reindex needed. It is categorised as a Write tool in the Knowledge Rag MCP Server, which means it can create or modify data. Consider rate limits to prevent runaway writes.
Register the Knowledge Rag MCP server in PolicyLayer and add a rule for add_from_url: allow, deny, rate-limit, or require approval. Point your MCP client at the PolicyLayer proxy URL and the rule is enforced on every call, before it reaches Knowledge Rag. Nothing to install.
add_from_url is a Write tool with medium risk. Write tools should be rate-limited to prevent accidental bulk modifications.
Yes. Add a rate_limit block to the add_from_url rule in your PolicyLayer policy. For example, setting max: 10 and window: 60 limits the tool to 10 calls per minute. Rate limits are tracked per agent session and reset automatically.
Set action: deny in the PolicyLayer policy for add_from_url. The AI agent will receive a policy violation error and cannot call the tool. You can also include a reason field to explain why the tool is blocked.
add_from_url is provided by the Knowledge Rag MCP server (lyonzin/knowledge-rag). PolicyLayer sits as a proxy in front of this server to enforce policies before tool calls reach the server.
Start from Knowledge Rag, add the rest of your stack, and see everything your agents can call. Then put policy on all of it.
Free to start. No card required.
12 Knowledge Rag tools catalogued and risk-classified — across an index of 43,000+ MCP servers.