crawl_documentation · Mcp For Docs MCP: Risk & Policy

WHAT IT DOES

What crawl_documentation does on Mcp For Docs

AI agents use crawl_documentation to create or update resources in Mcp For Docs — usually the action step of a workflow, after the agent has gathered context. Every call changes real data in your Mcp For Docs environment.

WHY IT NEEDS A POLICY

Why crawl_documentation needs a policy

This tool retrieves remote content (documentation) and writes it to local storage. While the retrieval phase is read-only, the net effect is creating or modifying local files persistently. This is Write category rather than Read because it has side effects (disk usage, file creation). It is not Destructive because the operation is reversible (files can be deleted).

From the tool's definition Tool description states 'Crawl and download documentation from a website' — the download operation creates or modifies local files/data structures. The sibling tool list suggests this server creates organized markdown files and stores documentation locally.

Attacks that exploit this kind of access

Recommended rule Ready

Verdict

crawl_documentation Rate-limited

crawl_documentation stays usable, but capped — an agent stuck in a loop can't make hundreds of changes a minute. Everything else on the server is denied unless you say otherwise.

View as policy code

policy.json

{
  "version": "1",
  "default": "deny",
  "tools": {
    "crawl_documentation": {
      "limits": [
        {
          "counter": "crawl_documentation_rate",
          "window": "minute",
          "max": 30,
          "scope": "grant"
        }
      ]
    }
  }
}

LIMIT THIS TOOL →

Instant setup, no code required.

PolicyLayer is an MCP gateway: every tool call is checked against your rule before it reaches Mcp For Docs, so you can run agents against it safely.

FAQ

Questions about crawl_documentation

What does the crawl_documentation tool do? +

Crawl and download documentation from a website. It is categorised as a Write tool in the Mcp For Docs MCP Server, which means it can create or modify data. Consider rate limits to prevent runaway writes.

How do I enforce a policy on crawl_documentation? +

Register the Mcp For Docs MCP server in PolicyLayer and add a rule for crawl_documentation: allow, deny, rate-limit, or require approval. Point your MCP client at the PolicyLayer proxy URL and the rule is enforced on every call, before it reaches Mcp For Docs. Nothing to install.

What risk level is crawl_documentation? +

crawl_documentation is a Write tool with medium risk. Write tools should be rate-limited to prevent accidental bulk modifications.

Can I rate-limit crawl_documentation? +

Yes. Add a rate_limit block to the crawl_documentation rule in your PolicyLayer policy. For example, setting max: 10 and window: 60 limits the tool to 10 calls per minute. Rate limits are tracked per agent session and reset automatically.

How do I block crawl_documentation completely? +

Set action: deny in the PolicyLayer policy for crawl_documentation. The AI agent will receive a policy violation error and cannot call the tool. You can also include a reason field to explain why the tool is blocked.

What MCP server provides crawl_documentation? +

crawl_documentation is provided by the Mcp For Docs MCP server (shayonpal/mcp-for-docs). PolicyLayer sits as a proxy in front of this server to enforce policies before tool calls reach the server.