crawl_web_page · Dev Tool MCP: Risk & Policy

WHAT IT DOES

What crawl_web_page does on Dev Tool MCP

AI agents use crawl_web_page to create or update resources in Dev Tool MCP — usually the action step of a workflow, after the agent has gathered context. Every call changes real data in your Dev Tool MCP environment.

WHY IT NEEDS A POLICY

Why crawl_web_page needs a policy

The tool does more than read — it saves files to disk in multiple formats and downloads file resources, constituting a Write action. While it reads from the web, the persistent side effects (saving files locally) push it beyond a pure Read classification. It is not Destructive or Execute since it doesn't run arbitrary code or delete data, though misuse could fill disk space or save malicious content.

From the tool's definition 'save in multiple formats (HTML, JSON, PDF, screenshots) while downloading file resources from the page'

Attacks that exploit this kind of access

Recommended rule Ready

Verdict

crawl_web_page Rate-limited

crawl_web_page stays usable, but capped — an agent stuck in a loop can't make hundreds of changes a minute. Everything else on the server is denied unless you say otherwise.

View as policy code

policy.json

{
  "version": "1",
  "default": "deny",
  "tools": {
    "crawl_web_page": {
      "limits": [
        {
          "counter": "crawl_web_page_rate",
          "window": "minute",
          "max": 30,
          "scope": "grant"
        }
      ]
    }
  }
}

LIMIT THIS TOOL →

Instant setup, no code required.

PolicyLayer is an MCP gateway: every tool call is checked against your rule before it reaches Dev Tool MCP, so you can run agents against it safely.

FAQ

Questions about crawl_web_page

What does the crawl_web_page tool do? +

Crawl web page content and save in multiple formats (HTML, JSON, PDF, screenshots) while downloading file resources from the page. It is categorised as a Write tool in the Dev Tool MCP MCP Server, which means it can create or modify data. Consider rate limits to prevent runaway writes.

How do I enforce a policy on crawl_web_page? +

Register the Dev Tool MCP server in PolicyLayer and add a rule for crawl_web_page: allow, deny, rate-limit, or require approval. Point your MCP client at the PolicyLayer proxy URL and the rule is enforced on every call, before it reaches Dev Tool MCP. Nothing to install.

What risk level is crawl_web_page? +

crawl_web_page is a Write tool with medium risk. Write tools should be rate-limited to prevent accidental bulk modifications.

Can I rate-limit crawl_web_page? +

Yes. Add a rate_limit block to the crawl_web_page rule in your PolicyLayer policy. For example, setting max: 10 and window: 60 limits the tool to 10 calls per minute. Rate limits are tracked per agent session and reset automatically.

How do I block crawl_web_page completely? +

Set action: deny in the PolicyLayer policy for crawl_web_page. The AI agent will receive a policy violation error and cannot call the tool. You can also include a reason field to explain why the tool is blocked.

What MCP server provides crawl_web_page? +

crawl_web_page is provided by the Dev Tool MCP server (osins/dev-tool-mcp). PolicyLayer sits as a proxy in front of this server to enforce policies before tool calls reach the server.