crawl_website
AI agents call crawl_website to retrieve information from Mcp Webscraper without modifying anything — typically the context-gathering step in research, monitoring, and reporting workflows, before the agent takes action elsewhere.
The tool crawls websites to extract and analyze data—a read-only operation with no data modification, deletion, or code execution. Severity is medium rather than low because web crawling can strain server resources, trigger rate-limiting, potentially violate Terms of Service, or extract sensitive information if applied to restricted sites.
From the tool's definition Tool name 'crawl_website' combined with server's stated purpose to perform 'web scraping and crawling operations, extracting structured data, analyzing website architectures' indicates a retrieval operation.
Attacks that exploit this kind of access
crawl_website. It is categorised as a Read tool in the Mcp Webscraper MCP Server, which means it retrieves data without modifying state.
Register the Mcp Webscraper MCP server in PolicyLayer and add a rule for crawl_website: allow, deny, rate-limit, or require approval. Point your MCP client at the PolicyLayer proxy URL and the rule is enforced on every call, before it reaches Mcp Webscraper. Nothing to install.
crawl_website is a Read tool with low risk. Read-only tools are generally safe to allow by default.
Yes. Add a rate_limit block to the crawl_website rule in your PolicyLayer policy. For example, setting max: 10 and window: 60 limits the tool to 10 calls per minute. Rate limits are tracked per agent session and reset automatically.
Set action: deny in the PolicyLayer policy for crawl_website. The AI agent will receive a policy violation error and cannot call the tool. You can also include a reason field to explain why the tool is blocked.
crawl_website is provided by the Mcp Webscraper MCP server (samirsaci/mcp-webscraper). PolicyLayer sits as a proxy in front of this server to enforce policies before tool calls reach the server.
crawl_website is one line of Mcp Webscraper's registry record.
The record carries the whole server: verified identity, auth posture, risk grade, every tool classified, recommended policy — re-checked continuously.
Teams ship this data inside their own products. See what a licence covers →