Crawl a website using its sitemap.xml. Auto-discovers sitemaps from robots.txt or /sitemap.xml. Supports sitemap index files and URL filtering.\n\nWhen to use: Extracting content from many pages of a site that publishes a sitemap.xml.\nWhen NOT to use: Use crawl for BFS discovery when no sitemap ...
AI agents call crawl_sitemap to retrieve information from OpenChrome without modifying anything — typically the context-gathering step in research, monitoring, and reporting workflows, before the agent takes action elsewhere.
This tool retrieves and reads content from multiple pages by following sitemap.xml files. It has no capability to modify data, execute arbitrary commands, delete resources, or move financial assets. The operations are purely informational retrieval with read-only semantics.
From the tool's definition crawl_sitemap performs 'Crawl a website using its sitemap.xml' and is described for 'Extracting content from many pages of a site'. It auto-discovers and filters URLs without modifying or executing operations on the target system.
Documented attack patterns abuse exactly the kind of access crawl_sitemap gives an agent:
PolicyLayer is an MCP gateway — it sits between your AI agents and OpenChrome, and nothing reaches the server without passing your rules. This is the rule we recommend for crawl_sitemap:
{
"version": "1",
"default": "deny",
"tools": {
"crawl_sitemap": {}
}
}crawl_sitemap is read-only, so it stays allowed — but everything else on the server is denied unless you say otherwise.
Free to start. No card required.
Crawl a website using its sitemap.xml. Auto-discovers sitemaps from robots.txt or /sitemap.xml. Supports sitemap index files and URL filtering.\n\nWhen to use: Extracting content from many pages of a site that publishes a sitemap.xml.\nWhen NOT to use: Use crawl for BFS discovery when no sitemap exists, or navigate for a single page. It is categorised as a Read tool in the OpenChrome MCP Server, which means it retrieves data without modifying state.
Register the OpenChrome MCP server in PolicyLayer and add a rule for crawl_sitemap: allow, deny, rate-limit, or require approval. Point your MCP client at the PolicyLayer proxy URL and the rule is enforced on every call, before it reaches OpenChrome. Nothing to install.
crawl_sitemap is a Read tool with low risk. Read-only tools are generally safe to allow by default.
Yes. Add a rate_limit block to the crawl_sitemap rule in your PolicyLayer policy. For example, setting max: 10 and window: 60 limits the tool to 10 calls per minute. Rate limits are tracked per agent session and reset automatically.
Set action: deny in the PolicyLayer policy for crawl_sitemap. The AI agent will receive a policy violation error and cannot call the tool. You can also include a reason field to explain why the tool is blocked.
crawl_sitemap is provided by the OpenChrome MCP server (shaun0927/openchrome). PolicyLayer sits as a proxy in front of this server to enforce policies before tool calls reach the server.
Deterministic rules across all 106 OpenChrome tools. Per-identity grants. Full audit log. Live in minutes. Nothing to install.
Free to start. No card required.
106 OpenChrome tools catalogued and risk-classified — across an index of 42,500+ MCP servers.