Crawl a website via BFS and return per-page extraction results (markdown, links, tables, images, metadata). Price: $0.10 USDC per crawl (paid mode) | Free test: returns fixture data. Crawls up to max_pages pages starting from the seed URL, up to max_depth link hops deep. Same extraction pipeline ...
AI agents call x402_crawl_site to retrieve information from x402 MCP Server without modifying anything — typically the context-gathering step in research, monitoring, and reporting workflows, before the agent takes action elsewhere.
The tool retrieves and extracts data from a website through breadth-first search traversal. It reads publicly or authorized content and returns structured information without modifying the target website.
From the tool's definition Crawl a website via BFS and return per-page extraction results (markdown, links, tables, images, metadata). Returns fixture data in test mode, no deletion or modification of target site. No code execution on user's system.
Attacks that exploit this kind of access
Crawl a website via BFS and return per-page extraction results (markdown, links, tables, images, metadata). Price: $0.10 USDC per crawl (paid mode) | Free test: returns fixture data. Crawls up to max_pages pages starting from the seed URL, up to max_depth link hops deep. Same extraction pipeline as x402_scrape_url — each page returns markdown, links, tables, images, metadata. Optional include_paths/exclude_paths glob filters (e.g. It is categorised as a Read tool in the x402 MCP Server MCP Server, which means it retrieves data without modifying state.
Register the x402 MCP Server MCP server in PolicyLayer and add a rule for x402_crawl_site: allow, deny, rate-limit, or require approval. Point your MCP client at the PolicyLayer proxy URL and the rule is enforced on every call, before it reaches x402 MCP Server. Nothing to install.
x402_crawl_site is a Read tool with low risk. Read-only tools are generally safe to allow by default.
Yes. Add a rate_limit block to the x402_crawl_site rule in your PolicyLayer policy. For example, setting max: 10 and window: 60 limits the tool to 10 calls per minute. Rate limits are tracked per agent session and reset automatically.
Set action: deny in the PolicyLayer policy for x402_crawl_site. The AI agent will receive a policy violation error and cannot call the tool. You can also include a reason field to explain why the tool is blocked.
x402_crawl_site is provided by the x402 MCP Server MCP server (jameswilliamwisdom/x402-mcp-server). PolicyLayer sits as a proxy in front of this server to enforce policies before tool calls reach the server.
Every MCP server has a record like this.
Type a name, get the same breakdown: verified identity, auth posture, risk grade, capabilities, recommended policy.
Teams ship this data inside their own products. See what a licence covers →