// SCAN REPORT

Crawl-MCP

19 tools. 0 can modify or destroy data without limits.

Read-only server. Low risk, but rate limits prevent runaway API costs.

Last updated:

0 can modify or destroy data
19 read-only
19 tools total

Community server · catalogue entry verified 11/06/2026

How to control Crawl-MCP ↓

TOOL MAP

Read (19) Write / Execute (0) Destructive / Financial (0)

WHAT CAN GO WRONG

✕

Even read-only tools carry cost. An agent in a retry loop can make thousands of API calls per minute, exhausting rate limits and running up bills.

HOW TO CONTROL CRAWL-MCP

PolicyLayer is an MCP gateway — it sits between your AI agents and Crawl-MCP, and nothing reaches the server without passing your rules. These are the rules we recommend:

Cap read operations
{
  "batch_crawl": {
    "limits": [
      {
        "counter": "batch_crawl_per_minute",
        "window": "minute",
        "max": 60,
        "scope": "grant"
      }
    ]
  }
}

Controls API costs and prevents retry loops from exhausting upstream rate limits.

  1. Create a free account and register Crawl-MCP — nothing to install.
  2. Add these rules — paste them, or build them visually. Tune the limits to your setup.
  3. Point your MCP client (Claude, Cursor, anything) at your gateway URL.
ENFORCE POLICY ON CRAWL-MCP →

Free to start. No card required.

ALL 19 CRAWL-MCP TOOLS

READ 19 tools
Read batch_crawl batch_crawl Read batch_extract_youtube_transcripts batch_extract_youtube_transcripts Read batch_search_google batch_search_google Read crawl_url crawl_url Read crawl_url_with_fallback crawl_url_with_fallback Read deep_crawl_site deep_crawl_site Read enhanced_process_large_content enhanced_process_large_content Read extract_entities extract_entities Read extract_structured_data extract_structured_data Read extract_youtube_comments extract_youtube_comments Read extract_youtube_transcript extract_youtube_transcript Read get_search_genres Get available search genres for targeted searching. Read get_supported_file_formats Get supported file formats (PDF, Office, ZIP) and their capabilities. Read get_youtube_video_info get_youtube_video_info Read intelligent_extract intelligent_extract Read multi_url_crawl multi_url_crawl Read process_file process_file Read search_and_crawl search_and_crawl Read search_google search_google

FAQ

Is the Crawl- MCP server safe to use without restrictions? +

The Crawl-MCP server is primarily read-only with 19 read tools. While it cannot modify data, an agent in a retry loop can make thousands of API calls per minute, exhausting rate limits and running up costs. Rate limiting is still recommended.

How many tools does the Crawl- MCP server expose? +

19 tools across 1 categories: Read. 19 are read-only. 0 can modify, create, or delete data.

How do I enforce a policy on Crawl-MCP? +

Register the Crawl- MCP server in PolicyLayer, apply the suggested rules above (adjust the limits to your use case), and point your AI client at the PolicyLayer proxy URL instead of the server directly. Your agents keep the same tools; PolicyLayer evaluates every call against policy before it executes. Nothing to install, live in minutes.

Enforce policy on every Crawl-MCP tool call.

Deterministic rules across all 19 Crawl-MCP tools. Per-identity grants. Full audit log. Live in minutes. Nothing to install.

ENFORCE POLICY ON CRAWL-MCP →

Free to start. No card required.

19 Crawl-MCP tools catalogued and risk-classified — across an index of 42,500+ MCP servers.

// GET IN TOUCH

Have a question or want to learn more? Send us a message.

Message sent.

We'll get back to you soon.