Scrape webpage content using intelligent automatic strategy selection with built-in caching. This tool fetches content from any URL with flexible result handling options. Result handling modes: - returnOnly: Returns scraped content without saving (uses maxChars for size limits) - saveAndReturn: ...
Accepts URL/endpoint input (url)
Part of the Pulse Fetch MCP server. Enforce policies on this tool with Intercept, the open-source MCP proxy.
AI agents call scrape to retrieve information from Pulse Fetch without modifying any data. This is common in research, monitoring, and reporting workflows where the agent needs context before taking action. Because read operations don't change state, they are generally safe to allow without restrictions -- but you may still want rate limits to control API costs.
Even though scrape only reads data, uncontrolled read access can leak sensitive information or rack up API costs. An agent caught in a retry loop could make thousands of calls per minute. A rate limit gives you a safety net without blocking legitimate use.
Read-only tools are safe to allow by default. No rate limit needed unless you want to control costs.
tools:
scrape:
rules:
- action: allowSee the full Pulse Fetch policy for all 1 tools.
Agents calling read-class tools like scrape have been implicated in these attack patterns. Read the full case and prevention policy for each:
Other tools in the Read risk category across the catalogue. The same policy patterns (rate-limit, allow) apply to each.
Scrape webpage content using intelligent automatic strategy selection with built-in caching. This tool fetches content from any URL with flexible result handling options. Result handling modes: - returnOnly: Returns scraped content without saving (uses maxChars for size limits) - saveAndReturn: Saves content as MCP Resource AND returns it (default, best for reuse) - saveOnly: Saves content as MCP Resource, returns only resource link (no content) Example responses by mode: returnOnly: { "content": [ { "type": "text", "text": "Article content here... --- Scraped using: native" } ] } saveAndReturn (embedded resource): { "content": [ { "type": "resource", "resource": { "uri": "scraped://example.com/article_2024-01-15T10:30:00Z", "name": "https://example.com/article", "text": "Full article content..." } } ] } saveOnly (linked resource): { "content": [ { "type": "resource_link", "uri": "scraped://example.com/article_2024-01-15T10:30:00Z", "name": "https://example.com/article" } ] } Caching behavior: - Previously scraped URLs are automatically cached as MCP Resources - Subsequent requests return cached content (unless forceRescrape: true) - saveOnly mode bypasses cache lookup for efficiency Scraping strategies: - native: Direct HTTP fetch (fastest, works for most public sites) - firecrawl: Advanced scraping with JavaScript rendering (requires FIRECRAWL_API_KEY) - brightdata: Premium scraping for heavily protected sites (requires BRIGHTDATA_API_KEY) The tool automatically: 1. Checks cache first (except in saveOnly mode) 2. Tries the most appropriate scraping method based on domain patterns 3. Falls back to alternative methods if needed 4. Remembers successful strategies for future requests. It is categorised as a Read tool in the Pulse Fetch MCP Server, which means it retrieves data without modifying state.
Add a rule in your Intercept YAML policy under the tools section for scrape. You can allow, deny, rate-limit, or validate arguments. Then run Intercept as a proxy in front of the Pulse Fetch MCP server.
scrape is a Read tool with low risk. Read-only tools are generally safe to allow by default.
Yes. Add a rate_limit block to the scrape rule in your Intercept policy. For example, setting max: 10 and window: 60 limits the tool to 10 calls per minute. Rate limits are tracked per agent session and reset automatically.
Set action: deny in the Intercept policy for scrape. The AI agent will receive a policy violation error and cannot call the tool. You can also include a reason field to explain why the tool is blocked.
scrape is provided by the Pulse Fetch MCP server (@pulsemcp/pulse-fetch). Intercept sits as a proxy in front of this server to enforce policies before tool calls reach the server.
Open source. One binary. Zero dependencies.
npx -y @policylayer/intercept