scrape_url · TDZ C64 Knowledge MCP: Risk & Policy

WHAT IT DOES

What scrape_url does on TDZ C64 Knowledge

AI agents use scrape_url to create or update resources in TDZ C64 Knowledge — usually the action step of a workflow, after the agent has gathered context. Every call changes real data in your TDZ C64 Knowledge environment.

WHY IT NEEDS A POLICY

Why scrape_url needs a policy

This tool creates and modifies the knowledge base by scraping remote websites and ingesting their content as searchable documents. While it does not delete data (ruling out Destructive) or execute arbitrary code beyond web scraping (ruling out Execute in the strictest sense), it does write/add content to the system.

From the tool's definition The tool description states it will 'add all pages to the knowledge base' and 'ingest online documentation', indicating creation of data records. The term 'scrape' combined with 'add' shows data modification rather than read-only retrieval.

Risk signalsBulk/mass operation — affects multiple targets

Attacks that exploit this kind of access

Recommended rule Ready

Verdict

scrape_url Rate-limited

scrape_url stays usable, but capped — an agent stuck in a loop can't make hundreds of changes a minute. Everything else on the server is denied unless you say otherwise.

View as policy code

policy.json

{
  "version": "1",
  "default": "deny",
  "tools": {
    "scrape_url": {
      "limits": [
        {
          "counter": "scrape_url_rate",
          "window": "minute",
          "max": 30,
          "scope": "grant"
        }
      ]
    }
  }
}

LIMIT THIS TOOL →

Instant setup, no code required.

PolicyLayer is an MCP gateway: every tool call is checked against your rule before it reaches TDZ C64 Knowledge, so you can run agents against it safely.

FAQ

Questions about scrape_url

What does the scrape_url tool do? +

Scrape a documentation website and add all pages to the knowledge base. Supports recursive scraping of entire sites by following links. Great for ingesting online documentation like http://www.sidmusic.org/sid/. Converts HTML to searchable markdown. It is categorised as a Write tool in the TDZ C64 Knowledge MCP Server, which means it can create or modify data. Consider rate limits to prevent runaway writes.

How do I enforce a policy on scrape_url? +

Register the TDZ C64 Knowledge MCP server in PolicyLayer and add a rule for scrape_url: allow, deny, rate-limit, or require approval. Point your MCP client at the PolicyLayer proxy URL and the rule is enforced on every call, before it reaches TDZ C64 Knowledge. Nothing to install.

What risk level is scrape_url? +

scrape_url is a Write tool with medium risk. Write tools should be rate-limited to prevent accidental bulk modifications.

Can I rate-limit scrape_url? +

Yes. Add a rate_limit block to the scrape_url rule in your PolicyLayer policy. For example, setting max: 10 and window: 60 limits the tool to 10 calls per minute. Rate limits are tracked per agent session and reset automatically.

How do I block scrape_url completely? +

Set action: deny in the PolicyLayer policy for scrape_url. The AI agent will receive a policy violation error and cannot call the tool. You can also include a reason field to explain why the tool is blocked.

What MCP server provides scrape_url? +

scrape_url is provided by the TDZ C64 Knowledge MCP server (michaeltroelsen/tdz-c64-knowledge). PolicyLayer sits as a proxy in front of this server to enforce policies before tool calls reach the server.