# util_robots_inspect Read robots.txt rules and sitemap declarations before crawling or indexing a domain. Agent View of the PolicyLayer registry record for `util_robots_inspect`. HTML page: https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-robots-inspect ## Facts - Tool: `util_robots_inspect` - Server: Delx Protocol — Agent Recovery & Continuity (`https://api.delx.ai/v1/mcp?src=registry`) — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol.md - Homepage: https://github.com/davidmosiah/delx-witness-protocol - Risk category: Read (Low risk) - Registry record: grade F, identity unverified - Server auth posture: open - Server CORS policy: * - Server rate-limited: yes - Parameters: 2 (1 required) - Recommended policy verdict: Allowed ## Parameters | Parameter | Type | Required | Description | | --- | --- | --- | --- | | `url` | string | yes | Domain or URL to inspect | | `timeout` | integer | no | Timeout in seconds (1-15) | Parameters from the server's own tool schema. ## Example call (MCP tools/call, JSON-RPC 2.0) ```json { "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "util_robots_inspect", "arguments": { "url": "" } } } ``` ## Why util_robots_inspect is rated Low This tool performs information retrieval without modification, deletion, or execution of code. Reading robots.txt and sitemap files is a passive, non-destructive operation commonly used for reconnaissance before web operations. The low severity reflects the minimal blast radius—such files are publicly accessible and designed to be read by crawlers. No side effects occur from retrieving this metadata. From the tool's own definition: "Tool name 'util_robots_inspect' and description 'Read robots.txt rules and sitemap declarations' explicitly indicate a read-only operation that retrieves and queries static files (robots.txt and sitemaps) from a domain." Risk signals: Accepts URL/endpoint input (url) ## Use case AI agents call util_robots_inspect to retrieve information from Delx Protocol — Agent Recovery & Continuity without modifying anything. It is typically the context-gathering step in research, monitoring, and reporting workflows, before the agent takes action elsewhere. ## Recommended policy (PolicyLayer) Verdict: **Allowed**. Enforced by the PolicyLayer MCP gateway (https://policylayer.com/mcp-gateway) before a call reaches Delx Protocol — Agent Recovery & Continuity: ```json { "version": "1", "default": "deny", "tools": { "util_robots_inspect": {} } } ``` ## Other tools on Delx Protocol — Agent Recovery & Continuity (1075) - `util_arr_compact` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-arr-compact.md - `util_arr_drop` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-arr-drop.md - `util_dedent_text` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-dedent-text.md - `util_dedupe_lines` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-dedupe-lines.md - `util_header_map_normalize` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-header-map-normalize.md - `util_http_request_canonicalize` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-http-request-canonicalize.md - `util_remove_punctuation` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-remove-punctuation.md - `util_text_truncate` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-text-truncate.md - `util_truncate_middle` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-truncate-middle.md - `crisis_intervention` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/crisis-intervention.md - `explain_delx_rewards` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/explain-delx-rewards.md - `grounding_protocol` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/grounding-protocol.md - `group_therapy_round` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/group-therapy-round.md - `quick_operational_recovery` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/quick-operational-recovery.md - `quick_session` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/quick-session.md - `start_therapy_session` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/start-therapy-session.md - `util_bool_parse` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-bool-parse.md - `util_cache_control_parse` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-cache-control-parse.md - `util_cagr_simple` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-cagr-simple.md - `util_caip2_build` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-caip2-build.md - `util_caip2_parse` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-caip2-parse.md - `util_categorical_distribution` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-categorical-distribution.md - `util_content_disposition_parse` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-content-disposition-parse.md - `util_content_range_parse` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-content-range-parse.md - `util_csp_directive_parse` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-csp-directive-parse.md - `util_deployment_readiness` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-deployment-readiness.md - `util_dmarc_policy_parse` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-dmarc-policy-parse.md - `util_dns_dmarc_policy_parse` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-dns-dmarc-policy-parse.md - `util_dns_https_record_parse` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-dns-https-record-parse.md - `util_dns_spf_record_parse` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-protocol/util-dns-spf-record-parse.md - …and 1045 more: https://policylayer.com/tools/io-github-davidmosiah-delx-protocol.md ## For agents This record is a snapshot. Live verdicts and the full registry: - Check every server in your MCP config at once: `npx -y policylayer stack` - Vet a server before you add it: install the mcp-precheck skill — `npx skills add https://policylayer.com` (skill text: https://policylayer.com/skill.md) - Query the registry over MCP: endpoint `https://api.policylayer.com/mcp` — tools `check_mcp_server`, `check_mcp_stack`, `check_tool`, `search_registry`, `get_change_events` --- Source: the PolicyLayer MCP registry — one continuously verified record per MCP server. Full record: https://policylayer.com/registry?q=io-github-davidmosiah-delx-protocol · API: https://policylayer.com/registry/api · Policy library: https://policylayer.com/policies/io-github-davidmosiah-delx-protocol