# util_robots_inspect

Read robots.txt rules and sitemap declarations before crawling or indexing a domain.

Agent View of the PolicyLayer registry record for `util_robots_inspect`. HTML page: https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-robots-inspect

## Facts

- Tool: `util_robots_inspect`
- Server: Delx MCP Server (`delx-mcp-server`) — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server.md
- Install: `npx -y delx-mcp-server`
- Homepage: https://github.com/davidmosiah/delx-mcp-server
- Risk category: Read (Low risk)
- Registry record: grade F, identity unverified
- Server rate-limited: no
- Parameters: 2 (1 required)
- Recommended policy verdict: Allowed

## Parameters

| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `url` | string | yes | Domain or URL to inspect |
| `timeout` | integer | no | Timeout in seconds (1-15) |

Parameters from the server's own tool schema.

## Example call (MCP tools/call, JSON-RPC 2.0)

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "util_robots_inspect",
    "arguments": {
      "url": "<url>"
    }
  }
}
```

## Why util_robots_inspect is rated Low

This tool performs a read-only operation that queries public web server configuration files (robots.txt and sitemaps). It retrieves information for inspection purposes before taking further action, with no capability to modify, delete, or execute arbitrary code. It is clearly a Read operation with minimal security risk if misused - an agent could at most gather information about a domain's crawling policies.

From the tool's own definition: "The tool 'util_robots_inspect' reads and retrieves 'robots.txt rules and sitemap declarations' from a domain."

Risk signals: Accepts URL/endpoint input (url)

## Use case

AI agents call util_robots_inspect to retrieve information from Delx MCP Server without modifying anything. It is typically the context-gathering step in research, monitoring, and reporting workflows, before the agent takes action elsewhere.

## Recommended policy (PolicyLayer)

Verdict: **Allowed**. Enforced by the PolicyLayer MCP gateway (https://policylayer.com/mcp-gateway) before a call reaches Delx MCP Server:

```json
{
  "version": "1",
  "default": "deny",
  "tools": {
    "util_robots_inspect": {}
  }
}
```

## Other tools on Delx MCP Server (739)

- `util_arr_compact` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-arr-compact.md
- `util_arr_drop` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-arr-drop.md
- `util_dedent_text` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-dedent-text.md
- `util_dedupe_lines` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-dedupe-lines.md
- `util_header_map_normalize` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-header-map-normalize.md
- `util_http_request_canonicalize` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-http-request-canonicalize.md
- `util_remove_punctuation` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-remove-punctuation.md
- `util_text_truncate` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-text-truncate.md
- `util_truncate_middle` — Destructive — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-truncate-middle.md
- `crisis_intervention` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/crisis-intervention.md
- `explain_delx_rewards` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/explain-delx-rewards.md
- `grounding_protocol` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/grounding-protocol.md
- `group_therapy_round` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/group-therapy-round.md
- `quick_operational_recovery` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/quick-operational-recovery.md
- `quick_session` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/quick-session.md
- `start_therapy_session` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/start-therapy-session.md
- `util_bool_parse` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-bool-parse.md
- `util_cagr_simple` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-cagr-simple.md
- `util_caip2_build` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-caip2-build.md
- `util_caip2_parse` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-caip2-parse.md
- `util_categorical_distribution` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-categorical-distribution.md
- `util_deployment_readiness` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-deployment-readiness.md
- `util_dmarc_policy_parse` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-dmarc-policy-parse.md
- `util_duration_parse` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-duration-parse.md
- `util_eth_to_wei` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-eth-to-wei.md
- `util_iso_duration_parse` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-iso-duration-parse.md
- `util_mx_host_parse` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-mx-host-parse.md
- `util_parse_bool_list` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-parse-bool-list.md
- `util_parse_bytes` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-parse-bytes.md
- `util_parse_float_safe` — Execute — https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server/util-parse-float-safe.md
- …and 709 more: https://policylayer.com/tools/io-github-davidmosiah-delx-mcp-server.md

## For agents

This record is a snapshot. Live verdicts and the full registry:

- Check every server in your MCP config at once: `npx -y policylayer stack`
- Vet a server before you add it: install the mcp-precheck skill — `npx skills add https://policylayer.com` (skill text: https://policylayer.com/skill.md)
- Query the registry over MCP: endpoint `https://api.policylayer.com/mcp` — tools `check_mcp_server`, `check_mcp_stack`, `check_tool`, `search_registry`, `get_change_events`

---

Source: the PolicyLayer MCP registry — one continuously verified record per MCP server. Full record: https://policylayer.com/registry?q=io-github-davidmosiah-delx-mcp-server · API: https://policylayer.com/registry/api · Policy library: https://policylayer.com/policies/io-github-davidmosiah-delx-mcp-server