# run_eval

Run the offline eval harness against the held-out intent set. Returns aggregate scores and per-intent results.

Agent View of the PolicyLayer registry record for `run_eval`. HTML page: https://policylayer.com/tools/adia-ai-mcp/run-eval

## Facts

- Tool: `run_eval`
- Server: Adia Ai (`@adia-ai/mcp`) — https://policylayer.com/tools/adia-ai-mcp.md
- Install: `npx -y @adia-ai/mcp`
- Homepage: https://www.npmjs.com/package/@adia-ai/mcp
- Risk category: Execute (High risk)
- Registry record: grade C, identity unverified
- Server rate-limited: no
- Parameters: 0
- Recommended policy verdict: Rate-limited

## Example call (MCP tools/call, JSON-RPC 2.0)

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "run_eval",
    "arguments": {}
  }
}
```

## Why run_eval is rated High

run_eval triggers real processes with real consequences. An agent gone sideways doesn't fire it once. It starts dozens of builds, sends mass notifications, or burns through compute before anyone looks up.

## Use case

AI agents invoke run_eval to trigger actions in Adia Ai. What it does depends on the arguments the agent supplies, and its effects often reach beyond the immediate call: builds kicked off, notifications sent, workflows started.

## Recommended policy (PolicyLayer)

Verdict: **Rate-limited**. Enforced by the PolicyLayer MCP gateway (https://policylayer.com/mcp-gateway) before a call reaches Adia Ai:

```json
{
  "version": "1",
  "default": "deny",
  "tools": {
    "run_eval": {
      "limits": [
        {
          "counter": "run_eval_rate",
          "window": "minute",
          "max": 10,
          "scope": "grant"
        }
      ]
    }
  }
}
```

## Other tools on Adia Ai (42)

- `submit_feedback` — Other — https://policylayer.com/tools/adia-ai-mcp/submit-feedback.md
- `assemble_context` — Read — https://policylayer.com/tools/adia-ai-mcp/assemble-context.md
- `audit_structure` — Read — https://policylayer.com/tools/adia-ai-mcp/audit-structure.md
- `check_anti_patterns` — Read — https://policylayer.com/tools/adia-ai-mcp/check-anti-patterns.md
- `classify_intent` — Read — https://policylayer.com/tools/adia-ai-mcp/classify-intent.md
- `compose_from_chunks` — Read — https://policylayer.com/tools/adia-ai-mcp/compose-from-chunks.md
- `factory_status` — Read — https://policylayer.com/tools/adia-ai-mcp/factory-status.md
- `get_catalog_ladder` — Read — https://policylayer.com/tools/adia-ai-mcp/get-catalog-ladder.md
- `get_catalog_tiers` — Read — https://policylayer.com/tools/adia-ai-mcp/get-catalog-tiers.md
- `get_chunk` — Read — https://policylayer.com/tools/adia-ai-mcp/get-chunk.md
- `get_component_map` — Read — https://policylayer.com/tools/adia-ai-mcp/get-component-map.md
- `get_composition` — Read — https://policylayer.com/tools/adia-ai-mcp/get-composition.md
- `get_graph` — Read — https://policylayer.com/tools/adia-ai-mcp/get-graph.md
- `get_quality_metrics` — Read — https://policylayer.com/tools/adia-ai-mcp/get-quality-metrics.md
- `get_registry_map` — Read — https://policylayer.com/tools/adia-ai-mcp/get-registry-map.md
- `get_state` — Read — https://policylayer.com/tools/adia-ai-mcp/get-state.md
- `get_training_gaps` — Read — https://policylayer.com/tools/adia-ai-mcp/get-training-gaps.md
- `get_traits` — Read — https://policylayer.com/tools/adia-ai-mcp/get-traits.md
- `get_wiring_catalog` — Read — https://policylayer.com/tools/adia-ai-mcp/get-wiring-catalog.md
- `get_wiring_registry` — Read — https://policylayer.com/tools/adia-ai-mcp/get-wiring-registry.md
- `list_patterns` — Read — https://policylayer.com/tools/adia-ai-mcp/list-patterns.md
- `lookup_chunk` — Read — https://policylayer.com/tools/adia-ai-mcp/lookup-chunk.md
- `lookup_component` — Read — https://policylayer.com/tools/adia-ai-mcp/lookup-component.md
- `orient_app` — Read — https://policylayer.com/tools/adia-ai-mcp/orient-app.md
- `protocol_status` — Read — https://policylayer.com/tools/adia-ai-mcp/protocol-status.md
- `refine_ui` — Read — https://policylayer.com/tools/adia-ai-mcp/refine-ui.md
- `scaffold_app` — Read — https://policylayer.com/tools/adia-ai-mcp/scaffold-app.md
- `scaffold_page` — Read — https://policylayer.com/tools/adia-ai-mcp/scaffold-page.md
- `search_chunks` — Read — https://policylayer.com/tools/adia-ai-mcp/search-chunks.md
- `search_patterns` — Read — https://policylayer.com/tools/adia-ai-mcp/search-patterns.md
- …and 12 more: https://policylayer.com/tools/adia-ai-mcp.md

## For agents

This record is a snapshot. Live verdicts and the full registry:

- Check every server in your MCP config at once: `npx -y policylayer stack`
- Vet a server before you add it: install the mcp-precheck skill — `npx skills add https://policylayer.com` (skill text: https://policylayer.com/skill.md)
- Query the registry over MCP: endpoint `https://api.policylayer.com/mcp` — tools `check_mcp_server`, `check_mcp_stack`, `check_tool`, `search_registry`, `get_change_events`

---

Source: the PolicyLayer MCP registry — one continuously verified record per MCP server. Full record: https://policylayer.com/registry?q=adia-ai-mcp · API: https://policylayer.com/registry/api · Policy library: https://policylayer.com/policies/adia-ai-mcp