# start_eval_run

Start a new eval run. Define the test batch upfront with test cases (input, intent, expected behavior), then record results as each case is executed. Rule: no change ships without an eval improvement.

Agent View of the PolicyLayer registry record for `start_eval_run`. HTML page: https://policylayer.com/tools/io-github-homenshum-nodebench/start-eval-run

## Facts

- Tool: `start_eval_run`
- Server: Nodebench (`nodebench-mcp`) — https://policylayer.com/tools/io-github-homenshum-nodebench.md
- Install: `npx -y nodebench-mcp`
- Homepage: https://github.com/HomenShum/nodebench-ai
- Risk category: Execute (High risk)
- Registry record: grade F, identity unverified
- Server rate-limited: no
- Parameters: 0
- Recommended policy verdict: Rate-limited

## Example call (MCP tools/call, JSON-RPC 2.0)

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "start_eval_run",
    "arguments": {}
  }
}
```

## Why start_eval_run is rated High

This tool initiates automated test execution across a batch of predefined test cases. While not destructive or financial, it executes external operations (test runs) whose effects depend on the test case definitions. The phrase 'record results as each case is executed' confirms active execution rather than passive reading.

From the tool's own definition: "'Start a new eval run' with 'test cases' and 'record results as each case is executed' indicates triggering and executing a defined test batch"

## Use case

AI agents invoke start_eval_run to trigger actions in Nodebench. What it does depends on the arguments the agent supplies, and its effects often reach beyond the immediate call: builds kicked off, notifications sent, workflows started.

## Recommended policy (PolicyLayer)

Verdict: **Rate-limited**. Enforced by the PolicyLayer MCP gateway (https://policylayer.com/mcp-gateway) before a call reaches Nodebench:

```json
{
  "version": "1",
  "default": "deny",
  "tools": {
    "start_eval_run": {
      "limits": [
        {
          "counter": "start_eval_run_rate",
          "window": "minute",
          "max": 10,
          "scope": "grant"
        }
      ]
    }
  }
}
```

## Other tools on Nodebench (823)

- `abandon_cycle` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/abandon-cycle.md
- `archiveDocument` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/archivedocument.md
- `cleanup_stale_runs` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/cleanup-stale-runs.md
- `deeptrace_revoke_passport` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/deeptrace-revoke-passport.md
- `delete_learning` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/delete-learning.md
- `delete_sandbox_policy` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/delete-sandbox-policy.md
- `deleteAgentMemory` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/deleteagentmemory.md
- `removeDocumentFromFolder` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/removedocumentfromfolder.md
- `share_revoke_packet_link` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/share-revoke-packet-link.md
- `unload_toolset` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/unload-toolset.md
- `benchmark_models` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/benchmark-models.md
- `bootstrap_parallel_agents` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/bootstrap-parallel-agents.md
- `build_banking_packet` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/build-banking-packet.md
- `build_company_profile_starter` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/build-company-profile-starter.md
- `build_founder_operating_model` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/build-founder-operating-model.md
- `build_research_digest` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/build-research-digest.md
- `build_shared_context_subscription` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/build-shared-context-subscription.md
- `build_shared_context_subscription_manifest` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/build-shared-context-subscription-manifest.md
- `build_submission_export` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/build-submission-export.md
- `build_temporal_graph` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/build-temporal-graph.md
- `burst_capture` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/burst-capture.md
- `call_driver_tool` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/call-driver-tool.md
- `call_llm` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/call-llm.md
- `call_openclaw_skill` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/call-openclaw-skill.md
- `call_webmcp_tool` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/call-webmcp-tool.md
- `capture_surface_stats` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/capture-surface-stats.md
- `capture_ui_screenshot` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/capture-ui-screenshot.md
- `compile_decision_packet` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/compile-decision-packet.md
- `compile_scenarios` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/compile-scenarios.md
- `compile_tension_model` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/compile-tension-model.md
- …and 793 more: https://policylayer.com/tools/io-github-homenshum-nodebench.md

## For agents

This record is a snapshot. Live verdicts and the full registry:

- Check every server in your MCP config at once: `npx -y policylayer stack`
- Vet a server before you add it: install the mcp-precheck skill — `npx skills add https://policylayer.com` (skill text: https://policylayer.com/skill.md)
- Query the registry over MCP: endpoint `https://api.policylayer.com/mcp` — tools `check_mcp_server`, `check_mcp_stack`, `check_tool`, `search_registry`, `get_change_events`

---

Source: the PolicyLayer MCP registry — one continuously verified record per MCP server. Full record: https://policylayer.com/registry?q=io-github-homenshum-nodebench · API: https://policylayer.com/registry/api · Policy library: https://policylayer.com/policies/io-github-homenshum-nodebench