# benchmark_models

Run the same prompt against multiple LLM providers and compare responses. Returns side-by-side results with latency, token usage, and a summary. Useful for model selection, quality comparison, and cost analysis.

Agent View of the PolicyLayer registry record for `benchmark_models`. HTML page: https://policylayer.com/tools/io-github-homenshum-nodebench/benchmark-models

## Facts

- Tool: `benchmark_models`
- Server: Nodebench (`nodebench-mcp`) — https://policylayer.com/tools/io-github-homenshum-nodebench.md
- Install: `npx -y nodebench-mcp`
- Homepage: https://github.com/HomenShum/nodebench-ai
- Risk category: Execute (High risk)
- Registry record: grade F, identity unverified
- Server rate-limited: no
- Parameters: 0
- Recommended policy verdict: Rate-limited

## Example call (MCP tools/call, JSON-RPC 2.0)

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "benchmark_models",
    "arguments": {}
  }
}
```

## Why benchmark_models is rated High

This tool executes prompts against multiple LLM providers, triggering external API calls whose effects (costs, rate limiting, data sent) depend on the prompt content and selected providers. While not destructive or financial by itself, it performs active execution that impacts external systems.

From the tool's own definition: "Tool description states 'Run the same prompt against multiple LLM providers' — the verb 'Run' and the action of sending prompts to external LLM services indicates execution of operations with external side effects."

## Use case

AI agents invoke benchmark_models to trigger actions in Nodebench. What it does depends on the arguments the agent supplies, and its effects often reach beyond the immediate call: builds kicked off, notifications sent, workflows started.

## Recommended policy (PolicyLayer)

Verdict: **Rate-limited**. Enforced by the PolicyLayer MCP gateway (https://policylayer.com/mcp-gateway) before a call reaches Nodebench:

```json
{
  "version": "1",
  "default": "deny",
  "tools": {
    "benchmark_models": {
      "limits": [
        {
          "counter": "benchmark_models_rate",
          "window": "minute",
          "max": 10,
          "scope": "grant"
        }
      ]
    }
  }
}
```

## Other tools on Nodebench (823)

- `abandon_cycle` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/abandon-cycle.md
- `archiveDocument` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/archivedocument.md
- `cleanup_stale_runs` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/cleanup-stale-runs.md
- `deeptrace_revoke_passport` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/deeptrace-revoke-passport.md
- `delete_learning` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/delete-learning.md
- `delete_sandbox_policy` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/delete-sandbox-policy.md
- `deleteAgentMemory` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/deleteagentmemory.md
- `removeDocumentFromFolder` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/removedocumentfromfolder.md
- `share_revoke_packet_link` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/share-revoke-packet-link.md
- `unload_toolset` — Destructive — https://policylayer.com/tools/io-github-homenshum-nodebench/unload-toolset.md
- `bootstrap_parallel_agents` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/bootstrap-parallel-agents.md
- `build_banking_packet` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/build-banking-packet.md
- `build_company_profile_starter` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/build-company-profile-starter.md
- `build_founder_operating_model` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/build-founder-operating-model.md
- `build_research_digest` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/build-research-digest.md
- `build_shared_context_subscription` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/build-shared-context-subscription.md
- `build_shared_context_subscription_manifest` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/build-shared-context-subscription-manifest.md
- `build_submission_export` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/build-submission-export.md
- `build_temporal_graph` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/build-temporal-graph.md
- `burst_capture` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/burst-capture.md
- `call_driver_tool` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/call-driver-tool.md
- `call_llm` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/call-llm.md
- `call_openclaw_skill` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/call-openclaw-skill.md
- `call_webmcp_tool` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/call-webmcp-tool.md
- `capture_surface_stats` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/capture-surface-stats.md
- `capture_ui_screenshot` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/capture-ui-screenshot.md
- `compile_decision_packet` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/compile-decision-packet.md
- `compile_scenarios` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/compile-scenarios.md
- `compile_tension_model` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/compile-tension-model.md
- `compute_ssim_analysis` — Execute — https://policylayer.com/tools/io-github-homenshum-nodebench/compute-ssim-analysis.md
- …and 793 more: https://policylayer.com/tools/io-github-homenshum-nodebench.md

## For agents

This record is a snapshot. Live verdicts and the full registry:

- Check every server in your MCP config at once: `npx -y policylayer stack`
- Vet a server before you add it: install the mcp-precheck skill — `npx skills add https://policylayer.com` (skill text: https://policylayer.com/skill.md)
- Query the registry over MCP: endpoint `https://api.policylayer.com/mcp` — tools `check_mcp_server`, `check_mcp_stack`, `check_tool`, `search_registry`, `get_change_events`

---

Source: the PolicyLayer MCP registry — one continuously verified record per MCP server. Full record: https://policylayer.com/registry?q=io-github-homenshum-nodebench · API: https://policylayer.com/registry/api · Policy library: https://policylayer.com/policies/io-github-homenshum-nodebench