create_evaluation_pipeline · Respan MCP Server MCP

WHAT IT DOES

What create_evaluation_pipeline does on Respan MCP Server

AI agents use create_evaluation_pipeline to create or update resources in Respan MCP Server — usually the action step of a workflow, after the agent has gathered context. Every call changes real data in your Respan MCP Server environment.

WHY IT NEEDS A POLICY

Why create_evaluation_pipeline needs a policy

This tool creates and persists a new evaluation pipeline configuration, which is a reversible write operation. It does not execute arbitrary code or queries, delete data, or move money. The medium severity reflects that creating evaluation pipelines could affect monitoring/management workflows, but the effect is contained to configuration creation with no broad destructive capability.

From the tool's definition Tool name contains 'create' and description states 'Create an evaluator pipeline' which establishes a new object in the system. The description indicates this tool creates workflows for graders in the Evaluators page UI.

Attacks that exploit this kind of access

Recommended rule Ready

Verdict

create_evaluation_pipeline Rate-limited

create_evaluation_pipeline stays usable, but capped — an agent stuck in a loop can't make hundreds of changes a minute. Everything else on the server is denied unless you say otherwise.

View as policy code

policy.json

{
  "version": "1",
  "default": "deny",
  "tools": {
    "create_evaluation_pipeline": {
      "limits": [
        {
          "counter": "create_evaluation_pipeline_rate",
          "window": "minute",
          "max": 30,
          "scope": "grant"
        }
      ]
    }
  }
}

LIMIT THIS TOOL →

Instant setup, no code required.

PolicyLayer is an MCP gateway: every tool call is checked against your rule before it reaches Respan MCP Server, so you can run agents against it safely.

FAQ

Questions about create_evaluation_pipeline

What does the create_evaluation_pipeline tool do? +

Create an evaluator pipeline (V2 — Blockly visual editor compatible) that renders in the Evaluators page UI. Pipelines wrap committed graders into a workflow. Use this AFTER creating + committing a grader with create_evaluator + commit_evaluator. PATTERNS: - Single grader: steps=[{grader_id:. It is categorised as a Write tool in the Respan MCP Server MCP Server, which means it can create or modify data. Consider rate limits to prevent runaway writes.

How do I enforce a policy on create_evaluation_pipeline? +

Register the Respan MCP Server MCP server in PolicyLayer and add a rule for create_evaluation_pipeline: allow, deny, rate-limit, or require approval. Point your MCP client at the PolicyLayer proxy URL and the rule is enforced on every call, before it reaches Respan MCP Server. Nothing to install.

What risk level is create_evaluation_pipeline? +

create_evaluation_pipeline is a Write tool with medium risk. Write tools should be rate-limited to prevent accidental bulk modifications.

Can I rate-limit create_evaluation_pipeline? +

Yes. Add a rate_limit block to the create_evaluation_pipeline rule in your PolicyLayer policy. For example, setting max: 10 and window: 60 limits the tool to 10 calls per minute. Rate limits are tracked per agent session and reset automatically.

How do I block create_evaluation_pipeline completely? +

Set action: deny in the PolicyLayer policy for create_evaluation_pipeline. The AI agent will receive a policy violation error and cannot call the tool. You can also include a reason field to explain why the tool is blocked.

What MCP server provides create_evaluation_pipeline? +

create_evaluation_pipeline is provided by the Respan MCP Server MCP server (respanai/respan-mcp). PolicyLayer sits as a proxy in front of this server to enforce policies before tool calls reach the server.