generate_test_suite · Rockfish MCP Server MCP: Risk & Policy

WHAT IT DOES

What generate_test_suite does on Rockfish MCP Server

AI agents use generate_test_suite to create or update resources in Rockfish MCP Server — usually the action step of a workflow, after the agent has gathered context. Every call changes real data in your Rockfish MCP Server environment.

WHY IT NEEDS A POLICY

Why generate_test_suite needs a policy

The tool's primary action is to create/generate test cases, which is a reversible write operation. It doesn't execute arbitrary code or commands on external systems, doesn't delete data irreversibly, and doesn't move money. While it interacts with ML data, the generated test cases are new outputs that can be modified or removed.

From the tool's definition Tool generates new test cases from a dataset, which creates new artifacts (Q&A test cases) in the system without permanently destroying existing data.

Attacks that exploit this kind of access

Recommended rule Ready

Verdict

generate_test_suite Rate-limited

generate_test_suite stays usable, but capped — an agent stuck in a loop can't make hundreds of changes a minute. Everything else on the server is denied unless you say otherwise.

View as policy code

policy.json

{
  "version": "1",
  "default": "deny",
  "tools": {
    "generate_test_suite": {
      "limits": [
        {
          "counter": "generate_test_suite_rate",
          "window": "minute",
          "max": 30,
          "scope": "grant"
        }
      ]
    }
  }
}

LIMIT THIS TOOL →

Instant setup, no code required.

PolicyLayer is an MCP gateway: every tool call is checked against your rule before it reaches Rockfish MCP Server, so you can run agents against it safely.

FAQ

Questions about generate_test_suite

What does the generate_test_suite tool do? +

Generate Q&A test cases for a dataset from ID or raw CSV. It is categorised as a Write tool in the Rockfish MCP Server MCP Server, which means it can create or modify data. Consider rate limits to prevent runaway writes.

How do I enforce a policy on generate_test_suite? +

Register the Rockfish MCP Server MCP server in PolicyLayer and add a rule for generate_test_suite: allow, deny, rate-limit, or require approval. Point your MCP client at the PolicyLayer proxy URL and the rule is enforced on every call, before it reaches Rockfish MCP Server. Nothing to install.

What risk level is generate_test_suite? +

generate_test_suite is a Write tool with medium risk. Write tools should be rate-limited to prevent accidental bulk modifications.

Can I rate-limit generate_test_suite? +

Yes. Add a rate_limit block to the generate_test_suite rule in your PolicyLayer policy. For example, setting max: 10 and window: 60 limits the tool to 10 calls per minute. Rate limits are tracked per agent session and reset automatically.

How do I block generate_test_suite completely? +

Set action: deny in the PolicyLayer policy for generate_test_suite. The AI agent will receive a policy violation error and cannot call the tool. You can also include a reason field to explain why the tool is blocked.

What MCP server provides generate_test_suite? +

generate_test_suite is provided by the Rockfish MCP Server MCP server (wolfdancer/rockfish-mcp). PolicyLayer sits as a proxy in front of this server to enforce policies before tool calls reach the server.