createDatasetRunItem · Langfuse Mcp Extended MCP

WHAT IT DOES

What createDatasetRunItem does on Langfuse Mcp Extended

AI agents use createDatasetRunItem to create or update resources in Langfuse Mcp Extended — usually the action step of a workflow, after the agent has gathered context. Every call changes real data in your Langfuse Mcp Extended environment.

WHY IT NEEDS A POLICY

Why createDatasetRunItem needs a policy

This tool creates a new record (dataset run item) that establishes a relationship between existing entities (trace/observation and dataset item). This is a reversible write operation—the created link can be undone (particularly evident from sibling tool 'deleteScore' and 'deleteTrace' showing deletion capabilities exist). It does not execute arbitrary code, delete data irreversibly, or involve financial transactions.

From the tool's definition Tool name 'createDatasetRunItem' and description 'Create a dataset run item' indicate a creation operation that modifies data by linking a trace/observation to a dataset item.

Attacks that exploit this kind of access

Recommended rule Ready

Verdict

createDatasetRunItem Rate-limited

createDatasetRunItem stays usable, but capped — an agent stuck in a loop can't make hundreds of changes a minute. Everything else on the server is denied unless you say otherwise.

View as policy code

policy.json

{
  "version": "1",
  "default": "deny",
  "tools": {
    "createDatasetRunItem": {
      "limits": [
        {
          "counter": "createdatasetrunitem_rate",
          "window": "minute",
          "max": 30,
          "scope": "grant"
        }
      ]
    }
  }
}

LIMIT THIS TOOL →

Instant setup, no code required.

PolicyLayer is an MCP gateway: every tool call is checked against your rule before it reaches Langfuse Mcp Extended, so you can run agents against it safely.

FAQ

Questions about createDatasetRunItem

What does the createDatasetRunItem tool do? +

Create a dataset run item linking a trace/observation to a dataset item for evaluation. It is categorised as a Write tool in the Langfuse Mcp Extended MCP Server, which means it can create or modify data. Consider rate limits to prevent runaway writes.

How do I enforce a policy on createDatasetRunItem? +

Register the Langfuse Mcp Extended MCP server in PolicyLayer and add a rule for createDatasetRunItem: allow, deny, rate-limit, or require approval. Point your MCP client at the PolicyLayer proxy URL and the rule is enforced on every call, before it reaches Langfuse Mcp Extended. Nothing to install.

What risk level is createDatasetRunItem? +

createDatasetRunItem is a Write tool with medium risk. Write tools should be rate-limited to prevent accidental bulk modifications.

Can I rate-limit createDatasetRunItem? +

Yes. Add a rate_limit block to the createDatasetRunItem rule in your PolicyLayer policy. For example, setting max: 10 and window: 60 limits the tool to 10 calls per minute. Rate limits are tracked per agent session and reset automatically.

How do I block createDatasetRunItem completely? +

Set action: deny in the PolicyLayer policy for createDatasetRunItem. The AI agent will receive a policy violation error and cannot call the tool. You can also include a reason field to explain why the tool is blocked.

What MCP server provides createDatasetRunItem? +

createDatasetRunItem is provided by the Langfuse Mcp Extended MCP server (langfuse-mcp-extended). PolicyLayer sits as a proxy in front of this server to enforce policies before tool calls reach the server.