fill_by_label · Visual Annotation MCP: Risk & Policy

WHAT IT DOES

What fill_by_label does on Visual Annotation MCP

AI agents use fill_by_label to create or update resources in Visual Annotation MCP — usually the action step of a workflow, after the agent has gathered context. Every call changes real data in your Visual Annotation MCP environment.

WHY IT NEEDS A POLICY

Why fill_by_label needs a policy

The tool fills form fields, constituting data modification (Write category). Severity is medium because the impact depends on context—filling a benign form is low risk, but an unvetted agent could fill payment forms, login credentials, or other sensitive fields.

From the tool's definition Tool description: 'Fill a field by its associated label text.' This performs form field population, which modifies data in the web page being interacted with.

Attacks that exploit this kind of access

Recommended rule Ready

Verdict

fill_by_label Rate-limited

fill_by_label stays usable, but capped — an agent stuck in a loop can't make hundreds of changes a minute. Everything else on the server is denied unless you say otherwise.

View as policy code

policy.json

{
  "version": "1",
  "default": "deny",
  "tools": {
    "fill_by_label": {
      "limits": [
        {
          "counter": "fill_by_label_rate",
          "window": "minute",
          "max": 30,
          "scope": "grant"
        }
      ]
    }
  }
}

LIMIT THIS TOOL →

Instant setup, no code required.

PolicyLayer is an MCP gateway: every tool call is checked against your rule before it reaches Visual Annotation MCP, so you can run agents against it safely.

FAQ

Questions about fill_by_label

What does the fill_by_label tool do? +

Fill a field by its associated label text. It is categorised as a Write tool in the Visual Annotation MCP MCP Server, which means it can create or modify data. Consider rate limits to prevent runaway writes.

How do I enforce a policy on fill_by_label? +

Register the Visual Annotation MCP server in PolicyLayer and add a rule for fill_by_label: allow, deny, rate-limit, or require approval. Point your MCP client at the PolicyLayer proxy URL and the rule is enforced on every call, before it reaches Visual Annotation MCP. Nothing to install.

What risk level is fill_by_label? +

fill_by_label is a Write tool with medium risk. Write tools should be rate-limited to prevent accidental bulk modifications.

Can I rate-limit fill_by_label? +

Yes. Add a rate_limit block to the fill_by_label rule in your PolicyLayer policy. For example, setting max: 10 and window: 60 limits the tool to 10 calls per minute. Rate limits are tracked per agent session and reset automatically.

How do I block fill_by_label completely? +

Set action: deny in the PolicyLayer policy for fill_by_label. The AI agent will receive a policy violation error and cannot call the tool. You can also include a reason field to explain why the tool is blocked.

What MCP server provides fill_by_label? +

fill_by_label is provided by the Visual Annotation MCP server (mstocker1/visual_annotation_mcp). PolicyLayer sits as a proxy in front of this server to enforce policies before tool calls reach the server.