检测当前窗口中所有 UI 元素的位置。返回每个元素相对于窗口的位置（百分比和区域）。可选择将结果保存到程序对应的 MD 布局文件。
AI agents call screen_detect_ui to retrieve information from Screen Agent without modifying anything — typically the context-gathering step in research, monitoring, and reporting workflows, before the agent takes action elsewhere.
This tool performs UI element detection and spatial analysis through OCR and UI framework inspection. It retrieves information about screen layout without executing commands, modifying data, or triggering external operations. The saving of results to a local MD file is metadata collection, not a destructive or write operation on user data or system state.
From the tool's definition Tool description states it 'detects the location of all UI elements in the current window' and 'returns the position of each element relative to the window (percentage and area)'.
Attacks that exploit this kind of access
检测当前窗口中所有 UI 元素的位置。返回每个元素相对于窗口的位置（百分比和区域）。可选择将结果保存到程序对应的 MD 布局文件。. It is categorised as a Read tool in the Screen Agent MCP Server, which means it retrieves data without modifying state.
Register the Screen Agent MCP server in PolicyLayer and add a rule for screen_detect_ui: allow, deny, rate-limit, or require approval. Point your MCP client at the PolicyLayer proxy URL and the rule is enforced on every call, before it reaches Screen Agent. Nothing to install.
screen_detect_ui is a Read tool with low risk. Read-only tools are generally safe to allow by default.
Yes. Add a rate_limit block to the screen_detect_ui rule in your PolicyLayer policy. For example, setting max: 10 and window: 60 limits the tool to 10 calls per minute. Rate limits are tracked per agent session and reset automatically.
Set action: deny in the PolicyLayer policy for screen_detect_ui. The AI agent will receive a policy violation error and cannot call the tool. You can also include a reason field to explain why the tool is blocked.
screen_detect_ui is provided by the Screen Agent MCP server (lqszhsp/screen-agent). PolicyLayer sits as a proxy in front of this server to enforce policies before tool calls reach the server.
Every MCP server has a record like this.
Type a name, get the same breakdown: verified identity, auth posture, risk grade, capabilities, recommended policy.
Teams ship this data inside their own products. See what a licence covers →