An AI agent that controls a desktop or browser via screenshots and clicks.
1 min readupdated 2026-07-06
/ quick answer
A computer-use agent perceives a screen (screenshots or DOM snapshots), reasons about the UI, and issues mouse/keyboard actions to complete a task. Anthropic's Claude Computer Use and OpenAI's Operator are the canonical examples. An AI agent that controls a desktop or browser via screenshots and clicks.
A computer-use agent is an AI that operates a real computer the way a person does — moving the mouse, typing on the keyboard, reading the screen with vision, and switching between apps. Anthropic shipped the first production version (Claude with computer use) in October 2024; OpenAI's Operator and Google's Project Mariner followed in 2025. These agents matter because they can automate any web or desktop task, including tools without APIs, without waiting for a vendor integration. The trade-offs today are still real: they are slower than an API call, more expensive per action, and can drift on long tasks. But for repetitive, brittle, GUI-only workflows — legacy admin panels, portals with no API, cross-app reconciliation — they are already the fastest way to ship an automation.
Definition
A computer-use agent perceives a screen (screenshots or DOM snapshots), reasons about the UI, and issues mouse/keyboard actions to complete a task. Anthropic's Claude Computer Use and OpenAI's Operator are the canonical examples.
Example
Ask the agent to 'download last month's Stripe payout CSV and email it to finance' — it opens the browser, logs in with saved credentials, downloads the file and drafts the email.
Are computer-use agents safe to run on my own machine?
Best practice is to run them inside a sandboxed VM or container with scoped credentials and no access to sensitive data. Do not point one at your primary workstation.
How reliable are computer-use agents in 2026?
Reliable for short well-scoped tasks (5–20 steps). Long-horizon tasks still drift and need checkpoints, retries or human review at key steps.
When should I use a computer-use agent instead of an API integration?
When the target app has no API, the API is worse than the UI, or the workflow spans several apps and building custom glue would cost more than tolerating a slower agent run.