Computer Use
AI controlling a computer interface through screenshots, clicks and typing to complete tasks.
In plain language
Computer use extends agents beyond APIs to arbitrary graphical interfaces, including legacy software, browsers and desktop apps. The loop is usually screenshot → visual understanding → action generation → execution → observation. Challenges include UI changes, coordinate errors, long-horizon planning and irreversible actions. Production deployments must limit scope, require confirmation for high-risk actions, log every step and provide human takeover. It suits repetitive, well-defined workflows, not high-risk financial transactions.
Example
An agent logs into a supplier portal daily, downloads statements and enters them into a spreadsheet; payment steps must pause for human approval.
Recommended tools
Related terms
Agentic Workflow
Breaking a task into steps planned by AI, executed by tools and reviewed by humans.
AI Agent
An AI system that perceives a goal, plans steps, calls tools and executes tasks.
Tool Use
A model using external tools such as search, code execution or APIs during reasoning.
Agent Memory
A mechanism for an agent to store and retrieve user preferences, past tasks and key facts across sessions.