Computer use gives Miteos agents the ability to see and interact with your entire desktop — not just web pages or APIs, but any application running on your machine. The agent works the same way a person does: it looks at the screen, decides what to do next, performs the action, checks the result, and repeats. This loop runs continuously until your task is complete, with no manual hand-holding required.
Computer use requires the Miteos Desktop app. This feature is not available in the web browser interface.
How It Works
Each computer-use action follows a tight perception-to-execution loop:
- The agent captures a screenshot of your current screen so it can see exactly what’s on display.
- The screenshot is sent to a vision-capable LLM (Claude with computer-use) that can understand what it’s looking at — buttons, text fields, menus, dialogs, and more.
- The LLM identifies the relevant UI elements and plans the single best next action to move toward your goal.
- The agent executes the action using your mouse and keyboard, with behavior designed to look and feel human.
- The loop repeats — a new screenshot is taken, the result is evaluated, and the next action is chosen — until the task is finished.
Supported Actions
| Action | What It Does |
|---|
click | Move the mouse to a screen coordinate and click |
double_click | Double-click at a coordinate |
type | Type text with variable speed and natural rhythm |
press | Press a single key (Enter, Tab, Escape, and others) |
hotkey | Execute a key combination (Ctrl+C, Cmd+V, and others) |
scroll | Smooth scroll up, down, left, or right |
screenshot | Capture the current screen state for decision-making |
wait | Pause for a specified duration before the next action |
Human-Like Behavior
Miteos agents don’t move the mouse in straight lines or type at a perfectly constant speed — because real people don’t either. Human-like behavior makes automation more reliable on applications that monitor for robotic input patterns, and it produces smoother, easier-to-follow playback in your task timeline.
Here’s what that looks like in practice:
- Bezier mouse curves — The mouse follows a natural curved path to every target, with realistic acceleration at the start and deceleration as it arrives.
- Variable typing speed — Common letter pairs (like “th” or “er”) are typed faster than less common ones, mirroring natural typing rhythm.
- Position jitter on clicks — Clicks land within a small natural radius of the target coordinate rather than hitting the exact pixel every time.
- Micro-delays between actions — Brief, randomized pauses between individual actions break the mechanical regularity that bot-detection systems look for.
Safety Controls
Computer use is powerful, so Miteos includes several layers of control to keep you in charge:
- Approval for risky actions — Before the agent does anything consequential — installing software, deleting files, submitting forms with irreversible effects — it pauses and asks for your confirmation.
- Pause and stop at any time — Use the pause or stop controls in the agent panel to immediately halt execution at any point during a task.
- Takeover Mode — Click Take Over to instantly grab control of your mouse and keyboard. The agent pauses while you’re in control and resumes when you hand it back.
- Full action log — Every screenshot, action, and decision the agent makes is recorded in the task timeline so you can review exactly what happened, step by step.
Use Cases
Computer use is especially valuable when there’s no API or no other way to automate a workflow programmatically:
- Legacy desktop software — Fill forms, extract data, and navigate applications that predate modern APIs.
- Trading platforms — Interact with TradingView and other charting or brokerage platforms that don’t expose developer access.
- Data entry workflows — Move data between systems that can’t be connected directly, including spreadsheets, internal tools, and government portals.
- Any desktop application — If a human can do it by looking at a screen and using a mouse and keyboard, an agent can do it too.
