Computer Use Fundamentals

What You Need to Know

Computer Use is where Cowork crosses the line from processing files and querying APIs to physically driving your computer. When enabled, Claude can see your screen, move your cursor, click buttons, type text, and navigate between applications — the same actions you perform yourself, executed remotely by Claude through periodic screenshots and simulated input.

This isn't a metaphor. Claude literally takes a photograph of your display, analyses what it sees, decides where to click, moves the cursor, and repeats the cycle. It's the most powerful capability in Cowork, and the most dangerous if used carelessly. Understanding exactly how it works — and when not to use it — is the core of Domain 4.

Enabling Computer Use

Computer Use is disabled by default and available only on Pro and Max individual plans. You enable it in Settings > General via a toggle switch. This is a deliberate opt-in: Claude never touches your screen without your explicit decision to turn this on.

How Claude sees your screen

Claude doesn't have direct access to your operating system, application APIs, or the accessibility layer. Instead, it perceives your screen by taking periodic screenshots and analysing the image. It reads text, identifies buttons, recognises UI elements, and determines what's visible — all through visual interpretation of a photograph.

This has a critical security implication: anything visible on your screen is visible to Claude. That includes the target application you want Claude to work in, but also every background window, notification banner, browser tab, and desktop icon. If your email shows a confidential client contract behind the app Claude is working in, that information is captured in the screenshot and processed by the model.

Operational control: the screenshot-analyse-act cycle

Once Claude can see the screen, it controls your computer through a repeating cycle:

1. Screenshot — Claude captures an image of your current display
2. Analyse — the model interprets what it sees: text, buttons, fields, menus, layout
3. Act — Claude moves the cursor, clicks, types, scrolls, or navigates based on its analysis
4. Repeat — another screenshot is taken to observe the result, and the cycle continues

This cycle is inherently slower than connector-based operations. A Connector delivers structured data in seconds through an API. Computer Use must photograph the screen, process the image, decide on an action, execute it, wait for the UI to respond, and photograph again. A task that takes two seconds via a Connector might take two minutes via Computer Use.

The per-app permission model

Computer Use follows a zero-trust model. There's no "allow everything" switch. Each time Claude needs to interact with a new application, it must request your explicit permission. Approve Calculator, and Claude can use Calculator. That approval doesn't extend to Safari, Mail, Finder, or any other application. Every app is a separate door that requires its own key.

This means the permission overhead scales with the number of applications involved in a task. A simple task using one app requires one approval. A complex workflow spanning five apps requires five separate approvals. Factor this into your workflow planning.

The tool priority hierarchy

Claude follows an efficiency hierarchy when deciding how to complete a task:

1. Connectors — structured API access, fastest and most reliable (pre-prepped ingredients in the fridge)
2. Browser — web navigation, slower but works for sites without Connectors (the pantry down the hall)
3. Computer Use — screen interaction, slowest but handles any visible application (walking out to the garden)

This hierarchy isn't just a performance recommendation — it reflects reliability too. Connectors deliver precise, structured data with near-100% success. Browser navigation is less reliable because websites change layout, load slowly, or require authentication. Computer Use has roughly a 50% success rate on complex multi-step tasks because it depends on visual interpretation of an ever-changing screen.

The practical rule: always use the highest-priority tool available for each step. If a Google Calendar Connector is installed, never ask Claude to visually navigate the Calendar website. If a Salesforce Connector exists, never ask Claude to drive the Salesforce desktop app via screen clicks. Reserve Computer Use for applications that have no API, no web interface, and no Connector — legacy desktop software that can only be operated through a GUI.

Computer Use operates outside the sandbox

This is the most important distinction to grasp. File processing, Skills, and Connectors all run inside Cowork's sandboxed environment. Computer Use doesn't. It interacts with your actual desktop, your physical mouse and keyboard, your real applications. There's no VM, no container, no safety net beyond the per-app permission model and the blocked application categories.

This is why Domain 2's folder scoping and Domain 3's Connector-based workflows are preferred: they operate within controlled boundaries. Computer Use breaks those boundaries by design — which is precisely what makes it both powerful and risky.

---

Common Mistakes

---

Hands-On Activity

---

Practice Question

---

Sources

• Let Claude use your computer in Cowork — Anthropic
• Put Claude to work on your computer — Anthropic
• Claude Dispatch: Control Cowork From Your Phone — FindSkill.ai