operations / agent
typesafe-computer-use
A command-line agent that drives a Mac toward a plain-English goal by reading the screen with OCR and asking a TypeSafe classifier for the next action.
At a glance
- Purpose
- A command-line agent that drives a Mac toward a plain-English goal by reading the screen with OCR and asking a TypeSafe classifier for the next action.
- Runs in
- Local agent with shell
- Operating systems
- macOS, Linux, Windows
- Needs
- Runtime: Python 3.12+ and uv; Needs internet: Yes; Needs accounts or API keys beyond your agent: Not stated
- Files and data
- Reads files: Yes; Writes files: Yes; Sends data to other services: Not stated
- Tool type
- Agent
- Delivery
- From GitHub
- Version
- Latest on the main branch
- Access
- Code on GitHub
- Compatibility
- According to the README: macOS 14 or newer with Python 3.12 or newer and uv. The README calls Windows 10 and 11 experimental. The README lists Google Chrome as the default browser and CLICKER_BROWSER as its configuration variable. The README lists ANTHROPIC_API_KEY for the writer and names alternative models or endpoints including LM Studio, Ollama and any OpenAI-compatible server. The README links docs for a browser backend that uses Chrome DevTools, and for a sandbox that is a Linux computer in a container. The OSWorld benchmark needs a Linux host with KVM.
- External costs
- According to the README, TYPESAFE_API_KEY is required for every decision, and the README says TypeSafe sells the decision model. ANTHROPIC_API_KEY is optional and is used for type_text, writer-proposed URLs and the final answer. The README says the writer can run on another model or endpoint. The README gives the author's own cost and speed comparison against a frontier model but no plan or subscription prices. The model providers you connect may charge for usage.
- Permissions
- The README's setup installs dependencies with uv sync and reads keys from a repository-root .env file, with TYPESAFE_API_KEY required and ANTHROPIC_API_KEY available for the writer. The README says the agent reads the screen through Vision OCR and the accessibility tree and drives the real mouse and keyboard. The README requires Screen Recording and Accessibility permissions for the terminal. The README lists Google Chrome as the default browser and CLICKER_EMAIL as enabling type_email. The README describes a dry run without --act and machine control with --act, up to 100 steps by default. The README gives Ctrl-C or moving the mouse to the top-left corner as stop controls and says each run writes runs/<timestamp>/. The README also describes scripts/osworld setup fetching OSWorld and installing jev beside it.
- Usage terms
- MIT License, as stated in the repository. See its LICENSE file for the exact terms.
Code on GitHub
awlevin/typesafe-computer-use
MITComputer use for about $0.0002 a step: OCR the screen, classify the next action with TypeSafe, click. macOS.
Topics: ai-agents automation computer-use macos ocr typesafe
Public GitHub data fetched · 0 days old
View on GitHub ↗What it does
According to the README, typesafe-computer-use, or jev, is a computer-use loop for plain-English goals on a Mac. The README describes reading a crop of the frontmost window with Vision OCR and labelled controls from the accessibility tree. The README says code adds state such as focused fields and previously attempted actions, and a TypeSafe request selects the action kind, item and site.The README describes the classifier as selecting actions and the writer as supplying free text or reading the screen after a classifier stop. The README says the writer can answer or return control with a focus or a question, and the loop stops on low confidence, a stall or its step limit.
The README labels the project beta and under heavy development, and advises a dry run because it drives the real mouse and keyboard.
Who it is for
Mac users and developers who want to try a computer-use agent that picks actions with a TypeSafe classifier.Example tasks
Run a dry run with uv run clicker "open the Playground" to see what the agent would do.Drive the machine toward a goal with uv run clicker "open the Playground" --act.
Run a longer, slower task with uv run clicker "log in" --act --steps 20 --delay 3.
Capture and open the annotated screen with uv run clicker-inspect "any goal".
Replay a stalled run offline from its run folder.