Show HN: Orbit – Structured Python control over AI computer use agents
TL;DR
Orbit is an open-source Python framework for structured control over AI computer use agents (CUAs), avoiding black-box behavior.
Key Points
- Each workflow step gets its own model, budget, and typed output via Pydantic, while sharing session context across steps.
- Instead of screenshots, Orbit uses the OS accessibility tree – faster and more reliable than pure vision models.
- Developers can mix cheap and expensive models per step and steer the agent mid-task when it gets stuck.
Nauti's Take
One GitHub star and zero HN comments – this screams 'early proof of concept'. That said, the core idea is sound: anyone serious about CUAs in production pipelines needs exactly this kind of structured layer between natural language and Python logic.
Choosing the accessibility tree over screenshots is a smart move – less token-heavy, less brittle. The open question is whether Orbit holds up against the chaos of real desktop environments or only shines in controlled demo setups.
Video
Sources