v0.7.1 154 KB static binary MIT

linux-use

Agent-first, CLI-first GUI control for Linux desktops — the Linux answer to Windows-Use. It reads the AT-SPI2 accessibility tree and acts on it, so an agent drives real applications without screenshots or pixel coordinates.

It ships zero LLM code. Windows-Use is a Python library with the model inside it; linux-use inverts that — it is a pure sensor/actuator binary, and your agent is the loop.

The loop

# what is running and exposes accessibility
$ linux-use apps

# interactive elements, each with a ref
$ linux-use state --app gnome-calculator
{"count":28,"walk_ms":48,"method":"walk","elements":[
  {"ref":"gnome-calculator:/0/1/0/1/0/0/1/0/22#18124260",
   "role":"push button","name":"7","cx":114,"cy":376,"action":"click"}, ...]}

# act on the widget itself — no pointer, no focus, works occluded
$ linux-use act 'gnome-calculator:/0/1/0/1/0/0/1/0/22#18124260'

# wait for the UI to change instead of polling it
$ linux-use watch --app gnome-calculator --role editbar --duration 5000

What makes it different

Acts on widgets, not pixels

Invokes a widget's own accessible action — no focus stealing, no pointer movement, and it works while the window is fully hidden behind another one.

Wrong answers are refused

Refs carry a fingerprint. A stale one exits 83 with a fix suggestion; an element whose identity can't be proven is reported unusable rather than guessed.

Events, not polling

Real AT-SPI callbacks streamed as NDJSON, filterable by app, role and event type — with exact stop conditions so an agent can block on a change.

Incomplete answers are loud

A truncated or depth-limited read says so, in the JSON and on stderr — because "0 elements" silently meaning "I didn't look" is how agents get misled.

What works, and what is untested

GTK 3/4 apps	verified	6 apps: read, act, type, watch
Qt 5 apps	verified	menus actuated via ShowMenu
Chromium/Edge web content	needs a flag	`--force-renderer-accessibility`
Electron, Java, Flutter	not tested	—
Wayland	click/key broken	AT-SPI paths still work

Tested on exactly one configuration — Ubuntu 22.04.3, GNOME 42.9, X11, x86-64 — by hand. There is no automated test suite yet: the numbers here are real measurements, but nothing re-checks them on commit. Treat it as a v0.x tool.

Install

sudo apt install libatspi2.0-dev libxtst-dev libx11-dev libglib2.0-dev
gsettings set org.gnome.desktop.interface toolkit-accessibility true
./build.sh          # or grab the binary from Releases
linux-use doctor    # tells you what's missing

X11 today. On Wayland the accessibility paths still work; synthetic input does not.

Built with machin

linux-use is written in machin (MFL) — the Machine-First Language: terse enough for an agent to write, compiled through C to a single tiny native binary with HTTP, SQLite and crypto built in. No runtime, no dependencies.

The whole tool is one source file plus a small C shim for the AT-SPI FFI boundary. See awesome-machin for other things built with it.

It follows the agent-first CLI conventions at cli-specs.intrane.fr: JSON on stdout, context on stderr, semantic exit codes, typed errors with suggestions, an embedded guide, and a daemon lifecycle.