Why ax exec
ax exec gives a remote command a job identity and lets the job outlive the caller. The runner accepts the request and supervises the process and its descendants. It records the outcome and keeps the output. If the client stops waiting, closes its terminal or loses its session, the job keeps running.
The failure modes
| Failure | Without a job identity | With ax exec |
|---|---|---|
| A wait loop matches its own process name | Shells spin after the work is done; one session found two shells from a 36-hour-old wait loop still stuck | ax exec watch follows the recorded job; no pgrep or pkill loop is needed |
| A cleanup command matches the controlling shell | The remote shell dies, the command chain stops, and the command's own exit status is lost | ax exec cancel "$job_id" addresses one supervised job; ax exec status confirms the terminal state |
| The SSH connection times out | The client knows it stopped waiting; the remote outcome is unknown, and acceptance may be uncertain | A saved acceptance keeps the job on the runner; reconcile the saved request ID before retrying |
| A tool displays only part of a long log | The agent reruns the command or reads a partial file | ax exec output reads retained stdout and stderr from a cursor; next_cursor continues |
| The agent session ends | The next session has no command context | The local journal keeps request and job IDs, the command, remote cwd and caller-supplied origin labels |
| The submission reply is lost | A blind retry can run the command twice | The CLI saves the request before sending; retry the same --request-id with unchanged arguments |
| A command is quiet | Silence is read as a hang, or a hang is read as progress | Status, timestamps and retained output record what happened; they do not prove progress. A deadline and cancellation are separate controls |
Jobs across sessions
A submission creates two references. The request ID names the request the client saved before it contacted the runner, and that request never changes. The job ID names the accepted execution on the runner; status, output and cancellation all use it. The receipt also carries the target ID and the runner name.
The local journal records the request, job and target references, the executable and argument array, and the remote cwd and local workspace. It also keeps the harness and session labels the caller supplied. A later session using the same client identity can list and recover the job instead of inferring ownership from process names.
This entry comes from a real run on the documentation machine. Your IDs and workspace differ.
ax exec ls -a --format json | jq '
.[] | select(.job.job_id == "e222c08a-3be8-499b-adfe-18737cabb544") |
{request_id, job_id: .job.job_id, state, executable, argv,
harness: .origin.harness, session: .origin.session_id}'
{
"request_id": "b24f540e-7e85-4fc4-b060-4f507d55fe0d",
"job_id": "e222c08a-3be8-499b-adfe-18737cabb544",
"state": "succeeded",
"executable": "/bin/echo",
"argv": [
"ax-docs-attribution"
],
"harness": "claude",
"session": null
}
The harness label is there because the submission supplied --harness claude. A session label appears when the caller supplies --harness-session. Both labels come from the caller. They do not grant access.
ax exec ls lists the current caller's jobs in this workspace. -a adds that caller's other workspaces. --refresh asks the runner for current status, gives up after ten seconds, and keeps the last known status if it fails.
ax exec ls --refresh
ax exec ls -a --format json
ax exec status "$job_id"
ax exec watch "$job_id"
ax exec output "$job_id" --cursor 0 --format json
watch prints retained output and follows until the job ends and the output runs out. Pressing Ctrl-C stops the local watch and leaves the remote job running. output returns a bounded page with chunks, next_cursor, eof and truncated. Pass next_cursor to read more without repeating old chunks.
If a submission's reply is uncertain, retry the original request with the same --request-id, unchanged arguments and the same stdin bytes when --stdin was used. The CLI rejects changed arguments, and it rejects a retry after the request-retention window. New work needs a fresh request ID.
Cancelling the right job
Cancellation names the job ID returned at submission. There is no process-name pattern to write, so the cancel cannot hit another session's shell or the client that sent it.
ax exec cancel "$job_id"
ax exec status "$job_id" --format json | jq '{state, signal, reason_code}'
{
"state": "cancelled",
"signal": 9,
"reason_code": "cancelled"
}
The job was running when the cancel was sent; the terminal state appeared a few seconds later. A cancel is not an instant stop. A client timeout or quiet output does not mean the command stopped, so check the status after the request.
On Linux, cancellation targets the supervised cgroup, including descendants. On macOS, it signals the payload's process group; a command that moves descendants into a different session can escape that group.
What survives
- An accepted job can continue while the client is offline. Reconnect to retrieve its recorded state and retained output with the same job reference.
- Restarting the runner API does not stop accepted jobs on Linux with systemd or macOS with launchd. A host reboot can.
- New deployments retain output for one day and request identities for seven days. Collect output you need to keep longer.
- The local journal belongs to the submitting client identity and local workspace.
-aadds that caller's other workspaces; the journal is not shared between devices. - The
openexecution profile inherits the configured account's filesystem and network access. It is not a sandbox. Commands run with that account's permissions. - A Linux system installation separates the service account, the privileged helper and the job account. A user installation runs the service and the job as one account.
- A normal submission does not copy project files. To transfer them, see Workspace file transfer.
- The release targets are macOS arm64, Linux x86_64, and Linux aarch64. Windows agents use the Linux backend through WSL.
Try it
Your first remote job deploys a runner, submits a command, closes the client, then recovers the result. Agent prompts has prompts for transcript mining, runner discovery, job recovery and messaging.