Workspace file transfer

Copy your working files to a runner, then start a supervised job against the completed transfer. Repeated pushes leave unchanged files and their modification times untouched. Pull copies runner files back to the client through the same transfer path, reversed: same selection, verification, records and recovery. You can recover every transfer and job by their IDs.

Workspace transfer is built on Portal Sync, the transfer engine that resolves the selection, plans the change set, moves payload over the authenticated connection, and checks delivered bytes before reporting completion.

Compared with rsync and scp

rsync and scp copy files over SSH, and the copy ends with the session. Workspace transfer runs over ax's authenticated connection and adds selection safety, verified delivery, records and job binding:

Workspace transfer is not a synchronization daemon: nothing runs continuously, and rsync remains the right tool for arbitrary mirroring.

Workspace policy

The first push or pull — including ax exec run --workspace — sets up the workspace policy automatically before transferring, so there is nothing extra to run. Setup uses the selected profile’s existing root key and records the client and runner’s access. File contents travel over Aster RPC; this does not start continuous file synchronization.

A --dry-run preview never changes anything, so it does not provision the policy either: on a fresh runner it reports missing setup instead of creating one. To review a dry run before your first transfer, prepare the policy first with ax workspace setup --node build. Repeating setup keeps that policy, and a revoked or deleted policy record is not reactivated by setup.

Preview and push

Push copies into a directory on the runner. Choose an existing one that the execution account can use, and create it through a normal job if needed:

ax exec run --node build --follow -- /bin/mkdir -p /home/builder/project

Run these on the client computer:

ax workspace push --node build --source . --to /home/builder/project --dry-run
ax workspace push --node build --source . --to /home/builder/project

The preview lists planned changes, source selections and exclusions, and free space on the destination. It does not publish files or change the destination. The real push returns after every selected path has completed. Files sent over the connection are checked against their delivered bytes; unchanged files may use filesystem identity and timestamps.

At a Git worktree root, selection defaults to tracked working-tree files, including staged additions and local edits. Untracked files require an explicit --include. Git history and toolchains are not copied. Use --directory to select directory contents instead.

Sensitive names are excluded even when tracked. The filter applies to every path segment, case-insensitively: names starting with .env, ending in .pem or .key, and the exact names .ssh, .aws, .cloudflare, .gnupg, id_rsa, id_dsa, id_ecdsa and id_ed25519. So config/.env.production and nested/.aws/credentials are both skipped. --allow-sensitive relative/path lets one exact relative path through (repeatable); it never disables the filter, and --include does not override it. Repository internals such as .git stay excluded even when named. Use repeatable --exclude patterns and --include paths to adjust selection, and review the dry run.

Nothing is deleted by default. --delete currently removes tracked paths that are missing from the Git working tree. Use repeatable --delete-path path for explicit source-absent paths. This first version does not perform a full mirror of arbitrary destination contents.

Run with your files

ax exec run --node build --cwd /home/builder/project --workspace . -- cargo test

This records the request, publishes the selected input, waits for transfer completion and binds that directory to the allocated job before launch. Submission then returns the normal job receipt. Use --follow, or recover the job later with ax exec status and ax exec output.

--workspace selects the local files; --cwd selects their remote destination. If --cwd is omitted, ax uses the saved runner default, which may be the account's home directory. Use a project directory when that default contains the runner's private state. Ax reports both remote paths if they overlap.

The combined form uses the default source selection. For custom exclusions, push explicitly first, then run against the remote directory. Ordinary jobs use the account’s filesystem access; the workspace is not a sandbox.

Pull files back

Pulls run the same transfer in the other direction: the runner serves the source and your client receives into a local directory. Bring build outputs or logs home with:

mkdir -p results
ax workspace pull --node build --from /home/builder/project --to ./results --dry-run
ax workspace pull --node build --from /home/builder/project --to ./results

--from is an absolute path on the runner; --to is a local directory that must already exist. The dry run reports the selection the runner resolved, the planned changes and free space on the local destination. The real pull returns after every selected path has completed.

Selection resolves on the runner. At a source root with a .git directory it defaults to tracked working-tree files, as with push; --directory selects directory contents and --git states the Git selection explicitly. The same sensitive-name exclusions, --exclude and --include patterns, and --allow-sensitive overrides apply on the way back. There is no --delete for pulls; nothing is removed from the destination.

Pulls use the same IDs, records and status as pushes: --id names one pull, and repeating it returns or waits for the recorded outcome without copying again. The default --durability reconstructible completes without syncing each local file; --durability durable syncs and audits the result. See Input durability below.

How transfer moves bytes

Connection. Transfers use the peer connection between your client and the runner — the same authenticated endpoint and identity as execution and messaging, over Portal's QUIC transport. No SSH session is opened for file movement. Payload streams run on their own protocol connections through that endpoint, so a large transfer does not block job status, output or cancellation calls.

Delta. Pushes and pulls are delta transfers. When the destination already holds an older copy of a large file, the receiving side splits that copy into content-defined chunks and sends their identifiers; the sending side chunks the current file the same way and transmits only the chunks that changed. The reassembled file is checked against its whole-file digest before the transfer reports completion. Chunk sizes scale with file size, from a few KiB up to 96 KiB for multi-gigabyte files. SQLite databases are matched block by block at their declared page size, which resends less than content-defined chunks when edits stay in place.

When the whole file is sent instead. Files under 1 MiB are sent whole — the extra negotiation would cost more than it saves — as are files with no older copy at the destination. On a fast link the receiver also skips the delta for large files: chunking costs about 1 GiB/s per core, so once the transfer's own observed rate exceeds roughly half of that, resending the file is faster. The receiver caches chunk indexes of files it delivers, so a later edit of the same file skips the re-chunking pass. Unchanged files are recognised from filesystem identity and timestamps and contribute zero file payload. The first push sends the tree's payload in full; deltas apply from the second transfer onward. The Recover and measure counters below report which of these paths each transfer took.

Recover and measure

Ax generates a transfer UUID automatically and prints it before starting. Supply --id to choose one, and keep it when retrying an uncertain transfer. Uppercase UUIDs, such as macOS uuidgen output, are accepted and printed in lowercase. run --workspace derives the transfer UUID from its request ID while preserving the original Exec request ID and request digest:

ax workspace status --node build TRANSFER_ID --format json
ax workspace push --node build --source . --to /home/builder/project --id TRANSFER_ID

An ID names one transfer request and source publication. Repeating it checks that the options match and returns or waits for the recorded outcome; it does not copy files again. A completed status is historical, so it does not establish that the destination still contains those files. Use a new ID to send changes.

If the broker or runner restarts during transfer, inspect the recorded status. A lost live source publication cannot be recaptured under the same ID. Reconcile any associated job before starting another transfer. For a run, keep the original request ID and arguments; an exact retry recovers its existing job.

Results report fetched_payload_bytes, delta_copied_bytes, delta_paths, delta_basis_bytes, delta_cached_bases, delta_skipped_fast_link, elapsed time and observed payload rate. Reused bytes come from files already present at the destination. An unchanged transfer can send zero file payload. These counters exclude QUIC overhead and shared protocol traffic. They are not wire-byte measurements. Interrupted accounting may be incomplete after a host crash.

Structured commands support --format json, csv and tsv. Transfer completion and job completion are separate results. A disconnected client does not cancel an accepted transfer or job.

Input durability

Transfers use --durability reconstructible by default. This completes delivery without syncing each destination file to disk or auditing the whole result as a snapshot. The serving side publishes the selected source files directly and retains no immutable recovery copy. Avoid editing them during a transfer: the source is your client for a push and the runner for a pull.

Use --durability durable to sync prepared data and audit the selected result before completion. Keep the same mode when retrying an ID. Neither mode makes the directory an atomic filesystem snapshot. After a power failure, inspect the transfer and job, then use a new transfer to establish current inputs. Recovery never automatically reruns a job.

The live transfer protocol requires matching current client and runner builds. Restart the local broker after upgrading: ax exec daemon restart.

Notes and limits

Workspace transfer supports runners installed under the SSH user. System installations with a separate payload account are not yet qualified.

After upgrading an older installation, policy writes may require a one-time migration. Upgrade every client and runner using that workspace policy, then run:

ax workspace migrate --node build

This checks each admitted workspace node's live layout support before moving the existing records. An unreachable or incompatible node blocks migration. New installations do not need this step.