Autonomous GPU training under a hard dollar ceiling
Symbioza

Training that stops at the ceiling you set.

Submit it over MCP with a dollar ceiling and close the laptop. Symbioza rents a machine on the GPU marketplaces, restarts the run on another box when one dies, confirms the dead one stopped billing, and hands back hashed artifacts with an itemized bill.

Built for fine-tuning shops and small applied-ML teams with runs to finish and no cluster to finish them on.

Send us your SSH public key. We return a host entry and get you running. One customer at a time, by invitation — no account needed to start.

Session exec_9f3ka1 · MCP Symbioza
$6.41 of $12.00 ceiling
1
you › train the baseline — 24 GB VRAM, hard ceiling $12
2
→ estimateExecution(spec)
3
vast · RTX 3090 · 24 GB · est $4.87 — free, nothing booked
4
→ submitJob(spec)
5
{ executionId: "exec_9f3ka1", status: "queued" }
6
── disconnected · symbioza owns the run ──
7
14 h later, a fresh session
8
→ getStatus("exec_9f3ka1")
9
running · spend $3.12 / $12.00 · heartbeat 41 s ago
10
09:14 host vanished — restarted on a new box · 27 checkpoints offsite
11
→ getArtifact("exec_9f3ka1")
12
exit 0 · 3/3 delivered, re-hashed · 3 download links (24 h)
Receipt · exec_9f3ka1
gpu · box 1 · RTX 3090 · 2 h 00 m · host vanished$1.95
gpu · box 2 · RTX 3090 · 3 h 40 m$2.95
disk · 120 GB$0.26
transfer · in 41 GB · out 2 GB$0.41
symbioza spread · 15%, published$0.84
charged · ceiling $12.00$6.41

Session and receipt above are illustrative: a made-up run shown the way a real one is recorded, priced at the production-cut spread. Pre-production the spread is 0%; nothing here is a current price.

What holds on every run

Hard ceiling

budgetUsd is enforced by two independent watchdogs, not a billing alert. The box comes down inside the ceiling. Teardown is confirmed, then an orphan sweep checks that nothing is still billing. If the run reaches the ceiling it stops there: the checkpoints already offsite are yours, and the ledger itemizes every line we are billed for.

The box is disposable

Hosts reboot, disks fill, instances vanish. Your checkpoints leave the machine every five minutes. If the host disappears, you lose GPU time, not the checkpoint: the run restarts on another machine under the same ceiling, and each machine's failure history feeds the next choice. Automatic resume is next.

Delivered and re-hashed

Your success check runs inside your own command, and its exit code is the verdict. Every artifact is re-hashed at its destination against the recorded sha256, then tiered to offsite object storage.

One ledger

Every provision, retry and transfer is a line in one per-job ledger. The final charge is itemized, so you can see what the retries cost and not only the total.

Fig. 01

Two runs, as the ledger recorded them.

Not a benchmark: two records from paid work for one research customer, itemized the way every job is. Ids withheld; the records are on file.

Run 1 audio model · 50k steps
record on file
wall time6 h 36 m
interruptions0
deliveredcomplete
final charge$2.07
Run 2 short job · 7 artifacts
record on file
wall time71.9 min
GPU utilization97%
delivered7 / 7, re-hashed
final charge · ceiling$0.19 · $5.00
Fig. 02

When the box dies.

Marketplace machines are cheap because they are individually unreliable. Your checkpoints leave the machine every five minutes; if the host disappears you lose GPU time, not the checkpoint. The run restarts on another box under the same ceiling. Automatic resume is next.

The run streams a checkpoint off the box every five minutes; at 09:14 the host vanishes; the run restarts on another box under the same ceiling and is delivered and re-hashed. SUBMIT · 00:0009:14 HOST VANISHEDRESTARTED ON ANOTHER BOX · SAME CEILINGDELIVERED · RE-HASHEDA CHECKPOINT LEAVES THE BOX EVERY 5 MIN Vertical version of the same recovery path: checkpoints every five minutes, the host vanishes, the run restarts under the same ceiling and is delivered. SUBMIT · A CHECKPOINT EVERY 5 MIN09:14 HOST VANISHEDRESTARTED · SAME CEILINGDELIVERED · RE-HASHED

Illustration of the recovery path, not a recorded incident: the injected host-loss test has not been run yet.

What leaves the box early

A closed checkpoint is pulled off the machine every five minutes, so your newest checkpoint is at most five minutes old when a host dies. You lose GPU time, not the checkpoint.

What decides the next move

Every exit is classified before anything spends again: out-of-memory re-sizes VRAM, a full disk re-sizes disk, a pre-empted or faulty host moves the run to another box, a network blip retries. Your own code failing stops the run; it is never auto-patched.

What you see

getStatus lists each incident with a timestamp. The ledger lists what the restart cost as its own lines. The number you set still bounds the whole job, restarts included. Automatic resume from the harvested checkpoints is next; today a restarted run begins again.

Fig. 03

Four calls from spec to artifacts.

You describe the run and the ceiling. After submitJob, nothing needs you at the keyboard.

01

Describe

One strict JSON object: image, command, a GPU floor such as minVramGb, budgetUsd, maxRuntimeSeconds, checkpoint policy, optional datasets with checksums. Unknown fields are rejected. The rented box fetches the data itself — your laptop is never on the data path.

02

Estimate

estimateExecution prices that spec against the live GPU market and books nothing. You see what the run should cost before anything spends. Raise the GPU floor or the ceiling, price it again, decide.

03

Submit and disconnect

submitJob returns a durable executionId in seconds. The session that submits does not stay connected — an always-on daemon owns the run. Close the laptop; a fresh session tomorrow calls getStatus for progress, spend, heartbeat and incidents.

04

Collect

getArtifact returns the exit code, the final itemized charge, and a sha256 manifest with a download link per file, valid for 24 hours. cancelJob stops the spend at any point.

Fig. 04

One strict spec. Five tools.

Your agent already knows how to use it: describe the job in your own words and let it make the calls.

estimateExecution
price a spec against the live marketfree
submitJob
enqueue the run · returns a durable executionIdspends
getStatus
progress, spend so far, heartbeat, incidentsfree
getArtifact
exit code, final charge, manifest with 24 h download linksfree
cancelJob
stop a queued or running jobstops spend
The spec — unknown fields are rejected
{
  "image": "ghcr.io/you/train:latest",
  "command": ["python", "train.py",
              "--epochs", "40"],
  "gpu": { "minVramGb": 24, "cuda": true },
  "budgetUsd": 12,
  "maxRuntimeSeconds": 43200,
  "checkpoint": { "enabled": true },
  "datasets": [{
    "source": "https://<signed-url>/train.tar",
    "sha256": "<sha256>", "sizeBytes": 41000000000,
    "destination": "/data/train.tar",
    "transferMode": "https"
  }]
}

A typo can never become a default. What you send is what runs; the rented box fetches the dataset itself, verified against the checksum you declared.

.mcp.json — the whole integration
{
  "mcpServers": {
    "symbioza": {
      "command": "ssh",
      "args": ["-o", "BatchMode=yes",
               "symbioza",
               "/srv/symbioza/bin/symbioza-mcp"]
    }
  }
}

The MCP server runs over ssh. Your enrolled key is the credential; over that connection the session is already authenticated and there is no second secret to paste. The getting-started guide walks through it.

Fig. 05

How the bill works.

Provider cost plus a published Symbioza spread, line by line, under the ceiling you set.

Provider cost, per line

Every ledger line is what the marketplace charged us for it — GPU hours, disk, transfer — plus Symbioza's published spread: 15% at the production cut, 0% while we are pre-production. No bundles, no rounding, no per-job minimum.

Retries by fault class

A retry caused by your code or by the provider is charged like any other line. A retry caused by us is not debited. The classification is the same one the recovery loop uses, and it is on the ledger.

The ceiling caps all attempts

budgetUsd bounds the whole job, restarts included. Today, pre-production, the one invited customer is settled by hand from the ledger. At the production cut you top up a prepaid balance by card or crypto, each ledger line is debited from it as it is consumed, and top-ups are not refunded.

Fig. 06

What happens to your data.

What is true today, and what is not yet promised.

Isolation

Your image runs as the rented instance itself, the way the marketplace runs any container. The watchdog we add inside it holds no credential that could touch anything else, and no control-plane secret ever reaches that machine. We have not tested this isolation to production grade and do not claim it.

In transit, at rest

The MCP session and any pushed dataset travel over ssh; fetched datasets and artifact uploads travel over https. Artifacts land in object storage that verifies our checksum on upload and encrypts at rest. Nothing is encrypted at rest on the rented box itself.

What we do not yet promise

That the operator of a marketplace machine cannot inspect their own hardware while your job runs. At teardown the rented instance is destroyed; we do not separately scrub the host's disk. Do not send data you could not stand to have on a stranger's rig.

Fig. 07

What you own. What we own.

The boundary is the contract: you write what runs and how much it may cost; we decide where and how it runs.

You own
The image, the command, and what counts as success.
The success check whose exit code is the verdict.
The budget ceiling and the maximum runtime.
The ssh public key that is your only credential on the supported path.

Nothing runs that you did not write, and nothing spends past the number you set.

Symbioza owns
Finding and renting the machine, and paying the marketplace.
Keeping the run alive across reboots, evictions, dead hosts.
Checkpoints, retries, restarts on a new box, teardown inside the ceiling.
Hashing artifacts, moving them offsite, itemizing the bill.

If the machine dies at 3 am, that is our problem, not your morning. Best effort, one operator, no SLA yet.

Fig. 08

The honest print.

What is true today, what is not promised, and who should not sign up.

Live, by invitation

The control plane is live and has run paid work for one research customer, whose four-arm training bakeoff we delivered. One customer at a time, one operator, best effort: no uptime or response-time SLA yet.

ssh first

The supported path is ssh + stdio. You enroll a public key; a forced command runs the MCP server and nothing else. No second secret travels over that connection.

Not for you if you need
a run that resumes from its last checkpoint after a host dies — today it restarts on a new box, checkpoints kept; automatic resume is next
multi-node distributed training — jobs are single-node today
inference endpoints, notebooks, anything interactive
running on hardware you already own — supply comes through the marketplaces
artifacts in the hundreds of gigabytes — download links expire after 24 hours; keep deliverables to models, logs and metrics
a dashboard and a hosted API this quarter — the front door is MCP over ssh

Bring one run you’d rather not babysit.

Send the spec you already have. We price it against today’s market for free, tell you plainly if it is a bad fit, and if it is not, we run it under your ceiling.