End-to-end setup

This is the full, start-to-finish guide to running a real training job on a rented GPU with gpu-train. It threads together every piece — install, credentials, choosing hardware (GPU model, VRAM, multi-GPU), launching three different ways, monitoring, checkpoints, and automatic teardown — into one path you can follow top to bottom.

If you just want the 60-second version, see the Quickstart.

The mental model#

Your local machine is the control plane. For each run it:

1. provisions a co-located GPU box at one provider,
2. rsyncs your project to it,
3. installs deps and launches the run (torchrun for DDP, or Ray),
4. streams logs + metrics back to your machine, and
5. terminates the box when the run ends (or fails, or you Ctrl-C).

Your laptop never sits in the training hot path — distributed training stays inside one provider over its fast interconnect.

1. Install#

gpu-train installs from the prebuilt wheel on GitHub Releases. The core install includes the Python API and the zero-cost local provider; cloud providers and the dashboard are opt-in extras.

# core + dashboard + RunPod + W&B — a common starter combo
pip install "gpu-train[server,runpod,wandb] @ https://github.com/Research-Commons/gpu-train/releases/download/v0.0.6/gpu_train-0.0.6-py3-none-any.whl"

Requirements: Python 3.10+, plus ssh and rsync on your PATH (used to drive remote boxes; preinstalled on macOS and most Linux). See Installation for every extra.

2. Connect a provider#

You can provide credentials two ways — pick whichever fits your workflow. Both store nothing sensitive in your code.

Option A — environment variables#

export RUNPOD_API_KEY="..."        # RunPod
export VAST_API_KEY="..."          # Vast.ai
export WANDB_API_KEY="..."         # optional: Weights & Biases

Then point a secret_ref at the variable in your registry (next step). Env vars always take precedence over anything stored in the dashboard.

Option B — the dashboard#

gpu-train serve        # opens http://127.0.0.1:8780

Go to Providers → Connect, paste your key, and it's saved locally to ~/.gpu-train/credentials.json (chmod 600). The API never returns keys in plaintext. See Credentials & secrets for the full matrix (including GCP service accounts and the Colab tunnel).

3. Pick the right hardware#

Specify hardware with the gpus="MODEL:COUNT" string. gpu-train ships a curated catalog of the GPUs that matter for training so you don't have to guess model names or VRAM.

The GPU catalog#

Query it from the SDK or the dashboard's launch dialog (which shows provider-aware preset cards with VRAM):

import gpu_train
 
gpu_train.list_gpus("vastai")     # GPUs Vast.ai commonly offers
gpu_train.vram_for("A100")        # -> 80  (typical per-GPU VRAM in GiB)

curl "http://127.0.0.1:8780/v1/gpus?provider=vastai"

The catalog is the human-facing menu; you can still type any token into the gpus field — unknown names are mapped to each provider's own identifiers (and pass through unchanged if there's no alias).

Multiple GPUs (single box)#

The :COUNT suffix asks for that many GPUs on one machine. torchrun then launches DDP with --nproc_per_node=COUNT automatically:

gpus="A10:1"     # one A10
gpus="A100:4"    # four A100s on one box, DDP across all four
gpus="H100:8"    # eight H100s on one box

Minimum VRAM, disk, and region#

For marketplace providers you often care about "any card with ≥ N GB", not a specific model. Use min_vram_gb — Vast.ai wires it straight into its offer search (gpu_ram filter):

job = gpu_train.run(
    task={"entrypoint": "train.py"},
    provider="vastai",
    gpus="A100:2",
    min_vram_gb=80,     # only A100/H100-class boxes with ≥80GB per GPU
    disk_gb=200,        # scratch space on the box
    region="us-east",   # provider-specific hint
    price_cap=3.00,
)

4. Launch#

There are three ways to launch the exact same job. All of them go through the same control-plane path.

a. Python SDK#

from gpu_train import configure, run, gpu
 
configure(
    registry={
        "credentials": [
            {"cred_id": "runpod-1", "provider": "runpod",
             "secret_ref": "env://RUNPOD_API_KEY"},
        ],
        "tracking": {"wandb": {"secret_ref": "env://WANDB_API_KEY", "project": "my-proj"}},
    },
    use=["runpod-1"],
)
 
job = run(
    task={"entrypoint": "train.py", "args": ["--epochs", "3"]},
    provider="runpod",
    gpus="A100:4",
    price_cap=2.50,
    project_dir=".",     # rsynced to the box
)
print(job.id, job.status)

run() is non-blocking by default — it returns the JobRecord immediately and drives the job on a background thread. Pass wait=True to block until the job reaches a terminal state.

b. Dashboard#

gpu-train serve

Click + New run, pick a provider, select a GPU preset card (model + VRAM), set the count, and optionally min-VRAM / disk / region. It submits to the control plane and opens the live run view. See Dashboard (UI).

c. HTTP API#

The dashboard talks to the same endpoint you can call yourself:

curl -X POST http://127.0.0.1:8780/jobs \
  -H 'Content-Type: application/json' \
  -d '{
    "label": "llama-sft",
    "spec": {
      "provider": "runpod",
      "gpus": "A100:4",
      "entrypoint": "train.py",
      "price_cap": 2.50,
      "min_vram_gb": 80,
      "disk_gb": 200
    }
  }'

5. Log metrics from your training script#

Call gpu_train.log() inside train.py to chart scalars on the local dashboard — no W&B account required, on every provider including local:

import gpu_train
 
for step in range(steps):
    loss = train_one_step()
    gpu_train.log({"loss": loss, "lr": lr}, step=step)

Values ride the log stream back, are stored locally, and render as native charts on the run page. In a DDP job only rank 0 emits by default (all_ranks=True to override). If W&B is configured, the same call also forwards to wandb.log(). See Metrics & monitoring.

6. Monitor & operate#

From the SDK:

gpu.wait(job.id)                    # block until terminal
for line in gpu.stream(job.id):     # tail logs until the job ends
    print(line)
gpu.job(job.id)                     # latest status / cost / exit code
gpu.jobs()                          # recent jobs

From the CLI (reads the same local registry — works standalone):

gpu-train jobs                      # list recent jobs
gpu-train logs -f <job-id>          # follow logs until the job ends
gpu-train kill <job-id>             # cancel one job + terminate its box
gpu-train kill --all                # the panic button

7. Capture checkpoints (and pull a base model)#

Write checkpoints to the directory gpu-train hands you; they're rsynced back to ~/.gpu-train/artifacts/<job>/checkpoints when the run ends and listed with one-click download on the run page:

import gpu_train
 
ckpt_dir = gpu_train.checkpoint_dir()           # GPU_TRAIN_CHECKPOINT_DIR, created for you
model.save_pretrained(ckpt_dir / f"step-{step}")

To start from a Hugging Face baseline (pip install "gpu-train[hf]"), pull it to the local cache; a configured token is injected into every job so gated models download on the box:

local = gpu_train.hf.pull("meta-llama/Llama-3.2-1B")

Code and data are versioned automatically too — see Checkpoints & versioning and Hugging Face.

8. Teardown is automatic#

You generally don't clean up by hand. gpu-train terminates boxes:

• on job completion and failure,
• when an idle-timeout watchdog fires,
• when the control plane exits or is Ctrl-C'd (terminate_on_exit, default on),
• and it reconciles orphans from a crashed control plane on next startup.

price_cap refuses to provision above your $/hr ceiling in the first place. Manual escape hatches: gpu.reconcile() or gpu-train kill --all. See Cost safety.

Full worked example#

A complete A10:4 run on Vast.ai with a VRAM floor, metrics, and a checkpoint — end to end:

from gpu_train import configure, run, gpu
import gpu_train
 
configure(
    registry={
        "credentials": [
            {"cred_id": "vast-1", "provider": "vastai",
             "secret_ref": "env://VAST_API_KEY",
             "ssh_key_ref": "~/.ssh/id_ed25519"},
        ],
        "tracking": {"wandb": {"secret_ref": "env://WANDB_API_KEY", "project": "demo"}},
    },
    use=["vast-1"],
)
 
job = run(
    task={"entrypoint": "train.py", "args": ["--epochs", "3"]},
    provider="vastai",
    gpus="A10:4",          # four A10s on one box (DDP)
    min_vram_gb=24,        # only boxes with ≥24GB per GPU
    disk_gb=100,
    price_cap=1.50,
    wait=True,             # block until done for this example
)
 
print(job.status, job.exit_code, f"${job.cost_usd:.2f}")

# train.py (excerpt)
import gpu_train
 
ckpt = gpu_train.checkpoint_dir()
for step in range(steps):
    loss = train_one_step()
    gpu_train.log({"loss": loss}, step=step)
model.save_pretrained(ckpt / "final")

When this finishes, the box is gone, the checkpoint is on your machine under ~/.gpu-train/artifacts/<job>/checkpoints, and the loss curve is on the run page.

Where to go next#

• Providers — connect RunPod, Vast.ai, GCP, Colab.
• Configuration — the registry, defaults, overrides.
• Distributed training — multi-GPU & multi-node.
• CLI and API reference.