GPU selection & VRAM

gpu-train requests hardware with a single string: gpus="MODEL:COUNT" (or "cpu"). This page is the reference for what you can put there, how VRAM filtering works, and how friendly names map onto each provider.

The gpus string#

gpus="A100:4"     # GPU model A100, 4 of them on one box
gpus="H100:8"     # 8x H100 on one box
gpus="RTX4090:1"  # a single RTX 4090
gpus="cpu"        # no GPU — local subprocess / CPU fallback

Format is MODEL:COUNT. Omit the count (gpus="A100") and it defaults to 1. MODEL is matched against the curated catalog first, then passed to the provider's own name mapping.

The curated catalog#

Rather than hit a slow, credential-gated marketplace query for a dropdown, gpu-train ships a hand-maintained list of training GPUs with typical VRAM and where to find them. Read it from the SDK or the HTTP API:

import gpu_train
 
gpu_train.list_gpus()             # the whole catalog (list[GpuModel])
gpu_train.list_gpus("vastai")     # narrowed to one provider
gpu_train.list_gpus("local")      # [] — local is CPU-only
gpu_train.vram_for("A100")        # -> 80
gpu_train.find("rtx4090")         # GpuModel(name="RTX4090", vram_gb=24, ...)

curl "http://127.0.0.1:8780/v1/gpus?provider=runpod"

Each entry is a GpuModel with name, label, vram_gb, providers, and notes. The dashboard's + New run dialog renders these as provider-aware preset cards (model + VRAM); selecting one fills the gpus field.

Multiple GPUs on one box#

The :COUNT suffix requests that many GPUs on a single instance. The runtime launches DDP across them automatically — torchrun --nproc_per_node=COUNT:

run(task={"entrypoint": "train.py"}, provider="runpod", gpus="A100:8")

This rents one 8-GPU box. Provisioning asks the provider for that GPU count (Vast.ai num_gpus, RunPod gpuCount); if no single box with that many cards is available under your price_cap, you get a clear ProvisionError.

Filtering by minimum VRAM#

Often you care about "any card with at least N GB", not a specific model. Set min_vram_gb on the run; it's a real provisioning constraint, not just a label:

run(
    task={"entrypoint": "train.py"},
    provider="vastai",
    gpus="A100:2",
    min_vram_gb=80,     # only offers with ≥80GB per GPU
)

On Vast.ai this is pushed into the marketplace search as a gpu_ram ≥ 80×1024 MiB filter, so you don't accidentally rent a 40GB variant. The field is part of the spec everywhere — SDK run(), POST /jobs, and the dashboard's launch dialog.

Other spawn settings#

Per-provider name mapping#

The same friendly name resolves to each provider's own identifier:

¹ RunPod has no dedicated alias for some cards yet, so the token is passed through as-is. If a card doesn't provision on RunPod, try the exact RunPod GPU name. Unknown names everywhere fall back to a sensible transform (e.g. Vast.ai turns underscores into spaces).

See also#

• End-to-end setup — the full run walkthrough.
• Providers — connecting each backend.
• Distributed training — DDP & multi-node.