Skip to content

Set up a CUDA Python project with uv

Keep PyTorch training and RAPIDS data preparation in one Linux x86-64 uv project, with CPU checks for CI. You will record all three dependency selections in one lockfileA file that records the exact version of every installed package, so everyone working on the project gets identical installs. and run them independently to avoid mixing incompatible CUDA packages.

You will verify actual tensor, cuDF, and cuML operations. Successful installation or import alone does not prove that a CUDA workload can use the GPU.

Prerequisites

  • uv installed
  • Python 3.12
  • A Linux x86-64 host for both the CPU and GPU checks
  • An NVIDIA driver compatible with CUDA 12.8 on the GPU host
  • For the RAPIDS checks, a supported NVIDIA GPU and system CUDA toolkit; consult the RAPIDS platform matrix before choosing a host

The CUDA and RAPIDS environments in this tutorial target Linux x86-64. Use WSL2 for supported Windows GPU hosts; native Windows and macOS cannot run this RAPIDS environment. Follow the PyTorch installation guide when a CPU-only project needs to support other platforms.

Check the GPU host

On the GPU host, inspect the driver and toolkit separately:

nvidia-smi --query-gpu=name,driver_version --format=csv,noheader
nvcc --version

Expect the first command to print the GPU name and driver version. The second prints the installed CUDA compiler version. The CUDA Version shown by a plain nvidia-smi command describes the driver’s supported level, not the installed toolkit.

Before continuing, compare the output with the CUDA 12 column for the RAPIDS release selected by the project in the RAPIDS platform matrix:

  • Check that the GPU model belongs to a supported architecture and meets its compute-capability requirement.
  • Check that the release value from nvcc --version falls within the supported toolkit range.
  • Check that driver_version meets the RAPIDS minimum and supports the PyTorch CUDA 12.8 runtime. NVIDIA’s CUDA toolkit and corresponding driver table lists a Linux driver target for CUDA 12.8.

For example, an NVIDIA T4 with the CUDA 12.8 toolkit and Linux driver 570.26 or newer satisfies these version checks for the RAPIDS requirements used here. Continue to the execution checks to verify that the Python packages can use the GPU.

If nvidia-smi cannot see a device, stop and fix the driver or container GPU access before creating the Python environment. If nvcc is missing or a compatibility check fails, follow the RAPIDS toolkit and NVRTC prerequisites before continuing.

Create the project

Create a flat project pinned to Python 3.12:

$ uv init --no-package --python 3.12 cuda-project
Initialized project `cuda-project` at `/path/to/cuda-project`
$ cd cuda-project

The absolute path in the initialization message depends on the working directory.

--no-package keeps the project layout flat (a main.py at the top level instead of a src/ package). Remove the generated file because the tutorial replaces it with a backend check:

rm main.py

Split the CPU and GPU dependencies

An extra is a named set of optional dependencies. This project defines cpu for CPU PyTorch, torch-cu128 for CUDA PyTorch, and rapids-cu12 for RAPIDS.

The commands install one extra at a time into the project’s .venv virtual environmentAn isolated folder where Python installs packages for one project, keeping them separate from other projects and your system Python. Learn more → . Switching extras with uv sync replaces packages in that same directory. Run the PyTorch and RAPIDS scripts independently; this setup does not support importing both libraries in one Python process.

Replace the pyproject.tomlThe standard configuration file for Python projects. Declares the project name, version, dependencies, build system, and tool settings in one place. Learn more → file with this configuration:

pyproject.toml
[project]
name = "cuda-project"
version = "0.1.0"
requires-python = ">=3.12,<3.13"
dependencies = []

[project.optional-dependencies]
cpu = [
  "torch>=2.11.0",
  "torchvision>=0.26.0",
]
torch-cu128 = [
  "torch>=2.11.0",
  "torchvision>=0.26.0",
]
rapids-cu12 = [
  "cudf-cu12>=26.10; sys_platform == 'linux' and platform_machine == 'x86_64'",
  "cuml-cu12>=26.10; sys_platform == 'linux' and platform_machine == 'x86_64'",
]

[tool.uv]
conflicts = [
  [
    { extra = "cpu" },
    { extra = "torch-cu128" },
  ],
  [
    { extra = "cpu" },
    { extra = "rapids-cu12" },
  ],
  [
    { extra = "torch-cu128" },
    { extra = "rapids-cu12" },
  ],
]

[tool.uv.sources]
torch = [
  { index = "pytorch-cpu", extra = "cpu" },
  { index = "pytorch-cu128", extra = "torch-cu128" },
]
torchvision = [
  { index = "pytorch-cpu", extra = "cpu" },
  { index = "pytorch-cu128", extra = "torch-cu128" },
]

[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true

[[tool.uv.index]]
name = "pytorch-cu128"
url = "https://download.pytorch.org/whl/cu128"
explicit = true

The cpu and torch-cu128 extras select different PyTorch indexes. The rapids-cu12 extra uses CUDA-suffixed package names and stays inside the Linux x86-64 branch.

The three conflict entries force uv to solve each environment separately. The selected PyTorch and RAPIDS requirements conflict on nvidia-nvjitlink-cu12, so uv cannot install both extras together. A shared CUDA major version does not guarantee compatible runtime package pins.

explicit = true prevents uv from searching the PyTorch indexes for unrelated dependencies. Both GPU extras target the CUDA 12 family, but each still needs its own host execution check.

Create the lockfile without installing any of the three dependency selections:

uv lock

Expect uv.lock to appear. Commit it with pyproject.toml; the lock records Python packages, not the host driver, toolkit, or GPU model.

Write the backend check

Create check_backend.py:

check_backend.py
import argparse


def check_torch(device: str) -> None:
    import torch

    if device == "cuda":
        assert torch.cuda.is_available(), "CUDA is not available to PyTorch"

    x = torch.ones((32, 32), device=device)
    y = x @ x
    assert y.device.type == device
    assert y.sum().item() == 32768
    print(f"{device} PyTorch matrix multiplication passed")


def check_rapids() -> None:
    import cudf
    import cupy as cp
    from cuml.decomposition import PCA

    assert cp.cuda.runtime.getDeviceCount() > 0
    print(cp.cuda.runtime.getDeviceProperties(0)["name"].decode())

    series = cudf.Series([1, 2, 3])
    assert int((series * series).sum()) == 14

    x = cp.asarray([[1.0, 2.0], [2.0, 4.0], [3.0, 6.0], [4.0, 8.0]])
    transformed = PCA(n_components=1, output_type="cupy").fit_transform(x)
    assert isinstance(transformed, cp.ndarray)
    assert transformed.device.id == 0
    assert transformed.shape == (4, 1)
    assert bool(cp.all(cp.isfinite(transformed)))
    print("cuDF and cuML GPU operations passed")


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("backend", choices=("cpu", "torch-cuda", "rapids"))
    args = parser.parse_args()

    if args.backend == "cpu":
        check_torch("cpu")
    elif args.backend == "torch-cuda":
        check_torch("cuda")
    else:
        check_rapids()

The CPU and PyTorch CUDA branches run the same tensor operation on different devices. The RAPIDS branch checks a cuDF computation and cuML output that remains in GPU memory.

Run the CPU environment

Install and run the CPU extra:

uv sync --locked --extra cpu
uv run --extra cpu python check_backend.py cpu

uv run executes the command inside the project’s virtual environment, ensuring all dependencies are installed first. Expect uv to report the synchronized CPU packages, followed by:

cpu PyTorch matrix multiplication passed

Inspect the selected build if the check fails:

uv run --extra cpu python -c "import torch; print(torch.__version__, torch.version.cuda)"

The version contains +cpu, and the CUDA value is None.

Run the PyTorch CUDA environment

On the GPU host, replace the CPU environment with the PyTorch CUDA extra:

uv sync --locked --extra torch-cu128
uv run --extra torch-cu128 python check_backend.py torch-cuda

uv removes packages that belong only to the CPU environment and installs the CUDA selection. Expect uv to report those changes, followed by:

cuda PyTorch matrix multiplication passed

Inspect the runtime and device when this fails:

uv run --extra torch-cu128 python -c "import torch; print(torch.__version__, torch.version.cuda, torch.cuda.get_device_name(0))"

Expect a version containing +cu128, CUDA 12.8, and the GPU name.

Run the RAPIDS environment

Replace the PyTorch environment with the RAPIDS extra:

uv sync --locked --extra rapids-cu12
uv run --extra rapids-cu12 python check_backend.py rapids

Expect uv to report the RAPIDS packages, followed by output in this shape:

NVIDIA T4
cuDF and cuML GPU operations passed

The GPU name varies by host. An assertion or CUDA loader error is a failed environment check, even if every package installed successfully. Keeping PyTorch and RAPIDS in separate environments avoids their runtime-package conflict; it does not change the host driver or toolkit requirements.

Optionally test the environments in GitHub Actions

The local checks complete the project setup. To automate them, use a GitHub repository containing pyproject.toml, .python-version, uv.lock, and check_backend.py.

The CPU job uses a GitHub-hosted runner. For GPU CI, first register your GPU host as a self-hosted runner, add the custom gpu label, and confirm that GitHub shows it as online and idle. The workflow matches all four labels: self-hosted, linux, x64, and gpu.

Run the local GPU checks under the runner’s account to confirm that it has device access and the required driver and toolkit. Without a registered GPU runner, omit the entire cuda job and keep the cpu job; leaving cuda enabled without a matching online runner leaves it queued.

Add .github/workflows/accelerators.yml, then commit and push it with the project files:

.github/workflows/accelerators.yml
name: Accelerator checks

on:
  push:
    branches: [main]
  pull_request:

permissions:
  contents: read

jobs:
  cpu:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false
      - uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3
        with:
          python-version: "3.12"
          enable-cache: true
      - run: uv sync --locked --extra cpu
      - run: uv run --extra cpu python check_backend.py cpu

  cuda:
    if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository
    runs-on: [self-hosted, linux, x64, gpu]
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false
      - uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3
        with:
          python-version: "3.12"
          enable-cache: true
      - run: nvidia-smi --query-gpu=name,driver_version --format=csv,noheader
      - run: uv sync --locked --extra torch-cu128
      - run: uv run --extra torch-cu128 python check_backend.py torch-cuda
      - run: uv sync --locked --extra rapids-cu12
      - run: uv run --extra rapids-cu12 python check_backend.py rapids

The job-level condition skips the persistent GPU runner for pull requests from forks. Do not expose a self-hosted runner to untrusted pull-request code; use an isolated, disposable GPU runner when outside contributors need GPU validation.

On a push to main or a pull request, expect the CPU log to contain cpu PyTorch matrix multiplication passed. With GPU CI configured, a push to main or a pull request from the same repository also runs the CUDA job; its log contains the PyTorch message, the GPU name, and cuDF and cuML GPU operations passed. For pull requests from forks, expect the CUDA job to be skipped.

Last updated on