How to Install Hugging Face Transformers with uv

The Transformers library from Hugging Face provides access to thousands of pretrained models for text, image, audio, and multimodal tasks. Installing it with uv is straightforward for CPU inference but requires PyTorch index configuration for GPU acceleration, the same CUDA routing pattern covered in How to Install PyTorch with uv.

This guide covers three install paths: CPU-only inference, GPU training and inference with CUDA, and quantized model loading with accelerate and bitsandbytes.

Install for CPU inference

For tasks that run on CPU (sentiment analysis, text generation with small models, embeddings), add Transformers to your project:

uv add transformers

This installs Transformers and its core dependencies (huggingface-hub, tokenizers, safetensors) but not PyTorch. Most inference and training features require a deep learning backend, so install PyTorch alongside it:

uv add "transformers[torch]"

The [torch] extra adds PyTorch and Accelerate; it does not select a CPU-only build. PyPI’s Linux wheels can install CUDA dependencies even when inference runs on the CPU. For a smaller CPU-only environment, use the explicit CPU index in How to Install PyTorch with uv. Transformers requires Python 3.10 or newer; check wheel availability for the chosen PyTorch release too.

Verify the install:

uv run python -c "from transformers import pipeline; print(pipeline('sentiment-analysis', model='distilbert/distilbert-base-uncased-finetuned-sst-2-english', device='cpu')('uv is fast'))"

uv run executes the command inside the project’s virtual environment, ensuring all dependencies are installed first. The first run downloads the public model. The output contains a label and confidence score; the exact score varies:

[{'label': 'POSITIVE', 'score': 0.9789}]

Install with GPU support

NVIDIA GPU acceleration requires a CUDA-enabled PyTorch build and a compatible host driver. Choose an explicit index for reproducible CUDA selection rather than relying on PyPI’s platform- and release-dependent default. The wheels supply CUDA runtime libraries; a system CUDA toolkit is only needed when compiling extensions that require it.

Configure CUDA in pyproject.toml

Add PyTorch as a direct dependency and route it to the CUDA index. tool.uv.sources applies to direct project dependencies; listing only transformers[torch] leaves its transitive PyTorch dependency on the default index. This example uses CUDA 12.8:

[project]
name = "my-ml-project"
version = "0.1.0"
requires-python = ">=3.10"
dependencies = [
    "transformers[torch]",
    "torch",
]

[[tool.uv.index]]
name = "pytorch-cu128"
url = "https://download.pytorch.org/whl/cu128"
explicit = true

[tool.uv.sources]
torch = [
  { index = "pytorch-cu128", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
]

Then lock and sync:

uv lock
uv sync

The platform markers restrict CUDA builds to Linux and Windows. macOS falls back to PyPI’s native build, which supports CPU execution and MPS on supported Apple hardware. These markers select by operating system, not by GPU presence: Linux machines without an NVIDIA GPU still receive the CUDA build. See How to Install PyTorch with uv for CPU and ROCm configurations.

Verify GPU access

uv run python -c "import torch; assert torch.cuda.is_available(), 'CUDA unavailable'; x = torch.ones(2, device='cuda'); assert (x + x).cpu().tolist() == [2.0, 2.0]; print(torch.cuda.get_device_name(0), torch.__version__, torch.version.cuda)"

The command must complete a CUDA operation and print the GPU name, PyTorch version, and CUDA runtime version. If the assertion fails, check GPU access and the NVIDIA driver with nvidia-smi. Its CUDA field describes the driver’s supported CUDA level, not an installed toolkit; it need not equal torch.version.cuda. Installing wheels in a GPU-less container build does not verify GPU access when that image runs.

Install GPU support without a project

For one-off experimentation without a pyproject.toml, use uv pip with --torch-backend:

uv venv --python 3.12 --managed-python
uv pip install "transformers[torch]" --torch-backend=auto

uv creates .venv and installs into it without requiring activation. --torch-backend=auto queries visible driver and accelerator information, then chooses a PyTorch index. With no detected accelerator it selects CPU wheels. For GPU-less image builds, choose the deployment backend explicitly, such as --torch-backend=cu128, then verify execution on the target GPU.

Install extras for quantization and distributed training

Quantization reduces model memory use. transformers[torch] already includes Accelerate; add bitsandbytes for its supported quantization backends:

uv add bitsandbytes

The command adds bitsandbytes to the project. The following example targets an NVIDIA GPU and downloads a public model on its first run:

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(load_in_4bit=True)

model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-0.5B-Instruct",
    quantization_config=quantization_config,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")

inputs = tokenizer("Explain virtual environments in one sentence.", return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

device_map="auto" dispatches the model for inference; it does not launch distributed training. See Set up a multi-GPU training environment with uv to verify an Accelerate environment across multiple GPUs.

Choose the right extras

Transformers ships several optional dependency groups that pull in libraries for specific use cases:

Extras can be combined: uv add "transformers[torch,vision]" installs the PyTorch and vision dependencies. Their native wheels must support the project’s Python version and platform.