# How to Install bitsandbytes


bitsandbytes provides 4-bit and 8-bit quantization for large language models, cutting GPU memory usage enough to run models that would otherwise not fit. Unlike PyTorch (see [Why Installing GPU Python Packages Is So Complicated](https://pydevtools.com/handbook/explanation/installing-cuda-python-packages.md)), bitsandbytes publishes platform-specific wheels to [PyPI](https://pydevtools.com/handbook/explanation/what-is-pypi.md) that bundle precompiled CUDA libraries for multiple toolkit versions. A plain `pip install` works on Linux, Windows, and macOS without extra index URLs.

## Requirements

- Python >= 3.10
- PyTorch >= 2.3 (see [How to Install PyTorch with uv](https://pydevtools.com/handbook/how-to/how-to-install-pytorch-with-uv.md))
- NVIDIA GPU with compute capability 6.0+ for GPU quantization (Pascal or newer)
- NVIDIA driver that supports CUDA 11.8 or later

bitsandbytes also supports CPU-only, AMD ROCm (preview), Intel XPU, and Apple Silicon. GPU quantization requires an NVIDIA GPU.

## Install with pip or uv

Install PyTorch first, then bitsandbytes:

```shell
uv pip install torch
uv pip install bitsandbytes
```
```shell
pip install torch
pip install bitsandbytes
```
The PyPI [wheel](https://pydevtools.com/handbook/reference/wheel.md) ships with precompiled binaries for CUDA 11.8 through 13.0. At runtime, bitsandbytes detects the CUDA version provided by the installed PyTorch and loads the matching binary. No `--index-url` flag or CUDA version suffix is needed.

> [!IMPORTANT]
> Install PyTorch before bitsandbytes. bitsandbytes uses PyTorch's CUDA runtime to detect the correct backend at import time. If PyTorch is missing or CPU-only, bitsandbytes falls back to CPU mode and quantization functions will raise errors.

## Add to a uv project

For a [uv](https://pydevtools.com/handbook/reference/uv.md) project, add both packages as dependencies:

```shell
uv add torch bitsandbytes
```

If the project needs a specific CUDA build of PyTorch, configure the PyTorch index in `pyproject.toml` as described in [How to Install PyTorch with uv](https://pydevtools.com/handbook/how-to/how-to-install-pytorch-with-uv.md). bitsandbytes itself resolves from PyPI with no extra configuration.

## Install with conda

bitsandbytes is available on [conda-forge](https://pydevtools.com/handbook/reference/conda-forge.md) with the same version as PyPI (0.49.2 as of May 2026). Conda resolves the CUDA dependency alongside PyTorch, which avoids the install-order requirement that pip and uv have.

```shell
pixi add bitsandbytes
```
```shell
conda install -c conda-forge bitsandbytes
```
See [uv vs pixi vs conda for scientific Python](https://pydevtools.com/handbook/explanation/uv-vs-pixi-vs-conda-for-scientific-python.md) for guidance on when conda-based tooling makes sense.

## Verify the installation

Run this check to confirm bitsandbytes loaded the CUDA backend:

```python
import bitsandbytes as bnb
import torch

# Should print the CUDA version bitsandbytes is using
print(bnb.cextension.BNB_BACKEND)

# Quick functional test: create a quantized linear layer
linear = bnb.nn.Linear8bitLt(256, 128, has_fp16_weights=False)
x = torch.randn(1, 256, dtype=torch.float16, device="cuda")
output = linear.to("cuda")(x)
print(f"Output shape: {output.shape}")
```

If the backend prints `CUDA`, the GPU path is active. If it prints `CPU`, PyTorch either lacks CUDA support or no GPU was detected.

## Troubleshooting

### `RuntimeError` when calling quantization functions

bitsandbytes imports without error even when the native CUDA library fails to load. The error surfaces later, when quantization code runs. Check that:

1. PyTorch was installed with CUDA support (`torch.cuda.is_available()` returns `True`)
2. The NVIDIA driver is recent enough for the installed CUDA toolkit

### Wrong CUDA version detected

bitsandbytes reads the CUDA version from PyTorch, not from the system `nvcc`. If `torch.version.cuda` reports a different version than expected, reinstall PyTorch with the correct CUDA backend.

Override the detected version by setting the `BNB_CUDA_VERSION` environment variable:

```shell
BNB_CUDA_VERSION=128 python -c "import bitsandbytes"
```

### `libcudart.so` not found

This occurs when the CUDA runtime shared library is missing from the environment. PyTorch wheels bundle their own copy of `libcudart`, so this usually means PyTorch was installed as CPU-only. Reinstall PyTorch with CUDA support.

## Related

Handbook articles:

- [Why Installing GPU Python Packages Is So Complicated](https://pydevtools.com/handbook/explanation/installing-cuda-python-packages.md)
- [How to Install PyTorch with uv](https://pydevtools.com/handbook/how-to/how-to-install-pytorch-with-uv.md)
- [uv vs pixi vs conda for Scientific Python](https://pydevtools.com/handbook/explanation/uv-vs-pixi-vs-conda-for-scientific-python.md)

External links:

- [bitsandbytes documentation](https://huggingface.co/docs/bitsandbytes/main)
- [bitsandbytes GitHub repository](https://github.com/bitsandbytes-foundation/bitsandbytes)