Skip to content

How to Install Flash-Attention

by Tim Hopper · Markdown
uv Scientific Python verified with pip 26.2 verified with uv 0.12

PyPI serves only a source distribution for flash-attn, so pip install flash-attn has no wheel to fall back on and compiles CUDA kernels for hours (see Why Installing GPU Python Packages Is So Complicated).

Prebuilt wheels exist in two places. The Astral GPU indexes publish one wheel per CUDA and PyTorch pair for the resolver to match. GitHub Releases hosts wheels that setup.py downloads by guessing a filename.

Pick an install path

Check your platform and driver

  • Supported GPUs: NVIDIA Ampere (A100, A10, RTX 3090), Ada Lovelace (RTX 4090), Hopper (H100), or Blackwell (B200, RTX 5090). The kernels require compute capability 8.0 or higher.
  • Unsupported GPUs: Turing (T4, RTX 20-series) and older. The install succeeds and the first kernel call raises RuntimeError: FlashAttention only supports Ampere GPUs or newer. The separate flash-attention-turing project covers a subset of features on those cards.
  • Blackwell: use cu128 or newer. setup.py emits the sm_100 and sm_120 gencode flags only from CUDA 12.8, so cu126 wheels carry no Blackwell kernels.
  • Operating systems: Linux. Windows support is experimental, and macOS has none.
  • Driver: 525.60.13 or newer for any CUDA 12.x index, 580 or newer for a CUDA 13 index. Under CUDA minor version compatibility, a driver whose nvidia-smi header reports 12.4 still runs cu128 wheels, so a header number below the index does not rule it out.
  • Toolkit: needed only for the GitHub Releases and source build paths. The Astral index installs a prebuilt wheel and touches no compiler.

Install from the Astral GPU index

Each index covers one CUDA version, and each wheel targets one PyTorch version. Both appear in the version segment rather than in a platform tag, so flash-attn resolves like any other dependency:

flash_attn-2.8.3.post1+cu.12.8.torch.2.8-cp312-cp312-manylinux_2_24_x86_64.whl

Route torch and flash-attn to indexes on the same CUDA line. Pick the CUDA version your driver supports (cu126, cu128, cu130, and others listed at wheels.astral.sh), then add both:

uv add torch --index pytorch-cu128=https://download.pytorch.org/whl/cu128
uv add flash-attn --index astral-cu128=https://wheels.astral.sh/simple/cu128/
$ uv add flash-attn --index astral-cu128=https://wheels.astral.sh/simple/cu128/
Resolved 28 packages in 222ms
Downloading flash-attn (241.7MiB)
Installed 3 packages in 5ms
 + einops==0.8.2
 + flash-attn==2.8.3.post1+cu.12.8.torch.2.11
 ~ myproject==0.1.0 (from file:///home/you/myproject)

Warning

Routing torch is not optional. The Astral indexes serve no torch, so an unrouted torch comes from PyPI, whose CUDA build changes between releases: PyPI ships PyTorch 2.11 as a CUDA 13 build. Pairing it with a cu128 wheel installs cleanly and then fails at import flash_attn with ImportError: libcudart.so.12: cannot open shared object file.

Each wheel declares its PyTorch as a dependency (torch==2.11.*), so the resolver picks the build matching the project’s torch. Routing torch to a CUDA-specific index pins the other half of the pair.

Installing flash-attn as a dependency of something else (vLLM, transformers) skips this routing, because [tool.uv.sources] applies only to declared dependencies. Declare flash-attn directly too.

Configure the index in pyproject.toml

uv add --index writes both indexes and the source mappings:

[tool.uv.sources]
torch = { index = "pytorch-cu128" }
flash-attn = { index = "astral-cu128" }

[[tool.uv.index]]
name = "pytorch-cu128"
url = "https://download.pytorch.org/whl/cu128"
explicit = true   # add by hand; uv add omits it

[[tool.uv.index]]
name = "astral-cu128"
url = "https://wheels.astral.sh/simple/cu128/"
explicit = true

A named index outranks PyPI for every package it carries, so flash-attn comes from Astral. The cu128 index also carries vllm, deepspeed, mmcv, and 18 others; explicit = true keeps it from silently supplying those. uv’s PyTorch guide recommends the flag.

Important

explicit = true without a matching [tool.uv.sources] entry removes the index from the pool and resolves flash-attn from PyPI instead. uv lock reports no error, and uv sync then starts the source build this page exists to avoid. Add both or neither.

Commit pyproject.toml and uv.lock, and uv sync rebuilds the environment on another machine. The lock records the index URL and the CUDA-specific version, so it reproduces across machines sharing that CUDA generation, not across all machines.

Install outside a project

A bare package name resolves to the highest version on the index, which is the build for the newest PyTorch it carries. Name the full version to choose a different pairing:

uv pip install 'flash-attn==2.8.3.post1+cu.12.8.torch.2.8' \
  --index astral-cu128=https://wheels.astral.sh/simple/cu128/

The local segment is valid only with ==, so >=2.8.3.post1+cu.12.8.torch.2.8 is rejected. Install a matching torch first, since neither command routes it, and skip --no-build-isolation, because neither builds anything.

pip has no equivalent of explicit = true: --extra-index-url merges every index into one pool and takes the highest version, across all 22 packages the index carries.

Check the Astral index coverage

Coverage for flash-attn on the CUDA 12.8 index, across both the 2.8.3 and 2.8.3.post1 builds:

PyTorch Python Platforms
2.7 3.9-3.13 x86_64, aarch64
2.8 3.9-3.13 x86_64
2.9 3.10-3.14 x86_64, aarch64
2.10 3.10-3.14 x86_64, aarch64
2.11 3.10-3.14 x86_64, aarch64

Coverage is uneven. PyTorch 2.8 ships no aarch64 wheel on this index, and Python 3.14 appears only on the 2.8.3 build for PyTorch 2.9. On aarch64, take torch from the PyTorch CUDA index rather than PyPI, whose aarch64 torch wheels are CPU-only. Other indexes carry different ranges; wheels.astral.sh is the current manifest.

Install from GitHub Releases

flash-attn’s setup.py runs a CachedWheelsCommand that downloads a matching wheel from GitHub Releases before falling back to compilation. Both pip and uv need --no-build-isolation here, because setup.py imports torch and packaging at the top level.

This path needs the full CUDA toolkit even when it finds a wheel. setup.py constructs a torch CUDAExtension at module level before it looks for one, so without CUDA_HOME the install stops at OSError: CUDA_HOME environment variable is not set and never reaches the download.

uv pip install torch==2.7.1 packaging   # a PyTorch the release covers
uv pip install flash-attn --no-build-isolation

When a wheel matches, the install finishes in seconds; otherwise the command falls back to compiling without warning, so an install running past a minute means nothing matched.

Check the GitHub release assets

PyPI serves flash-attn 2.8.3.post1. Its release carries these wheels:

PyTorch CUDA Python Platform
2.4 12 3.9-3.12 linux_x86_64
2.5-2.8 12 3.9-3.13 linux_x86_64
2.9 13 3.12 linux_x86_64, linux_aarch64

The CUDA 12 wheels ship in both CXX11 ABI TRUE and FALSE variants; the CUDA 13 wheels are TRUE only. The v2.8.3 release covers more: CUDA 12 wheels for PyTorch 2.9 and CUDA 13 wheels for PyTorch 2.10, all cp312.

A listed asset can still fail to download. The PyTorch 2.9 assets in the v2.8.3.post1 release carry the version 2.8.3, while automatic download builds the filename from the installed 2.8.3.post1 and hits a 404. Use Pin a wheel by URL there.

Coverage changes with every release; check the flash-attention releases page before picking a wheel.

Pin a wheel by URL

For a requirements file that needs one fixed artifact, install the wheel URL directly. The URL pattern is:

https://github.com/Dao-AILab/flash-attention/releases/download/v{release}/flash_attn-{version}+cu{cuda}torch{torch}cxx11abi{abi}-cp{py}-cp{py}-linux_x86_64.whl

To fill in the blanks, check your environment:

python -c "import torch; print('torch:', '.'.join(torch.__version__.split('.')[:2]))"
python -c "import torch; print('cxx11abi:', torch._C._GLIBCXX_USE_CXX11_ABI)"
python -c "import sys; print('python: cp' + ''.join(map(str, sys.version_info[:2])))"

Install the matching PyTorch before the wheel. These wheels declare an unpinned torch dependency, so installing one into an empty environment pulls whatever PyTorch is current, and import flash_attn then fails with the undefined symbol error from Verify the install. For Python 3.12, PyTorch 2.7, and CXX11 ABI True:

uv pip install torch==2.7.1
uv pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.3.post1/flash_attn-2.8.3.post1+cu12torch2.7cxx11abiTRUE-cp312-cp312-linux_x86_64.whl

Filenames do not always carry the release tag. The torch 2.9 wheels inside the v2.8.3.post1 release are named flash_attn-2.8.3+cu13torch2.9..., so copy the exact filename off the release page rather than assembling it from the pattern.

Compile from source

Source builds need the CUDA toolkit 12.0 or newer with nvcc on PATH, and PyTorch 2.2 or newer already installed. When no prebuilt wheel matches, install the build dependencies into the same environment as PyTorch and compile with a constrained job count:

uv pip install ninja packaging psutil
MAX_JOBS=4 uv pip install flash-attn --no-build-isolation

psutil matters only when MAX_JOBS is unset. setup.py imports it then to size the job count from free memory and cores, and aborts with ModuleNotFoundError: No module named 'psutil' if it is missing.

The toolkit’s CUDA major version must match PyTorch’s. Compare nvcc --version against python -c "import torch; print(torch.version.cuda)". A minor mismatch only warns; a major mismatch aborts the build with The detected CUDA version ... mismatches the version that was used to compile PyTorch.

The project’s README puts a compile without ninja at roughly two hours, against 3-5 minutes with ninja on a 64-core machine. It recommends capping MAX_JOBS on machines with less than 96GB of RAM, where ninja launches enough parallel jobs to exhaust memory.

In a uv project, route the build dependencies through extra-build-dependencies instead of a command-line flag:

[project]
dependencies = ["flash-attn", "torch==2.7.1"]

[tool.uv.extra-build-dependencies]
flash-attn = ["packaging", "psutil", { requirement = "torch", match-runtime = true }]

[tool.uv.extra-build-variables]
flash-attn = { MAX_JOBS = "4" }

match-runtime = true makes the build use the same torch version the project resolves at runtime.

Skip the wheels with conda-forge or pixi

conda and pixi skip wheel selection and compilation. The conda-forge build resolves the CUDA toolkit through the solver, so nothing here needs nvcc, ABI variants, or --no-build-isolation.

pixi add flash-attn

conda-forge builds linux-64 and linux-aarch64 packages. uv vs pixi vs conda for Scientific Python compares these tools for GPU workloads.

Verify the install

Run one attention call to confirm that the package imports and that its CUDA kernels load against your GPU. The first line reports torch’s CUDA build, which must sit on the same major version as the index:

check_flash_attn.py
import torch
from flash_attn import flash_attn_func

print(torch.__version__, torch.version.cuda, torch.cuda.is_available())
q = torch.randn(2, 256, 8, 64, dtype=torch.float16, device="cuda")
k, v = torch.randn_like(q), torch.randn_like(q)
out = flash_attn_func(q, k, v, causal=True)
print(tuple(out.shape), out.dtype)
$ uv run python check_flash_attn.py
2.11.0+cu128 12.8 True
(2, 256, 8, 64) torch.float16

A ModuleNotFoundError means the installation did not complete; check the install output for errors. An ImportError naming an undefined symbol in flash_attn_2_cuda means the wheel was built against a different PyTorch than the one installed:

ImportError: .../flash_attn_2_cuda.cpython-312-x86_64-linux-gnu.so: undefined symbol:
_ZN3c104cuda29c10_cuda_check_implementationEiPKcS2_ib

Reinstall the wheel matching both torch.__version__ and, on the GitHub Releases path, the CXX11 ABI variant. A CUDA error naming the device or its compute capability points at a GPU architecture mismatch.

Fix a failed install

  • ImportError: libcudart.so.12: cannot open shared object file. The installed torch is a CUDA 13 build and the wheel is a CUDA 12 build. Route torch to an index on the same CUDA line as the flash-attn index, as in Install from the Astral GPU index.

  • No solution found when resolving dependencies. No wheel covers your combination of platform, Python version, and pinned PyTorch. uv names the missing tag and the platforms that do have wheels, and the last three lines carry the answer:

    flash-attn==2.8.3+cu.12.8.torch.2.8 has no wheels with a matching platform tag
(e.g., `manylinux_2_28_aarch64`)
hint: Wheels are available for `flash-attn` (v2.8.3+cu.12.8.torch.2.8) on the
following platform: `manylinux_2_24_x86_64`

    Try another CUDA index, since torch coverage differs between them, or fall back to conda-forge when the project’s torch pin cannot move.

  • ModuleNotFoundError: No module named 'torch' while building. The installer is building the PyPI source distribution instead of downloading a wheel, and the isolated build has no torch. Point it at an index, or pass --no-build-isolation with torch already installed.

  • OSError: CUDA_HOME environment variable is not set. The GitHub Releases path needs the CUDA toolkit. Install it from NVIDIA’s CUDA toolkit archive, use a CUDA devel image such as nvidia/cuda:12.8.0-devel-ubuntu22.04, or switch to the Astral index, which needs no toolkit.

  • ModuleNotFoundError: No module named 'packaging'. The setup.py imports packaging before it does anything else, including downloading prebuilt wheels. Run uv pip install packaging first, then retry.

  • The build compiles instead of downloading a wheel. On the GitHub Releases path, setup.py prints Guessing wheel URL: before it tries, so compare that line against the release page. PyTorch 2.9 is the known case: the guessed filename carries a .post1 that the asset lacks.

  • The build is killed by the OOM killer. Set MAX_JOBS=2 or MAX_JOBS=1. Each compilation job can consume several gigabytes of memory.

  • The build runs for hours. Install ninja (uv pip install ninja) and retry. Without ninja, the CUDA extensions compile one file at a time.

Note

FlashAttention-3 is a Hopper-targeted beta, published on the Astral GPU indexes as flash-attn-3. FlashAttention-4 is a separate package (uv pip install --pre flash-attn-4) written in CuTeDSL for Hopper and Blackwell GPUs, shipping as a pure Python wheel that compiles kernels at runtime and needs no CUDA compiler.

Learn More

Handbook articles:

External resources:

Last updated on
How to Install DeepSpeedHow to install from a pylock.toml lockfile with pip