How to test a Python library against multiple type checkers

Python users don’t all run the same type checker. Some reach for mypy, others pyright or Pyrefly. Each sees a library’s public API through its own rules. What passes mypy can fail pyright; what works in Pyrefly can be rejected by mypy.

Running all five type checkers on source code is not the answer. That path leads to dozens of checker-specific # type: ignore comments scattered through internal files that users never see. Instead, focus on what users actually encounter: the public API.

The Pyrefly team’s recommendation for library maintainers: run one type checker on source code for internal consistency, and run multiple type checkers on the test suite. Tests call the library the way users do, so tests that type-check correctly under mypy, pyright, and Pyrefly confirm that users of all three can consume the API without error.

Structure the test suite

If the test suite already calls public functions and constructors, it already exercises the public API surface. To make the type assertions explicit, add assert_type calls. assert_type is available in typing from Python 3.11; for earlier versions, install typing_extensions:

from typing import assert_type  # or: from typing_extensions import assert_type
import mylib

result = mylib.process("hello")
assert_type(result, str)

obj = mylib.Widget(name="test")
assert_type(obj.width, int)

Each checker verifies that the inferred type matches the declared type. A mismatch is a type error.

Install nox

uv tool install nox

Write the noxfile

import nox

nox.options.default_venv_backend = "uv"


@nox.session
def typecheck_src(session: nox.Session) -> None:
    session.install(".", "pyrefly")
    session.run("pyrefly", "check", "src/")


@nox.session
@nox.parametrize("checker", ["mypy", "pyright", "pyrefly"])
def typecheck_api(session: nox.Session, checker: str) -> None:
    session.install(".", checker)
    if checker == "mypy":
        session.run("mypy", "tests/")
    elif checker == "pyright":
        session.run("pyright", "tests/")
    else:
        session.run("pyrefly", "check", "tests/")

typecheck_src runs Pyrefly on src/ once. Swap in the checker you use day-to-day if you prefer a different one.

typecheck_api is parametrized: nox creates three separate sessions, each with its own virtual environment and checker installed. Failures in these sessions reveal public API types that checker’s users would hit.

Run locally

nox                                        # run all sessions
nox -s typecheck_src                       # source check only
nox -s "typecheck_api(checker='mypy')"     # API check with mypy
nox -s "typecheck_api(checker='pyright')"  # API check with pyright
nox -s "typecheck_api(checker='pyrefly')"  # API check with Pyrefly

Add to GitHub Actions CI

name: Type Check

on: [push, pull_request]

jobs:
  typecheck:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v8.1.0
      - run: uv tool install nox
      - run: nox

All four sessions run on every push. The parametrized typecheck_api session runs three times, once per checker.

Fix checker-specific failures

Failures in typecheck_api mean one checker rejects a type annotation that the others accept. Two causes to check first:

• Ambiguous container types: x = [] followed by x.append(1): pyright infers list[Unknown], mypy infers list[int]. Add an explicit annotation: x: list[int] = [].
• Inferred None vs explicit Optional: checkers disagree on when a missing return implies -> None. Annotate return types explicitly.

When the only fix is a suppression comment, use the checker-specific format to keep the suppression targeted:

x = get_items()  # type: ignore[assignment]              # mypy / Zuban
x = get_items()  # pyright: ignore[reportAssignmentType] # pyright / Basedpyright
x = get_items()  # pyrefly: ignore                       # Pyrefly

Start with two checkers

Adding a third checker means another CI session to maintain. Start with mypy and pyright:

@nox.session
@nox.parametrize("checker", ["mypy", "pyright"])
def typecheck_api(session: nox.Session, checker: str) -> None:
    session.install(".", checker)
    if checker == "mypy":
        session.run("mypy", "tests/")
    else:
        session.run("pyright", "tests/")

Add Pyrefly when users ask for it or when it becomes standard in the library’s ecosystem.

Learn More

• How do Python type checkers compare? covers speed benchmarks, conformance scores, and recommendations
• How to gradually adopt type checking in an existing Python project
• nox reference
• mypy reference
• pyright reference
• Pyrefly reference
• Testing and Ensuring Type Annotation Quality in the Python typing documentation