How to enforce annotation coverage on a Python library with Pyrefly

Annotation coverage tracks what fraction of a library’s public API carries type annotations. Pyrefly ships a coverage subcommand that reports this percentage per symbol and module, then exits non-zero in CI when the number falls below a threshold.

This is distinct from type checking: pyrefly check verifies that annotations are consistent; pyrefly coverage measures whether annotations exist. Ship type annotations with a py.typed marker first, then use this guide to track completeness.

Prerequisites

• uv installed
• A library project with a src/ layout and a pyproject.toml

Add Pyrefly as a development dependencyA package needed during development (testing, linting, formatting) that is not shipped to users of your project. Installed with the --dev flag. :

uv add --dev pyrefly

Check current annotation coverage

pyrefly coverage report emits a JSON document with per-symbol and per-module counts. It always exits 0, making it safe for dashboards and log capture:

uv run pyrefly coverage report

The summary block gives the project-wide percentage:

{
  "summary": {
    "n_typable": 4,
    "n_typed": 2,
    "n_any": 0,
    "n_untyped": 2,
    "coverage": 50.0,
    "strict_coverage": 50.0
  }
}

n_typable counts function parameters, return types, and class attributes. Local variables are excluded. To extract a single metric for a CI log, pipe the output to jq:

uv run pyrefly coverage report | jq .summary.coverage

Find what’s unannotated

pyrefly coverage check prints unannotated symbols with their source locations:

$ uv run pyrefly coverage check
 WARN `mylib.farewell` is untyped [coverage-missing]
 --> src/mylib/__init__.py:6:1
  |
6 | / def farewell(name):
7 | |     return f"Goodbye, {name}!"
  | |______________________________-
  |
ERROR type coverage 50.00% (2 of 4 typable) is below the 100.00% threshold

The default threshold is 100%, so the command exits 1 whenever any annotatable symbol lacks an annotation. Each coverage-missing warning identifies the exact function or method to annotate next.

Focus on the public API

--public-only restricts coverage to symbols reachable from the library’s __all__ and re-export chains. Internal modules and private helpers don’t affect the score:

uv run pyrefly coverage check --public-only

For a library with private implementation details that users never import, --public-only gives the number that actually matters to consumers. Start with this flag rather than measuring the whole project.

Set a minimum threshold

--fail-under N exits non-zero when coverage falls below N percent and exits 0 when coverage meets or exceeds it:

$ uv run pyrefly coverage check --public-only --fail-under 100
 WARN `mylib.farewell` is untyped [coverage-missing]
 --> src/mylib/__init__.py:6:1
  |
6 | / def farewell(name):
7 | |     return f"Goodbye, {name}!"
  | |______________________________-
  |
ERROR type coverage 50.00% (2 of 4 typable) is below the 100.00% threshold

When coverage meets the threshold, the command shows a summary and exits 0:

$ uv run pyrefly coverage check --public-only --fail-under 50
 WARN `mylib.farewell` is untyped [coverage-missing]
 --> src/mylib/__init__.py:6:1
  |
6 | / def farewell(name):
7 | |     return f"Goodbye, {name}!"
  | |______________________________-
  |
 INFO type coverage 50.00% (2 of 4 typable)

Note that coverage-missing warnings still appear even when the threshold passes. The exit code governs CI; the warnings identify what to annotate next.

Add to CI

Add the check as a step in a GitHub Actions workflow:

- name: Check annotation coverage
  run: uv run pyrefly coverage check --public-only --fail-under 80

To set the initial threshold without failing on the first run, capture the current percentage with coverage report and set --fail-under to that value. Raise the threshold as annotations accumulate over time.

Use pyrefly infer to auto-generate annotations for the symbols the coverage-missing warnings surface.