# How to enforce annotation coverage on a Python library with Pyrefly


Annotation coverage tracks what fraction of a library's public API carries type annotations. [Pyrefly](https://pydevtools.com/handbook/reference/pyrefly.md) ships a `coverage` subcommand that reports this percentage per symbol and module, then exits non-zero in CI when the number falls below a threshold.

This is distinct from type checking: `pyrefly check` verifies that annotations are *consistent*; `pyrefly coverage` measures whether annotations *exist*. Ship type annotations with a [py.typed marker](https://pydevtools.com/handbook/how-to/how-to-ship-type-annotations-with-a-python-library.md) first, then use this guide to track completeness.

## Prerequisites

- [uv](https://pydevtools.com/handbook/reference/uv.md) installed
- A library project with a `src/` layout and a `pyproject.toml`

Add [Pyrefly](https://pydevtools.com/handbook/reference/pyrefly.md) as a {{< term "dev-dependency" "development dependency" >}}:

```bash
uv add --dev pyrefly
```

## Check current annotation coverage

`pyrefly coverage report` emits a JSON document with per-symbol and per-module counts. It always exits 0, making it safe for dashboards and log capture:

```bash
uv run pyrefly coverage report
```

The `summary` block gives the project-wide percentage:

```json
{
  "summary": {
    "n_typable": 4,
    "n_typed": 2,
    "n_any": 0,
    "n_untyped": 2,
    "coverage": 50.0,
    "strict_coverage": 50.0
  }
}
```

`n_typable` counts function parameters, return types, and class attributes. Local variables are excluded. To extract a single metric for a CI log, pipe the output to `jq`:

```bash
uv run pyrefly coverage report | jq .summary.coverage
```

## Find what's unannotated

`pyrefly coverage check` prints unannotated symbols with their source locations:

```console
$ uv run pyrefly coverage check
 WARN `mylib.farewell` is untyped [coverage-missing]
 --> src/mylib/__init__.py:6:1
  |
6 | / def farewell(name):
7 | |     return f"Goodbye, {name}!"
  | |______________________________-
  |
ERROR type coverage 50.00% (2 of 4 typable) is below the 100.00% threshold
```

The default threshold is 100%, so the command exits 1 whenever any annotatable symbol lacks an annotation. Each `coverage-missing` warning identifies the exact function or method to annotate next.

## Focus on the public API

`--public-only` restricts coverage to symbols reachable from the library's `__all__` and re-export chains. Internal modules and private helpers don't affect the score:

```bash
uv run pyrefly coverage check --public-only
```

For a library with private implementation details that users never import, `--public-only` gives the number that actually matters to consumers. Start with this flag rather than measuring the whole project.

## Set a minimum threshold

`--fail-under N` exits non-zero when coverage falls below `N` percent and exits 0 when coverage meets or exceeds it:

```console
$ uv run pyrefly coverage check --public-only --fail-under 100
 WARN `mylib.farewell` is untyped [coverage-missing]
 --> src/mylib/__init__.py:6:1
  |
6 | / def farewell(name):
7 | |     return f"Goodbye, {name}!"
  | |______________________________-
  |
ERROR type coverage 50.00% (2 of 4 typable) is below the 100.00% threshold
```

When coverage meets the threshold, the command shows a summary and exits 0:

```console
$ uv run pyrefly coverage check --public-only --fail-under 50
 WARN `mylib.farewell` is untyped [coverage-missing]
 --> src/mylib/__init__.py:6:1
  |
6 | / def farewell(name):
7 | |     return f"Goodbye, {name}!"
  | |______________________________-
  |
 INFO type coverage 50.00% (2 of 4 typable)
```

Note that `coverage-missing` warnings still appear even when the threshold passes. The exit code governs CI; the warnings identify what to annotate next.

## Add to CI

Add the check as a step in a GitHub Actions workflow:

```yaml
- name: Check annotation coverage
  run: uv run pyrefly coverage check --public-only --fail-under 80
```

To set the initial threshold without failing on the first run, capture the current percentage with `coverage report` and set `--fail-under` to that value. Raise the threshold as annotations accumulate over time.

Use [pyrefly infer](https://pydevtools.com/handbook/how-to/how-to-add-type-annotations-with-pyrefly-infer.md) to auto-generate annotations for the symbols the `coverage-missing` warnings surface.