# How to Combine Coverage Across a CI Matrix


When a test suite runs across a Python-version matrix, each job measures coverage for only the lines that version executes. A line guarded by `if sys.version_info >= (3, 13)` runs on 3.13 but not on 3.10, so checking any single job reports it as uncovered. The accurate number is the union of every job. [coverage.py](https://coverage.readthedocs.io/) merges per-job data into one report with no external service.

## Prerequisites

- A project with [pytest](https://pydevtools.com/handbook/reference/pytest.md) tests that runs across a version matrix in GitHub Actions (see [Setting up GitHub Actions with uv](https://pydevtools.com/handbook/tutorial/setting-up-github-actions-with-uv.md))
- [uv](https://pydevtools.com/handbook/reference/uv.md) installed, with `coverage` as a dev dependency: `uv add --dev coverage pytest`

## Configure coverage for combinable data files

In [pyproject.toml](https://pydevtools.com/handbook/reference/pyproject.toml.md), turn on parallel mode:

```toml {filename="pyproject.toml"}
[tool.coverage.run]
source = ["src/mypackage"]
parallel = true
relative_files = true
```

`parallel = true` makes each run write a uniquely named fragment file (`.coverage.<host>.<pid>.<random>`) instead of overwriting a single `.coverage`. Without it, two jobs that both produced `.coverage` would clobber each other once their artifacts landed in the same directory.

`relative_files = true` records source paths relative to the project root. The job that combines the data resolves those paths against its own checkout, so the absolute runner path baked into each fragment never has to match.

## Record coverage in each matrix job

Run the suite under `coverage run` instead of plain `pytest`, then upload the fragment file as an artifact named per version:

```yaml {filename=".github/workflows/test.yml"}
jobs:
  tests:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
    steps:
      - uses: actions/checkout@v6.0.3
      - uses: astral-sh/setup-uv@v8.2.0
      - run: uv run --python ${{ matrix.python-version }} coverage run -m pytest
      - name: Upload coverage data
        uses: actions/upload-artifact@v7.0.1
        with:
          name: coverage-data-${{ matrix.python-version }}
          path: .coverage.*
          include-hidden-files: true
          if-no-files-found: ignore
```

> [!IMPORTANT]
> coverage.py data files start with a dot, and `actions/upload-artifact` skips hidden files by default. Set `include-hidden-files: true` or the artifact uploads empty and the combine step later reports `No data to combine`.

`if-no-files-found: ignore` keeps the step green for matrix entries that produce no data, so you can measure coverage on a subset of versions without failing the others.

## Combine the data in a dependent job

Add a job that waits for the matrix, pulls every artifact into one directory, and merges the fragments:

```yaml {filename=".github/workflows/test.yml"}
  coverage:
    name: Combine coverage and enforce threshold
    runs-on: ubuntu-latest
    needs: tests
    if: always()
    steps:
      - uses: actions/checkout@v6.0.3
      - uses: actions/setup-python@v6.2.0
        with:
          python-version: "3.14"
      - uses: astral-sh/setup-uv@v8.2.0
      - name: Download coverage data
        uses: actions/download-artifact@v8.0.1
        with:
          pattern: coverage-data-*
          merge-multiple: true
      - name: Combine and report
        run: |
          uvx coverage combine
          uvx coverage html --skip-covered --skip-empty
          uvx coverage report --format=markdown >> "$GITHUB_STEP_SUMMARY"
          uvx coverage report --fail-under=100
      - name: Upload HTML report on failure
        if: failure()
        uses: actions/upload-artifact@v7.0.1
        with:
          name: html-coverage-report
          path: htmlcov
```

`needs: tests` holds the job until the matrix finishes. `if: always()` runs it even when a test job fails, so a single broken version still produces a coverage report. `merge-multiple: true` flattens every `coverage-data-*` artifact into the working directory, where `coverage combine` reads all the `.coverage.*` fragments and writes one `.coverage`.

## Report the total to the run summary

`coverage report --format=markdown` emits a Markdown table, and appending it to `$GITHUB_STEP_SUMMARY` renders it on the workflow run page:

```console
$ coverage report --format=markdown
| Name                    |    Stmts |     Miss |   Cover |
|------------------------ | -------: | -------: | ------: |
| src/mypackage/core.py   |       45 |        0 |    100% |
| **TOTAL**               |   **45** |    **0** | **100%** |
```

Reviewers see the combined number on the run without downloading anything.

## Enforce one threshold for the matrix

`coverage report --fail-under=100` exits nonzero when the combined total drops below the target, which fails the job and blocks the merge. Lower the number to whatever you commit to holding:

```bash
coverage report --fail-under=90
```

Because the check runs after `coverage combine`, the threshold gates the union of every version at once. A line covered on any job in the matrix counts as covered. The `coverage html --skip-covered --skip-empty` step writes a browsable `htmlcov/` report that uploads only when the threshold check fails, so a failed build hands you a clickable list of the uncovered lines.

## Learn More

- [How to measure code coverage with pytest-cov](https://pydevtools.com/handbook/how-to/how-to-measure-code-coverage-with-pytest-cov.md) covers single-run, line-level reports for local development
- [How to test against multiple Python versions using uv](https://pydevtools.com/handbook/how-to/how-to-test-against-multiple-python-versions-using-uv.md) builds the matrix this workflow aggregates
- [coverage.py: Combining data files](https://coverage.readthedocs.io/en/latest/cmd.html#combining-data-files) documents `coverage combine` and parallel mode
- [GitHub Actions: workflow commands](https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#adding-a-job-summary) documents `$GITHUB_STEP_SUMMARY`