How to Combine Coverage Across a CI Matrix

When a test suite runs across a Python-version matrix, each job measures coverage for only the lines that version executes. A line guarded by if sys.version_info >= (3, 13) runs on 3.13 but not on 3.10, so checking any single job reports it as uncovered. The accurate number is the union of every job. coverage.py merges per-job data into one report with no external service.

Prerequisites

• A project with pytest tests that runs across a version matrix in GitHub Actions (see Setting up GitHub Actions with uv)
• uv installed, with coverage as a dev dependency: uv add --dev coverage pytest

Configure coverage for combinable data files

In pyproject.toml, turn on parallel mode:

[tool.coverage.run]
source = ["src/mypackage"]
parallel = true
relative_files = true

parallel = true makes each run write a uniquely named fragment file (.coverage.<host>.<pid>.<random>) instead of overwriting a single .coverage. Without it, two jobs that both produced .coverage would clobber each other once their artifacts landed in the same directory.

relative_files = true records source paths relative to the project root. The job that combines the data resolves those paths against its own checkout, so the absolute runner path baked into each fragment never has to match.

Record coverage in each matrix job

Run the suite under coverage run instead of plain pytest, then upload the fragment file as an artifact named per version:

jobs:
  tests:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
    steps:
      - uses: actions/[email protected]
      - uses: astral-sh/[email protected]
      - run: uv run --python ${{ matrix.python-version }} coverage run -m pytest
      - name: Upload coverage data
        uses: actions/[email protected]
        with:
          name: coverage-data-${{ matrix.python-version }}
          path: .coverage.*
          include-hidden-files: true
          if-no-files-found: ignore

if-no-files-found: ignore keeps the step green for matrix entries that produce no data, so you can measure coverage on a subset of versions without failing the others.

Combine the data in a dependent job

Add a job that waits for the matrix, pulls every artifact into one directory, and merges the fragments:

  coverage:
    name: Combine coverage and enforce threshold
    runs-on: ubuntu-latest
    needs: tests
    if: always()
    steps:
      - uses: actions/[email protected]
      - uses: actions/[email protected]
        with:
          python-version: "3.14"
      - uses: astral-sh/[email protected]
      - name: Download coverage data
        uses: actions/[email protected]
        with:
          pattern: coverage-data-*
          merge-multiple: true
      - name: Combine and report
        run: |
          uvx coverage combine
          uvx coverage html --skip-covered --skip-empty
          uvx coverage report --format=markdown >> "$GITHUB_STEP_SUMMARY"
          uvx coverage report --fail-under=100
      - name: Upload HTML report on failure
        if: failure()
        uses: actions/[email protected]
        with:
          name: html-coverage-report
          path: htmlcov

needs: tests holds the job until the matrix finishes. if: always() runs it even when a test job fails, so a single broken version still produces a coverage report. merge-multiple: true flattens every coverage-data-* artifact into the working directory, where coverage combine reads all the .coverage.* fragments and writes one .coverage.

Report the total to the run summary

coverage report --format=markdown emits a Markdown table, and appending it to $GITHUB_STEP_SUMMARY renders it on the workflow run page:

$ coverage report --format=markdown
| Name                    |    Stmts |     Miss |   Cover |
|------------------------ | -------: | -------: | ------: |
| src/mypackage/core.py   |       45 |        0 |    100% |
| **TOTAL**               |   **45** |    **0** | **100%** |

Reviewers see the combined number on the run without downloading anything.

Enforce one threshold for the matrix

coverage report --fail-under=100 exits nonzero when the combined total drops below the target, which fails the job and blocks the merge. Lower the number to whatever you commit to holding:

coverage report --fail-under=90

Because the check runs after coverage combine, the threshold gates the union of every version at once. A line covered on any job in the matrix counts as covered. The coverage html --skip-covered --skip-empty step writes a browsable htmlcov/ report that uploads only when the threshold check fails, so a failed build hands you a clickable list of the uncovered lines.

Learn More

• How to measure code coverage with pytest-cov covers single-run, line-level reports for local development
• How to test against multiple Python versions using uv builds the matrix this workflow aggregates
• coverage.py: Combining data files documents coverage combine and parallel mode
• GitHub Actions: workflow commands documents $GITHUB_STEP_SUMMARY