Uploading Test Reports and Coverages as Artifacts

Right now, when your workflow finishes, everything it produced disappears with it. The runner VM is destroyed the moment the job ends — logs stick around in the Actions tab, but any file your steps actually created is gone. That’s fine for a pass/fail checkmark. It’s not fine the day you need to actually look at what failed, or hand a test report to someone who wasn’t watching the run live. This chapter fixes that with artifacts — files a workflow can hand off before the VM disappears.

Why You’d Want Anything Off The Runner At All

Right now, python -m pytest either passes or fails, and that result shows up as a green or red step in the log. That’s enough to know whether something broke. It’s not enough to easily see what broke without scrolling through raw console output, and it’s definitely not enough to hand to a teammate, attach to a bug report, or compare against last week’s run.

An artifact solves this by letting a workflow save a file — from any step, on any runner — to a place that survives after the job (and the VM it ran on) is gone. Anyone with access to the repo can go to that specific workflow run and download it, long after the runner that produced it no longer exists.

Generating A JUnit XML Report

pytest can already produce a structured report of what happened, instead of just printing to the console. Pass --junitxml with a file path:

python -m pytest --junitxml=test-report.xml

JUnit XML is a long-standing, widely supported format for test results — every test case, its outcome, and (for failures) the error message, all in one machine-readable file. You’re not writing anything to produce this; pytest generates it for you as a side effect of the run you’re already doing.

Uploading It As An Artifact

actions/upload-artifact takes a file (or files) from the runner’s disk and stores them alongside the workflow run:

- name: Upload Test Report
  uses: actions/upload-artifact@v7
  with:
    name: test-report
    path: test-report.xml

name is what shows up in the run’s Artifacts section — think of it as the label on the download button. path is what gets packaged up; here it’s a single file, but it can be a directory or a glob pattern just as easily.

Before Hands-On

We’ve opt out ci.yml while learning Matrix and introduced matrix-tests.yml for the exact same purpose. From now on let’s do everything (linting, testing, etc.) on ci.yml and keep matrix-tests.yml for the sake of history that will trigger only manually on workflow_dispatch event as this final version:

name: Matrix Tests

on: [workflow_dispatch]

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        python-version: ["3.12", "3.13", "3.14"]
        os: [windows-latest, ubuntu-latest, macos-latest]

    steps:
      - name: Checkout repo
        uses: actions/checkout@v7

      - name: Setup Python
        uses: actions/setup-python@v7
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install pytest
        run: python -m pip install --upgrade pip pytest

      - name: Lint (ruff)
        uses: astral-sh/[email protected]

      - name: Run Tests
        run: python -m pytest

We could have removed this matrix-tests.yml but it’s good to keep history of what we have learnt. Even in production enviornment, when you are evolving, you often hit and trial for the best possible workflow.

Commit and push:

git add .github/workflows/matrix-tests.yml
git commit -m "Keep Matrix Tests for the sake of learning history"
git push

This won’t trigger matrix-tests.yml but still triggers ci.yml because the triggering event is push for CI and that’s fine.

Hands-On: Wire It In And Download It Manually

Update .github/workflows/ci.yml so the test step writes a report, and a new step uploads it:

name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  build:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        python-version: ["3.12", "3.13", "3.14"]
        os: [windows-latest, ubuntu-latest, macos-latest]
    steps:
      - name: Checkout
        uses: actions/checkout@v7

      - name: Setup Python
        uses: actions/setup-python@v7
        with:
          python-version: ${{ matrix.python-version }}
          cache: "pip"
          cache-dependency-path: requirements.txt

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

      - name: Lint (ruff)
        uses: astral-sh/[email protected]

      - name: Run Tests
        run: python -m pytest --junitxml=test-report.xml

      - name: Upload Test Report
        uses: actions/upload-artifact@v7
        with:
          name: test-report
          path: test-report.xml

Commit and push:

git add .github/workflows/ci.yml
git commit -m "Generate JUnit XML report and upload it as an artifact"
git push

Open the Actions tab and click into this run. Scroll to the bottom of the run summary page — you’ll see an “Artifacts” section listing test-report. Actually you will see nine different files but with the same name test-report that’s because we keep the same name for each Matrix run. We could have name differently like test-report-${{ matrix.os }}-${{ matrix.python-version }} to give unique name but that’s fine because we will be downloading this automatically in the next chapter.

For now, download one of them, unzip it, and open test-report.xml in a text editor. You’ll see the raw structure pytest produced: a <testsuite> element with counts for tests, failures, and errors, and one <testcase> per test, each carrying its own pass/fail status.

That file now exists independently of the runner that created it — you could close this laptop, come back next week, and it would still be sitting there waiting to be downloaded.

Right now, getting anything useful out of it means downloading it by hand and reading raw XML. The next chapter automates that: a second, independent job that pulls this same artifact down and turns it into something readable, right on the workflow run page.