Downloading and Pulishing Test Artifacts as Summary
An artifact sitting in the run summary, waiting to be downloaded by hand, proves the mechanism works — but it’s not that different from just reading the raw pytest output in the log. The real value of artifacts shows up when a second job, running on its own separate VM, pulls that file back down and does something with it. That’s what turns “a save button” into a genuine way to move data between two isolated machines.
Retention, Revisited
You already know the basics from the last chapter: an artifact outlives the job that created it, and GitHub holds onto it for 90 days by default. Worth being precise about what “the job that created it” means here — artifacts in this newer generation of the upload/download actions are scoped to the specific job that uploaded them, and become available for download (via the UI or the API) the moment the upload step finishes, not after the whole workflow run completes. That’s what makes the pattern in this chapter possible at all: a downstream job doesn’t have to wait for the entire workflow to wrap up before it can grab what an earlier job produced.
Downloading An Artifact In A Different Job
actions/download-artifact is the counterpart to the upload action from the last chapter. Point it at the artifact’s name, and it pulls the file (or files) back down onto whatever runner the current step is executing on:
- name: Download Test Report
uses: actions/download-artifact@v8
with:
name: test-reportWith no path given, it extracts directly into the job’s working directory — so test-report.xml lands right where a script in that job can read it, exactly as if that job had generated the file itself. The fact that it was actually produced on a completely different VM, possibly already destroyed by the time this download happens, is invisible to anything downstream.
The Payoff: A Second Job That Actually Uses The Report
Here’s the proof that this is real data crossing a real boundary between two independent machines, not just a convenience wrapper: a new job, summarize, that depends on build finishing, downloads the report build produced, and turns it into a readable table — visible right on the workflow run page, with no one needing to download anything by hand.
GitHub Actions gives every job a special file path in the GITHUB_STEP_SUMMARY environment variable. Anything written to that file (in Markdown) renders directly on the run’s summary page. That’s the target for this job’s output.
The Parsing Script
Using only Python’s standard library — no new dependency to install — here’s a script that reads the JUnit XML report and writes a Markdown summary:
import sys
import xml.etree.ElementTree as ET
from pathlib import Path
def main() -> None:
xml_file = Path(sys.argv[1]) if len(sys.argv) > 1 else Path("test-report.xml")
root = ET.parse(xml_file).getroot()
# pytest generates:
#
# <testsuites>
# <testsuite ...>
# <testcase ... />
# </testsuite>
# </testsuites>
#
# Aggregate all testsuites rather than reading attributes from
# the <testsuites> wrapper.
suites = list(root.iter("testsuite"))
tests = sum(int(suite.attrib.get("tests", 0)) for suite in suites)
failures = sum(int(suite.attrib.get("failures", 0)) for suite in suites)
errors = sum(int(suite.attrib.get("errors", 0)) for suite in suites)
skipped = sum(int(suite.attrib.get("skipped", 0)) for suite in suites)
duration = sum(float(suite.attrib.get("time", 0)) for suite in suites)
passed = tests - failures - errors - skipped
if failures == 0 and errors == 0:
status = "✅ Passed"
else:
status = "❌ Failed"
# Overall Results (table)
print(f"# Pytest Results {status}")
print()
print("| Metric | Count |")
print("|---|---:|")
print(f"| Total | {tests} |")
print(f"| ✅ Passed | {passed} |")
print(f"| ❌ Failed | {failures} |")
print(f"| 💥 Errors | {errors} |")
print(f"| ⏭️ Skipped | {skipped} |")
print(f"| ⏱️ Duration | {duration:.3f}s |")
print()
# Test Details (Table)
print("## Test Details")
print()
print("| Test | Result | Duration |")
print("|---|---|---:|")
for suite in suites:
for case in suite.findall("testcase"):
name = case.attrib.get("name", "")
classname = case.attrib.get("classname", "")
case_time = float(case.attrib.get("time", 0))
failure = case.find("failure")
error = case.find("error")
skipped_node = case.find("skipped")
if failure is not None:
result = "❌ Failed"
elif error is not None:
result = "💥 Error"
elif skipped_node is not None:
result = "⏭️ Skipped"
else:
result = "✅ Passed"
test_name = f"`{classname}.{name}`" if classname else f"`{name}`"
print(f"| {test_name} | {result} | {case_time:.3f}s |")
# Failure/error details
node = failure if failure is not None else error
if node is not None:
message = node.attrib.get("message", "Unknown error")
details = node.text or ""
print()
print("<details>")
print(f"<summary>❌ {name} failure details</summary>")
print()
print("```text")
print(message)
if details.strip():
print(details)
print("```")
print()
print("</details>")
print()
if __name__ == "__main__":
main()Walking through it:
sys reads the XML file path from the command line, while Path makes file handling cleaner. If no path is provided, the script uses tests-report.xml.
ElementTree parses the XML using Python’s built-in XML parser. Instead of assuming there is one <testsuite>, root.iter("testsuite") collects all suites and aggregates their tests, failures, errors, skipped, and time values.
Passed tests are calculated as tests - failures - errors - skipped. The overall status is ✅ Passed when there are no failures or errors; otherwise, it is ❌ Failed. And yes you can’t avoid emojis in these AI days.
The script then prints a summary table followed by every individual test with its result and duration. Each testcase is classified by checking for <failure>, <error>, or <skipped> elements.
For failed or errored tests, it also prints the error message and traceback inside an expandable <details> section, making the CI output easier to read.
Important
The script writes to stdout, you need to redirect it to $GITHUB_STEP_SUMMARY file to view it in Github Step Summary.
Wiring It Into The Workflow
Save the script to the repo, so it’s version-controlled rather than typed inline into the workflow:
mkdir -p scripts
cat > scripts/summarize_report.py << 'EOF'
import sys
import xml.etree.ElementTree as ET
from pathlib import Path
def main() -> None:
xml_file = Path(sys.argv[1]) if len(sys.argv) > 1 else Path("test-report.xml")
root = ET.parse(xml_file).getroot()
# pytest generates:
#
# <testsuites>
# <testsuite ...>
# <testcase ... />
# </testsuite>
# </testsuites>
#
# Aggregate all testsuites rather than reading attributes from
# the <testsuites> wrapper.
suites = list(root.iter("testsuite"))
tests = sum(int(suite.attrib.get("tests", 0)) for suite in suites)
failures = sum(int(suite.attrib.get("failures", 0)) for suite in suites)
errors = sum(int(suite.attrib.get("errors", 0)) for suite in suites)
skipped = sum(int(suite.attrib.get("skipped", 0)) for suite in suites)
duration = sum(float(suite.attrib.get("time", 0)) for suite in suites)
passed = tests - failures - errors - skipped
if failures == 0 and errors == 0:
status = "✅ Passed"
else:
status = "❌ Failed"
# Overall Results (table)
print(f"# Pytest Results {status}")
print()
print("| Metric | Count |")
print("|---|---:|")
print(f"| Total | {tests} |")
print(f"| ✅ Passed | {passed} |")
print(f"| ❌ Failed | {failures} |")
print(f"| 💥 Errors | {errors} |")
print(f"| ⏭️ Skipped | {skipped} |")
print(f"| ⏱️ Duration | {duration:.3f}s |")
print()
# Test Details (Table)
print("## Test Details")
print()
print("| Test | Result | Duration |")
print("|---|---|---:|")
for suite in suites:
for case in suite.findall("testcase"):
name = case.attrib.get("name", "")
classname = case.attrib.get("classname", "")
case_time = float(case.attrib.get("time", 0))
failure = case.find("failure")
error = case.find("error")
skipped_node = case.find("skipped")
if failure is not None:
result = "❌ Failed"
elif error is not None:
result = "💥 Error"
elif skipped_node is not None:
result = "⏭️ Skipped"
else:
result = "✅ Passed"
test_name = f"`{classname}.{name}`" if classname else f"`{name}`"
print(f"| {test_name} | {result} | {case_time:.3f}s |")
# Failure/error details
node = failure if failure is not None else error
if node is not None:
message = node.attrib.get("message", "Unknown error")
details = node.text or ""
print()
print("<details>")
print(f"<summary>❌ {name} failure details</summary>")
print()
print("```text")
print(message)
if details.strip():
print(details)
print("```")
print()
print("</details>")
print()
if __name__ == "__main__":
main()
EOFNow add the second job to .github/workflows/ci.yml:
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
matrix:
python-version: ["3.12", "3.13", "3.14"]
os: [windows-latest, ubuntu-latest, macos-latest]
steps:
- name: Checkout
uses: actions/checkout@v7
- name: Setup Python
uses: actions/setup-python@v7
with:
python-version: ${{ matrix.python-version }}
cache: "pip"
cache-dependency-path: requirements.txt
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Lint (ruff)
uses: astral-sh/[email protected]
- name: Run Tests
run: python -m pytest --junitxml=test-report.xml
- name: Upload Test Report
uses: actions/upload-artifact@v7
with:
name: test-report-${{ matrix.os }}-${{ matrix.python-version }}
path: test-report.xml
summarize:
needs: build
runs-on: ${{ matrix.os }}
strategy:
matrix:
python-version: ["3.12", "3.13", "3.14"]
os: [windows-latest, ubuntu-latest, macos-latest]
steps:
- name: Checkout
uses: actions/checkout@v7
- name: Setup Python
uses: actions/setup-python@v7
with:
python-version: ${{ matrix.python-version }}
- name: Download Test Report
uses: actions/download-artifact@v8
with:
name: test-report-${{ matrix.os }}-${{ matrix.python-version }}
- name: Parse Report And Write Job Summary
env:
PYTHONIOENCODING: utf-8
run: python scripts/summarize_report.py >> "$GITHUB_STEP_SUMMARY"
shell: bashsummarize needs its own Checkout step to get scripts/summarize_report.py onto its runner — remember, build and summarize are two completely separate VMs. Nothing build did to its filesystem carries over automatically; the artifact is the only thing that crosses that boundary, which is exactly the point being demonstrated here.
Note
We did not pass test-report.xml to scripts/summarize_report.py because it’s the default one the script searches for.
Commit and push:
git add scripts/summarize_report.py .github/workflows/ci.yml
git commit -m "Add summarize job: download report, parse it, write a job summary"
git pushOpen the Actions tab and watch both jobs run — summarize will wait for build to finish (that’s what needs: build enforces) before it starts. Once both complete, open the run and look at the summary page itself, above the list of jobs: you’ll see a rendered Markdown table with your pass/fail counts, generated entirely by a job that never ran your tests — it only ever saw the report a different VM handed it.
That’s the real proof this chapter set out to give you: an artifact isn’t just a save button for later. It’s a genuine way to move a real file between two isolated, independent machines that otherwise share nothing.
There’s a trap waiting in this exact setup, though — one that won’t show up until a test actually fails. The next chapter covers a different topic first (speeding up dependency installation with caching), and then chapter 37 comes back to fix it.