Skip to content

Requirements Gathering

Every script in this course so far — including both full projects — was built alongside you, one working version at a time. This capstone works differently, on purpose: this chapter defines the problem completely, as a real specification, and the next chapter hands you the challenge to build it yourself, from scratch, applying everything this course has covered without a version-by-version walkthrough to lean on. This chapter is the spec. No code appears here — that’s deliberate.

The Problem

Log directories grow without bound if nothing manages them. Build a script that keeps a log directory under control by compressing old log files into a dated archive location, and permanently deleting archived files once they’re old enough that they’re no longer worth keeping at all.

Functional Requirements

  1. Given a log directory, find files older than a configurable archive age (based on modification time) that aren’t already compressed.
  2. Compress each qualifying file (gzip is the expected tool), preserving the original filename with .gz appended.
  3. Move each compressed file into an archive directory, organized by the date it was archived (for example, archive_root/2026-08-23/).
  4. Within the archive directory structure, find .gz files older than a configurable retention age and delete them permanently.
  5. Report a summary at the end: how many files were archived, how many were deleted, and whether any individual failures occurred along the way.

Constraints And Assumptions

  • This is a Bash script, consistent with the rest of this course — no assumption of POSIX sh portability.
  • gzip and find are assumed available.
  • The log directory may contain filenames with spaces or other characters requiring careful handling — nothing about this script may assume simple, space-free filenames.
  • The log directory may be empty, or contain zero files old enough to qualify — this must be handled as a normal, non-error outcome, not a failure.
  • This script is expected to run unattended, typically via a scheduler — it must behave safely with no human present to react to anything, and must not leave stray state behind if killed uncleanly.

Expected Behavior — Happy Path And Failure Paths

Happy path: the script runs, archives every eligible file, deletes every expired archive, logs a clear summary, and exits 0.

Failure paths that must be handled explicitly, each with its own behavior:

  • The log directory doesn’t exist — distinct exit code, clear log message, no action taken.
  • The archive root doesn’t exist or isn’t writable — distinct exit code.
  • Compression fails on a specific file — that failure is logged, but does not abort the run; every other eligible file still gets processed, and the script’s final exit status reflects that at least one failure occurred.
  • A second instance is already running — the new invocation logs this and exits cleanly, without disturbing the instance already in progress.
  • The log directory or archive root resolve to an empty string or / — the script refuses to proceed at all, immediately, before any operation is attempted.
  • The script is interrupted mid-run (a scheduler timeout, a manual kill, a system restart) — no file is left in an ambiguous, half-compressed or half-moved state; whatever cleanup is needed happens automatically.

CLI Interface 

Usage: log-archiver.sh [-l LOG_DIR] [-a ARCHIVE_ROOT] [-d ARCHIVE_DAYS] [-r RETENTION_DAYS] [-n] [-h] [-v]

  -l LOG_DIR         Directory containing logs to archive
  -a ARCHIVE_ROOT    Directory to store compressed archives
  -d ARCHIVE_DAYS    Age in days before a log is compressed and archived (default: 7)
  -r RETENTION_DAYS  Age in days before an archived file is permanently deleted (default: 90)
  -n, --dry-run      Show what would happen without making any changes
  -h, --help         Show this help message and exit
  -v, --version      Show version information and exit

Configuration

A sourced config file, following the same convention established in the backup project, should provide defaults for every one of the above values that isn’t hardcoded — with the config file’s own path itself overridable via an environment variable, falling back to a sensible system default. Loading order matters, and should follow the precedence rule established in that project’s final chapter: defaults, then config file, then command-line arguments — each layer able to override the one before it, with the command line always having the final say.

Safety Requirements

Every one of these is a hard requirement, not a suggestion — and each is grounded directly in a specific danger this course has already demonstrated concretely:

  1. Validate log_dir and archive_root — refuse to proceed if either is empty or /, exactly as the backup project’s path validation did, and for exactly the same reason: this script performs destructive operations, and an empty or root-valued path next to a destructive command is a genuine, well-established danger.
  2. Use a lockfile with kill -0 liveness checking to prevent overlapping runs — this script deletes files, making the consequences of a concurrency bug considerably higher-stakes than the duplicate-alert problem the health-check project demonstrated.
  3. Iterate files with find ... -print0 and while read -r -d '', never an unquoted glob or $(ls) — log filenames containing spaces must be handled correctly, and this is the fully robust pattern established for exactly that requirement.
  4. Never delete a file without first confirming it’s genuinely inside the expected archive directory structure and genuinely past the retention threshold — a deletion step is the single highest-stakes operation in this entire script, and deserves more scrutiny than any other single line in it.
  5. Use mktemp for any staging this script needs, paired with a trap ... EXIT for guaranteed cleanup regardless of how the script terminates.
  6. set -euo pipefail as a baseline, with deliberate, explicit error handling layered on top of it — including, specifically, avoiding the local var=$(cmd) masking mistake when checking any command substitution’s success.
  7. Log every action taken — or that would be taken, under --dry-run — with levels and timestamps, to stderr.
  8. Distinct, documented exit codes for each meaningfully different failure category.
  9. A single file’s failure must not abort the entire run — log it, continue processing everything else, and ensure the final exit status still reflects that a failure happened somewhere.

Implementation Plan

A suggested build order for the next chapter — each stage independently testable before moving to the next:

  1. Argument parsing, config loading, and path validation, with no real archiving logic yet — confirm the CLI and config precedence work correctly first.
  2. Logging functions.
  3. Locking.
  4. The “find eligible files to archive” step — build and test this with --dry-run only, before writing anything that actually touches a file.
  5. The real compress-and-move step, with explicit error handling per file.
  6. The “find and delete expired archives” step — again, test with --dry-run first, given the stakes involved.
  7. Summary reporting.
  8. trap-based cleanup, wired in throughout rather than bolted on at the end.

Best Practices

  • Write every failure path down in prose before writing any code. It’s far easier to notice a missing case while writing a sentence than while debugging a script that’s already misbehaving.
  • Decide exit codes and the CLI interface before implementation begins — retrofitting either onto a script that’s already written tends to produce an inconsistent, half-applied version of both.
  • State constraints and assumptions explicitly, even ones that feel obvious in the moment — “filenames may contain spaces” is exactly the kind of assumption that’s easy to silently forget three hours into writing the actual logic.

Shell-Safety Considerations

This capstone is the first script in this entire course that permanently deletes data as a normal part of its intended behavior — every previous project either only read data (the health check) or only ever added new data in a clearly-named, staged, reversible way (the backup script). That distinction raises the stakes on every safety pattern this course has covered: a word-splitting bug in a script that only copies files produces, at worst, an incomplete copy; the same category of bug in a script that deletes files can destroy data that existed nowhere else.

Every requirement in the Safety Requirements section above exists specifically because this script’s failure modes are more serious than either prior project’s — treat the retention-deletion step in particular as the single place in this entire course most deserving of the extra caution, and confirm, before writing that step, that you can articulate exactly why each safety requirement above applies to it.

Hands-On: Preparing Your Capstone Workspace

This chapter’s hands-on step doesn’t build anything yet — it sets up a realistic test environment you’ll use to validate your own implementation in the next chapter.

1. Create a test log directory with files of varying ages.

mkdir -p ~/shell-course/capstone/logs
cd ~/shell-course/capstone/logs

echo "recent log entry" > app.log
touch -d "2 days ago" app.log

echo "old log entry" > old-app.log
touch -d "10 days ago" old-app.log

echo "very old log entry" > ancient.log
touch -d "100 days ago" ancient.log

echo "a log with a space in its name" > "service report.log"
touch -d "15 days ago" "service report.log"

2. Create an archive root and confirm gzip is available.

mkdir -p ~/shell-course/capstone/archive
gzip --version | head -n 1

3. Confirm find’s modification-time filtering works as expected against your test files.

find ~/shell-course/capstone/logs -type f -mtime +7

This should list old-app.log, ancient.log, and service report.log — everything older than 7 days — while leaving out app.log, which is only 2 days old. If your results don’t match this, check your system’s date and find behavior before moving on to the next chapter.

4. Leave this workspace in place. You’ll build directly against it in the next chapter, rather than recreating it from scratch.

With the problem fully specified and a real test environment in place, the next chapter is the challenge itself: build this script, from nothing, applying everything this course has covered — without a version-by-version walkthrough guiding each step this time.

Last updated on
Building From Scratch