Backup Script With Rsync
v4 works, but cp -r recopies every single file on every single run, whether or not it actually changed — fine for a small sample directory, genuinely wasteful for anything real. rsync is what actual backup tooling uses instead: it transfers only what’s changed, preserves permissions and timestamps correctly, and — with one additional flag — can turn each backup into a space-efficient incremental snapshot instead of a full copy every time. This chapter swaps rsync into the exact staging step v3 introduced, without touching anything else this project already got right.
Why rsync Instead Of cp
Three concrete advantages, all relevant to a real backup tool:
- Only transfers what’s changed.
cp -rreads and writes every file, every run, regardless of whether it’s identical to what’s already backed up.rsynccompares source and destination and skips anything unchanged — dramatically faster on repeat runs against a large, mostly-static source tree. - Preserves permissions, timestamps, and symlinks correctly, with a single flag —
cp -rdoesn’t guarantee this without carefully chosen extra options of its own.
Basic rsync Syntax
rsync -a "$source_dir" "$staging_dir"Preserving Permissions And Metadata With -a
-a (archive mode) is shorthand for a bundle of options: recursive copying, plus preservation of symlinks, permissions, timestamps, and ownership wherever possible. For backup purposes, this is essentially always what you want — a restored file should look exactly like the original, not like a fresh file created at restore time.
Swapping cp For rsync In The Staging Step
The change to the script itself is small — everything else from v3 and v4 (staging, atomic mv, locking, validation, logging, --dry-run) stays exactly as it was:
create_backup() {
...
# stage first
log_info "Staging before committing..."
staging_rysnc_output=$(rsync -a "$source_dir" "$staging_dir" 2>&1)
staging_rysnc_status=$?
# exit if staging failed
if [[ "$staging_rysnc_status" -ne 0 ]]; then
log_warn "$staging_rysnc_output"
log_fatal "Failed to sync files from $source to staging area, could be permission issues."
notify "FAILURE" "Sync failed for $source_dir"
rm -rf "$destination" # cleanup
exit "$EXIT_CP_FAILED"
fi
log_info "Synced files to staging area: $staging_dir"
...
}Everything else from v4 works with rsync exactly as it did with cp.
Incremental Backups With --link-dest
This is rsync’s standout feature for exactly this use case: --link-dest=PREVIOUS_BACKUP_DIR tells rsync to hardlink any file that hasn’t changed since the referenced previous backup, instead of copying it again. The result: each backup still appears as a complete, independently browsable directory, but unchanged files consume no additional disk space at all — only genuinely new or modified files use real storage.
Finding the most recent existing backup to link against, and building the rsync arguments conditionally with a command array:
create_backup() {
...
destination_dirname=$(dirname $destination)
previous_backup=""
# gt 1 because shouldn't count current timestamped backup dir ready to backup
if [[ "$(ls $destination_dirname | wc -l)" -gt 1 ]]; then
# normally tail -n 1 should get but we have fresh timestamped dir also, so tail -n 2 | head -n 1
previous_backup=$(find "$destination_dirname" -maxdepth 1 -type d 2>/dev/null | sort | tail -n 2 | head -n 1)
fi
rsync_args=(-a)
if [[ -n "$previous_backup" ]]; then
rsync_args+=(--link-dest="$previous_backup")
log_info "Using previous backup for incremental sync: $previous_backup"
fi
staging_rysnc_output=$(rsync ${rsync_args[@]} "$source_dir" "$staging_dir" 2>&1)
staging_rysnc_status=$?
# exit if staging failed
...Building rsync_args as an array, rather than trying to conditionally splice an optional flag into a plain string — the --link-dest=... argument is added as one clean, complete element only when there’s actually a previous backup to reference, with no risk of it being mangled by unquoted expansion later.
The Full v5 Script
Only the sync step changes from v4 — everything else (config loading, argument parsing, locking, validation, the atomic mv commit, notifications) is identical:
#!/usr/bin/env bash
set -euo pipefail
# ---------------------------
# Usage and Log Functions
# ---------------------------
usage() {
cat << 'EOF'
Usage: backup.sh -s SOURCE [-b BACKUP_ROOT] [-n] [-h] [-v]
-s SOURCE Source directory to back up
-b BACKUP_ROOT Root directory to store backups in (Default: $HOME/backup)
-n, --dry-run Show what would happen without making changes
-h, --help Show this help message and exit
-v, --version Show version information and exit
EOF
}
# reusable log function
log() {
local level="$1"
shift
local ts
ts=$(date '+%Y-%m-%d %H:%M:%S')
printf "[$ts] [$level] $*\n" >&2
}
# log function by type which uses log() under the hood, takes log message as argument
log_info() { log "INFO" "$@"; }
log_warn() { log "WARN" "$@"; }
log_error() { log "ERROR" "$@"; }
log_fatal() { log "FATAL" "$@"; }
get_value() {
if [[ $# -lt 2 ]]; then
echo "Option -s requires an argument" >&2
usage >&2
exit "$EXIT_MISSING_ARGS"
fi
# return value
echo "$2"
}
# ---------------------------
# Readonly vars
# ---------------------------
# define exit codes
readonly EXIT_BACKUP_FAILED=1
readonly EXIT_CP_FAILED=1 # treat as script/backup failure
readonly EXIT_SOURCE_MISSING=2
readonly EXIT_MKDIR_FAILED=3
readonly EXIT_ALREADY_RUNNING=4
readonly EXIT_INVALID_PATH=5
readonly EXIT_MISSING_ARGS=6
# ---------------------------
# Argument Parsing
# ---------------------------
dry_run=false
source_dir=""
backup_root=""
if [[ $# -lt 1 ]]; then
usage
exit 0
fi
while [[ $# -gt 0 ]]; do
case "$1" in
-s)
source_dir="$(get_value $@)"
shift 2
;;
-b)
backup_root="$(get_value $@)"
shift 2
;;
-n|--dry-run)
dry_run=true
shift
;;
-h|--help)
usage
exit 0
;;
-v|--version)
echo "backup.sh 4.0.0"
exit 0
;;
*)
echo "Invalid argument: $1"
usage
exit "$EXIT_MISSING_ARGS"
;;
esac
done
# ---------------------------
# Constants
# ---------------------------
SOURCE="$source_dir"
backup_root="${backup_root%/}" # remove trailing slash
BACKUP_ROOT="${backup_root:-$HOME/backup}"
# ---------------------------
# User Defined Functions
# ---------------------------
cleanup() {
set +u # staging_dir may not be avl if exit by trying ro run concurrently
rm -f "$lock_file"
rm -rf "$staging_dir"
}
trap cleanup EXIT
# notify
notify() {
local status="$1"
local message="$2"
log_info "NOTIFY [$status]: $message"
}
# check if dir exists, takes 1st arg as dir name
dir_exists() {
local dir_name="$1"
[[ -d "$dir_name" ]] && return 0
}
# create dir (including missing parent), 1st arg is dir name
make_dir() {
local dir_name="$1"
mkdir -p "$dir_name" || return 1
}
# print check msg for dir, 1st arg is dir name, 2nd arg is identifier of dir
print_dir_check_msg_for () {
local dir_name="$1"
local dir_identifier="$2"
log_info "Checking $dir_identifier directory: '$dir_name'"
}
# create dir only if it doesn't exists, takes 1st arg is dir name, 2nd arg is dir identifier to print
create_dir_if_not_exists() {
local dir_name="$1"
local dir_identifier="$2"
print_dir_check_msg_for "$dir_name" "$dir_identifier"
if ! dir_exists "$dir_name"; then
log_warn "Directory doesn't exists: '$dir_name', Creating..."
if ! make_dir "$dir_name"; then
log_error "Creating directory failed: '$dir_name'"
return 1
else
log_info "Directory created successfully: '$dir_name'"
fi
fi
}
# create backup, takes 1st arg as source dir, 2nd as staging_dir (temp). 3rd as destination to backup
create_backup() {
local source="$1"
local staging_dir="$2"
local destination="$3"
log_info "Backup process start: '$source' to '$destination'"
# stage first
log_info "Staging before committing..."
destination_dirname=$(dirname $destination)
previous_backup=""
# gt 1 because shouldn't count current timestamped backup dir ready to backup
if [[ "$(ls $destination_dirname | wc -l)" -gt 1 ]]; then
# normally tail -n 1 should get but we have fresh timestamped dir also, so tail -n 2 | head -n 1
previous_backup=$(find "$destination_dirname" -maxdepth 1 -type d 2>/dev/null | sort | tail -n 2 | head -n 1)
fi
rsync_args=(-a)
if [[ -n "$previous_backup" ]]; then
rsync_args+=(--link-dest="$previous_backup")
log_info "Using previous backup for incremental sync: $previous_backup"
fi
staging_rysnc_output=$(rsync ${rsync_args[@]} "$source_dir/" "$staging_dir/" 2>&1)
staging_rysnc_status=$?
# exit if staging failed
if [[ "$staging_rysnc_status" -ne 0 ]]; then
log_warn "$staging_rysnc_output"
log_fatal "Failed to sync files from $source to staging area, could be permission issues."
notify "FAILURE" "Sync failed for $source_dir"
rm -rf "$destination" # cleanup
exit "$EXIT_CP_FAILED"
fi
log_info "Synced files to staging area: $staging_dir"
log_info "Committing staged changes..."
if [[ "$dry_run" == true ]]; then
log_info "[DRY RUN] Would move $staging_dir to $destination"
rm -rf "$destination" # cleanup
mv_status=0 # delibarately set status true
mv_output=""
else
shopt -s dotglob
mv_output=$(mv $staging_dir/* "$destination" 2>&1)
mv_status=$?
shopt -u dotglob
fi
if [[ "$mv_status" -ne 0 ]]; then
log_warn "$mv_output"
log_error "Failed to move staged backup into place: $destination"
notify "FAILURE" "Move to destination failed: $destination"
return 1
fi
printf "Backup success: '$source' to '$destination'\n"
notify "SUCCESS" "Backup complete: $destination"
}
# just print section end, no args required
print_section_end() {
printf "\n"
}
# create lock file
set_lock () {
local lock_file="/tmp/backup-script.lock"
if [[ -f "$lock_file" ]]; then
existing_pid=$(cat "$lock_file")
if kill -0 "$existing_pid" 2>/dev/null; then
log_fatal "Backup already running as PID $existing_pid"
return 1
else
log_warn "Removing stale lock file: (PID $existing_pid no longer running)"
rm -f "$lock_file"
fi
fi
echo "$$" > "$lock_file"
log_info "Lock set: (PID $$)"
printf "$lock_file"
}
# validate dangerous paths, 1st arg path, 2nd arg path identifier
validate_path() {
local path="$1"
local path_identifier="$2"
if [[ -z "$path" || "$path" == "/" ]]; then
log_fatal "Refusing to operate on dangerous path for $path_identifier: '$path'"
return 1
fi
}
# ---------------------------
# Driver Code
# ---------------------------
# set lock first
lock_file=$(set_lock) || exit "$EXIT_ALREADY_RUNNING"
# validate before starting
validate_path "$SOURCE" "source dir" || exit "$EXIT_INVALID_PATH"
validate_path "$BACKUP_ROOT" "backup root" || exit "$EXIT_INVALID_PATH"
# exit if source doesn't exists
if ! dir_exists "$SOURCE"; then
log_fatal "Source directory doesn't exists: '$SOURCE', Exiting..."
exit "$EXIT_SOURCE_MISSING"
fi
# create if backup root does not exists, exit if creation failed
create_dir_if_not_exists "$BACKUP_ROOT" "backup root" || (log_fatal "Backup root creation failed, Exiting..." && exit "$EXIT_MKDIR_FAILED")
print_section_end
# create destination, exit if failed
timestamp=$(date +%Y%m%d_%H%M%S)
source_abs_path=$(realpath $SOURCE)
destination="$BACKUP_ROOT/$source_abs_path/$timestamp"
create_dir_if_not_exists "$destination" "destination" || (log_fatal "Creating backup destination failed, Exiting..." && exit "$EXIT_MKDIR_FAILED")
print_section_end
# create backup, exit if failed (script failure exit 1)
staging_dir=$(mktemp -d)
create_backup "$source_abs_path" "$staging_dir" "$destination" || (printf "Backup failed: partial or full, Exiting...\n" && exit "$EXIT_BACKUP_FAILED")What Changed From v4
rsync -areplacescp -r, fixing the hidden-file gap and cutting repeat-run time significantly on an unchanged or mostly-unchanged source.--link-destincremental backups, turning unchanged files into free hardlinks against the previous backup instead of full copies, without sacrificing each backup’s independence.- A command array (
rsync_args) for conditionally-included arguments. destinationis full absolute path (destination="$BACKUP_ROOT/$source_abs_path/$timestamp"), both source and destination is passed as absolute path increate_backup
Best Practices
- Always use
-afor backup-style copying withrsync— it’s the difference between a restorable, faithful copy and one that’s silently missing permissions or timestamps. - Double-check trailing slashes on
rsyncsource paths every time — it’s a one-character difference with a completely different result, and it’s worth verifying with a small test run before trusting it against real data. - Build conditional command-line arguments as arrays, never as a string with an optional piece spliced in..
Shell-Safety Considerations
Here’s the trailing-slash mistake from earlier in this chapter, demonstrated directly — a genuinely common, very well-known rsync gotcha that produces a working-looking result with completely the wrong structure.
Naive (broken) version — no trailing slash on the source path:
mkdir -p /tmp/shell-course-demo/source_dir
touch /tmp/shell-course-demo/source_dir/file1.txt
mkdir -p /tmp/shell-course-demo/dest_broken
rsync -a /tmp/shell-course-demo/source_dir /tmp/shell-course-demo/dest_broken
ls /tmp/shell-course-demo/dest_brokensource_dirfile1.txt isn’t directly inside dest_broken at all — rsync copied source_dir itself as a subdirectory, since there was no trailing slash telling it to copy that directory’s contents instead. Applied to this project’s actual script, this would mean $staging_dir ends up containing a nested source_dir/ folder instead of the backed-up files directly — and the subsequent mv "$staging_dir" "$destination" would produce a backup with a completely different internal structure than every previous version of this script has produced, breaking anything that expects to find files directly at the top level of a backup.
Corrected version — trailing slash on the source path:
mkdir -p /tmp/shell-course-demo/dest_fixed
rsync -a /tmp/shell-course-demo/source_dir/ /tmp/shell-course-demo/dest_fixed
ls /tmp/shell-course-demo/dest_fixedfile1.txtfile1.txt now lands directly inside dest_fixed, exactly matching the flat structure every prior version of this backup script has relied on. The rule worth memorizing, since it’s easy to forget under pressure: a trailing slash on rsync’s source means “copy what’s inside,” no trailing slash means “copy this directory itself.”
It Works
Invoke and see the output:
/backup.sh -s database_project/ -b /tmp/temp/[2026-08-31 09:25:53] [INFO] Lock set: (PID 45690)
[2026-08-31 09:25:53] [INFO] Checking backup root directory: '/tmp/temp'
[2026-08-31 09:25:53] [INFO] Checking destination directory: '/tmp/temp//home/alice/shell-projects/backup-keeping-script/database_project/20260831_092553'
[2026-08-31 09:25:53] [WARN] Directory doesn't exists: '/tmp/temp//home/alice/shell-projects/backup-keeping-script/database_project/20260831_092553', Creating...
[2026-08-31 09:25:53] [INFO] Directory created successfully: '/tmp/temp//home/alice/shell-projects/backup-keeping-script/database_project/20260831_092553'
[2026-08-31 09:25:53] [INFO] Backup process start: '/home/alice/shell-projects/backup-keeping-script/database_project' to '/tmp/temp//home/alice/shell-projects/backup-keeping-script/database_project/20260831_092553'
[2026-08-31 09:25:53] [INFO] Staging before committing...
[2026-08-31 09:25:53] [INFO] Using previous backup for incremental sync: /tmp/temp//home/alice/shell-projects/backup-keeping-script/database_project/20260831_090459
[2026-08-31 09:25:53] [INFO] Synced files to staging area: /tmp/tmp.vziFF8PnXO
[2026-08-31 09:25:53] [INFO] Committing staged changes...
Backup success: '/home/alice/shell-projects/backup-keeping-script/database_project' to '/tmp/temp//home/alice/shell-projects/backup-keeping-script/database_project/20260831_092553'
[2026-08-31 09:25:53] [INFO] NOTIFY [SUCCESS]: Backup complete: /tmp/temp//home/alice/shell-projects/backup-keeping-script/database_project/20260831_092553