Skip to content

Backup Script With Rsync


v4 works, but cp -r recopies every single file on every single run, whether or not it actually changed — fine for a small sample directory, genuinely wasteful for anything real. rsync is what actual backup tooling uses instead: it transfers only what’s changed, preserves permissions and timestamps correctly, and — with one additional flag — can turn each backup into a space-efficient incremental snapshot instead of a full copy every time. This chapter swaps rsync into the exact staging step v3 introduced, without touching anything else this project already got right.

Why rsync Instead Of cp

Three concrete advantages, all relevant to a real backup tool:

  • Only transfers what’s changed. cp -r reads and writes every file, every run, regardless of whether it’s identical to what’s already backed up. rsync compares source and destination and skips anything unchanged — dramatically faster on repeat runs against a large, mostly-static source tree.
  • Preserves permissions, timestamps, and symlinks correctly, with a single flag — cp -r doesn’t guarantee this without carefully chosen extra options of its own.

Basic rsync Syntax

rsync -a "$source_dir" "$staging_dir"

Preserving Permissions And Metadata With -a

-a (archive mode) is shorthand for a bundle of options: recursive copying, plus preservation of symlinks, permissions, timestamps, and ownership wherever possible. For backup purposes, this is essentially always what you want — a restored file should look exactly like the original, not like a fresh file created at restore time.

Swapping cp For rsync In The Staging Step

The change to the script itself is small — everything else from v3 and v4 (staging, atomic mv, locking, validation, logging, --dry-run) stays exactly as it was:

create_backup() {
    ...
    # stage first
    log_info "Staging before committing..."
    staging_rysnc_output=$(rsync -a "$source_dir" "$staging_dir" 2>&1)
    staging_rysnc_status=$?

    # exit if staging failed
    if [[ "$staging_rysnc_status" -ne 0 ]]; then
        log_warn "$staging_rysnc_output"
        log_fatal "Failed to sync files from $source to staging area, could be permission issues."
        notify "FAILURE" "Sync failed for $source_dir"
        rm -rf "$destination" # cleanup
        exit "$EXIT_CP_FAILED"
    fi 

    log_info "Synced files to staging area: $staging_dir"
    ...
}

Everything else from v4 works with rsync exactly as it did with cp.

Incremental Backups With --link-dest

This is rsync’s standout feature for exactly this use case: --link-dest=PREVIOUS_BACKUP_DIR tells rsync to hardlink any file that hasn’t changed since the referenced previous backup, instead of copying it again. The result: each backup still appears as a complete, independently browsable directory, but unchanged files consume no additional disk space at all — only genuinely new or modified files use real storage.

Finding the most recent existing backup to link against, and building the rsync arguments conditionally with a command array:

create_backup() {
    ...
    destination_dirname=$(dirname $destination)
    previous_backup=""
    # gt 1 because shouldn't count current timestamped backup dir ready to backup
    if [[ "$(ls $destination_dirname | wc -l)" -gt 1 ]]; then
        # normally tail -n 1 should get but we have fresh timestamped dir also, so tail -n 2 | head -n 1
        previous_backup=$(find "$destination_dirname" -maxdepth 1 -type d 2>/dev/null | sort | tail -n 2 | head -n 1)
    fi

    rsync_args=(-a)
    if [[ -n "$previous_backup" ]]; then
        rsync_args+=(--link-dest="$previous_backup")
        log_info "Using previous backup for incremental sync: $previous_backup"
    fi

    staging_rysnc_output=$(rsync ${rsync_args[@]} "$source_dir" "$staging_dir" 2>&1)
    staging_rysnc_status=$?

    # exit if staging failed
    ...

Building rsync_args as an array, rather than trying to conditionally splice an optional flag into a plain string — the --link-dest=... argument is added as one clean, complete element only when there’s actually a previous backup to reference, with no risk of it being mangled by unquoted expansion later.

The Full v5 Script

Only the sync step changes from v4 — everything else (config loading, argument parsing, locking, validation, the atomic mv commit, notifications) is identical:

#!/usr/bin/env bash

set -euo pipefail

# ---------------------------
# Usage and Log Functions
# ---------------------------
usage() {
    cat << 'EOF'
Usage: backup.sh -s SOURCE [-b BACKUP_ROOT] [-n] [-h] [-v]

  -s SOURCE         Source directory to back up
  -b BACKUP_ROOT    Root directory to store backups in (Default: $HOME/backup)
  -n, --dry-run     Show what would happen without making changes
  -h, --help        Show this help message and exit
  -v, --version     Show version information and exit
EOF
}

# reusable log function
log() {
    local level="$1"
    shift
    local ts
    ts=$(date '+%Y-%m-%d %H:%M:%S')
    printf "[$ts] [$level] $*\n" >&2
}

# log function by type which uses log() under the hood, takes log message as argument
log_info()  { log "INFO"  "$@"; }
log_warn()  { log "WARN"  "$@"; }
log_error() { log "ERROR" "$@"; }
log_fatal() { log "FATAL" "$@"; }

get_value() {
    if [[ $# -lt 2 ]]; then
        echo "Option -s requires an argument" >&2
        usage >&2
        exit "$EXIT_MISSING_ARGS"
    fi
    # return value 
    echo "$2"
}

# ---------------------------
# Readonly vars
# ---------------------------

# define exit codes 
readonly EXIT_BACKUP_FAILED=1
readonly EXIT_CP_FAILED=1 # treat as script/backup failure
readonly EXIT_SOURCE_MISSING=2
readonly EXIT_MKDIR_FAILED=3
readonly EXIT_ALREADY_RUNNING=4
readonly EXIT_INVALID_PATH=5
readonly EXIT_MISSING_ARGS=6

# ---------------------------
# Argument Parsing
# ---------------------------

dry_run=false
source_dir=""
backup_root=""

if [[ $# -lt 1 ]]; then
    usage
    exit 0
fi

while [[ $# -gt 0 ]]; do
    case "$1" in
        -s)
            source_dir="$(get_value $@)"
            shift 2
            ;;

        -b)
            backup_root="$(get_value $@)"
            shift 2
            ;;

        -n|--dry-run)
            dry_run=true
            shift
            ;;

        -h|--help)
            usage
            exit 0
            ;;

        -v|--version)
            echo "backup.sh 4.0.0"
            exit 0
            ;;

        *)
            echo "Invalid argument: $1"
            usage
            exit "$EXIT_MISSING_ARGS"
            ;;
    esac
done

# ---------------------------
# Constants
# ---------------------------
SOURCE="$source_dir"
backup_root="${backup_root%/}" # remove trailing slash
BACKUP_ROOT="${backup_root:-$HOME/backup}"

# ---------------------------
# User Defined Functions
# ---------------------------

cleanup() {
    set +u # staging_dir may not be avl if exit by trying ro run concurrently
    rm -f "$lock_file"
    rm -rf "$staging_dir"
}
trap cleanup EXIT

# notify 
notify() {
    local status="$1"
    local message="$2"
    log_info "NOTIFY [$status]: $message"
}

# check if dir exists, takes 1st arg as dir name
dir_exists() {
    local dir_name="$1"
    [[ -d "$dir_name" ]] && return 0
}

# create dir (including missing parent), 1st arg is dir name
make_dir() {
    local dir_name="$1"
    mkdir -p "$dir_name" || return 1
}

# print check msg for dir, 1st arg is dir name, 2nd arg is identifier of dir
print_dir_check_msg_for () {
    local dir_name="$1"
    local dir_identifier="$2"
    log_info "Checking $dir_identifier directory: '$dir_name'"
}

# create dir only if it doesn't exists, takes 1st arg is dir name, 2nd arg is dir identifier to print 
create_dir_if_not_exists() {
    local dir_name="$1"
    local dir_identifier="$2"
    print_dir_check_msg_for "$dir_name" "$dir_identifier"
    if ! dir_exists "$dir_name"; then
        log_warn "Directory doesn't exists: '$dir_name', Creating..."
        if ! make_dir "$dir_name"; then
            log_error "Creating directory failed: '$dir_name'"
            return 1
        else
            log_info "Directory created successfully: '$dir_name'"
        fi
    fi
}

# create backup, takes 1st arg as source dir, 2nd as staging_dir (temp). 3rd as destination to backup 
create_backup() {
    local source="$1"
    local staging_dir="$2"
    local destination="$3"
    log_info "Backup process start: '$source' to '$destination'"

    # stage first
    log_info "Staging before committing..."

    destination_dirname=$(dirname $destination)
    previous_backup=""
    # gt 1 because shouldn't count current timestamped backup dir ready to backup  
    if [[ "$(ls $destination_dirname | wc -l)" -gt 1 ]]; then
    # normally tail -n 1 should get but we have fresh timestamped dir also, so tail -n 2 | head -n 1
        previous_backup=$(find "$destination_dirname" -maxdepth 1 -type d 2>/dev/null | sort | tail -n 2 | head -n 1)
    fi

    rsync_args=(-a)
    if [[ -n "$previous_backup" ]]; then
        rsync_args+=(--link-dest="$previous_backup")
        log_info "Using previous backup for incremental sync: $previous_backup"
    fi

    staging_rysnc_output=$(rsync ${rsync_args[@]} "$source_dir/" "$staging_dir/" 2>&1)
    staging_rysnc_status=$?

    # exit if staging failed
    if [[ "$staging_rysnc_status" -ne 0 ]]; then
        log_warn "$staging_rysnc_output"
        log_fatal "Failed to sync files from $source to staging area, could be permission issues."
        notify "FAILURE" "Sync failed for $source_dir"
        rm -rf "$destination" # cleanup
        exit "$EXIT_CP_FAILED"
    fi 

    log_info "Synced files to staging area: $staging_dir"

    log_info "Committing staged changes..."
    if [[ "$dry_run" == true ]]; then
        log_info "[DRY RUN] Would move $staging_dir to $destination"
        rm -rf "$destination" # cleanup
        mv_status=0 # delibarately set status true
        mv_output=""
    else
        shopt -s dotglob
        mv_output=$(mv $staging_dir/* "$destination" 2>&1)
        mv_status=$?
        shopt -u dotglob
    fi

    if [[ "$mv_status" -ne 0 ]]; then
        log_warn "$mv_output"
        log_error "Failed to move staged backup into place: $destination"
        notify "FAILURE" "Move to destination failed: $destination"
        return 1
    fi
    printf "Backup success: '$source' to '$destination'\n"
    notify "SUCCESS" "Backup complete: $destination"
}

# just print section end, no args required
print_section_end() {
    printf "\n"
}

# create lock file
set_lock () {
    local lock_file="/tmp/backup-script.lock"
    if [[ -f "$lock_file" ]]; then
        existing_pid=$(cat "$lock_file")
        if kill -0 "$existing_pid" 2>/dev/null; then
            log_fatal "Backup already running as PID $existing_pid"
            return 1
        else
            log_warn "Removing stale lock file: (PID $existing_pid no longer running)"
            rm -f "$lock_file"
        fi
    fi
    echo "$$" > "$lock_file"
    log_info "Lock set: (PID $$)"
    printf "$lock_file"
}

# validate dangerous paths, 1st arg path, 2nd arg path identifier
validate_path() {
    local path="$1"
    local path_identifier="$2"
    if [[ -z "$path" || "$path" == "/" ]]; then
        log_fatal "Refusing to operate on dangerous path for $path_identifier: '$path'"
        return 1
    fi
}

# ---------------------------
# Driver Code
# ---------------------------

# set lock first
lock_file=$(set_lock) || exit "$EXIT_ALREADY_RUNNING"

# validate before starting
validate_path "$SOURCE" "source dir" || exit "$EXIT_INVALID_PATH"
validate_path "$BACKUP_ROOT" "backup root" || exit "$EXIT_INVALID_PATH"

# exit if source doesn't exists 
if ! dir_exists "$SOURCE"; then
    log_fatal "Source directory doesn't exists: '$SOURCE', Exiting..."
    exit "$EXIT_SOURCE_MISSING"
fi

# create if backup root does not exists, exit if creation failed 
create_dir_if_not_exists "$BACKUP_ROOT" "backup root" || (log_fatal "Backup root creation failed, Exiting..." && exit "$EXIT_MKDIR_FAILED")
print_section_end

# create destination, exit if failed
timestamp=$(date +%Y%m%d_%H%M%S)
source_abs_path=$(realpath $SOURCE)
destination="$BACKUP_ROOT/$source_abs_path/$timestamp"
create_dir_if_not_exists "$destination" "destination" || (log_fatal "Creating backup destination failed, Exiting..." && exit "$EXIT_MKDIR_FAILED")
print_section_end

# create backup, exit if failed (script failure exit 1)
staging_dir=$(mktemp -d)
create_backup "$source_abs_path" "$staging_dir" "$destination" || (printf "Backup failed: partial or full, Exiting...\n" && exit "$EXIT_BACKUP_FAILED")

What Changed From v4

  • rsync -a replaces cp -r, fixing the hidden-file gap and cutting repeat-run time significantly on an unchanged or mostly-unchanged source.
  • --link-dest incremental backups, turning unchanged files into free hardlinks against the previous backup instead of full copies, without sacrificing each backup’s independence.
  • A command array (rsync_args) for conditionally-included arguments.
  • destination is full absolute path (destination="$BACKUP_ROOT/$source_abs_path/$timestamp"), both source and destination is passed as absolute path in create_backup

Best Practices

  • Always use -a for backup-style copying with rsync — it’s the difference between a restorable, faithful copy and one that’s silently missing permissions or timestamps.
  • Double-check trailing slashes on rsync source paths every time — it’s a one-character difference with a completely different result, and it’s worth verifying with a small test run before trusting it against real data.
  • Build conditional command-line arguments as arrays, never as a string with an optional piece spliced in..

Shell-Safety Considerations

Here’s the trailing-slash mistake from earlier in this chapter, demonstrated directly — a genuinely common, very well-known rsync gotcha that produces a working-looking result with completely the wrong structure.

Naive (broken) version — no trailing slash on the source path:

mkdir -p /tmp/shell-course-demo/source_dir
touch /tmp/shell-course-demo/source_dir/file1.txt
mkdir -p /tmp/shell-course-demo/dest_broken

rsync -a /tmp/shell-course-demo/source_dir /tmp/shell-course-demo/dest_broken
ls /tmp/shell-course-demo/dest_broken
source_dir

file1.txt isn’t directly inside dest_broken at all — rsync copied source_dir itself as a subdirectory, since there was no trailing slash telling it to copy that directory’s contents instead. Applied to this project’s actual script, this would mean $staging_dir ends up containing a nested source_dir/ folder instead of the backed-up files directly — and the subsequent mv "$staging_dir" "$destination" would produce a backup with a completely different internal structure than every previous version of this script has produced, breaking anything that expects to find files directly at the top level of a backup.

Corrected version — trailing slash on the source path:

mkdir -p /tmp/shell-course-demo/dest_fixed
rsync -a /tmp/shell-course-demo/source_dir/ /tmp/shell-course-demo/dest_fixed
ls /tmp/shell-course-demo/dest_fixed
file1.txt

file1.txt now lands directly inside dest_fixed, exactly matching the flat structure every prior version of this backup script has relied on. The rule worth memorizing, since it’s easy to forget under pressure: a trailing slash on rsync’s source means “copy what’s inside,” no trailing slash means “copy this directory itself.”

It Works

Invoke and see the output:

/backup.sh -s database_project/ -b /tmp/temp/
[2026-08-31 09:25:53] [INFO] Lock set: (PID 45690)
[2026-08-31 09:25:53] [INFO] Checking backup root directory: '/tmp/temp'

[2026-08-31 09:25:53] [INFO] Checking destination directory: '/tmp/temp//home/alice/shell-projects/backup-keeping-script/database_project/20260831_092553'
[2026-08-31 09:25:53] [WARN] Directory doesn't exists: '/tmp/temp//home/alice/shell-projects/backup-keeping-script/database_project/20260831_092553', Creating...
[2026-08-31 09:25:53] [INFO] Directory created successfully: '/tmp/temp//home/alice/shell-projects/backup-keeping-script/database_project/20260831_092553'

[2026-08-31 09:25:53] [INFO] Backup process start: '/home/alice/shell-projects/backup-keeping-script/database_project' to '/tmp/temp//home/alice/shell-projects/backup-keeping-script/database_project/20260831_092553'
[2026-08-31 09:25:53] [INFO] Staging before committing...
[2026-08-31 09:25:53] [INFO] Using previous backup for incremental sync: /tmp/temp//home/alice/shell-projects/backup-keeping-script/database_project/20260831_090459
[2026-08-31 09:25:53] [INFO] Synced files to staging area: /tmp/tmp.vziFF8PnXO
[2026-08-31 09:25:53] [INFO] Committing staged changes...
Backup success: '/home/alice/shell-projects/backup-keeping-script/database_project' to '/tmp/temp//home/alice/shell-projects/backup-keeping-script/database_project/20260831_092553'
[2026-08-31 09:25:53] [INFO] NOTIFY [SUCCESS]: Backup complete: /tmp/temp//home/alice/shell-projects/backup-keeping-script/database_project/20260831_092553
Last updated on