Background Jobs and Waiting

Every command so far in this course has run to completion before the next line executes. & breaks that: it starts a command running in the background and immediately hands control back, letting your script keep going while that command works independently. This is genuinely useful for running independent tasks concurrently — but a background job’s exit status doesn’t behave the way you’d naturally guess, and this chapter’s safety section shows exactly where that assumption breaks.

Running A Command In The Background

Append & to send a command to the background:

sleep 3 &
echo "Started sleep in the background — script continues immediately"

[1] 48213
Started sleep in the background — script continues immediately

That [1] 48213 line is Bash’s job-control notification: job number 1, process ID 48213. Control returned to your script (or interactive prompt) instantly — it didn’t wait for sleep 3 to actually finish.

$! — Capturing The Background Process’s PID

$!, introduced back in Chapter 10, holds the PID of the most recently backgrounded job — the same number Bash just printed:

sleep 3 &
pid=$!
echo "Backgrounded sleep with PID $pid"

Capture this immediately after starting the background job if you’ll need to reference it specifically later — starting another background job overwrites $! with the new one.

wait — Waiting For A Background Job To Finish

wait pauses the script until a background job completes:

sleep 2 &
pid=$!
echo "Waiting for PID $pid..."
wait "$pid"
echo "Done waiting."

Called with no arguments, wait waits for every currently running background job. Called with a specific PID (or job number), it waits for just that one, letting others continue running independently in the meantime.

Capturing A Background Job’s Exit Status

Here’s the part that catches people off guard: checking $? immediately after command & does not tell you whether command succeeded — it tells you whether the act of backgrounding it succeeded, which is almost always a trivial, immediate success regardless of what the backgrounded command eventually does. To get the backgrounded command’s actual exit status, wait on its specific PID — wait itself returns that job’s real exit status as its own:

false &
pid=$!
wait "$pid"
echo "Actual exit status of the background job: $?"

Actual exit status of the background job: 1

This distinction, and the misleading version it corrects, is covered in full in this chapter’s Shell-Safety Considerations section below.

Running Multiple Independent Tasks Concurrently

The real payoff of background jobs is running several independent pieces of work at once instead of strictly one after another. Track each job’s PID as you launch it, then wait for all of them:

pids=()
for n in 1 2 3; do
    ( sleep "$n"; echo "Task $n finished" ) &
    pids+=("$!")
done

for pid in "${pids[@]}"; do
    wait "$pid"
done
echo "All tasks complete"

Task 1 finished
Task 2 finished
Task 3 finished
All tasks complete

Each task runs concurrently — the whole thing takes roughly as long as the slowest task (about 3 seconds here), not the sum of all three, because they’re genuinely running in parallel rather than one after another.

Avoiding Unmanaged Background Processes

A background job whose output is never redirected still writes directly to the terminal (or wherever the script’s own stdout/stderr go), which means its output can interleave unpredictably with whatever your script prints on its own — two streams of text arriving at the same destination with no guaranteed ordering between them. In practice, redirect a background job’s output somewhere dedicated unless you specifically want it mixed in:

long_running_task > task.log 2>&1 &

A script that starts background jobs and then exits without ever calling wait also leaves those jobs running detached, with no further connection to the script that started them — sometimes intentional, but worth being a deliberate choice rather than an oversight. If a script’s correctness depends on background work actually finishing, wait for it explicitly before the script itself exits.

Best Practices

• Capture $! immediately after backgrounding a job you’ll need to reference specifically later — it gets overwritten by the next backgrounded command.
• Use wait "$pid" to get a background job’s real exit status — never trust $? checked immediately after the & line itself.
• Redirect a background job’s output to a file unless you deliberately want it interleaved with the rest of your script’s output on the terminal.
• Wait for every background job your script depends on before the script exits, rather than leaving work running unmanaged and disconnected from the script that started it.

Shell-Safety Considerations

Checking $? right after backgrounding a command, expecting it to reflect that command’s eventual success or failure, is an easy and very natural mistake — it just happens to be checking the wrong thing entirely.

Naive (broken) version:

false &
echo "Exit status immediately after backgrounding: $?"

Exit status immediately after backgrounding: 0

false is a command that always fails — yet the reported exit status is 0, success. $? here reflects whether Bash successfully started the background job (it did — starting a job that will later fail is still a successful start), not whether false itself succeeded once it actually ran. Depending on timing, false may not have even finished running yet by the time this echo executes.

Corrected version — capture the PID and wait on it specifically:

false &
pid=$!
wait "$pid"
echo "Actual background job exit status: $?"

Actual background job exit status: 1

1 — the real result of false. This is the only reliable way to learn a background job’s actual outcome: capture its PID with $! right after starting it, then wait on that specific PID and read $? immediately afterward.

Hands-On: Backgrounding, Waiting, And Capturing Real Exit Status

1. Set up a working directory.

mkdir -p ~/shell-course/ch16
cd ~/shell-course/ch16

2. Start a background job and wait for it.

sleep 2 &
pid=$!
echo "Waiting for PID $pid..."
wait "$pid"
echo "Background job finished."

3. Reproduce the misleading immediate-$? check, then fix it with wait.

echo "--- broken ---"
false &
echo "Exit status immediately after: $?"

echo "--- fixed ---"
false &
pid=$!
wait "$pid"
echo "Actual exit status: $?"

4. Run several tasks concurrently and wait for all of them.

pids=()
for n in 1 2 3; do
    ( sleep "$n"; echo "Task $n finished" ) &
    pids+=("$!")
done

for pid in "${pids[@]}"; do
    wait "$pid"
done
echo "All tasks complete"

5. Redirect a background job’s output to avoid interleaving.

( echo "background output line 1"; sleep 1; echo "background output line 2" ) > background.log 2>&1 &
bg_pid=$!

echo "Main script output line 1"
echo "Main script output line 2"

wait "$bg_pid"
cat background.log

6. Clean up.

cd ~
rm -rf ~/shell-course/ch16

You’ve now started background jobs, tracked them with $!, waited for one or several of them, and confirmed directly why $? checked immediately after & tells you nothing about the backgrounded command’s real outcome. The next chapter covers getopts — proper command-line option parsing, replacing the manual $1/$2-checking approach you’ve been using for arguments so far.