Redirections and Pipelines

Every command you run is quietly connected to three data streams the moment it starts — one it reads from, two it writes to — whether or not you ever think about them. Redirection is just rewiring those connections: sending output to a file instead of your screen, feeding a file in as if it were typed input, or chaining commands together so one’s output becomes another’s input. This chapter covers all of that machinery, plus a pipeline exit-status gotcha that catches even experienced scripters off guard.

The Three Standard Streams

Every process starts with three open file descriptors, numbered:

Notice stdout and stderr both go to your terminal by default — which is why, without redirection, normal output and error messages appear mixed together on screen, even though they’re two genuinely separate streams under the hood. Splitting them apart, or sending either one somewhere else entirely, is what the rest of this chapter is about.

Redirecting Output: > and >>

> sends a command’s stdout to a file instead of the terminal, overwriting that file completely if it already exists:

echo "First line" > output.txt
cat output.txt

First line

echo "This replaces everything" > output.txt
cat output.txt

This replaces everything

>> does the same thing but appends instead of overwriting:

echo "Added line" >> output.txt
cat output.txt

This replaces everything
Added line

The difference between these two is a common source of accidentally destroyed data — reaching for > when you meant >> silently erases whatever was there before, with no warning or confirmation.

Redirecting Input: <

< feeds a file’s contents to a command’s stdin, as though you’d typed them:

printf "line one\nline two\nline three\n" > lines.txt
cat < lines.txt

line one
line two
line three

This particular example doesn’t look different from cat lines.txt directly — cat accepts a filename argument as well — but the distinction matters for commands and scripts that only read from stdin and have no filename-argument option at all. Redirecting < gives you a way to feed such a command file-based input regardless.

Redirecting Standard Error: 2> and 2>>

Since stderr is file descriptor 2, redirecting it uses the same operators with 2 prefixed:

ls existing-report.txt does-not-exist.txt

ls: cannot access 'does-not-exist.txt': No such file or directory
existing-report.txt

Both lines appear together on screen because stdout and stderr both default to the terminal — but they’re genuinely separate streams, and you can prove it by redirecting only one:

ls existing-report.txt does-not-exist.txt 2> errors.txt

existing-report.txt

cat errors.txt

ls: cannot access 'does-not-exist.txt': No such file or directory

The normal filename listing still printed to the terminal — only the error line was diverted into errors.txt. 2>> appends to an existing error log the same way >> does for stdout.

Merging Streams: 2>&1

Sometimes you want both stdout and stderr going to the same place — a single combined log file, for example. 2>&1 means “make file descriptor 2 point wherever file descriptor 1 currently points”:

ls existing-report.txt does-not-exist.txt > combined.log 2>&1
cat combined.log

ls: cannot access 'does-not-exist.txt': No such file or directory
existing-report.txt

Both the normal output and the error message ended up in the same file, in the order the command actually produced them.

Redirecting Stdout To Stderr: >&2

There’s a reverse case worth knowing too: sending something that would normally go to stdout over to stderr instead, using >&2 (short for 1>&2):

echo "This is a diagnostic message" >&2

Why deliberately do this? Stdout is the stream callers actually care about — piping into another command, redirecting to a file, or capturing the output for later use. A script’s diagnostic or progress messages don’t belong mixed into that stream; if they are, anything consuming the script’s real output ends up parsing your status noise right along with it. Sending diagnostics to stderr keeps stdout clean for whatever the script is actually meant to produce.

Try it yourself:

cat > status-and-result.sh << 'EOF'
#!/usr/bin/env bash

echo "Starting process..." >&2
echo "42"
EOF

chmod +x status-and-result.sh
./status-and-result.sh > result.txt

See what gets in the file:

cat result.txt

42

The "Starting process..." line never touched result.txt — it went to the terminal via stderr, while only the real result, 42, landed in the file.

Discarding Output: /dev/null

/dev/null is a special file that discards anything written to it and always reads as empty. Redirect output there to silence it entirely:

ls existing-report.txt does-not-exist.txt > /dev/null 2>&1
echo "Exit status: $?"

Exit status: 1

Nothing printed at all — both streams were discarded — but the command still ran and still produced a real exit status you can check with $?. This pattern, > /dev/null 2>&1, is extremely common for scripts that need to run a command purely for its side effects or exit status, with no interest in its output either way.

Pipelines

A pipe (|) connects one command’s stdout directly to the next command’s stdin, without ever touching a file:

printf "banana\napple\ncherry\n" | sort

apple
banana
cherry

Chain more than two commands together the same way:

printf "banana\napple\ncherry\napple\n" | sort | uniq | wc -l

3

Each stage runs as its own separate process, connected to the next by a direct pipe rather than an intermediate file — faster, and without leaving temporary files behind. One consequence worth knowing now: because each stage is a separate process, changes a stage makes to shell state — variables it sets, for instance — don’t persist once the pipeline finishes. The mechanics of exactly why fall under subshells, covered fully later in the course.

tee — Splitting A Stream

tee reads from stdin and writes the same data to both a file and stdout simultaneously, letting you save output partway through a pipeline without breaking the chain:

printf "banana\napple\ncherry\n" | sort | tee sorted.txt | tail -n 1

cherry

cat sorted.txt

apple
banana
cherry

The full sorted list landed in sorted.txt, while the pipeline continued on to tail -n 1, which still received the complete sorted output and printed just the last line. tee -a appends instead of overwriting, and tee can take multiple filenames to write the same stream to several files at once.

Pipeline Exit Status & PIPESTATUS

Checking $? after a pipeline only tells you about the last command in it — every earlier stage’s success or failure is invisible by default:

false | true
echo "Exit status: $?"

Exit status: 0

false (a builtin that always fails) ran first and failed, but because true — the last stage — succeeded, $? reports success for the whole pipeline. A failure in an earlier stage was silently swallowed.

Bash tracks every stage’s individual exit status in a special array variable, PIPESTATUS, immediately after a pipeline runs:

false | true
echo "${PIPESTATUS[0]} ${PIPESTATUS[1]}"

1 0

PIPESTATUS is indexed starting at 0 for the first command in the pipeline, 1 for the second, and so on — you’ll cover array syntax in full later in this course, but this much is enough to read individual stage results right now: ${PIPESTATUS[0]} is the first command’s exit status, ${PIPESTATUS[1]} the second’s, and so on down the pipeline.

Best Practices

• Default to 2>&1 placed immediately after your main redirection (> file 2>&1, not the reverse) — treat the correct order as a fixed pattern to reach for, not something to reconstruct from first principles each time.
• Use >> deliberately, not > out of habit, for anything that logs or accumulates over multiple runs — a script that overwrites its own log file every time it runs isn’t logging anything useful.
• Check PIPESTATUS whenever a pipeline’s correctness genuinely depends on every stage succeeding, not just the last one — especially for pipelines involving filtering or processing stages where a silent early failure could produce misleadingly “successful” output.
• Reach for tee when debugging a pipeline you’re not sure is behaving correctly — inserting tee debug.txt between stages lets you inspect exactly what’s flowing through at that point, without altering the pipeline’s actual behavior.

Shell-Safety Considerations

The 2>&1 ordering issue mentioned above is common enough, and easy enough to get backwards under pressure, that it’s worth seeing both versions run side by side.

Naive (broken) version — stderr merged before stdout is redirected:

ls existing-report.txt does-not-exist.txt 2>&1 > combined.log

ls: cannot access 'does-not-exist.txt': No such file or directory

That error message printed straight to your terminal — it was never captured. Check the file to confirm:

cat combined.log

existing-report.txt

Only the normal output made it into combined.log. The error is gone from the file entirely, sitting only in whatever terminal happened to be watching when the command ran — exactly the kind of gap that turns into a real problem when this pattern is used inside an unattended script whose “log file” was supposed to capture everything.

Corrected version — stdout redirected first, stderr merged second:

ls existing-report.txt does-not-exist.txt > combined.log 2>&1
cat combined.log

ls: cannot access 'does-not-exist.txt': No such file or directory
existing-report.txt

Both lines are captured this time. The rule to hold onto: read 2>&1 as “point stderr at wherever stdout currently points” — which means whatever redirects stdout has to come first, or 2>&1 is duplicating the wrong target.

Hands-On: Redirection, Pipelines, And Catching A Hidden Pipeline Failure

1. Set up a working directory.

mkdir -p ~/shell-course/ch5
cd ~/shell-course/ch5

2. Practice > vs. >>.

echo "First run" > log.txt
cat log.txt
echo "Second run" > log.txt
cat log.txt
echo "Third run" >> log.txt
cat log.txt

Confirm the second echo completely replaced the first, while the third one added on rather than replacing.

3. Separate stdout and stderr from a command that produces both.

touch existing-report.txt
ls existing-report.txt does-not-exist.txt 1> stdout.txt 2> stderr.txt
cat stdout.txt
cat stderr.txt

4. Reproduce the 2>&1 ordering bug, then fix it.

ls existing-report.txt does-not-exist.txt 2>&1 > combined-broken.log
cat combined-broken.log

existing-report.txt

ls existing-report.txt does-not-exist.txt > combined-fixed.log 2>&1
cat combined-fixed.log

ls: cannot access 'does-not-exist.txt': No such file or directory
existing-report.txt

5. Build a small pipeline and split it with tee.

printf "banana\napple\ncherry\napple\n" | sort | uniq | tee unique-sorted.txt | wc -l
cat unique-sorted.txt

6. Catch a hidden pipeline failure with PIPESTATUS.

false | true
echo "Pipeline \$?        : $?"
echo "First stage status : ${PIPESTATUS[0]}"
echo "Second stage status: ${PIPESTATUS[1]}"

Pipeline $?        : 0
First stage status : 1
Second stage status: 0

7. Clean up.

cd ~
rm -rf ~/shell-course/ch5

You’ve now redirected each stream independently, merged them correctly (and seen exactly how getting the order backwards silently drops error output), built and split a pipeline with tee, and used PIPESTATUS to catch a failure that $? alone would have hidden completely. The next chapter covers here documents and here strings — feeding multi-line input to a command without a separate file at all.