wget Command Deep Dive

If curl is a precise HTTP client, wget approaches the same network from a different direction: retrieving files and resources. The commands can overlap, but the difference becomes obvious when you move beyond “download this one URL” into recursive downloads, continuing interrupted transfers, and mirroring a site.

The useful question isn’t “which command is better?” It is:

Am I trying to interact with a specific HTTP request, or am I primarily trying to retrieve resources?

What Is wget?

wget is a command-line utility designed primarily for downloading resources over protocols such as HTTP and HTTPS.

The simplest example is:

wget https://example.com/file.tar.gz

Instead of writing the response body directly to your terminal, wget normally saves the downloaded resource as a file.

Conceptually:

URL
 ↓
wget
 ↓
HTTP request
 ↓
Response
 ↓
Local file

That default behavior immediately makes wget convenient for ordinary downloads.

Compare it with:

curl https://example.com/file.tar.gz

which writes the response body to standard output unless you tell it otherwise.

With curl, you explicitly choose how to save the result:

curl -O https://example.com/file.tar.gz

With wget, saving the resource is already the normal operation:

wget https://example.com/file.tar.gz

A Basic Download

Run:

wget https://example.com/file.tar.gz

If the server allows the download, you’ll see progress information and a local file will be created.

List the directory:

ls

You should find:

file.tar.gz

The important difference from a simple curl command is the default intent:

curl
    → give me the HTTP response

wget
    → download this resource

Both can perform the other task, but their defaults reflect their different strengths.

Choose The Output Filename

You can choose the local filename with:

wget -O archive.tar.gz https://example.com/file.tar.gz

Here -O means:

Write the downloaded response to this file.

This is similar in purpose to:

curl -o archive.tar.gz https://example.com/file.tar.gz

Notice that the capitalization differs:

curl
    -o

wget
    -O

Don’t rely on the option name alone when switching between the two tools.

Continue An Interrupted Download

One of wget’s particularly useful download-oriented features is continuing an existing download.

Suppose:

large-file.iso

was only partially downloaded.

You can try:

wget -c https://example.com/large-file.iso

The -c option means continue.

Conceptually:

Already downloaded:
[====================          ]

wget -c
        ↓
Continue from here
[==============================]

This is useful for large files or unreliable connections.

The remote server must support the necessary range requests for resuming to work correctly.

With curl, the equivalent pattern is:

curl -C - -O https://example.com/large-file.iso

So both tools can resume downloads, but wget makes the operation feel more natural because downloading is its primary job.

Follow Redirects

Modern URLs frequently redirect.

For example:

http://example.com/file.tar.gz
        ↓
https://downloads.example.com/file.tar.gz

wget normally follows HTTP redirects when retrieving resources.

This is one reason it is convenient for downloads: you usually don’t need to explicitly add a redirect-following option for the ordinary case.

With curl, you commonly write:

curl -L -O https://example.com/file.tar.gz

The difference illustrates the broader design philosophy:

curl
    → explicit control over the request

wget
    → convenient retrieval behavior

See HTTP Headers

wget can display server response information using:

wget --server-response https://example.com

You may see information including:

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: ...

This is useful when you need to understand what the server returned but your primary goal is still downloading the resource.

With curl, the corresponding common inspection patterns are:

curl -i https://example.com

or:

curl -v https://example.com

curl generally gives you a more direct request/response debugging experience.

Quiet Downloads

If you don’t want the normal progress display, use:

wget -q https://example.com/file.tar.gz

The -q option means quiet.

This is useful in scripts where normal progress output isn’t useful.

You can combine it with other options when you only care whether the download succeeds.

Set A Timeout

Network operations should not always be allowed to wait indefinitely.

You can set a timeout with:

wget --timeout=10 https://example.com/file.tar.gz

You can also control retry behavior.

For example:

wget --tries=3 https://example.com/file.tar.gz

This asks wget to make a limited number of attempts rather than retrying indefinitely.

These options become useful when wget is used in scripts or scheduled jobs.

Download Multiple URLs

wget can work with multiple URLs directly:

wget https://example.com/file1.tar.gz \
     https://example.com/file2.tar.gz  \
     https://example.com/file3.tar.gz

It can also read URLs from a file.

Create:

urls.txt

with:

https://example.com/file1.tar.gz
https://example.com/file2.tar.gz
https://example.com/file3.tar.gz

Then:

wget -i urls.txt

The -i option tells wget to read URLs from the specified input file.

This is useful when the download list is generated separately from the actual downloading step.

Why wget Is Interesting For Website Downloads

Now we reach the area where wget starts to become substantially different from the way you normally use curl.

Suppose this page:

https://example.com/docs/

contains links to:

/docs/install.html
/docs/config.html
/docs/networking.html

A normal curl command retrieves one response:

curl https://example.com/docs/

It does not automatically crawl the links found in that HTML.

wget can recursively retrieve linked resources.

For example:

wget --recursive https://example.com/docs/

Now the operation is closer to:

docs/
 │
 ├── index.html
 ├── install.html
 ├── config.html
 └── networking.html

Instead of manually discovering every URL and downloading it yourself, wget can follow links and retrieve additional resources.

This is one of its biggest practical advantages over curl.

Limit Recursion

Recursive downloading needs boundaries.

You normally don’t want to tell a downloader:

"Follow everything you can find on the Internet."

You can limit recursion depth.

For example:

wget --recursive --level=2 https://example.com/docs/

This limits how deeply wget follows links.

Conceptually:

Start page
   │
   ├── Level 1
   │     ├── page A
   │     └── page B
   │
   └── Level 2
         ├── page C
         └── page D

Without a sensible boundary, recursive retrieval can produce far more content than you intended.

Download A Site’s Dependencies

A common goal is to retrieve a page and the resources required to view it locally.

For example:

index.html
   ├── style.css
   ├── app.js
   └── image.png

You can use:

wget --page-requisites https://example.com/

The --page-requisites option tells wget to retrieve resources needed by the page.

This is different from simply downloading the HTML document.

The distinction is:

wget URL
    → download the resource

wget --page-requisites URL
    → download the page and resources it needs

Convert Links For Local Browsing

If you download a collection of HTML pages, their links may still point to the original website.

For a local copy, you may want those links rewritten.

wget supports this with:

wget --convert-links https://example.com/docs/

Now links in downloaded documents can be adjusted so that they point to the corresponding local files.

This becomes especially useful when combined with recursive retrieval.

For example:

wget   --recursive   --convert-links   https://example.com/docs/

The result is much closer to a browsable local copy.

Mirror A Website

wget provides a higher-level mode specifically intended for mirroring.

For example:

wget --mirror https://example.com/docs/

--mirror enables a collection of options appropriate for mirroring a site.

A more explicit form might be:

wget   --mirror   --convert-links   --page-requisites   https://example.com/docs/

This is the kind of operation where wget becomes much more convenient than curl.

With curl, you’d have to build the crawling, URL discovery, file naming, and link handling yourself.

With wget, those concerns are already part of the tool’s purpose.

Restrict Downloads To One Domain

Recursive downloading can become dangerous if links lead outside the site you intended to retrieve.

You can restrict recursion to the target domain:

wget --recursive --no-parent --domains=example.com https://example.com/docs/

Here:

--recursive
    follow links

--no-parent
    don't move above the starting directory

--domains=example.com
    restrict retrieval to the specified domain

This is a much safer pattern for a controlled documentation download.

A Practical Documentation Download

Suppose you want a local copy of:

https://example.com/docs/

and the documentation consists of multiple linked pages.

A reasonable demonstration is:

wget   --recursive   --convert-links   --page-requisites   --no-parent   --domains=example.com   https://example.com/docs/

The important parts are:

recursive
    follow documentation links

convert-links
    make downloaded links useful locally

page-requisites
    retrieve resources required by pages

no-parent
    stay below /docs/

domains
    don't wander onto unrelated domains

This is a good example of why wget exists as a distinct tool rather than simply being another spelling of curl.

wget And HTTP Requests

Although wget is download-oriented, it can still make HTTP requests and provide useful control.

For example:

wget --header='Accept: application/json' https://example.com/api/users

This adds a request header.

You can also send data:

wget   --header='Content-Type: application/json'   --post-data='{"name":"alice"}'   https://example.com/api/users

So wget is not limited to anonymous file downloads.

However, once your task becomes:

inspect exact request
debug headers
send several custom headers
reproduce an API request
inspect TLS/connection details

curl usually becomes the more natural tool.

curl Versus wget

The overlap is large enough that people often ask which one they should use.

The better way to think about it is by task.

Neither tool is universally better.

A useful rule is:

Need to interact with an HTTP service?
        ↓
       curl

Need to retrieve resources?
        ↓
       wget

It’s not an absolute rule, but it is a very good starting point.

The Same Download With Both Tools

Suppose you need:

https://example.com/file.tar.gz

With curl:

curl -L -O https://example.com/file.tar.gz

With wget:

wget https://example.com/file.tar.gz

Both accomplish the same basic task.

Now suppose you want to inspect the HTTP exchange.

With curl:

curl -v https://example.com/file.tar.gz

This is where curl starts to feel more natural.

Now suppose you want to recursively download an entire documentation tree:

wget --recursive https://example.com/docs/

This is where wget has a clear advantage.

The important lesson is not to memorize a winner.

It is to recognize the problem shape.

wget And The Networking Layers

Like curl, wget exercises the layers you’ve already studied:

wget https://example.com/file.tar.gz
              │
              ▼
           DNS
              │
              ▼
          IP address
              │
              ▼
           Routing
              │
              ▼
       Network interface
              │
              ▼
       TCP connection :443
              │
              ▼
             TLS
              │
              ▼
            HTTP
              │
              ▼
       downloaded file

The difference is mainly in what the application does with the HTTP interaction.

curl exposes the request/response exchange as the primary object.

wget treats retrieval as the primary object and adds features around that workflow.

A Useful wget Pattern Collection

You don’t need to memorize the entire manual. These patterns cover the most useful operations for this course.

Download:

wget https://example.com/file.tar.gz

Choose the local filename:

wget -O archive.tar.gz https://example.com/file.tar.gz

Continue an interrupted download:

wget -c https://example.com/large-file.iso

Read URLs from a file:

wget -i urls.txt

Show server response information:

wget --server-response https://example.com

Quiet operation:

wget -q https://example.com/file.tar.gz

Recursive download:

wget --recursive https://example.com/docs/

Limit recursion:

wget --recursive --level=2 https://example.com/docs/

Download page dependencies:

wget --page-requisites https://example.com/

Rewrite links for local browsing:

wget --convert-links https://example.com/docs/

Mirror a site:

wget --mirror https://example.com/docs/

Restrict a recursive download:

wget --recursive --no-parent --domains=example.com https://example.com/docs/

The important thing is to understand what each pattern is trying to accomplish rather than memorizing every option.

What You Should Remember

wget and curl overlap, but they have different centers of gravity.

curl is particularly strong when you want to control and inspect a specific HTTP interaction:

request
   ↓
response

wget is particularly strong when you want to retrieve resources:

URL
 ↓
download
 ↓
local files

And when the task becomes recursive:

page
 ↓
linked pages
 ↓
linked resources
 ↓
local copy

wget becomes especially convenient.

The practical distinction is:

Reach for curl when the HTTP request itself is the thing you’re investigating or constructing. Reach for wget when downloading and retrieving resources is the main job.

Knowing both means you don’t have to force one tool into a job the other handles naturally.

What’s Next

We’ve now used curl and wget to interact with network services from the command line. The next topic brings the pieces together into network debugging: how to determine whether a failure is caused by the interface, IP configuration, routing, DNS, a listening service, or the connection itself.