Skip to content

wget Command: Download Files from the Web

Summary

wget is a command-line utility for retrieving files using HTTP, HTTPS, and FTP, the most widely-used Internet protocols. It supports proxy servers, persistent HTTP connections, background downloading, and much more.

Introduction

wget (which stands for "World Wide Web Get") is a powerful and non-interactive command-line download manager. It's particularly useful for downloading files or entire websites from a remote server. Because it's non-interactive, wget can easily be incorporated into scripts and cron jobs for automated downloads.

Use case and Examples

Download a single file

wget https://example.com/document.pdf
This command downloads the file document.pdf from example.com and saves it in the current directory.

Download a file with a different name

wget -O my_document.pdf https://example.com/document.pdf
This command downloads document.pdf from example.com but saves it as my_document.pdf in the current directory. The -O option specifies the output filename.

Download a file in the background

wget -b https://example.com/large_file.iso
This starts the download of large_file.iso in the background. You can check the progress by viewing the wget-log file.

Download multiple files from a list

First, create a file named urls.txt containing a list of URLs, one URL per line: 

https://example.com/file1.txt
https://example.com/file2.txt
https://example.com/file3.txt
Then, use wget to download all the files: 
wget -i urls.txt
The -i option tells wget to read URLs from the specified file.

Limit download speed

wget --limit-rate=200k https://example.com/big_file.zip
This command limits the download speed to 200 kilobytes per second.

Resume an interrupted download

wget -c https://example.com/very_large_file.zip
If a download is interrupted, the -c option allows wget to continue downloading from where it left off.

Commonly used flags

Flag Description Example
-O Specify the output filename. wget -O new_name.txt https://example.com/file.txt
-b Run wget in the background. Output is written to wget-log. wget -b https://example.com/large_file.iso
-i Read URLs from a file. wget -i urls.txt
-c Continue an interrupted download. wget -c https://example.com/very_large_file.zip
-q Quiet mode (suppress output). wget -q https://example.com/file.txt
--limit-rate Limit the download speed. e.g., 200k, 1m. wget --limit-rate=100k https://example.com/big_file.zip
-r Recursive download (download entire website). wget -r https://example.com
-l Sets recursion level, used with -r. wget -r -l 2 https://example.com
-A Specify accepted file extensions. wget -A .pdf,.txt https://example.com
-nH Don't create host directories (Useful when recursive download is happening). wget -r -nH https://example.com


Share on Share on

Comments