Bash Downloader (wget)

Common wget Commands

wget https://site.com/file.zip

wget -c https://site.com/large.iso

wget -i urls.txt

wget -m https://example.com

🔹 Basic wget Usage

The wget command is a robust, non-interactive network downloader for retrieving files via HTTP, HTTPS, and FTP. Its simplest form downloads a resource directly to your current directory while preserving the original filename. During transfer, it displays a detailed progress bar with download speed, time remaining, and completion percentage. Unlike browser-based downloads, wget handles network interruptions gracefully and is highly reliable for large files or unstable connections. It’s widely used in scripts, automated pipelines, and server environments where a GUI is unavailable.

# Download a file
wget https://example.com/document.pdf

# Download and save with different name
wget -O myfile.pdf https://example.com/document.pdf

🔹 Resume Interrupted Downloads

Resuming partial downloads is a critical feature of wget for handling unstable connections or large files. By adding the -c (continue) option, wget checks the local file’s size and requests only the remaining bytes from the server, avoiding a full restart. This capability is invaluable in low-bandwidth or unreliable network conditions, such as when downloading ISO images, video files, or software packages. It works with most HTTP/1.1 and FTP servers that support range requests, significantly saving time and bandwidth.

# Resume download if interrupted
wget -c https://example.com/large-file.iso

# Resume with custom filename
wget -c -O ubuntu.iso https://releases.ubuntu.com/ubuntu.iso

🔹 Download Multiple Files

Batch downloading multiple files with wget streamlines workflows when you have a list of URLs. First, create a plain text file with one URL per line. Then, use the -i (input file) option to process all listed URLs sequentially. This approach is perfect for mirroring asset collections, downloading datasets, or automating nightly backups without manual intervention. You can combine it with other flags like -c for resumability or -b for background operation, creating a powerful, scriptable bulk-download solution.

# Create a file with URLs
echo "https://example.com/file1.zip" > urls.txt
echo "https://example.com/file2.zip" >> urls.txt

# Download all files from list
wget -i urls.txt

🔹 Background Downloads

Running wget in the background allows you to continue using your terminal while downloads proceed. The -b (background) option detaches wget after startup, logging output to a default file named wget-log in the current directory. This is especially useful for lengthy downloads, such as system images or media archives, where you need the shell for other tasks. You can monitor progress later by examining the log file, and combine background mode with resume and limit-rate features for efficient, unattended file retrieval.

# Download in background
wget -b https://example.com/large-file.zip

# Check download progress
tail -f wget-log

# Background with custom log file
wget -b -o download.log https://example.com/file.zip

🔹 Limit Download Speed

Bandwidth throttling with wget prevents it from consuming your entire network capacity. Use the --limit-rate option followed by a value like 100k (100 kilobytes/second) or 2m (2 megabytes/second). This ensures other critical applications—video calls, web browsing, or real-time services—remain responsive during large downloads. It’s essential in shared or metered network environments, such as offices, data centers, or home networks with multiple users. Throttling can also avoid triggering rate limits on remote servers.

# Limit to 500 KB/s
wget --limit-rate=500k https://example.com/file.zip

# Limit to 1 MB/s
wget --limit-rate=1m https://example.com/large-file.iso

🔹 Download Entire Website

Mirroring an entire website for offline viewing is a powerful wget feature for archiving or local analysis. The -m (mirror) option enables recursive downloading while respecting robots.txt and preserving directory structure. Combine it with -p to fetch all page requisites (images, CSS, JavaScript) and --convert-links to rewrite URLs for local browsing. This creates a fully functional, offline copy of the site, useful for developers, researchers, or content archivists who need reliable access without an internet connection.

# Mirror a website
wget -m https://example.com

# Mirror with converted links for offline viewing
wget -m -p -k https://example.com

# Mirror with depth limit
wget -m -l 2 https://example.com

🔹 Quiet and Verbose Modes

Controlling output verbosity in wget helps tailor its behavior for scripts or debugging. Use -q (quiet) to suppress all non‑error output, ideal for cron jobs or automated scripts where silence is preferred. Conversely, -v (verbose) provides detailed connection, HTTP header, and transfer data—invaluable for troubleshooting failed downloads, SSL issues, or server misconfigurations. You can also use -nv (non‑verbose) for a concise summary. Choosing the right output level enhances script cleanliness and diagnostic clarity.

# Quiet mode (no output)
wget -q https://example.com/file.zip

# Verbose mode (detailed output)
wget -v https://example.com/file.zip

# No verbose (default)
wget -nv https://example.com/file.zip