A Beginners Guide to Using wget in Windows

a-beginners-guide-to-using-wget-in-windows-1

Wget is a GNU program primarily used in Linux and Unix to download files from the internet. It is a simple command line tool that can also be used within a Windows environment. With wget, you can download anything you like from entire websites to movies, music, podcasts and large files from anywhere online.

Not many Microsoft users know about this neat tool, which is why I wrote the beginners guide to using wget in Windows. We tend to use our browser for everything, which is fine but it isn’t always the most efficient way to achieve something. Wget is just one of the many tools that have been around for eons but very few people know about. That all changes today.

Once installed within Windows, you can download what you like, where you like, when you like. You just tell wget what to get and where from and it takes care of everything else. You need do nothing, hence the moniker ‘non-interactive network retriever’.

So here is my guide to using wget in Windows.

Using wget in Windows

Using wget in Windows is quite straightforward once set up. Follow this guide to installing and configuring wget.

  1. Download wget from here and install it. Make sure it is the setup program and not just the source otherwise it won’t work.
  2. Once installed, you should now be able to access the wget command from a command line window. Open a CMD window as an administrator and type ‘wget -h’ to test. If it works, you’re golden, if you get ‘unrecognized command’ you downloaded the wrong package. Try again.
  3. Set a download directory to save all your files. Type ‘md \directory name’ to create a download directory. I called mine ‘downloadz’ to be recognizable.

a-beginners-guide-to-using-wget-in-windows-2

Once installed, you’re ready to set to work. Below I have listed a selection of popular wget commands that can achieve a wide range of things.

Download a single file

wget http://website.com/file.zip

Download a single file but save it as something else

wget ‐‐output-document=newname.html website.com

Download to a specific folder

wget ‐‐directory-prefix=folder/subfolder website.com/file.zip

Resume an interrupted download

wget ‐‐continue website.com /file.zip

Download a newer version of a file

wget ‐‐continue ‐‐timestamping website.com/file.zip

a-beginners-guide-to-using-wget-in-windows-3

Download multiple web pages

For this you need to create a list in Notepad or other text editor. Add a new full URL (with http://) onto a separate line. Then point wget to the file. In this example I named the file Filelist.txt and saved it in the wget folder.

wget ‐‐input Filelist.txt

Download an entire website

wget ‐‐execute robots=off ‐‐recursive ‐‐no-parent ‐‐continue ‐‐no-clobber http://website.com

You might find, as I often do that web hosts block wget commands. You can try to spoof these blocks by impersonating Googlebot. Try typing this:

wget –user-agent=”Googlebot/2.1 (+http://www.googlebot.com/bot.html)” -r http://website.com

Download a specific file type from a website

wget ‐‐level=1 ‐‐recursive ‐‐no-parent ‐‐accept FILETYPE http://website.com / FILETYPE/

For example, change FILETYPE for MP3, MP4, .zip or whatever you like.

Download all website images

wget ‐‐directory-prefix=files/pictures ‐‐no-directories ‐‐recursive ‐‐no-clobber ‐‐accept jpg,gif,png,jpeg http://website.com/images/

Check a website for broken links

wget ‐‐output-file=logfile.txt ‐‐recursive ‐‐spider http://website.com

Download files without overloading the web server

wget ‐‐limit-rate=20k ‐‐wait=60 ‐‐random-wait ‐‐mirror http://website.com

There are hundreds, if not thousands of wget commands and I only know a few of them. Now you’re familiar with the tool and how it works, it’s up to you what you use it for!

Do you have any cool commands that can achieve wonders? Share them with us below!

Posted by Jamie on November 23, 2016

Leave a Reply

Your email address will not be published. Required fields are marked *


TechJunkie Also Recommends