Most of Linux users are using wget from time to time, sometimes even when they don’t know about it – many GUI download managers for Linux are based on wget. Anyways wget is command line tool for downloading files over HTTP, HTTPs and FTP protocols within single session. It works like a charm with default settings for downloading simple static files, at the same time if content is protected by cookies and/or referrer then wget may seem useless but it’s actually not.

For example, imagine you need to download some time from restricted area of website: it requires to enter username/password to access it and then stores cookies with session id in browsers cache. When that cookies file is in place you’ll be permitted to download files but once cookies file is missing you’ll get username/password prompt to establish new session. In regards of wget such attempt to download file without cookies will result in html page with login page layout inside file instead of its actual contents.

There is simple solution for this. First, you need to establish session with wget, save resulting cookies file somewhere and then load that cookies to download files from restricted area:

1.

wget --post-data="username=artiomix&pwd=yourpassword" --cookies=on --keep-session-cookies --save-cookies=cookie.txt "http://www.someserver.com/login.php" -O

2.

wget --referer="http://www.someserver.com/login.php" --cookies=on --keep-session-cookies --load-cookies=cookie.txt http://www.someserver.com/somefile.iso

Key --referer is optional but most probably you’ll need it besides cookies to get access to content protected by username/password.

I had been using this wget’s functionality inside the script that establishes session to Cacti and creates graphs for dozens of hosts. Without wget cookies support I would have need to create those graphs by hands (that sucks).

 

4 Comments

 

  1. March 20, 2012  3:26 pm by Yu-Jie Lin Reply

    I think it's not a good idea to save cookie file in /tmp. Everyone can read files in it if you don't change permission.

  2. March 20, 2012  4:12 pm by artiomix Reply

    You're right, thanks Yu-Jie Lin.

  3. March 25, 2012  10:08 pm by link it Reply

    This is a comment to the website owner. Does your website get enough traffic or rank for keywords with Google? Well we can help! We can provide you with tens of thousands of backlinks to your site! This will help your rankings on Google and make your website more visible to your target audience. Take a quick look as I am sure you will be interested.

  4. March 31, 2012  1:49 am by fruit mocking party Reply

    This website doesn't display correctly on my blackberry - you may wanna try and repair that

Leave a reply

 

Your email address will not be published.