Monday, March 5, 2012

Mirroring Using WGET and LFTP

Mirroring a website is sometime a useful trick to have an offline version of a website. Several ways can be used to get a full copy of the website and what's good about it is that some of the tools are already available at Slackware's default packages. They are wget and lftp.

In wget, you can use -m or --mirror option to turn on recursion which acts like a mirroring program, like this wget -m <host>

In lftp, we can use mirror sub command to perform a mirroring operation and then add several options to make it work like rsync, like this: lftp -e "mirror --delete --only-newer" <host> <target>

Be very cautious with --delete option as it might delete files available on your system, but not on the remote site, so it's safer to run this script in an empty directory and move it later when it's completed.

If you prefer to use GUI application, HTTrack is one of the best tool available. It's also available for other platform as well.