Tutorial on Downloading Websites

By | July 16, 2012

Do you ever need to:

  • Take sites you created and designed with you to clients?
  • Keep a backup if important information found on sites online?
  • Browse sites while offline, e.g. when traveling in areas with no internet?
  • Provide alternative mirrors for sites on the intranet and internet.?

If you can answer yes to any of above, you will want to read our tutorial on using our software solution on downloading websites. (Our tool can be used to all of above things, but it has even more uses. )

Finding the Right Tool for The Job

Finding the best web site download solution is not an easy task. Here is a short list of the capabilities in the A1 Website Download software (a tool we have developed) for Windows:

  • Download and convert large dynamic database sites to static ones.
  • Customize download speed through crawl delay and max simultaneous connections.
  • Includes support for a wide range of crawl filters to finely control the download.
  • Can follow and correct a variety of file uses including redirects, frames and images.

Downloading Websites

Have you ever felt you just had to to store a local copy of a website? Perhaps even save it to a CD or USB stick, so you could take it with you. (This can be very useful in many cases, e.g. if helping someone with a computer problem that has no internet access.) This guide will demonstrate how to download entire sites in a snap.

We will be using the A1 Website Download program, our own developed software for the Windows platform. (It can also run other Mac and Linux if using virtualization solutions.) As the default options setup works with the vast majority of all sites, we will get started right-away without any delays 🙂

As a general rule of thumb, it matters little if anything what the website you are trying to download uses as back-end platform, e.g. if it uses PHP as server side language, uses databases or has query parameters in its URLs. The only setting you really need to setup before downloading is the “website root“.

In case you are planning on downloading multiple websites, you may need to configure “Save files crawled to disk directory path” to avoid downloaded sites to overwrite each another on your local disk.

When done, you can click the “Start scan” button to initiate the site crawl and download.

site copying setup

After The Website Download

The A1 Website Download tool will default to show you the site content retrieved when it has finished downloading it all. At this point, you can, e.g. by using “Windows Explorer”, move and store the downloaded website somewhere else, e.g. on an USB stick or disk, and take the complete website with you. You can also browse the web site offline with your preferred internet browser.

Viewing website downloaded

It is worth noting that our websites download tool can also be configured to e.g. only download website images – this being useful in case you are mainly interested in downloading e.g. picture or photo galleries. The same concept can however be used for files of any type such as .PDF documents whatever kind of files.

Tip for Web Developers: Discover Broken Links

If you are helping customers finding broken links in their websites, it can sometimes be difficult to prove and track down error if their websites change content (e.g. stuff keywords), add links (which may be broken) etc. depending on the crawler user agent, cookies support, the last visited page and similar.

However, by downloading the website, you can verify the broken links reported in the software with the actual downloaded files. This makes it much easier to point out exactly where a broken link occurs, even on websites that changes their content randomly.

Leave a Reply