Reworked the code to work from top to bottom.
It now has a function to open webpages and extract the contents if they are possibly gzipped.
Merge branch 'master' of https://github.com/klaxa/batotocrawler