Initial import.
Support sequential grabbing. Change db to record visited.
Adjust sleeping times.
Raise exception on no iframe instead
Show some counts.
Adds sleep-max option.
Uses gzip compression.
Catches http errors
Adds export script.
Show some stats
Adds logging to file
Close connection on error. Don't mark url as not existing on error.
Skips no iframe.
Trims url msg
Reduces number of stats msgs
Adds license and readme.
Adds reverse-sequential, quiet, database-dir, log-dir options.
Adds example host config files.
Adds missing database_dir
Updates sample configs