Some fixes for the python web crawler, now works.
Added gitignore