crawling.py: Improved crawling algorithm for speed.
crawling.py: Small adjustions for pep8 and pyflakes