webvulnscan.py: Completeted crawler functionality