Updated crawler.py

Integratable to django code