Fix bug in py26 where timedelta.total_seconds is not defined
Fix failing tests in py33
Update docs
Refactor crawler
Add more tests for crawler