Removed Seen as it wasen't fruitful. Just a simple list will do. Sorting plus lookup takes O(nlogn) time whereas with simple lists, look up will take O(n) time.
Added documentation. Also added tests
Cleaning up
Added error checking but there is a bug in crawl method of Bot
Removed the url attribute of Process class. The run now takes a single url argument and downloads from there. Simplifies the design due to rid of excessive variables
Adding duplicate url checking
Merge https://github.com/SaurabhJha/Web-Crawler

Conflicts:
	tools.py