Crawler modified to store graph links and key(md5) value(url)
Version control files
Handled exceptions
Be Polite with randomness