Added error checking but there is a bug in crawl method of Bot