finished the web crawler

added the code that follows the links and then adds those links to the
crawlers list. until no more links are left