Added spider to crawl links, not working as intended. Not checking for visited url.