This web crawler will give you all the links present on any website, it will also crawl through all those links and will give you all the links present on all those crawled links