add links to visited in crawl links