simple-site-crawler (pypi): Simple website crawler that asynchronously crawls a website and all subpages that it can find, along with static content that they rely on.
Keywords: website|crawler|sitemap