dr4g0nsr/sitemap-crawler (packagist): Crawler for any type of site using robots.txt and sitemap.xml as the source of URL. Useful for cache regenerating.
Keywords: crawler|php