github.com/dinofizz/sitemapper (go): sitemapper is a site mapping tool which provides a JSON output listing each internal URL and the internal links found at that URL. The crawl depth is configurable, as well as the mode of operation: "synchronous", "concurrent" and "concurrent limited". The tool runs stand-alone or as a distributed crawl engine running in a Kubernetes cluster.
Keywords: astradb|cassandra|concurrency|crawler|go|golang|kubernetes|nats|sitemap