GitHub repository: spire-rs/kit
Primary language: Rust
Description: 📚 Libraries for retrieving and parsing of robots.txt & sitemaps.