GitHub repository: Kikobeats/top-crawler-agents
Primary language: JavaScript
Description: A list of common crawler user agents useful for retrieving metadata from links.