GitHub repository: CitizensFoundation/pace-commoncrawl-scanner
Primary language: JavaScript
Description: Scans CommonCrawl datasets for keywords. Scans the whole month of CommonCrawl data using Amazon EC2 c5.18xlarge instance for hundreds of keywords in about 3 hours. Developed with support from the EU and the Populism & Civic Engagement H2020 project.