GitHub repository: marianna13/pile_tokenizer
Created: 2022-06-14T15:17:13Z
Description: Downloads, extracts and tokenizes pile (https://the-eye.eu/public/AI/pile) data.