GitHub repository: tianjianjiang/bigscience-metadata
Primary language: Python
Description: Experiments on including metadata such as URLs, timestamps, website descriptions and HTML tags during pretraining.