Omkar2118/datasets huggingface dataset:
Expository Prose
Full name: Expository-Prose-V1

Expository-Prose-V1 is a collection of specially-curated corpora gathered from diverse sources, ranging from research papers (arXiv) to European Parliament proceedings (EuroParl). It has been specially filtered and curated for the quality of text, depth of reasoning and breadth of knowledge to faciliate an effective pre-train. It was used to pre-train 1.5-Pints, a small but powerful Large Language Model developed by the Pints Research Team.

Introducing paper: 1.5-Pints Technical Report: Pretraining in Days, Not Months -- Your Language Model Thrives on Quality Data