Key papers on small LLMs, data-centric training, and multilingual NLP.
-
The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
Paper • 2406.17557 • Published • 105 -
SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
Paper • 2502.02737 • Published • 261 -
The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text
Paper • 2506.05209 • Published • 63
