Tiny News

For a detailed overview of this project from start to finish, check out GregSamek.github.io/TinyNews

TinyNews is a collection of one million synthetically generated news bulletins and several language models scratch-trained on this data. Evaluations suggests that TinyNews retains ~80% of the quality of the training data while using ~1/1000th the number of parameters as the models used to generate it.

To run these models, git clone the repository

Trained models and training data are available in this 🤗 Hugging Face Collection

This project is essentially a modified reimplementation of the Microsoft Research TinyStories project.

Downloads last month
51
Safetensors
Model size
2.91M params
Tensor type
F32
·
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no pipeline_tag.

Dataset used to train GregSamek/TinyNews-3M

Collection including GregSamek/TinyNews-3M