Pythia Scaling Suite

EleutherAI 's Collections

Quirky Models and Datasets

Pile-T5

Quirky Math (archive)

Pile-T5 T5x Checkpoints

updated Nov 21, 2023

Pythia is the first LLM suite designed specifically to enable scientific research on LLMs. To learn more see https://github.com/EleutherAI/pythia

Upvote

Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling

Paper • 2304.01373 • Published Apr 3, 2023 • 7

Note The paper introduced the Pythia model suite
EleutherAI/pythia-14m

Text Generation • Updated Jul 26, 2023 • 57.4k • 11

Note This model was trained after the paper was released.
EleutherAI/pythia-70m

Updated Nov 21, 2023 • 71.6k • 40

Note This model was trained after the paper was released.
EleutherAI/pythia-160m

Text Generation • Updated Jul 9, 2023 • 58.7k • 19
EleutherAI/pythia-410m

Text Generation • Updated Jul 9, 2023 • 30.6k • 17
EleutherAI/pythia-1b

Text Generation • Updated Jul 9, 2023 • 21.8k • 32
EleutherAI/pythia-1.4b

Text Generation • Updated Jul 9, 2023 • 33.8k • 21
EleutherAI/pythia-2.8b

Text Generation • Updated Jun 9, 2023 • 37.7k • 21
EleutherAI/pythia-6.9b

Text Generation • Updated Jun 8, 2023 • 34k • 38
EleutherAI/pythia-12b

Text Generation • Updated Jun 8, 2023 • 42.6k • 128
EleutherAI/pythia-70m-deduped

Text Generation • Updated Jul 9, 2023 • 198k • 22
EleutherAI/pythia-160m-deduped

Text Generation • Updated Jul 9, 2023 • 76.3k • 2
EleutherAI/pythia-410m-deduped

Text Generation • Updated Jul 9, 2023 • 69k • 21
EleutherAI/pythia-1b-deduped

Text Generation • Updated Jul 10, 2023 • 40.3k • 15
EleutherAI/pythia-1.4b-deduped

Text Generation • Updated Jun 8, 2023 • 16.5k • 19
EleutherAI/pythia-2.8b-deduped

Text Generation • Updated Jul 9, 2023 • 16k • 13
EleutherAI/pythia-6.9b-deduped

Text Generation • Updated Jun 8, 2023 • 13.5k • 7
EleutherAI/pythia-12b-deduped

Text Generation • Updated Jun 8, 2023 • 11.3k • 50

Upvote