---
datasets:
- EleutherAI/pile
language:
- en
---
# Model Card

This model is an Attention (Llama architecture) model pretrained on 30Bn tokens of the Pile corpus.


### Model Sources

The model implementation and training code that produced the model are provided here: https://github.com/HazyResearch/based