YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
Training Details:
- 200,000 steps on the Pile
- Denoising (span corruption) objective
- Batch size 256, 512 sequence length
- 25B tokens total
- Modified GPT NeoX Tokenizer
- Downloads last month
- 3