File size: 595 Bytes
4439967 028ef7e 4439967 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
---
language:
- eng
pipeline_tag: text-generation
Trained: Pretrain
Config file: 1.3B
Data: English News Dataset 2GB
---
# Model Card for Model ID
<!-- Provide a quick summary of what the model is/does. -->
Pretrained GPT-NeoX model with 2.06GB English news dataset. Took about 10 hours to reach 20,000 iterations. Trained on p3.16xlarge.
Different hyperparameter: gradient_accumulation_step 4
## Model Details
### Model Description
<!-- Provide a longer summary of what this model is. -->
- **Developed by:** Eunyoung Lee
- **Model type:** GPT-NeoX
- **Language(s) (NLP):** English
|