---
language:
- eng
pipeline_tag: text-generation

Trained: Pretrain
Config file: 1.3B
Data: English News Dataset 2GB
---
# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

Pretrained GPT-NeoX model with 2.06GB English news dataset. Took about 10 hours to reach 20,000 iterations. Trained on p3.16xlarge.  
Different hyperparameter: gradient_accumulation_step 4 

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->



- **Developed by:** Eunyoung Lee
- **Model type:** GPT-NeoX
- **Language(s) (NLP):** English