| license: mit | |
| datasets: | |
| - bookcorpus/bookcorpus | |
| language: | |
| - en | |
| library_name: transformers | |
| * The GPT -2 model was trained on the BookCorpus dataset for 60K steps. | |
| * No position embedding was used (NoPE). | |
| * [Here](https://wandb.ai/a-arun283-iit-madras/gpt-2-BooKcorpus-WarmUpLr/reports/Pretraining-GPT-2---Vmlldzo5MDY3MDk5) is the wandb report | |
| * This is for educational purposes only. |