---
library_name: transformers
tags: []
---



## Model Details

- **Developed by:** Rafael Espinosa Mena
- **Model type:** GPT2 124M
- **Language(s) (NLP):** English

## Uses

Pre-Trained from scratch on 200 wikipedia articles and intended for fine tunning.


## Training Details

Trained for 200 epochs on 200 wikipedia articles. Used a learning rate of 3e-5, and a sliding window approach with 1024 tokens per chunk, and a 200 token window.

### Training Data

https://huggingface.co/datasets/wikipedia

#### Hardware

Trained on a single V100 GPU

## Model Card Authors

Rafael Espinosa Mena rafaelespinosamena@gmail.com