| datasets: | |
| - xzuyn/Stable-Diffusion-Prompts-Deduped-2.008M | |
| language: | |
| - en | |
| pipeline_tag: text-generation | |
| # Latest Version: *130,000* / 97,106,275 Steps | |
| - 8,320,000 tokens seen. | |
| # Model Info: | |
| Test **aitextgen** GPT-2 Model. Trained from scratch. | |
| - 6.9M parameters. | |
| - 64 context length. | |
| # Config | |
| ``` | |
| batch_size: 1 | |
| dropout: 0 | |
| learning_rate: 0.0001 | |
| max_length: 64 | |
| n_embed: 256 | |
| n_head: 8 | |
| n_layer: 8 | |
| vocab_size: 2,048 | |
| ``` |