|
--- |
|
language: |
|
- en |
|
pipeline_tag: text-generation |
|
datasets: |
|
- practicaldreamer/RPGPT_PublicDomain-alpaca |
|
--- |
|
|
|
# Latest Version: *10,000* Steps |
|
- 2,560,000 tokens seen. |
|
|
|
|
|
# Model Info: |
|
- Trained from scratch. |
|
- 8.5M parameters. |
|
- 256 context length. |
|
- Test model. Likely needs at least 512 context to function "properly". |
|
- Trained with a dataset that overlaps by a quarter of the context length (Shifts by 64 tokens for each subset). |
|
|
|
# Format: |
|
``` |
|
<|characters|> |
|
Nancy (Oliver Twist): Female, early 20s, ESFP, Cockney accent. Loyal... |
|
Mr. Edward Hyde (Dr. Jekyll and Mr. Hyde): Male, late 30s, ESTP... |
|
|
|
<|scenario|> |
|
In an alternate Victorian London where the city's poor and downtrodden... |
|
|
|
<|response|> |
|
Nancy: *gently brushes her fingers across the worn book spine, before suddenly stopping as she feels another hand... |
|
Mr. Edward Hyde: *glances at Nancy with a sinister grin, slowly pulling his hand back* No need to apologize, miss... |
|
``` |
|
|
|
# Example Output: |
|
Step 10,000. |
|
``` |
|
<|characters|> |
|
The Intended (The Iliad): Male, late 20s, ENFJ, British accent. Intelligent, empathetic, seeking the same challenge by Jane Austen's challenges. |
|
|
|
<|scenario|> |
|
In a strange sense of a mysterious time, The Intended finds himself in the same place. The Red and the |
|
``` |
|
|
|
# Config: |
|
``` |
|
batch_size: 1 |
|
dropout: 0 |
|
learning_rate: 0.0001 |
|
max_length: 256 |
|
n_embed: 256 |
|
n_head: 8 |
|
n_layer: 8 |
|
vocab_size: 8192 |
|
``` |
|
|
