File size: 949 Bytes
6e46e7d 8dfcf0f 6e46e7d 3301852 54fbe61 6e46e7d 2ba61d6 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
---
license: apache-2.0
inference: false
---
**NOTE: This "delta model" cannot be used directly.**
Users have to apply it on top of the original LLaMA weights.
See https://github.com/lm-sys/FastChat#vicuna-weights for instructions.
<br>
<br>
# scifi-fantasy-author Model Card
`scifi-fantasy-author` is a finetuned LLaMA-7B model to generate narrative fiction,
paricularly in the Science Fiction and Fantasy genres.
The following hyperparameters were used
|Batch Size|Epochs|Context length|Learning rate|Scheduler|Weight decay|Warmup ratio|
|---------:|-----:|-------------:|------------:|--------:|-----------:|-----------:|
| 128 | 3 | 8192 | 2e-5 | Cosine | 0. | 0.03 |
The model reached a training loss of 2.008 and took approximately 8 hours on 8x A100 80 GB GPUs.
The specific training script can be found [here](https://github.com/hooloovoo-ai/cyoa-backend/blob/master/backend/scripts/train.py). |