|
--- |
|
datasets: |
|
- lemonilia/LimaRP |
|
language: |
|
- en |
|
tags: |
|
- llama |
|
- llama 2 |
|
- smol_llama |
|
--- |
|
# smol_llama-220M-GQA-32k-theta-sft-limarp |
|
|
|
Experimental model meant to serve as a long-context speculative decoding model. This one is specifically for models trained on the LimaRP prompt format. |
|
|
|
Created using [Doctor-Shotgun/smol_llama-220M-GQA-32k-theta-sft](https://huggingface.co/Doctor-Shotgun/smol_llama-220M-GQA-32k-theta-sft) and finetuning at 32768 context length on the LimaRP dataset. |
|
|
|
This variant uses the rope theta (rope frequency base) method for context extension. |
|
|
|
The trained instruction format is LimaRP Alpaca: |
|
``` |
|
### Instruction: |
|
Character's Persona: {bot character description} |
|
|
|
User's Persona: {user character description} |
|
|
|
Scenario: {what happens in the story} |
|
|
|
Play the role of Character. Taking the above information into consideration, you must engage in a roleplaying chat with User below this line. Do not write dialogues and narration for User. |
|
|
|
### Input: |
|
User: {utterance} |
|
|
|
### Response: |
|
Character: {utterance} |
|
|
|
### Input |
|
User: {utterance} |
|
|
|
### Response: |
|
Character: {utterance} |
|
|
|
(etc.) |
|
``` |
