Update README.md
Browse files
README.md
CHANGED
@@ -6,14 +6,16 @@ license: apache-2.0
|
|
6 |
|
7 |
This is an experimental version of LimaRP using a somewhat updated dataset (1800 training samples)
|
8 |
and a 2-pass training procedure. The first pass includes unsupervised tuning on 2800 stories within
|
9 |
-
4k tokens and the second is LimaRP.
|
10 |
|
11 |
For more details about LimaRP, see the model page for the [previously released version](https://huggingface.co/lemonilia/limarp-llama2-v2).
|
12 |
Most details written there apply for this version as well.
|
13 |
|
14 |
-
## Prompt
|
15 |
-
Same as before. It uses Alpaca format,
|
16 |
-
immediately preceding
|
|
|
|
|
17 |
|
18 |
```
|
19 |
### Instruction:
|
@@ -30,12 +32,21 @@ User: {utterance}
|
|
30 |
|
31 |
### Response:
|
32 |
Character: {utterance}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
33 |
```
|
34 |
|
35 |
### Other notes
|
36 |
- Replace all the text in curly braces (curly braces included) with your own text.
|
37 |
- `User` and `Character` should be replaced with appropriate names.
|
38 |
|
|
|
39 |
## Training Hyperparameters
|
40 |
[Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) was used for training.
|
41 |
The model has been trained as a 4-bit LoRA adapter. It's so large because a LoRA rank
|
|
|
6 |
|
7 |
This is an experimental version of LimaRP using a somewhat updated dataset (1800 training samples)
|
8 |
and a 2-pass training procedure. The first pass includes unsupervised tuning on 2800 stories within
|
9 |
+
4k tokens length and the second is LimaRP.
|
10 |
|
11 |
For more details about LimaRP, see the model page for the [previously released version](https://huggingface.co/lemonilia/limarp-llama2-v2).
|
12 |
Most details written there apply for this version as well.
|
13 |
|
14 |
+
## Prompt format
|
15 |
+
Same as before. It uses the [extended Alpaca format](https://github.com/tatsu-lab/stanford_alpaca),
|
16 |
+
with `### Input:` immediately preceding user inputs and `### Response:` immediately preceding
|
17 |
+
model outputs. While Alpaca wasn't originally intended for multi-turn responses, in practice this
|
18 |
+
is not a problem; the format follows a pattern already used by other models.
|
19 |
|
20 |
```
|
21 |
### Instruction:
|
|
|
32 |
|
33 |
### Response:
|
34 |
Character: {utterance}
|
35 |
+
|
36 |
+
### Input
|
37 |
+
User: {utterance}
|
38 |
+
|
39 |
+
### Response:
|
40 |
+
Character: {utterance}
|
41 |
+
|
42 |
+
(etc.)
|
43 |
```
|
44 |
|
45 |
### Other notes
|
46 |
- Replace all the text in curly braces (curly braces included) with your own text.
|
47 |
- `User` and `Character` should be replaced with appropriate names.
|
48 |
|
49 |
+
|
50 |
## Training Hyperparameters
|
51 |
[Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) was used for training.
|
52 |
The model has been trained as a 4-bit LoRA adapter. It's so large because a LoRA rank
|