Update README.md
Browse files
README.md
CHANGED
@@ -3,6 +3,7 @@ datasets:
|
|
3 |
- ewof/koishi-instruct-metharme
|
4 |
---
|
5 |
|
|
|
6 |
|
7 |
[axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) was used for training
|
8 |
on a 4x nvidia a40 gpu cluster.
|
@@ -10,4 +11,11 @@ on a 4x nvidia a40 gpu cluster.
|
|
10 |
the a40 GPU cluster has been graciously provided by [Arc Compute](https://www.arccompute.io/).
|
11 |
rank 8 lora merge
|
12 |
|
13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
- ewof/koishi-instruct-metharme
|
4 |
---
|
5 |
|
6 |
+
## Training
|
7 |
|
8 |
[axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) was used for training
|
9 |
on a 4x nvidia a40 gpu cluster.
|
|
|
11 |
the a40 GPU cluster has been graciously provided by [Arc Compute](https://www.arccompute.io/).
|
12 |
rank 8 lora merge
|
13 |
|
14 |
+
## Base Model
|
15 |
+
|
16 |
+
native fine tune of mistralai/Mistral-7B-v0.1
|
17 |
+
## Prompting
|
18 |
+
|
19 |
+
The current model version has been trained on prompts using three different roles, which are denoted by the following tokens: `<|system|>`, `<|user|>` and `<|model|>`.
|
20 |
+
|
21 |
+
The `<|system|>` prompt can be used to inject out-of-channel information behind the scenes, while the `<|user|>` prompt should be used to indicate user input. The `<|model|>` token should then be used to indicate that the model should generate a response. These tokens can happen multiple times and be chained up to form a conversation history.
|