ewof commited on
Commit
5b94ad2
1 Parent(s): d8cbff2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -1
README.md CHANGED
@@ -3,6 +3,7 @@ datasets:
3
  - ewof/koishi-instruct-metharme
4
  ---
5
 
 
6
 
7
  [axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) was used for training
8
  on a 4x nvidia a40 gpu cluster.
@@ -10,4 +11,11 @@ on a 4x nvidia a40 gpu cluster.
10
  the a40 GPU cluster has been graciously provided by [Arc Compute](https://www.arccompute.io/).
11
  rank 8 lora merge
12
 
13
- tune of mistral 7b
 
 
 
 
 
 
 
 
3
  - ewof/koishi-instruct-metharme
4
  ---
5
 
6
+ ## Training
7
 
8
  [axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) was used for training
9
  on a 4x nvidia a40 gpu cluster.
 
11
  the a40 GPU cluster has been graciously provided by [Arc Compute](https://www.arccompute.io/).
12
  rank 8 lora merge
13
 
14
+ ## Base Model
15
+
16
+ native fine tune of mistralai/Mistral-7B-v0.1
17
+ ## Prompting
18
+
19
+ The current model version has been trained on prompts using three different roles, which are denoted by the following tokens: `<|system|>`, `<|user|>` and `<|model|>`.
20
+
21
+ The `<|system|>` prompt can be used to inject out-of-channel information behind the scenes, while the `<|user|>` prompt should be used to indicate user input. The `<|model|>` token should then be used to indicate that the model should generate a response. These tokens can happen multiple times and be chained up to form a conversation history.