R136a1 commited on
Commit
8b85303
1 Parent(s): 31238b2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -19
README.md CHANGED
@@ -6,24 +6,14 @@ tags:
6
  - safetensors
7
  - mixtral
8
  ---
9
- Test model.
10
 
11
- Under testing...
12
 
13
- Recipe:
14
- ```yaml
15
- base_model: /content/InfinityRP
16
- gate_mode: random
17
- dtype: bfloat16 # output dtype (float32, float16, or bfloat16)
18
- ## (optional)
19
- experts_per_token: 2
20
- experts:
21
- - source_model: /content/WestLake
22
- positive_prompts: []
23
- - source_model: /content/Kuno
24
- positive_prompts: []
25
- - source_model: /content/InfinityRP
26
- positive_prompts: []
27
- - source_model: /content/LemonadeRP
28
- positive_prompts: []
29
- ```
 
6
  - safetensors
7
  - mixtral
8
  ---
 
9
 
 
10
 
11
+ Testing done.
12
+
13
+ It performs really well in complex scenario and follows the character card quite well. The char card and previous message can affect a lot to the next reply style.
14
+
15
+ The main idea is instead of _merging_ models to create new model, I try to put these best model into mixtral so it can work together. And the result is good, every model has its uniqueness and strength.
16
+
17
+ Downside? it only support 8k (8192) context length...
18
+
19
+ Alpaca prompting format.