ttagu99 commited on
Commit
8829a22
1 Parent(s): adcaca4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +34 -0
README.md CHANGED
@@ -20,6 +20,36 @@ based mistral, instruction tuned and dpo.
20
 
21
  merge mncai/mistral-7b-dpo-v6, rwitz2/go-bruins-v2.1.1, ignos/LeoScorpius-GreenNode-Alpaca-7B-v1.
22
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
 
24
  ### How to Use
25
  Here give some examples of how to use our model.
@@ -43,6 +73,10 @@ for seq in sequences:
43
  print(f"Result: {seq['generated_text']}")
44
  ```
45
 
 
 
 
 
46
 
47
  ### Contact
48
  If you have any questions, please raise an issue or contact us at dwmyoung@mnc.ai
 
20
 
21
  merge mncai/mistral-7b-dpo-v6, rwitz2/go-bruins-v2.1.1, ignos/LeoScorpius-GreenNode-Alpaca-7B-v1.
22
 
23
+ ### Details
24
+
25
+ ties
26
+
27
+ ```
28
+ models:
29
+ - model: rwitz2/go-bruins-v2.1.1
30
+ # no parameters necessary for base model
31
+ - model: janai-hq/trinity-v1 # psmathur/orca_mini_v3_13b
32
+ parameters:
33
+ density: [1, 0.7, 0.1] # density gradient
34
+ weight: 1.0
35
+ - model: ignos/LeoScorpius-GreenNode-Alpaca-7B-v1
36
+ parameters:
37
+ density: 0.5
38
+ weight: [0, 0.3, 0.7, 1] # weight gradient
39
+ - model: mncai/mistral-7b-dpo-v6
40
+ parameters:
41
+ density: 0.33
42
+ weight:
43
+ - filter: mlp
44
+ value: 0.5
45
+ - value: 0
46
+ merge_method: ties
47
+ base_model: rwitz2/go-bruins-v2.1.1
48
+ parameters:
49
+ normalize: true
50
+ int8_mask: true
51
+ dtype: float16
52
+ ```
53
 
54
  ### How to Use
55
  Here give some examples of how to use our model.
 
73
  print(f"Result: {seq['generated_text']}")
74
  ```
75
 
76
+ ### Warnings
77
+ Currently, the leaderboard is overfitted. It is inevitable because, unlike Kaggle, where there's private scoring followed by the end of the competition, here the scores are continuously open.
78
+ Even among my models, some received lower scores in internal data evaluations. mncai/agiin-13.6B-v0.1 > mncai/agiin-11.1B-v0.1 > mncai/mistral-7b-dpo-v6. However, on the leaderboard, mncai/mistral-7b-dpo-v6 has the highest score.
79
+ When choosing a model to use on the open LLM leaderboard, it would be best to evaluate with your own private dataset that is not publicly available.
80
 
81
  ### Contact
82
  If you have any questions, please raise an issue or contact us at dwmyoung@mnc.ai