TeeZee commited on
Commit
19252b2
1 Parent(s): 6121f69

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -128,5 +128,6 @@ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-le
128
 
129
  ### Results
130
  - small quality loss can be observed comparing to base model, as described in the DUS paper
131
- - this merge has best evaluation results, os it will be finetuned to 'recover' from the merge
 
132
  - v03 > v01 > v02 - based on average evaluation scores, removing 1/4 of total layers seems to be the correct way to scale DUS
 
128
 
129
  ### Results
130
  - small quality loss can be observed comparing to base model, as described in the DUS paper
131
+ - this merge has best evaluation results, ss it will be finetuned to 'recover' from the merge
132
+ - finetunig will be done on 5-10% of openorca dataset and full DPO datasets used by SOLAR
133
  - v03 > v01 > v02 - based on average evaluation scores, removing 1/4 of total layers seems to be the correct way to scale DUS