Update README.md
Browse files
README.md
CHANGED
@@ -128,6 +128,6 @@ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-le
|
|
128 |
|
129 |
### Results
|
130 |
- small quality loss can be observed comparing to base model, as described in the DUS paper
|
131 |
-
- this merge has best evaluation results,
|
132 |
- finetunig will be done on 5-10% of openorca dataset and full DPO datasets used by SOLAR
|
133 |
- v03 > v01 > v02 - based on average evaluation scores, removing 1/4 of total layers seems to be the correct way to scale DUS
|
|
|
128 |
|
129 |
### Results
|
130 |
- small quality loss can be observed comparing to base model, as described in the DUS paper
|
131 |
+
- this merge has best evaluation results, so it will be finetuned to 'recover' from the merge
|
132 |
- finetunig will be done on 5-10% of openorca dataset and full DPO datasets used by SOLAR
|
133 |
- v03 > v01 > v02 - based on average evaluation scores, removing 1/4 of total layers seems to be the correct way to scale DUS
|