Update README.md
Browse files
README.md
CHANGED
@@ -1,7 +1,7 @@
|
|
1 |
---
|
2 |
license: llama2
|
3 |
---
|
4 |
-
Experiment for DARE(Drop and REscale),
|
5 |
|
6 |
|
7 |
| Model | Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K | DROP |
|
|
|
1 |
---
|
2 |
license: llama2
|
3 |
---
|
4 |
+
Experiment for DARE(Drop and REscale), most of the delta parameters can be directly set to zeros without affecting the capabilities of SFT LMs and larger models can tolerate a higher proportion of discarded parameters.
|
5 |
|
6 |
|
7 |
| Model | Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K | DROP |
|