Commit
•
d7aada5
1
Parent(s):
b316047
changed model name
Browse files
README.md
CHANGED
@@ -12,7 +12,7 @@ library_name: transformers
|
|
12 |
---
|
13 |
|
14 |
|
15 |
-
# Yi RMU
|
16 |
|
17 |
Yi 34B Chat model with hazardous knowledge about biosecurity and cybersecurity "unlearned" using Representation Misdirection for Unlearning (RMU). For more details, please check [our paper](https://arxiv.org/abs/2403.03218).
|
18 |
|
@@ -24,12 +24,12 @@ Yi 34B Chat model with hazardous knowledge about biosecurity and cybersecurity "
|
|
24 |
|
25 |
## Performance
|
26 |
|
27 |
-
Yi RMU has been evaluated on [WMDP](https://huggingface.co/datasets/cais/wmdp), [MMLU](https://huggingface.co/datasets/cais/mmlu) and [MT-Bench](https://huggingface.co/spaces/lmsys/mt-bench). Higher accuracy on MMLU and MT-Bench, and lower accuracy on WMDP are preferred.
|
28 |
|
29 |
-
|
|
30 |
-
|
31 |
-
| Yi
|
32 |
-
|
|
33 |
|
34 |
|
35 |
|
|
|
12 |
---
|
13 |
|
14 |
|
15 |
+
# Yi 34B Chat RMU
|
16 |
|
17 |
Yi 34B Chat model with hazardous knowledge about biosecurity and cybersecurity "unlearned" using Representation Misdirection for Unlearning (RMU). For more details, please check [our paper](https://arxiv.org/abs/2403.03218).
|
18 |
|
|
|
24 |
|
25 |
## Performance
|
26 |
|
27 |
+
Yi 34B Chat RMU has been evaluated on [WMDP](https://huggingface.co/datasets/cais/wmdp), [MMLU](https://huggingface.co/datasets/cais/mmlu) and [MT-Bench](https://huggingface.co/spaces/lmsys/mt-bench). Higher accuracy on MMLU and MT-Bench, and lower accuracy on WMDP are preferred.
|
28 |
|
29 |
+
| | WMDP-Bio | WMDP-Cyber | MMLU | MT-Bench |
|
30 |
+
|-----------------|:---------:|:----------:|:------:|:--------:|
|
31 |
+
| Yi 34B Chat | 75.3 | 49.7 | 72.6 | 7.65 |
|
32 |
+
| Yi 34B Chat RMU | 30.7 | 29.0 | 70.6 | 7.59 |
|
33 |
|
34 |
|
35 |
|