Update README.md
Browse files
README.md
CHANGED
@@ -25,6 +25,14 @@ It achieves the following results on the evaluation set:
|
|
25 |
- Logits/rejected: -2.5535
|
26 |
- Logits/chosen: -2.7973
|
27 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
28 |
## Model description
|
29 |
|
30 |
**It seems to outperforms the original Zephyr in most of the tasks.**
|
@@ -46,6 +54,15 @@ Research purposes.
|
|
46 |
alignment-handbook DPO with UNA on top of the SFT lora.
|
47 |
|
48 |
### Evaluation lm-evaluation-harness
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
49 |
#### 0-Shot
|
50 |
```
|
51 |
hf (pretrained=fblgit/juanako-7b-v1,load_in_4bit=False,dtype=float16), limit: None, num_fewshot: 0, batch_size: 8
|
|
|
25 |
- Logits/rejected: -2.5535
|
26 |
- Logits/chosen: -2.7973
|
27 |
|
28 |
+
|
29 |
+
```
|
30 |
+
hf (pretrained=fblgit/juanako-7b-v1,load_in_4bit=False,dtype=float16), limit: None, num_fewshot: 3, batch_size: 4
|
31 |
+
```
|
32 |
+
|Tasks|Version| Filter | Metric |Value | |Stderr|
|
33 |
+
|-----|-------|----------|-----------|-----:|---|-----:|
|
34 |
+
|gsm8k|Yaml |get-answer|exact_match|0.4556|± |0.0137|
|
35 |
+
|
36 |
## Model description
|
37 |
|
38 |
**It seems to outperforms the original Zephyr in most of the tasks.**
|
|
|
54 |
alignment-handbook DPO with UNA on top of the SFT lora.
|
55 |
|
56 |
### Evaluation lm-evaluation-harness
|
57 |
+
|
58 |
+
#### GSM8K
|
59 |
+
```
|
60 |
+
hf (pretrained=/root/juanako-7b-v1-beta,load_in_4bit=False,dtype=float16), limit: None, num_fewshot: 3, batch_size: 4
|
61 |
+
```
|
62 |
+
|Tasks|Version| Filter | Metric |Value | |Stderr|
|
63 |
+
|-----|-------|----------|-----------|-----:|---|-----:|
|
64 |
+
|gsm8k|Yaml |get-answer|exact_match|0.4556|± |0.0137|
|
65 |
+
|
66 |
#### 0-Shot
|
67 |
```
|
68 |
hf (pretrained=fblgit/juanako-7b-v1,load_in_4bit=False,dtype=float16), limit: None, num_fewshot: 0, batch_size: 8
|