enkhtogtokh/mistral-dpo
Browse files
README.md
CHANGED
@@ -17,15 +17,15 @@ should probably proofread and complete it, then remove this comment. -->
|
|
17 |
|
18 |
This model is a fine-tuned version of [TheBloke/OpenHermes-2-Mistral-7B-GPTQ](https://huggingface.co/TheBloke/OpenHermes-2-Mistral-7B-GPTQ) on the None dataset.
|
19 |
It achieves the following results on the evaluation set:
|
20 |
-
- Loss: 0.
|
21 |
-
- Rewards/chosen:
|
22 |
-
- Rewards/rejected: -6.
|
23 |
- Rewards/accuracies: 1.0
|
24 |
-
- Rewards/margins:
|
25 |
-
- Logps/rejected: -
|
26 |
-
- Logps/chosen: -
|
27 |
-
- Logits/rejected: -1.
|
28 |
-
- Logits/chosen: -1.
|
29 |
|
30 |
## Model description
|
31 |
|
@@ -58,11 +58,11 @@ The following hyperparameters were used during training:
|
|
58 |
|
59 |
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|
60 |
|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
|
61 |
-
| 0.
|
62 |
-
| 0.
|
63 |
-
| 0.
|
64 |
-
| 0.
|
65 |
-
| 0.
|
66 |
|
67 |
|
68 |
### Framework versions
|
|
|
17 |
|
18 |
This model is a fine-tuned version of [TheBloke/OpenHermes-2-Mistral-7B-GPTQ](https://huggingface.co/TheBloke/OpenHermes-2-Mistral-7B-GPTQ) on the None dataset.
|
19 |
It achieves the following results on the evaluation set:
|
20 |
+
- Loss: 0.0012
|
21 |
+
- Rewards/chosen: 23.3318
|
22 |
+
- Rewards/rejected: -6.7489
|
23 |
- Rewards/accuracies: 1.0
|
24 |
+
- Rewards/margins: 30.0806
|
25 |
+
- Logps/rejected: -87.3513
|
26 |
+
- Logps/chosen: -337.4500
|
27 |
+
- Logits/rejected: -1.2781
|
28 |
+
- Logits/chosen: -1.6769
|
29 |
|
30 |
## Model description
|
31 |
|
|
|
58 |
|
59 |
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|
60 |
|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
|
61 |
+
| 0.5937 | 0.01 | 10 | 0.3601 | 0.7833 | -0.3607 | 0.9904 | 1.1441 | -23.4698 | -562.9344 | -1.1371 | -1.4401 |
|
62 |
+
| 0.0908 | 0.02 | 20 | 1.1245 | 8.8420 | -2.7352 | 0.9615 | 11.5772 | -47.2141 | -482.3473 | -1.1942 | -1.5504 |
|
63 |
+
| 0.0683 | 0.03 | 30 | 0.2541 | 17.6490 | -4.7403 | 0.9904 | 22.3893 | -67.2654 | -394.2778 | -1.2341 | -1.6426 |
|
64 |
+
| 0.0009 | 0.04 | 40 | 0.0015 | 22.5664 | -5.9863 | 1.0 | 28.5527 | -79.7251 | -345.1035 | -1.2763 | -1.6781 |
|
65 |
+
| 0.0003 | 0.05 | 50 | 0.0012 | 23.3318 | -6.7489 | 1.0 | 30.0806 | -87.3513 | -337.4500 | -1.2781 | -1.6769 |
|
66 |
|
67 |
|
68 |
### Framework versions
|
adapter_config.json
CHANGED
@@ -19,8 +19,8 @@
|
|
19 |
"rank_pattern": {},
|
20 |
"revision": null,
|
21 |
"target_modules": [
|
22 |
-
"
|
23 |
-
"
|
24 |
],
|
25 |
"task_type": "CAUSAL_LM"
|
26 |
}
|
|
|
19 |
"rank_pattern": {},
|
20 |
"revision": null,
|
21 |
"target_modules": [
|
22 |
+
"q_proj",
|
23 |
+
"v_proj"
|
24 |
],
|
25 |
"task_type": "CAUSAL_LM"
|
26 |
}
|
adapter_model.safetensors
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 6832600
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:f80f99035c681c845263561663e67f34c2d083918f74e36fab1f698934cf052c
|
3 |
size 6832600
|
runs/Jan16_06-26-11_c6a9bcfdc196/events.out.tfevents.1705386478.c6a9bcfdc196.594.0
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:2c4ca71aaa1cf21c95c01c219082e5654ba85094a7d9b554c8a1c73bb881a789
|
3 |
+
size 12559
|
training_args.bin
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 4091
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:63b7be98043f6e4899e95c30b2a63ccbdfb2a8332e1daa417547649d18aa0c8d
|
3 |
size 4091
|