Model save

README.md

@@ -14,6 +14,16 @@ should probably proofread and complete it, then remove this comment. -->
 # zephyr-7b-dpo-full
 
 This model was trained from scratch on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0216
+- Rewards/chosen: -2.0336
+- Rewards/rejected: -3.3217
+- Rewards/accuracies: 0.7227
+- Rewards/margins: 1.2881
+- Logps/rejected: -643.7943
+- Logps/chosen: -506.9436
+- Logits/rejected: 0.6159
+- Logits/chosen: 0.5330
 
 ## Model description
 
@@ -33,12 +43,12 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 1e-06
-- train_batch_size: 2
+- train_batch_size: 4
 - eval_batch_size: 8
-- seed: 4
+- seed: 5
 - distributed_type: multi-GPU
 - num_devices: 8
-- gradient_accumulation_steps: 8
+- gradient_accumulation_steps: 4
 - total_train_batch_size: 128
 - total_eval_batch_size: 64
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
@@ -48,6 +58,17 @@ The following hyperparameters were used during training:
 
 ### Training results
 
+| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
+|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
+| 0.1161        | 0.21  | 100  | 0.1131          | -0.3418        | -0.6647          | 0.6836             | 0.3229          | -378.0938      | -337.7654    | 0.4986          | 0.4461        |
+| 0.0696        | 0.42  | 200  | 0.0678          | -0.7178        | -1.3697          | 0.7109             | 0.6519          | -448.5950      | -375.3677    | 0.3246          | 0.2679        |
+| 0.0546        | 0.63  | 300  | 0.0566          | -0.8496        | -1.6574          | 0.7383             | 0.8078          | -477.3692      | -388.5492    | 0.3538          | 0.2934        |
+| 0.0503        | 0.84  | 400  | 0.0567          | -0.8246        | -1.6719          | 0.7188             | 0.8473          | -478.8125      | -386.0466    | 0.3896          | 0.3333        |
+| 0.0189        | 1.05  | 500  | 0.0289          | -1.9029        | -3.2435          | 0.7344             | 1.3406          | -635.9754      | -493.8704    | 0.4162          | 0.3568        |
+| 0.0121        | 1.26  | 600  | 0.0237          | -1.9630        | -3.2842          | 0.7266             | 1.3212          | -640.0466      | -499.8828    | 0.5605          | 0.4812        |
+| 0.0113        | 1.47  | 700  | 0.0215          | -2.0722        | -3.4058          | 0.7266             | 1.3336          | -652.2046      | -510.8033    | 0.5884          | 0.5062        |
+| 0.0119        | 1.67  | 800  | 0.0225          | -1.9782        | -3.2475          | 0.7188             | 1.2693          | -636.3755      | -501.4035    | 0.5996          | 0.5166        |
+| 0.0103        | 1.88  | 900  | 0.0216          | -2.0336        | -3.3217          | 0.7227             | 1.2881          | -643.7943      | -506.9436    | 0.6159          | 0.5330        |
 
 
 ### Framework versions

all_results.json

@@ -1,8 +1,8 @@
 {
     "epoch": 2.0,
-    "train_loss": 0.11496639693108927,
-    "train_runtime": 23193.7036,
-    "train_samples": 106682,
-    "train_samples_per_second": 9.199,
-    "train_steps_per_second": 0.072
+    "train_loss": 0.05042304046280229,
+    "train_runtime": 8923.6937,
+    "train_samples": 61134,
+    "train_samples_per_second": 13.702,
+    "train_steps_per_second": 0.107
 }

model-00001-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1b548fafbd9efff630df189b0bf00fc664acb471a1214bad1dee2a5ea6974fe6
+oid sha256:4f07e4e59297222e87a201a992cae0f88a9a9667a88343813d686d79d1bd05cd
 size 4976698672

model-00002-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5b67fc61d7577b913d32c3bfd018089599c95780f44fe89e5e1dbedb56a40586
+oid sha256:6ced690ef624a7753e879b8c0fad2db8e6663519b82e4fd1f1fcd24309e291f3
 size 4999802720

model-00003-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fdd3743d2c166fcadbed6ea88c7574aec5bcf890a0dddb4fb54b675acfb7c9e8
+oid sha256:8cbb0b93b3dcda24ebda2c2b48122f43df269487fa73f2c3efe50a1cae56bdae
 size 4915916176

model-00004-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3f3acebda78f6b10ff7250ea1c5c6805c4ea09dbadf2d15c05e328527d01207c
+oid sha256:62e4fdd625a71f4a8c2517c9a87c86b92901aa52f4fe9e780f3f74a12a96e127
 size 1168138808

train_results.json

@@ -1,8 +1,8 @@
 {
     "epoch": 2.0,
-    "train_loss": 0.11496639693108927,
-    "train_runtime": 23193.7036,
-    "train_samples": 106682,
-    "train_samples_per_second": 9.199,
-    "train_steps_per_second": 0.072
+    "train_loss": 0.05042304046280229,
+    "train_runtime": 8923.6937,
+    "train_samples": 61134,
+    "train_samples_per_second": 13.702,
+    "train_steps_per_second": 0.107
 }

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8fcb49e2510ad8daccc633592dcdb9ab896e9fd597c24b453bda8c8d73ec2183
+oid sha256:1efe5818c60138c7516bdee959b1c547110ddbf35ce1b9d021048a3d3e59f522
 size 6648