Model save

README.md

@@ -14,6 +14,16 @@ should probably proofread and complete it, then remove this comment. -->
 # zephyr-7b-dpo-full
 
 This model was trained from scratch on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.5286
+- Rewards/chosen: -1.7068
+- Rewards/rejected: -3.1572
+- Rewards/accuracies: 0.7695
+- Rewards/margins: 1.4504
+- Logps/rejected: -627.3446
+- Logps/chosen: -474.2680
+- Logits/rejected: -0.7503
+- Logits/chosen: -0.5802
 
 ## Model description
 
@@ -33,12 +43,12 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 1e-06
-- train_batch_size: 2
+- train_batch_size: 4
 - eval_batch_size: 8
-- seed: 4
+- seed: 5
 - distributed_type: multi-GPU
 - num_devices: 8
-- gradient_accumulation_steps: 8
+- gradient_accumulation_steps: 4
 - total_train_batch_size: 128
 - total_eval_batch_size: 64
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
@@ -48,6 +58,17 @@ The following hyperparameters were used during training:
 
 ### Training results
 
+| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
+|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
+| 0.6254        | 0.21  | 100  | 0.6260          | -0.1988        | -0.5392          | 0.6992             | 0.3404          | -365.5406      | -323.4606    | 0.3870          | 0.3426        |
+| 0.5841        | 0.42  | 200  | 0.5597          | -0.4205        | -1.0478          | 0.7305             | 0.6273          | -416.4040      | -345.6356    | 0.1437          | 0.0907        |
+| 0.5389        | 0.63  | 300  | 0.5285          | -0.6859        | -1.5998          | 0.7773             | 0.9140          | -471.6094      | -372.1726    | 0.2331          | 0.2134        |
+| 0.5188        | 0.84  | 400  | 0.5197          | -0.7311        | -1.7606          | 0.7852             | 1.0295          | -487.6861      | -376.6970    | -0.1165         | -0.1000       |
+| 0.3402        | 1.05  | 500  | 0.5344          | -1.5025        | -2.9522          | 0.7773             | 1.4497          | -606.8411      | -453.8337    | -0.4375         | -0.3802       |
+| 0.3141        | 1.26  | 600  | 0.5426          | -1.7806        | -3.3337          | 0.7539             | 1.5531          | -644.9940      | -481.6454    | -0.6733         | -0.5363       |
+| 0.3324        | 1.47  | 700  | 0.5322          | -1.7213        | -3.2147          | 0.7773             | 1.4934          | -633.0936      | -475.7130    | -0.9457         | -0.7473       |
+| 0.3372        | 1.67  | 800  | 0.5313          | -1.7652        | -3.2295          | 0.7656             | 1.4643          | -634.5750      | -480.1067    | -0.7581         | -0.5822       |
+| 0.3058        | 1.88  | 900  | 0.5286          | -1.7068        | -3.1572          | 0.7695             | 1.4504          | -627.3446      | -474.2680    | -0.7503         | -0.5802       |
 
 
 ### Framework versions

all_results.json

@@ -1,8 +1,8 @@
 {
     "epoch": 2.0,
-    "train_loss": 0.11496639693108927,
-    "train_runtime": 24857.3347,
-    "train_samples": 106682,
-    "train_samples_per_second": 8.584,
-    "train_steps_per_second": 0.067
+    "train_loss": 0.44928585458351633,
+    "train_runtime": 8755.9554,
+    "train_samples": 61134,
+    "train_samples_per_second": 13.964,
+    "train_steps_per_second": 0.109
 }

model-00001-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1b548fafbd9efff630df189b0bf00fc664acb471a1214bad1dee2a5ea6974fe6
+oid sha256:bb6dae14a8a45dc2dba2faa238cabeb3dd475cf38582ca55e0db21e1bcba298b
 size 4976698672

model-00002-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5b67fc61d7577b913d32c3bfd018089599c95780f44fe89e5e1dbedb56a40586
+oid sha256:d4a9b139ac61337a3f9804165615d4b66c3ac23c8aa785fe62d6ed1e6762d8c7
 size 4999802720

model-00003-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fdd3743d2c166fcadbed6ea88c7574aec5bcf890a0dddb4fb54b675acfb7c9e8
+oid sha256:891c89339231c57c3d361d8e3316ff43c01cf327000837bc937ebf6b25a6bc24
 size 4915916176

model-00004-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3f3acebda78f6b10ff7250ea1c5c6805c4ea09dbadf2d15c05e328527d01207c
+oid sha256:c1a6e1fa0a8e051df6b214baf25d5e7ecf4e020f3d12403c1e06564d15c9d5dd
 size 1168138808

train_results.json

@@ -1,8 +1,8 @@
 {
     "epoch": 2.0,
-    "train_loss": 0.11496639693108927,
-    "train_runtime": 24857.3347,
-    "train_samples": 106682,
-    "train_samples_per_second": 8.584,
-    "train_steps_per_second": 0.067
+    "train_loss": 0.44928585458351633,
+    "train_runtime": 8755.9554,
+    "train_samples": 61134,
+    "train_samples_per_second": 13.964,
+    "train_steps_per_second": 0.109
 }

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fd7260ee010ca4a604f5b589d0f690f6be98dc3409be725d29e2e433c0a137f5
+oid sha256:2a6b1b9121f5e07a38d89a34290e13b8df30cc0da45f64840437b68179a1db9a
 size 6648