Model save

README.md

@@ -14,6 +14,16 @@ should probably proofread and complete it, then remove this comment. -->
 # zephyr-7b-dpo-full
 
 This model was trained from scratch on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.5304
+- Rewards/chosen: -2.0877
+- Rewards/rejected: -3.5813
+- Rewards/accuracies: 0.7969
+- Rewards/margins: 1.4935
+- Logps/rejected: -669.7546
+- Logps/chosen: -512.3596
+- Logits/rejected: -0.0805
+- Logits/chosen: 0.0044
 
 ## Model description
 
@@ -33,12 +43,12 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 1e-06
-- train_batch_size: 2
+- train_batch_size: 4
 - eval_batch_size: 8
-- seed: 5
+- seed: 4
 - distributed_type: multi-GPU
 - num_devices: 8
-- gradient_accumulation_steps: 8
+- gradient_accumulation_steps: 4
 - total_train_batch_size: 128
 - total_eval_batch_size: 64
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
@@ -48,6 +58,17 @@ The following hyperparameters were used during training:
 
 ### Training results
 
+| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
+|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
+| 0.6209        | 0.21  | 100  | 0.6260          | -0.2545        | -0.6360          | 0.6953             | 0.3815          | -375.2259      | -329.0332    | 0.3909          | 0.3389        |
+| 0.5447        | 0.42  | 200  | 0.5506          | -0.6231        | -1.4102          | 0.7852             | 0.7871          | -452.6489      | -365.8961    | 0.5763          | 0.5112        |
+| 0.5257        | 0.63  | 300  | 0.5325          | -0.9411        | -1.9562          | 0.7617             | 1.0151          | -507.2468      | -397.6915    | 0.2916          | 0.2656        |
+| 0.5016        | 0.84  | 400  | 0.5168          | -1.1370        | -2.2739          | 0.7930             | 1.1368          | -539.0118      | -417.2896    | 0.1301          | 0.1581        |
+| 0.3557        | 1.05  | 500  | 0.5232          | -1.5956        | -3.0234          | 0.7852             | 1.4277          | -613.9626      | -463.1488    | 0.1606          | 0.1971        |
+| 0.3459        | 1.26  | 600  | 0.5179          | -1.8408        | -3.1490          | 0.7969             | 1.3082          | -626.5206      | -487.6650    | 0.1677          | 0.2149        |
+| 0.3321        | 1.47  | 700  | 0.5331          | -2.0696        | -3.5323          | 0.7891             | 1.4626          | -664.8507      | -510.5476    | -0.0690         | 0.0033        |
+| 0.2983        | 1.67  | 800  | 0.5289          | -2.0385        | -3.4952          | 0.7930             | 1.4566          | -661.1429      | -507.4386    | -0.0746         | 0.0060        |
+| 0.3235        | 1.88  | 900  | 0.5304          | -2.0877        | -3.5813          | 0.7969             | 1.4935          | -669.7546      | -512.3596    | -0.0805         | 0.0044        |
 
 
 ### Framework versions

all_results.json

@@ -1,8 +1,8 @@
 {
     "epoch": 2.0,
-    "train_loss": 0.11547961056518669,
-    "train_runtime": 23573.0656,
-    "train_samples": 106682,
-    "train_samples_per_second": 9.051,
-    "train_steps_per_second": 0.071
+    "train_loss": 0.44723546317538376,
+    "train_runtime": 8668.6976,
+    "train_samples": 61134,
+    "train_samples_per_second": 14.105,
+    "train_steps_per_second": 0.11
 }

model-00001-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b3923b40d0bbe0fbb4239bba707a55c077de441f88e83016dc12be92aa19fa46
+oid sha256:4400d5cc92103182c0288602258bcb6a357097c64fdd3c11599cc42caa1ffea8
 size 4976698672

model-00002-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:09bc3c0542093c9fee78d6eaa2a803b32670ea4206ea182b40b1eec04497c535
+oid sha256:d49e56f1ffb7e3b39edf0324f5f7b5fa8049cb6b5243bb8606d58cea6fd54f4c
 size 4999802720

model-00003-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3a8a9a3fe6dd4932209eea72e8bfe225b8068e4ba8e093c9951dc2ddc3892b3d
+oid sha256:be757ff962fe896c71369dfc072edc1b45a8a8170b10760f068e2d5c05ec06f0
 size 4915916176

model-00004-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:627e9b3dd2c96dfa2a32ebab95fcbaab2565940fc256159eb68c24932a34ee24
+oid sha256:4f1b4fe6c8278ecf71a36df42146155fd60f564232f4a0499a05cab556a07fb1
 size 1168138808

train_results.json

@@ -1,8 +1,8 @@
 {
     "epoch": 2.0,
-    "train_loss": 0.11547961056518669,
-    "train_runtime": 23573.0656,
-    "train_samples": 106682,
-    "train_samples_per_second": 9.051,
-    "train_steps_per_second": 0.071
+    "train_loss": 0.44723546317538376,
+    "train_runtime": 8668.6976,
+    "train_samples": 61134,
+    "train_samples_per_second": 14.105,
+    "train_steps_per_second": 0.11
 }

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1aa79c700934ffba8c1d63a094dba92ff61f81afadd70c22c9c307e8b5a8030d
+oid sha256:bf44d047df68bef9d42ea33d5bfa7fdecd645ea74f86b115b9c82ad801a096e6
 size 6648