Model save

README.md

@@ -14,16 +14,6 @@ should probably proofread and complete it, then remove this comment. -->
 # zephyr-7b-dpo-full
 
 This model was trained from scratch on the None dataset.
-It achieves the following results on the evaluation set:
-- Loss: 0.5304
-- Rewards/chosen: -2.0877
-- Rewards/rejected: -3.5813
-- Rewards/accuracies: 0.7969
-- Rewards/margins: 1.4935
-- Logps/rejected: -669.7546
-- Logps/chosen: -512.3596
-- Logits/rejected: -0.0805
-- Logits/chosen: 0.0044
 
 ## Model description
 
@@ -43,12 +33,12 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 1e-06
-- train_batch_size: 4
+- train_batch_size: 2
 - eval_batch_size: 8
 - seed: 4
 - distributed_type: multi-GPU
 - num_devices: 8
-- gradient_accumulation_steps: 4
+- gradient_accumulation_steps: 8
 - total_train_batch_size: 128
 - total_eval_batch_size: 64
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
@@ -58,17 +48,6 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
-|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
-| 0.6209        | 0.21  | 100  | 0.6260          | -0.2545        | -0.6360          | 0.6953             | 0.3815          | -375.2259      | -329.0332    | 0.3909          | 0.3389        |
-| 0.5447        | 0.42  | 200  | 0.5506          | -0.6231        | -1.4102          | 0.7852             | 0.7871          | -452.6489      | -365.8961    | 0.5763          | 0.5112        |
-| 0.5257        | 0.63  | 300  | 0.5325          | -0.9411        | -1.9562          | 0.7617             | 1.0151          | -507.2468      | -397.6915    | 0.2916          | 0.2656        |
-| 0.5016        | 0.84  | 400  | 0.5168          | -1.1370        | -2.2739          | 0.7930             | 1.1368          | -539.0118      | -417.2896    | 0.1301          | 0.1581        |
-| 0.3557        | 1.05  | 500  | 0.5232          | -1.5956        | -3.0234          | 0.7852             | 1.4277          | -613.9626      | -463.1488    | 0.1606          | 0.1971        |
-| 0.3459        | 1.26  | 600  | 0.5179          | -1.8408        | -3.1490          | 0.7969             | 1.3082          | -626.5206      | -487.6650    | 0.1677          | 0.2149        |
-| 0.3321        | 1.47  | 700  | 0.5331          | -2.0696        | -3.5323          | 0.7891             | 1.4626          | -664.8507      | -510.5476    | -0.0690         | 0.0033        |
-| 0.2983        | 1.67  | 800  | 0.5289          | -2.0385        | -3.4952          | 0.7930             | 1.4566          | -661.1429      | -507.4386    | -0.0746         | 0.0060        |
-| 0.3235        | 1.88  | 900  | 0.5304          | -2.0877        | -3.5813          | 0.7969             | 1.4935          | -669.7546      | -512.3596    | -0.0805         | 0.0044        |
 
 
 ### Framework versions

all_results.json

@@ -1,8 +1,8 @@
 {
     "epoch": 2.0,
-    "train_loss": 0.44723546317538376,
-    "train_runtime": 8668.6976,
-    "train_samples": 61134,
-    "train_samples_per_second": 14.105,
-    "train_steps_per_second": 0.11
+    "train_loss": 0.11496639693108927,
+    "train_runtime": 24857.3347,
+    "train_samples": 106682,
+    "train_samples_per_second": 8.584,
+    "train_steps_per_second": 0.067
 }

model-00001-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4400d5cc92103182c0288602258bcb6a357097c64fdd3c11599cc42caa1ffea8
+oid sha256:1b548fafbd9efff630df189b0bf00fc664acb471a1214bad1dee2a5ea6974fe6
 size 4976698672

model-00002-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d49e56f1ffb7e3b39edf0324f5f7b5fa8049cb6b5243bb8606d58cea6fd54f4c
+oid sha256:5b67fc61d7577b913d32c3bfd018089599c95780f44fe89e5e1dbedb56a40586
 size 4999802720

model-00003-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:be757ff962fe896c71369dfc072edc1b45a8a8170b10760f068e2d5c05ec06f0
+oid sha256:fdd3743d2c166fcadbed6ea88c7574aec5bcf890a0dddb4fb54b675acfb7c9e8
 size 4915916176

model-00004-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4f1b4fe6c8278ecf71a36df42146155fd60f564232f4a0499a05cab556a07fb1
+oid sha256:3f3acebda78f6b10ff7250ea1c5c6805c4ea09dbadf2d15c05e328527d01207c
 size 1168138808

train_results.json

@@ -1,8 +1,8 @@
 {
     "epoch": 2.0,
-    "train_loss": 0.44723546317538376,
-    "train_runtime": 8668.6976,
-    "train_samples": 61134,
-    "train_samples_per_second": 14.105,
-    "train_steps_per_second": 0.11
+    "train_loss": 0.11496639693108927,
+    "train_runtime": 24857.3347,
+    "train_samples": 106682,
+    "train_samples_per_second": 8.584,
+    "train_steps_per_second": 0.067
 }

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bf44d047df68bef9d42ea33d5bfa7fdecd645ea74f86b115b9c82ad801a096e6
+oid sha256:fd7260ee010ca4a604f5b589d0f690f6be98dc3409be725d29e2e433c0a137f5
 size 6648