Model save

README.md

@@ -14,16 +14,6 @@ should probably proofread and complete it, then remove this comment. -->
 # zephyr-7b-dpo-full
 
 This model was trained from scratch on the None dataset.
-It achieves the following results on the evaluation set:
-- Loss: 0.5286
-- Rewards/chosen: -1.7068
-- Rewards/rejected: -3.1572
-- Rewards/accuracies: 0.7695
-- Rewards/margins: 1.4504
-- Logps/rejected: -627.3446
-- Logps/chosen: -474.2680
-- Logits/rejected: -0.7503
-- Logits/chosen: -0.5802
 
 ## Model description
 
@@ -43,12 +33,12 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 1e-06
-- train_batch_size: 4
+- train_batch_size: 2
 - eval_batch_size: 8
 - seed: 5
 - distributed_type: multi-GPU
 - num_devices: 8
-- gradient_accumulation_steps: 4
+- gradient_accumulation_steps: 8
 - total_train_batch_size: 128
 - total_eval_batch_size: 64
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
@@ -58,17 +48,6 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
-|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
-| 0.6254        | 0.21  | 100  | 0.6260          | -0.1988        | -0.5392          | 0.6992             | 0.3404          | -365.5406      | -323.4606    | 0.3870          | 0.3426        |
-| 0.5841        | 0.42  | 200  | 0.5597          | -0.4205        | -1.0478          | 0.7305             | 0.6273          | -416.4040      | -345.6356    | 0.1437          | 0.0907        |
-| 0.5389        | 0.63  | 300  | 0.5285          | -0.6859        | -1.5998          | 0.7773             | 0.9140          | -471.6094      | -372.1726    | 0.2331          | 0.2134        |
-| 0.5188        | 0.84  | 400  | 0.5197          | -0.7311        | -1.7606          | 0.7852             | 1.0295          | -487.6861      | -376.6970    | -0.1165         | -0.1000       |
-| 0.3402        | 1.05  | 500  | 0.5344          | -1.5025        | -2.9522          | 0.7773             | 1.4497          | -606.8411      | -453.8337    | -0.4375         | -0.3802       |
-| 0.3141        | 1.26  | 600  | 0.5426          | -1.7806        | -3.3337          | 0.7539             | 1.5531          | -644.9940      | -481.6454    | -0.6733         | -0.5363       |
-| 0.3324        | 1.47  | 700  | 0.5322          | -1.7213        | -3.2147          | 0.7773             | 1.4934          | -633.0936      | -475.7130    | -0.9457         | -0.7473       |
-| 0.3372        | 1.67  | 800  | 0.5313          | -1.7652        | -3.2295          | 0.7656             | 1.4643          | -634.5750      | -480.1067    | -0.7581         | -0.5822       |
-| 0.3058        | 1.88  | 900  | 0.5286          | -1.7068        | -3.1572          | 0.7695             | 1.4504          | -627.3446      | -474.2680    | -0.7503         | -0.5802       |
 
 
 ### Framework versions

all_results.json

@@ -1,8 +1,8 @@
 {
     "epoch": 2.0,
-    "train_loss": 0.44928585458351633,
-    "train_runtime": 8755.9554,
-    "train_samples": 61134,
-    "train_samples_per_second": 13.964,
-    "train_steps_per_second": 0.109
+    "train_loss": 0.11559809408500558,
+    "train_runtime": 23407.6305,
+    "train_samples": 106682,
+    "train_samples_per_second": 9.115,
+    "train_steps_per_second": 0.071
 }

model-00001-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bb6dae14a8a45dc2dba2faa238cabeb3dd475cf38582ca55e0db21e1bcba298b
+oid sha256:e0e47c941b4dacd35c121d5e4bc0286cf8b5e81054c59e2d56144e860df9597c
 size 4976698672

model-00002-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d4a9b139ac61337a3f9804165615d4b66c3ac23c8aa785fe62d6ed1e6762d8c7
+oid sha256:dca0065a230087c3b55705c6a6f7054e64f0b81d4cd9031df0f8aad008ede197
 size 4999802720

model-00003-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:891c89339231c57c3d361d8e3316ff43c01cf327000837bc937ebf6b25a6bc24
+oid sha256:1d1d4980829c9d3d06ef7a81e02fffa0e40201c4532305ec6a04aba5f5e1b689
 size 4915916176

model-00004-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c1a6e1fa0a8e051df6b214baf25d5e7ecf4e020f3d12403c1e06564d15c9d5dd
+oid sha256:cdae88e9a153704c05b7398a5c9237f862194fe810bc9fe187286c41d9552058
 size 1168138808

train_results.json

@@ -1,8 +1,8 @@
 {
     "epoch": 2.0,
-    "train_loss": 0.44928585458351633,
-    "train_runtime": 8755.9554,
-    "train_samples": 61134,
-    "train_samples_per_second": 13.964,
-    "train_steps_per_second": 0.109
+    "train_loss": 0.11559809408500558,
+    "train_runtime": 23407.6305,
+    "train_samples": 106682,
+    "train_samples_per_second": 9.115,
+    "train_steps_per_second": 0.071
 }

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2a6b1b9121f5e07a38d89a34290e13b8df30cc0da45f64840437b68179a1db9a
+oid sha256:53e04de78c43c6d34212d0523f5accae0cd42011de8bac7befb2084cd87faa70
 size 6648