Model save

README.md

@@ -14,6 +14,16 @@ should probably proofread and complete it, then remove this comment. -->
 # zephyr-7b-dpo-full
 
 This model was trained from scratch on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0224
+- Rewards/chosen: -1.9945
+- Rewards/rejected: -3.2919
+- Rewards/accuracies: 0.7148
+- Rewards/margins: 1.2974
+- Logps/rejected: -640.8138
+- Logps/chosen: -503.0325
+- Logits/rejected: 0.3215
+- Logits/chosen: 0.2841
 
 ## Model description
 
@@ -33,12 +43,12 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 1e-06
-- train_batch_size: 2
+- train_batch_size: 4
 - eval_batch_size: 8
-- seed: 3
+- seed: 4
 - distributed_type: multi-GPU
 - num_devices: 8
-- gradient_accumulation_steps: 8
+- gradient_accumulation_steps: 4
 - total_train_batch_size: 128
 - total_eval_batch_size: 64
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
@@ -48,6 +58,17 @@ The following hyperparameters were used during training:
 
 ### Training results
 
+| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
+|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
+| 0.111         | 0.21  | 100  | 0.1080          | -0.3300        | -0.6434          | 0.7148             | 0.3134          | -375.9606      | -336.5851    | 0.4520          | 0.3976        |
+| 0.0697        | 0.42  | 200  | 0.0728          | -0.5844        | -1.2213          | 0.7422             | 0.6369          | -433.7567      | -362.0242    | 0.4101          | 0.3267        |
+| 0.055         | 0.63  | 300  | 0.0610          | -0.7945        | -1.5421          | 0.7266             | 0.7476          | -465.8376      | -383.0369    | 0.2780          | 0.2451        |
+| 0.0573        | 0.84  | 400  | 0.0566          | -0.8305        | -1.5952          | 0.7383             | 0.7647          | -471.1477      | -386.6394    | 0.2561          | 0.2348        |
+| 0.0215        | 1.05  | 500  | 0.0327          | -1.6150        | -2.8668          | 0.7305             | 1.2517          | -598.3008      | -465.0880    | 0.2419          | 0.2221        |
+| 0.0139        | 1.26  | 600  | 0.0260          | -1.8080        | -3.0895          | 0.7227             | 1.2815          | -620.5768      | -484.3871    | 0.2916          | 0.2601        |
+| 0.0125        | 1.47  | 700  | 0.0247          | -1.9121        | -3.1886          | 0.7305             | 1.2765          | -630.4850      | -494.7950    | 0.2947          | 0.2614        |
+| 0.0107        | 1.67  | 800  | 0.0226          | -1.9947        | -3.2951          | 0.7188             | 1.3004          | -641.1344      | -503.0576    | 0.3196          | 0.2841        |
+| 0.0106        | 1.88  | 900  | 0.0224          | -1.9945        | -3.2919          | 0.7148             | 1.2974          | -640.8138      | -503.0325    | 0.3215          | 0.2841        |
 
 
 ### Framework versions

all_results.json

@@ -1,8 +1,8 @@
 {
     "epoch": 2.0,
-    "train_loss": 0.11590367368682951,
-    "train_runtime": 24766.7088,
-    "train_samples": 106682,
-    "train_samples_per_second": 8.615,
-    "train_steps_per_second": 0.067
+    "train_loss": 0.049936374161290924,
+    "train_runtime": 8881.7089,
+    "train_samples": 61134,
+    "train_samples_per_second": 13.766,
+    "train_steps_per_second": 0.107
 }

model-00001-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3489fcaec81823ca8ae60e4455bb4632a4b4e86e4b8a920a3533de4f1482436e
+oid sha256:6d02de6c8edfeeeb0253ca2e615c4ad095be502f4f2daa1d2584121e533850d7
 size 4976698672

model-00002-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9ebdc8cfbfe9010ea09e753efce060c7a93f352e7a3742747bb6e04afa768130
+oid sha256:8b4c182af3e082a3edf41d75086f984893666c4aff034fb3c79740eb7488540c
 size 4999802720

model-00003-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ea5399d071d1636e27fbee607807a2231bf69ac74f9997d515ea28fdc2b5b54f
+oid sha256:a36a65b6cc24b5ae3dfaf97458a75ad18ec6c0ec44c61a174fa593e52e1ad78b
 size 4915916176

model-00004-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:38702b95a0f22b50e9667d98ca314733d2fc34909db10f2ad9f44cbca40397fa
+oid sha256:1eef956d179706a12a747534eb1331caf296a451da0e621b616adc9e84c06df3
 size 1168138808

train_results.json

@@ -1,8 +1,8 @@
 {
     "epoch": 2.0,
-    "train_loss": 0.11590367368682951,
-    "train_runtime": 24766.7088,
-    "train_samples": 106682,
-    "train_samples_per_second": 8.615,
-    "train_steps_per_second": 0.067
+    "train_loss": 0.049936374161290924,
+    "train_runtime": 8881.7089,
+    "train_samples": 61134,
+    "train_samples_per_second": 13.766,
+    "train_steps_per_second": 0.107
 }

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4a874454b0be106b09135ca7d876005da005cccf11147d71834b9b4e2669c3e1
+oid sha256:bb017de948f5de7f3516c58b8c593bf371eefcb77e51739c8cde03e1e7e3faed
 size 6648