Hugging Face's logo

tongliuphysics
/
Llama0-3-8b-v0.1-dpo-lr1e-6-e1

Text Generation
Transformers
Safetensors
llama
trl
dpo
Generated from Trainer
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
tongliuphysics commited on
Commit
36396b5
·
verified ·
1 Parent(s): 0556788

Model save

Browse files
Files changed (8) hide show
  1. README.md +13 -13
  2. all_results.json +12 -12
  3. eval_results.json +12 -12
  4. model-00001-of-00004.safetensors +1 -1
  5. model-00002-of-00004.safetensors +1 -1
  6. model-00003-of-00004.safetensors +1 -1
  7. model-00004-of-00004.safetensors +1 -1
  8. training_args.bin +1 -1
README.md CHANGED
@@ -18,15 +18,15 @@ should probably proofread and complete it, then remove this comment. -->
18
 
19
  This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) on an unknown dataset.
20
  It achieves the following results on the evaluation set:
21
- - Loss: 0.6324
22
- - Rewards/chosen: -1.3540
23
- - Rewards/rejected: -1.5425
24
- - Rewards/accuracies: 0.6169
25
- - Rewards/margins: 0.1885
26
- - Logps/rejected: -241.0062
27
- - Logps/chosen: -223.6923
28
- - Logits/rejected: 0.3721
29
- - Logits/chosen: 0.3664
30
 
31
  ## Model description
32
 
@@ -62,10 +62,10 @@ The following hyperparameters were used during training:
62
 
63
  | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
64
  |:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
65
- | 0.6748 | 0.2137 | 100 | 0.6726 | -0.3070 | -0.3472 | 0.6169 | 0.0403 | -121.4768 | -118.9880 | 0.1390 | 0.1219 |
66
- | 0.6509 | 0.4275 | 200 | 0.6504 | -0.8078 | -0.9126 | 0.6089 | 0.1047 | -178.0135 | -169.0766 | 0.2864 | 0.2760 |
67
- | 0.6393 | 0.6412 | 300 | 0.6399 | -1.1579 | -1.3156 | 0.6048 | 0.1577 | -218.3196 | -204.0809 | 0.3588 | 0.3523 |
68
- | 0.6325 | 0.8549 | 400 | 0.6336 | -1.2896 | -1.4695 | 0.6129 | 0.1799 | -233.7066 | -217.2510 | 0.3661 | 0.3609 |
69
 
70
 
71
  ### Framework versions
 
18
 
19
  This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) on an unknown dataset.
20
  It achieves the following results on the evaluation set:
21
+ - Loss: 0.6081
22
+ - Rewards/chosen: -1.5097
23
+ - Rewards/rejected: -1.7233
24
+ - Rewards/accuracies: 0.6129
25
+ - Rewards/margins: 0.2136
26
+ - Logps/rejected: -259.0845
27
+ - Logps/chosen: -239.2623
28
+ - Logits/rejected: 0.4158
29
+ - Logits/chosen: 0.4094
30
 
31
  ## Model description
32
 
 
62
 
63
  | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
64
  |:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
65
+ | 0.652 | 0.2137 | 100 | 0.6498 | -0.3163 | -0.3568 | 0.6169 | 0.0405 | -122.4376 | -119.9265 | 0.1437 | 0.1272 |
66
+ | 0.6273 | 0.4275 | 200 | 0.6269 | -0.8960 | -1.0115 | 0.6129 | 0.1155 | -187.9061 | -177.8965 | 0.3135 | 0.3044 |
67
+ | 0.6152 | 0.6412 | 300 | 0.6155 | -1.2887 | -1.4649 | 0.6129 | 0.1762 | -233.2506 | -217.1634 | 0.4062 | 0.4005 |
68
+ | 0.608 | 0.8549 | 400 | 0.6094 | -1.4301 | -1.6328 | 0.6089 | 0.2027 | -250.0380 | -231.3053 | 0.4062 | 0.4004 |
69
 
70
 
71
  ### Framework versions
all_results.json CHANGED
@@ -1,15 +1,15 @@
1
  {
2
  "epoch": 0.9981298423724285,
3
- "eval_logits/chosen": 0.3663541078567505,
4
- "eval_logits/rejected": 0.37210896611213684,
5
- "eval_logps/chosen": -223.69229125976562,
6
- "eval_logps/rejected": -241.00619506835938,
7
- "eval_loss": 0.632370114326477,
8
- "eval_rewards/accuracies": 0.6169354915618896,
9
- "eval_rewards/chosen": -1.3539990186691284,
10
- "eval_rewards/margins": 0.1885022521018982,
11
- "eval_rewards/rejected": -1.5425012111663818,
12
- "eval_runtime": 171.0384,
13
- "eval_samples_per_second": 11.465,
14
- "eval_steps_per_second": 0.181
15
  }
 
1
  {
2
  "epoch": 0.9981298423724285,
3
+ "eval_logits/chosen": 0.409419447183609,
4
+ "eval_logits/rejected": 0.41578754782676697,
5
+ "eval_logps/chosen": -239.2622528076172,
6
+ "eval_logps/rejected": -259.08453369140625,
7
+ "eval_loss": 0.6081043481826782,
8
+ "eval_rewards/accuracies": 0.6129032373428345,
9
+ "eval_rewards/chosen": -1.5096983909606934,
10
+ "eval_rewards/margins": 0.21358628571033478,
11
+ "eval_rewards/rejected": -1.7232848405838013,
12
+ "eval_runtime": 177.9012,
13
+ "eval_samples_per_second": 11.023,
14
+ "eval_steps_per_second": 0.174
15
  }
eval_results.json CHANGED
@@ -1,15 +1,15 @@
1
  {
2
  "epoch": 0.9981298423724285,
3
- "eval_logits/chosen": 0.3663541078567505,
4
- "eval_logits/rejected": 0.37210896611213684,
5
- "eval_logps/chosen": -223.69229125976562,
6
- "eval_logps/rejected": -241.00619506835938,
7
- "eval_loss": 0.632370114326477,
8
- "eval_rewards/accuracies": 0.6169354915618896,
9
- "eval_rewards/chosen": -1.3539990186691284,
10
- "eval_rewards/margins": 0.1885022521018982,
11
- "eval_rewards/rejected": -1.5425012111663818,
12
- "eval_runtime": 171.0384,
13
- "eval_samples_per_second": 11.465,
14
- "eval_steps_per_second": 0.181
15
  }
 
1
  {
2
  "epoch": 0.9981298423724285,
3
+ "eval_logits/chosen": 0.409419447183609,
4
+ "eval_logits/rejected": 0.41578754782676697,
5
+ "eval_logps/chosen": -239.2622528076172,
6
+ "eval_logps/rejected": -259.08453369140625,
7
+ "eval_loss": 0.6081043481826782,
8
+ "eval_rewards/accuracies": 0.6129032373428345,
9
+ "eval_rewards/chosen": -1.5096983909606934,
10
+ "eval_rewards/margins": 0.21358628571033478,
11
+ "eval_rewards/rejected": -1.7232848405838013,
12
+ "eval_runtime": 177.9012,
13
+ "eval_samples_per_second": 11.023,
14
+ "eval_steps_per_second": 0.174
15
  }
model-00001-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a3a80e17a4818f4128ab68c5cfcfd52051507d3bde475437719c9ce7cf56622d
3
  size 4976698672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7f08cd636808435c7266ce8c1c7a9796034f8f893c27cf79b449ee5cf7d28c98
3
  size 4976698672
model-00002-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2e4f8fbed2ab857830f437e2e48a5393248d6c4c1d3fc8d6c843dbd034745ead
3
  size 4999802720
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:79907330806b1ee161c8e733a3b01db1990e1c15faf20b9864cb61191118e2c0
3
  size 4999802720
model-00003-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ba184c8561458ab27eab718a20a62910cbd766bf0fe01704c0f7a543ed40ae8e
3
  size 4915916176
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c0b27c0f207ebb45479d1b135a2ec15fa6362fce5599bf9eb1fcd3ee757278e2
3
  size 4915916176
model-00004-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:22ae4dfa795744cad3daf1c81992cc6d3ffa911ca3a00efaa8dcfea9015580b4
3
  size 1168138808
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:981cba4b62131c3684835bb9fa4f75af8be60beb5df267f13ce0270af22cf664
3
  size 1168138808
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:87e2b81217281de710d0a5428aaaa179721605c619386f43b32e0eaed4e250ad
3
  size 7288
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:81f2fd967aff15b09fd1764742096386cce96a2062598ebd936eceabc08a43ba
3
  size 7288