Hugging Face's logo

thorirhrafn
/
llama_DPO_model_e3

PEFT
TensorBoard
Safetensors
trl
dpo
Generated from Trainer
Model card Files Metrics Community
thorirhrafn commited on
Commit
ed39636
•
1 Parent(s): 3f16323

End of training

Browse files
Files changed (1) hide show
  1. README.md +100 -0
README.md ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: llama2
3
+ library_name: peft
4
+ tags:
5
+ - trl
6
+ - dpo
7
+ - generated_from_trainer
8
+ base_model: meta-llama/Llama-2-7b-hf
9
+ model-index:
10
+ - name: llama_DPO_model_e3
11
+ results: []
12
+ ---
13
+
14
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
+ should probably proofread and complete it, then remove this comment. -->
16
+
17
+ # llama_DPO_model_e3
18
+
19
+ This model is a fine-tuned version of [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) on an unknown dataset.
20
+ It achieves the following results on the evaluation set:
21
+ - Loss: 0.0722
22
+ - Rewards/chosen: 0.4618
23
+ - Rewards/rejected: -2.3246
24
+ - Rewards/accuracies: 1.0
25
+ - Rewards/margins: 2.7864
26
+ - Logps/rejected: -208.0558
27
+ - Logps/chosen: -156.0157
28
+ - Logits/rejected: -1.0512
29
+ - Logits/chosen: -0.8590
30
+
31
+ ## Model description
32
+
33
+ More information needed
34
+
35
+ ## Intended uses & limitations
36
+
37
+ More information needed
38
+
39
+ ## Training and evaluation data
40
+
41
+ More information needed
42
+
43
+ ## Training procedure
44
+
45
+ ### Training hyperparameters
46
+
47
+ The following hyperparameters were used during training:
48
+ - learning_rate: 7e-07
49
+ - train_batch_size: 1
50
+ - eval_batch_size: 1
51
+ - seed: 42
52
+ - gradient_accumulation_steps: 8
53
+ - total_train_batch_size: 8
54
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
55
+ - lr_scheduler_type: linear
56
+ - num_epochs: 3
57
+
58
+ ### Training results
59
+
60
+ | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
61
+ |:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
62
+ | 0.675 | 0.1 | 25 | 0.6531 | 0.0248 | -0.0584 | 0.8667 | 0.0832 | -185.3936 | -160.3859 | -1.0523 | -0.8549 |
63
+ | 0.5865 | 0.2 | 50 | 0.5720 | 0.0730 | -0.1895 | 0.9933 | 0.2625 | -186.7048 | -159.9039 | -1.0525 | -0.8552 |
64
+ | 0.5203 | 0.3 | 75 | 0.4808 | 0.1258 | -0.3673 | 1.0 | 0.4931 | -188.4825 | -159.3763 | -1.0520 | -0.8543 |
65
+ | 0.4291 | 0.4 | 100 | 0.3986 | 0.1804 | -0.5547 | 1.0 | 0.7352 | -190.3568 | -158.8295 | -1.0527 | -0.8559 |
66
+ | 0.3712 | 0.5 | 125 | 0.3264 | 0.2303 | -0.7594 | 1.0 | 0.9897 | -192.4033 | -158.3308 | -1.0528 | -0.8572 |
67
+ | 0.2856 | 0.6 | 150 | 0.2612 | 0.2765 | -0.9893 | 1.0 | 1.2658 | -194.7025 | -157.8685 | -1.0531 | -0.8592 |
68
+ | 0.2433 | 0.7 | 175 | 0.2086 | 0.3223 | -1.2201 | 1.0 | 1.5424 | -197.0102 | -157.4110 | -1.0526 | -0.8573 |
69
+ | 0.1822 | 0.79 | 200 | 0.1673 | 0.3627 | -1.4385 | 1.0 | 1.8012 | -199.1950 | -157.0071 | -1.0529 | -0.8606 |
70
+ | 0.1511 | 0.89 | 225 | 0.1354 | 0.3921 | -1.6585 | 1.0 | 2.0506 | -201.3948 | -156.7133 | -1.0522 | -0.8601 |
71
+ | 0.1211 | 0.99 | 250 | 0.1134 | 0.4119 | -1.8492 | 1.0 | 2.2612 | -203.3017 | -156.5144 | -1.0526 | -0.8591 |
72
+ | 0.113 | 1.09 | 275 | 0.0999 | 0.4261 | -1.9792 | 1.0 | 2.4054 | -204.6017 | -156.3724 | -1.0511 | -0.8578 |
73
+ | 0.087 | 1.19 | 300 | 0.0912 | 0.4374 | -2.0704 | 1.0 | 2.5078 | -205.5134 | -156.2602 | -1.0521 | -0.8612 |
74
+ | 0.0808 | 1.29 | 325 | 0.0846 | 0.4439 | -2.1510 | 1.0 | 2.5949 | -206.3199 | -156.1949 | -1.0515 | -0.8600 |
75
+ | 0.0875 | 1.39 | 350 | 0.0814 | 0.4537 | -2.1942 | 1.0 | 2.6479 | -206.7517 | -156.0968 | -1.0520 | -0.8589 |
76
+ | 0.0826 | 1.49 | 375 | 0.0785 | 0.4559 | -2.2325 | 1.0 | 2.6884 | -207.1346 | -156.0752 | -1.0516 | -0.8585 |
77
+ | 0.0717 | 1.59 | 400 | 0.0768 | 0.4564 | -2.2611 | 1.0 | 2.7175 | -207.4205 | -156.0697 | -1.0517 | -0.8595 |
78
+ | 0.0694 | 1.69 | 425 | 0.0750 | 0.4602 | -2.2778 | 1.0 | 2.7380 | -207.5878 | -156.0322 | -1.0516 | -0.8590 |
79
+ | 0.0809 | 1.79 | 450 | 0.0739 | 0.4647 | -2.2925 | 1.0 | 2.7572 | -207.7341 | -155.9865 | -1.0514 | -0.8586 |
80
+ | 0.0747 | 1.89 | 475 | 0.0736 | 0.4595 | -2.3075 | 1.0 | 2.7670 | -207.8848 | -156.0394 | -1.0515 | -0.8584 |
81
+ | 0.0751 | 1.99 | 500 | 0.0726 | 0.4643 | -2.3130 | 1.0 | 2.7773 | -207.9396 | -155.9911 | -1.0516 | -0.8589 |
82
+ | 0.069 | 2.09 | 525 | 0.0725 | 0.4608 | -2.3223 | 1.0 | 2.7831 | -208.0324 | -156.0257 | -1.0512 | -0.8598 |
83
+ | 0.0658 | 2.19 | 550 | 0.0724 | 0.4670 | -2.3178 | 1.0 | 2.7847 | -207.9872 | -155.9642 | -1.0514 | -0.8580 |
84
+ | 0.0659 | 2.29 | 575 | 0.0720 | 0.4650 | -2.3217 | 1.0 | 2.7867 | -208.0269 | -155.9841 | -1.0516 | -0.8592 |
85
+ | 0.0732 | 2.38 | 600 | 0.0725 | 0.4585 | -2.3236 | 1.0 | 2.7821 | -208.0455 | -156.0485 | -1.0511 | -0.8591 |
86
+ | 0.0802 | 2.48 | 625 | 0.0723 | 0.4611 | -2.3249 | 1.0 | 2.7859 | -208.0582 | -156.0233 | -1.0511 | -0.8582 |
87
+ | 0.0734 | 2.58 | 650 | 0.0723 | 0.4646 | -2.3213 | 1.0 | 2.7859 | -208.0227 | -155.9879 | -1.0510 | -0.8591 |
88
+ | 0.068 | 2.68 | 675 | 0.0723 | 0.4627 | -2.3230 | 1.0 | 2.7857 | -208.0397 | -156.0069 | -1.0512 | -0.8585 |
89
+ | 0.0708 | 2.78 | 700 | 0.0720 | 0.4617 | -2.3278 | 1.0 | 2.7895 | -208.0874 | -156.0165 | -1.0508 | -0.8592 |
90
+ | 0.0621 | 2.88 | 725 | 0.0719 | 0.4613 | -2.3296 | 1.0 | 2.7909 | -208.1059 | -156.0208 | -1.0511 | -0.8585 |
91
+ | 0.0708 | 2.98 | 750 | 0.0722 | 0.4618 | -2.3246 | 1.0 | 2.7864 | -208.0558 | -156.0157 | -1.0512 | -0.8590 |
92
+
93
+
94
+ ### Framework versions
95
+
96
+ - PEFT 0.8.2
97
+ - Transformers 4.38.1
98
+ - Pytorch 2.2.0+cu118
99
+ - Datasets 2.17.1
100
+ - Tokenizers 0.15.2