Hugging Face's logo

Basha738
/
outputs

PEFT
TensorBoard
Safetensors
trl
sft
Generated from Trainer
Model card Files Metrics Community
Basha738 commited on
Commit
b925d9f
·
verified ·
1 Parent(s): 800de26

llama2-13B-supervised-eos-ft-10-epochs-351

Browse files
Files changed (4) hide show
  1. README.md +23 -44
  2. adapter_config.json +2 -2
  3. adapter_model.safetensors +1 -1
  4. training_args.bin +1 -1
README.md CHANGED
@@ -17,7 +17,7 @@ should probably proofread and complete it, then remove this comment. -->
17
 
18
  This model was trained from scratch on the None dataset.
19
  It achieves the following results on the evaluation set:
20
- - Loss: 1.4471
21
 
22
  ## Model description
23
 
@@ -44,55 +44,34 @@ The following hyperparameters were used during training:
44
  - total_train_batch_size: 16
45
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
  - lr_scheduler_type: linear
47
- - num_epochs: 10
48
  - mixed_precision_training: Native AMP
49
 
50
  ### Training results
51
 
52
  | Training Loss | Epoch | Step | Validation Loss |
53
  |:-------------:|:-----:|:----:|:---------------:|
54
- | 2.0121 | 0.24 | 4 | 1.9623 |
55
- | 2.0559 | 0.47 | 8 | 1.8792 |
56
- | 1.8364 | 0.71 | 12 | 1.8000 |
57
- | 1.6852 | 0.94 | 16 | 1.7083 |
58
- | 1.7672 | 1.18 | 20 | 1.6258 |
59
- | 1.6344 | 1.41 | 24 | 1.5731 |
60
- | 1.5026 | 1.65 | 28 | 1.5477 |
61
- | 1.4473 | 1.88 | 32 | 1.5344 |
62
- | 1.5297 | 2.12 | 36 | 1.5254 |
63
- | 1.5623 | 2.35 | 40 | 1.5183 |
64
- | 1.3866 | 2.59 | 44 | 1.5122 |
65
- | 1.5727 | 2.82 | 48 | 1.5067 |
66
- | 1.3696 | 3.06 | 52 | 1.5018 |
67
- | 1.5159 | 3.29 | 56 | 1.4973 |
68
- | 1.5107 | 3.53 | 60 | 1.4932 |
69
- | 1.4175 | 3.76 | 64 | 1.4893 |
70
- | 1.404 | 4.0 | 68 | 1.4854 |
71
- | 1.4115 | 4.24 | 72 | 1.4821 |
72
- | 1.5804 | 4.47 | 76 | 1.4788 |
73
- | 1.4163 | 4.71 | 80 | 1.4759 |
74
- | 1.4404 | 4.94 | 84 | 1.4729 |
75
- | 1.4331 | 5.18 | 88 | 1.4700 |
76
- | 1.4269 | 5.41 | 92 | 1.4674 |
77
- | 1.4902 | 5.65 | 96 | 1.4650 |
78
- | 1.493 | 5.88 | 100 | 1.4627 |
79
- | 1.4476 | 6.12 | 104 | 1.4608 |
80
- | 1.4142 | 6.35 | 108 | 1.4592 |
81
- | 1.4147 | 6.59 | 112 | 1.4573 |
82
- | 1.3399 | 6.82 | 116 | 1.4558 |
83
- | 1.4323 | 7.06 | 120 | 1.4543 |
84
- | 1.2752 | 7.29 | 124 | 1.4531 |
85
- | 1.4907 | 7.53 | 128 | 1.4524 |
86
- | 1.3854 | 7.76 | 132 | 1.4510 |
87
- | 1.3615 | 8.0 | 136 | 1.4498 |
88
- | 1.5176 | 8.24 | 140 | 1.4492 |
89
- | 1.3634 | 8.47 | 144 | 1.4490 |
90
- | 1.3644 | 8.71 | 148 | 1.4486 |
91
- | 1.4832 | 8.94 | 152 | 1.4481 |
92
- | 1.4305 | 9.18 | 156 | 1.4476 |
93
- | 1.5344 | 9.41 | 160 | 1.4473 |
94
- | 1.4204 | 9.65 | 164 | 1.4471 |
95
- | 1.3488 | 9.88 | 168 | 1.4471 |
96
 
97
 
98
  ### Framework versions
 
17
 
18
  This model was trained from scratch on the None dataset.
19
  It achieves the following results on the evaluation set:
20
+ - Loss: 1.4939
21
 
22
  ## Model description
23
 
 
44
  - total_train_batch_size: 16
45
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
  - lr_scheduler_type: linear
47
+ - num_epochs: 5
48
  - mixed_precision_training: Native AMP
49
 
50
  ### Training results
51
 
52
  | Training Loss | Epoch | Step | Validation Loss |
53
  |:-------------:|:-----:|:----:|:---------------:|
54
+ | 2.0181 | 0.24 | 4 | 1.9684 |
55
+ | 2.0616 | 0.47 | 8 | 1.8863 |
56
+ | 1.8467 | 0.71 | 12 | 1.8116 |
57
+ | 1.707 | 0.94 | 16 | 1.7309 |
58
+ | 1.7886 | 1.18 | 20 | 1.6529 |
59
+ | 1.6539 | 1.41 | 24 | 1.5884 |
60
+ | 1.5149 | 1.65 | 28 | 1.5568 |
61
+ | 1.4526 | 1.88 | 32 | 1.5390 |
62
+ | 1.5335 | 2.12 | 36 | 1.5283 |
63
+ | 1.5668 | 2.35 | 40 | 1.5211 |
64
+ | 1.3914 | 2.59 | 44 | 1.5158 |
65
+ | 1.5769 | 2.82 | 48 | 1.5113 |
66
+ | 1.3794 | 3.06 | 52 | 1.5075 |
67
+ | 1.5274 | 3.29 | 56 | 1.5043 |
68
+ | 1.5247 | 3.53 | 60 | 1.5016 |
69
+ | 1.4291 | 3.76 | 64 | 1.4993 |
70
+ | 1.4233 | 4.0 | 68 | 1.4974 |
71
+ | 1.4353 | 4.24 | 72 | 1.4960 |
72
+ | 1.6016 | 4.47 | 76 | 1.4949 |
73
+ | 1.4416 | 4.71 | 80 | 1.4942 |
74
+ | 1.4654 | 4.94 | 84 | 1.4939 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
75
 
76
 
77
  ### Framework versions
adapter_config.json CHANGED
@@ -19,12 +19,12 @@
19
  "rank_pattern": {},
20
  "revision": null,
21
  "target_modules": [
22
- "k_proj",
23
- "up_proj",
24
  "gate_proj",
25
  "v_proj",
26
  "q_proj",
 
27
  "down_proj",
 
28
  "o_proj"
29
  ],
30
  "task_type": "CAUSAL_LM",
 
19
  "rank_pattern": {},
20
  "revision": null,
21
  "target_modules": [
 
 
22
  "gate_proj",
23
  "v_proj",
24
  "q_proj",
25
+ "up_proj",
26
  "down_proj",
27
+ "k_proj",
28
  "o_proj"
29
  ],
30
  "task_type": "CAUSAL_LM",
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a7270f8b2ce500576568e6d0d99dc89289051a76cac3bfc66b49b84b0c3bdb80
3
  size 2002857080
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:018bc976adbc8ea0c2a4f8aba4c440d7e960ccd7b14a4e3d881b02373eaed436
3
  size 2002857080
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2feb3a72004a78345189fddde57ecae17643f1dfe32c31b72068642d138e3242
3
  size 4664
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7921f3d55a49dada1b2321ee29cf6d83a7052e21522778adb18c865bfa58831e
3
  size 4664