Hugging Face's logo

sujithatz
/
checkpoint_dir

PEFT
Safetensors
trl
sft
Generated from Trainer
Model card Files Community
checkpoint_dir / README.md
sujithatz's picture
sujithatz
sujithatz/finbot-transofrmer-based-phi3.5_adapter
fa5219a verified
preview code
|
raw
history blame
11.6 kB
metadata
base_model: microsoft/Phi-3.5-mini-instruct
library_name: peft
license: mit
tags:
  - trl
  - sft
  - generated_from_trainer
model-index:
  - name: checkpoint_dir
    results: []

checkpoint_dir

This model is a fine-tuned version of microsoft/Phi-3.5-mini-instruct on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5253

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 0
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.01
  • num_epochs: 8

Training results

Training Loss Epoch Step Validation Loss
1.7134 0.0405 3 1.8221
1.7619 0.0811 6 1.6961
1.5449 0.1216 9 1.5453
1.3625 0.1622 12 1.3982
1.1389 0.2027 15 1.2786
1.1091 0.2432 18 1.1889
1.0605 0.2838 21 1.1050
0.9908 0.3243 24 1.0395
0.9653 0.3649 27 0.9886
0.9258 0.4054 30 0.9401
0.8964 0.4459 33 0.8945
0.8189 0.4865 36 0.8615
0.7202 0.5270 39 0.8325
0.7553 0.5676 42 0.8109
0.7415 0.6081 45 0.7911
0.6421 0.6486 48 0.7730
0.7638 0.6892 51 0.7411
0.7495 0.7297 54 0.7208
0.7678 0.7703 57 0.7102
0.7027 0.8108 60 0.7002
0.7106 0.8514 63 0.6892
0.8461 0.8919 66 0.6852
0.5863 0.9324 69 0.6826
0.7466 0.9730 72 0.6802
0.5847 1.0135 75 0.6696
0.5349 1.0541 78 0.6590
0.5991 1.0946 81 0.6560
0.5777 1.1351 84 0.6526
0.6342 1.1757 87 0.6488
0.5053 1.2162 90 0.6494
0.4909 1.2568 93 0.6485
0.5154 1.2973 96 0.6458
0.4728 1.3378 99 0.6375
0.5648 1.3784 102 0.6327
0.4878 1.4189 105 0.6260
0.5677 1.4595 108 0.6165
0.6598 1.5 111 0.6059
0.5811 1.5405 114 0.6021
0.5984 1.5811 117 0.6018
0.4477 1.6216 120 0.6010
0.5762 1.6622 123 0.5944
0.7896 1.7027 126 0.5924
0.449 1.7432 129 0.5849
0.6014 1.7838 132 0.5793
0.4798 1.8243 135 0.5744
0.4943 1.8649 138 0.5715
0.3907 1.9054 141 0.5692
0.6352 1.9459 144 0.5631
0.469 1.9865 147 0.5633
0.4819 2.0270 150 0.5623
0.7567 2.0676 153 0.5610
0.533 2.1081 156 0.5641
0.4195 2.1486 159 0.5615
0.4015 2.1892 162 0.5609
0.2958 2.2297 165 0.5642
0.4477 2.2703 168 0.5602
0.4111 2.3108 171 0.5530
0.3958 2.3514 174 0.5495
0.3053 2.3919 177 0.5437
0.4952 2.4324 180 0.5400
0.5617 2.4730 183 0.5322
0.298 2.5135 186 0.5273
0.5439 2.5541 189 0.5256
0.5791 2.5946 192 0.5215
0.4429 2.6351 195 0.5205
0.4454 2.6757 198 0.5251
0.4071 2.7162 201 0.5267
0.3948 2.7568 204 0.5327
0.3196 2.7973 207 0.5342
0.3567 2.8378 210 0.5344
0.5284 2.8784 213 0.5292
0.491 2.9189 216 0.5182
0.4267 2.9595 219 0.5137
0.3587 3.0 222 0.5098
0.3587 3.0405 225 0.5131
0.377 3.0811 228 0.5200
0.6423 3.1216 231 0.5214
0.4839 3.1622 234 0.5139
0.566 3.2027 237 0.5123
0.38 3.2432 240 0.5172
0.3995 3.2838 243 0.5207
0.3486 3.3243 246 0.5148
0.2418 3.3649 249 0.5104
0.3178 3.4054 252 0.5086
0.4065 3.4459 255 0.5031
0.3472 3.4865 258 0.5050
0.4543 3.5270 261 0.5046
0.4066 3.5676 264 0.5020
0.2606 3.6081 267 0.5010
0.2332 3.6486 270 0.5007
0.5026 3.6892 273 0.5003
0.3901 3.7297 276 0.5057
0.3552 3.7703 279 0.5126
0.3921 3.8108 282 0.5179
0.3366 3.8514 285 0.5092
0.3706 3.8919 288 0.5008
0.2791 3.9324 291 0.4961
0.2247 3.9730 294 0.4968
0.2879 4.0135 297 0.4971
0.3355 4.0541 300 0.5036
0.3928 4.0946 303 0.5023
0.2399 4.1351 306 0.5056
0.3396 4.1757 309 0.5089
0.2602 4.2162 312 0.5091
0.2565 4.2568 315 0.5110
0.24 4.2973 318 0.5156
0.2364 4.3378 321 0.5216
0.3694 4.3784 324 0.5224
0.2185 4.4189 327 0.5183
0.337 4.4595 330 0.5119
0.3404 4.5 333 0.5084
0.3049 4.5405 336 0.5071
0.4811 4.5811 339 0.5098
0.338 4.6216 342 0.5092
0.305 4.6622 345 0.5090
0.5273 4.7027 348 0.5079
0.3122 4.7432 351 0.5044
0.2995 4.7838 354 0.4991
0.2654 4.8243 357 0.4935
0.3992 4.8649 360 0.4946
0.2272 4.9054 363 0.5003
0.3094 4.9459 366 0.5026
0.2773 4.9865 369 0.5021
0.3934 5.0270 372 0.4993
0.271 5.0676 375 0.5015
0.3928 5.1081 378 0.5040
0.2105 5.1486 381 0.5134
0.2548 5.1892 384 0.5182
0.2424 5.2297 387 0.5104
0.4469 5.2703 390 0.5122
0.2866 5.3108 393 0.5112
0.2958 5.3514 396 0.5090
0.2034 5.3919 399 0.5051
0.4091 5.4324 402 0.5023
0.1415 5.4730 405 0.5059
0.4137 5.5135 408 0.5098
0.2784 5.5541 411 0.5134
0.158 5.5946 414 0.5160
0.4701 5.6351 417 0.5183
0.2256 5.6757 420 0.5168
0.1868 5.7162 423 0.5147
0.2868 5.7568 426 0.5130
0.2142 5.7973 429 0.5147
0.2693 5.8378 432 0.5130
0.2882 5.8784 435 0.5108
0.3243 5.9189 438 0.5098
0.343 5.9595 441 0.5067
0.2602 6.0 444 0.5002
0.2237 6.0405 447 0.5001
0.3727 6.0811 450 0.5039
0.2471 6.1216 453 0.5076
0.4095 6.1622 456 0.5145
0.2445 6.2027 459 0.5188
0.2387 6.2432 462 0.5231
0.2322 6.2838 465 0.5258
0.2998 6.3243 468 0.5270
0.2463 6.3649 471 0.5251
0.1931 6.4054 474 0.5237
0.2254 6.4459 477 0.5187
0.278 6.4865 480 0.5177
0.3654 6.5270 483 0.5162
0.2886 6.5676 486 0.5130
0.229 6.6081 489 0.5150
0.2361 6.6486 492 0.5158
0.1497 6.6892 495 0.5165
0.2926 6.7297 498 0.5179
0.2979 6.7703 501 0.5211
0.244 6.8108 504 0.5200
0.2846 6.8514 507 0.5197
0.1897 6.8919 510 0.5200
0.2106 6.9324 513 0.5210
0.3168 6.9730 516 0.5210
0.2002 7.0135 519 0.5192
0.3515 7.0541 522 0.5202
0.1807 7.0946 525 0.5214
0.2331 7.1351 528 0.5212
0.1571 7.1757 531 0.5215
0.186 7.2162 534 0.5194
0.2281 7.2568 537 0.5207
0.2534 7.2973 540 0.5219
0.3643 7.3378 543 0.5212
0.4516 7.3784 546 0.5203
0.181 7.4189 549 0.5226
0.256 7.4595 552 0.5214
0.2802 7.5 555 0.5212
0.1913 7.5405 558 0.5196
0.2293 7.5811 561 0.5207
0.2282 7.6216 564 0.5213
0.1954 7.6622 567 0.5225
0.3199 7.7027 570 0.5216
0.2687 7.7432 573 0.5231
0.2122 7.7838 576 0.5218
0.3616 7.8243 579 0.5228
0.1206 7.8649 582 0.5212
0.148 7.9054 585 0.5216
0.3779 7.9459 588 0.5224
0.272 7.9865 591 0.5253

Framework versions

  • PEFT 0.13.2
  • Transformers 4.45.2
  • Pytorch 2.4.0
  • Datasets 3.0.1
  • Tokenizers 0.20.0