Edit model card

cs_mT5-large_0.01_100_v0.1

This model is a fine-tuned version of google/mt5-large on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 6.2112
  • Bleu: 0.8171
  • Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.01
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 100

Training results

Training Loss Epoch Step Validation Loss Bleu Gen Len
9.5426 1.0 6 13.2737 0.0 19.0
9.5598 2.0 12 57.9184 0.2088 19.0
8.9859 3.0 18 7.1357 0.2088 19.0
4.8419 4.0 24 6.5896 0.0 2.0
6.0099 5.0 30 5.9797 0.0 19.0
5.4071 6.0 36 6.0228 0.0 19.0
5.4716 7.0 42 6.0132 0.0 19.0
5.6419 8.0 48 5.9242 0.0 19.0
5.7044 9.0 54 6.0529 0.0 19.0
5.7007 10.0 60 5.8730 0.0 19.0
5.0052 11.0 66 6.0392 0.0 19.0
6.3889 12.0 72 6.0776 0.0 19.0
5.2703 13.0 78 70.6639 0.0 19.0
7.1444 14.0 84 7.6067 0.0 19.0
4.7785 15.0 90 6.5610 0.0 19.0
5.6738 16.0 96 6.0522 0.0 19.0
5.5087 17.0 102 6.0558 0.0 19.0
5.4367 18.0 108 5.9737 0.0 19.0
5.5081 19.0 114 6.0431 0.0 19.0
5.2506 20.0 120 5.9623 0.0 19.0
5.354 21.0 126 6.0081 0.0 19.0
5.5891 22.0 132 5.9859 0.0 19.0
5.2457 23.0 138 5.9296 0.0 19.0
4.9566 24.0 144 6.0038 0.0 19.0
5.3327 25.0 150 6.0421 0.0 19.0
4.946 26.0 156 6.0225 0.0 19.0
5.1903 27.0 162 5.9587 0.0 19.0
5.0797 28.0 168 5.9780 0.0 19.0
4.8033 29.0 174 6.0577 0.0 19.0
5.559 30.0 180 6.0250 0.0 19.0
5.7859 31.0 186 5.9493 0.0 19.0
5.4172 32.0 192 6.0647 0.0 19.0
4.9906 33.0 198 6.0617 0.0 19.0
4.9745 34.0 204 5.9800 0.0 19.0
5.2086 35.0 210 5.9942 0.0 19.0
5.7047 36.0 216 5.9996 0.0 19.0
4.4275 37.0 222 6.0826 0.0 19.0
4.9545 38.0 228 6.0865 0.0 19.0
5.1466 39.0 234 5.9571 0.0 19.0
5.5095 40.0 240 5.9970 0.0 19.0
5.1998 41.0 246 5.9978 0.0 19.0
4.8406 42.0 252 6.0314 0.0 19.0
5.0467 43.0 258 6.0444 0.0 19.0
5.2282 44.0 264 6.0295 0.0 19.0
4.8847 45.0 270 6.0284 0.0 19.0
5.5734 46.0 276 6.0598 0.0 19.0
4.743 47.0 282 6.0396 0.0 19.0
5.3795 48.0 288 6.0567 0.0 19.0
4.9066 49.0 294 6.0615 0.0 19.0
4.9682 50.0 300 6.1018 0.0 19.0
4.828 51.0 306 6.0605 0.0 19.0
4.5153 52.0 312 6.0531 0.0 19.0
5.2316 53.0 318 5.9855 0.0 19.0
4.8071 54.0 324 6.0292 0.0 19.0
5.106 55.0 330 6.0541 0.0 19.0
4.9581 56.0 336 5.9499 0.0 19.0
4.8037 57.0 342 6.1083 0.0 19.0
4.7738 58.0 348 6.0111 0.0 19.0
5.3786 59.0 354 6.0164 0.0 19.0
4.8782 60.0 360 5.9442 0.0 19.0
4.8589 61.0 366 5.9036 0.8171 19.0
4.8486 62.0 372 5.7896 0.8171 19.0
4.4303 63.0 378 5.8475 0.8171 19.0
5.116 64.0 384 5.7361 0.8171 19.0
4.9206 65.0 390 5.7211 0.8171 19.0
4.5294 66.0 396 5.6845 0.8171 19.0
5.0969 67.0 402 5.6964 0.8171 19.0
4.4403 68.0 408 5.7035 0.8171 19.0
4.3498 69.0 414 5.7088 0.8171 19.0
5.0456 70.0 420 5.6742 0.8171 19.0
4.9812 71.0 426 5.6820 0.8171 19.0
4.4053 72.0 432 5.7010 0.8171 19.0
4.8459 73.0 438 5.8511 0.8171 19.0
4.3272 74.0 444 5.7204 0.8171 19.0
4.4791 75.0 450 5.7542 0.8171 19.0
4.5272 76.0 456 5.7444 0.8171 19.0
4.2581 77.0 462 5.7456 0.879 19.0
4.718 78.0 468 5.7187 0.8171 19.0
4.3661 79.0 474 5.8472 0.8291 19.0
4.8016 80.0 480 5.7478 0.8171 19.0
4.1973 81.0 486 5.8850 0.8171 19.0
4.0916 82.0 492 5.7678 0.8171 19.0
4.1624 83.0 498 5.8662 0.8171 19.0
4.2458 84.0 504 5.9224 0.8171 19.0
3.7141 85.0 510 5.8928 0.8171 19.0
3.5796 86.0 516 6.0489 0.937 19.0
4.8417 87.0 522 6.1602 0.8171 19.0
4.3568 88.0 528 5.9343 0.8171 19.0
4.6028 89.0 534 5.9039 0.8171 19.0
3.6638 90.0 540 6.1188 0.879 19.0
4.1465 91.0 546 6.0166 0.8171 19.0
4.32 92.0 552 6.0690 0.8171 19.0
4.0945 93.0 558 6.0812 0.8171 19.0
3.9572 94.0 564 5.9877 0.8171 19.0
3.9032 95.0 570 6.0960 0.2223 19.0
4.3571 96.0 576 6.1585 0.8171 19.0
3.768 97.0 582 6.1953 0.8171 19.0
3.94 98.0 588 6.2025 0.8171 19.0
3.8452 99.0 594 6.2129 0.8171 19.0
4.4174 100.0 600 6.2112 0.8171 19.0

Framework versions

  • Transformers 4.35.2
  • Pytorch 1.13.1+cu117
  • Datasets 2.17.0
  • Tokenizers 0.15.2
Downloads last month
1
Safetensors
Model size
1.23B params
Tensor type
F32
·

Finetuned from