Hugging Face's logo

ldos
/
text_shortening_model_v10

Text2Text Generation
Transformers
PyTorch
t5
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Community
text_shortening_model_v10 / README.md
ldos's picture
ldos
End of training
695353d
preview code
|
raw
history blame contribute delete
25.4 kB
metadata
license: apache-2.0
base_model: t5-small
tags:
  - generated_from_trainer
metrics:
  - rouge
model-index:
  - name: text_shortening_model_v10
    results: []

text_shortening_model_v10

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.7343
  • Rouge1: 0.5944
  • Rouge2: 0.3803
  • Rougel: 0.5562
  • Rougelsum: 0.5556
  • Bert precision: 0.8982
  • Bert recall: 0.9028
  • Average word count: 11.1571
  • Max word count: 16
  • Min word count: 7
  • Average token count: 16.4
  • % shortened texts with length > 12: 22.1429

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 100

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Bert precision Bert recall Average word count Max word count Min word count Average token count % shortened texts with length > 12
2.1519 1.0 16 1.6818 0.5512 0.3185 0.4947 0.4946 0.8804 0.8891 11.8643 18 5 17.0071 45.7143
1.824 2.0 32 1.5909 0.5691 0.3406 0.5167 0.5163 0.8858 0.8957 11.8714 18 3 16.8429 47.1429
1.6784 3.0 48 1.5260 0.582 0.3562 0.5308 0.5295 0.8871 0.8997 12.0214 18 4 17.0929 47.1429
1.5584 4.0 64 1.4877 0.5749 0.3431 0.5258 0.5249 0.8878 0.8996 11.8 17 6 16.9786 42.1429
1.4831 5.0 80 1.4597 0.5849 0.3527 0.5337 0.5329 0.8871 0.901 12.0071 17 6 17.2857 44.2857
1.4536 6.0 96 1.4384 0.587 0.3562 0.5375 0.536 0.8889 0.9015 11.8929 16 6 17.2071 41.4286
1.3631 7.0 112 1.4238 0.5928 0.3616 0.5438 0.5434 0.89 0.9024 11.7643 16 7 17.1 40.0
1.285 8.0 128 1.4128 0.5876 0.3566 0.5364 0.5355 0.8898 0.9008 11.55 16 6 16.9286 36.4286
1.2515 9.0 144 1.4009 0.5953 0.3631 0.5444 0.5436 0.8913 0.9015 11.6286 16 6 16.9857 36.4286
1.2159 10.0 160 1.3975 0.5898 0.3583 0.54 0.5398 0.8915 0.9017 11.5714 16 6 16.9643 33.5714
1.1865 11.0 176 1.3955 0.5977 0.3641 0.5465 0.5457 0.8933 0.9034 11.4857 16 6 16.7643 32.8571
1.1476 12.0 192 1.3925 0.5903 0.3584 0.5436 0.5429 0.8906 0.9026 11.6571 17 7 17.0 34.2857
1.1196 13.0 208 1.3878 0.5972 0.3667 0.5482 0.5471 0.8935 0.9036 11.4857 16 6 16.7643 33.5714
1.0767 14.0 224 1.3949 0.5978 0.3687 0.5481 0.5474 0.8946 0.9041 11.45 16 6 16.7643 32.1429
1.0343 15.0 240 1.3930 0.6013 0.3776 0.5547 0.5544 0.8961 0.9046 11.4357 16 6 16.6714 30.0
1.0302 16.0 256 1.3959 0.597 0.3702 0.5453 0.5446 0.8921 0.9057 11.75 17 6 17.2071 34.2857
1.0251 17.0 272 1.3876 0.5999 0.3752 0.5524 0.5519 0.893 0.9061 11.7429 16 6 17.1643 34.2857
0.9773 18.0 288 1.3929 0.6002 0.3724 0.5463 0.5462 0.8946 0.9041 11.4929 16 6 16.7929 30.7143
0.9437 19.0 304 1.3921 0.6038 0.3824 0.5555 0.5553 0.897 0.9053 11.2857 16 6 16.5571 27.8571
0.9267 20.0 320 1.4016 0.6046 0.3774 0.5542 0.554 0.8923 0.9048 11.8429 16 7 17.1929 35.7143
0.9178 21.0 336 1.4047 0.6054 0.3775 0.5553 0.5545 0.8957 0.9056 11.5429 16 4 16.8643 29.2857
0.8941 22.0 352 1.4112 0.6039 0.3775 0.5556 0.555 0.8937 0.9052 11.6143 17 4 17.0 29.2857
0.8715 23.0 368 1.4156 0.602 0.3811 0.5552 0.5548 0.8947 0.9057 11.4929 16 6 16.8714 27.8571
0.847 24.0 384 1.4283 0.6013 0.3771 0.5516 0.5509 0.8935 0.905 11.5071 16 6 16.9571 30.0
0.8319 25.0 400 1.4349 0.5991 0.3747 0.5525 0.5519 0.8951 0.9049 11.3 16 6 16.7857 27.1429
0.8081 26.0 416 1.4421 0.6014 0.3797 0.5529 0.5528 0.8946 0.9056 11.5 16 6 16.9071 30.0
0.8098 27.0 432 1.4406 0.6038 0.3811 0.5546 0.5543 0.8965 0.9053 11.3429 16 6 16.6786 28.5714
0.7738 28.0 448 1.4544 0.5993 0.3774 0.5544 0.5535 0.8951 0.9042 11.3 17 5 16.7429 25.7143
0.7651 29.0 464 1.4711 0.6024 0.383 0.5573 0.5568 0.8979 0.9053 11.15 17 6 16.4357 22.1429
0.7495 30.0 480 1.4666 0.6066 0.3842 0.5609 0.5597 0.8979 0.9065 11.1643 16 5 16.6429 21.4286
0.7216 31.0 496 1.4779 0.6009 0.3801 0.5555 0.5549 0.8968 0.9048 11.2643 16 6 16.7143 25.7143
0.7074 32.0 512 1.4918 0.6 0.3792 0.5547 0.5537 0.8959 0.9041 11.3 16 7 16.7643 26.4286
0.7241 33.0 528 1.4914 0.6029 0.3859 0.5606 0.5596 0.8984 0.9044 11.1071 16 7 16.4857 20.7143
0.7001 34.0 544 1.4851 0.6035 0.3831 0.5591 0.5584 0.8971 0.9044 11.25 16 7 16.65 26.4286
0.6863 35.0 560 1.4942 0.6001 0.3818 0.556 0.5551 0.8961 0.902 11.2429 16 7 16.6214 25.7143
0.6724 36.0 576 1.4976 0.5958 0.3721 0.5496 0.5493 0.8959 0.9018 11.2143 16 7 16.5429 25.0
0.6652 37.0 592 1.5006 0.5976 0.3714 0.5522 0.5513 0.8964 0.9018 11.1071 16 7 16.4714 22.8571
0.6326 38.0 608 1.5064 0.6013 0.3802 0.5573 0.5563 0.8975 0.9017 11.0643 16 7 16.3071 22.1429
0.6408 39.0 624 1.5209 0.5969 0.3797 0.5557 0.555 0.8947 0.9025 11.35 16 7 16.6 27.8571
0.6209 40.0 640 1.5194 0.599 0.3773 0.5547 0.5538 0.8961 0.9036 11.35 16 7 16.6429 27.1429
0.6094 41.0 656 1.5285 0.6076 0.3917 0.5677 0.5667 0.9002 0.9064 11.1071 16 7 16.35 21.4286
0.6007 42.0 672 1.5403 0.607 0.3844 0.5657 0.5646 0.8991 0.9052 11.2214 15 6 16.3714 22.1429
0.5916 43.0 688 1.5546 0.5991 0.3768 0.5608 0.5602 0.8964 0.9044 11.3857 15 7 16.7357 25.7143
0.5816 44.0 704 1.5533 0.5955 0.3687 0.556 0.5556 0.8959 0.9017 11.1857 15 7 16.5571 22.1429
0.5714 45.0 720 1.5604 0.6025 0.3785 0.5592 0.5589 0.8978 0.9037 11.2357 16 7 16.4786 22.8571
0.563 46.0 736 1.5673 0.6034 0.3795 0.5604 0.5598 0.8969 0.9027 11.3 16 7 16.5571 25.0
0.546 47.0 752 1.5723 0.6005 0.381 0.5595 0.5592 0.8979 0.9035 11.2714 17 7 16.4429 25.0
0.5386 48.0 768 1.5735 0.5942 0.3768 0.5541 0.554 0.898 0.9028 11.1929 16 7 16.3214 25.0
0.5489 49.0 784 1.5781 0.5923 0.372 0.5527 0.5531 0.8966 0.9017 11.1857 16 7 16.3643 24.2857
0.5267 50.0 800 1.5837 0.5928 0.3729 0.5519 0.5513 0.8966 0.9019 11.2786 17 7 16.3929 25.0
0.5274 51.0 816 1.5907 0.5974 0.3751 0.5586 0.558 0.8988 0.9029 11.1857 17 6 16.25 26.4286
0.5206 52.0 832 1.5964 0.5913 0.3673 0.5515 0.5515 0.8966 0.9014 11.2429 16 7 16.3857 25.0
0.4979 53.0 848 1.6073 0.59 0.3719 0.5546 0.555 0.8965 0.9012 11.1357 17 7 16.3143 22.8571
0.5007 54.0 864 1.6126 0.5923 0.3733 0.5561 0.5559 0.8961 0.9012 11.2643 17 7 16.4286 24.2857
0.5035 55.0 880 1.6188 0.5972 0.3749 0.5567 0.5567 0.8985 0.9024 11.0786 16 7 16.2143 20.7143
0.504 56.0 896 1.6320 0.5996 0.3776 0.5593 0.5597 0.8985 0.9038 11.1357 16 7 16.3143 21.4286
0.4908 57.0 912 1.6333 0.5941 0.3757 0.5552 0.5554 0.897 0.9034 11.1929 16 7 16.3857 23.5714
0.4748 58.0 928 1.6339 0.5968 0.3704 0.5541 0.554 0.8977 0.9025 11.1571 16 7 16.3786 23.5714
0.4751 59.0 944 1.6352 0.6006 0.3791 0.5601 0.5599 0.8988 0.9032 11.1214 17 7 16.3071 20.7143
0.474 60.0 960 1.6349 0.6006 0.3865 0.5618 0.5609 0.8987 0.9045 11.1786 17 7 16.3786 22.8571
0.4673 61.0 976 1.6357 0.5924 0.3756 0.5561 0.5559 0.8982 0.9024 11.0786 16 6 16.2 20.7143
0.4708 62.0 992 1.6488 0.5964 0.3833 0.5607 0.5602 0.8985 0.9043 11.2214 16 7 16.4143 25.0
0.4721 63.0 1008 1.6555 0.5949 0.3781 0.5569 0.5563 0.8966 0.9036 11.2643 16 7 16.5071 25.0
0.46 64.0 1024 1.6644 0.5955 0.3749 0.5536 0.5536 0.8982 0.903 11.1714 16 7 16.3286 20.7143
0.4469 65.0 1040 1.6648 0.5962 0.3791 0.558 0.5581 0.8968 0.9041 11.3714 16 7 16.5429 27.1429
0.4395 66.0 1056 1.6675 0.5912 0.3767 0.5526 0.5523 0.8976 0.903 11.2357 16 7 16.4071 22.8571
0.4363 67.0 1072 1.6699 0.5913 0.3747 0.5538 0.5533 0.8987 0.902 11.0 16 7 16.0786 19.2857
0.4313 68.0 1088 1.6751 0.5854 0.3666 0.5464 0.5452 0.897 0.9 10.9929 16 7 16.1071 20.0
0.4237 69.0 1104 1.6777 0.5956 0.3787 0.554 0.5535 0.8975 0.9024 11.2 16 7 16.3571 22.8571
0.4265 70.0 1120 1.6787 0.5907 0.3713 0.552 0.5516 0.8964 0.9011 11.1429 15 7 16.3286 20.7143
0.4219 71.0 1136 1.6846 0.5868 0.3699 0.5487 0.5481 0.895 0.9004 11.2357 15 7 16.4214 22.1429
0.4237 72.0 1152 1.6891 0.5895 0.3702 0.5506 0.5503 0.895 0.9013 11.2643 15 7 16.5071 24.2857
0.4146 73.0 1168 1.6914 0.5963 0.3772 0.5552 0.5548 0.8967 0.9019 11.2429 15 7 16.4357 22.1429
0.4103 74.0 1184 1.6948 0.5878 0.372 0.5496 0.549 0.8962 0.9006 11.2 16 7 16.3571 21.4286
0.4099 75.0 1200 1.6970 0.5932 0.3755 0.5566 0.5563 0.897 0.9021 11.2357 16 7 16.3286 22.8571
0.403 76.0 1216 1.6966 0.5922 0.3768 0.5545 0.5542 0.8987 0.9026 11.0857 16 7 16.2214 20.7143
0.3999 77.0 1232 1.6991 0.5946 0.3778 0.5549 0.5552 0.8986 0.9026 11.1 16 6 16.2286 21.4286
0.4176 78.0 1248 1.7002 0.5963 0.3783 0.5568 0.5571 0.8984 0.9032 11.1643 16 6 16.3286 23.5714
0.4007 79.0 1264 1.7038 0.5921 0.3729 0.553 0.5529 0.8976 0.9015 11.1071 16 6 16.2286 22.1429
0.3918 80.0 1280 1.7114 0.595 0.3745 0.5551 0.5544 0.8982 0.9021 11.1714 16 6 16.3071 22.1429
0.3936 81.0 1296 1.7153 0.5914 0.3724 0.5527 0.5524 0.8979 0.9014 11.0929 16 6 16.2286 21.4286
0.3997 82.0 1312 1.7154 0.5924 0.3755 0.5528 0.5527 0.8972 0.9021 11.2286 16 6 16.3571 23.5714
0.396 83.0 1328 1.7187 0.5943 0.3765 0.5552 0.5549 0.897 0.9029 11.3 16 6 16.4571 24.2857
0.4049 84.0 1344 1.7198 0.5958 0.3757 0.5555 0.5548 0.8972 0.9031 11.2929 16 6 16.4929 25.0
0.3983 85.0 1360 1.7201 0.5948 0.3776 0.558 0.5574 0.8974 0.9025 11.25 16 6 16.4286 24.2857
0.3936 86.0 1376 1.7211 0.5945 0.3764 0.5579 0.5572 0.8973 0.9028 11.2286 16 6 16.4143 24.2857
0.3847 87.0 1392 1.7211 0.5965 0.3808 0.5582 0.5583 0.8977 0.9032 11.1929 16 6 16.3857 24.2857
0.3979 88.0 1408 1.7227 0.5928 0.374 0.5552 0.555 0.8973 0.902 11.15 16 6 16.3286 22.8571
0.3851 89.0 1424 1.7262 0.5908 0.3731 0.5538 0.5532 0.8978 0.9016 11.1143 16 6 16.25 20.7143
0.3762 90.0 1440 1.7262 0.591 0.3726 0.5542 0.5536 0.8975 0.9017 11.1214 16 6 16.3 20.0
0.3752 91.0 1456 1.7250 0.5924 0.3756 0.5555 0.555 0.8969 0.902 11.2 16 6 16.3571 23.5714
0.3825 92.0 1472 1.7273 0.5905 0.3728 0.5542 0.5541 0.8968 0.9015 11.2357 16 6 16.3714 23.5714
0.3731 93.0 1488 1.7295 0.5916 0.373 0.5541 0.5536 0.8972 0.902 11.2143 16 7 16.4286 22.1429
0.3707 94.0 1504 1.7313 0.5928 0.3746 0.5548 0.5545 0.8975 0.9023 11.1786 16 7 16.3929 22.1429
0.3708 95.0 1520 1.7323 0.5919 0.3737 0.5536 0.5536 0.8972 0.9019 11.2071 16 7 16.4286 22.1429
0.372 96.0 1536 1.7334 0.5914 0.3737 0.5537 0.5536 0.8971 0.9019 11.2 16 7 16.4 22.1429
0.3754 97.0 1552 1.7339 0.5905 0.3733 0.5527 0.5521 0.8975 0.9018 11.1714 16 7 16.3857 21.4286
0.3829 98.0 1568 1.7342 0.5923 0.3779 0.5548 0.5545 0.8979 0.9025 11.1571 16 7 16.3786 22.1429
0.3723 99.0 1584 1.7343 0.5936 0.3795 0.5556 0.5549 0.8978 0.9026 11.1643 16 7 16.4 22.1429
0.3846 100.0 1600 1.7343 0.5944 0.3803 0.5562 0.5556 0.8982 0.9028 11.1571 16 7 16.4 22.1429

Framework versions

  • Transformers 4.33.0
  • Pytorch 2.0.1+cu118
  • Datasets 2.14.4
  • Tokenizers 0.13.3