Edit model card

bart-qmsum-meeting-summarization

This model is a fine-tuned version of sshleifer/distilbart-cnn-12-6 on the QMSum dataset. It achieves the following results on the evaluation set:

  • Loss: 4.3354
  • Rouge1: 39.5539
  • Rouge2: 12.1134
  • Rougel: 23.9163
  • Rougelsum: 36.0299
  • Gen Len: 117.225

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-07
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • num_epochs: 200
  • label_smoothing_factor: 0.1

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
5.5573 2.17 100 5.4074 23.6282 4.1122 14.584 21.2263 84.75
5.4721 4.35 200 5.2899 24.61 4.272 15.2096 22.2997 87.2
5.3407 6.52 300 5.1360 25.8272 4.3314 15.9926 23.3416 87.95
5.1527 8.7 400 4.9751 27.7207 5.31 16.7055 24.8357 88.35
5.0058 10.87 500 4.8372 30.1847 6.8615 18.934 27.2424 89.95
4.8807 13.04 600 4.7488 33.1208 9.1784 20.655 30.1198 101.3
4.7931 15.22 700 4.6891 33.2266 8.4253 20.0334 30.4093 108.925
4.7272 17.39 800 4.6467 35.0475 9.326 21.0655 31.8413 111.7
4.6904 19.57 900 4.6102 34.869 9.6046 21.395 32.4346 115.05
4.6547 21.74 1000 4.5829 36.3392 10.9936 22.1524 33.6863 119.875
4.594 23.91 1100 4.5602 35.9717 10.3827 21.6118 32.8302 119.5
4.5714 26.09 1200 4.5424 36.3656 10.6282 22.2187 33.6494 118.0
4.542 28.26 1300 4.5256 36.7386 10.615 22.2487 34.1927 115.675
4.5092 30.43 1400 4.5116 37.1597 10.7751 22.6747 34.396 118.55
4.5031 32.61 1500 4.4981 37.6108 10.9732 22.8342 34.6833 117.125
4.4682 34.78 1600 4.4875 37.5057 11.1328 22.8973 34.7114 117.65
4.4387 36.96 1700 4.4775 38.1278 11.3597 23.1307 35.1869 115.65
4.4085 39.13 1800 4.4682 37.9578 11.4355 23.1149 35.4961 119.6
4.4166 41.3 1900 4.4592 38.1467 11.3208 23.045 35.0824 120.05
4.3971 43.48 2000 4.4517 37.9922 11.5071 23.3983 34.6918 114.425
4.3638 45.65 2100 4.4438 38.1666 11.4985 23.5518 35.1484 117.2
4.3522 47.83 2200 4.4377 37.7572 11.3984 23.4437 35.0453 113.725
4.3398 50.0 2300 4.4320 38.5833 11.4575 23.6411 35.3437 116.125
4.3341 52.17 2400 4.4247 38.2705 12.0374 23.5807 34.9985 110.8
4.3024 54.35 2500 4.4201 39.0206 12.2041 23.4394 35.6291 114.5
4.3117 56.52 2600 4.4147 38.6555 12.1079 23.5655 35.5287 111.325
4.2659 58.7 2700 4.4107 39.2235 12.025 23.934 36.2243 113.3
4.2946 60.87 2800 4.4055 39.0301 12.1833 23.8999 36.0487 110.325
4.2431 63.04 2900 4.4009 39.0498 12.3215 23.9686 36.0277 112.775
4.2439 65.22 3000 4.3968 38.8786 12.0985 23.8308 35.8575 115.175
4.2244 67.39 3100 4.3922 38.7614 12.1721 23.7736 35.6744 113.55
4.235 69.57 3200 4.3895 38.6858 11.3994 23.6392 35.3456 114.125
4.2064 71.74 3300 4.3859 39.0258 12.0435 24.2528 35.8378 113.5
4.1934 73.91 3400 4.3835 39.0467 11.5556 23.6704 35.5643 111.5
4.1859 76.09 3500 4.3800 38.776 11.729 24.1254 35.3894 112.9
4.1762 78.26 3600 4.3775 38.9465 11.9112 23.8123 35.5453 114.125
4.1848 80.43 3700 4.3744 39.2783 11.6539 23.8236 35.8465 110.225
4.1386 82.61 3800 4.3730 38.8894 11.4784 23.7534 35.5464 113.15
4.1483 84.78 3900 4.3710 39.2734 12.0285 23.8171 35.6884 115.95
4.1428 86.96 4000 4.3688 39.6134 12.0616 23.7454 36.0363 113.375
4.133 89.13 4100 4.3663 38.935 11.4781 23.8766 35.4061 114.15
4.1211 91.3 4200 4.3648 39.1488 11.8399 23.9935 35.3107 113.975
4.1076 93.48 4300 4.3650 38.9764 11.9963 23.4994 35.7214 116.25
4.121 95.65 4400 4.3597 38.9418 11.8416 24.0272 35.6597 111.325
4.0936 97.83 4500 4.3602 39.266 12.5616 24.2046 36.1883 114.275
4.0841 100.0 4600 4.3588 39.4659 12.2132 24.0521 36.249 115.475
4.0768 102.17 4700 4.3578 39.4167 12.0587 24.025 35.9668 114.375
4.0711 104.35 4800 4.3541 39.6943 12.1095 24.0925 36.3496 115.65
4.072 106.52 4900 4.3539 40.2024 12.4618 24.2863 36.8844 113.475
4.0646 108.7 5000 4.3540 39.4299 11.8085 23.686 36.0454 113.975
4.0508 110.87 5100 4.3517 39.9217 11.9379 24.2299 36.6362 115.5
4.0549 113.04 5200 4.3498 40.3496 12.2558 24.0271 36.9715 112.5
4.0428 115.22 5300 4.3497 40.1349 12.0628 24.0622 36.9169 113.95
4.0391 117.39 5400 4.3480 40.1209 12.3587 24.3456 36.8411 116.025
4.0195 119.57 5500 4.3474 39.5209 12.1325 24.2622 36.4357 111.975
4.0054 121.74 5600 4.3468 40.2885 12.4453 24.2373 36.932 117.375
4.0286 123.91 5700 4.3465 39.3943 11.8399 23.9786 35.991 116.475
4.005 126.09 5800 4.3459 38.7442 11.7408 23.8948 35.3673 117.625
3.991 128.26 5900 4.3444 39.6276 12.1549 23.9542 36.3832 115.675
4.0137 130.43 6000 4.3427 39.8331 12.2687 24.187 36.6144 115.475
3.9755 132.61 6100 4.3438 39.1907 12.1033 24.2339 35.9126 114.525
4.0134 134.78 6200 4.3422 39.4298 11.862 24.0847 35.5744 115.025
3.9935 136.96 6300 4.3416 39.4158 11.6968 23.9636 35.8155 114.35
3.9606 139.13 6400 4.3409 39.1239 11.7046 23.6846 36.0431 114.775
3.9834 141.3 6500 4.3404 39.6375 12.2746 24.2636 36.1425 116.175
3.9687 143.48 6600 4.3409 39.1494 12.1404 24.0778 35.4932 118.05
3.9861 145.65 6700 4.3394 39.6258 12.2497 23.9662 36.4054 116.8
3.9755 147.83 6800 4.3400 39.3121 11.7831 23.6584 35.9636 118.125
3.9591 150.0 6900 4.3390 39.6957 11.9406 24.0599 36.3021 114.9
3.9599 152.17 7000 4.3389 39.4271 11.4159 24.1437 35.9056 115.8
3.9456 154.35 7100 4.3384 39.4862 11.726 23.883 35.9839 116.375
3.9341 156.52 7200 4.3386 39.6915 11.8028 24.346 36.406 116.425
3.9648 158.7 7300 4.3383 39.9311 11.7135 23.985 36.2617 118.075
3.9486 160.87 7400 4.3372 39.8375 12.0014 24.0969 36.5902 118.8
3.9533 163.04 7500 4.3371 40.2678 12.3137 24.1916 37.1632 118.075
3.9344 165.22 7600 4.3369 39.5588 11.6805 24.1474 36.2021 114.875
3.9314 167.39 7700 4.3368 39.8649 11.9824 24.5459 36.3921 113.65
3.9558 169.57 7800 4.3363 39.8428 12.0892 24.0175 36.67 112.7
3.928 171.74 7900 4.3364 39.2281 11.8456 23.7212 36.2005 113.95
3.9351 173.91 8000 4.3363 39.9798 12.4387 23.7687 36.6472 115.45
3.9326 176.09 8100 4.3363 39.9772 12.1193 24.1518 36.5791 117.4
3.9387 178.26 8200 4.3363 39.8629 12.1719 23.9446 36.345 115.075
3.9204 180.43 8300 4.3358 39.9738 12.3072 23.8641 36.4802 116.3
3.9418 182.61 8400 4.3357 40.1451 12.4144 24.1553 36.4251 116.025
3.9289 184.78 8500 4.3357 39.7241 12.0543 24.0752 36.0847 115.8
3.9176 186.96 8600 4.3358 39.7969 12.0967 24.123 36.2664 118.6
3.9097 189.13 8700 4.3356 39.4096 11.9872 24.0609 35.8662 117.2
3.938 191.3 8800 4.3354 39.4695 11.9343 24.0295 35.9372 117.025
3.9239 193.48 8900 4.3352 39.3231 12.0965 23.9131 35.9555 117.275
3.91 195.65 9000 4.3354 39.5932 12.1808 23.9233 36.0864 116.925
3.9234 197.83 9100 4.3354 39.5539 12.1134 23.9163 36.0299 117.225
3.9263 200.0 9200 4.3354 39.5539 12.1134 23.9163 36.0299 117.225

Framework versions

  • Transformers 4.18.0
  • Pytorch 1.11.0+cu113
  • Datasets 2.1.0
  • Tokenizers 0.12.1
Downloads last month
29
Hosted inference API
This model can be loaded on the Inference API on-demand.

Dataset used to train mikeadimech/bart-qmsum-meeting-summarization

Evaluation results