--- license: apache-2.0 tags: - generated_from_trainer metrics: - rouge model-index: - name: bart-qmsum-meeting-summarization results: [] datasets: - yawnick/QMSum --- # bart-qmsum-meeting-summarization This model is a fine-tuned version of [sshleifer/distilbart-cnn-12-6](https://huggingface.co/sshleifer/distilbart-cnn-12-6) on the QMSum dataset. It achieves the following results on the evaluation set: - Loss: 4.3354 - Rouge1: 39.5539 - Rouge2: 12.1134 - Rougel: 23.9163 - Rougelsum: 36.0299 - Gen Len: 117.225 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 3e-07 - train_batch_size: 4 - eval_batch_size: 4 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 500 - num_epochs: 200 - label_smoothing_factor: 0.1 ### Training results | Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len | |:-------------:|:------:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:| | 5.5573 | 2.17 | 100 | 5.4074 | 23.6282 | 4.1122 | 14.584 | 21.2263 | 84.75 | | 5.4721 | 4.35 | 200 | 5.2899 | 24.61 | 4.272 | 15.2096 | 22.2997 | 87.2 | | 5.3407 | 6.52 | 300 | 5.1360 | 25.8272 | 4.3314 | 15.9926 | 23.3416 | 87.95 | | 5.1527 | 8.7 | 400 | 4.9751 | 27.7207 | 5.31 | 16.7055 | 24.8357 | 88.35 | | 5.0058 | 10.87 | 500 | 4.8372 | 30.1847 | 6.8615 | 18.934 | 27.2424 | 89.95 | | 4.8807 | 13.04 | 600 | 4.7488 | 33.1208 | 9.1784 | 20.655 | 30.1198 | 101.3 | | 4.7931 | 15.22 | 700 | 4.6891 | 33.2266 | 8.4253 | 20.0334 | 30.4093 | 108.925 | | 4.7272 | 17.39 | 800 | 4.6467 | 35.0475 | 9.326 | 21.0655 | 31.8413 | 111.7 | | 4.6904 | 19.57 | 900 | 4.6102 | 34.869 | 9.6046 | 21.395 | 32.4346 | 115.05 | | 4.6547 | 21.74 | 1000 | 4.5829 | 36.3392 | 10.9936 | 22.1524 | 33.6863 | 119.875 | | 4.594 | 23.91 | 1100 | 4.5602 | 35.9717 | 10.3827 | 21.6118 | 32.8302 | 119.5 | | 4.5714 | 26.09 | 1200 | 4.5424 | 36.3656 | 10.6282 | 22.2187 | 33.6494 | 118.0 | | 4.542 | 28.26 | 1300 | 4.5256 | 36.7386 | 10.615 | 22.2487 | 34.1927 | 115.675 | | 4.5092 | 30.43 | 1400 | 4.5116 | 37.1597 | 10.7751 | 22.6747 | 34.396 | 118.55 | | 4.5031 | 32.61 | 1500 | 4.4981 | 37.6108 | 10.9732 | 22.8342 | 34.6833 | 117.125 | | 4.4682 | 34.78 | 1600 | 4.4875 | 37.5057 | 11.1328 | 22.8973 | 34.7114 | 117.65 | | 4.4387 | 36.96 | 1700 | 4.4775 | 38.1278 | 11.3597 | 23.1307 | 35.1869 | 115.65 | | 4.4085 | 39.13 | 1800 | 4.4682 | 37.9578 | 11.4355 | 23.1149 | 35.4961 | 119.6 | | 4.4166 | 41.3 | 1900 | 4.4592 | 38.1467 | 11.3208 | 23.045 | 35.0824 | 120.05 | | 4.3971 | 43.48 | 2000 | 4.4517 | 37.9922 | 11.5071 | 23.3983 | 34.6918 | 114.425 | | 4.3638 | 45.65 | 2100 | 4.4438 | 38.1666 | 11.4985 | 23.5518 | 35.1484 | 117.2 | | 4.3522 | 47.83 | 2200 | 4.4377 | 37.7572 | 11.3984 | 23.4437 | 35.0453 | 113.725 | | 4.3398 | 50.0 | 2300 | 4.4320 | 38.5833 | 11.4575 | 23.6411 | 35.3437 | 116.125 | | 4.3341 | 52.17 | 2400 | 4.4247 | 38.2705 | 12.0374 | 23.5807 | 34.9985 | 110.8 | | 4.3024 | 54.35 | 2500 | 4.4201 | 39.0206 | 12.2041 | 23.4394 | 35.6291 | 114.5 | | 4.3117 | 56.52 | 2600 | 4.4147 | 38.6555 | 12.1079 | 23.5655 | 35.5287 | 111.325 | | 4.2659 | 58.7 | 2700 | 4.4107 | 39.2235 | 12.025 | 23.934 | 36.2243 | 113.3 | | 4.2946 | 60.87 | 2800 | 4.4055 | 39.0301 | 12.1833 | 23.8999 | 36.0487 | 110.325 | | 4.2431 | 63.04 | 2900 | 4.4009 | 39.0498 | 12.3215 | 23.9686 | 36.0277 | 112.775 | | 4.2439 | 65.22 | 3000 | 4.3968 | 38.8786 | 12.0985 | 23.8308 | 35.8575 | 115.175 | | 4.2244 | 67.39 | 3100 | 4.3922 | 38.7614 | 12.1721 | 23.7736 | 35.6744 | 113.55 | | 4.235 | 69.57 | 3200 | 4.3895 | 38.6858 | 11.3994 | 23.6392 | 35.3456 | 114.125 | | 4.2064 | 71.74 | 3300 | 4.3859 | 39.0258 | 12.0435 | 24.2528 | 35.8378 | 113.5 | | 4.1934 | 73.91 | 3400 | 4.3835 | 39.0467 | 11.5556 | 23.6704 | 35.5643 | 111.5 | | 4.1859 | 76.09 | 3500 | 4.3800 | 38.776 | 11.729 | 24.1254 | 35.3894 | 112.9 | | 4.1762 | 78.26 | 3600 | 4.3775 | 38.9465 | 11.9112 | 23.8123 | 35.5453 | 114.125 | | 4.1848 | 80.43 | 3700 | 4.3744 | 39.2783 | 11.6539 | 23.8236 | 35.8465 | 110.225 | | 4.1386 | 82.61 | 3800 | 4.3730 | 38.8894 | 11.4784 | 23.7534 | 35.5464 | 113.15 | | 4.1483 | 84.78 | 3900 | 4.3710 | 39.2734 | 12.0285 | 23.8171 | 35.6884 | 115.95 | | 4.1428 | 86.96 | 4000 | 4.3688 | 39.6134 | 12.0616 | 23.7454 | 36.0363 | 113.375 | | 4.133 | 89.13 | 4100 | 4.3663 | 38.935 | 11.4781 | 23.8766 | 35.4061 | 114.15 | | 4.1211 | 91.3 | 4200 | 4.3648 | 39.1488 | 11.8399 | 23.9935 | 35.3107 | 113.975 | | 4.1076 | 93.48 | 4300 | 4.3650 | 38.9764 | 11.9963 | 23.4994 | 35.7214 | 116.25 | | 4.121 | 95.65 | 4400 | 4.3597 | 38.9418 | 11.8416 | 24.0272 | 35.6597 | 111.325 | | 4.0936 | 97.83 | 4500 | 4.3602 | 39.266 | 12.5616 | 24.2046 | 36.1883 | 114.275 | | 4.0841 | 100.0 | 4600 | 4.3588 | 39.4659 | 12.2132 | 24.0521 | 36.249 | 115.475 | | 4.0768 | 102.17 | 4700 | 4.3578 | 39.4167 | 12.0587 | 24.025 | 35.9668 | 114.375 | | 4.0711 | 104.35 | 4800 | 4.3541 | 39.6943 | 12.1095 | 24.0925 | 36.3496 | 115.65 | | 4.072 | 106.52 | 4900 | 4.3539 | 40.2024 | 12.4618 | 24.2863 | 36.8844 | 113.475 | | 4.0646 | 108.7 | 5000 | 4.3540 | 39.4299 | 11.8085 | 23.686 | 36.0454 | 113.975 | | 4.0508 | 110.87 | 5100 | 4.3517 | 39.9217 | 11.9379 | 24.2299 | 36.6362 | 115.5 | | 4.0549 | 113.04 | 5200 | 4.3498 | 40.3496 | 12.2558 | 24.0271 | 36.9715 | 112.5 | | 4.0428 | 115.22 | 5300 | 4.3497 | 40.1349 | 12.0628 | 24.0622 | 36.9169 | 113.95 | | 4.0391 | 117.39 | 5400 | 4.3480 | 40.1209 | 12.3587 | 24.3456 | 36.8411 | 116.025 | | 4.0195 | 119.57 | 5500 | 4.3474 | 39.5209 | 12.1325 | 24.2622 | 36.4357 | 111.975 | | 4.0054 | 121.74 | 5600 | 4.3468 | 40.2885 | 12.4453 | 24.2373 | 36.932 | 117.375 | | 4.0286 | 123.91 | 5700 | 4.3465 | 39.3943 | 11.8399 | 23.9786 | 35.991 | 116.475 | | 4.005 | 126.09 | 5800 | 4.3459 | 38.7442 | 11.7408 | 23.8948 | 35.3673 | 117.625 | | 3.991 | 128.26 | 5900 | 4.3444 | 39.6276 | 12.1549 | 23.9542 | 36.3832 | 115.675 | | 4.0137 | 130.43 | 6000 | 4.3427 | 39.8331 | 12.2687 | 24.187 | 36.6144 | 115.475 | | 3.9755 | 132.61 | 6100 | 4.3438 | 39.1907 | 12.1033 | 24.2339 | 35.9126 | 114.525 | | 4.0134 | 134.78 | 6200 | 4.3422 | 39.4298 | 11.862 | 24.0847 | 35.5744 | 115.025 | | 3.9935 | 136.96 | 6300 | 4.3416 | 39.4158 | 11.6968 | 23.9636 | 35.8155 | 114.35 | | 3.9606 | 139.13 | 6400 | 4.3409 | 39.1239 | 11.7046 | 23.6846 | 36.0431 | 114.775 | | 3.9834 | 141.3 | 6500 | 4.3404 | 39.6375 | 12.2746 | 24.2636 | 36.1425 | 116.175 | | 3.9687 | 143.48 | 6600 | 4.3409 | 39.1494 | 12.1404 | 24.0778 | 35.4932 | 118.05 | | 3.9861 | 145.65 | 6700 | 4.3394 | 39.6258 | 12.2497 | 23.9662 | 36.4054 | 116.8 | | 3.9755 | 147.83 | 6800 | 4.3400 | 39.3121 | 11.7831 | 23.6584 | 35.9636 | 118.125 | | 3.9591 | 150.0 | 6900 | 4.3390 | 39.6957 | 11.9406 | 24.0599 | 36.3021 | 114.9 | | 3.9599 | 152.17 | 7000 | 4.3389 | 39.4271 | 11.4159 | 24.1437 | 35.9056 | 115.8 | | 3.9456 | 154.35 | 7100 | 4.3384 | 39.4862 | 11.726 | 23.883 | 35.9839 | 116.375 | | 3.9341 | 156.52 | 7200 | 4.3386 | 39.6915 | 11.8028 | 24.346 | 36.406 | 116.425 | | 3.9648 | 158.7 | 7300 | 4.3383 | 39.9311 | 11.7135 | 23.985 | 36.2617 | 118.075 | | 3.9486 | 160.87 | 7400 | 4.3372 | 39.8375 | 12.0014 | 24.0969 | 36.5902 | 118.8 | | 3.9533 | 163.04 | 7500 | 4.3371 | 40.2678 | 12.3137 | 24.1916 | 37.1632 | 118.075 | | 3.9344 | 165.22 | 7600 | 4.3369 | 39.5588 | 11.6805 | 24.1474 | 36.2021 | 114.875 | | 3.9314 | 167.39 | 7700 | 4.3368 | 39.8649 | 11.9824 | 24.5459 | 36.3921 | 113.65 | | 3.9558 | 169.57 | 7800 | 4.3363 | 39.8428 | 12.0892 | 24.0175 | 36.67 | 112.7 | | 3.928 | 171.74 | 7900 | 4.3364 | 39.2281 | 11.8456 | 23.7212 | 36.2005 | 113.95 | | 3.9351 | 173.91 | 8000 | 4.3363 | 39.9798 | 12.4387 | 23.7687 | 36.6472 | 115.45 | | 3.9326 | 176.09 | 8100 | 4.3363 | 39.9772 | 12.1193 | 24.1518 | 36.5791 | 117.4 | | 3.9387 | 178.26 | 8200 | 4.3363 | 39.8629 | 12.1719 | 23.9446 | 36.345 | 115.075 | | 3.9204 | 180.43 | 8300 | 4.3358 | 39.9738 | 12.3072 | 23.8641 | 36.4802 | 116.3 | | 3.9418 | 182.61 | 8400 | 4.3357 | 40.1451 | 12.4144 | 24.1553 | 36.4251 | 116.025 | | 3.9289 | 184.78 | 8500 | 4.3357 | 39.7241 | 12.0543 | 24.0752 | 36.0847 | 115.8 | | 3.9176 | 186.96 | 8600 | 4.3358 | 39.7969 | 12.0967 | 24.123 | 36.2664 | 118.6 | | 3.9097 | 189.13 | 8700 | 4.3356 | 39.4096 | 11.9872 | 24.0609 | 35.8662 | 117.2 | | 3.938 | 191.3 | 8800 | 4.3354 | 39.4695 | 11.9343 | 24.0295 | 35.9372 | 117.025 | | 3.9239 | 193.48 | 8900 | 4.3352 | 39.3231 | 12.0965 | 23.9131 | 35.9555 | 117.275 | | 3.91 | 195.65 | 9000 | 4.3354 | 39.5932 | 12.1808 | 23.9233 | 36.0864 | 116.925 | | 3.9234 | 197.83 | 9100 | 4.3354 | 39.5539 | 12.1134 | 23.9163 | 36.0299 | 117.225 | | 3.9263 | 200.0 | 9200 | 4.3354 | 39.5539 | 12.1134 | 23.9163 | 36.0299 | 117.225 | ### Framework versions - Transformers 4.18.0 - Pytorch 1.11.0+cu113 - Datasets 2.1.0 - Tokenizers 0.12.1