Edit model card

longformer-qmsum-meeting-summarization

This model is a fine-tuned version of allenai/led-base-16384 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 4.2055
  • Rouge1: 20.5333
  • Rouge2: 7.6756
  • Rougel: 16.2531
  • Rougelsum: 19.0336
  • Gen Len: 20.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-07
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • num_epochs: 200
  • label_smoothing_factor: 0.1

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
5.4071 1.09 100 5.2910 6.012 0.5556 4.936 5.6141 20.0
5.269 2.17 200 5.1446 6.7419 0.9713 5.2774 6.3003 20.0
5.1153 3.26 300 4.9976 8.1369 1.2365 6.391 7.5911 20.0
4.9888 4.35 400 4.8763 9.9113 1.4239 8.0574 9.3442 20.0
4.8687 5.43 500 4.7889 10.504 1.5638 8.1191 9.817 20.0
4.7936 6.52 600 4.7226 12.6475 2.4733 9.968 11.541 20.0
4.713 7.61 700 4.6770 15.2998 3.6209 11.8629 14.2323 20.0
4.6843 8.7 800 4.6428 15.8299 4.4128 12.7301 14.8795 20.0
4.6453 9.78 900 4.6105 16.3702 4.7356 13.1566 15.4497 20.0
4.6212 10.87 1000 4.5849 16.9765 5.1101 13.617 15.9401 20.0
4.5761 11.96 1100 4.5649 17.3024 5.2494 13.79 16.3173 20.0
4.564 13.04 1200 4.5447 18.7699 6.2331 14.8264 17.645 20.0
4.5393 14.13 1300 4.5277 19.1495 6.6082 15.1392 18.2546 20.0
4.5069 15.22 1400 4.5132 20.3648 7.3895 16.018 19.1503 20.0
4.4985 16.3 1500 4.4973 20.165 7.3477 16.1161 18.7585 20.0
4.4476 17.39 1600 4.4859 20.4691 7.5734 16.438 19.1045 20.0
4.4421 18.48 1700 4.4758 20.4402 7.7674 16.3998 19.1045 20.0
4.4554 19.57 1800 4.4648 20.5992 7.3522 16.185 19.2869 20.0
4.4138 20.65 1900 4.4560 20.497 7.1732 16.2177 19.0912 20.0
4.4447 21.74 2000 4.4465 21.2936 7.8856 16.8994 19.7994 20.0
4.3636 22.83 2100 4.4373 21.1015 7.6466 16.787 19.6918 20.0
4.3647 23.91 2200 4.4288 21.3408 7.8052 17.1431 20.1456 20.0
4.3707 25.0 2300 4.4217 21.523 8.017 17.1586 20.2724 20.0
4.3503 26.09 2400 4.4145 21.485 8.015 17.064 20.209 20.0
4.3295 27.17 2500 4.4069 21.5167 7.6749 16.9976 20.265 20.0
4.3444 28.26 2600 4.4004 21.748 7.8808 17.1592 20.4054 20.0
4.3135 29.35 2700 4.3958 21.5523 7.5449 17.2103 20.5405 20.0
4.3028 30.43 2800 4.3880 21.3016 7.6531 17.1515 20.3301 20.0
4.3406 31.52 2900 4.3834 21.4169 7.5647 16.9477 20.3379 20.0
4.286 32.61 3000 4.3760 21.4684 7.4776 17.1018 20.5254 20.0
4.2717 33.7 3100 4.3736 21.596 7.514 17.164 20.6272 20.0
4.285 34.78 3200 4.3666 21.3495 7.676 17.0703 20.3182 20.0
4.2496 35.87 3300 4.3628 21.5539 7.6574 17.1393 20.5116 20.0
4.2618 36.96 3400 4.3591 21.08 7.6814 16.6941 20.2386 20.0
4.255 38.04 3500 4.3522 21.1979 7.7334 16.8281 20.3095 20.0
4.2353 39.13 3600 4.3502 21.1162 8.0427 16.9948 20.3903 20.0
4.2556 40.22 3700 4.3462 21.3417 7.7851 16.6548 20.5316 20.0
4.207 41.3 3800 4.3401 21.4329 7.948 16.944 20.5075 20.0
4.234 42.39 3900 4.3388 21.6109 8.033 16.9375 20.6668 20.0
4.2118 43.48 4000 4.3347 21.5051 7.9239 16.7403 20.6123 20.0
4.1898 44.57 4100 4.3319 21.2644 7.8222 16.7109 20.3999 20.0
4.1951 45.65 4200 4.3265 21.3383 7.997 16.7605 20.4542 20.0
4.1851 46.74 4300 4.3248 21.3509 7.9038 16.9098 20.4593 20.0
4.1674 47.83 4400 4.3223 21.3516 8.0058 17.0061 20.4199 20.0
4.1785 48.91 4500 4.3182 21.4118 8.0755 16.959 20.5154 20.0
4.1599 50.0 4600 4.3175 21.2748 7.8562 16.8107 20.3536 20.0
4.1564 51.09 4700 4.3141 21.1811 7.8563 16.7687 20.2242 20.0
4.1513 52.17 4800 4.3101 21.1557 7.6616 16.8105 19.8191 20.0
4.1234 53.26 4900 4.3083 21.0718 7.8625 16.7849 20.0014 20.0
4.1532 54.35 5000 4.3041 21.4241 7.984 16.6561 20.3073 20.0
4.1371 55.43 5100 4.3035 21.259 7.6476 16.9931 20.3421 20.0
4.1342 56.52 5200 4.3009 21.0745 7.386 16.7976 20.1148 20.0
4.1146 57.61 5300 4.2985 21.0796 7.6743 16.5062 19.8702 20.0
4.0774 58.7 5400 4.2965 21.2129 7.2871 17.0019 20.3176 20.0
4.1726 59.78 5500 4.2930 21.159 7.4045 16.7762 19.9886 20.0
4.0931 60.87 5600 4.2900 20.957 7.2307 16.784 19.8402 20.0
4.0838 61.96 5700 4.2887 21.13 7.2664 16.7837 19.951 20.0
4.0878 63.04 5800 4.2853 21.0281 7.2664 16.6847 19.7843 20.0
4.1067 64.13 5900 4.2848 20.941 7.2307 16.74 19.8262 20.0
4.0743 65.22 6000 4.2817 21.1234 7.4612 16.755 20.027 20.0
4.103 66.3 6100 4.2807 21.2852 7.4802 16.8037 20.2316 20.0
4.0434 67.39 6200 4.2777 21.236 7.3169 16.7967 20.0534 20.0
4.0829 68.48 6300 4.2793 20.947 7.3164 16.8597 19.7938 20.0
4.0619 69.57 6400 4.2736 21.4626 7.7245 16.8395 20.2035 20.0
4.079 70.65 6500 4.2729 21.163 7.6397 16.7826 20.0295 20.0
4.0411 71.74 6600 4.2721 20.8673 7.3841 16.6784 19.6854 20.0
4.046 72.83 6700 4.2697 20.9774 7.3325 16.7779 19.761 20.0
4.0384 73.91 6800 4.2684 21.0736 7.6569 16.7631 19.992 20.0
4.0401 75.0 6900 4.2670 21.2708 7.8224 16.5649 20.2364 20.0
4.0153 76.09 7000 4.2669 21.3638 7.7586 16.765 19.9744 20.0
4.0227 77.17 7100 4.2652 21.0611 7.709 16.3201 20.0516 20.0
4.0264 78.26 7200 4.2634 21.3766 7.7666 16.7508 20.0938 20.0
4.0475 79.35 7300 4.2615 21.2356 7.5533 16.6339 19.9254 20.0
4.0145 80.43 7400 4.2580 20.7689 7.3386 16.287 19.7335 20.0
4.0087 81.52 7500 4.2580 20.9816 7.343 16.4598 19.701 20.0
3.9835 82.61 7600 4.2577 21.1001 7.5887 16.5226 19.714 20.0
4.0029 83.7 7700 4.2562 21.1875 7.7333 16.4799 19.9907 20.0
3.9912 84.78 7800 4.2549 20.8265 7.3897 16.2191 19.4398 20.0
4.008 85.87 7900 4.2541 21.4955 7.7602 16.4989 20.1402 20.0
3.9659 86.96 8000 4.2523 21.687 7.9463 16.5832 20.1598 20.0
3.9923 88.04 8100 4.2505 21.4615 7.817 16.3628 19.9159 20.0
3.9811 89.13 8200 4.2498 21.1917 7.5813 16.3066 19.4905 20.0
3.9819 90.22 8300 4.2488 21.239 7.4585 16.4297 19.5213 20.0
3.9889 91.3 8400 4.2456 21.5052 7.7994 16.3783 19.8739 20.0
3.942 92.39 8500 4.2468 21.3482 7.7517 16.34 19.764 20.0
3.9959 93.48 8600 4.2446 21.4615 7.817 16.3628 19.9159 20.0
3.987 94.57 8700 4.2438 21.1265 7.6497 16.4132 19.5981 20.0
3.9803 95.65 8800 4.2420 21.2956 7.7796 16.3643 19.8607 20.0
3.9415 96.74 8900 4.2410 20.8332 7.5468 16.1678 19.316 20.0
3.97 97.83 9000 4.2407 21.4223 7.8688 16.533 19.8081 20.0
3.9495 98.91 9100 4.2400 21.5678 7.9698 16.5492 19.9404 20.0
3.9489 100.0 9200 4.2391 21.3928 7.8416 16.3595 19.7579 20.0
3.9194 101.09 9300 4.2394 21.2216 7.8416 16.2499 19.5661 20.0
3.966 102.17 9400 4.2372 21.2756 7.8798 16.3124 19.6303 20.0
3.934 103.26 9500 4.2367 21.3106 7.8585 16.3937 19.7289 20.0
3.9316 104.35 9600 4.2349 21.3296 7.9392 16.3574 19.8031 20.0
3.9586 105.43 9700 4.2366 21.0662 7.771 16.2242 19.4813 20.0
3.9189 106.52 9800 4.2338 21.1348 7.8414 16.2757 19.7301 20.0
3.937 107.61 9900 4.2350 21.2434 7.7611 16.4693 19.6923 20.0
3.911 108.7 10000 4.2331 21.2697 7.8282 16.3636 19.6627 20.0
3.8956 109.78 10100 4.2312 21.2697 7.8117 16.3636 19.6321 20.0
3.9396 110.87 10200 4.2303 21.0842 7.7105 16.221 19.4378 20.0
3.9058 111.96 10300 4.2290 21.1633 7.8117 16.3196 19.5575 20.0
3.9198 113.04 10400 4.2278 21.1633 7.8117 16.3196 19.5311 20.0
3.9104 114.13 10500 4.2276 21.0784 7.6899 16.3248 19.5625 20.0
3.915 115.22 10600 4.2282 20.9369 7.6522 16.1615 19.4826 20.0
3.8748 116.3 10700 4.2268 20.9369 7.6522 16.1615 19.4826 20.0
3.9341 117.39 10800 4.2252 21.0067 7.7263 16.3314 19.5589 20.0
3.8713 118.48 10900 4.2253 20.7028 7.5712 16.0398 19.2212 20.0
3.8861 119.57 11000 4.2243 20.7075 7.6844 16.0626 19.2959 20.0
3.8905 120.65 11100 4.2252 20.6546 7.5642 15.9451 19.1838 20.0
3.8682 121.74 11200 4.2238 20.8809 7.6536 16.1667 19.4217 20.0
3.904 122.83 11300 4.2241 20.6916 7.5324 15.9692 19.1791 20.0
3.8577 123.91 11400 4.2231 20.9271 7.6536 16.2314 19.4695 20.0
3.8851 125.0 11500 4.2230 20.8097 7.6891 16.1087 19.3872 20.0
3.8725 126.09 11600 4.2219 20.8965 7.6891 16.197 19.4319 20.0
3.8918 127.17 11700 4.2210 20.8203 7.6562 16.1283 19.388 20.0
3.845 128.26 11800 4.2210 20.7633 7.6883 16.0813 19.3537 20.0
3.8812 129.35 11900 4.2197 20.6605 7.6351 15.9703 19.2425 20.0
3.8734 130.43 12000 4.2208 20.6164 7.601 15.9703 19.1967 20.0
3.8704 131.52 12100 4.2201 20.533 7.5141 15.941 19.1898 20.0
3.8302 132.61 12200 4.2194 20.6164 7.601 15.9703 19.1967 20.0
3.8793 133.7 12300 4.2178 20.5427 7.5674 15.9591 19.2078 20.0
3.8631 134.78 12400 4.2181 20.6953 7.6549 16.0402 19.2734 20.0
3.8565 135.87 12500 4.2173 20.6168 7.5808 16.0402 19.2734 20.0
3.8842 136.96 12600 4.2163 20.6525 7.5782 16.0402 19.3124 20.0
3.8183 138.04 12700 4.2165 20.6168 7.5808 16.0402 19.2734 20.0
3.8482 139.13 12800 4.2155 20.6953 7.6154 16.0402 19.2734 20.0
3.8689 140.22 12900 4.2158 20.8264 7.7844 16.1396 19.4834 20.0
3.8361 141.3 13000 4.2144 20.8264 7.6986 16.2466 19.5192 20.0
3.8336 142.39 13100 4.2148 20.7613 7.7027 16.2516 19.4307 20.0
3.8532 143.48 13200 4.2155 20.6905 7.6695 16.1708 19.3584 20.0
3.8424 144.57 13300 4.2137 20.7613 7.7027 16.2516 19.4307 20.0
3.8781 145.65 13400 4.2128 20.6905 7.6695 16.1708 19.3584 20.0
3.8693 146.74 13500 4.2128 20.5395 7.4561 16.1388 19.1866 20.0
3.8304 147.83 13600 4.2123 20.6345 7.7324 16.1761 19.2764 20.0
3.8434 148.91 13700 4.2123 20.7145 7.6768 16.1729 19.3787 20.0
3.8348 150.0 13800 4.2123 20.7859 7.7023 16.2986 19.4932 20.0
3.8375 151.09 13900 4.2126 20.6319 7.5676 16.325 19.2512 20.0
3.8421 152.17 14000 4.2120 20.6665 7.5619 16.3257 19.2911 20.0
3.831 153.26 14100 4.2110 20.609 7.4912 16.2881 19.2953 20.0
3.8172 154.35 14200 4.2112 20.7352 7.6588 16.2115 19.3408 20.0
3.7853 155.43 14300 4.2107 20.6635 7.5987 16.2131 19.2667 20.0
3.8274 156.52 14400 4.2109 20.7352 7.7559 16.3035 19.3408 20.0
3.8362 157.61 14500 4.2099 20.7559 7.6865 16.325 19.4191 20.0
3.8561 158.7 14600 4.2098 20.6225 7.6943 16.3448 19.1425 20.0
3.7832 159.78 14700 4.2098 20.6307 7.6684 16.2469 19.269 20.0
3.8409 160.87 14800 4.2092 20.683 7.7924 16.2986 19.2414 20.0
3.821 161.96 14900 4.2092 20.5235 7.6721 16.2191 18.9879 20.0
3.8343 163.04 15000 4.2089 20.5235 7.6721 16.2191 18.9879 20.0
3.8279 164.13 15100 4.2087 20.5304 7.5448 16.2106 19.0909 20.0
3.7874 165.22 15200 4.2083 20.6319 7.6145 16.3035 19.2294 20.0
3.8316 166.3 15300 4.2076 20.5759 7.6145 16.2528 19.1508 20.0
3.7817 167.39 15400 4.2084 20.4845 7.5473 16.2067 19.0683 20.0
3.8338 168.48 15500 4.2075 20.5375 7.614 16.2509 19.1047 20.0
3.8515 169.57 15600 4.2069 20.4845 7.5473 16.2067 19.0683 20.0
3.7895 170.65 15700 4.2074 20.4845 7.5473 16.2067 19.0683 20.0
3.8129 171.74 15800 4.2076 20.4845 7.5473 16.2067 19.0683 20.0
3.8582 172.83 15900 4.2073 20.4845 7.5473 16.2067 19.0683 20.0
3.7716 173.91 16000 4.2073 20.5333 7.6756 16.2531 19.0336 20.0
3.8142 175.0 16100 4.2069 20.5333 7.6756 16.2531 19.0336 20.0
3.8186 176.09 16200 4.2068 20.5333 7.6756 16.2531 19.0336 20.0
3.8323 177.17 16300 4.2065 20.5333 7.6281 16.2531 19.0336 20.0
3.774 178.26 16400 4.2064 20.5724 7.677 16.2545 19.0747 20.0
3.8123 179.35 16500 4.2062 20.5333 7.6281 16.2531 19.0336 20.0
3.7914 180.43 16600 4.2066 20.5333 7.6281 16.2531 19.0336 20.0
3.7988 181.52 16700 4.2063 20.5724 7.6287 16.2545 19.0747 20.0
3.8331 182.61 16800 4.2059 20.6225 7.7265 16.3103 19.1036 20.0
3.8125 183.7 16900 4.2061 20.494 7.5897 16.2303 18.9697 20.0
3.8069 184.78 17000 4.2059 20.5333 7.6756 16.2531 19.0336 20.0
3.7933 185.87 17100 4.2058 20.5333 7.6281 16.2531 19.0336 20.0
3.807 186.96 17200 4.2058 20.5333 7.6756 16.2531 19.0336 20.0
3.8 188.04 17300 4.2055 20.5333 7.6756 16.2531 19.0336 20.0
3.776 189.13 17400 4.2057 20.5333 7.6756 16.2531 19.0336 20.0
3.7976 190.22 17500 4.2057 20.5333 7.6756 16.2531 19.0336 20.0
3.8293 191.3 17600 4.2057 20.5333 7.6756 16.2531 19.0336 20.0
3.7807 192.39 17700 4.2057 20.5333 7.6756 16.2531 19.0336 20.0
3.8246 193.48 17800 4.2055 20.5333 7.6756 16.2531 19.0336 20.0
3.7719 194.57 17900 4.2055 20.5333 7.6756 16.2531 19.0336 20.0
3.8055 195.65 18000 4.2055 20.5333 7.6756 16.2531 19.0336 20.0
3.7803 196.74 18100 4.2055 20.5333 7.6756 16.2531 19.0336 20.0
3.8287 197.83 18200 4.2055 20.5333 7.6756 16.2531 19.0336 20.0
3.8066 198.91 18300 4.2055 20.5333 7.6756 16.2531 19.0336 20.0
3.8011 200.0 18400 4.2055 20.5333 7.6756 16.2531 19.0336 20.0

Framework versions

  • Transformers 4.18.0
  • Pytorch 1.11.0+cu113
  • Datasets 2.1.0
  • Tokenizers 0.12.1
Downloads last month
31
Hosted inference API
This model can be loaded on the Inference API on-demand.

Evaluation results