Commit
•
cca8c58
1
Parent(s):
166c2ea
update model card README.md
Browse files
README.md
ADDED
@@ -0,0 +1,156 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: apache-2.0
|
3 |
+
tags:
|
4 |
+
- generated_from_trainer
|
5 |
+
metrics:
|
6 |
+
- rouge
|
7 |
+
model-index:
|
8 |
+
- name: bart-qmsum-meeting-summarization
|
9 |
+
results: []
|
10 |
+
---
|
11 |
+
|
12 |
+
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
13 |
+
should probably proofread and complete it, then remove this comment. -->
|
14 |
+
|
15 |
+
# bart-qmsum-meeting-summarization
|
16 |
+
|
17 |
+
This model is a fine-tuned version of [sshleifer/distilbart-cnn-12-6](https://huggingface.co/sshleifer/distilbart-cnn-12-6) on the None dataset.
|
18 |
+
It achieves the following results on the evaluation set:
|
19 |
+
- Loss: 4.3354
|
20 |
+
- Rouge1: 39.5539
|
21 |
+
- Rouge2: 12.1134
|
22 |
+
- Rougel: 23.9163
|
23 |
+
- Rougelsum: 36.0299
|
24 |
+
- Gen Len: 117.225
|
25 |
+
|
26 |
+
## Model description
|
27 |
+
|
28 |
+
More information needed
|
29 |
+
|
30 |
+
## Intended uses & limitations
|
31 |
+
|
32 |
+
More information needed
|
33 |
+
|
34 |
+
## Training and evaluation data
|
35 |
+
|
36 |
+
More information needed
|
37 |
+
|
38 |
+
## Training procedure
|
39 |
+
|
40 |
+
### Training hyperparameters
|
41 |
+
|
42 |
+
The following hyperparameters were used during training:
|
43 |
+
- learning_rate: 3e-07
|
44 |
+
- train_batch_size: 4
|
45 |
+
- eval_batch_size: 4
|
46 |
+
- seed: 42
|
47 |
+
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
48 |
+
- lr_scheduler_type: linear
|
49 |
+
- lr_scheduler_warmup_steps: 500
|
50 |
+
- num_epochs: 200
|
51 |
+
- label_smoothing_factor: 0.1
|
52 |
+
|
53 |
+
### Training results
|
54 |
+
|
55 |
+
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|
56 |
+
|:-------------:|:------:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
|
57 |
+
| 5.5573 | 2.17 | 100 | 5.4074 | 23.6282 | 4.1122 | 14.584 | 21.2263 | 84.75 |
|
58 |
+
| 5.4721 | 4.35 | 200 | 5.2899 | 24.61 | 4.272 | 15.2096 | 22.2997 | 87.2 |
|
59 |
+
| 5.3407 | 6.52 | 300 | 5.1360 | 25.8272 | 4.3314 | 15.9926 | 23.3416 | 87.95 |
|
60 |
+
| 5.1527 | 8.7 | 400 | 4.9751 | 27.7207 | 5.31 | 16.7055 | 24.8357 | 88.35 |
|
61 |
+
| 5.0058 | 10.87 | 500 | 4.8372 | 30.1847 | 6.8615 | 18.934 | 27.2424 | 89.95 |
|
62 |
+
| 4.8807 | 13.04 | 600 | 4.7488 | 33.1208 | 9.1784 | 20.655 | 30.1198 | 101.3 |
|
63 |
+
| 4.7931 | 15.22 | 700 | 4.6891 | 33.2266 | 8.4253 | 20.0334 | 30.4093 | 108.925 |
|
64 |
+
| 4.7272 | 17.39 | 800 | 4.6467 | 35.0475 | 9.326 | 21.0655 | 31.8413 | 111.7 |
|
65 |
+
| 4.6904 | 19.57 | 900 | 4.6102 | 34.869 | 9.6046 | 21.395 | 32.4346 | 115.05 |
|
66 |
+
| 4.6547 | 21.74 | 1000 | 4.5829 | 36.3392 | 10.9936 | 22.1524 | 33.6863 | 119.875 |
|
67 |
+
| 4.594 | 23.91 | 1100 | 4.5602 | 35.9717 | 10.3827 | 21.6118 | 32.8302 | 119.5 |
|
68 |
+
| 4.5714 | 26.09 | 1200 | 4.5424 | 36.3656 | 10.6282 | 22.2187 | 33.6494 | 118.0 |
|
69 |
+
| 4.542 | 28.26 | 1300 | 4.5256 | 36.7386 | 10.615 | 22.2487 | 34.1927 | 115.675 |
|
70 |
+
| 4.5092 | 30.43 | 1400 | 4.5116 | 37.1597 | 10.7751 | 22.6747 | 34.396 | 118.55 |
|
71 |
+
| 4.5031 | 32.61 | 1500 | 4.4981 | 37.6108 | 10.9732 | 22.8342 | 34.6833 | 117.125 |
|
72 |
+
| 4.4682 | 34.78 | 1600 | 4.4875 | 37.5057 | 11.1328 | 22.8973 | 34.7114 | 117.65 |
|
73 |
+
| 4.4387 | 36.96 | 1700 | 4.4775 | 38.1278 | 11.3597 | 23.1307 | 35.1869 | 115.65 |
|
74 |
+
| 4.4085 | 39.13 | 1800 | 4.4682 | 37.9578 | 11.4355 | 23.1149 | 35.4961 | 119.6 |
|
75 |
+
| 4.4166 | 41.3 | 1900 | 4.4592 | 38.1467 | 11.3208 | 23.045 | 35.0824 | 120.05 |
|
76 |
+
| 4.3971 | 43.48 | 2000 | 4.4517 | 37.9922 | 11.5071 | 23.3983 | 34.6918 | 114.425 |
|
77 |
+
| 4.3638 | 45.65 | 2100 | 4.4438 | 38.1666 | 11.4985 | 23.5518 | 35.1484 | 117.2 |
|
78 |
+
| 4.3522 | 47.83 | 2200 | 4.4377 | 37.7572 | 11.3984 | 23.4437 | 35.0453 | 113.725 |
|
79 |
+
| 4.3398 | 50.0 | 2300 | 4.4320 | 38.5833 | 11.4575 | 23.6411 | 35.3437 | 116.125 |
|
80 |
+
| 4.3341 | 52.17 | 2400 | 4.4247 | 38.2705 | 12.0374 | 23.5807 | 34.9985 | 110.8 |
|
81 |
+
| 4.3024 | 54.35 | 2500 | 4.4201 | 39.0206 | 12.2041 | 23.4394 | 35.6291 | 114.5 |
|
82 |
+
| 4.3117 | 56.52 | 2600 | 4.4147 | 38.6555 | 12.1079 | 23.5655 | 35.5287 | 111.325 |
|
83 |
+
| 4.2659 | 58.7 | 2700 | 4.4107 | 39.2235 | 12.025 | 23.934 | 36.2243 | 113.3 |
|
84 |
+
| 4.2946 | 60.87 | 2800 | 4.4055 | 39.0301 | 12.1833 | 23.8999 | 36.0487 | 110.325 |
|
85 |
+
| 4.2431 | 63.04 | 2900 | 4.4009 | 39.0498 | 12.3215 | 23.9686 | 36.0277 | 112.775 |
|
86 |
+
| 4.2439 | 65.22 | 3000 | 4.3968 | 38.8786 | 12.0985 | 23.8308 | 35.8575 | 115.175 |
|
87 |
+
| 4.2244 | 67.39 | 3100 | 4.3922 | 38.7614 | 12.1721 | 23.7736 | 35.6744 | 113.55 |
|
88 |
+
| 4.235 | 69.57 | 3200 | 4.3895 | 38.6858 | 11.3994 | 23.6392 | 35.3456 | 114.125 |
|
89 |
+
| 4.2064 | 71.74 | 3300 | 4.3859 | 39.0258 | 12.0435 | 24.2528 | 35.8378 | 113.5 |
|
90 |
+
| 4.1934 | 73.91 | 3400 | 4.3835 | 39.0467 | 11.5556 | 23.6704 | 35.5643 | 111.5 |
|
91 |
+
| 4.1859 | 76.09 | 3500 | 4.3800 | 38.776 | 11.729 | 24.1254 | 35.3894 | 112.9 |
|
92 |
+
| 4.1762 | 78.26 | 3600 | 4.3775 | 38.9465 | 11.9112 | 23.8123 | 35.5453 | 114.125 |
|
93 |
+
| 4.1848 | 80.43 | 3700 | 4.3744 | 39.2783 | 11.6539 | 23.8236 | 35.8465 | 110.225 |
|
94 |
+
| 4.1386 | 82.61 | 3800 | 4.3730 | 38.8894 | 11.4784 | 23.7534 | 35.5464 | 113.15 |
|
95 |
+
| 4.1483 | 84.78 | 3900 | 4.3710 | 39.2734 | 12.0285 | 23.8171 | 35.6884 | 115.95 |
|
96 |
+
| 4.1428 | 86.96 | 4000 | 4.3688 | 39.6134 | 12.0616 | 23.7454 | 36.0363 | 113.375 |
|
97 |
+
| 4.133 | 89.13 | 4100 | 4.3663 | 38.935 | 11.4781 | 23.8766 | 35.4061 | 114.15 |
|
98 |
+
| 4.1211 | 91.3 | 4200 | 4.3648 | 39.1488 | 11.8399 | 23.9935 | 35.3107 | 113.975 |
|
99 |
+
| 4.1076 | 93.48 | 4300 | 4.3650 | 38.9764 | 11.9963 | 23.4994 | 35.7214 | 116.25 |
|
100 |
+
| 4.121 | 95.65 | 4400 | 4.3597 | 38.9418 | 11.8416 | 24.0272 | 35.6597 | 111.325 |
|
101 |
+
| 4.0936 | 97.83 | 4500 | 4.3602 | 39.266 | 12.5616 | 24.2046 | 36.1883 | 114.275 |
|
102 |
+
| 4.0841 | 100.0 | 4600 | 4.3588 | 39.4659 | 12.2132 | 24.0521 | 36.249 | 115.475 |
|
103 |
+
| 4.0768 | 102.17 | 4700 | 4.3578 | 39.4167 | 12.0587 | 24.025 | 35.9668 | 114.375 |
|
104 |
+
| 4.0711 | 104.35 | 4800 | 4.3541 | 39.6943 | 12.1095 | 24.0925 | 36.3496 | 115.65 |
|
105 |
+
| 4.072 | 106.52 | 4900 | 4.3539 | 40.2024 | 12.4618 | 24.2863 | 36.8844 | 113.475 |
|
106 |
+
| 4.0646 | 108.7 | 5000 | 4.3540 | 39.4299 | 11.8085 | 23.686 | 36.0454 | 113.975 |
|
107 |
+
| 4.0508 | 110.87 | 5100 | 4.3517 | 39.9217 | 11.9379 | 24.2299 | 36.6362 | 115.5 |
|
108 |
+
| 4.0549 | 113.04 | 5200 | 4.3498 | 40.3496 | 12.2558 | 24.0271 | 36.9715 | 112.5 |
|
109 |
+
| 4.0428 | 115.22 | 5300 | 4.3497 | 40.1349 | 12.0628 | 24.0622 | 36.9169 | 113.95 |
|
110 |
+
| 4.0391 | 117.39 | 5400 | 4.3480 | 40.1209 | 12.3587 | 24.3456 | 36.8411 | 116.025 |
|
111 |
+
| 4.0195 | 119.57 | 5500 | 4.3474 | 39.5209 | 12.1325 | 24.2622 | 36.4357 | 111.975 |
|
112 |
+
| 4.0054 | 121.74 | 5600 | 4.3468 | 40.2885 | 12.4453 | 24.2373 | 36.932 | 117.375 |
|
113 |
+
| 4.0286 | 123.91 | 5700 | 4.3465 | 39.3943 | 11.8399 | 23.9786 | 35.991 | 116.475 |
|
114 |
+
| 4.005 | 126.09 | 5800 | 4.3459 | 38.7442 | 11.7408 | 23.8948 | 35.3673 | 117.625 |
|
115 |
+
| 3.991 | 128.26 | 5900 | 4.3444 | 39.6276 | 12.1549 | 23.9542 | 36.3832 | 115.675 |
|
116 |
+
| 4.0137 | 130.43 | 6000 | 4.3427 | 39.8331 | 12.2687 | 24.187 | 36.6144 | 115.475 |
|
117 |
+
| 3.9755 | 132.61 | 6100 | 4.3438 | 39.1907 | 12.1033 | 24.2339 | 35.9126 | 114.525 |
|
118 |
+
| 4.0134 | 134.78 | 6200 | 4.3422 | 39.4298 | 11.862 | 24.0847 | 35.5744 | 115.025 |
|
119 |
+
| 3.9935 | 136.96 | 6300 | 4.3416 | 39.4158 | 11.6968 | 23.9636 | 35.8155 | 114.35 |
|
120 |
+
| 3.9606 | 139.13 | 6400 | 4.3409 | 39.1239 | 11.7046 | 23.6846 | 36.0431 | 114.775 |
|
121 |
+
| 3.9834 | 141.3 | 6500 | 4.3404 | 39.6375 | 12.2746 | 24.2636 | 36.1425 | 116.175 |
|
122 |
+
| 3.9687 | 143.48 | 6600 | 4.3409 | 39.1494 | 12.1404 | 24.0778 | 35.4932 | 118.05 |
|
123 |
+
| 3.9861 | 145.65 | 6700 | 4.3394 | 39.6258 | 12.2497 | 23.9662 | 36.4054 | 116.8 |
|
124 |
+
| 3.9755 | 147.83 | 6800 | 4.3400 | 39.3121 | 11.7831 | 23.6584 | 35.9636 | 118.125 |
|
125 |
+
| 3.9591 | 150.0 | 6900 | 4.3390 | 39.6957 | 11.9406 | 24.0599 | 36.3021 | 114.9 |
|
126 |
+
| 3.9599 | 152.17 | 7000 | 4.3389 | 39.4271 | 11.4159 | 24.1437 | 35.9056 | 115.8 |
|
127 |
+
| 3.9456 | 154.35 | 7100 | 4.3384 | 39.4862 | 11.726 | 23.883 | 35.9839 | 116.375 |
|
128 |
+
| 3.9341 | 156.52 | 7200 | 4.3386 | 39.6915 | 11.8028 | 24.346 | 36.406 | 116.425 |
|
129 |
+
| 3.9648 | 158.7 | 7300 | 4.3383 | 39.9311 | 11.7135 | 23.985 | 36.2617 | 118.075 |
|
130 |
+
| 3.9486 | 160.87 | 7400 | 4.3372 | 39.8375 | 12.0014 | 24.0969 | 36.5902 | 118.8 |
|
131 |
+
| 3.9533 | 163.04 | 7500 | 4.3371 | 40.2678 | 12.3137 | 24.1916 | 37.1632 | 118.075 |
|
132 |
+
| 3.9344 | 165.22 | 7600 | 4.3369 | 39.5588 | 11.6805 | 24.1474 | 36.2021 | 114.875 |
|
133 |
+
| 3.9314 | 167.39 | 7700 | 4.3368 | 39.8649 | 11.9824 | 24.5459 | 36.3921 | 113.65 |
|
134 |
+
| 3.9558 | 169.57 | 7800 | 4.3363 | 39.8428 | 12.0892 | 24.0175 | 36.67 | 112.7 |
|
135 |
+
| 3.928 | 171.74 | 7900 | 4.3364 | 39.2281 | 11.8456 | 23.7212 | 36.2005 | 113.95 |
|
136 |
+
| 3.9351 | 173.91 | 8000 | 4.3363 | 39.9798 | 12.4387 | 23.7687 | 36.6472 | 115.45 |
|
137 |
+
| 3.9326 | 176.09 | 8100 | 4.3363 | 39.9772 | 12.1193 | 24.1518 | 36.5791 | 117.4 |
|
138 |
+
| 3.9387 | 178.26 | 8200 | 4.3363 | 39.8629 | 12.1719 | 23.9446 | 36.345 | 115.075 |
|
139 |
+
| 3.9204 | 180.43 | 8300 | 4.3358 | 39.9738 | 12.3072 | 23.8641 | 36.4802 | 116.3 |
|
140 |
+
| 3.9418 | 182.61 | 8400 | 4.3357 | 40.1451 | 12.4144 | 24.1553 | 36.4251 | 116.025 |
|
141 |
+
| 3.9289 | 184.78 | 8500 | 4.3357 | 39.7241 | 12.0543 | 24.0752 | 36.0847 | 115.8 |
|
142 |
+
| 3.9176 | 186.96 | 8600 | 4.3358 | 39.7969 | 12.0967 | 24.123 | 36.2664 | 118.6 |
|
143 |
+
| 3.9097 | 189.13 | 8700 | 4.3356 | 39.4096 | 11.9872 | 24.0609 | 35.8662 | 117.2 |
|
144 |
+
| 3.938 | 191.3 | 8800 | 4.3354 | 39.4695 | 11.9343 | 24.0295 | 35.9372 | 117.025 |
|
145 |
+
| 3.9239 | 193.48 | 8900 | 4.3352 | 39.3231 | 12.0965 | 23.9131 | 35.9555 | 117.275 |
|
146 |
+
| 3.91 | 195.65 | 9000 | 4.3354 | 39.5932 | 12.1808 | 23.9233 | 36.0864 | 116.925 |
|
147 |
+
| 3.9234 | 197.83 | 9100 | 4.3354 | 39.5539 | 12.1134 | 23.9163 | 36.0299 | 117.225 |
|
148 |
+
| 3.9263 | 200.0 | 9200 | 4.3354 | 39.5539 | 12.1134 | 23.9163 | 36.0299 | 117.225 |
|
149 |
+
|
150 |
+
|
151 |
+
### Framework versions
|
152 |
+
|
153 |
+
- Transformers 4.18.0
|
154 |
+
- Pytorch 1.11.0+cu113
|
155 |
+
- Datasets 2.1.0
|
156 |
+
- Tokenizers 0.12.1
|