mt5-summarize-ch_trad-v2
This model is a fine-tuned version of t5-small on the xlsum dataset. It achieves the following results on the evaluation set:
- Loss: 3.1706
- Rouge1: 0.292
- Rouge2: 0.1413
- Rougel: 0.2218
- Rougelsum: 0.2383
- Gen Len: 126.9946
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- num_epochs: 10
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|---|---|---|---|---|---|---|---|---|
| 7.599 | 0.43 | 500 | 5.9495 | 0.2214 | 0.0975 | 0.1686 | 0.1785 | 124.4867 |
| 6.051 | 0.86 | 1000 | 5.1437 | 0.2508 | 0.1156 | 0.1915 | 0.2024 | 126.4152 |
| 5.2303 | 1.28 | 1500 | 4.4085 | 0.2586 | 0.1206 | 0.1985 | 0.2091 | 126.5906 |
| 4.6814 | 1.71 | 2000 | 4.0174 | 0.281 | 0.1314 | 0.2124 | 0.2282 | 126.8248 |
| 4.388 | 2.14 | 2500 | 3.7829 | 0.2732 | 0.1278 | 0.2101 | 0.223 | 126.8782 |
| 4.1681 | 2.57 | 3000 | 3.6421 | 0.2655 | 0.1251 | 0.2068 | 0.2171 | 126.8794 |
| 4.0634 | 3.0 | 3500 | 3.5647 | 0.2732 | 0.129 | 0.2099 | 0.2217 | 126.9833 |
| 3.9309 | 3.42 | 4000 | 3.4990 | 0.2758 | 0.1295 | 0.2114 | 0.2254 | 126.9901 |
| 3.868 | 3.85 | 4500 | 3.4264 | 0.2769 | 0.1328 | 0.2152 | 0.2252 | 126.9861 |
| 3.7944 | 4.28 | 5000 | 3.4014 | 0.2857 | 0.1378 | 0.2187 | 0.2326 | 126.9694 |
| 3.7583 | 4.71 | 5500 | 3.3351 | 0.2822 | 0.136 | 0.2186 | 0.2311 | 126.9944 |
| 3.6907 | 5.14 | 6000 | 3.3172 | 0.2792 | 0.1335 | 0.2144 | 0.2273 | 126.9874 |
| 3.6542 | 5.57 | 6500 | 3.2911 | 0.2798 | 0.1343 | 0.2147 | 0.228 | 126.9916 |
| 3.6186 | 5.99 | 7000 | 3.2548 | 0.2802 | 0.134 | 0.2152 | 0.2277 | 126.9916 |
| 3.5894 | 6.42 | 7500 | 3.2287 | 0.2859 | 0.1376 | 0.2181 | 0.2328 | 126.9972 |
| 3.5615 | 6.85 | 8000 | 3.2264 | 0.2872 | 0.1374 | 0.2179 | 0.2343 | 126.9972 |
| 3.5321 | 7.28 | 8500 | 3.2069 | 0.2861 | 0.1374 | 0.2178 | 0.233 | 126.9972 |
| 3.5242 | 7.71 | 9000 | 3.2076 | 0.289 | 0.1385 | 0.2193 | 0.2357 | 126.9919 |
| 3.5195 | 8.13 | 9500 | 3.1825 | 0.2878 | 0.1384 | 0.2189 | 0.2352 | 126.9919 |
| 3.4815 | 8.56 | 10000 | 3.1852 | 0.289 | 0.1386 | 0.22 | 0.2358 | 126.9944 |
| 3.4823 | 8.99 | 10500 | 3.1775 | 0.2918 | 0.1413 | 0.2218 | 0.2383 | 127.0 |
| 3.4705 | 9.42 | 11000 | 3.1704 | 0.2912 | 0.1407 | 0.2218 | 0.2375 | 126.9972 |
| 3.4634 | 9.85 | 11500 | 3.1706 | 0.292 | 0.1413 | 0.2218 | 0.2383 | 126.9946 |
Framework versions
- Transformers 4.26.1
- Pytorch 1.13.0
- Datasets 2.1.0
- Tokenizers 0.13.2
- Downloads last month
- 0