---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: t5-small-mse-summarization
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# t5-small-mse-summarization

This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.1108
- Rouge1: 43.1145
- Rouge2: 23.2262
- Rougel: 37.218
- Rougelsum: 41.0897
- Bleurt: -0.8051
- Gen Len: 18.549

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 64
- eval_batch_size: 256
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum | Bleurt  | Gen Len |
|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|:-------:|
| 1.5207        | 1.0   | 267  | 1.2922          | 38.8738 | 19.1958 | 32.8458 | 36.9993   | -0.9061 | 18.668  |
| 1.363         | 2.0   | 534  | 1.2340          | 39.8466 | 20.0452 | 33.9101 | 37.7708   | -0.8925 | 18.657  |
| 1.3062        | 3.0   | 801  | 1.2057          | 40.5536 | 20.8249 | 34.5221 | 38.4648   | -0.8625 | 18.602  |
| 1.272         | 4.0   | 1068 | 1.1782          | 41.0078 | 21.2186 | 35.0101 | 38.9186   | -0.8595 | 18.602  |
| 1.2312        | 5.0   | 1335 | 1.1688          | 41.521  | 21.7934 | 35.704  | 39.4718   | -0.842  | 18.486  |
| 1.2052        | 6.0   | 1602 | 1.1557          | 42.1037 | 22.4291 | 36.3554 | 40.1124   | -0.8432 | 18.533  |
| 1.1842        | 7.0   | 1869 | 1.1440          | 42.4438 | 22.6456 | 36.5729 | 40.3134   | -0.8288 | 18.553  |
| 1.1643        | 8.0   | 2136 | 1.1408          | 42.245  | 22.4859 | 36.3637 | 40.2193   | -0.8284 | 18.622  |
| 1.1495        | 9.0   | 2403 | 1.1320          | 42.5362 | 22.5034 | 36.5092 | 40.4552   | -0.8211 | 18.57   |
| 1.1368        | 10.0  | 2670 | 1.1301          | 42.5159 | 22.462  | 36.4646 | 40.3968   | -0.819  | 18.538  |
| 1.1203        | 11.0  | 2937 | 1.1243          | 42.2803 | 22.5963 | 36.3454 | 40.2987   | -0.8242 | 18.522  |
| 1.1116        | 12.0  | 3204 | 1.1197          | 42.8078 | 22.8409 | 36.7344 | 40.8186   | -0.821  | 18.565  |
| 1.099         | 13.0  | 3471 | 1.1193          | 42.7423 | 22.9397 | 36.7894 | 40.7298   | -0.8125 | 18.552  |
| 1.0976        | 14.0  | 3738 | 1.1176          | 42.9002 | 23.2394 | 37.0215 | 40.9211   | -0.8156 | 18.568  |
| 1.0816        | 15.0  | 4005 | 1.1133          | 43.0007 | 23.3093 | 37.2037 | 40.9719   | -0.8059 | 18.519  |
| 1.084         | 16.0  | 4272 | 1.1146          | 42.9053 | 23.2391 | 37.0542 | 40.8826   | -0.8104 | 18.533  |
| 1.0755        | 17.0  | 4539 | 1.1124          | 43.0429 | 23.2773 | 37.1389 | 41.0755   | -0.8086 | 18.544  |
| 1.0748        | 18.0  | 4806 | 1.1121          | 43.2243 | 23.4179 | 37.2039 | 41.143    | -0.8048 | 18.548  |
| 1.072         | 19.0  | 5073 | 1.1106          | 43.1776 | 23.3061 | 37.3105 | 41.1392   | -0.8039 | 18.549  |
| 1.0671        | 20.0  | 5340 | 1.1108          | 43.1145 | 23.2262 | 37.218  | 41.0897   | -0.8051 | 18.549  |


### Framework versions

- Transformers 4.21.2
- Pytorch 1.12.1+cu113
- Datasets 2.4.0
- Tokenizers 0.12.1