---
license: apache-2.0
base_model: google/flan-t5-base
tags:
- generated_from_trainer
datasets:
- xlsum
model-index:
- name: flan-t5-base-xlsum
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# flan-t5-base-xlsum

This model is a fine-tuned version of [google/flan-t5-base](https://huggingface.co/google/flan-t5-base) on the xlsum dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3989

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 6
- eval_batch_size: 12
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 200
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 29.1967       | 0.05  | 100   | 4.4538          |
| 2.3457        | 0.09  | 200   | 0.4789          |
| 0.5064        | 0.14  | 300   | 0.4116          |
| 0.4857        | 0.18  | 400   | 0.4077          |
| 0.4749        | 0.23  | 500   | 0.4069          |
| 0.4695        | 0.28  | 600   | 0.4059          |
| 0.4754        | 0.32  | 700   | 0.4041          |
| 0.4609        | 0.37  | 800   | 0.4044          |
| 0.4703        | 0.42  | 900   | 0.4037          |
| 0.4656        | 0.46  | 1000  | 0.4025          |
| 0.464         | 0.51  | 1100  | 0.4020          |
| 0.4679        | 0.55  | 1200  | 0.4026          |
| 0.462         | 0.6   | 1300  | 0.4021          |
| 0.4665        | 0.65  | 1400  | 0.4006          |
| 0.4617        | 0.69  | 1500  | 0.4013          |
| 0.4541        | 0.74  | 1600  | 0.4000          |
| 0.4566        | 0.79  | 1700  | 0.3997          |
| 0.4646        | 0.83  | 1800  | 0.3995          |
| 0.4445        | 0.88  | 1900  | 0.3999          |
| 0.4733        | 0.92  | 2000  | 0.3993          |
| 0.4703        | 0.97  | 2100  | 0.3995          |
| 0.4412        | 1.02  | 2200  | 0.3998          |
| 0.4249        | 1.06  | 2300  | 0.4000          |
| 0.436         | 1.11  | 2400  | 0.3995          |
| 0.4333        | 1.16  | 2500  | 0.3989          |
| 0.4249        | 1.2   | 2600  | 0.3984          |
| 0.4312        | 1.25  | 2700  | 0.3988          |
| 0.4376        | 1.29  | 2800  | 0.3992          |
| 0.4276        | 1.34  | 2900  | 0.3990          |
| 0.4258        | 1.39  | 3000  | 0.3983          |
| 0.4411        | 1.43  | 3100  | 0.3986          |
| 0.4352        | 1.48  | 3200  | 0.3989          |
| 0.4429        | 1.53  | 3300  | 0.3974          |
| 0.4466        | 1.57  | 3400  | 0.3980          |
| 0.4311        | 1.62  | 3500  | 0.3977          |
| 0.427         | 1.66  | 3600  | 0.3976          |
| 0.4433        | 1.71  | 3700  | 0.3977          |
| 0.4228        | 1.76  | 3800  | 0.3984          |
| 0.4247        | 1.8   | 3900  | 0.3980          |
| 0.4275        | 1.85  | 4000  | 0.3980          |
| 0.4523        | 1.9   | 4100  | 0.3970          |
| 0.4258        | 1.94  | 4200  | 0.3976          |
| 0.4329        | 1.99  | 4300  | 0.3978          |
| 0.4146        | 2.03  | 4400  | 0.3988          |
| 0.4025        | 2.08  | 4500  | 0.3997          |
| 0.3944        | 2.13  | 4600  | 0.3989          |
| 0.4034        | 2.17  | 4700  | 0.3984          |
| 0.4099        | 2.22  | 4800  | 0.3987          |
| 0.3989        | 2.27  | 4900  | 0.3983          |
| 0.4269        | 2.31  | 5000  | 0.3990          |
| 0.4273        | 2.36  | 5100  | 0.3988          |
| 0.4117        | 2.4   | 5200  | 0.3981          |
| 0.4117        | 2.45  | 5300  | 0.3984          |
| 0.4037        | 2.5   | 5400  | 0.3978          |
| 0.4158        | 2.54  | 5500  | 0.3981          |
| 0.4081        | 2.59  | 5600  | 0.3982          |
| 0.4125        | 2.64  | 5700  | 0.3982          |
| 0.4086        | 2.68  | 5800  | 0.3988          |
| 0.4143        | 2.73  | 5900  | 0.3986          |
| 0.4025        | 2.77  | 6000  | 0.3981          |
| 0.4141        | 2.82  | 6100  | 0.3979          |
| 0.4239        | 2.87  | 6200  | 0.3975          |
| 0.4217        | 2.91  | 6300  | 0.3979          |
| 0.4099        | 2.96  | 6400  | 0.3972          |
| 0.4008        | 3.01  | 6500  | 0.3977          |
| 0.4092        | 3.05  | 6600  | 0.3998          |
| 0.3898        | 3.1   | 6700  | 0.4000          |
| 0.3978        | 3.14  | 6800  | 0.3985          |
| 0.4004        | 3.19  | 6900  | 0.3996          |
| 0.3998        | 3.24  | 7000  | 0.3996          |
| 0.3908        | 3.28  | 7100  | 0.3993          |
| 0.4021        | 3.33  | 7200  | 0.3994          |
| 0.3889        | 3.37  | 7300  | 0.3993          |
| 0.4009        | 3.42  | 7400  | 0.3984          |
| 0.3835        | 3.47  | 7500  | 0.3988          |
| 0.3999        | 3.51  | 7600  | 0.3986          |
| 0.409         | 3.56  | 7700  | 0.3985          |
| 0.3927        | 3.61  | 7800  | 0.3984          |
| 0.407         | 3.65  | 7900  | 0.3980          |
| 0.389         | 3.7   | 8000  | 0.3989          |
| 0.3976        | 3.74  | 8100  | 0.3981          |
| 0.4075        | 3.79  | 8200  | 0.3982          |
| 0.3897        | 3.84  | 8300  | 0.3981          |
| 0.3805        | 3.88  | 8400  | 0.3983          |
| 0.393         | 3.93  | 8500  | 0.3983          |
| 0.398         | 3.98  | 8600  | 0.3980          |
| 0.3832        | 4.02  | 8700  | 0.3985          |
| 0.384         | 4.07  | 8800  | 0.3989          |
| 0.3787        | 4.11  | 8900  | 0.3989          |
| 0.3816        | 4.16  | 9000  | 0.3994          |
| 0.3857        | 4.21  | 9100  | 0.3991          |
| 0.3909        | 4.25  | 9200  | 0.3990          |
| 0.3858        | 4.3   | 9300  | 0.3993          |
| 0.4021        | 4.35  | 9400  | 0.3993          |
| 0.3879        | 4.39  | 9500  | 0.3991          |
| 0.3752        | 4.44  | 9600  | 0.3994          |
| 0.3882        | 4.48  | 9700  | 0.3994          |
| 0.3881        | 4.53  | 9800  | 0.3992          |
| 0.4089        | 4.58  | 9900  | 0.3988          |
| 0.3801        | 4.62  | 10000 | 0.3989          |
| 0.3925        | 4.67  | 10100 | 0.3989          |
| 0.3858        | 4.72  | 10200 | 0.3990          |
| 0.3883        | 4.76  | 10300 | 0.3988          |
| 0.3808        | 4.81  | 10400 | 0.3989          |
| 0.4012        | 4.85  | 10500 | 0.3989          |
| 0.384         | 4.9   | 10600 | 0.3989          |
| 0.3828        | 4.95  | 10700 | 0.3989          |
| 0.3899        | 4.99  | 10800 | 0.3989          |


### Framework versions

- Transformers 4.31.0
- Pytorch 2.0.1+cu117
- Datasets 2.13.1
- Tokenizers 0.13.3