---
base_model: kravchenko/uk-mt5-base
tags:
- summarization
- generated_from_trainer
datasets:
- xlsum
metrics:
- rouge
model-index:
- name: uk-mt5-base-xlsum-4000
  results:
  - task:
      name: Sequence-to-sequence Language Modeling
      type: text2text-generation
    dataset:
      name: xlsum
      type: xlsum
      config: ukrainian
      split: validation
      args: ukrainian
    metrics:
    - name: Rouge1
      type: rouge
      value: 4.2038
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# uk-mt5-base-xlsum-4000

This model is a fine-tuned version of [kravchenko/uk-mt5-base](https://huggingface.co/kravchenko/uk-mt5-base) on the xlsum dataset.
It achieves the following results on the evaluation set:
- Loss: 1.7909
- Rouge1: 4.2038
- Rouge2: 0.6736
- Rougel: 4.1229
- Rougelsum: 4.1353

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5.6e-05
- train_batch_size: 6
- eval_batch_size: 6
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 8

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum |
|:-------------:|:-----:|:-----:|:---------------:|:------:|:------:|:------:|:---------:|
| 2.871         | 1.0   | 7201  | 1.9992          | 3.157  | 0.5155 | 3.1283 | 3.1298    |
| 2.3902        | 2.0   | 14402 | 1.9162          | 3.6231 | 0.595  | 3.5878 | 3.6125    |
| 2.2273        | 3.0   | 21603 | 1.8681          | 3.8688 | 0.5949 | 3.8101 | 3.8106    |
| 2.1219        | 4.0   | 28804 | 1.8264          | 3.7935 | 0.58   | 3.741  | 3.7647    |
| 2.0448        | 5.0   | 36005 | 1.8062          | 3.9388 | 0.7156 | 3.8877 | 3.9098    |
| 1.9898        | 6.0   | 43206 | 1.8077          | 4.3916 | 0.8113 | 4.3133 | 4.327     |
| 1.9483        | 7.0   | 50407 | 1.7935          | 4.2474 | 0.7119 | 4.1732 | 4.197     |
| 1.9209        | 8.0   | 57608 | 1.7909          | 4.2038 | 0.6736 | 4.1229 | 4.1353    |


### Framework versions

- Transformers 4.34.0
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.14.1