File size: 10,342 Bytes
e26cfc9
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
---
license: apache-2.0
base_model: google/mt5-base
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: mt5-lithuanian-simplifier-full
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# mt5-lithuanian-simplifier-full

This model is a fine-tuned version of [google/mt5-base](https://huggingface.co/google/mt5-base) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0771
- Rouge1: 0.7828
- Rouge2: 0.6494
- Rougel: 0.7787
- Gen Len: 48.0191

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 8

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Rouge1 | Rouge2 | Rougel | Gen Len |
|:-------------:|:-----:|:-----:|:---------------:|:------:|:------:|:------:|:-------:|
| 24.351        | 0.08  | 200   | 18.6244         | 0.0226 | 0.0018 | 0.0207 | 512.0   |
| 3.0331        | 0.16  | 400   | 0.6830          | 0.0549 | 0.0018 | 0.0497 | 49.0191 |
| 0.2076        | 0.24  | 600   | 0.1642          | 0.6417 | 0.4986 | 0.6328 | 48.0191 |
| 0.2019        | 0.32  | 800   | 0.1303          | 0.6713 | 0.5243 | 0.6633 | 48.0191 |
| 0.1573        | 0.4   | 1000  | 0.1242          | 0.7007 | 0.5589 | 0.6937 | 48.0191 |
| 0.1687        | 0.48  | 1200  | 0.1158          | 0.712  | 0.569  | 0.7055 | 48.0191 |
| 0.1315        | 0.56  | 1400  | 0.1225          | 0.6923 | 0.5361 | 0.6851 | 48.0191 |
| 0.1376        | 0.64  | 1600  | 0.1108          | 0.7171 | 0.5695 | 0.7105 | 48.0191 |
| 0.158         | 0.72  | 1800  | 0.1074          | 0.7229 | 0.574  | 0.7169 | 48.0191 |
| 0.1221        | 0.8   | 2000  | 0.1064          | 0.7227 | 0.5761 | 0.7166 | 48.0191 |
| 0.1371        | 0.88  | 2200  | 0.1049          | 0.7282 | 0.5827 | 0.7223 | 48.0191 |
| 0.1376        | 0.96  | 2400  | 0.1043          | 0.73   | 0.5861 | 0.7239 | 48.0191 |
| 0.1116        | 1.04  | 2600  | 0.1021          | 0.733  | 0.5888 | 0.727  | 48.0191 |
| 0.132         | 1.12  | 2800  | 0.1012          | 0.7338 | 0.5899 | 0.7277 | 48.0191 |
| 0.131         | 1.2   | 3000  | 0.0997          | 0.7365 | 0.5936 | 0.7307 | 48.0191 |
| 0.1001        | 1.28  | 3200  | 0.0950          | 0.7408 | 0.5977 | 0.7355 | 48.0191 |
| 0.1398        | 1.36  | 3400  | 0.0964          | 0.7418 | 0.599  | 0.7364 | 48.0191 |
| 0.1085        | 1.44  | 3600  | 0.0962          | 0.744  | 0.6015 | 0.7386 | 48.0191 |
| 0.097         | 1.52  | 3800  | 0.0967          | 0.743  | 0.6009 | 0.7377 | 48.0191 |
| 0.1178        | 1.6   | 4000  | 0.0955          | 0.7446 | 0.6035 | 0.7391 | 48.0191 |
| 0.1214        | 1.68  | 4200  | 0.0939          | 0.7452 | 0.6036 | 0.7403 | 48.0191 |
| 0.1539        | 1.76  | 4400  | 0.0909          | 0.7486 | 0.6068 | 0.7436 | 48.0191 |
| 0.1141        | 1.83  | 4600  | 0.0900          | 0.7518 | 0.6104 | 0.7467 | 48.0191 |
| 0.0795        | 1.91  | 4800  | 0.0891          | 0.7513 | 0.6097 | 0.7466 | 48.0191 |
| 0.0856        | 1.99  | 5000  | 0.0915          | 0.7513 | 0.6099 | 0.7463 | 48.0191 |
| 0.0954        | 2.07  | 5200  | 0.0898          | 0.753  | 0.6126 | 0.7482 | 48.0191 |
| 0.1271        | 2.15  | 5400  | 0.0901          | 0.7534 | 0.6125 | 0.7486 | 48.0191 |
| 0.0816        | 2.23  | 5600  | 0.0893          | 0.7553 | 0.6148 | 0.7506 | 48.0191 |
| 0.0922        | 2.31  | 5800  | 0.0881          | 0.7569 | 0.6163 | 0.7521 | 48.0191 |
| 0.1177        | 2.39  | 6000  | 0.0878          | 0.7575 | 0.6176 | 0.7532 | 48.0191 |
| 0.0916        | 2.47  | 6200  | 0.0874          | 0.7585 | 0.618  | 0.7541 | 48.0191 |
| 0.1349        | 2.55  | 6400  | 0.0861          | 0.76   | 0.62   | 0.7555 | 48.0191 |
| 0.1196        | 2.63  | 6600  | 0.0833          | 0.7617 | 0.6212 | 0.7572 | 48.0191 |
| 0.0841        | 2.71  | 6800  | 0.0848          | 0.7621 | 0.6219 | 0.7576 | 48.0191 |
| 0.0934        | 2.79  | 7000  | 0.0854          | 0.7622 | 0.6227 | 0.7577 | 48.0191 |
| 0.1246        | 2.87  | 7200  | 0.0835          | 0.7652 | 0.6256 | 0.7606 | 48.0191 |
| 0.0762        | 2.95  | 7400  | 0.0835          | 0.7649 | 0.6262 | 0.7606 | 48.0191 |
| 0.0924        | 3.03  | 7600  | 0.0828          | 0.7662 | 0.6276 | 0.7618 | 48.0191 |
| 0.0822        | 3.11  | 7800  | 0.0834          | 0.7664 | 0.6284 | 0.7621 | 48.0191 |
| 0.0856        | 3.19  | 8000  | 0.0836          | 0.7647 | 0.627  | 0.7603 | 48.0191 |
| 0.0798        | 3.27  | 8200  | 0.0829          | 0.7657 | 0.6284 | 0.7614 | 48.0191 |
| 0.0959        | 3.35  | 8400  | 0.0828          | 0.7671 | 0.6302 | 0.7629 | 48.0191 |
| 0.0871        | 3.43  | 8600  | 0.0820          | 0.7672 | 0.6297 | 0.763  | 48.0191 |
| 0.1068        | 3.51  | 8800  | 0.0827          | 0.7683 | 0.6307 | 0.7641 | 48.0191 |
| 0.072         | 3.59  | 9000  | 0.0820          | 0.7684 | 0.632  | 0.764  | 48.0191 |
| 0.0964        | 3.67  | 9200  | 0.0838          | 0.7692 | 0.6333 | 0.7645 | 48.0191 |
| 0.0946        | 3.75  | 9400  | 0.0809          | 0.7707 | 0.6348 | 0.7663 | 48.0191 |
| 0.0822        | 3.83  | 9600  | 0.0825          | 0.7708 | 0.6347 | 0.7666 | 48.0191 |
| 0.1019        | 3.91  | 9800  | 0.0788          | 0.7733 | 0.6373 | 0.7692 | 48.0191 |
| 0.08          | 3.99  | 10000 | 0.0797          | 0.7727 | 0.6369 | 0.7686 | 48.0191 |
| 0.0989        | 4.07  | 10200 | 0.0818          | 0.7724 | 0.6367 | 0.7681 | 48.0191 |
| 0.0693        | 4.15  | 10400 | 0.0804          | 0.7737 | 0.6378 | 0.7697 | 48.0191 |
| 0.0763        | 4.23  | 10600 | 0.0814          | 0.7741 | 0.6379 | 0.7699 | 48.0191 |
| 0.0956        | 4.31  | 10800 | 0.0815          | 0.7726 | 0.6369 | 0.7683 | 48.0191 |
| 0.0728        | 4.39  | 11000 | 0.0800          | 0.7738 | 0.6374 | 0.7696 | 48.0191 |
| 0.0652        | 4.47  | 11200 | 0.0795          | 0.7747 | 0.6388 | 0.7708 | 48.0191 |
| 0.0706        | 4.55  | 11400 | 0.0798          | 0.7742 | 0.6388 | 0.7703 | 48.0191 |
| 0.0979        | 4.63  | 11600 | 0.0788          | 0.7748 | 0.6387 | 0.7708 | 48.0191 |
| 0.0771        | 4.71  | 11800 | 0.0797          | 0.775  | 0.6402 | 0.771  | 48.0191 |
| 0.1067        | 4.79  | 12000 | 0.0779          | 0.7757 | 0.6404 | 0.7717 | 48.0191 |
| 0.0773        | 4.87  | 12200 | 0.0783          | 0.7759 | 0.6411 | 0.7721 | 48.0191 |
| 0.0866        | 4.95  | 12400 | 0.0780          | 0.7773 | 0.6437 | 0.7734 | 48.0191 |
| 0.0611        | 5.03  | 12600 | 0.0785          | 0.7761 | 0.6418 | 0.7723 | 48.0191 |
| 0.0685        | 5.11  | 12800 | 0.0781          | 0.777  | 0.6421 | 0.773  | 48.0191 |
| 0.0501        | 5.19  | 13000 | 0.0788          | 0.7764 | 0.6411 | 0.7721 | 48.0191 |
| 0.0626        | 5.27  | 13200 | 0.0792          | 0.7762 | 0.6416 | 0.7721 | 48.0191 |
| 0.0708        | 5.35  | 13400 | 0.0795          | 0.7761 | 0.6408 | 0.772  | 48.0191 |
| 0.055         | 5.42  | 13600 | 0.0779          | 0.7773 | 0.642  | 0.7733 | 48.0191 |
| 0.0749        | 5.5   | 13800 | 0.0789          | 0.7783 | 0.6431 | 0.7742 | 48.0191 |
| 0.0771        | 5.58  | 14000 | 0.0779          | 0.778  | 0.6437 | 0.774  | 48.0191 |
| 0.0906        | 5.66  | 14200 | 0.0779          | 0.7781 | 0.6431 | 0.7742 | 48.0191 |
| 0.0679        | 5.74  | 14400 | 0.0778          | 0.7783 | 0.6449 | 0.7745 | 48.0191 |
| 0.0605        | 5.82  | 14600 | 0.0786          | 0.7778 | 0.6439 | 0.7738 | 48.0191 |
| 0.0647        | 5.9   | 14800 | 0.0781          | 0.7785 | 0.6445 | 0.7743 | 48.0191 |
| 0.058         | 5.98  | 15000 | 0.0775          | 0.7792 | 0.6448 | 0.7749 | 48.0191 |
| 0.0574        | 6.06  | 15200 | 0.0788          | 0.7793 | 0.6451 | 0.7752 | 48.0191 |
| 0.0545        | 6.14  | 15400 | 0.0778          | 0.7802 | 0.6464 | 0.7759 | 48.0191 |
| 0.079         | 6.22  | 15600 | 0.0781          | 0.7801 | 0.6466 | 0.7759 | 48.0191 |
| 0.0474        | 6.3   | 15800 | 0.0782          | 0.7809 | 0.6477 | 0.7768 | 48.0191 |
| 0.0517        | 6.38  | 16000 | 0.0788          | 0.7809 | 0.6481 | 0.7769 | 48.0191 |
| 0.0613        | 6.46  | 16200 | 0.0782          | 0.7814 | 0.6481 | 0.7773 | 48.0191 |
| 0.0517        | 6.54  | 16400 | 0.0785          | 0.7807 | 0.6468 | 0.7767 | 48.0191 |
| 0.0549        | 6.62  | 16600 | 0.0778          | 0.7817 | 0.6485 | 0.7777 | 48.0191 |
| 0.0727        | 6.7   | 16800 | 0.0774          | 0.7824 | 0.6493 | 0.7785 | 48.0191 |
| 0.0768        | 6.78  | 17000 | 0.0784          | 0.7826 | 0.6495 | 0.7785 | 48.0191 |
| 0.0612        | 6.86  | 17200 | 0.0772          | 0.7818 | 0.6485 | 0.7779 | 48.0191 |
| 0.0735        | 6.94  | 17400 | 0.0778          | 0.7817 | 0.6484 | 0.7777 | 48.0191 |
| 0.0662        | 7.02  | 17600 | 0.0780          | 0.7819 | 0.6483 | 0.7778 | 48.0191 |
| 0.0769        | 7.1   | 17800 | 0.0777          | 0.7823 | 0.6488 | 0.7784 | 48.0191 |
| 0.0649        | 7.18  | 18000 | 0.0775          | 0.7818 | 0.6482 | 0.7778 | 48.0191 |
| 0.0749        | 7.26  | 18200 | 0.0774          | 0.7822 | 0.6486 | 0.7781 | 48.0191 |
| 0.0568        | 7.34  | 18400 | 0.0772          | 0.7825 | 0.6488 | 0.7784 | 48.0191 |
| 0.0751        | 7.42  | 18600 | 0.0774          | 0.7822 | 0.6486 | 0.7783 | 48.0191 |
| 0.0564        | 7.5   | 18800 | 0.0773          | 0.7823 | 0.6487 | 0.7782 | 48.0191 |
| 0.0593        | 7.58  | 19000 | 0.0767          | 0.7826 | 0.6492 | 0.7786 | 48.0191 |
| 0.0563        | 7.66  | 19200 | 0.0773          | 0.7826 | 0.6497 | 0.7786 | 48.0191 |
| 0.0686        | 7.74  | 19400 | 0.0771          | 0.7828 | 0.6494 | 0.7789 | 48.0191 |
| 0.0728        | 7.82  | 19600 | 0.0772          | 0.7823 | 0.6494 | 0.7784 | 48.0191 |
| 0.06          | 7.9   | 19800 | 0.0772          | 0.7826 | 0.6491 | 0.7786 | 48.0191 |
| 0.0557        | 7.98  | 20000 | 0.0771          | 0.7828 | 0.6494 | 0.7787 | 48.0191 |


### Framework versions

- Transformers 4.36.2
- Pytorch 2.1.1
- Datasets 2.16.1
- Tokenizers 0.15.0