Hugging Face's logo

eglkan1
/
mt5-lithuanian-simplifier-full

Text2Text Generation
Transformers
TensorBoard
Safetensors
t5
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
eglkan1 commited on
Commit
e26cfc9
•
1 Parent(s): 4131cd5

End of training

Browse files
Files changed (9) hide show
  1. README.md +163 -0
  2. config.json +33 -0
  3. generation_config.json +7 -0
  4. logs/events.out.tfevents.1706106930.ESBeastModeOn.27508.0 +3 -0
  5. model.safetensors +3 -0
  6. special_tokens_map.json +23 -0
  7. spiece.model +3 -0
  8. tokenizer_config.json +38 -0
  9. training_args.bin +3 -0
README.md ADDED
@@ -0,0 +1,163 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: google/mt5-base
4
+ tags:
5
+ - generated_from_trainer
6
+ metrics:
7
+ - rouge
8
+ model-index:
9
+ - name: mt5-lithuanian-simplifier-full
10
+ results: []
11
+ ---
12
+
13
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
+ should probably proofread and complete it, then remove this comment. -->
15
+
16
+ # mt5-lithuanian-simplifier-full
17
+
18
+ This model is a fine-tuned version of [google/mt5-base](https://huggingface.co/google/mt5-base) on the None dataset.
19
+ It achieves the following results on the evaluation set:
20
+ - Loss: 0.0771
21
+ - Rouge1: 0.7828
22
+ - Rouge2: 0.6494
23
+ - Rougel: 0.7787
24
+ - Gen Len: 48.0191
25
+
26
+ ## Model description
27
+
28
+ More information needed
29
+
30
+ ## Intended uses & limitations
31
+
32
+ More information needed
33
+
34
+ ## Training and evaluation data
35
+
36
+ More information needed
37
+
38
+ ## Training procedure
39
+
40
+ ### Training hyperparameters
41
+
42
+ The following hyperparameters were used during training:
43
+ - learning_rate: 0.0001
44
+ - train_batch_size: 4
45
+ - eval_batch_size: 4
46
+ - seed: 42
47
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
48
+ - lr_scheduler_type: linear
49
+ - lr_scheduler_warmup_steps: 500
50
+ - num_epochs: 8
51
+
52
+ ### Training results
53
+
54
+ | Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Gen Len |
55
+ |:-------------:|:-----:|:-----:|:---------------:|:------:|:------:|:------:|:-------:|
56
+ | 24.351 | 0.08 | 200 | 18.6244 | 0.0226 | 0.0018 | 0.0207 | 512.0 |
57
+ | 3.0331 | 0.16 | 400 | 0.6830 | 0.0549 | 0.0018 | 0.0497 | 49.0191 |
58
+ | 0.2076 | 0.24 | 600 | 0.1642 | 0.6417 | 0.4986 | 0.6328 | 48.0191 |
59
+ | 0.2019 | 0.32 | 800 | 0.1303 | 0.6713 | 0.5243 | 0.6633 | 48.0191 |
60
+ | 0.1573 | 0.4 | 1000 | 0.1242 | 0.7007 | 0.5589 | 0.6937 | 48.0191 |
61
+ | 0.1687 | 0.48 | 1200 | 0.1158 | 0.712 | 0.569 | 0.7055 | 48.0191 |
62
+ | 0.1315 | 0.56 | 1400 | 0.1225 | 0.6923 | 0.5361 | 0.6851 | 48.0191 |
63
+ | 0.1376 | 0.64 | 1600 | 0.1108 | 0.7171 | 0.5695 | 0.7105 | 48.0191 |
64
+ | 0.158 | 0.72 | 1800 | 0.1074 | 0.7229 | 0.574 | 0.7169 | 48.0191 |
65
+ | 0.1221 | 0.8 | 2000 | 0.1064 | 0.7227 | 0.5761 | 0.7166 | 48.0191 |
66
+ | 0.1371 | 0.88 | 2200 | 0.1049 | 0.7282 | 0.5827 | 0.7223 | 48.0191 |
67
+ | 0.1376 | 0.96 | 2400 | 0.1043 | 0.73 | 0.5861 | 0.7239 | 48.0191 |
68
+ | 0.1116 | 1.04 | 2600 | 0.1021 | 0.733 | 0.5888 | 0.727 | 48.0191 |
69
+ | 0.132 | 1.12 | 2800 | 0.1012 | 0.7338 | 0.5899 | 0.7277 | 48.0191 |
70
+ | 0.131 | 1.2 | 3000 | 0.0997 | 0.7365 | 0.5936 | 0.7307 | 48.0191 |
71
+ | 0.1001 | 1.28 | 3200 | 0.0950 | 0.7408 | 0.5977 | 0.7355 | 48.0191 |
72
+ | 0.1398 | 1.36 | 3400 | 0.0964 | 0.7418 | 0.599 | 0.7364 | 48.0191 |
73
+ | 0.1085 | 1.44 | 3600 | 0.0962 | 0.744 | 0.6015 | 0.7386 | 48.0191 |
74
+ | 0.097 | 1.52 | 3800 | 0.0967 | 0.743 | 0.6009 | 0.7377 | 48.0191 |
75
+ | 0.1178 | 1.6 | 4000 | 0.0955 | 0.7446 | 0.6035 | 0.7391 | 48.0191 |
76
+ | 0.1214 | 1.68 | 4200 | 0.0939 | 0.7452 | 0.6036 | 0.7403 | 48.0191 |
77
+ | 0.1539 | 1.76 | 4400 | 0.0909 | 0.7486 | 0.6068 | 0.7436 | 48.0191 |
78
+ | 0.1141 | 1.83 | 4600 | 0.0900 | 0.7518 | 0.6104 | 0.7467 | 48.0191 |
79
+ | 0.0795 | 1.91 | 4800 | 0.0891 | 0.7513 | 0.6097 | 0.7466 | 48.0191 |
80
+ | 0.0856 | 1.99 | 5000 | 0.0915 | 0.7513 | 0.6099 | 0.7463 | 48.0191 |
81
+ | 0.0954 | 2.07 | 5200 | 0.0898 | 0.753 | 0.6126 | 0.7482 | 48.0191 |
82
+ | 0.1271 | 2.15 | 5400 | 0.0901 | 0.7534 | 0.6125 | 0.7486 | 48.0191 |
83
+ | 0.0816 | 2.23 | 5600 | 0.0893 | 0.7553 | 0.6148 | 0.7506 | 48.0191 |
84
+ | 0.0922 | 2.31 | 5800 | 0.0881 | 0.7569 | 0.6163 | 0.7521 | 48.0191 |
85
+ | 0.1177 | 2.39 | 6000 | 0.0878 | 0.7575 | 0.6176 | 0.7532 | 48.0191 |
86
+ | 0.0916 | 2.47 | 6200 | 0.0874 | 0.7585 | 0.618 | 0.7541 | 48.0191 |
87
+ | 0.1349 | 2.55 | 6400 | 0.0861 | 0.76 | 0.62 | 0.7555 | 48.0191 |
88
+ | 0.1196 | 2.63 | 6600 | 0.0833 | 0.7617 | 0.6212 | 0.7572 | 48.0191 |
89
+ | 0.0841 | 2.71 | 6800 | 0.0848 | 0.7621 | 0.6219 | 0.7576 | 48.0191 |
90
+ | 0.0934 | 2.79 | 7000 | 0.0854 | 0.7622 | 0.6227 | 0.7577 | 48.0191 |
91
+ | 0.1246 | 2.87 | 7200 | 0.0835 | 0.7652 | 0.6256 | 0.7606 | 48.0191 |
92
+ | 0.0762 | 2.95 | 7400 | 0.0835 | 0.7649 | 0.6262 | 0.7606 | 48.0191 |
93
+ | 0.0924 | 3.03 | 7600 | 0.0828 | 0.7662 | 0.6276 | 0.7618 | 48.0191 |
94
+ | 0.0822 | 3.11 | 7800 | 0.0834 | 0.7664 | 0.6284 | 0.7621 | 48.0191 |
95
+ | 0.0856 | 3.19 | 8000 | 0.0836 | 0.7647 | 0.627 | 0.7603 | 48.0191 |
96
+ | 0.0798 | 3.27 | 8200 | 0.0829 | 0.7657 | 0.6284 | 0.7614 | 48.0191 |
97
+ | 0.0959 | 3.35 | 8400 | 0.0828 | 0.7671 | 0.6302 | 0.7629 | 48.0191 |
98
+ | 0.0871 | 3.43 | 8600 | 0.0820 | 0.7672 | 0.6297 | 0.763 | 48.0191 |
99
+ | 0.1068 | 3.51 | 8800 | 0.0827 | 0.7683 | 0.6307 | 0.7641 | 48.0191 |
100
+ | 0.072 | 3.59 | 9000 | 0.0820 | 0.7684 | 0.632 | 0.764 | 48.0191 |
101
+ | 0.0964 | 3.67 | 9200 | 0.0838 | 0.7692 | 0.6333 | 0.7645 | 48.0191 |
102
+ | 0.0946 | 3.75 | 9400 | 0.0809 | 0.7707 | 0.6348 | 0.7663 | 48.0191 |
103
+ | 0.0822 | 3.83 | 9600 | 0.0825 | 0.7708 | 0.6347 | 0.7666 | 48.0191 |
104
+ | 0.1019 | 3.91 | 9800 | 0.0788 | 0.7733 | 0.6373 | 0.7692 | 48.0191 |
105
+ | 0.08 | 3.99 | 10000 | 0.0797 | 0.7727 | 0.6369 | 0.7686 | 48.0191 |
106
+ | 0.0989 | 4.07 | 10200 | 0.0818 | 0.7724 | 0.6367 | 0.7681 | 48.0191 |
107
+ | 0.0693 | 4.15 | 10400 | 0.0804 | 0.7737 | 0.6378 | 0.7697 | 48.0191 |
108
+ | 0.0763 | 4.23 | 10600 | 0.0814 | 0.7741 | 0.6379 | 0.7699 | 48.0191 |
109
+ | 0.0956 | 4.31 | 10800 | 0.0815 | 0.7726 | 0.6369 | 0.7683 | 48.0191 |
110
+ | 0.0728 | 4.39 | 11000 | 0.0800 | 0.7738 | 0.6374 | 0.7696 | 48.0191 |
111
+ | 0.0652 | 4.47 | 11200 | 0.0795 | 0.7747 | 0.6388 | 0.7708 | 48.0191 |
112
+ | 0.0706 | 4.55 | 11400 | 0.0798 | 0.7742 | 0.6388 | 0.7703 | 48.0191 |
113
+ | 0.0979 | 4.63 | 11600 | 0.0788 | 0.7748 | 0.6387 | 0.7708 | 48.0191 |
114
+ | 0.0771 | 4.71 | 11800 | 0.0797 | 0.775 | 0.6402 | 0.771 | 48.0191 |
115
+ | 0.1067 | 4.79 | 12000 | 0.0779 | 0.7757 | 0.6404 | 0.7717 | 48.0191 |
116
+ | 0.0773 | 4.87 | 12200 | 0.0783 | 0.7759 | 0.6411 | 0.7721 | 48.0191 |
117
+ | 0.0866 | 4.95 | 12400 | 0.0780 | 0.7773 | 0.6437 | 0.7734 | 48.0191 |
118
+ | 0.0611 | 5.03 | 12600 | 0.0785 | 0.7761 | 0.6418 | 0.7723 | 48.0191 |
119
+ | 0.0685 | 5.11 | 12800 | 0.0781 | 0.777 | 0.6421 | 0.773 | 48.0191 |
120
+ | 0.0501 | 5.19 | 13000 | 0.0788 | 0.7764 | 0.6411 | 0.7721 | 48.0191 |
121
+ | 0.0626 | 5.27 | 13200 | 0.0792 | 0.7762 | 0.6416 | 0.7721 | 48.0191 |
122
+ | 0.0708 | 5.35 | 13400 | 0.0795 | 0.7761 | 0.6408 | 0.772 | 48.0191 |
123
+ | 0.055 | 5.42 | 13600 | 0.0779 | 0.7773 | 0.642 | 0.7733 | 48.0191 |
124
+ | 0.0749 | 5.5 | 13800 | 0.0789 | 0.7783 | 0.6431 | 0.7742 | 48.0191 |
125
+ | 0.0771 | 5.58 | 14000 | 0.0779 | 0.778 | 0.6437 | 0.774 | 48.0191 |
126
+ | 0.0906 | 5.66 | 14200 | 0.0779 | 0.7781 | 0.6431 | 0.7742 | 48.0191 |
127
+ | 0.0679 | 5.74 | 14400 | 0.0778 | 0.7783 | 0.6449 | 0.7745 | 48.0191 |
128
+ | 0.0605 | 5.82 | 14600 | 0.0786 | 0.7778 | 0.6439 | 0.7738 | 48.0191 |
129
+ | 0.0647 | 5.9 | 14800 | 0.0781 | 0.7785 | 0.6445 | 0.7743 | 48.0191 |
130
+ | 0.058 | 5.98 | 15000 | 0.0775 | 0.7792 | 0.6448 | 0.7749 | 48.0191 |
131
+ | 0.0574 | 6.06 | 15200 | 0.0788 | 0.7793 | 0.6451 | 0.7752 | 48.0191 |
132
+ | 0.0545 | 6.14 | 15400 | 0.0778 | 0.7802 | 0.6464 | 0.7759 | 48.0191 |
133
+ | 0.079 | 6.22 | 15600 | 0.0781 | 0.7801 | 0.6466 | 0.7759 | 48.0191 |
134
+ | 0.0474 | 6.3 | 15800 | 0.0782 | 0.7809 | 0.6477 | 0.7768 | 48.0191 |
135
+ | 0.0517 | 6.38 | 16000 | 0.0788 | 0.7809 | 0.6481 | 0.7769 | 48.0191 |
136
+ | 0.0613 | 6.46 | 16200 | 0.0782 | 0.7814 | 0.6481 | 0.7773 | 48.0191 |
137
+ | 0.0517 | 6.54 | 16400 | 0.0785 | 0.7807 | 0.6468 | 0.7767 | 48.0191 |
138
+ | 0.0549 | 6.62 | 16600 | 0.0778 | 0.7817 | 0.6485 | 0.7777 | 48.0191 |
139
+ | 0.0727 | 6.7 | 16800 | 0.0774 | 0.7824 | 0.6493 | 0.7785 | 48.0191 |
140
+ | 0.0768 | 6.78 | 17000 | 0.0784 | 0.7826 | 0.6495 | 0.7785 | 48.0191 |
141
+ | 0.0612 | 6.86 | 17200 | 0.0772 | 0.7818 | 0.6485 | 0.7779 | 48.0191 |
142
+ | 0.0735 | 6.94 | 17400 | 0.0778 | 0.7817 | 0.6484 | 0.7777 | 48.0191 |
143
+ | 0.0662 | 7.02 | 17600 | 0.0780 | 0.7819 | 0.6483 | 0.7778 | 48.0191 |
144
+ | 0.0769 | 7.1 | 17800 | 0.0777 | 0.7823 | 0.6488 | 0.7784 | 48.0191 |
145
+ | 0.0649 | 7.18 | 18000 | 0.0775 | 0.7818 | 0.6482 | 0.7778 | 48.0191 |
146
+ | 0.0749 | 7.26 | 18200 | 0.0774 | 0.7822 | 0.6486 | 0.7781 | 48.0191 |
147
+ | 0.0568 | 7.34 | 18400 | 0.0772 | 0.7825 | 0.6488 | 0.7784 | 48.0191 |
148
+ | 0.0751 | 7.42 | 18600 | 0.0774 | 0.7822 | 0.6486 | 0.7783 | 48.0191 |
149
+ | 0.0564 | 7.5 | 18800 | 0.0773 | 0.7823 | 0.6487 | 0.7782 | 48.0191 |
150
+ | 0.0593 | 7.58 | 19000 | 0.0767 | 0.7826 | 0.6492 | 0.7786 | 48.0191 |
151
+ | 0.0563 | 7.66 | 19200 | 0.0773 | 0.7826 | 0.6497 | 0.7786 | 48.0191 |
152
+ | 0.0686 | 7.74 | 19400 | 0.0771 | 0.7828 | 0.6494 | 0.7789 | 48.0191 |
153
+ | 0.0728 | 7.82 | 19600 | 0.0772 | 0.7823 | 0.6494 | 0.7784 | 48.0191 |
154
+ | 0.06 | 7.9 | 19800 | 0.0772 | 0.7826 | 0.6491 | 0.7786 | 48.0191 |
155
+ | 0.0557 | 7.98 | 20000 | 0.0771 | 0.7828 | 0.6494 | 0.7787 | 48.0191 |
156
+
157
+
158
+ ### Framework versions
159
+
160
+ - Transformers 4.36.2
161
+ - Pytorch 2.1.1
162
+ - Datasets 2.16.1
163
+ - Tokenizers 0.15.0
config.json ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "google/mt5-base",
3
+ "architectures": [
4
+ "T5ForConditionalGeneration"
5
+ ],
6
+ "classifier_dropout": 0.0,
7
+ "d_ff": 2048,
8
+ "d_kv": 64,
9
+ "d_model": 768,
10
+ "decoder_start_token_id": 0,
11
+ "dense_act_fn": "gelu_new",
12
+ "dropout_rate": 0.1,
13
+ "eos_token_id": 1,
14
+ "feed_forward_proj": "gated-gelu",
15
+ "initializer_factor": 1.0,
16
+ "is_encoder_decoder": true,
17
+ "is_gated_act": true,
18
+ "layer_norm_epsilon": 1e-06,
19
+ "model_type": "t5",
20
+ "num_decoder_layers": 12,
21
+ "num_heads": 12,
22
+ "num_layers": 12,
23
+ "output_past": true,
24
+ "pad_token_id": 0,
25
+ "relative_attention_max_distance": 128,
26
+ "relative_attention_num_buckets": 32,
27
+ "tie_word_embeddings": false,
28
+ "tokenizer_class": "T5Tokenizer",
29
+ "torch_dtype": "float32",
30
+ "transformers_version": "4.36.2",
31
+ "use_cache": true,
32
+ "vocab_size": 250112
33
+ }
generation_config.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "decoder_start_token_id": 0,
4
+ "eos_token_id": 1,
5
+ "pad_token_id": 0,
6
+ "transformers_version": "4.36.2"
7
+ }
logs/events.out.tfevents.1706106930.ESBeastModeOn.27508.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:27a0216cecef302f69897473728cbda53dc7264f67c3fbaf4ea3d0316abc5e5e
3
+ size 368158
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5191fd20b961116cff835c6cdef4e9327c390141a95c7525582b01235c581d56
3
+ size 2329638768
special_tokens_map.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "eos_token": {
3
+ "content": "</s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "pad_token": {
10
+ "content": "<pad>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "unk_token": {
17
+ "content": "<unk>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ }
23
+ }
spiece.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef78f86560d809067d12bac6c09f19a462cb3af3f54d2b8acbba26e1433125d6
3
+ size 4309802
tokenizer_config.json ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "<pad>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "</s>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "<unk>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ }
27
+ },
28
+ "additional_special_tokens": [],
29
+ "clean_up_tokenization_spaces": true,
30
+ "eos_token": "</s>",
31
+ "extra_ids": 0,
32
+ "legacy": true,
33
+ "model_max_length": 1000000000000000019884624838656,
34
+ "pad_token": "<pad>",
35
+ "sp_model_kwargs": {},
36
+ "tokenizer_class": "T5Tokenizer",
37
+ "unk_token": "<unk>"
38
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c6475e95b39404a5c2b714f9aea7139913707c518ae4b839226f54f407642918
3
+ size 4728