asafaya commited on
Commit
89b8bd0
1 Parent(s): 98030fb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +21 -36
README.md CHANGED
@@ -7,7 +7,7 @@ datasets:
7
  metrics:
8
  - rouge
9
  model-index:
10
- - name: eval-mt5-base-aggressive
11
  results:
12
  - task:
13
  name: Summarization
@@ -22,33 +22,21 @@ model-index:
22
  value: 47.4222
23
  ---
24
 
25
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
26
- should probably proofread and complete it, then remove this comment. -->
27
 
28
- # eval-mt5-base-aggressive
 
 
 
 
29
 
30
- This model is a fine-tuned version of [google/mt5-base](https://huggingface.co/google/mt5-base) on the mlsum tu dataset.
31
  It achieves the following results on the evaluation set:
32
- - Loss: 2.7801
33
  - Rouge1: 47.4222
34
  - Rouge2: 34.8624
35
  - Rougel: 42.2487
36
  - Rougelsum: 43.9494
37
- - Gen Len: 51.3525
38
-
39
- ## Model description
40
-
41
- More information needed
42
-
43
- ## Intended uses & limitations
44
-
45
- More information needed
46
 
47
- ## Training and evaluation data
48
-
49
- More information needed
50
-
51
- ## Training procedure
52
 
53
  ### Training hyperparameters
54
 
@@ -67,25 +55,22 @@ The following hyperparameters were used during training:
67
  - num_epochs: 10.0
68
  - label_smoothing_factor: 0.1
69
 
70
- ### Training results
71
-
72
- | Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
73
- |:-------------:|:-----:|:-----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
74
- | 3.084 | 1.0 | 3895 | 2.9282 | 31.6872 | 22.1113 | 29.2851 | 29.7608 | 18.9861 |
75
- | 2.9162 | 2.0 | 7790 | 2.8552 | 32.1716 | 22.5001 | 29.6845 | 30.1887 | 18.9938 |
76
- | 2.8149 | 3.0 | 11685 | 2.8089 | 32.5681 | 22.689 | 30.0409 | 30.5507 | 18.9959 |
77
- | 2.7325 | 4.0 | 15580 | 2.7948 | 33.1236 | 23.1775 | 30.5156 | 31.0461 | 18.9958 |
78
- | 2.6679 | 5.0 | 19475 | 2.7810 | 33.1766 | 23.162 | 30.4802 | 31.0527 | 18.9967 |
79
- | 2.6237 | 6.0 | 23370 | 2.7790 | 33.1118 | 23.2043 | 30.5064 | 31.0096 | 18.9978 |
80
- | 2.5711 | 7.0 | 27265 | 2.7801 | 33.2033 | 23.2957 | 30.59 | 31.1504 | 18.9979 |
81
- | 2.538 | 8.0 | 31160 | 2.7777 | 33.0256 | 23.0621 | 30.3818 | 30.978 | 18.998 |
82
- | 2.5 | 9.0 | 35055 | 2.7839 | 33.2288 | 23.2361 | 30.5421 | 31.1573 | 18.998 |
83
- | 2.4719 | 10.0 | 38950 | 2.7832 | 33.2098 | 23.2274 | 30.5164 | 31.1094 | 18.9981 |
84
-
85
-
86
  ### Framework versions
87
 
88
  - Transformers 4.11.3
89
  - Pytorch 1.8.2+cu111
90
  - Datasets 1.14.0
91
  - Tokenizers 0.10.3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  metrics:
8
  - rouge
9
  model-index:
10
+ - name: mt5-base-turkish-sum
11
  results:
12
  - task:
13
  name: Summarization
 
22
  value: 47.4222
23
  ---
24
 
 
 
25
 
26
+ # [Mukayese: Turkish NLP Strikes Back](https://arxiv.org/abs/2203.01215)
27
+
28
+ ## Summarization: mukayese/mbart-large-turkish-sum
29
+
30
+ This model is a fine-tuned version of [google/mt5-base](https://huggingface.co/google/mt5-base) on the mlsum/tu dataset.
31
 
 
32
  It achieves the following results on the evaluation set:
33
+
34
  - Rouge1: 47.4222
35
  - Rouge2: 34.8624
36
  - Rougel: 42.2487
37
  - Rougelsum: 43.9494
 
 
 
 
 
 
 
 
 
38
 
39
+ Check [this](https://arxiv.org/abs/2203.01215) paper for more details on the model and the dataset.
 
 
 
 
40
 
41
  ### Training hyperparameters
42
 
 
55
  - num_epochs: 10.0
56
  - label_smoothing_factor: 0.1
57
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
58
  ### Framework versions
59
 
60
  - Transformers 4.11.3
61
  - Pytorch 1.8.2+cu111
62
  - Datasets 1.14.0
63
  - Tokenizers 0.10.3
64
+
65
+ ### Citation
66
+
67
+ ```
68
+ @misc{safaya-etal-2022-mukayese,
69
+ title={Mukayese: Turkish NLP Strikes Back},
70
+ author={Ali Safaya and Emirhan Kurtuluş and Arda Göktoğan and Deniz Yuret},
71
+ year={2022},
72
+ eprint={2203.01215},
73
+ archivePrefix={arXiv},
74
+ primaryClass={cs.CL}
75
+ }
76
+ ```