Update README.md
Browse files
README.md
CHANGED
@@ -81,12 +81,12 @@ model-index:
|
|
81 |
type: squad_it
|
82 |
name: SQuAD-IT
|
83 |
metrics:
|
|
|
|
|
|
|
84 |
- type: f1
|
85 |
value: 0.775
|
86 |
name: Test F1
|
87 |
-
- type: exact-match
|
88 |
-
value: 0.66
|
89 |
-
name: Test Exact Match
|
90 |
train-eval-index:
|
91 |
- config: default
|
92 |
task: question-answering
|
@@ -162,20 +162,20 @@ Answers:
|
|
162 |
|
163 |
## Comparison ⚖️
|
164 |
|
165 |
-
| Model
|
166 |
-
|
167 |
-
| it5/it5-large-question-answering
|
168 |
-
|
|
169 |
-
| it5/it5-base-question-answering
|
170 |
-
| it5/mt5-base-question-answering
|
171 |
-
| antoniocappiello/bert-base-italian-uncased-squad-it
|
172 |
-
| luigisaetta/squad_it_xxl_cased_hub1
|
173 |
-
| it5/it5-efficient-small-el32-question-answering
|
174 |
-
| mrm8488/bert-italian-finedtuned-squadv1-it-alfa
|
175 |
-
| mrm8488/umberto-wikipedia-uncased-v1-finetuned-squadv1-it
|
176 |
-
| it5/it5-small-question-answering
|
177 |
-
| it5/mt5-small-question-answering
|
178 |
-
| DrQA-it trained on SQuAD-it
|
179 |
|
180 |
## Training details 🏋️
|
181 |
[Training notebook](https://github.com/anakin87/electra-italian-xxl-cased-squad-it/blob/main/training.ipynb)
|
|
|
81 |
type: squad_it
|
82 |
name: SQuAD-IT
|
83 |
metrics:
|
84 |
+
- type: exact-match
|
85 |
+
value: 0.66
|
86 |
+
name: Test Exact Match
|
87 |
- type: f1
|
88 |
value: 0.775
|
89 |
name: Test F1
|
|
|
|
|
|
|
90 |
train-eval-index:
|
91 |
- config: default
|
92 |
task: question-answering
|
|
|
162 |
|
163 |
## Comparison ⚖️
|
164 |
|
165 |
+
| Model | EM | F1 | Model size (pytorch) | Architecture |
|
166 |
+
|-----------------------------------------------------------|-------|-------|----------------------|------------------|
|
167 |
+
| it5/it5-large-question-answering | 69.10 | 78.00 | 3.13 GB | encoder-decoder |
|
168 |
+
| anakin87/electra-italian-xxl-cased-squad-it (this one) | 66.03 | 77.47 | 437 MB | encoder |
|
169 |
+
| it5/it5-base-question-answering | 66.30 | 76.10 | 990 MB | encoder-decoder |
|
170 |
+
| it5/mt5-base-question-answering | 66.30 | 75.70 | 2.33 GB | encoder-decoder |
|
171 |
+
| antoniocappiello/bert-base-italian-uncased-squad-it | 63.80 | 75.30 | 440 MB | encoder |
|
172 |
+
| luigisaetta/squad_it_xxl_cased_hub1 | 63.95 | 75.27 | 440 MB | encoder |
|
173 |
+
| it5/it5-efficient-small-el32-question-answering | 64.50 | 74.70 | 569 MB | encoder-decoder |
|
174 |
+
| mrm8488/bert-italian-finedtuned-squadv1-it-alfa | 62.51 | 74.16 | 440 MB | encoder |
|
175 |
+
| mrm8488/umberto-wikipedia-uncased-v1-finetuned-squadv1-it | 60.50 | 72.41 | 443 MB | encoder |
|
176 |
+
| it5/it5-small-question-answering | 61.90 | 71.60 | 308 MB | encoder-decoder |
|
177 |
+
| it5/mt5-small-question-answering | 56.00 | 66.00 | 1.2 GB | encoder-decoder |
|
178 |
+
| DrQA-it trained on SQuAD-it | 56.10 | 65.90 | ? | ? |
|
179 |
|
180 |
## Training details 🏋️
|
181 |
[Training notebook](https://github.com/anakin87/electra-italian-xxl-cased-squad-it/blob/main/training.ipynb)
|