osiria commited on
Commit
002a527
1 Parent(s): 605fe9a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +103 -0
README.md CHANGED
@@ -1,3 +1,106 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ language:
4
+ - it
5
+ datasets:
6
+ - squad_it
7
+ widget:
8
+ - text: Quale libro fu scritto da Alessandro Manzoni?
9
+ context: Alessandro Manzoni pubblicò la prima versione de I Promessi Sposi nel 1827
10
+ - text: In quali competizioni gareggia la Ferrari?
11
+ context: La Scuderia Ferrari è una squadra corse italiana di Formula 1 con sede a Maranello
12
+ - text: Quale sport è riferito alla Serie A?
13
+ context: Il campionato di Serie A è la massima divisione professionistica del campionato italiano di calcio maschile
14
+ model-index:
15
+ - name: osiria/bert-italian-cased-question-answering
16
+ results:
17
+ - task:
18
+ type: question-answering
19
+ name: Question Answering
20
+ dataset:
21
+ name: squad_it
22
+ type: squad_it
23
+ metrics:
24
+ - type: exact-match
25
+ value: 0.6572
26
+ name: Exact Match
27
+ - type: f1
28
+ value: 0.7706
29
+ name: F1
30
+ pipeline_tag: question-answering
31
  ---
32
+
33
+ --------------------------------------------------------------------------------------------------
34
+
35
+ <body>
36
+ <span class="vertical-text" style="background-color:lightgreen;border-radius: 3px;padding: 3px;"> </span>
37
+ <br>
38
+ <span class="vertical-text" style="background-color:orange;border-radius: 3px;padding: 3px;">    Task: Question Answering</span>
39
+ <br>
40
+ <span class="vertical-text" style="background-color:lightblue;border-radius: 3px;padding: 3px;">    Model: BERT</span>
41
+ <br>
42
+ <span class="vertical-text" style="background-color:tomato;border-radius: 3px;padding: 3px;">    Lang: IT</span>
43
+ <br>
44
+ <span class="vertical-text" style="background-color:lightgrey;border-radius: 3px;padding: 3px;">  </span>
45
+ <br>
46
+ <span class="vertical-text" style="background-color:#CF9FFF;border-radius: 3px;padding: 3px;"> </span>
47
+ </body>
48
+
49
+ --------------------------------------------------------------------------------------------------
50
+
51
+ <h3>Model description</h3>
52
+
53
+ This is a <b>BERT</b> <b>[1]</b> model for the <b>Italian</b> language, fine-tuned for <b>Extractive Question Answering</b> on the [SQuAD-IT](https://huggingface.co/datasets/squad_it) dataset <b>[2]</b>
54
+
55
+ If you are looking for a more accurate (but slightly heavier) model, you can refer to: https://huggingface.co/osiria/deberta-italian-question-answering
56
+
57
+ <b>update: version 2.0</b>
58
+
59
+ The 2.0 version further improves the performances by exploiting a 2-phases fine-tuning strategy: the model is first fine-tuned on the English SQuAD v2 (1 epoch, 20% warmup ratio, and max learning rate of 3e-5) then further fine-tuned on the Italian SQuAD (2 epochs, no warmup, initial learning rate of 3e-5)
60
+
61
+ In order to maximize the benefits of the multilingual procedure, [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) is used as a pre-trained model. When the double fine-tuning is completed, the embedding layer is then compressed as in [bert-base-italian-cased](https://huggingface.co/osiria/bert-base-italian-cased) to obtain a mono-lingual model size
62
+
63
+
64
+ <h3>Training and Performances</h3>
65
+
66
+ The model is trained to perform question answering, given a context and a question (under the assumption that the context contains the answer to the question). It has been fine-tuned for Extractive Question Answering, using the SQuAD-IT dataset, for 2 epochs with a linearly decaying learning rate starting from 3e-5, maximum sequence length of 384 and document stride of 128.
67
+ <br>The dataset includes 54.159 training instances and 7.609 test instances
68
+
69
+ The performances on the test set are reported in the following table:
70
+
71
+ | EM | F1 |
72
+ | ------ | ------ |
73
+ | 65.72 | 77.06 |
74
+
75
+ Testing notebook: https://huggingface.co/osiria/bert-italian-cased-question-answering/blob/main/osiria_bert_italian_cased_qa_evaluation.ipynb
76
+
77
+ <h3>Quick usage</h3>
78
+
79
+ ```python
80
+ from transformers import BertTokenizerFast, BertForQuestionAnswering
81
+ from transformers import pipeline
82
+
83
+ tokenizer = BertTokenizerFast.from_pretrained("osiria/bert-italian-cased-question-answering")
84
+ model = BertForQuestionAnswering.from_pretrained("osiria/bert-italian-cased-question-answering")
85
+
86
+ pipeline_qa = pipeline("question-answering", model = model, tokenizer = tokenizer)
87
+ pipeline_qa(context = "Alessandro Manzoni è nato a Milano nel 1785", question = "Dove è nato Manzoni?")
88
+
89
+ # {'score': 0.9922313690185547, 'start': 28, 'end': 34, 'answer': 'Milano'}
90
+ ```
91
+
92
+ <h3>References</h3>
93
+
94
+ [1] https://arxiv.org/abs/1810.04805
95
+
96
+ [2] https://link.springer.com/chapter/10.1007/978-3-030-03840-3_29
97
+
98
+ <h3>Limitations</h3>
99
+
100
+ This model was trained SQuAD-IT which is mainly a machine translated version of the original SQuAD v1.1. This means that the quality of the training set is limited by the machine translation.
101
+ Moreover, the model is meant to answer questions under the assumption that the required information is actually contained in the given context (which is the underlying assumption of SQuAD v1.1).
102
+ If the assumption is violated, the model will try to return an answer in any case, which is going to be incorrect.
103
+
104
+ <h3>License</h3>
105
+
106
+ The model is released under <b>Apache-2.0</b> license