gonzalez-agirre commited on
Commit
686c8d3
1 Parent(s): 6727ae7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +63 -8
README.md CHANGED
@@ -1,18 +1,42 @@
1
  ---
2
  language:
 
3
  - es
 
4
  license: apache-2.0
 
5
  tags:
 
6
  - "national library of spain"
 
7
  - "spanish"
 
8
  - "bne"
 
9
  - "qa"
 
10
  - "question answering"
 
11
  datasets:
 
12
  - "PlanTL-GOB-ES/SQAC"
 
13
  metrics:
14
  - "f1"
15
-
 
 
 
 
 
 
 
 
 
 
 
 
 
16
  ---
17
 
18
  # Spanish RoBERTa-base trained on BNE finetuned for Spanish Question Answering Corpus (SQAC) dataset.
@@ -26,7 +50,13 @@ metrics:
26
  - [How to use](#how-to-use)
27
  - [Limitations and bias](#limitations-and-bias)
28
  - [Training](#training)
 
 
 
 
29
  - [Evaluation](#evaluation)
 
 
30
  - [Additional information](#additional-information)
31
  - [Author](#author)
32
  - [Contact information](#contact-information)
@@ -39,24 +69,49 @@ metrics:
39
  </details>
40
 
41
  ## Model description
42
- RoBERTa-base-bne is a transformer-based masked language model for the Spanish language. It is based on the [RoBERTa](https://arxiv.org/abs/1907.11692) base model and has been pre-trained using the largest Spanish corpus known to date, with a total of 570GB of clean and deduplicated text processed for this work, compiled from the web crawlings performed by the [National Library of Spain (Biblioteca Nacional de España)](http://www.bne.es/en/Inicio/index.html) from 2009 to 2019.
43
-
44
- Original pre-trained model can be found here: https://huggingface.co/PlanTL-GOB-ES/roberta-base-bne
45
 
46
  ## Intended uses and limitations
47
 
 
 
48
  ## How to use
49
 
 
 
 
 
 
 
 
 
 
 
50
  ## Limitations and bias
51
  At the time of submission, no measures have been taken to estimate the bias embedded in the model. However, we are well aware that our models may be biased since the corpora have been collected using crawling techniques on multiple web sources. We intend to conduct research in these areas in the future, and if completed, this model card will be updated.
52
 
53
  ## Training
54
- The dataset used is the [SQAC corpus](https://huggingface.co/datasets/PlanTL-GOB-ES/SQAC).
55
 
56
- ## Evaluation
57
- F1 Score: 0.7923 (average of 5 runs).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
58
 
59
- For evaluation details visit our [GitHub repository](https://github.com/PlanTL-GOB-ES/lm-spanish).
60
 
61
  ## Additional information
62
 
 
1
  ---
2
  language:
3
+
4
  - es
5
+
6
  license: apache-2.0
7
+
8
  tags:
9
+
10
  - "national library of spain"
11
+
12
  - "spanish"
13
+
14
  - "bne"
15
+
16
  - "qa"
17
+
18
  - "question answering"
19
+
20
  datasets:
21
+
22
  - "PlanTL-GOB-ES/SQAC"
23
+
24
  metrics:
25
  - "f1"
26
+ - "exact match"
27
+
28
+ model-index:
29
+ - name: roberta-base-bne-sqac
30
+ results:
31
+ - task:
32
+ type: question-answering
33
+ dataset:
34
+ type: "PlanTL-GOB-ES/SQAC"
35
+ name: SQAC
36
+ metrics:
37
+ - name: F1
38
+ type: f1
39
+ value: 0.7923
40
  ---
41
 
42
  # Spanish RoBERTa-base trained on BNE finetuned for Spanish Question Answering Corpus (SQAC) dataset.
 
50
  - [How to use](#how-to-use)
51
  - [Limitations and bias](#limitations-and-bias)
52
  - [Training](#training)
53
+ - [Training](#training)
54
+ - [Training data](#training-data)
55
+ - [Training procedure](#training-procedure)
56
+ - [Evaluation](#evaluation)
57
  - [Evaluation](#evaluation)
58
+ - [Variable and metrics](#variable-and-metrics)
59
+ - [Evaluation results](#evaluation-results)
60
  - [Additional information](#additional-information)
61
  - [Author](#author)
62
  - [Contact information](#contact-information)
 
69
  </details>
70
 
71
  ## Model description
72
+ The **roberta-base-bne-sqac** is a Question Answering (QA) model for the Spanish language fine-tuned from the [roberta-base-bne](https://huggingface.co/PlanTL-GOB-ES/roberta-base-bne) model, a [RoBERTa](https://arxiv.org/abs/1907.11692) base model pre-trained using the largest Spanish corpus known to date, with a total of 570GB of clean and deduplicated text, processed for this work, compiled from the web crawlings performed by the [National Library of Spain (Biblioteca Nacional de España)](http://www.bne.es/en/Inicio/index.html) from 2009 to 2019.
 
 
73
 
74
  ## Intended uses and limitations
75
 
76
+ **roberta-base-bne-sqac** model can be used for extractive question answering. The model is limited by its training dataset and may not generalize well for all use cases.
77
+
78
  ## How to use
79
 
80
+ ```python
81
+ from transformers import pipeline
82
+ nlp = pipeline("question-answering", model="PlanTL-GOB-ES/roberta-base-bne-sqac")
83
+ text = "¿Dónde vivo?"
84
+ context = "Me llamo Wolfgang y vivo en Berlin"
85
+
86
+ qa_results = nlp(text, context)
87
+ print(qa_results)
88
+ ```
89
+
90
  ## Limitations and bias
91
  At the time of submission, no measures have been taken to estimate the bias embedded in the model. However, we are well aware that our models may be biased since the corpora have been collected using crawling techniques on multiple web sources. We intend to conduct research in these areas in the future, and if completed, this model card will be updated.
92
 
93
  ## Training
 
94
 
95
+ ### Training data
96
+ We used the QA dataset in Spanish called [SQAC corpus](https://huggingface.co/datasets/PlanTL-GOB-ES/SQAC) for training and evaluation.
97
+
98
+ ### Training procedure
99
+ The model was trained with a batch size of 16 and a learning rate of 5e-5 for 5 epochs. We then selected the best checkpoint using the downstream task metric in the corresponding development set and then evaluated it on the test set.
100
+
101
+ ## Evaluation results
102
+ We evaluated the **roberta-base-bne-sqac** on the SQAC test set against standard multilingual and monolingual baselines:
103
+
104
+
105
+ | Model | SQAC (F1) |
106
+ | ------------|:----|
107
+ | roberta-large-bne-sqac | **82.02** |
108
+ | roberta-base-bne-sqac | 79.23|
109
+ | BETO | 79.23 |
110
+ | mBERT | 75.62 |
111
+ | BERTIN | 76.78 |
112
+ | ELECTRA | 73.83 |
113
 
114
+ For more details, check the fine-tuning and evaluation scripts in the official [GitHub repository](https://github.com/PlanTL-GOB-ES/lm-spanish).
115
 
116
  ## Additional information
117