ixa-ehu commited on
Commit
f825e2b
1 Parent(s): 1b16b99

roberta-eus-mc4-base-cased upload

Browse files
README.md CHANGED
@@ -1,3 +1,41 @@
1
- ---
2
- license: cc-by-nc-4.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: eu
3
+ license: cc-by-nc-4.0
4
+ tags:
5
+ - basque
6
+ - roberta
7
+ ---
8
+
9
+ # Roberta-eus Euscrawl base cased
10
+
11
+ This is a RoBERTa model for Basque model presented in [Does corpus quality really matter for low-resource languages?](https://arxiv.org/abs/2203.08111). There are several models for Basque using the RoBERTa architecture, using different corpora:
12
+
13
+ - roberta-eus-euscrawl-base-cased: Basque RoBERTa model trained on Euscrawl, a corpus created using tailored crawling from Basque sites, and which is distributed under a CC-BY license. EusCrawl It contains 12,528k documents and 423M tokens.
14
+ - roberta-eus-euscrawl-large-cased: RoBERTa large trained on EusCrawl.
15
+ - roberta-eus-mC4-base-cased: Basque RoBERTa model trained on the Basque portion of mc4 dataset.
16
+ - roberta-eus-CC100-base-cased: Basque RoBERTa model trained on Basque portion of cc100 dataset.
17
+
18
+ The models have been tested on five different downstream tasks for Basque: Topic classification, Sentiment analysis, Stance detection, Named Entity Recognition (NER), and Question Answering (refer to the [paper](https://arxiv.org/abs/2203.08111) for more details). See summary of results below:
19
+
20
+
21
+ | Model | Topic class. | Sentiment | Stance det. | NER | QA | Average |
22
+ |----------------------------------|--------------|-----------|-------------|----------|----------|----------|
23
+ | roberta-eus-euscrawl-base-cased | 76.2 | 77.7 | 57.4 | 86.8 | 34.6 | 66.5 |
24
+ | roberta-eus-euscrawl-large-cased | **77.6** | 78.8 | 62.9 | **87.2** | **38.3** | **69.0** |
25
+ | roberta-eus-mC4-base-cased | 75.3 | **80.4** | 59.1 | 86.0 | 35.2 | 67.2 |
26
+ | roberta-eus-CC100-base-cased | 76.2 | 78.8 | **63.4** | 85.2 | 35.8 | 67.9 |
27
+
28
+
29
+ If you use any of these models, please cite the following paper:
30
+
31
+ ```
32
+ @misc{artetxe2022euscrawl,
33
+ title={Does corpus quality really matter for low-resource languages?},
34
+ author={Mikel Artetxe, Itziar Aldabe, Rodrigo Agerri,
35
+ Olatz Perez-de-Viñaspre, Aitor Soroa},
36
+ year={2022},
37
+ eprint={2203.08111},
38
+ archivePrefix={arXiv},
39
+ primaryClass={cs.CL}
40
+ }
41
+ ```
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:79f0d38a703743006c582e05c2cf373336b00bedcf1c111dbfbe8756e8df9a32
3
+ size 651891543
sentencepiece.bpe.model ADDED
@@ -0,0 +1,3 @@
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:81d2a5b4e48753cf629457c3cdfa55f1d5c369194db112550e5e1ea101342e70
3
+ size 1117898
special_tokens_map.json ADDED
@@ -0,0 +1 @@
 
1
+ {"bos_token": "<s>", "eos_token": "</s>", "unk_token": "<unk>", "sep_token": "</s>", "pad_token": "<pad>", "cls_token": "<s>", "mask_token": {"content": "<mask>", "single_word": false, "lstrip": true, "rstrip": false, "normalized": true}}
tokenizer_config.json ADDED
@@ -0,0 +1 @@
 
1
+ {"do_lower_case":false, "bos_token": "<s>", "eos_token": "</s>", "unk_token": "<unk>", "sep_token": "</s>", "cls_token": "<s>", "pad_token": "<pad>", "mask_token": {"content": "<mask>", "single_word": false, "lstrip": true, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "sp_model_kwargs": {}, "tokenizer_class": "XLMRobertaTokenizer"}