ixa-ehu commited on
Commit
75ed8dd
1 Parent(s): 845d307

README updated

Browse files
Files changed (1) hide show
  1. README.md~ +0 -41
README.md~ DELETED
@@ -1,41 +0,0 @@
1
- ---
2
- language: eu
3
- license: cc-by-4.0
4
- tags:
5
- - basque
6
- - roberta
7
- ---
8
-
9
- # Roberta-eus Euscrawl base cased
10
-
11
- This is a RoBERTa model for Basque model presented in [Does corpus quality really matter for low-resource languages?](https://arxiv.org/abs/2203.08111). There are several models for Basque using the RoBERTa architecture, which are pre-trained using different corpora:
12
-
13
- - roberta-eus-euscrawl-base-cased: Basque RoBERTa trained on Euscrawl, a corpus created using tailored crawling from Basque sites, and which is distributed under a CC-BY license. EusCrawl It contains 12,528k documents and 423M tokens.
14
- - roberta-eus-euscrawl-large-cased: Basque RoBERTa large trained on EusCrawl.
15
- - roberta-eus-mC4-base-cased: Basque RoBERTa trained on the Basque portion of mc4 dataset.
16
- - roberta-eus-CC100-base-cased: Basque RoBERTa trained on Basque portion of cc100 dataset.
17
-
18
- The models have been tested on five different downstream tasks for Basque: Topic classification, Sentiment analysis, Stance detection, Named Entity Recognition (NER), and Question Answering (refer to the [paper](https://arxiv.org/abs/2203.08111) for more details). See summary of results below:
19
-
20
-
21
- | Model | Topic class. | Sentiment | Stance det. | NER | QA | Average |
22
- |----------------------------------|--------------|-----------|-------------|----------|----------|----------|
23
- | roberta-eus-euscrawl-base-cased | 76.2 | 77.7 | 57.4 | 86.8 | 34.6 | 66.5 |
24
- | roberta-eus-euscrawl-large-cased | **77.6** | 78.8 | 62.9 | **87.2** | **38.3** | **69.0** |
25
- | roberta-eus-mC4-base-cased | 75.3 | **80.4** | 59.1 | 86.0 | 35.2 | 67.2 |
26
- | roberta-eus-CC100-base-cased | 76.2 | 78.8 | **63.4** | 85.2 | 35.8 | 67.9 |
27
-
28
-
29
- If you use any of these models, please cite the following paper:
30
-
31
- ```
32
- @misc{artetxe2022euscrawl,
33
- title={Does corpus quality really matter for low-resource languages?},
34
- author={Mikel Artetxe, Itziar Aldabe, Rodrigo Agerri,
35
- Olatz Perez-de-Viñaspre, Aitor Soroa},
36
- year={2022},
37
- eprint={2203.08111},
38
- archivePrefix={arXiv},
39
- primaryClass={cs.CL}
40
- }
41
- ```