roberta-eus-mc4-base-cased upload

README.md

@@ -1,3 +1,41 @@
----
-license: cc-by-nc-4.0
----
+---
+language: eu
+license: cc-by-nc-4.0
+tags:
+- basque
+- roberta
+---
+
+# Roberta-eus Euscrawl base cased
+
+This is a RoBERTa model for Basque model presented in [Does corpus quality really matter for low-resource languages?](https://arxiv.org/abs/2203.08111). There are several models for Basque using the RoBERTa architecture, using different corpora:
+
+- roberta-eus-euscrawl-base-cased: Basque RoBERTa model trained on Euscrawl, a corpus created using tailored crawling from Basque sites, and which is distributed under a CC-BY license. EusCrawl It contains 12,528k documents and 423M tokens.
+- roberta-eus-euscrawl-large-cased: RoBERTa large trained on EusCrawl.
+- roberta-eus-mC4-base-cased: Basque RoBERTa model trained on the Basque portion of mc4 dataset.
+- roberta-eus-CC100-base-cased: Basque RoBERTa model trained on  Basque portion of cc100 dataset.
+
+The models have been tested on five different downstream tasks for Basque: Topic classification, Sentiment analysis, Stance detection, Named Entity Recognition (NER), and Question Answering (refer to the [paper](https://arxiv.org/abs/2203.08111) for more details). See summary of results below:
+
+
+| Model                            | Topic class. | Sentiment | Stance det. |     NER  |     QA   | Average  |
+|----------------------------------|--------------|-----------|-------------|----------|----------|----------|
+| roberta-eus-euscrawl-base-cased  |         76.2 |      77.7 |        57.4 |    86.8  |    34.6  |    66.5  |
+| roberta-eus-euscrawl-large-cased |     **77.6** |      78.8 |        62.9 | **87.2** | **38.3** | **69.0** |
+| roberta-eus-mC4-base-cased       |         75.3 |  **80.4** |        59.1 |    86.0  |    35.2  |    67.2  |
+| roberta-eus-CC100-base-cased     |         76.2 |      78.8 |    **63.4** |    85.2  |    35.8  |    67.9  |
+
+
+If you use any of these models, please cite the following paper:
+
+```
+@misc{artetxe2022euscrawl,
+ title={Does corpus quality really matter for low-resource languages?},
+ author={Mikel Artetxe, Itziar Aldabe, Rodrigo Agerri,
+         Olatz Perez-de-Viñaspre, Aitor Soroa},
+ year={2022},
+ eprint={2203.08111},
+ archivePrefix={arXiv},
+ primaryClass={cs.CL}
+}
+```

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79f0d38a703743006c582e05c2cf373336b00bedcf1c111dbfbe8756e8df9a32
+size 651891543

sentencepiece.bpe.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:81d2a5b4e48753cf629457c3cdfa55f1d5c369194db112550e5e1ea101342e70
+size 1117898

special_tokens_map.json

@@ -0,0 +1 @@
+{"bos_token": "<s>", "eos_token": "</s>", "unk_token": "<unk>", "sep_token": "</s>", "pad_token": "<pad>", "cls_token": "<s>", "mask_token": {"content": "<mask>", "single_word": false, "lstrip": true, "rstrip": false, "normalized": true}}

tokenizer_config.json

@@ -0,0 +1 @@
+{"do_lower_case":false, "bos_token": "<s>", "eos_token": "</s>", "unk_token": "<unk>", "sep_token": "</s>", "cls_token": "<s>", "pad_token": "<pad>", "mask_token": {"content": "<mask>", "single_word": false, "lstrip": true, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "sp_model_kwargs": {}, "tokenizer_class": "XLMRobertaTokenizer"}