gonzalez-agirre commited on
Commit
33654b2
1 Parent(s): 73725ac

Upload model.

Browse files
README.md CHANGED
@@ -1,3 +1,97 @@
1
  ---
2
- license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+
3
+ language:
4
+
5
+ - ca
6
+
7
+ tags:
8
+
9
+ - "catalan"
10
+
11
+ - "text classification"
12
+
13
+ - "tecla"
14
+
15
+ - "CaText"
16
+
17
+ - "Catalan Textual Corpus"
18
+
19
+ datasets:
20
+
21
+ - "projecte-aina/tecla"
22
+
23
+ metrics:
24
+ - accuracy
25
+
26
+ model-index:
27
+ - name: roberta-base-ca-v2-cased-tc
28
+ results:
29
+ - task:
30
+ type: text-classification
31
+ dataset:
32
+ name: TeCla
33
+ type: projecte-aina/tecla
34
+ metrics:
35
+ - name: Accuracy
36
+ type: accuracy
37
+ value: 0.7426
38
+
39
+ widget:
40
+
41
+ - text: "Els Pets presenten el seu nou treball al Palau Sant Jordi."
42
+
43
+ - text: "Els barcelonins incrementen un 23% l’ús del cotxe des de l’inici de la pandèmia."
44
+
45
+ - text: "Retards a quatre línies de Rodalies per una avaria entre Sants i plaça de Catalunya."
46
+
47
+ - text: "Majors de 60 anys i sanitaris començaran a rebre la tercera dosi de la vacuna covid els propers dies."
48
+
49
+ - text: "Els cinemes Verdi estrenen Verdi Classics, un nou canal de televisió."
50
+
51
  ---
52
+
53
+ # Catalan BERTa-v2 (roberta-base-ca-v2) finetuned for Text Classification.
54
+
55
+ The **roberta-base-ca-v2-cased-tc** is a Text Classification (TC) model for the Catalan language fine-tuned from the [roberta-base-ca-v2](https://huggingface.co/projecte-aina/roberta-base-ca-v2) model, a [RoBERTa](https://arxiv.org/abs/1907.11692) base model pre-trained on a medium-size corpus collected from publicly available corpora and crawlers (check the roberta-base-ca-v2 model card for more details).
56
+
57
+ ## Datasets
58
+ We used the TC dataset in Catalan called [TeCla](https://huggingface.co/datasets/projecte-aina/tecla) for training and evaluation.
59
+
60
+ ## Evaluation and results
61
+ We evaluated the _roberta-base-ca-v2-cased-tc_ on the TeCla test set against standard multilingual and monolingual baselines:
62
+
63
+ | Model | TeCla (Accuracy) |
64
+ | ------------|:-------------|
65
+ | roberta-base-ca-v2-cased-tc | **74.26** |
66
+ | roberta-base-ca-cased-tc | 73.65 |
67
+ | mBERT | 69.90 |
68
+ | XLM-RoBERTa | 70.14 |
69
+
70
+ For more details, check the fine-tuning and evaluation scripts in the official [GitHub repository](https://github.com/projecte-aina/club).
71
+
72
+ ## Citing
73
+ If you use any of these resources (datasets or models) in your work, please cite our latest paper:
74
+ ```bibtex
75
+ @inproceedings{armengol-estape-etal-2021-multilingual,
76
+ title = "Are Multilingual Models the Best Choice for Moderately Under-resourced Languages? {A} Comprehensive Assessment for {C}atalan",
77
+ author = "Armengol-Estap{\'e}, Jordi and
78
+ Carrino, Casimiro Pio and
79
+ Rodriguez-Penagos, Carlos and
80
+ de Gibert Bonet, Ona and
81
+ Armentano-Oller, Carme and
82
+ Gonzalez-Agirre, Aitor and
83
+ Melero, Maite and
84
+ Villegas, Marta",
85
+ booktitle = "Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021",
86
+ month = aug,
87
+ year = "2021",
88
+ address = "Online",
89
+ publisher = "Association for Computational Linguistics",
90
+ url = "https://aclanthology.org/2021.findings-acl.437",
91
+ doi = "10.18653/v1/2021.findings-acl.437",
92
+ pages = "4933--4946",
93
+ }
94
+ ```
95
+
96
+ ### Funding
97
+ This work was funded by the [Catalan Government](https://politiquesdigitals.gencat.cat/en/inici/index.html) within the framework of the [AINA project.](https://politiquesdigitals.gencat.cat/ca/economia/catalonia-ai/aina).
config.json ADDED
@@ -0,0 +1,70 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "projecte-aina/roberta-base-ca-v2-cased-tc",
3
+ "architectures": [
4
+ "RobertaForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": 0,
8
+ "eos_token_id": 2,
9
+ "finetuning_task": "tecla",
10
+ "gradient_checkpointing": false,
11
+ "hidden_act": "gelu",
12
+ "hidden_dropout_prob": 0.1,
13
+ "hidden_size": 768,
14
+ "id2label": {
15
+ "0": "Medi ambient",
16
+ "1": "Societat",
17
+ "2": "Policial",
18
+ "3": "Judicial",
19
+ "4": "Empresa",
20
+ "5": "Partits",
21
+ "6": "Política",
22
+ "7": "Successos",
23
+ "8": "Salut",
24
+ "9": "Infraestructures",
25
+ "10": "Parlament",
26
+ "11": "Música",
27
+ "12": "Govern",
28
+ "13": "Unió Europea",
29
+ "14": "Economia",
30
+ "15": "Mobilitat",
31
+ "16": "Treball",
32
+ "17": "Cultura",
33
+ "18": "Educació"
34
+ },
35
+ "initializer_range": 0.02,
36
+ "intermediate_size": 3072,
37
+ "label2id": {
38
+ "Medi ambient": 0,
39
+ "Societat": 1,
40
+ "Policial": 2,
41
+ "Judicial": 3,
42
+ "Empresa": 4,
43
+ "Partits": 5,
44
+ "Política": 6,
45
+ "Successos": 7,
46
+ "Salut": 8,
47
+ "Infraestructures": 9,
48
+ "Parlament": 10,
49
+ "Música": 11,
50
+ "Govern": 12,
51
+ "Unió Europea": 13,
52
+ "Economia": 14,
53
+ "Mobilitat": 15,
54
+ "Treball": 16,
55
+ "Cultura": 17,
56
+ "Educació": 18
57
+ },
58
+ "layer_norm_eps": 1e-05,
59
+ "max_position_embeddings": 514,
60
+ "model_type": "roberta",
61
+ "num_attention_heads": 12,
62
+ "num_hidden_layers": 12,
63
+ "pad_token_id": 1,
64
+ "position_embedding_type": "absolute",
65
+ "problem_type": "single_label_classification",
66
+ "transformers_version": "4.6.1",
67
+ "type_vocab_size": 1,
68
+ "use_cache": true,
69
+ "vocab_size": 50262
70
+ }
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b5da19d73d770844b657a2e4489d742af5e843a480c8b5fc46cbb61d1a6698e2
3
+ size 498717165
special_tokens_map.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"bos_token": {"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "eos_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "unk_token": {"content": "<unk>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "sep_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "pad_token": {"content": "<pad>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "cls_token": {"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "mask_token": {"content": "<mask>", "single_word": false, "lstrip": true, "rstrip": false, "normalized": true}}
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"unk_token": {"content": "<unk>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "bos_token": {"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "eos_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "add_prefix_space": true, "errors": "replace", "sep_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "cls_token": {"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "pad_token": {"content": "<pad>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "mask_token": {"content": "<mask>", "single_word": false, "lstrip": true, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "max_len": 512, "special_tokens_map_file": null, "name_or_path": "/gpfs/projects/bsc88/BERTs/models/roberta_base_ca_jsc/transformed_lr0.0005"}
vocab.json ADDED
The diff for this file is too large to render. See raw diff