First version

README.md

@@ -0,0 +1,55 @@
+# Sentiment Analysis in Spanish
+## robertuito-sentiment-analysis
+
+Repository: [https://github.com/pysentimiento/pysentimiento/](https://github.com/finiteautomata/pysentimiento/)
+
+
+
+Model trained with SemEval 2019 Task 5: HatEval (SubTask B) corpus for Hate Speech detection in Spanish. Base model is [RoBERTuito](https://github.com/pysentimiento/robertuito), a RoBERTa model trained in Spanish tweets.
+
+It is a multi-classifier model, with the following classes:
+
+- **HS**: is it hate speech?
+- **TR**: is it targeted to a specific individual?
+- **AG**: is it aggressive?
+
+## Results
+
+Results for the four tasks evaluated in `pysentimiento`. Results are expressed as Macro F1 scores
+
+
+| model         | emotion       | hate_speech   | irony         | sentiment     |
+|:--------------|:--------------|:--------------|:--------------|:--------------|
+| robertuito    | 0.560 ± 0.010 | 0.759 ± 0.007 | 0.739 ± 0.005 | 0.705 ± 0.003 |
+| roberta       | 0.527 ± 0.015 | 0.741 ± 0.012 | 0.721 ± 0.008 | 0.670 ± 0.006 |
+| bertin        | 0.524 ± 0.007 | 0.738 ± 0.007 | 0.713 ± 0.012 | 0.666 ± 0.005 |
+| beto_uncased  | 0.532 ± 0.012 | 0.727 ± 0.016 | 0.701 ± 0.007 | 0.651 ± 0.006 |
+| beto_cased    | 0.516 ± 0.012 | 0.724 ± 0.012 | 0.705 ± 0.009 | 0.662 ± 0.005 |
+| mbert_uncased | 0.493 ± 0.010 | 0.718 ± 0.011 | 0.681 ± 0.010 | 0.617 ± 0.003 |
+| biGRU         | 0.264 ± 0.007 | 0.592 ± 0.018 | 0.631 ± 0.011 | 0.585 ± 0.011 |
+
+
+Note that for Hate Speech, these are the results for Semeval 2019, Task 5 Subtask B (HS+TR+AG detection)
+
+## Citation
+
+If you use this model in your research, please cite pysentimiento and RoBERTuito papers:
+
+```
+@misc{perez2021pysentimiento,
+      title={pysentimiento: A Python Toolkit for Sentiment Analysis and SocialNLP tasks},
+      author={Juan Manuel Pérez and Juan Carlos Giudici and Franco Luque},
+      year={2021},
+      eprint={2106.09462},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL}
+}
+@misc{perez2021robertuito,
+      title={RoBERTuito: a pre-trained language model for social media text in Spanish},
+      author={Juan Manuel Pérez and Damián A. Furman and Laura Alonso Alemany and Franco Luque},
+      year={2021},
+      eprint={2111.09453},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL}
+}
+```

config.json

@@ -0,0 +1,39 @@
+{
+  "_name_or_path": "pysentimiento/robertuito-base-uncased",
+  "architectures": [
+    "RobertaForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "hateful",
+    "1": "targeted",
+    "2": "aggressive"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "aggressive": 2,
+    "hateful": 0,
+    "targeted": 1
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 130,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "problem_type": "multi_label_classification",
+  "torch_dtype": "float32",
+  "transformers_version": "4.11.3",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 30002
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:169eabf45b7e489e38711439c9c217e2ad1fcf955fb14cad32816e835ae4a688
+size 435246509

special_tokens_map.json

@@ -0,0 +1 @@
+{"bos_token": "<s>", "eos_token": "</s>", "unk_token": "<unk>", "sep_token": "</s>", "pad_token": "<pad>", "cls_token": "<s>", "mask_token": "<mask>"}

test_results.json

@@ -0,0 +1,21 @@
+{
+    "test_loss": 0.4132111370563507,
+    "test_hs_f1": 0.7729658792650919,
+    "test_hs_precision": 0.6817129629629629,
+    "test_hs_recall": 0.8924242424242425,
+    "test_tr_f1": 0.8071428571428573,
+    "test_tr_precision": 0.8129496402877698,
+    "test_tr_recall": 0.8014184397163121,
+    "test_ag_f1": 0.7058823529411765,
+    "test_ag_precision": 0.6111111111111112,
+    "test_ag_recall": 0.8354430379746836,
+    "test_macro_hs_f1_score": 0.7832609825919732,
+    "test_emr_no_gating": 0.67,
+    "test_emr": 0.67125,
+    "test_macro_f1": 0.7619970440864563,
+    "test_macro_precision": 0.7019245624542236,
+    "test_macro_recall": 0.8430952429771423,
+    "test_runtime": 3.1919,
+    "test_samples_per_second": 501.263,
+    "test_steps_per_second": 31.329
+}

tokenizer_config.json

@@ -0,0 +1 @@
+{"bos_token": "<s>", "eos_token": "</s>", "sep_token": "</s>", "cls_token": "<s>", "unk_token": "<unk>", "pad_token": "<pad>", "mask_token": "<mask>", "special_tokens_map_file": "models/twerto-base-uncased/special_tokens_map.json", "name_or_path": "pysentimiento/robertuito-base-uncased", "tokenizer_class": "PreTrainedTokenizerFast"}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d7c98dc95c82c2b1466b8a6920bab4012a36d689e7a2c2b18fd1bb425546b5a
+size 2799