First version

README.md

@@ -0,0 +1,51 @@
+# Irony detection in Spanish
+## robertuito-irony
+
+Repository: [https://github.com/pysentimiento/pysentimiento/](https://github.com/finiteautomata/pysentimiento/)
+
+
+
+Model trained with IRosVA 2019 dataset for irony detection. Base model is [RoBERTuito](https://github.com/pysentimiento/robertuito), a RoBERTa model trained in Spanish tweets.
+
+The positive class marks irony, the negative class marks not irony.
+
+## Results
+
+Results for the four tasks evaluated in `pysentimiento`. Results are expressed as Macro F1 scores
+
+
+| model         | emotion       | hate_speech   | irony         | sentiment     |
+|:--------------|:--------------|:--------------|:--------------|:--------------|
+| robertuito    | 0.560 ± 0.010 | 0.759 ± 0.007 | 0.739 ± 0.005 | 0.705 ± 0.003 |
+| roberta       | 0.527 ± 0.015 | 0.741 ± 0.012 | 0.721 ± 0.008 | 0.670 ± 0.006 |
+| bertin        | 0.524 ± 0.007 | 0.738 ± 0.007 | 0.713 ± 0.012 | 0.666 ± 0.005 |
+| beto_uncased  | 0.532 ± 0.012 | 0.727 ± 0.016 | 0.701 ± 0.007 | 0.651 ± 0.006 |
+| beto_cased    | 0.516 ± 0.012 | 0.724 ± 0.012 | 0.705 ± 0.009 | 0.662 ± 0.005 |
+| mbert_uncased | 0.493 ± 0.010 | 0.718 ± 0.011 | 0.681 ± 0.010 | 0.617 ± 0.003 |
+| biGRU         | 0.264 ± 0.007 | 0.592 ± 0.018 | 0.631 ± 0.011 | 0.585 ± 0.011 |
+
+
+Note that for Hate Speech, these are the results for Semeval 2019, Task 5 Subtask B (HS+TR+AG detection)
+
+## Citation
+
+If you use this model in your research, please cite pysentimiento and RoBERTuito papers:
+
+```
+@misc{perez2021pysentimiento,
+      title={pysentimiento: A Python Toolkit for Sentiment Analysis and SocialNLP tasks},
+      author={Juan Manuel Pérez and Juan Carlos Giudici and Franco Luque},
+      year={2021},
+      eprint={2106.09462},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL}
+}
+@misc{perez2021robertuito,
+      title={RoBERTuito: a pre-trained language model for social media text in Spanish},
+      author={Juan Manuel Pérez and Damián A. Furman and Laura Alonso Alemany and Franco Luque},
+      year={2021},
+      eprint={2111.09453},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL}
+}
+```

config.json

@@ -0,0 +1,37 @@
+{
+  "_name_or_path": "pysentimiento/robertuito-base-uncased",
+  "architectures": [
+    "RobertaForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "not ironic",
+    "1": "ironic"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "ironic": 1,
+    "not ironic": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 130,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "problem_type": "single_label_classification",
+  "torch_dtype": "float32",
+  "transformers_version": "4.11.3",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 30002
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb554c7fed80b7220134bad550f5eb0ff13485f269210e2f59bbdecb9de03c37
+size 435243437

special_tokens_map.json

@@ -0,0 +1 @@
+{"bos_token": "<s>", "eos_token": "</s>", "unk_token": "<unk>", "sep_token": "</s>", "pad_token": "<pad>", "cls_token": "<s>", "mask_token": "<mask>"}

test_results.json

@@ -0,0 +1,17 @@
+{
+    "test_loss": 1.247560739517212,
+    "test_not ironic_f1": 0.8370808678500986,
+    "test_not ironic_precision": 0.795352323838081,
+    "test_not ironic_recall": 0.8834304746044963,
+    "test_ironic_f1": 0.612206572769953,
+    "test_ironic_precision": 0.6995708154506438,
+    "test_ironic_recall": 0.5442404006677797,
+    "test_micro_f1": 0.7705555555555557,
+    "test_acc": 0.7705555555555555,
+    "test_macro_f1": 0.7246437072753906,
+    "test_macro_precision": 0.7474615573883057,
+    "test_macro_recall": 0.7138354778289795,
+    "test_runtime": 4.0387,
+    "test_samples_per_second": 445.685,
+    "test_steps_per_second": 27.979
+}

tokenizer_config.json

@@ -0,0 +1 @@
+{"bos_token": "<s>", "eos_token": "</s>", "sep_token": "</s>", "cls_token": "<s>", "unk_token": "<unk>", "pad_token": "<pad>", "mask_token": "<mask>", "special_tokens_map_file": "models/twerto-base-uncased/special_tokens_map.json", "name_or_path": "pysentimiento/robertuito-base-uncased", "tokenizer_class": "PreTrainedTokenizerFast"}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6acd820218ac5099b6d42c9140518f868886df394f2a21d30f220636ff08adc8
+size 2799