File size: 1,816 Bytes
853149d 90d2663 853149d 90d2663 853149d 90d2663 25022d1 853149d b990e47 d0a1aa1 90d2663 25022d1 e969c33 d0a1aa1 e969c33 853149d 90d2663 853149d e969c33 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
Welcome to **RoBERTArg**!
🤖 **Model description**:
This model was trained on ~40k heterogeneous manually annotated sentences (📚 Stab et al. 2018) of controversial topics (abortion etc.) to classify text into one of two labels: 🏷 **NON-ARGUMENT** (0) and **ARGUMENT** (1).
**Dataset**
Please note that the label distribution in the dataset is imbalanced:
* NON-ARGUMENTS:
* ARGUMENTS:
**Model training**
**RoBERTArg** was fine-tuned on a $RoBERTA_{base}$ pre-trained model using the HuggingFace trainer with the following hyperparameters. The hyperparameters were determined using a hyperparameter search on a 20% validation set.
```
training_args = TrainingArguments(
num_train_epochs=2,
learning_rate=2.3102e-06,
seed=8,
per_device_train_batch_size=64,
per_device_eval_batch_size=64,
)
```
**Evaluation**
The model was evaluated using 20% of the sentences (80-20 train-test split).
| Model | Acc | F1 | R arg | R non | P arg | P non |
|----|----|----|----|----|----|----|
| RoBERTArg | 0.8193 | 0.8021 | 0.8463 | 0.7986 | 0.7623 | 0.8719 |
Showing the **confusion matrix** using the 20% of the sentences as an evaluation set:
| | ARGUMENT | NON-ARGUMENT |
|----|----|----|
| ARGUMENT | 2213 | 558 |
| NON-ARGUMENT | 325 | 1790 |
**Intended Uses & Potential Limitations**
The model can be a practical starting point to the complex topic **Argument Mining**. It is a quite challenging task due to the different conceptions of an argument.
This model is a part of an open-source project providing several models to detect arguments in text.
👉🏾 Check out _chkla/argument-analyzer/_ for more details.
Enjoy and stay tuned! 🚀
📚Stab et al. (2018) https://public.ukp.informatik.tu-darmstadt.de/UKP_Webpage/publications/2018/2018_EMNLP_CS_Cross-topicArgumentMining.pdf |