![autoevaluator's picture](https://cdn-avatars.huggingface.co/v1/production/uploads/1654180084862-6297236d64501abb8dfde40d.png)
autoevaluator
HF staff
Add evaluation results on the offensive config and train split of tweet_eval
598924d
datasets: | |
- tweet_eval | |
metrics: | |
- f1 | |
- accuracy | |
pipeline_tag: text-classification | |
widget: | |
- text: Get the all-analog Classic Vinyl Edition of "Takin Off" Album from {@herbiehancock@} | |
via {@bluenoterecords@} link below {{URL}} | |
example_title: topic_classification 1 | |
- text: Yes, including Medicare and social security saving👍 | |
example_title: sentiment 1 | |
- text: All two of them taste like ass. | |
example_title: offensive 1 | |
- text: If you wanna look like a badass, have drama on social media | |
example_title: irony 1 | |
- text: Whoever just unfollowed me you a bitch | |
example_title: hate 1 | |
- text: I love swimming for the same reason I love meditating...the feeling of weightlessness. | |
example_title: emotion 1 | |
- text: Beautiful sunset last night from the pontoon @TupperLakeNY | |
example_title: emoji 1 | |
model-index: | |
- name: cardiffnlp/twitter-roberta-base-2021-124m-offensive | |
results: | |
- task: | |
type: text-classification | |
name: Text Classification | |
dataset: | |
name: tweet_eval | |
type: offensive | |
split: test | |
metrics: | |
- type: micro_f1_tweet_eval/offensive | |
value: 0.858139534883721 | |
name: Micro F1 (tweet_eval/offensive) | |
- type: micro_f1_tweet_eval/offensive | |
value: 0.8232706055154664 | |
name: Macro F1 (tweet_eval/offensive) | |
- type: accuracy_tweet_eval/offensive | |
value: 0.858139534883721 | |
name: Accuracy (tweet_eval/offensive) | |
- task: | |
type: text-classification | |
name: Text Classification | |
dataset: | |
name: tweet_eval | |
type: tweet_eval | |
config: offensive | |
split: train | |
metrics: | |
- type: accuracy | |
value: 0.8682443773078214 | |
name: Accuracy | |
verified: true | |
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMGJkNTljNjU4ZmFmYjVlN2M5YjMxOTEyMzhhYzA0OWYzM2U1MWVlOGQ1ZTJmNGY3YTNjODNmYzgxZjA3NGE4NiIsInZlcnNpb24iOjF9.NY_hmncAf9F010pQucmbKhIWhVWfyGpcUNyavHA53tYC-sGQOzQde8sifPOGTVFq2u7n7YWEYM_6OUHY8Z2RAQ | |
- type: f1 | |
value: 0.8532408201027345 | |
name: F1 Macro | |
verified: true | |
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYzdlNjFiOWMzZTdkYmFmOWQ1ODU2YWU5OTg3YzQwZjY1ODdhYjE3ZDc0ODUxZDZiM2FmZDY4NzA1MzY3MGUxZSIsInZlcnNpb24iOjF9.8sqfFWNsda-hnnYSvHJQCJczkarfPftYaCKpYxIC3RFAc3XRCdpVvEBqOzhsJ3m2kgCVVdHXrg8WbSEDh9mEBA | |
- type: f1 | |
value: 0.8682443773078214 | |
name: F1 Micro | |
verified: true | |
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMmExOWFhODliM2IyMDU0Y2U1ZTZiOTM2MWM5MWExNGRjYjMwMmE0MmEyNDRjOTgxNDEwN2Q1M2I5Yjg4OWZhMyIsInZlcnNpb24iOjF9.KKiMpInK2uYzTZT7ECGwpF8uR9rEfi3NO-NySGhs42IVGDVekXlg6LB7JdMR7OVN5Hpj6KTAtX3B9Ku_0sMOCg | |
- type: f1 | |
value: 0.8691264762091178 | |
name: F1 Weighted | |
verified: true | |
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYjdlYWQ4N2NkODJmY2ZkZTNiZjgzOTIwMTMyYzg1MjgwOTc3Y2ZmMGE3NTQ1NGM4ODhhMTc5YTQzODVhMGRlNyIsInZlcnNpb24iOjF9.-CIWG5eSbCdonG6iIIKuzLFEfV2-zx7eNjNtagaKf0zccn0hgLN5pLgwwZQnieUb8cjLE5EjDtV6HUJbh_qBBw | |
- type: precision | |
value: 0.8489031737065875 | |
name: Precision Macro | |
verified: true | |
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTE1YmFiMDFhNjQyM2Q4YTYwZWU0NGU5MzAzYmU3NDBkODQzOGE1ODkwYWJkN2E5MzJmMDQxNTU5OTEwOWIzMyIsInZlcnNpb24iOjF9.8022ApT081k_TZGFFL1zdpZBQE6BXNDMLtkcLQBp1nkiyECh3AhHMw1y-YPO5zmMVwMwhjY7L0CjJZIye6ZcCw | |
- type: precision | |
value: 0.8682443773078214 | |
name: Precision Micro | |
verified: true | |
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTEyZDFlNWIyNzIzYWRkMzk1OTEzNGU3NDNiYWFlNTQyODJjNTc4YTMyYmEyZDdhZGJjY2E0MWVkY2Q1MWRmOCIsInZlcnNpb24iOjF9.GSHQgLs8ybqBHQHFinysfDnQmg1IZfbAsZP5fNDLIGi9F7uDhwyp2bovVRtcc7P5zsHKt0SvC1wOpZV4qL6RBg | |
- type: precision | |
value: 0.8706606793751257 | |
name: Precision Weighted | |
verified: true | |
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZmQyOTc3Y2IzNzNhYjczZWMyNGRjM2VlMTYyYzJlZWYyMWFlOWJlNjg4NjYyM2RkMGU4MTZhNjRiMGNmMDhjMCIsInZlcnNpb24iOjF9.ETcK2QE85cr6gUzQjuZAhZSitf7Q71bko0YgxejPRblaDgoFBIIDgvnwr2P0570Z_aY2phqLJVY4H0uWaQw3DA | |
- type: recall | |
value: 0.8583773352879741 | |
name: Recall Macro | |
verified: true | |
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMmJlYjVmYzg4OGEzNmFlYTVmNTAyMzEwNGQwMWYxNTNiOWY5ODRmZmUyYzZiODVlY2M2MGI5MDQ5NTNkNTA5MiIsInZlcnNpb24iOjF9.sSwm_LY3NFoisGUZntYDtRCZ6Qmi6cl1018LUgugDtIQBT2sOk_QyotICk4wuzrG6Z6K2MWYVrT7vP-vTEpfBA | |
- type: recall | |
value: 0.8682443773078214 | |
name: Recall Micro | |
verified: true | |
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMTNlNjBmMjc4MmI4YzNiOWQ2ZWUxNTJkN2Q0MDlmYWZjOGQ0NDI0Zjk0ZmNlYzM5Nzk3YzE1OWM5MGMxY2NlOSIsInZlcnNpb24iOjF9.Yb-IGL2iR1vuEbpN3dFDTnsh1w3yTsL5oP4WGnqmj01knATG45MJYtPyC_eKIGR20HaW9iR_aVCW8GE2TpZUAg | |
- type: recall | |
value: 0.8682443773078214 | |
name: Recall Weighted | |
verified: true | |
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGM2YmMxZGM0MmZiMzc1MmM3MTdjN2Q0ZGY0OTUzM2IzOWJhYjViNTI0YjU4NmQ3YTQzOTllODUzNjZiNzA2OCIsInZlcnNpb24iOjF9.tFJUmy4JxCO7R6tCalkA1T0251hrCB6GUUUt7WmeCDRRFfkZe8GLXqhnW1gBvgv0g5cGeNHw6uYwOh4hpa6DCA | |
- type: loss | |
value: 0.33106717467308044 | |
name: loss | |
verified: true | |
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYmIwODg3MzA0NWY1YTAwZjk5ZThhNzM2MzcxMTI1ZjdiOWM1ZTNlODMwZGFmZDMyYjY0NjA0NjRiNzhiNjEwZSIsInZlcnNpb24iOjF9.Habwn92oQOuGBmw_K5hZamuJItEz-XeKhL4Q3PJJyL_9L5xbD2FKGXaYHcjalJFqZRqwQbsQ8JqEjErxqQPKDg | |
# cardiffnlp/twitter-roberta-base-2021-124m-offensive | |
This model is a fine-tuned version of [cardiffnlp/twitter-roberta-base-2021-124m](https://huggingface.co/cardiffnlp/twitter-roberta-base-2021-124m) on the | |
[`tweet_eval (offensive)`](https://huggingface.co/datasets/tweet_eval) | |
via [`tweetnlp`](https://github.com/cardiffnlp/tweetnlp). | |
Training split is `train` and parameters have been tuned on the validation split `validation`. | |
Following metrics are achieved on the test split `test` ([link](https://huggingface.co/cardiffnlp/twitter-roberta-base-2021-124m-offensive/raw/main/metric.json)). | |
- F1 (micro): 0.858139534883721 | |
- F1 (macro): 0.8232706055154664 | |
- Accuracy: 0.858139534883721 | |
### Usage | |
Install tweetnlp via pip. | |
```shell | |
pip install tweetnlp | |
``` | |
Load the model in python. | |
```python | |
import tweetnlp | |
model = tweetnlp.Classifier("cardiffnlp/twitter-roberta-base-2021-124m-offensive", max_length=128) | |
model.predict('Get the all-analog Classic Vinyl Edition of "Takin Off" Album from {@herbiehancock@} via {@bluenoterecords@} link below {{URL}}') | |
``` | |
### Reference | |
``` | |
@inproceedings{camacho-collados-etal-2022-tweetnlp, | |
title={{T}weet{NLP}: {C}utting-{E}dge {N}atural {L}anguage {P}rocessing for {S}ocial {M}edia}, | |
author={Camacho-Collados, Jose and Rezaee, Kiamehr and Riahi, Talayeh and Ushio, Asahi and Loureiro, Daniel and Antypas, Dimosthenis and Boisson, Joanne and Espinosa-Anke, Luis and Liu, Fangyu and Mart{'\i}nez-C{'a}mara, Eugenio and others}, | |
author = "Ushio, Asahi and | |
Camacho-Collados, Jose", | |
booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations", | |
month = nov, | |
year = "2022", | |
address = "Abu Dhabi, U.A.E.", | |
publisher = "Association for Computational Linguistics", | |
} | |
``` | |