metadata

datasets:
  - tweet_eval
metrics:
  - f1
  - accuracy
pipeline_tag: text-classification
widget:
  - text: >-
      Get the all-analog Classic Vinyl Edition of "Takin Off" Album from
      {@herbiehancock@} via {@bluenoterecords@} link below {{URL}}
    example_title: topic_classification 1
  - text: Yes, including Medicare and social security saving👍
    example_title: sentiment 1
  - text: All two of them taste like ass.
    example_title: offensive 1
  - text: If you wanna look like a badass, have drama on social media
    example_title: irony 1
  - text: Whoever just unfollowed me you a bitch
    example_title: hate 1
  - text: >-
      I love swimming for the same reason I love meditating...the feeling of
      weightlessness.
    example_title: emotion 1
  - text: Beautiful sunset last night from the pontoon @TupperLakeNY
    example_title: emoji 1
model-index:
  - name: cardiffnlp/twitter-roberta-base-2021-124m-offensive
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: tweet_eval
          type: offensive
          split: test
        metrics:
          - type: micro_f1_tweet_eval/offensive
            value: 0.858139534883721
            name: Micro F1 (tweet_eval/offensive)
          - type: micro_f1_tweet_eval/offensive
            value: 0.8232706055154664
            name: Macro F1 (tweet_eval/offensive)
          - type: accuracy_tweet_eval/offensive
            value: 0.858139534883721
            name: Accuracy (tweet_eval/offensive)
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: tweet_eval
          type: tweet_eval
          config: offensive
          split: train
        metrics:
          - type: accuracy
            value: 0.8682443773078214
            name: Accuracy
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMGJkNTljNjU4ZmFmYjVlN2M5YjMxOTEyMzhhYzA0OWYzM2U1MWVlOGQ1ZTJmNGY3YTNjODNmYzgxZjA3NGE4NiIsInZlcnNpb24iOjF9.NY_hmncAf9F010pQucmbKhIWhVWfyGpcUNyavHA53tYC-sGQOzQde8sifPOGTVFq2u7n7YWEYM_6OUHY8Z2RAQ
          - type: f1
            value: 0.8532408201027345
            name: F1 Macro
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYzdlNjFiOWMzZTdkYmFmOWQ1ODU2YWU5OTg3YzQwZjY1ODdhYjE3ZDc0ODUxZDZiM2FmZDY4NzA1MzY3MGUxZSIsInZlcnNpb24iOjF9.8sqfFWNsda-hnnYSvHJQCJczkarfPftYaCKpYxIC3RFAc3XRCdpVvEBqOzhsJ3m2kgCVVdHXrg8WbSEDh9mEBA
          - type: f1
            value: 0.8682443773078214
            name: F1 Micro
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMmExOWFhODliM2IyMDU0Y2U1ZTZiOTM2MWM5MWExNGRjYjMwMmE0MmEyNDRjOTgxNDEwN2Q1M2I5Yjg4OWZhMyIsInZlcnNpb24iOjF9.KKiMpInK2uYzTZT7ECGwpF8uR9rEfi3NO-NySGhs42IVGDVekXlg6LB7JdMR7OVN5Hpj6KTAtX3B9Ku_0sMOCg
          - type: f1
            value: 0.8691264762091178
            name: F1 Weighted
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYjdlYWQ4N2NkODJmY2ZkZTNiZjgzOTIwMTMyYzg1MjgwOTc3Y2ZmMGE3NTQ1NGM4ODhhMTc5YTQzODVhMGRlNyIsInZlcnNpb24iOjF9.-CIWG5eSbCdonG6iIIKuzLFEfV2-zx7eNjNtagaKf0zccn0hgLN5pLgwwZQnieUb8cjLE5EjDtV6HUJbh_qBBw
          - type: precision
            value: 0.8489031737065875
            name: Precision Macro
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTE1YmFiMDFhNjQyM2Q4YTYwZWU0NGU5MzAzYmU3NDBkODQzOGE1ODkwYWJkN2E5MzJmMDQxNTU5OTEwOWIzMyIsInZlcnNpb24iOjF9.8022ApT081k_TZGFFL1zdpZBQE6BXNDMLtkcLQBp1nkiyECh3AhHMw1y-YPO5zmMVwMwhjY7L0CjJZIye6ZcCw
          - type: precision
            value: 0.8682443773078214
            name: Precision Micro
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTEyZDFlNWIyNzIzYWRkMzk1OTEzNGU3NDNiYWFlNTQyODJjNTc4YTMyYmEyZDdhZGJjY2E0MWVkY2Q1MWRmOCIsInZlcnNpb24iOjF9.GSHQgLs8ybqBHQHFinysfDnQmg1IZfbAsZP5fNDLIGi9F7uDhwyp2bovVRtcc7P5zsHKt0SvC1wOpZV4qL6RBg
          - type: precision
            value: 0.8706606793751257
            name: Precision Weighted
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZmQyOTc3Y2IzNzNhYjczZWMyNGRjM2VlMTYyYzJlZWYyMWFlOWJlNjg4NjYyM2RkMGU4MTZhNjRiMGNmMDhjMCIsInZlcnNpb24iOjF9.ETcK2QE85cr6gUzQjuZAhZSitf7Q71bko0YgxejPRblaDgoFBIIDgvnwr2P0570Z_aY2phqLJVY4H0uWaQw3DA
          - type: recall
            value: 0.8583773352879741
            name: Recall Macro
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMmJlYjVmYzg4OGEzNmFlYTVmNTAyMzEwNGQwMWYxNTNiOWY5ODRmZmUyYzZiODVlY2M2MGI5MDQ5NTNkNTA5MiIsInZlcnNpb24iOjF9.sSwm_LY3NFoisGUZntYDtRCZ6Qmi6cl1018LUgugDtIQBT2sOk_QyotICk4wuzrG6Z6K2MWYVrT7vP-vTEpfBA
          - type: recall
            value: 0.8682443773078214
            name: Recall Micro
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMTNlNjBmMjc4MmI4YzNiOWQ2ZWUxNTJkN2Q0MDlmYWZjOGQ0NDI0Zjk0ZmNlYzM5Nzk3YzE1OWM5MGMxY2NlOSIsInZlcnNpb24iOjF9.Yb-IGL2iR1vuEbpN3dFDTnsh1w3yTsL5oP4WGnqmj01knATG45MJYtPyC_eKIGR20HaW9iR_aVCW8GE2TpZUAg
          - type: recall
            value: 0.8682443773078214
            name: Recall Weighted
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGM2YmMxZGM0MmZiMzc1MmM3MTdjN2Q0ZGY0OTUzM2IzOWJhYjViNTI0YjU4NmQ3YTQzOTllODUzNjZiNzA2OCIsInZlcnNpb24iOjF9.tFJUmy4JxCO7R6tCalkA1T0251hrCB6GUUUt7WmeCDRRFfkZe8GLXqhnW1gBvgv0g5cGeNHw6uYwOh4hpa6DCA
          - type: loss
            value: 0.33106717467308044
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYmIwODg3MzA0NWY1YTAwZjk5ZThhNzM2MzcxMTI1ZjdiOWM1ZTNlODMwZGFmZDMyYjY0NjA0NjRiNzhiNjEwZSIsInZlcnNpb24iOjF9.Habwn92oQOuGBmw_K5hZamuJItEz-XeKhL4Q3PJJyL_9L5xbD2FKGXaYHcjalJFqZRqwQbsQ8JqEjErxqQPKDg

cardiffnlp/twitter-roberta-base-2021-124m-offensive

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2021-124m on the tweet_eval (offensive) via tweetnlp. Training split is train and parameters have been tuned on the validation split validation.

Following metrics are achieved on the test split test (link).

F1 (micro): 0.858139534883721
F1 (macro): 0.8232706055154664
Accuracy: 0.858139534883721

Usage

Install tweetnlp via pip.

pip install tweetnlp

Load the model in python.

import tweetnlp
model = tweetnlp.Classifier("cardiffnlp/twitter-roberta-base-2021-124m-offensive", max_length=128)
model.predict('Get the all-analog Classic Vinyl Edition of "Takin Off" Album from {@herbiehancock@} via {@bluenoterecords@} link below {{URL}}')

Reference

@inproceedings{camacho-collados-etal-2022-tweetnlp,
    title={{T}weet{NLP}: {C}utting-{E}dge {N}atural {L}anguage {P}rocessing for {S}ocial {M}edia},
    author={Camacho-Collados, Jose and Rezaee, Kiamehr and Riahi, Talayeh and Ushio, Asahi and Loureiro, Daniel and Antypas, Dimosthenis and Boisson, Joanne and Espinosa-Anke, Luis and Liu, Fangyu and Mart{'\i}nez-C{'a}mara, Eugenio and others},
    author = "Ushio, Asahi  and
      Camacho-Collados, Jose",
    booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
    month = nov,
    year = "2022",
    address = "Abu Dhabi, U.A.E.",
    publisher = "Association for Computational Linguistics",
}