Hugging Face's logo

cardiffnlp
/
roberta-base-offensive

Text Classification
Transformers
PyTorch
roberta
Eval Results
Inference Endpoints
Model card Files Community
3
roberta-base-offensive / README.md
autoevaluator's picture
autoevaluator HF staff
Add evaluation results on the offensive config and train split of tweet_eval
d54766d
preview code
|
raw
history blame
7.22 kB
metadata
datasets:
  - tweet_eval
metrics:
  - f1
  - accuracy
pipeline_tag: text-classification
widget:
  - text: >-
      Get the all-analog Classic Vinyl Edition of "Takin Off" Album from
      {@herbiehancock@} via {@bluenoterecords@} link below {{URL}}
    example_title: topic_classification 1
  - text: Yes, including Medicare and social security saving👍
    example_title: sentiment 1
  - text: All two of them taste like ass.
    example_title: offensive 1
  - text: If you wanna look like a badass, have drama on social media
    example_title: irony 1
  - text: Whoever just unfollowed me you a bitch
    example_title: hate 1
  - text: >-
      I love swimming for the same reason I love meditating...the feeling of
      weightlessness.
    example_title: emotion 1
  - text: Beautiful sunset last night from the pontoon @TupperLakeNY
    example_title: emoji 1
model-index:
  - name: cardiffnlp/roberta-base-offensive
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: tweet_eval
          type: offensive
          split: test
        metrics:
          - type: micro_f1_tweet_eval/offensive
            value: 0.8441860465116279
            name: Micro F1 (tweet_eval/offensive)
          - type: micro_f1_tweet_eval/offensive
            value: 0.8038468085106383
            name: Macro F1 (tweet_eval/offensive)
          - type: accuracy_tweet_eval/offensive
            value: 0.8441860465116279
            name: Accuracy (tweet_eval/offensive)
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: tweet_eval
          type: tweet_eval
          config: offensive
          split: train
        metrics:
          - type: accuracy
            value: 0.8775595837529372
            name: Accuracy
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZTQ3ZjIwZDAyZDc5MmU0ZWE3Mjg3MGZkMzJjYTA4ODYxMmI1NmUyNWUyMWQwYjhhOThiMjVlYzcwMTIyYWE3NiIsInZlcnNpb24iOjF9.FGKrLdRO1Iljnac-g6wty0HrcE5vwfHpWzRtzm-lPKsInyrGbtFC6mh6fpWHquoKZN_XVD-3Ju1ivROv3PsYDA
          - type: f1
            value: 0.8617195443801995
            name: F1 Macro
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYmE2MTJkNjUzOTczN2NkNzFmZmFkMmVlNzNkMTQxNThmYjY1NDJmYjI2MjJhZjc2M2I1OTJlYjg3ODQ5NTAwYiIsInZlcnNpb24iOjF9.JyPCbdFBiSnKAHO_fpGPKolFfS-QxCmgGILFTtRsO0yr53SFWZzvuWU3LF4eG_EkCskOCkhzJHe9ydFScf1cCg
          - type: f1
            value: 0.8775595837529372
            name: F1 Micro
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOWJjZDdiNGQxYTEzOWM1ZWU2NzI5MDkyNmQ2NjNhYWMzZmUyMmU4YWYyMjQ3ZjlkNDFhNGFiZmM0ZjEyMmE0MSIsInZlcnNpb24iOjF9.h4RE_k9PKIV2aoJxt9K_hStetS0jvvnZuumo6EWqZek1jrVdNCw8hecEfpDxCMuV1nJG_Nb1Qb2CPHaehoiaAA
          - type: f1
            value: 0.8775635113600219
            name: F1 Weighted
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZjczZjlhOTlmOTYyOGZkODFhMzBkNThjNmQ3ZTVhZTY0ZTQ2YWY3YTY4ZDU1M2E5ZmEyMmFmODI2NjJjMTc5YiIsInZlcnNpb24iOjF9.LNoaYMgzp63FR4pgt49Bi-6Fwb7ocicdGesMntzBV9Y_eNl7f4Jx-Jl1V8jjB-Mas_Fj1BHqYgmVnsokZEnDCA
          - type: precision
            value: 0.8616963464593261
            name: Precision Macro
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOWQ2MmI4YTI2NThlMDExODhkYzVhOGY4NmFlNjAxMzNjOTQ2NzNjYTBhYmM2NGRkYzIyYTEwYmQ2MDhhYzc5MyIsInZlcnNpb24iOjF9.dcwR0Y2MUzNt_-YSNFyLzxsVzCAglflGeLEm1EhQ2xU9cOpxKmGOADEETRVN-s8Qo-rfR0UTLBf8s1m_AJ01Bg
          - type: precision
            value: 0.8775595837529372
            name: Precision Micro
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYjdlN2VhNjllYjNhMDM1NmQwZGVkMTZiZjY1ZjgxNDZlMDRlZjY2NGE2NzkwOTIwMzBlODQ1NTIwOTUzODVhMiIsInZlcnNpb24iOjF9.bMukPZRCgLsH5bRqkUys1DjubnLFh39mj0JEmWkGNPKNqgRq11IDsHpMICK2l8_kW25_wpiThELRXlYWI8L6CA
          - type: precision
            value: 0.8775674524222297
            name: Precision Weighted
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOWU2NDE3NzkxYjFlNDM3ZmI4MzQwYTVjOWVkY2Q3MTIwZGVlZTUzYzBkNGFmMjU4ODVlZTQwYTdlYzBlNDRjNSIsInZlcnNpb24iOjF9.PLj9bhs5wyqcANvgiYVbf8Gnpkn7H1IWg7lUjXez60QxfOcN0LdXbGttxu_y13Q41mbF4RW9MkC_OlVgxgiOBw
          - type: recall
            value: 0.8617427589865883
            name: Recall Macro
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYzllZDBjMGY2ZTYyYTA5ODc4NjU1NWZkYTM4MWZlMDFkNjJhYTg3MTYwODYyZDYwYzc5MDliMTAzM2Q2NTk4YyIsInZlcnNpb24iOjF9.PUlMOsCQrowlUu1GGR9T2Hd50cOLsQHwu1FuwiLvWB25fLJYjFGTIai0UdBmtlTSKmviye_QzXrX1H_dJUAkBA
          - type: recall
            value: 0.8775595837529372
            name: Recall Micro
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZjhkMzYxMWNmZjYxYTE2N2JjMjRhOWQ4YjZhMzI3NWU2YWI2ODI2MGViZmE0M2NkYjdmYmRmNTBkMjkwOTVlNiIsInZlcnNpb24iOjF9.PT7NY-polKG346y1T7fq1vC_wtzI_niOFeIuCZqXbexwnmtPKQYZGW8ag3690u4D_8wP9KQlJuPimiiO5OzRBg
          - type: recall
            value: 0.8775595837529372
            name: Recall Weighted
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYmY5M2RkZmU2ZDQyNzIzYjA4MGY3MTZmMGViYTU3OWI4ODFlN2VhOWVhYWEwN2VkOWM3YTQ0ODU3NDk5MzNkMSIsInZlcnNpb24iOjF9.U1k9ishrbEKkcceXP-FgodUG-GbE-g1B1tK-hCpZNpCYKicZrxI7Ft5fNZ9jGjO8_eRZNpL8o1DYmON2-kjFBw
          - type: loss
            value: 0.31321173906326294
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNWMwYzM0MTg5YWEzYjBmMTg2NjgwMDc5NDY5NmQ4NWU1MjNjMTE4NzNmMDZmNWQzZGNlZDc3NGZjNzQzZTVjNiIsInZlcnNpb24iOjF9.YGXjIov_YlgdewGVUVySHZwVd874bUxvAkHcNXYf3j_at4DB14V1KLXmts0xXyHz0iTqJPjS6frr0aTHcixvBA

cardiffnlp/roberta-base-offensive

This model is a fine-tuned version of roberta-base on the tweet_eval (offensive) via tweetnlp. Training split is train and parameters have been tuned on the validation split validation.

Following metrics are achieved on the test split test (link).

  • F1 (micro): 0.8441860465116279
  • F1 (macro): 0.8038468085106383
  • Accuracy: 0.8441860465116279

Usage

Install tweetnlp via pip.

pip install tweetnlp

Load the model in python.

import tweetnlp
model = tweetnlp.Classifier("cardiffnlp/roberta-base-offensive", max_length=128)
model.predict('Get the all-analog Classic Vinyl Edition of "Takin Off" Album from {@herbiehancock@} via {@bluenoterecords@} link below {{URL}}')

Reference 

@inproceedings{camacho-collados-etal-2022-tweetnlp,
    title={{T}weet{NLP}: {C}utting-{E}dge {N}atural {L}anguage {P}rocessing for {S}ocial {M}edia},
    author={Camacho-Collados, Jose and Rezaee, Kiamehr and Riahi, Talayeh and Ushio, Asahi and Loureiro, Daniel and Antypas, Dimosthenis and Boisson, Joanne and Espinosa-Anke, Luis and Liu, Fangyu and Mart{'\i}nez-C{'a}mara, Eugenio and others},
    author = "Ushio, Asahi  and
      Camacho-Collados, Jose",
    booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
    month = nov,
    year = "2022",
    address = "Abu Dhabi, U.A.E.",
    publisher = "Association for Computational Linguistics",
}