Edit model card

Model trained from roberta-base on the go_emotions dataset for multi-label classification.

go_emotions is based on Reddit data and has 28 labels. It is a multi-label dataset where one or multiple labels may apply for any given input text, hence this model is a multi-label classification model with 28 'probability' float outputs for any given input text. Typically a threshold of 0.5 is applied to the probabilities for the prediction for each label.

The model was trained using AutoModelForSequenceClassification.from_pretrained with problem_type="multi_label_classification" for 3 epochs with a learning rate of 2e-5 and weight decay of 0.01.

Evaluation (of the 28 dim output via a threshold of 0.5 to binarize each) using the dataset test split gives:

  • Micro F1 0.585
  • ROC AUC 0.751
  • Accuracy 0.474

But the metrics would be more meaningful when measured per label given the multi-label nature.

Additionally some labels (E.g. gratitude) when considered independently perform very strongly with F1 around 0.9, whilst others (E.g. relief) perform very poorly. This is a challenging dataset. Labels such as relief do have much fewer examples in the training data (less than 100 out of the 40k+), but there is also some ambiguity and/or labelling errors visible in the training data of go_emotions that is suspected to constrain the performance.

Downloads last month
5
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train Rakshit122/roberta