metadata

language: en
tags:
  - text-classification
  - pytorch
  - tensorflow
datasets:
  - go_emotions
license: mit

distilbert-base-uncased-go-emotions-student

Model Description

This model is distilled from the zero-shot classification pipeline on the unlabeled GoEmotions dataset using this script. It is the result of the demo notebook here, where more details about the model can be found.

Teacher model: roberta-large-mnli

Intended Usage

The model can be used like any other model trained on GoEmotions, but will likely not perform as well as a model trained with full supervision. It is primarily intended as a demo of how an expensive NLI-based zero-shot model can be distilled to a more efficient student. Note that although the GoEmotions dataset allow multiple labels per instance, the teacher used single-label classification to create psuedo-labels.