Edit model card

xtremedistil-l6-h384-go-emotion

This model is a fine-tuned version of microsoft/xtremedistil-l6-h384-uncased on the go_emotions dataset.

See notebook for how the model was trained and converted to ONNX format Training Notebook

This model is deployed to aiserv.cloud for live demo of the model.

See https://github.com/jobergum/browser-ml-inference for how to reproduce.

Training hyperparameters

  • batch size 128
  • learning_rate=3e-05
  • epocs 4
    Num examples = 211225
    Num Epochs = 4
    Instantaneous batch size per device = 128
    Total train batch size (w. parallel, distributed & accumulation) = 128
    Gradient Accumulation steps = 1
    Total optimization steps = 6604
    [6604/6604 53:23, Epoch 4/4]
    Step    Training Loss
    500    0.263200
    1000    0.156900
    1500    0.152500
    2000    0.145400
    2500    0.140500
    3000    0.135900
    3500    0.132800
    4000    0.129400
    4500    0.127200
    5000    0.125700
    5500    0.124400
    6000    0.124100
    6500    0.123400
    
Downloads last month
797
Hosted inference API
Text Classification
Examples
Examples
This model can be loaded on the Inference API on-demand.

Dataset used to train bergum/xtremedistil-l6-h384-go-emotion

Evaluation results