metadata

license: mit
tags:
  - generated_from_trainer
  - nlu
  - intent-classification
metrics:
  - accuracy
  - f1
model-index:
  - name: mdeberta-v3-base_amazon-massive_intent
    results:
      - task:
          name: intent-classification
          type: intent-classification
        dataset:
          name: MASSIVE
          type: AmazonScience/massive
          split: test
        metrics:
          - name: F1
            type: f1
            value: 0.8136
datasets:
  - AmazonScience/massive
language:
  - en
pipeline_tag: text-classification

mdeberta-v3-base_amazon-massive_intent

This model is a fine-tuned version of microsoft/mdeberta-v3-base on the MASSIVE1.1 dataset. It achieves the following results on the evaluation set:

Loss: 1.1661
Accuracy: 0.8136
F1: 0.8136

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
3.6412	1.0	720	2.7536	0.3123	0.3123
2.8575	2.0	1440	1.8556	0.5303	0.5303
1.7284	3.0	2160	1.3758	0.6699	0.6699
1.3794	4.0	2880	1.1221	0.7236	0.7236
0.942	5.0	3600	0.9936	0.7609	0.7609
0.7672	6.0	4320	0.9411	0.7727	0.7727
0.602	7.0	5040	0.9196	0.7841	0.7841
0.4776	8.0	5760	0.9328	0.7895	0.7895
0.4347	9.0	6480	0.9602	0.7860	0.7860
0.2941	10.0	7200	0.9543	0.7949	0.7949
0.2783	11.0	7920	0.9979	0.8013	0.8013
0.2038	12.0	8640	0.9702	0.8062	0.8062
0.1827	13.0	9360	1.0121	0.8106	0.8106
0.1352	14.0	10080	1.0339	0.8136	0.8136
0.1115	15.0	10800	1.1091	0.8057	0.8057
0.0996	16.0	11520	1.1134	0.8151	0.8151
0.0837	17.0	12240	1.1288	0.8160	0.8160
0.0711	18.0	12960	1.1499	0.8155	0.8155
0.0594	19.0	13680	1.1622	0.8126	0.8126
0.0569	20.0	14400	1.1661	0.8136	0.8136

Framework versions

Transformers 4.24.0
Pytorch 1.12.1+cu113
Datasets 2.7.1
Tokenizers 0.13.2