metadata

base_model: gokuls/model_v1_complete_training_wt_init_48_small_freeze_new
tags:
  - generated_from_trainer
datasets:
  - massive
metrics:
  - accuracy
model-index:
  - name: hbertv1-massive-logit_KD-small
    results:
      - task:
          name: Text Classification
          type: text-classification
        dataset:
          name: massive
          type: massive
          config: en-US
          split: validation
          args: en-US
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.8735858337432366

hbertv1-massive-logit_KD-small

This model is a fine-tuned version of gokuls/model_v1_complete_training_wt_init_48_small_freeze_new on the massive dataset. It achieves the following results on the evaluation set:

Loss: 0.4139
Accuracy: 0.8736

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 64
eval_batch_size: 64
seed: 33
distributed_type: multi-GPU
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
2.2301	1.0	180	0.8611	0.7565
0.8039	2.0	360	0.5989	0.8151
0.5542	3.0	540	0.5036	0.8396
0.4134	4.0	720	0.4535	0.8569
0.3187	5.0	900	0.4432	0.8569
0.251	6.0	1080	0.4280	0.8637
0.2201	7.0	1260	0.4311	0.8598
0.1879	8.0	1440	0.4443	0.8608
0.168	9.0	1620	0.4136	0.8677
0.153	10.0	1800	0.4286	0.8598
0.137	11.0	1980	0.4148	0.8701
0.1276	12.0	2160	0.4158	0.8711
0.1196	13.0	2340	0.3975	0.8721
0.1137	14.0	2520	0.4221	0.8662
0.1066	15.0	2700	0.4085	0.8677
0.1024	16.0	2880	0.4048	0.8687
0.0995	17.0	3060	0.4139	0.8736
0.0949	18.0	3240	0.3953	0.8706
0.0908	19.0	3420	0.3984	0.8716
0.0882	20.0	3600	0.4006	0.8701
0.0864	21.0	3780	0.3943	0.8731
0.0837	22.0	3960	0.3912	0.8692

Framework versions

Transformers 4.35.2
Pytorch 1.14.0a0+410ce96
Datasets 2.15.0
Tokenizers 0.15.0