metadata

base_model: gokuls/model_v1_complete_training_wt_init_48_tiny_freeze_new
tags:
  - generated_from_trainer
datasets:
  - massive
metrics:
  - accuracy
model-index:
  - name: hbertv1-massive-logit_KD-tiny
    results:
      - task:
          name: Text Classification
          type: text-classification
        dataset:
          name: massive
          type: massive
          config: en-US
          split: validation
          args: en-US
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.8465322183964584

hbertv1-massive-logit_KD-tiny

This model is a fine-tuned version of gokuls/model_v1_complete_training_wt_init_48_tiny_freeze_new on the massive dataset. It achieves the following results on the evaluation set:

Loss: 0.5468
Accuracy: 0.8465

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 64
eval_batch_size: 64
seed: 33
distributed_type: multi-GPU
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
4.0471	1.0	180	3.2580	0.2258
2.9727	2.0	360	2.3478	0.3778
2.3183	3.0	540	1.8643	0.5081
1.9162	4.0	720	1.5331	0.6375
1.6284	5.0	900	1.3079	0.6931
1.4163	6.0	1080	1.1495	0.7241
1.263	7.0	1260	1.0287	0.7437
1.1491	8.0	1440	0.9566	0.7575
1.0652	9.0	1620	0.8881	0.7644
0.9661	10.0	1800	0.8426	0.7801
0.9077	11.0	1980	0.7980	0.7796
0.8466	12.0	2160	0.7675	0.7919
0.7996	13.0	2340	0.7422	0.7934
0.7605	14.0	2520	0.7323	0.7954
0.7156	15.0	2700	0.6864	0.8067
0.6867	16.0	2880	0.6730	0.8131
0.6493	17.0	3060	0.6548	0.8160
0.6245	18.0	3240	0.6495	0.8136
0.6038	19.0	3420	0.6282	0.8224
0.57	20.0	3600	0.6123	0.8224
0.556	21.0	3780	0.6020	0.8308
0.5334	22.0	3960	0.5943	0.8298
0.5101	23.0	4140	0.5778	0.8323
0.4948	24.0	4320	0.5740	0.8337
0.4824	25.0	4500	0.5772	0.8337
0.4728	26.0	4680	0.5712	0.8342
0.4596	27.0	4860	0.5691	0.8337
0.4436	28.0	5040	0.5670	0.8396
0.4367	29.0	5220	0.5542	0.8367
0.4249	30.0	5400	0.5512	0.8406
0.4117	31.0	5580	0.5450	0.8387
0.4051	32.0	5760	0.5468	0.8465
0.4	33.0	5940	0.5464	0.8401
0.3939	34.0	6120	0.5451	0.8446
0.3801	35.0	6300	0.5387	0.8441
0.3708	36.0	6480	0.5353	0.8421
0.3686	37.0	6660	0.5320	0.8455

Framework versions

Transformers 4.35.2
Pytorch 1.14.0a0+410ce96
Datasets 2.15.0
Tokenizers 0.15.0