hubert-base-ls960-finetuned-ic-slurp

This model is a fine-tuned version of facebook/hubert-base-ls960 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.9150
• Accuracy: 0.7349

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 24
• eval_batch_size: 24
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 96
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
4.0503	1.0	527	3.9739	0.0814
3.8351	2.0	1055	3.7950	0.0837
3.7053	3.0	1582	3.6592	0.1081
3.4539	4.0	2110	3.3374	0.1772
2.657	5.0	2637	2.5832	0.3443
2.1356	6.0	3165	2.0006	0.4873
1.7409	7.0	3692	1.7459	0.5627
1.4391	8.0	4220	1.6168	0.6104
1.1336	9.0	4747	1.5041	0.6489
1.0151	10.0	5275	1.4378	0.6786
0.8624	11.0	5802	1.4653	0.6880
0.6583	12.0	6330	1.4319	0.6998
0.7089	13.0	6857	1.4993	0.7095
0.6454	14.0	7385	1.5267	0.7036
0.5424	15.0	7912	1.5672	0.7152
0.425	16.0	8440	1.6051	0.7159
0.4016	17.0	8967	1.6342	0.7173
0.3563	18.0	9495	1.7061	0.7110
0.367	19.0	10022	1.6884	0.7177
0.3511	20.0	10550	1.7300	0.7154
0.3573	21.0	11077	1.7361	0.7230
0.2533	22.0	11605	1.7119	0.7279
0.2029	23.0	12132	1.7801	0.7279
0.3279	24.0	12660	1.8096	0.7324
0.2164	25.0	13187	1.8916	0.7237
0.2092	26.0	13715	1.8348	0.7274
0.1757	27.0	14242	1.8824	0.7286
0.2584	28.0	14770	1.9150	0.7349
0.1605	29.0	15297	1.9417	0.7305
0.1815	30.0	15825	1.8939	0.7309
0.1749	31.0	16352	1.9729	0.7327
0.1628	32.0	16880	1.9796	0.7275
0.1369	33.0	17407	2.0156	0.7322

Framework versions

• Transformers 4.37.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2