mms-1b-bigcgen-combined-30hrs-model

This model is a fine-tuned version of facebook/mms-1b-all on the BIGCGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: inf
• Wer: 0.5077

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
14.5866	0.0509	100	inf	1.0167
6.2466	0.1018	200	inf	1.0019
5.4142	0.1526	300	inf	0.9912
2.1374	0.2035	400	inf	0.5952
1.741	0.2544	500	inf	0.5641
1.6543	0.3053	600	inf	0.5607
1.6579	0.3561	700	inf	0.5585
1.676	0.4070	800	inf	0.5475
1.5245	0.4579	900	inf	0.5410
1.6324	0.5088	1000	inf	0.5278
1.6878	0.5597	1100	inf	0.5244
1.4994	0.6105	1200	inf	0.5259
1.544	0.6614	1300	inf	0.5211
1.5796	0.7123	1400	inf	0.5244
1.3625	0.7632	1500	inf	0.5235
1.4826	0.8140	1600	inf	0.5165
1.4439	0.8649	1700	inf	0.5227
1.4778	0.9158	1800	inf	0.5148
1.389	0.9667	1900	inf	0.5130
1.3863	1.0173	2000	inf	0.5177
1.516	1.0682	2100	inf	0.5082
1.474	1.1191	2200	inf	0.5106
1.465	1.1699	2300	inf	0.5077
1.484	1.2208	2400	inf	0.5090
1.3942	1.2717	2500	inf	0.5079

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0