mms-1b-bigcgen-female-10hrs-model

This model is a fine-tuned version of facebook/mms-1b-all on the BIGCGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: inf
• Wer: 0.5260

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
14.5837	0.1502	100	inf	1.0057
6.1158	0.3003	200	inf	1.0050
5.2493	0.4505	300	inf	0.9905
2.2567	0.6006	400	inf	0.5970
1.8394	0.7508	500	inf	0.6075
1.7984	0.9009	600	inf	0.5689
1.7749	1.0511	700	inf	0.5732
1.704	1.2012	800	inf	0.5666
1.6314	1.3514	900	inf	0.5632
1.5753	1.5015	1000	inf	0.5407
1.5827	1.6517	1100	inf	0.5442
1.5613	1.8018	1200	inf	0.5446
1.5646	1.9520	1300	inf	0.5267
1.563	2.1021	1400	inf	0.5521
1.4125	2.2523	1500	inf	0.5224
1.421	2.4024	1600	inf	0.5337
1.4159	2.5526	1700	inf	0.5260
1.5017	2.7027	1800	inf	0.5246
1.5513	2.8529	1900	inf	0.5530
1.4063	3.0030	2000	inf	0.5142
1.3611	3.1532	2100	inf	0.5308
1.4059	3.3033	2200	inf	0.5328
1.4289	3.4535	2300	inf	0.5373
1.3879	3.6036	2400	inf	0.5264
1.4592	3.7538	2500	inf	0.5267

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0