mHuBERT-147-upper-sorbian

This model is a fine-tuned version of utter-project/mHuBERT-147 on the common_voice_17_0 dataset. It achieves the following results on the evaluation set:

• Loss: 3.2172
• Wer: 1.0
• Cer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
3.8611	3.9216	100	3.8610	1.0	1.0
3.2233	7.8431	200	3.2270	1.0	1.0
3.1742	11.7647	300	3.2232	1.0	1.0
3.2292	15.6863	400	3.2238	1.0	1.0
3.2105	19.6078	500	3.2270	1.0	1.0
3.191	23.5294	600	3.2201	1.0	1.0
3.2626	27.4510	700	3.2177	1.0	1.0
3.21	31.3725	800	3.2230	1.0	1.0
3.1871	35.2941	900	3.2211	1.0	1.0
3.2221	39.2157	1000	3.2245	1.0	1.0
3.2408	43.1373	1100	3.2217	1.0	1.0
3.2	47.0588	1200	3.2193	1.0	1.0
3.202	50.9804	1300	3.2181	1.0	1.0
3.2286	54.9020	1400	3.2190	1.0	1.0
3.1863	58.8235	1500	3.2187	1.0	1.0
3.1868	62.7451	1600	3.2175	1.0	1.0
3.226	66.6667	1700	3.2199	1.0	1.0
3.1944	70.5882	1800	3.2196	1.0	1.0
3.1997	74.5098	1900	3.2180	1.0	1.0
3.2184	78.4314	2000	3.2200	1.0	1.0
3.2252	82.3529	2100	3.2188	1.0	1.0
3.208	86.2745	2200	3.2175	1.0	1.0
3.2122	90.1961	2300	3.2170	1.0	1.0
3.2307	94.1176	2400	3.2169	1.0	1.0
3.1852	98.0392	2500	3.2172	1.0	1.0

Framework versions

• Transformers 4.42.0.dev0
• Pytorch 2.3.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1