xls-r-300-cv17-upper-sorbian

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the common_voice_17_0 dataset. It achieves the following results on the evaluation set:

• Loss: 0.0001
• Wer: 0.0
• Cer: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
4.0153	1.4286	100	4.2633	1.0	1.0
3.2616	2.8571	200	3.2377	1.0	1.0
3.1754	4.2857	300	3.2002	0.9892	0.9774
0.686	5.7143	400	0.5966	0.65	0.1446
0.2345	7.1429	500	0.2319	0.2981	0.0567
0.1576	8.5714	600	0.1181	0.1759	0.0309
0.1403	10.0	700	0.0603	0.0956	0.0155
0.0833	11.4286	800	0.0297	0.0475	0.0077
0.0552	12.8571	900	0.0261	0.0405	0.0055
0.059	14.2857	1000	0.0225	0.0335	0.0054
0.093	15.7143	1100	0.0136	0.0203	0.0035
0.0785	17.1429	1200	0.0153	0.0253	0.0043
0.0551	18.5714	1300	0.0120	0.0215	0.0030
0.0742	20.0	1400	0.0074	0.0095	0.0013
0.0285	21.4286	1500	0.0053	0.0095	0.0014
0.021	22.8571	1600	0.0066	0.0108	0.0015
0.0297	24.2857	1700	0.0038	0.0057	0.0007
0.0451	25.7143	1800	0.0062	0.0095	0.0014
0.0353	27.1429	1900	0.0270	0.0222	0.0038
0.0426	28.5714	2000	0.0066	0.0095	0.0015
0.0296	30.0	2100	0.0042	0.0076	0.0011
0.0348	31.4286	2200	0.0031	0.0051	0.0008
0.0336	32.8571	2300	0.0047	0.0070	0.0010
0.0126	34.2857	2400	0.0021	0.0051	0.0007
0.0287	35.7143	2500	0.0031	0.0063	0.0008
0.0253	37.1429	2600	0.0046	0.0070	0.0012
0.0317	38.5714	2700	0.0038	0.0044	0.0007
0.1223	40.0	2800	0.0035	0.0076	0.0012
0.0337	41.4286	2900	0.0031	0.0032	0.0005
0.0125	42.8571	3000	0.0039	0.0076	0.0010
0.0043	44.2857	3100	0.0026	0.0013	0.0003
0.0261	45.7143	3200	0.0016	0.0013	0.0002
0.0129	47.1429	3300	0.0014	0.0038	0.0007
0.0168	48.5714	3400	0.0031	0.0044	0.0006
0.0274	50.0	3500	0.0005	0.0	0.0
0.0157	51.4286	3600	0.0014	0.0025	0.0003
0.0149	52.8571	3700	0.0010	0.0019	0.0003
0.0095	54.2857	3800	0.0009	0.0019	0.0003
0.0158	55.7143	3900	0.0031	0.0044	0.0005
0.0103	57.1429	4000	0.0015	0.0019	0.0004
0.0262	58.5714	4100	0.0024	0.0013	0.0001
0.0515	60.0	4200	0.0007	0.0032	0.0003
0.0085	61.4286	4300	0.0004	0.0	0.0
0.0169	62.8571	4400	0.0018	0.0032	0.0005
0.0096	64.2857	4500	0.0004	0.0	0.0
0.0052	65.7143	4600	0.0002	0.0	0.0
0.0219	67.1429	4700	0.0003	0.0	0.0
0.0031	68.5714	4800	0.0002	0.0	0.0
0.0033	70.0	4900	0.0003	0.0	0.0
0.0026	71.4286	5000	0.0008	0.0006	0.0001
0.0036	72.8571	5100	0.0005	0.0006	0.0001
0.0045	74.2857	5200	0.0002	0.0	0.0
0.0038	75.7143	5300	0.0025	0.0044	0.0007
0.0101	77.1429	5400	0.0003	0.0	0.0
0.0075	78.5714	5500	0.0002	0.0	0.0
0.0086	80.0	5600	0.0001	0.0	0.0
0.0047	81.4286	5700	0.0001	0.0	0.0
0.0009	82.8571	5800	0.0001	0.0	0.0
0.0056	84.2857	5900	0.0001	0.0	0.0
0.009	85.7143	6000	0.0001	0.0	0.0
0.0003	87.1429	6100	0.0001	0.0	0.0
0.0005	88.5714	6200	0.0001	0.0	0.0
0.0034	90.0	6300	0.0001	0.0	0.0
0.0103	91.4286	6400	0.0001	0.0	0.0
0.0027	92.8571	6500	0.0001	0.0	0.0
0.0029	94.2857	6600	0.0001	0.0	0.0
0.004	95.7143	6700	0.0001	0.0	0.0
0.0011	97.1429	6800	0.0001	0.0	0.0
0.0033	98.5714	6900	0.0001	0.0	0.0
0.0003	100.0	7000	0.0001	0.0	0.0

Framework versions

• Transformers 4.42.0.dev0
• Pytorch 2.3.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1