faiq-wav2vec2-large-xlsr-indo-demo-v100-newparameter

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the common_voice_11_0 dataset. It achieves the following results on the evaluation set:

• Loss: 0.2880
• Wer: 0.3926

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 300
• num_epochs: 60
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
11.1998	0.73	100	4.4160	1.0
3.166	1.46	200	2.9133	1.0
2.89	2.19	300	2.8775	1.0
2.8188	2.92	400	2.7385	1.0007
2.656	3.65	500	1.9852	1.0208
1.5392	4.38	600	0.6224	0.7274
1.0493	5.11	700	0.4720	0.6435
0.9097	5.84	800	0.4157	0.5892
0.8133	6.57	900	0.3720	0.5668
0.7818	7.3	1000	0.3570	0.5416
0.7376	8.03	1100	0.3346	0.5209
0.7131	8.76	1200	0.3234	0.5124
0.6624	9.49	1300	0.3313	0.5005
0.6609	10.22	1400	0.3162	0.4882
0.6331	10.95	1500	0.3257	0.4850
0.6123	11.68	1600	0.3271	0.4804
0.6018	12.41	1700	0.3052	0.4729
0.5822	13.14	1800	0.3048	0.4642
0.5724	13.87	1900	0.3058	0.4698
0.5519	14.6	2000	0.2952	0.4601
0.54	15.33	2100	0.2889	0.4582
0.5393	16.06	2200	0.2875	0.4491
0.5273	16.79	2300	0.2843	0.4465
0.5145	17.52	2400	0.2782	0.4403
0.515	18.25	2500	0.2916	0.4457
0.5006	18.98	2600	0.2795	0.4418
0.5025	19.71	2700	0.2788	0.4360
0.491	20.44	2800	0.2903	0.4348
0.478	21.17	2900	0.2819	0.4264
0.4721	21.9	3000	0.2842	0.4258
0.4709	22.63	3100	0.2865	0.4316
0.4618	23.36	3200	0.2911	0.4414
0.4592	24.09	3300	0.2901	0.4267
0.4448	24.82	3400	0.2871	0.4247
0.4347	25.55	3500	0.2772	0.4225
0.4515	26.28	3600	0.2907	0.4203
0.4368	27.01	3700	0.2749	0.4175
0.4317	27.74	3800	0.2781	0.4204
0.4158	28.47	3900	0.2847	0.4214
0.4225	29.2	4000	0.2815	0.4160
0.4205	29.93	4100	0.2792	0.4088
0.4035	30.66	4200	0.2801	0.4097
0.405	31.39	4300	0.2853	0.4105
0.404	32.12	4400	0.2728	0.4083
0.3929	32.85	4500	0.2801	0.4118
0.4022	33.58	4600	0.2801	0.4058
0.3951	34.31	4700	0.2857	0.4095
0.3716	35.04	4800	0.2882	0.4055
0.3786	35.77	4900	0.2866	0.4056
0.3886	36.5	5000	0.2924	0.4090
0.3746	37.23	5100	0.2820	0.4066
0.3883	37.96	5200	0.2730	0.4006
0.374	38.69	5300	0.2825	0.3992
0.3741	39.42	5400	0.2950	0.4011
0.3709	40.15	5500	0.2930	0.4015
0.3579	40.88	5600	0.2919	0.4015
0.3704	41.61	5700	0.2826	0.4014
0.3648	42.34	5800	0.2847	0.4001
0.3549	43.07	5900	0.2934	0.4020
0.3522	43.8	6000	0.2846	0.4004
0.3575	44.53	6100	0.2892	0.3986
0.3512	45.26	6200	0.2952	0.4008
0.3525	45.99	6300	0.2918	0.3961
0.3414	46.72	6400	0.2884	0.3945
0.3458	47.45	6500	0.2918	0.3979
0.337	48.18	6600	0.2838	0.3933
0.3352	48.91	6700	0.2872	0.3939
0.3374	49.64	6800	0.2860	0.3941
0.327	50.36	6900	0.2820	0.3920
0.3396	51.09	7000	0.2884	0.3946
0.3246	51.82	7100	0.2960	0.3930
0.322	52.55	7200	0.2881	0.3949
0.331	53.28	7300	0.2927	0.3930
0.3406	54.01	7400	0.2940	0.3954
0.3292	54.74	7500	0.2873	0.3946
0.3209	55.47	7600	0.2881	0.3915
0.3275	56.2	7700	0.2921	0.3922
0.3311	56.93	7800	0.2877	0.3915
0.3282	57.66	7900	0.2866	0.3931
0.3255	58.39	8000	0.2863	0.3917
0.3182	59.12	8100	0.2878	0.3931
0.3321	59.85	8200	0.2880	0.3926

Framework versions

• Transformers 4.28.0
• Pytorch 2.0.0+cu118
• Datasets 2.6.1
• Tokenizers 0.13.3