faiq-wav2vec2-large-xlsr-indo-demo-t4-newparam

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the common_voice_11_0 dataset. It achieves the following results on the evaluation set:

• Loss: 0.2923
• Wer: 0.4000

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 300
• num_epochs: 60
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
10.5954	0.73	100	3.9916	1.0
3.1442	1.46	200	2.9161	1.0
2.8909	2.19	300	2.8833	1.0
2.8336	2.92	400	2.7413	1.0
2.4988	3.65	500	1.3723	1.0262
1.3458	4.38	600	0.5772	0.6982
1.006	5.11	700	0.4542	0.6162
0.8851	5.84	800	0.3988	0.5887
0.7914	6.57	900	0.3782	0.5601
0.7634	7.3	1000	0.3500	0.5309
0.7245	8.03	1100	0.3378	0.5198
0.7061	8.76	1200	0.3362	0.5198
0.6586	9.49	1300	0.3286	0.5094
0.6506	10.22	1400	0.3176	0.4910
0.6216	10.95	1500	0.3051	0.4804
0.6056	11.68	1600	0.3109	0.4787
0.5956	12.41	1700	0.2930	0.4748
0.5748	13.14	1800	0.3028	0.4604
0.5635	13.87	1900	0.3050	0.4671
0.5488	14.6	2000	0.3043	0.4570
0.5401	15.33	2100	0.3005	0.4605
0.5354	16.06	2200	0.2925	0.4508
0.5217	16.79	2300	0.2852	0.4440
0.5103	17.52	2400	0.3000	0.4479
0.5109	18.25	2500	0.2837	0.4470
0.4992	18.98	2600	0.2855	0.4464
0.4967	19.71	2700	0.2842	0.4379
0.4892	20.44	2800	0.2872	0.4359
0.4771	21.17	2900	0.2982	0.4350
0.4664	21.9	3000	0.2998	0.4317
0.4667	22.63	3100	0.2853	0.4340
0.4566	23.36	3200	0.2884	0.4302
0.4533	24.09	3300	0.2960	0.4267
0.441	24.82	3400	0.2921	0.4292
0.4241	25.55	3500	0.2805	0.4271
0.4488	26.28	3600	0.2907	0.4267
0.4348	27.01	3700	0.2712	0.4211
0.4266	27.74	3800	0.2783	0.4216
0.4133	28.47	3900	0.2763	0.4224
0.4224	29.2	4000	0.2829	0.4232
0.4134	29.93	4100	0.2836	0.4135
0.3982	30.66	4200	0.2913	0.4199
0.3998	31.39	4300	0.2914	0.4192
0.3971	32.12	4400	0.2873	0.4174
0.3908	32.85	4500	0.2885	0.4253
0.4	33.58	4600	0.2871	0.4209
0.3915	34.31	4700	0.2897	0.4156
0.3735	35.04	4800	0.2893	0.4136
0.375	35.77	4900	0.2998	0.4133
0.3847	36.5	5000	0.2820	0.4088
0.3723	37.23	5100	0.2859	0.4110
0.3832	37.96	5200	0.2836	0.4107
0.3714	38.69	5300	0.2798	0.4066
0.3735	39.42	5400	0.2839	0.4064
0.365	40.15	5500	0.2834	0.4067
0.3618	40.88	5600	0.2903	0.4100
0.3654	41.61	5700	0.2829	0.4067
0.3546	42.34	5800	0.3003	0.4071
0.3496	43.07	5900	0.2914	0.4080
0.3465	43.8	6000	0.2867	0.4029
0.352	44.53	6100	0.2927	0.4041
0.349	45.26	6200	0.2900	0.4057
0.3494	45.99	6300	0.3001	0.4047
0.3408	46.72	6400	0.2859	0.4054
0.3424	47.45	6500	0.2917	0.4063
0.3365	48.18	6600	0.2961	0.4044
0.334	48.91	6700	0.2913	0.4035
0.3378	49.64	6800	0.2945	0.4027
0.3251	50.36	6900	0.2902	0.4015
0.3401	51.09	7000	0.2892	0.4026
0.3248	51.82	7100	0.2931	0.4028
0.3205	52.55	7200	0.2895	0.4040
0.3254	53.28	7300	0.2969	0.4008
0.3414	54.01	7400	0.2971	0.4009
0.3252	54.74	7500	0.2946	0.4022
0.3166	55.47	7600	0.2953	0.4019
0.3299	56.2	7700	0.2949	0.4015
0.3275	56.93	7800	0.2897	0.4005
0.325	57.66	7900	0.2904	0.4002
0.3278	58.39	8000	0.2885	0.3994
0.3118	59.12	8100	0.2931	0.3999
0.3342	59.85	8200	0.2923	0.4000

Framework versions

• Transformers 4.28.0
• Pytorch 2.0.0+cu118
• Datasets 2.6.1
• Tokenizers 0.13.3