wav2vec2-demo-F01-2

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6361
• Wer: 0.9025

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
23.7408	0.81	500	3.3782	1.0
3.348	1.62	1000	2.9501	1.0
2.8539	2.44	1500	2.6975	1.0
2.311	3.25	2000	1.7770	1.3175
1.5102	4.06	2500	1.3481	1.3515
1.0616	4.87	3000	1.4306	1.2313
0.8493	5.68	3500	1.2261	1.1701
0.7058	6.49	4000	1.2132	1.1111
0.6129	7.31	4500	1.4230	1.1429
0.5513	8.12	5000	1.2003	1.0499
0.4957	8.93	5500	1.5534	1.1043
0.4456	9.74	6000	1.2315	1.0658
0.4101	10.55	6500	1.1621	1.0680
0.3776	11.36	7000	1.4302	1.0385
0.3318	12.18	7500	1.3488	0.9977
0.3189	12.99	8000	1.4050	1.0295
0.3103	13.8	8500	1.4535	1.0385
0.2791	14.61	9000	1.3318	1.0181
0.2681	15.42	9500	1.5199	0.9909
0.2352	16.23	10000	1.5019	1.0023
0.235	17.05	10500	1.7984	0.9955
0.2319	17.86	11000	1.3399	0.9705
0.221	18.67	11500	1.8316	0.9342
0.2154	19.48	12000	1.6837	0.9637
0.1911	20.29	12500	1.6999	0.9388
0.1754	21.1	13000	1.4801	0.9274
0.1776	21.92	13500	1.7954	0.9206
0.1616	22.73	14000	1.7891	0.9320
0.1579	23.54	14500	1.5692	0.9116
0.173	24.35	15000	1.4928	0.9048
0.1561	25.16	15500	1.6492	0.9116
0.1542	25.97	16000	1.7356	0.9048
0.131	26.79	16500	1.7785	0.9048
0.1295	27.6	17000	1.6532	0.9116
0.1374	28.41	17500	1.6760	0.9093
0.1186	29.22	18000	1.6361	0.9025

Framework versions

• Transformers 4.23.1
• Pytorch 1.12.1+cu113
• Datasets 1.18.3
• Tokenizers 0.13.2