w2v-bert-2.0-lg-CV-Fleurs-5hrs-v10

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the None dataset. It achieves the following results on the evaluation set:

Model description

More information needed

More information needed

More information needed

The following hyperparameters were used during training:

learning_rate: 3e-05
train_batch_size: 8
eval_batch_size: 4
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 16
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
num_epochs: 100
mixed_precision_training: Native AMP

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
1.6631	1.0	163	0.5729	0.5671	0.1312
0.4064	2.0	326	0.4681	0.4766	0.1051
0.3071	3.0	489	0.4348	0.4344	0.0961
0.2518	4.0	652	0.4442	0.4112	0.0885
0.2154	5.0	815	0.4503	0.4042	0.0877
0.1824	6.0	978	0.4146	0.4287	0.0910
0.1625	7.0	1141	0.4245	0.4082	0.0878
0.1354	8.0	1304	0.4579	0.4335	0.0881
0.1182	9.0	1467	0.4593	0.4242	0.0916
0.1025	10.0	1630	0.4587	0.4046	0.0881
0.0863	11.0	1793	0.5591	0.3991	0.0854
0.0723	12.0	1956	0.4954	0.4041	0.0863
0.0619	13.0	2119	0.5618	0.4127	0.0890
0.0543	14.0	2282	0.5675	0.4115	0.0892
0.0461	15.0	2445	0.6027	0.3968	0.0861
0.0412	16.0	2608	0.5939	0.4138	0.0895
0.0348	17.0	2771	0.6687	0.4157	0.0894
0.0342	18.0	2934	0.7066	0.3849	0.0838
0.0288	19.0	3097	0.7669	0.3899	0.0849
0.0233	20.0	3260	0.6945	0.4000	0.0865
0.0218	21.0	3423	0.7192	0.4086	0.0883
0.02	22.0	3586	0.6980	0.3940	0.0843
0.017	23.0	3749	0.7983	0.4014	0.0873
0.0153	24.0	3912	0.7599	0.3942	0.0853
0.0142	25.0	4075	0.7761	0.3993	0.0858
0.0137	26.0	4238	0.7491	0.3996	0.0857
0.0157	27.0	4401	0.7682	0.3994	0.0858
0.0113	28.0	4564	0.7784	0.4006	0.0875
0.0111	29.0	4727	0.8020	0.4020	0.0864
0.0105	30.0	4890	0.8414	0.3891	0.0838