w2v-bert-2.0-luo_19_38h

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the CLEAR-GLOBAL/LUO_19_38H - NA dataset. It achieves the following results on the evaluation set:

• Loss: 0.2546
• Wer: 0.3108
• Cer: 0.0981

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• training_steps: 100000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.3406	1.4948	1000	0.8302	0.6124	0.1932
0.1199	2.9895	2000	0.5006	0.4101	0.1477
0.1048	4.4843	3000	0.3736	0.3655	0.1173
0.0645	5.9791	4000	0.3103	0.3541	0.1165
0.0521	7.4738	5000	0.2974	0.3157	0.0985
0.1057	8.9686	6000	0.2747	0.3197	0.1059
0.0489	10.4634	7000	0.2846	0.2937	0.0961
0.03	11.9581	8000	0.3065	0.3117	0.1018
0.2008	13.4529	9000	0.2546	0.3117	0.0977
0.0562	14.9477	10000	0.3030	0.2809	0.0926
0.026	16.4425	11000	0.2626	0.2923	0.0901
0.0314	17.9372	12000	0.2877	0.2994	0.0908
0.0211	19.4320	13000	0.3100	0.2875	0.0918
0.0175	20.9268	14000	0.3116	0.2888	0.0932

Framework versions

• Transformers 4.48.1
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1