jehone-shqip

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1423
• Wer: 65.3266

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 40
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.4839	0.3077	1	1.1928	64.3216
0.4172	0.6154	2	1.1452	62.3116
0.3064	0.9231	3	1.1150	61.8090
0.2727	1.2308	4	1.0900	62.3116
0.1152	1.5385	5	1.0778	60.3015
0.1147	1.8462	6	1.0780	62.3116
0.094	2.1538	7	1.0866	62.8141
0.0816	2.4615	8	1.0908	62.8141
0.0535	2.7692	9	1.0927	64.3216
0.0295	3.0769	10	1.0971	63.8191
0.0198	3.3846	11	1.1000	65.3266
0.0346	3.6923	12	1.1014	65.3266
0.0199	4.0	13	1.1062	67.3367
0.0161	4.3077	14	1.1124	67.8392
0.012	4.6154	15	1.1151	65.3266
0.0121	4.9231	16	1.1180	66.3317
0.0079	5.2308	17	1.1207	65.8291
0.0055	5.5385	18	1.1236	64.8241
0.0082	5.8462	19	1.1270	64.8241
0.0089	6.1538	20	1.1290	61.8090
0.0043	6.4615	21	1.1312	62.3116
0.0039	6.7692	22	1.1332	61.8090
0.0041	7.0769	23	1.1355	62.3116
0.0032	7.3846	24	1.1373	66.8342
0.0039	7.6923	25	1.1385	66.8342
0.0031	8.0	26	1.1397	66.8342
0.0042	8.3077	27	1.1409	66.3317
0.0026	8.6154	28	1.1416	66.3317
0.0027	8.9231	29	1.1418	65.8291
0.0029	9.2308	30	1.1419	65.8291
0.0026	9.5385	31	1.1419	65.8291
0.0022	9.8462	32	1.1424	65.8291
0.0024	10.1538	33	1.1420	65.3266
0.0025	10.4615	34	1.1423	65.3266
0.0022	10.7692	35	1.1422	65.3266
0.0021	11.0769	36	1.1424	65.3266
0.0023	11.3846	37	1.1423	65.3266
0.0021	11.6923	38	1.1423	65.3266
0.0023	12.0	39	1.1422	65.3266
0.0022	12.3077	40	1.1423	65.3266

Framework versions

• Transformers 4.43.0.dev0
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1