speller-t5-finetuned

This model is a fine-tuned version of UrukHan/t5-russian-spell on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0737
• Rouge1: 25.3988
• Rouge2: 11.756
• Rougel: 25.051
• Rougelsum: 25.2041
• Gen Len: 41.3929

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.1861	0.07	1000	0.1074	24.351	9.9107	24.0512	24.1443	41.4375
0.1629	0.14	2000	0.0918	25.1414	11.3393	24.8541	24.9175	41.3839
0.1545	0.21	3000	0.0898	25.2909	11.5882	24.981	25.0772	41.2857
0.1507	0.29	4000	0.0860	25.1414	11.3393	24.8541	24.9175	41.3571
0.1472	0.36	5000	0.0823	25.2909	11.5882	24.981	25.0772	41.3661
0.1417	0.43	6000	0.0794	24.6565	10.2976	24.265	24.4006	41.4107
0.1411	0.5	7000	0.0788	25.1414	11.3393	24.8541	24.9175	41.3482
0.1395	0.57	8000	0.0786	25.2909	11.5882	24.981	25.0772	41.3125
0.1318	0.64	9000	0.0774	25.2909	11.5882	24.981	25.0772	41.3929
0.1336	0.72	10000	0.0758	25.2394	11.5476	24.9777	25.0511	41.3839
0.1332	0.79	11000	0.0743	25.2394	11.5476	24.9777	25.0511	41.3482
0.1309	0.86	12000	0.0734	25.2394	11.5476	24.9777	25.0511	41.3393
0.1264	0.93	13000	0.0737	25.3988	11.756	25.051	25.2041	41.3929

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1+cu116
• Datasets 2.9.0
• Tokenizers 0.13.2