ruT5_cleaner_400

This model is a fine-tuned version of ai-forever/ruT5-base on the None dataset. It achieves the following results on the evaluation set:

• Rouge1: 34.5060
• Rouge2: 8.5
• Rougel: 34.3452
• Rougelsum: 34.1786
• Gen Len: 96.925
• Loss: 0.1071

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len	Validation Loss
No log	1.0	45	30.5417	10.0	30.3472	30.4971	127.0	0.1532
0.2302	2.0	90	30.6793	12.0	30.8033	30.9048	127.0	0.1490
0.1755	3.0	135	30.1766	9.5	29.9206	30.1131	126.875	0.1265
0.2222	4.0	180	27.5952	8.5	27.6310	27.8512	77.95	0.1163
0.1828	5.0	225	24.0357	5.7692	24.0476	23.8571	126.9	0.1069
0.1515	6.0	270	26.5952	6.0	26.7976	26.6667	126.95	0.1094
0.1376	7.0	315	33.75	8.75	33.3333	33.3333	116.0	0.1059
0.1247	8.0	360	34.75	8.75	34.5833	34.4167	97.975	0.1044
0.1114	9.0	405	32.8274	8.9286	32.7976	32.9167	126.925	0.1023
0.1027	10.0	450	34.6429	8.5	34.1667	34.5238	118.0	0.1028
0.1027	11.0	495	31.6667	5.8333	31.4955	31.4583	118.0	0.1050
0.1006	12.0	540	32.0833	6.0	32.0833	31.9643	97.975	0.1043
0.0878	13.0	585	34.7619	8.5	34.5833	34.5833	91.925	0.1037
0.087	14.0	630	33.7013	8.0000	33.5227	33.2955	127.0	0.1075
0.0794	15.0	675	33.0043	8.3803	32.8333	32.6667	126.925	0.1081
0.0958	16.0	720	35.2787	10.5000	35.2906	35.4340	126.925	0.1058
0.0779	17.0	765	34.3452	8.5	34.3452	34.0179	96.925	0.1077
0.0758	18.0	810	34.3452	8.5	34.3452	34.0179	96.925	0.1076
0.0872	19.0	855	34.5060	8.5	34.3452	34.1786	96.925	0.1070
0.0694	20.0	900	34.5060	8.5	34.3452	34.1786	96.925	0.1071

Framework versions

• Transformers 4.57.0
• Pytorch 2.8.0+cu126
• Datasets 4.0.0
• Tokenizers 0.22.1