ania3000/ossbert-lemm-v2-1

trainer_output

This model is a fine-tuned version of AlexeySorokin/ossbert-onc-unlab-from_multilingual-bs64-5epochs on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1022
• Lemma accuracy: 98.7973
• Sentence accuracy (lemmas): 85.5046

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	Lemma accuracy	Sentence accuracy (lemmas)
0.7217	1.0	546	0.2753	95.1391	56.5138
0.2237	2.0	1092	0.1710	97.1185	70.0917
0.1312	3.0	1638	0.1313	97.8452	77.2477
0.092	4.0	2184	0.1070	98.3463	80.7339
0.065	5.0	2730	0.1048	98.4089	81.6514
0.0483	6.0	3276	0.1053	98.3964	81.2844
0.0382	7.0	3822	0.1024	98.4716	82.5688
0.03	8.0	4368	0.1017	98.5968	84.0367
0.0233	9.0	4914	0.1034	98.6470	83.4862
0.0157	10.0	5460	0.0984	98.7597	85.3211
0.0099	11.0	6006	0.0891	98.8098	85.3211
0.0081	12.0	6552	0.0940	98.8725	85.8716
0.0052	13.0	7098	0.1023	98.7848	85.3211
0.0046	14.0	7644	0.0980	98.7848	85.5046
0.003	15.0	8190	0.1005	98.8224	85.6881
0.0026	16.0	8736	0.1022	98.7973	85.5046

Framework versions

• Transformers 4.57.3
• Pytorch 2.11.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2