distilbert-base-pl-cased-finetuned-eo

This model is a fine-tuned version of Geotrend/distilbert-base-pl-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4275

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
2.4541	1.0	20	1.5317
1.5138	2.0	40	1.2678
1.2893	3.0	60	1.0208
1.135	4.0	80	1.0185
1.0288	5.0	100	0.9068
0.9607	6.0	120	0.8102
0.8933	7.0	140	0.8191
0.8595	8.0	160	0.8191
0.8314	9.0	180	0.6913
0.7866	10.0	200	0.6317
0.7642	11.0	220	0.6345
0.7114	12.0	240	0.6749
0.7033	13.0	260	0.6147
0.6435	14.0	280	0.5894
0.6545	15.0	300	0.5822
0.6255	16.0	320	0.5452
0.602	17.0	340	0.5521
0.594	18.0	360	0.4672
0.5572	19.0	380	0.5322
0.5614	20.0	400	0.5200
0.556	21.0	420	0.5213
0.5616	22.0	440	0.5052
0.5249	23.0	460	0.4811
0.5403	24.0	480	0.4990
0.5081	25.0	500	0.4572
0.5153	26.0	520	0.4845
0.4962	27.0	540	0.4954
0.4834	28.0	560	0.4418
0.4782	29.0	580	0.4987
0.5126	30.0	600	0.5001
0.4829	31.0	620	0.4515
0.4671	32.0	640	0.4400
0.4514	33.0	660	0.4875
0.488	34.0	680	0.4000
0.4642	35.0	700	0.4538
0.4481	36.0	720	0.4452
0.4505	37.0	740	0.4636
0.4554	38.0	760	0.4645
0.4322	39.0	780	0.4615
0.4394	40.0	800	0.4676
0.4325	41.0	820	0.4072
0.4077	42.0	840	0.4518
0.416	43.0	860	0.4514
0.4382	44.0	880	0.4459
0.4395	45.0	900	0.4757
0.4188	46.0	920	0.4870
0.4052	47.0	940	0.4658
0.4273	48.0	960	0.4168
0.435	49.0	980	0.3387
0.432	50.0	1000	0.4673

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0