bert-base-multilingual-cased-finetuned-CAJ

This model is a fine-tuned version of bert-base-multilingual-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4226

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.0475	1.0	4	0.9036
0.856	2.0	8	0.7524
0.8014	3.0	12	0.9149
0.7855	4.0	16	0.8052
0.6329	5.0	20	0.8866
0.7714	6.0	24	0.9880
0.6925	7.0	28	0.7490
0.6408	8.0	32	0.6889
0.6983	9.0	36	0.7648
0.6028	10.0	40	0.4431
0.5899	11.0	44	0.6020
0.6032	12.0	48	0.5415
0.5282	13.0	52	0.5124
0.5528	14.0	56	0.6242
0.5191	15.0	60	0.4651
0.5307	16.0	64	0.7029
0.5309	17.0	68	0.5505
0.4425	18.0	72	0.4792
0.4594	19.0	76	0.3245
0.4425	20.0	80	0.5562
0.4409	21.0	84	0.4026
0.442	22.0	88	0.4993
0.4535	23.0	92	0.5693
0.3707	24.0	96	0.4002
0.3914	25.0	100	0.5969
0.3493	26.0	104	0.3247
0.3595	27.0	108	0.3832
0.395	28.0	112	0.4497
0.4186	29.0	116	0.3194
0.4131	30.0	120	0.3699
0.357	31.0	124	0.4968
0.3369	32.0	128	0.4404
0.3734	33.0	132	0.4266
0.342	34.0	136	0.5202
0.3643	35.0	140	0.3872
0.3362	36.0	144	0.5037
0.3302	37.0	148	0.5572
0.3241	38.0	152	0.4138
0.299	39.0	156	0.2888
0.3383	40.0	160	0.5453
0.3786	41.0	164	0.3909
0.3121	42.0	168	0.4414
0.3357	43.0	172	0.3216
0.3601	44.0	176	0.3046
0.2662	45.0	180	0.4090
0.2979	46.0	184	0.4571
0.4222	47.0	188	0.4513
0.3006	48.0	192	0.3829
0.3385	49.0	196	0.3473
0.2711	50.0	200	0.3419

Framework versions

• Transformers 4.39.3
• Pytorch 2.2.2+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2