2404v7

This model is a fine-tuned version of projecte-aina/roberta-base-ca-v2-cased-te on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6435
• Accuracy: 0.8487
• Precision: 0.8488
• Recall: 0.8487
• F1: 0.8487
• Ratio: 0.4916

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.06
• lr_scheduler_warmup_steps: 4
• num_epochs: 10
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	Recall	F1	Ratio
3.5609	0.2597	10	1.6738	0.5168	0.5191	0.5168	0.5020	0.3277
1.4146	0.5195	20	0.9720	0.5294	0.5309	0.5294	0.5237	0.6092
0.924	0.7792	30	0.9377	0.5	0.25	0.5	0.3333	1.0
0.9189	1.0390	40	0.8705	0.5042	0.7511	0.5042	0.3426	0.0042
0.8741	1.2987	50	0.8415	0.5	0.25	0.5	0.3333	1.0
0.8001	1.5584	60	0.7613	0.7311	0.7640	0.7311	0.7224	0.6765
0.7156	1.8182	70	0.6827	0.7773	0.8070	0.7773	0.7718	0.3445
0.7028	2.0779	80	0.6722	0.7941	0.7967	0.7941	0.7937	0.4538
0.5612	2.3377	90	0.6600	0.7689	0.7776	0.7689	0.7671	0.5882
0.5967	2.5974	100	0.6264	0.8067	0.8075	0.8067	0.8066	0.5252
0.6661	2.8571	110	0.6225	0.8193	0.8260	0.8193	0.8184	0.4286
0.6454	3.1169	120	0.6108	0.8109	0.8336	0.8109	0.8077	0.6303
0.541	3.3766	130	0.6002	0.8235	0.8311	0.8235	0.8225	0.4244
0.538	3.6364	140	0.5880	0.8403	0.8428	0.8403	0.8401	0.5420
0.5157	3.8961	150	0.5845	0.8529	0.8603	0.8529	0.8522	0.4286
0.4868	4.1558	160	0.6059	0.8403	0.8548	0.8403	0.8387	0.6008
0.4877	4.4156	170	0.5575	0.8571	0.8572	0.8571	0.8571	0.4916
0.4758	4.6753	180	0.5656	0.8403	0.8403	0.8403	0.8403	0.5
0.4686	4.9351	190	0.5736	0.8571	0.8581	0.8571	0.8571	0.5252
0.4311	5.1948	200	0.5755	0.8571	0.8571	0.8571	0.8571	0.5
0.4452	5.4545	210	0.6053	0.8487	0.8491	0.8487	0.8487	0.5168
0.4396	5.7143	220	0.5726	0.8613	0.8614	0.8613	0.8613	0.4958
0.4923	5.9740	230	0.5534	0.8487	0.8487	0.8487	0.8487	0.5
0.4002	6.2338	240	0.5841	0.8445	0.8446	0.8445	0.8445	0.5042
0.3887	6.4935	250	0.6092	0.8361	0.8402	0.8361	0.8356	0.5546
0.402	6.7532	260	0.6164	0.8445	0.8446	0.8445	0.8445	0.4958
0.4644	7.0130	270	0.6094	0.8571	0.8622	0.8571	0.8566	0.4412
0.3905	7.2727	280	0.5938	0.8487	0.8503	0.8487	0.8486	0.5336
0.3836	7.5325	290	0.6080	0.8487	0.8496	0.8487	0.8486	0.5252
0.3781	7.7922	300	0.6193	0.8529	0.8530	0.8529	0.8529	0.4958
0.4384	8.0519	310	0.6211	0.8529	0.8532	0.8529	0.8529	0.4874
0.3324	8.3117	320	0.6312	0.8445	0.8446	0.8445	0.8445	0.4958
0.3779	8.5714	330	0.6394	0.8529	0.8532	0.8529	0.8529	0.4874
0.3772	8.8312	340	0.6415	0.8529	0.8530	0.8529	0.8529	0.4958
0.3757	9.0909	350	0.6464	0.8403	0.8404	0.8403	0.8403	0.5084
0.3742	9.3506	360	0.6441	0.8487	0.8487	0.8487	0.8487	0.5
0.3647	9.6104	370	0.6437	0.8487	0.8488	0.8487	0.8487	0.4916
0.3502	9.8701	380	0.6435	0.8487	0.8488	0.8487	0.8487	0.4916

Framework versions

• Transformers 4.40.0
• Pytorch 2.2.1+cu121
• Datasets 2.19.0
• Tokenizers 0.19.1