longformer-base-4096-bne-es-finetuned-v2

This model is a fine-tuned version of joheras/longformer-base-4096-bne-es-finetuned-clinais on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.0205
• Precision: 0.4420
• Recall: 0.6075
• F1: 0.5117
• Accuracy: 0.8420

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	1.0	196	0.6855	0.1500	0.2840	0.1963	0.7991
No log	2.0	392	0.5677	0.2372	0.3962	0.2967	0.8271
0.7365	3.0	588	0.5457	0.3043	0.5142	0.3823	0.8412
0.7365	4.0	784	0.5818	0.3268	0.5208	0.4016	0.8303
0.7365	5.0	980	0.5958	0.3595	0.5538	0.4359	0.8385
0.3443	6.0	1176	0.6380	0.3916	0.5660	0.4630	0.8396
0.3443	7.0	1372	0.6835	0.3499	0.5594	0.4305	0.8272
0.2031	8.0	1568	0.6758	0.4088	0.5726	0.4770	0.8441
0.2031	9.0	1764	0.7236	0.3921	0.5792	0.4676	0.8397
0.2031	10.0	1960	0.7699	0.3941	0.5755	0.4678	0.8349
0.1283	11.0	2156	0.7788	0.4004	0.5745	0.4719	0.8315
0.1283	12.0	2352	0.7802	0.4164	0.6019	0.4923	0.8479
0.0861	13.0	2548	0.8092	0.4280	0.5915	0.4966	0.8394
0.0861	14.0	2744	0.8582	0.4211	0.5991	0.4945	0.8373
0.0861	15.0	2940	0.8581	0.3860	0.5925	0.4674	0.8407
0.0589	16.0	3136	0.9137	0.4213	0.6038	0.4963	0.8291
0.0589	17.0	3332	0.8669	0.4287	0.6038	0.5014	0.8448
0.0436	18.0	3528	0.8987	0.4365	0.6028	0.5063	0.8403
0.0436	19.0	3724	0.9389	0.4437	0.5991	0.5098	0.8360
0.0436	20.0	3920	0.9512	0.4479	0.6	0.5129	0.8348
0.0313	21.0	4116	0.9484	0.4535	0.6075	0.5194	0.8445
0.0313	22.0	4312	0.9715	0.4498	0.6123	0.5186	0.8438
0.0236	23.0	4508	0.9726	0.4542	0.6170	0.5232	0.8457
0.0236	24.0	4704	0.9586	0.4531	0.6066	0.5188	0.8427
0.0236	25.0	4900	0.9962	0.4634	0.6160	0.5290	0.8433
0.0206	26.0	5096	1.0098	0.4683	0.6198	0.5335	0.8429
0.0206	27.0	5292	0.9914	0.4527	0.6094	0.5195	0.8414
0.0206	28.0	5488	1.0146	0.4567	0.6113	0.5228	0.8436
0.0154	29.0	5684	1.0199	0.4468	0.6104	0.5159	0.8428
0.0154	30.0	5880	1.0205	0.4420	0.6075	0.5117	0.8420

Framework versions

• Transformers 4.28.1
• Pytorch 2.0.0+cu117
• Datasets 2.12.0
• Tokenizers 0.13.3