FinBERT_PEFT

This model is a fine-tuned version of ProsusAI/finbert on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3864
• F1: 0.8824

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 6
• eval_batch_size: 6
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	F1
1.4927	1.0	54	1.2082	0.6094
0.909	2.0	108	1.1464	0.6529
0.8769	3.0	162	0.8971	0.6800
0.8455	4.0	216	0.7712	0.8095
0.7669	5.0	270	0.6915	0.8776
0.7396	6.0	324	0.6472	0.8776
0.7756	7.0	378	0.6238	0.8776
0.7261	8.0	432	0.5821	0.7569
0.691	9.0	486	0.6744	0.7233
0.6826	10.0	540	0.5255	0.8218
0.6531	11.0	594	0.7270	0.7233
0.6737	12.0	648	0.5285	0.8095
0.6163	13.0	702	0.6131	0.6078
0.6001	14.0	756	0.5240	0.8776
0.6464	15.0	810	0.3864	0.8824
0.5858	16.0	864	0.5041	0.8776
0.5787	17.0	918	0.5269	0.7233
0.5524	18.0	972	0.6487	0.7233
0.6014	19.0	1026	0.4913	0.7339
0.5631	20.0	1080	0.5969	0.7233
0.5369	21.0	1134	0.4510	0.8824
0.5475	22.0	1188	0.5592	0.6774

Framework versions

• Transformers 4.30.2
• Pytorch 1.13.1+cpu
• Datasets 2.13.2
• Tokenizers 0.13.3