Regression_Albert_2

This model is a fine-tuned version of albert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.4677
• Mse: 3.4677
• Mae: 1.6443
• R2: -0.7892
• Accuracy: 0.1429

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Mse	Mae	R2	Accuracy
No log	1.0	3	1.5252	1.5252	1.1112	-0.4311	0.1429
No log	2.0	6	1.3872	1.3872	1.0182	-0.3016	0.2857
No log	3.0	9	2.5294	2.5294	1.2286	-1.3733	0.4286
No log	4.0	12	3.8938	3.8938	1.5973	-2.6535	0.1429
No log	5.0	15	5.5535	5.5535	2.0657	-4.2108	0.0
No log	6.0	18	7.0814	7.0814	2.3965	-5.6444	0.0
No log	7.0	21	7.5510	7.5510	2.4797	-6.0850	0.0
No log	8.0	24	6.6578	6.6578	2.2618	-5.2469	0.0
No log	9.0	27	5.7320	5.7320	2.0266	-4.3783	0.1429
No log	10.0	30	6.9700	6.9700	2.2615	-5.5398	0.1429
No log	11.0	33	7.9965	7.9965	2.4660	-6.5030	0.0
No log	12.0	36	7.7483	7.7483	2.4191	-6.2700	0.0
No log	13.0	39	7.6652	7.6652	2.3991	-6.1921	0.0
No log	14.0	42	7.7858	7.7858	2.4436	-6.3053	0.0
No log	15.0	45	7.8747	7.8747	2.4527	-6.3886	0.0
No log	16.0	48	7.6498	7.6498	2.4028	-6.1776	0.0
No log	17.0	51	7.3543	7.3543	2.3428	-5.9004	0.0
No log	18.0	54	7.3656	7.3656	2.3428	-5.9110	0.0
No log	19.0	57	7.3185	7.3185	2.3360	-5.8668	0.0
No log	20.0	60	7.3529	7.3529	2.3417	-5.8990	0.0

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu116
• Datasets 2.9.0
• Tokenizers 0.13.2