distilbert-base-uncased-lora-text-classification

This model is a fine-tuned version of distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.3734
• Accuracy: {'accuracy': 0.6005221932114883}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 12

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	192	1.8883	{'accuracy': 0.36945169712793735}
No log	2.0	384	1.6563	{'accuracy': 0.4216710182767624}
1.8736	3.0	576	1.5385	{'accuracy': 0.46736292428198434}
1.8736	4.0	768	1.4276	{'accuracy': 0.5274151436031331}
1.8736	5.0	960	1.3666	{'accuracy': 0.5613577023498695}
1.2939	6.0	1152	1.3688	{'accuracy': 0.5613577023498695}
1.2939	7.0	1344	1.3397	{'accuracy': 0.5783289817232375}
0.9487	8.0	1536	1.3576	{'accuracy': 0.5757180156657964}
0.9487	9.0	1728	1.3523	{'accuracy': 0.5939947780678851}
0.9487	10.0	1920	1.3819	{'accuracy': 0.5926892950391645}
0.7324	11.0	2112	1.3746	{'accuracy': 0.597911227154047}
0.7324	12.0	2304	1.3734	{'accuracy': 0.6005221932114883}

Framework versions

• Transformers 4.33.0
• Pytorch 2.0.0
• Datasets 2.1.0
• Tokenizers 0.13.3