nhankins/zh_euph_distilbert_lora_final

distilbert-base-multilingual-cased-lora-text-classification

This model is a fine-tuned version of distilbert/distilbert-base-multilingual-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4881
• Precision: 0.7966
• Recall: 0.9216
• F1 and accuracy: {'accuracy': 0.7605985037406484, 'f1': 0.8545454545454545}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1 and accuracy
No log	1.0	401	0.5429	0.7631	1.0	{'accuracy': 0.7630922693266833, 'f1': 0.8656294200848657}
0.5808	2.0	802	0.5361	0.7631	1.0	{'accuracy': 0.7630922693266833, 'f1': 0.8656294200848657}
0.5805	3.0	1203	0.5235	0.7631	1.0	{'accuracy': 0.7630922693266833, 'f1': 0.8656294200848657}
0.5554	4.0	1604	0.5096	0.7669	1.0	{'accuracy': 0.7680798004987531, 'f1': 0.8680851063829788}
0.5214	5.0	2005	0.5046	0.7734	0.9706	{'accuracy': 0.7605985037406484, 'f1': 0.8608695652173913}
0.5214	6.0	2406	0.4971	0.7950	0.9379	{'accuracy': 0.7680798004987531, 'f1': 0.8605697151424289}
0.5152	7.0	2807	0.4919	0.7983	0.9183	{'accuracy': 0.7605985037406484, 'f1': 0.8541033434650457}
0.4956	8.0	3208	0.4881	0.8017	0.9118	{'accuracy': 0.7605985037406484, 'f1': 0.8532110091743118}
0.4891	9.0	3609	0.4881	0.7972	0.9248	{'accuracy': 0.7630922693266833, 'f1': 0.8562783661119516}
0.5038	10.0	4010	0.4881	0.7966	0.9216	{'accuracy': 0.7605985037406484, 'f1': 0.8545454545454545}

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.17.0
• Tokenizers 0.15.2