nhankins's picture
nhankins/es_euph_distil_3.0
0fda3ca verified
metadata
license: apache-2.0
base_model: distilbert/distilbert-base-multilingual-cased
tags:
  - generated_from_trainer
metrics:
  - precision
  - recall
model-index:
  - name: distilbert-base-multilingual-cased-lora-text-classification
    results: []

distilbert-base-multilingual-cased-lora-text-classification

This model is a fine-tuned version of distilbert/distilbert-base-multilingual-cased on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5930
  • Precision: 0.7325
  • Recall: 0.7542
  • F1 and accuracy: {'accuracy': 0.6702412868632708, 'f1': 0.74321503131524}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss Precision Recall F1 and accuracy
No log 1.0 372 0.6533 0.6327 1.0 {'accuracy': 0.6327077747989276, 'f1': 0.7750410509031198}
0.67 2.0 744 0.6432 0.6327 1.0 {'accuracy': 0.6327077747989276, 'f1': 0.7750410509031198}
0.6548 3.0 1116 0.6197 0.6341 0.9915 {'accuracy': 0.6327077747989276, 'f1': 0.7735537190082644}
0.6548 4.0 1488 0.6020 0.6678 0.8178 {'accuracy': 0.6273458445040214, 'f1': 0.7352380952380952}
0.6211 5.0 1860 0.5969 0.696 0.7373 {'accuracy': 0.6300268096514745, 'f1': 0.7160493827160493}
0.5929 6.0 2232 0.5954 0.6980 0.7542 {'accuracy': 0.6380697050938338, 'f1': 0.7250509164969451}
0.5887 7.0 2604 0.5940 0.7412 0.7161 {'accuracy': 0.6621983914209115, 'f1': 0.728448275862069}
0.5887 8.0 2976 0.5937 0.7426 0.7458 {'accuracy': 0.675603217158177, 'f1': 0.7441860465116279}
0.5809 9.0 3348 0.5933 0.7247 0.7585 {'accuracy': 0.6648793565683646, 'f1': 0.7412008281573499}
0.5726 10.0 3720 0.5930 0.7325 0.7542 {'accuracy': 0.6702412868632708, 'f1': 0.74321503131524}

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.1.0+cu121
  • Datasets 2.17.0
  • Tokenizers 0.15.2