Edit model card

lora-roberta-large-finetuned-reduced_captures

This model is a fine-tuned version of roberta-large on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2458
  • Accuracy: 0.9345

Tests metrics:

  • loss: 0.23798689246177673
  • accuracy: 0.9321285694578563

Model description

Captures prediction based on LoRA-adapted RoBERTa-large. This includes labels 0-12 but excluding 9 (criminality).

Intended uses & limitations

Impero Safeguarding & Wellbeing

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 10
  • eval_batch_size: 10
  • seed: 42
  • distributed_type: multi-GPU
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 40
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss Accuracy
0.3924 0.9996 616 0.3862 0.8906
0.4385 1.9992 1232 0.3267 0.9048
0.388 2.9988 1848 0.2849 0.9175
0.3256 4.0 2465 0.2728 0.9207
0.2718 4.9996 3081 0.2939 0.9170
0.2877 5.9992 3697 0.2522 0.9267
0.233 6.9988 4313 0.2624 0.9260
0.1832 8.0 4930 0.2512 0.9317
0.2399 8.9996 5546 0.2458 0.9345
0.1506 9.9959 6160 0.2390 0.9336

Framework versions

  • PEFT 0.12.0
  • Transformers 4.40.2
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.0
Downloads last month
15
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for alunapr/lora-roberta-large-finetuned-reduced_captures

Adapter
(62)
this model