Edit model card

scenario-NON-KD-PO-COPY-CDF-CL-D2_data-cl-cardiff_cl_only_beta

This model is a fine-tuned version of haryoaw/scenario-TCR_data-cl-cardiff_cl_only2 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 5.5580
  • Accuracy: 0.4414
  • F1: 0.4359

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 112233
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 30

Training results

Training Loss Epoch Step Validation Loss Accuracy F1
No log 1.09 250 1.7211 0.4823 0.4838
0.5121 2.17 500 1.9396 0.4807 0.4817
0.5121 3.26 750 2.3456 0.4684 0.4700
0.2161 4.35 1000 2.7111 0.4730 0.4701
0.2161 5.43 1250 3.4354 0.4360 0.4244
0.1194 6.52 1500 3.4859 0.4606 0.4586
0.1194 7.61 1750 3.1266 0.4560 0.4519
0.0941 8.7 2000 3.5782 0.4707 0.4693
0.0941 9.78 2250 4.1522 0.4460 0.4409
0.06 10.87 2500 4.4567 0.4576 0.4546
0.06 11.96 2750 4.2186 0.4560 0.4541
0.0399 13.04 3000 4.6178 0.4614 0.4591
0.0399 14.13 3250 4.3070 0.4738 0.4737
0.0407 15.22 3500 4.5620 0.4645 0.4597
0.0407 16.3 3750 4.6845 0.4560 0.4528
0.0316 17.39 4000 4.9378 0.4560 0.4521
0.0316 18.48 4250 4.9708 0.4784 0.4765
0.0207 19.57 4500 5.0696 0.4630 0.4618
0.0207 20.65 4750 5.2248 0.4406 0.4375
0.0148 21.74 5000 5.4988 0.4406 0.4337
0.0148 22.83 5250 5.2698 0.4537 0.4512
0.0121 23.91 5500 5.4028 0.4491 0.4460
0.0121 25.0 5750 5.6057 0.4421 0.4346
0.0121 26.09 6000 5.3745 0.4522 0.4503
0.0121 27.17 6250 5.5548 0.4545 0.4503
0.0068 28.26 6500 5.5513 0.4491 0.4454
0.0068 29.35 6750 5.5580 0.4414 0.4359

Framework versions

  • Transformers 4.33.3
  • Pytorch 2.1.1+cu121
  • Datasets 2.14.5
  • Tokenizers 0.13.3
Downloads last month
5
Inference Examples
Inference API (serverless) is not available, repository is disabled.

Model tree for haryoaw/scenario-NON-KD-PO-COPY-CDF-CL-D2_data-cl-cardiff_cl_only_beta