Hugging Face's logo

haryoaw
/
scenario-NON-KD-PO-COPY-D2_data-AmazonScience_massive_all_1_155

Transformers
Safetensors
deberta-v2
Generated from Trainer
Inference Endpoints
Model card Files Community
scenario-NON-KD-PO-COPY-D2_data-AmazonScience_massive_all_1_155 / README.md
haryoaw's picture
haryoaw
Upload tokenizer
41a48a9 verified
preview code
|
raw
history blame contribute delete
4.26 kB
metadata
base_model: haryoaw/scenario-MDBT-TCR-MSV-CL
datasets:
  - massive
library_name: transformers
license: mit
metrics:
  - accuracy
  - f1
tags:
  - generated_from_trainer
model-index:
  - name: scenario-NON-KD-PO-COPY-D2_data-AmazonScience_massive_all_1_155
    results: []

scenario-NON-KD-PO-COPY-D2_data-AmazonScience_massive_all_1_155

This model is a fine-tuned version of haryoaw/scenario-MDBT-TCR-MSV-CL on the massive dataset. It achieves the following results on the evaluation set:

  • Loss: 1.5339
  • Accuracy: 0.8554
  • F1: 0.8295

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 55
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss Accuracy F1
0.564 0.2672 5000 0.6868 0.8215 0.7763
0.4391 0.5344 10000 0.6652 0.8372 0.8055
0.3679 0.8017 15000 0.6546 0.8445 0.8145
0.2392 1.0689 20000 0.7360 0.8438 0.8175
0.235 1.3361 25000 0.7624 0.8490 0.8230
0.2161 1.6033 30000 0.7277 0.8492 0.8275
0.2069 1.8706 35000 0.7749 0.8471 0.8235
0.145 2.1378 40000 0.8993 0.8425 0.8102
0.1482 2.4050 45000 0.8471 0.8474 0.8196
0.1479 2.6722 50000 0.8894 0.8511 0.8280
0.1379 2.9394 55000 0.8645 0.8489 0.8240
0.1063 3.2067 60000 0.9803 0.8516 0.8284
0.1077 3.4739 65000 0.9819 0.8475 0.8260
0.1036 3.7411 70000 0.9888 0.8478 0.8242
0.092 4.0083 75000 1.0324 0.8503 0.8273
0.0821 4.2756 80000 1.0974 0.8473 0.8174
0.0827 4.5428 85000 1.1001 0.8483 0.8255
0.0764 4.8100 90000 1.0927 0.8490 0.8267
0.0586 5.0772 95000 1.1587 0.8531 0.8259
0.0655 5.3444 100000 1.1937 0.8499 0.8260
0.0604 5.6117 105000 1.1844 0.8500 0.8258
0.0574 5.8789 110000 1.2190 0.8521 0.8271
0.045 6.1461 115000 1.2595 0.8533 0.8288
0.0473 6.4133 120000 1.2713 0.8527 0.8261
0.0477 6.6806 125000 1.2822 0.8527 0.8286
0.0392 6.9478 130000 1.2957 0.8562 0.8312
0.0365 7.2150 135000 1.3796 0.8526 0.8309
0.0372 7.4822 140000 1.3954 0.8526 0.8239
0.0344 7.7495 145000 1.3817 0.8542 0.8275
0.0257 8.0167 150000 1.4356 0.8554 0.8301
0.0237 8.2839 155000 1.4554 0.8546 0.8292
0.0235 8.5511 160000 1.4907 0.8553 0.8319
0.0243 8.8183 165000 1.4675 0.8540 0.8281
0.0171 9.0856 170000 1.5041 0.8552 0.8292
0.0158 9.3528 175000 1.5237 0.8558 0.8303
0.0192 9.6200 180000 1.5367 0.8548 0.8286
0.0159 9.8872 185000 1.5339 0.8554 0.8295

Framework versions

  • Transformers 4.44.2
  • Pytorch 2.1.1+cu121
  • Datasets 2.14.5
  • Tokenizers 0.19.1