Hugging Face's logo

haryoaw
/
scenario-NON-KD-PR-COPY-D2_data-AmazonScience_massive_all_1_1_betta-jason

Transformers
PyTorch
xlm-roberta
Generated from Trainer
Inference Endpoints
Model card Files Community
scenario-NON-KD-PR-COPY-D2_data-AmazonScience_massive_all_1_1_betta-jason / README.md
haryoaw's picture
haryoaw
Initial Commit
11110a4 verified
preview code
|
raw
history blame
9.64 kB
metadata
license: mit
base_model: xlm-roberta-base
tags:
  - generated_from_trainer
datasets:
  - massive
metrics:
  - accuracy
  - f1
model-index:
  - name: scenario-NON-KD-PR-COPY-D2_data-AmazonScience_massive_all_1_1_betta-jason
    results: []

scenario-NON-KD-PR-COPY-D2_data-AmazonScience_massive_all_1_1_betta-jason

This model is a fine-tuned version of xlm-roberta-base on the massive dataset. It achieves the following results on the evaluation set:

  • Loss: 1.4867
  • Accuracy: 0.8342
  • F1: 0.8095

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 222
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 30

Training results

Training Loss Epoch Step Validation Loss Accuracy F1
1.6555 0.27 5000 1.5175 0.5778 0.4204
1.1914 0.53 10000 1.1206 0.6936 0.5745
0.9781 0.8 15000 0.9691 0.7383 0.6438
0.7905 1.07 20000 0.8906 0.7631 0.6846
0.7201 1.34 25000 0.8162 0.7809 0.7151
0.6624 1.6 30000 0.7933 0.7914 0.7310
0.6399 1.87 35000 0.7562 0.8000 0.7518
0.5163 2.14 40000 0.7660 0.8057 0.7589
0.503 2.41 45000 0.7659 0.8060 0.7609
0.4855 2.67 50000 0.7446 0.8110 0.7707
0.4744 2.94 55000 0.7199 0.8160 0.7793
0.3807 3.21 60000 0.7582 0.8165 0.7787
0.3902 3.47 65000 0.7526 0.8177 0.7773
0.3757 3.74 70000 0.7647 0.8158 0.7771
0.3496 4.01 75000 0.7529 0.8234 0.7867
0.2917 4.28 80000 0.7961 0.8181 0.7830
0.3042 4.54 85000 0.7907 0.8190 0.7837
0.3098 4.81 90000 0.7699 0.8237 0.7878
0.2381 5.08 95000 0.8027 0.8224 0.7842
0.244 5.34 100000 0.8074 0.8242 0.7898
0.2509 5.61 105000 0.8052 0.8260 0.7924
0.2709 5.88 110000 0.8002 0.8258 0.7925
0.2001 6.15 115000 0.8449 0.8242 0.7909
0.2175 6.41 120000 0.8669 0.8236 0.7892
0.214 6.68 125000 0.8765 0.8252 0.7987
0.222 6.95 130000 0.8332 0.8288 0.8003
0.1713 7.22 135000 0.9019 0.8262 0.7951
0.1751 7.48 140000 0.8958 0.8255 0.7957
0.1794 7.75 145000 0.9065 0.8260 0.7982
0.1544 8.02 150000 0.9200 0.8263 0.8020
0.1445 8.28 155000 0.9510 0.8240 0.7975
0.155 8.55 160000 0.9418 0.8294 0.7999
0.157 8.82 165000 0.9463 0.8295 0.8055
0.1182 9.09 170000 0.9762 0.8273 0.8020
0.1303 9.35 175000 0.9829 0.8271 0.8032
0.131 9.62 180000 1.0006 0.8292 0.8019
0.127 9.89 185000 0.9976 0.8251 0.8009
0.092 10.15 190000 1.0280 0.8278 0.8002
0.1131 10.42 195000 1.0338 0.8271 0.8018
0.1135 10.69 200000 1.0388 0.8277 0.8009
0.115 10.96 205000 1.0341 0.8278 0.8010
0.0871 11.22 210000 1.0720 0.8282 0.8022
0.0992 11.49 215000 1.0691 0.8292 0.8040
0.1007 11.76 220000 1.0821 0.8279 0.8017
0.0776 12.03 225000 1.1169 0.8260 0.7960
0.0833 12.29 230000 1.1196 0.8283 0.8031
0.0843 12.56 235000 1.1386 0.8286 0.8035
0.0884 12.83 240000 1.1368 0.8281 0.8006
0.0637 13.09 245000 1.1611 0.8265 0.8032
0.0721 13.36 250000 1.1857 0.8248 0.7988
0.0731 13.63 255000 1.1788 0.8294 0.8028
0.0754 13.9 260000 1.1879 0.8266 0.8027
0.0615 14.16 265000 1.2059 0.8308 0.8049
0.0741 14.43 270000 1.2121 0.8280 0.8019
0.0677 14.7 275000 1.2192 0.8296 0.8033
0.0736 14.96 280000 1.2419 0.8266 0.7993
0.0561 15.23 285000 1.2439 0.8288 0.8006
0.0554 15.5 290000 1.2603 0.8282 0.7990
0.0634 15.77 295000 1.2692 0.8279 0.8009
0.0445 16.03 300000 1.2826 0.8284 0.8010
0.052 16.3 305000 1.2949 0.8287 0.8048
0.0568 16.57 310000 1.3029 0.8284 0.8031
0.0484 16.84 315000 1.2977 0.8298 0.8043
0.0446 17.1 320000 1.3212 0.8280 0.8031
0.0462 17.37 325000 1.3350 0.8277 0.8013
0.047 17.64 330000 1.3301 0.8297 0.8042
0.048 17.9 335000 1.3293 0.8297 0.8041
0.0421 18.17 340000 1.3249 0.8286 0.8023
0.0405 18.44 345000 1.3471 0.8277 0.8025
0.0458 18.71 350000 1.3654 0.8302 0.8037
0.0463 18.97 355000 1.3435 0.8311 0.8043
0.0392 19.24 360000 1.3816 0.8294 0.8051
0.0379 19.51 365000 1.3748 0.8315 0.8077
0.0358 19.77 370000 1.3599 0.8322 0.8064
0.0276 20.04 375000 1.3637 0.8318 0.8080
0.0359 20.31 380000 1.3649 0.8322 0.8068
0.0322 20.58 385000 1.3857 0.8305 0.8040
0.0404 20.84 390000 1.3926 0.8302 0.8048
0.0338 21.11 395000 1.3937 0.8311 0.8048
0.0307 21.38 400000 1.4248 0.8294 0.8043
0.0301 21.65 405000 1.4184 0.8296 0.8050
0.0289 21.91 410000 1.4154 0.8307 0.8053
0.0266 22.18 415000 1.4249 0.8304 0.8057
0.0282 22.45 420000 1.4311 0.8319 0.8085
0.0306 22.71 425000 1.4417 0.8306 0.8055
0.0272 22.98 430000 1.4490 0.8302 0.8036
0.0264 23.25 435000 1.4372 0.8321 0.8061
0.0232 23.52 440000 1.4548 0.8304 0.8057
0.0264 23.78 445000 1.4496 0.8317 0.8065
0.0194 24.05 450000 1.4454 0.8320 0.8073
0.0228 24.32 455000 1.4532 0.8327 0.8077
0.0229 24.58 460000 1.4574 0.8325 0.8068
0.0203 24.85 465000 1.4709 0.8310 0.8066
0.0183 25.12 470000 1.4707 0.8327 0.8090
0.0157 25.39 475000 1.4689 0.8334 0.8089
0.0175 25.65 480000 1.4704 0.8324 0.8076
0.0211 25.92 485000 1.4806 0.8319 0.8065
0.0158 26.19 490000 1.4881 0.8326 0.8067
0.0209 26.46 495000 1.4771 0.8335 0.8084
0.0193 26.72 500000 1.4882 0.8325 0.8076
0.0184 26.99 505000 1.4740 0.8333 0.8088
0.0145 27.26 510000 1.4818 0.8339 0.8095
0.0141 27.52 515000 1.4909 0.8327 0.8075
0.0157 27.79 520000 1.4787 0.8331 0.8086
0.0168 28.06 525000 1.4842 0.8336 0.8078
0.0179 28.33 530000 1.4847 0.8338 0.8084
0.014 28.59 535000 1.4846 0.8339 0.8089
0.0145 28.86 540000 1.4856 0.8333 0.8087
0.0135 29.13 545000 1.4864 0.8336 0.8085
0.0127 29.39 550000 1.4852 0.8340 0.8088
0.0159 29.66 555000 1.4874 0.8342 0.8093
0.0144 29.93 560000 1.4867 0.8342 0.8095

Framework versions

  • Transformers 4.33.3
  • Pytorch 2.1.1+cu121
  • Datasets 2.14.5
  • Tokenizers 0.13.3