Edit model card

deberta-v3-large-ddlm

This model is a fine-tuned version of microsoft/deberta-v3-large on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5241

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • gradient_accumulation_steps: 64
  • total_train_batch_size: 128
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 3.0

Training results

Training Loss Epoch Step Validation Loss
0.9823 0.01 1000 0.9163
0.8817 0.02 2000 0.9022
0.9647 0.03 3000 0.8879
0.8646 0.04 4000 0.8577
0.9159 0.06 5000 0.8677
0.8449 0.07 6000 0.8221
0.8681 0.08 7000 0.8332
0.8738 0.09 8000 0.8334
0.8638 0.1 9000 0.8236
0.9066 0.11 10000 0.8200
0.8686 0.12 11000 0.8092
0.7736 0.13 12000 0.8199
0.8054 0.14 13000 0.7972
0.8934 0.16 14000 0.7998
0.7884 0.17 15000 0.7895
0.8278 0.18 16000 0.7586
0.8482 0.19 17000 0.7562
0.8716 0.2 18000 0.7819
0.8881 0.21 19000 0.7878
0.8397 0.22 20000 0.7989
0.811 0.23 21000 0.7846
0.7762 0.24 22000 0.7753
0.7778 0.25 23000 0.7878
0.737 0.27 24000 0.7473
0.8451 0.28 25000 0.7460
0.823 0.29 26000 0.7300
0.7472 0.3 27000 0.7292
0.8048 0.31 28000 0.7697
0.7962 0.32 29000 0.7359
0.8048 0.33 30000 0.7409
0.8095 0.34 31000 0.7434
0.7451 0.35 32000 0.7534
0.6997 0.37 33000 0.7602
0.8116 0.38 34000 0.7566
0.7963 0.39 35000 0.7245
0.786 0.4 36000 0.7311
0.7991 0.41 37000 0.7230
0.723 0.42 38000 0.7209
0.789 0.43 39000 0.7418
0.7296 0.44 40000 0.7325
0.7363 0.45 41000 0.7134
0.758 0.47 42000 0.6948
0.711 0.48 43000 0.6992
0.7984 0.49 44000 0.7055
0.8402 0.5 45000 0.7108
0.8553 0.51 46000 0.7005
0.7538 0.52 47000 0.7208
0.7169 0.53 48000 0.7291
0.7345 0.54 49000 0.7195
0.758 0.55 50000 0.6694
0.7868 0.56 51000 0.6938
0.6966 0.58 52000 0.6867
0.7389 0.59 53000 0.6862
0.7529 0.6 54000 0.7175
0.7345 0.61 55000 0.6970
0.766 0.62 56000 0.7017
0.7043 0.63 57000 0.6916
0.6474 0.64 58000 0.7129
0.7456 0.65 59000 0.6802
0.7512 0.66 60000 0.6951
0.6816 0.68 61000 0.7072
0.7206 0.69 62000 0.6967
0.6439 0.7 63000 0.6798
0.7309 0.71 64000 0.7163
0.6925 0.72 65000 0.6794
0.6833 0.73 66000 0.6637
0.6643 0.74 67000 0.6855
0.6433 0.75 68000 0.7035
0.7595 0.76 69000 0.7008
0.7214 0.78 70000 0.6618
0.7111 0.79 71000 0.6850
0.7375 0.8 72000 0.6909
0.6779 0.81 73000 0.7042
0.6646 0.82 74000 0.6634
0.6616 0.83 75000 0.7020
0.6762 0.84 76000 0.6638
0.7509 0.85 77000 0.6541
0.6963 0.86 78000 0.6781
0.6949 0.87 79000 0.6576
0.6781 0.89 80000 0.6900
0.65 0.9 81000 0.6835
0.7205 0.91 82000 0.6712
0.6901 0.92 83000 0.6699
0.6972 0.93 84000 0.6456
0.7041 0.94 85000 0.6497
0.6864 0.95 86000 0.6432
0.7308 0.96 87000 0.6497
0.6886 0.97 88000 0.6674
0.6947 0.99 89000 0.6638
0.6567 1.0 90000 0.6242
0.7185 1.01 91000 0.6704
0.7435 1.02 92000 0.6681
0.7108 1.03 93000 0.6619
0.6942 1.04 94000 0.6306
0.6998 1.05 95000 0.6409
0.6481 1.06 96000 0.6476
0.727 1.07 97000 0.6354
0.647 1.09 98000 0.6222
0.6622 1.1 99000 0.6119
0.6346 1.11 100000 0.6471
0.6203 1.12 101000 0.6655
0.6765 1.13 102000 0.6473
0.6703 1.14 103000 0.6308
0.6793 1.15 104000 0.6531
0.683 1.16 105000 0.6693
0.6654 1.17 106000 0.6241
0.6626 1.18 107000 0.6215
0.6976 1.2 108000 0.6479
0.7494 1.21 109000 0.6345
0.691 1.22 110000 0.6322
0.6568 1.23 111000 0.6265
0.705 1.24 112000 0.6281
0.6307 1.25 113000 0.6202
0.6828 1.26 114000 0.6158
0.6403 1.27 115000 0.6495
0.6615 1.28 116000 0.6298
0.6237 1.3 117000 0.6234
0.6707 1.31 118000 0.6267
0.6823 1.32 119000 0.6299
0.6333 1.33 120000 0.6169
0.685 1.34 121000 0.6371
0.6941 1.35 122000 0.6245
0.6358 1.36 123000 0.6291
0.6754 1.37 124000 0.6400
0.6286 1.38 125000 0.6148
0.7036 1.4 126000 0.6033
0.645 1.41 127000 0.6295
0.6578 1.42 128000 0.6348
0.651 1.43 129000 0.6222
0.5558 1.44 130000 0.6231
0.6601 1.45 131000 0.6302
0.6304 1.46 132000 0.6127
0.6177 1.47 133000 0.6047
0.5933 1.48 134000 0.6169
0.6307 1.49 135000 0.6012
0.6018 1.51 136000 0.5900
0.6724 1.52 137000 0.6086
0.6367 1.53 138000 0.6414
0.6515 1.54 139000 0.6267
0.5902 1.55 140000 0.5913
0.6523 1.56 141000 0.5992
0.6005 1.57 142000 0.6128
0.6179 1.58 143000 0.6089
0.6154 1.59 144000 0.6353
0.6298 1.61 145000 0.5997
0.5623 1.62 146000 0.5974
0.5787 1.63 147000 0.6165
0.6099 1.64 148000 0.6246
0.658 1.65 149000 0.6116
0.6567 1.66 150000 0.5938
0.6227 1.67 151000 0.5948
0.5858 1.68 152000 0.5822
0.6227 1.69 153000 0.5802
0.6699 1.71 154000 0.6067
0.5989 1.72 155000 0.6073
0.6184 1.73 156000 0.6124
0.6404 1.74 157000 0.6169
0.639 1.75 158000 0.5997
0.6433 1.76 159000 0.5989
0.5574 1.77 160000 0.5796
0.5983 1.78 161000 0.6036
0.6532 1.79 162000 0.5888
0.6679 1.8 163000 0.6038
0.62 1.82 164000 0.5984
0.5541 1.83 165000 0.6003
0.6192 1.84 166000 0.5786
0.6613 1.85 167000 0.6064
0.5923 1.86 168000 0.6018
0.5894 1.87 169000 0.5912
0.6462 1.88 170000 0.5902
0.5811 1.89 171000 0.6030
0.6358 1.9 172000 0.5915
0.614 1.92 173000 0.5886
0.5969 1.93 174000 0.6084
0.6146 1.94 175000 0.6003
0.6051 1.95 176000 0.5835
0.6268 1.96 177000 0.5999
0.6436 1.97 178000 0.5965
0.6167 1.98 179000 0.5789
0.5647 1.99 180000 0.5669
0.6038 2.0 181000 0.6009
0.6082 2.02 182000 0.5799
0.6483 2.03 183000 0.5716
0.5503 2.04 184000 0.5806
0.6231 2.05 185000 0.5699
0.5892 2.06 186000 0.5979
0.5933 2.07 187000 0.5709
0.594 2.08 188000 0.5719
0.5838 2.09 189000 0.5879
0.6039 2.1 190000 0.5984
0.5911 2.11 191000 0.5953
0.563 2.13 192000 0.5772
0.5671 2.14 193000 0.5771
0.6051 2.15 194000 0.5972
0.5852 2.16 195000 0.5917
0.5757 2.17 196000 0.5819
0.6557 2.18 197000 0.5655
0.6055 2.19 198000 0.5820
0.6067 2.2 199000 0.5801
0.6422 2.21 200000 0.5590
0.624 2.23 201000 0.5573
0.6222 2.24 202000 0.5661
0.5597 2.25 203000 0.5786
0.5746 2.26 204000 0.5622
0.6269 2.27 205000 0.5804
0.6241 2.28 206000 0.5696
0.6519 2.29 207000 0.5367
0.6161 2.3 208000 0.5666
0.5415 2.31 209000 0.5633
0.633 2.33 210000 0.5499
0.5566 2.34 211000 0.5822
0.6158 2.35 212000 0.5826
0.5574 2.36 213000 0.5429
0.5748 2.37 214000 0.5736
0.5818 2.38 215000 0.5599
0.6226 2.39 216000 0.5407
0.5733 2.4 217000 0.5759
0.6268 2.41 218000 0.5725
0.5885 2.42 219000 0.5771
0.5708 2.44 220000 0.5654
0.5783 2.45 221000 0.5756
0.61 2.46 222000 0.5647
0.5848 2.47 223000 0.5532
0.5869 2.48 224000 0.5519
0.5717 2.49 225000 0.5621
0.5675 2.5 226000 0.5446
0.6321 2.51 227000 0.5812
0.568 2.52 228000 0.5673
0.5577 2.54 229000 0.5590
0.5888 2.55 230000 0.5628
0.6389 2.56 231000 0.5828
0.5782 2.57 232000 0.5543
0.5871 2.58 233000 0.5575
0.5593 2.59 234000 0.5625
0.6167 2.6 235000 0.5450
0.5828 2.61 236000 0.5627
0.5411 2.62 237000 0.5498
0.6168 2.64 238000 0.5891
0.6508 2.65 239000 0.5811
0.6322 2.66 240000 0.5649
0.6131 2.67 241000 0.5473
0.5419 2.68 242000 0.5583
0.5685 2.69 243000 0.5635
0.5267 2.7 244000 0.5481
0.5357 2.71 245000 0.5474
0.585 2.72 246000 0.5281
0.5894 2.73 247000 0.5457
0.5665 2.75 248000 0.5579
0.5409 2.76 249000 0.5412
0.6051 2.77 250000 0.5447
0.5866 2.78 251000 0.5535
0.5348 2.79 252000 0.5377
0.5606 2.8 253000 0.5524
0.5142 2.81 254000 0.5441
0.543 2.82 255000 0.5499
0.5763 2.83 256000 0.5241

Framework versions

  • Transformers 4.20.0.dev0
  • Pytorch 1.10.0+cu102
  • Datasets 1.15.1
  • Tokenizers 0.11.0
Downloads last month
138
Hosted inference API
Fill-Mask
Examples
Examples
Mask token: [MASK]
This model can be loaded on the Inference API on-demand.