Edit model card

fine-tuned-roberta2-nosql-injection

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0134

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 4
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 100
  • num_epochs: 500

Training results

Training Loss Epoch Step Validation Loss
1.2572 1.0 158 0.2235
0.1144 2.0 316 0.0546
0.0873 3.0 474 0.0943
0.0488 4.0 632 0.0324
0.0453 5.0 790 0.0970
0.1358 6.0 948 0.0003
0.0208 7.0 1106 0.0025
0.0409 8.0 1264 0.0000
0.0489 9.0 1422 0.0505
0.0808 10.0 1580 0.0000
0.0219 11.0 1738 0.0929
0.1204 12.0 1896 0.0000
0.0355 13.0 2054 0.0001
0.0093 14.0 2212 0.0671
0.0216 15.0 2370 0.0279
0.0388 16.0 2528 0.0173
0.0185 17.0 2686 0.0303
0.0069 18.0 2844 0.0160
0.0481 19.0 3002 0.0557
0.0459 20.0 3160 0.0022
0.0146 21.0 3318 0.0191
0.0313 22.0 3476 0.1119
0.025 23.0 3634 0.0000
0.0328 24.0 3792 0.0559
0.0251 25.0 3950 0.0343
0.0334 26.0 4108 0.0263
0.0048 27.0 4266 0.0424
0.0584 28.0 4424 0.0578
0.0861 29.0 4582 0.0495
0.0345 30.0 4740 0.0282
0.0193 31.0 4898 0.0412
0.0216 32.0 5056 0.0537
0.0241 33.0 5214 0.0124
0.009 34.0 5372 0.0272
0.0309 35.0 5530 0.0000
0.0069 36.0 5688 0.0000
0.0398 37.0 5846 0.0000
0.0474 38.0 6004 0.0017
0.0263 39.0 6162 0.0094
0.0128 40.0 6320 0.0000
0.0101 41.0 6478 0.0015
0.0415 42.0 6636 0.0004
0.0204 43.0 6794 0.0307
0.0342 44.0 6952 0.0348
0.0103 45.0 7110 0.0344
0.0389 46.0 7268 0.0238
0.0108 47.0 7426 0.0651
0.0486 48.0 7584 0.0290
0.0131 49.0 7742 0.0130
0.0027 50.0 7900 0.1448
0.0119 51.0 8058 0.0034
0.0142 52.0 8216 0.0014
0.0123 53.0 8374 0.0292
0.0364 54.0 8532 0.0000
0.0541 55.0 8690 0.0000
0.0309 56.0 8848 0.0032
0.0278 57.0 9006 0.0124
0.0247 58.0 9164 0.0000
0.0077 59.0 9322 0.0000
0.0104 60.0 9480 0.0150
0.0302 61.0 9638 0.0003
0.0225 62.0 9796 0.0358
0.0296 63.0 9954 0.0000
0.0533 64.0 10112 0.1224
0.0121 65.0 10270 0.0092
0.0249 66.0 10428 0.0087
0.0637 67.0 10586 0.0003
0.035 68.0 10744 0.0000
0.0117 69.0 10902 0.0393
0.0049 70.0 11060 0.0209
0.0225 71.0 11218 0.0003
0.0121 72.0 11376 0.0001
0.0079 73.0 11534 0.0006
0.0354 74.0 11692 0.0458
0.0071 75.0 11850 0.0009
0.0043 76.0 12008 0.0290
0.0239 77.0 12166 0.0001
0.0159 78.0 12324 0.0162
0.0051 79.0 12482 0.0673
0.0028 80.0 12640 0.0275
0.0155 81.0 12798 0.0101
0.0068 82.0 12956 0.0271
0.006 83.0 13114 0.0120
0.0107 84.0 13272 0.0008
0.0122 85.0 13430 0.0124
0.0028 86.0 13588 0.0000
0.0256 87.0 13746 0.0011
0.037 88.0 13904 0.0213
0.0044 89.0 14062 0.0000
0.003 90.0 14220 0.0000
0.0072 91.0 14378 0.0381
0.0309 92.0 14536 0.0000
0.0123 93.0 14694 0.0117
0.0002 94.0 14852 0.0024
0.0033 95.0 15010 0.0002
0.0181 96.0 15168 0.0071
0.0167 97.0 15326 0.0219
0.0136 98.0 15484 0.0001
0.0116 99.0 15642 0.0189
0.0118 100.0 15800 0.0000
0.0345 101.0 15958 0.0006
0.0032 102.0 16116 0.0355
0.0003 103.0 16274 0.0484
0.0281 104.0 16432 0.0000
0.029 105.0 16590 0.0319
0.006 106.0 16748 0.0016
0.0001 107.0 16906 0.0608
0.024 108.0 17064 0.0000
0.0187 109.0 17222 0.0000
0.0038 110.0 17380 0.0000
0.0046 111.0 17538 0.0045
0.0024 112.0 17696 0.0000
0.033 113.0 17854 0.0001
0.0048 114.0 18012 0.0511
0.0005 115.0 18170 0.0000
0.0166 116.0 18328 0.0000
0.0113 117.0 18486 0.0122
0.0043 118.0 18644 0.0122
0.0703 119.0 18802 0.0013
0.0051 120.0 18960 0.0009
0.048 121.0 19118 0.0082
0.0408 122.0 19276 0.0137
0.018 123.0 19434 0.0365
0.0125 124.0 19592 0.0111
0.0075 125.0 19750 0.0179
0.0379 126.0 19908 0.0000
0.0029 127.0 20066 0.0181
0.007 128.0 20224 0.0610
0.0069 129.0 20382 0.0203
0.0082 130.0 20540 0.0000
0.0136 131.0 20698 0.1019
0.0152 132.0 20856 0.0000
0.0078 133.0 21014 0.0089
0.0003 134.0 21172 0.0000
0.0069 135.0 21330 0.0278
0.0089 136.0 21488 0.0330
0.0642 137.0 21646 0.0251
0.0077 138.0 21804 0.0015
0.0062 139.0 21962 0.0218
0.0203 140.0 22120 0.0032
0.0173 141.0 22278 0.0000
0.0149 142.0 22436 0.0231
0.011 143.0 22594 0.0218
0.0142 144.0 22752 0.0068
0.0075 145.0 22910 0.0067
0.0021 146.0 23068 0.0172
0.009 147.0 23226 0.0134
0.0072 148.0 23384 0.0050
0.0297 149.0 23542 0.0162
0.0189 150.0 23700 0.0000
0.0065 151.0 23858 0.0006
0.0423 152.0 24016 0.0102
0.0001 153.0 24174 0.0147
0.0607 154.0 24332 0.0062
0.0247 155.0 24490 0.0000
0.0187 156.0 24648 0.0000
0.025 157.0 24806 0.0278
0.0069 158.0 24964 0.0002
0.0024 159.0 25122 0.0002
0.0194 160.0 25280 0.0000
0.0072 161.0 25438 0.0000
0.0552 162.0 25596 0.0304
0.0114 163.0 25754 0.0184
0.001 164.0 25912 0.0000
0.0076 165.0 26070 0.0004
0.0158 166.0 26228 0.0000
0.0017 167.0 26386 0.0145
0.0009 168.0 26544 0.0436
0.0199 169.0 26702 0.0147
0.0067 170.0 26860 0.0003
0.0082 171.0 27018 0.0151
0.0079 172.0 27176 0.0018
0.0016 173.0 27334 0.0002
0.0095 174.0 27492 0.0152
0.0166 175.0 27650 0.0000
0.0018 176.0 27808 0.0495
0.0025 177.0 27966 0.0824
0.01 178.0 28124 0.0000
0.0054 179.0 28282 0.0072
0.0082 180.0 28440 0.0019
0.0 181.0 28598 0.0131
0.0034 182.0 28756 0.0163
0.0083 183.0 28914 0.0175
0.0035 184.0 29072 0.0111
0.009 185.0 29230 0.0004
0.015 186.0 29388 0.0000
0.0011 187.0 29546 0.0182
0.0142 188.0 29704 0.0581
0.0048 189.0 29862 0.0002
0.0021 190.0 30020 0.0002
0.0 191.0 30178 0.0334
0.016 192.0 30336 0.0003
0.0109 193.0 30494 0.0000
0.0085 194.0 30652 0.0028
0.002 195.0 30810 0.0000
0.002 196.0 30968 0.0144
0.0142 197.0 31126 0.0000
0.0727 198.0 31284 0.0163
0.0238 199.0 31442 0.0000
0.0171 200.0 31600 0.0141
0.0158 201.0 31758 0.0000
0.0049 202.0 31916 0.0086
0.0016 203.0 32074 0.0000
0.0088 204.0 32232 0.0007
0.0048 205.0 32390 0.0164
0.0103 206.0 32548 0.0080
0.0039 207.0 32706 0.0441
0.0041 208.0 32864 0.0000
0.027 209.0 33022 0.0004
0.0229 210.0 33180 0.0149
0.01 211.0 33338 0.0154
0.0036 212.0 33496 0.0191
0.0036 213.0 33654 0.0491
0.0316 214.0 33812 0.0000
0.0014 215.0 33970 0.0198
0.0097 216.0 34128 0.0000
0.0011 217.0 34286 0.0001
0.0063 218.0 34444 0.0010
0.0143 219.0 34602 0.0243
0.0022 220.0 34760 0.0275
0.002 221.0 34918 0.0000
0.0095 222.0 35076 0.0119
0.0016 223.0 35234 0.0001
0.0165 224.0 35392 0.0000
0.0106 225.0 35550 0.0279
0.0316 226.0 35708 0.0006
0.0041 227.0 35866 0.0000
0.0492 228.0 36024 0.0123
0.0034 229.0 36182 0.0196
0.0035 230.0 36340 0.0113
0.0197 231.0 36498 0.0000
0.0234 232.0 36656 0.0026
0.0102 233.0 36814 0.0250
0.0113 234.0 36972 0.0001
0.0008 235.0 37130 0.0000
0.0032 236.0 37288 0.0001
0.1045 237.0 37446 0.0000
0.0028 238.0 37604 0.0143
0.0028 239.0 37762 0.0143
0.0002 240.0 37920 0.0171
0.0302 241.0 38078 0.0406
0.0046 242.0 38236 0.0273
0.0124 243.0 38394 0.0754
0.0055 244.0 38552 0.0396
0.0102 245.0 38710 0.0003
0.0139 246.0 38868 0.0201
0.0367 247.0 39026 0.0343
0.0216 248.0 39184 0.0015
0.0091 249.0 39342 0.0637
0.0115 250.0 39500 0.0053
0.0254 251.0 39658 0.0299
0.0202 252.0 39816 0.0011
0.0005 253.0 39974 0.0181
0.0024 254.0 40132 0.0223
0.0002 255.0 40290 0.0001
0.0102 256.0 40448 0.0593
0.0133 257.0 40606 0.0000
0.0051 258.0 40764 0.0001
0.0027 259.0 40922 0.0225
0.0053 260.0 41080 0.0000
0.0073 261.0 41238 0.0108
0.0033 262.0 41396 0.0118
0.0024 263.0 41554 0.0233
0.02 264.0 41712 0.0236
0.0078 265.0 41870 0.0180
0.0117 266.0 42028 0.0000
0.0156 267.0 42186 0.0000
0.0041 268.0 42344 0.0009
0.0003 269.0 42502 0.0108
0.0049 270.0 42660 0.0000
0.0021 271.0 42818 0.0174
0.0083 272.0 42976 0.0201
0.0002 273.0 43134 0.0371
0.0219 274.0 43292 0.0000
0.0014 275.0 43450 0.0156
0.0102 276.0 43608 0.0002
0.0311 277.0 43766 0.0008
0.0115 278.0 43924 0.0269
0.0122 279.0 44082 0.0131
0.0021 280.0 44240 0.0000
0.0032 281.0 44398 0.0157
0.0079 282.0 44556 0.0146
0.0344 283.0 44714 0.0142
0.0086 284.0 44872 0.0000
0.0045 285.0 45030 0.0011
0.0077 286.0 45188 0.0117
0.0026 287.0 45346 0.0006
0.0042 288.0 45504 0.0069
0.0528 289.0 45662 0.0199
0.0162 290.0 45820 0.0090
0.0598 291.0 45978 0.0000
0.0293 292.0 46136 0.0142
0.0021 293.0 46294 0.0330
0.0073 294.0 46452 0.0029
0.0006 295.0 46610 0.0007
0.0065 296.0 46768 0.0005
0.0075 297.0 46926 0.0371
0.0017 298.0 47084 0.0001
0.0023 299.0 47242 0.0442
0.0059 300.0 47400 0.0000
0.0041 301.0 47558 0.0198
0.0372 302.0 47716 0.0123
0.0004 303.0 47874 0.0177
0.0008 304.0 48032 0.0172
0.0002 305.0 48190 0.0005
0.0064 306.0 48348 0.0000
0.0116 307.0 48506 0.0142
0.0113 308.0 48664 0.0054
0.0046 309.0 48822 0.0090
0.0026 310.0 48980 0.0118
0.0051 311.0 49138 0.0012
0.0027 312.0 49296 0.0099
0.0031 313.0 49454 0.0104
0.0031 314.0 49612 0.0098
0.0085 315.0 49770 0.0115
0.0023 316.0 49928 0.0000
0.0143 317.0 50086 0.0000
0.0003 318.0 50244 0.0090
0.0179 319.0 50402 0.0124
0.0012 320.0 50560 0.0010
0.009 321.0 50718 0.0219
0.0082 322.0 50876 0.0102
0.0084 323.0 51034 0.0016
0.0029 324.0 51192 0.0227
0.0023 325.0 51350 0.0148
0.0022 326.0 51508 0.0157
0.0065 327.0 51666 0.0132
0.0257 328.0 51824 0.0162
0.0013 329.0 51982 0.0005
0.0135 330.0 52140 0.0278
0.0029 331.0 52298 0.0391
0.0139 332.0 52456 0.0000
0.0152 333.0 52614 0.0005
0.0042 334.0 52772 0.0008
0.0161 335.0 52930 0.0005
0.0008 336.0 53088 0.0000
0.0098 337.0 53246 0.0149
0.0096 338.0 53404 0.0097
0.0158 339.0 53562 0.0121
0.0001 340.0 53720 0.0168
0.0068 341.0 53878 0.0006
0.0026 342.0 54036 0.0000
0.0061 343.0 54194 0.0126
0.0071 344.0 54352 0.0154
0.0032 345.0 54510 0.0135
0.0034 346.0 54668 0.0046
0.003 347.0 54826 0.0137
0.0028 348.0 54984 0.0012
0.0157 349.0 55142 0.0193
0.0046 350.0 55300 0.0006
0.0023 351.0 55458 0.0137
0.0 352.0 55616 0.0111
0.0041 353.0 55774 0.0004
0.0032 354.0 55932 0.0000
0.0103 355.0 56090 0.0004
0.0 356.0 56248 0.0123
0.0074 357.0 56406 0.0205
0.0098 358.0 56564 0.0009
0.004 359.0 56722 0.0005
0.0018 360.0 56880 0.0134
0.0032 361.0 57038 0.0000
0.0063 362.0 57196 0.0230
0.0015 363.0 57354 0.0087
0.0041 364.0 57512 0.0110
0.0021 365.0 57670 0.0075
0.0019 366.0 57828 0.0000
0.0021 367.0 57986 0.0109
0.0018 368.0 58144 0.0228
0.0027 369.0 58302 0.0183
0.0036 370.0 58460 0.0116
0.0042 371.0 58618 0.0015
0.0029 372.0 58776 0.0010
0.0206 373.0 58934 0.0000
0.0071 374.0 59092 0.0000
0.0004 375.0 59250 0.0151
0.0009 376.0 59408 0.0000
0.0037 377.0 59566 0.0341
0.0097 378.0 59724 0.0239
0.0032 379.0 59882 0.0147
0.0041 380.0 60040 0.0006
0.002 381.0 60198 0.0003
0.0063 382.0 60356 0.0
0.0054 383.0 60514 0.0093
0.0069 384.0 60672 0.0089
0.0056 385.0 60830 0.0006
0.0071 386.0 60988 0.0219
0.0061 387.0 61146 0.0503
0.0068 388.0 61304 0.0028
0.002 389.0 61462 0.0011
0.0024 390.0 61620 0.0075
0.0028 391.0 61778 0.0009
0.0045 392.0 61936 0.0015
0.0009 393.0 62094 0.0350
0.0064 394.0 62252 0.0103
0.0003 395.0 62410 0.0118
0.0112 396.0 62568 0.0000
0.0046 397.0 62726 0.0118
0.0064 398.0 62884 0.0482
0.0021 399.0 63042 0.0228
0.0083 400.0 63200 0.0009
0.0008 401.0 63358 0.0005
0.0061 402.0 63516 0.0000
0.0038 403.0 63674 0.0031
0.0022 404.0 63832 0.0198
0.0074 405.0 63990 0.0000
0.0024 406.0 64148 0.0
0.0017 407.0 64306 0.0140
0.0004 408.0 64464 0.0210
0.003 409.0 64622 0.0096
0.0013 410.0 64780 0.0
0.0166 411.0 64938 0.0285
0.0029 412.0 65096 0.0000
0.0133 413.0 65254 0.0000
0.014 414.0 65412 0.0008
0.0044 415.0 65570 0.0163
0.0077 416.0 65728 0.0000
0.0002 417.0 65886 0.0004
0.0024 418.0 66044 0.0375
0.0092 419.0 66202 0.0008
0.0059 420.0 66360 0.0139
0.0005 421.0 66518 0.0121
0.0128 422.0 66676 0.0105
0.0035 423.0 66834 0.0005
0.0042 424.0 66992 0.0
0.0051 425.0 67150 0.0311
0.0089 426.0 67308 0.0084
0.0112 427.0 67466 0.0229
0.0002 428.0 67624 0.0156
0.0049 429.0 67782 0.0117
0.0051 430.0 67940 0.0150
0.0088 431.0 68098 0.0005
0.01 432.0 68256 0.0136
0.0032 433.0 68414 0.0
0.0003 434.0 68572 0.0155
0.0032 435.0 68730 0.0119
0.0041 436.0 68888 0.0003
0.0 437.0 69046 0.0168
0.0283 438.0 69204 0.0131
0.0032 439.0 69362 0.0000
0.007 440.0 69520 0.0000
0.0079 441.0 69678 0.0251
0.0121 442.0 69836 0.0003
0.0067 443.0 69994 0.0005
0.0017 444.0 70152 0.0000
0.0027 445.0 70310 0.0138
0.0013 446.0 70468 0.0322
0.0086 447.0 70626 0.0000
0.0066 448.0 70784 0.0112
0.003 449.0 70942 0.0000
0.0006 450.0 71100 0.0097
0.01 451.0 71258 0.0113
0.0003 452.0 71416 0.0098
0.0027 453.0 71574 0.0009
0.0034 454.0 71732 0.0119
0.0041 455.0 71890 0.0012
0.01 456.0 72048 0.0222
0.0032 457.0 72206 0.0111
0.0027 458.0 72364 0.0000
0.0064 459.0 72522 0.0149
0.0034 460.0 72680 0.0097
0.0083 461.0 72838 0.0146
0.002 462.0 72996 0.0237
0.004 463.0 73154 0.0135
0.0028 464.0 73312 0.0056
0.0008 465.0 73470 0.0000
0.0015 466.0 73628 0.0004
0.0023 467.0 73786 0.0320
0.008 468.0 73944 0.0108
0.0052 469.0 74102 0.0283
0.0088 470.0 74260 0.0009
0.0037 471.0 74418 0.0251
0.006 472.0 74576 0.0208
0.0008 473.0 74734 0.0237
0.005 474.0 74892 0.0198
0.0033 475.0 75050 0.0000
0.0014 476.0 75208 0.0000
0.0028 477.0 75366 0.0004
0.0082 478.0 75524 0.0149
0.0067 479.0 75682 0.0124
0.0062 480.0 75840 0.0327
0.0035 481.0 75998 0.0273
0.0013 482.0 76156 0.0007
0.0023 483.0 76314 0.0112
0.0042 484.0 76472 0.0101
0.0001 485.0 76630 0.0004
0.0 486.0 76788 0.0000
0.0069 487.0 76946 0.0278
0.0002 488.0 77104 0.0000
0.0045 489.0 77262 0.0163
0.0005 490.0 77420 0.0000
0.0005 491.0 77578 0.0272
0.0014 492.0 77736 0.0000
0.0088 493.0 77894 0.0215
0.0033 494.0 78052 0.0009
0.0031 495.0 78210 0.0261
0.0038 496.0 78368 0.0120
0.0091 497.0 78526 0.0151
0.0062 498.0 78684 0.0009
0.004 499.0 78842 0.0005
0.0025 500.0 79000 0.0134

Framework versions

  • Transformers 4.31.0.dev0
  • Pytorch 2.0.1+cu118
  • Datasets 2.13.1
  • Tokenizers 0.11.0
Downloads last month
11
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for ankush-003/fine-tuned-roberta2-nosql-injection

Finetuned
(1309)
this model