--- language: - en license: mit tags: - generated_from_trainer datasets: - glue metrics: - accuracy model-index: - name: roberta-base-sst2 results: - task: name: Text Classification type: text-classification dataset: name: GLUE SST2 type: glue args: sst2 metrics: - name: Accuracy type: accuracy value: 0.9323394495412844 --- # roberta-base-sst2 This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on the GLUE SST2 dataset. It achieves the following results on the evaluation set: - Loss: 0.1952 - Accuracy: 0.9323 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 16 - eval_batch_size: 8 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_ratio: 0.06 - num_epochs: 10.0 ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | |:-------------:|:-----:|:-----:|:---------------:|:--------:| | 0.575 | 0.12 | 500 | 0.2665 | 0.9071 | | 0.2989 | 0.24 | 1000 | 0.2088 | 0.9220 | | 0.2725 | 0.36 | 1500 | 0.2560 | 0.9243 | | 0.2814 | 0.48 | 2000 | 0.2016 | 0.9266 | | 0.2586 | 0.59 | 2500 | 0.2293 | 0.9174 | | 0.2536 | 0.71 | 3000 | 0.2340 | 0.9323 | | 0.2494 | 0.83 | 3500 | 0.1952 | 0.9323 | | 0.2396 | 0.95 | 4000 | 0.2494 | 0.9323 | | 0.2123 | 1.07 | 4500 | 0.2187 | 0.9381 | | 0.2042 | 1.19 | 5000 | 0.2812 | 0.9151 | | 0.2083 | 1.31 | 5500 | 0.2739 | 0.9346 | | 0.2041 | 1.43 | 6000 | 0.2087 | 0.9381 | | 0.1969 | 1.54 | 6500 | 0.2590 | 0.9255 | | 0.1982 | 1.66 | 7000 | 0.2445 | 0.9300 | | 0.1943 | 1.78 | 7500 | 0.2798 | 0.9266 | | 0.1848 | 1.9 | 8000 | 0.2844 | 0.9312 | | 0.1788 | 2.02 | 8500 | 0.2998 | 0.9255 | | 0.1623 | 2.14 | 9000 | 0.2696 | 0.9392 | | 0.1499 | 2.26 | 9500 | 0.2533 | 0.9278 | | 0.1426 | 2.38 | 10000 | 0.2971 | 0.9300 | | 0.1479 | 2.49 | 10500 | 0.2596 | 0.9358 | | 0.1405 | 2.61 | 11000 | 0.2945 | 0.9255 | | 0.1577 | 2.73 | 11500 | 0.4061 | 0.9002 | | 0.1521 | 2.85 | 12000 | 0.2724 | 0.9335 | | 0.1426 | 2.97 | 12500 | 0.2712 | 0.9427 | | 0.1206 | 3.09 | 13000 | 0.2954 | 0.9358 | | 0.1074 | 3.21 | 13500 | 0.2653 | 0.9392 | | 0.112 | 3.33 | 14000 | 0.2778 | 0.9346 | | 0.1147 | 3.44 | 14500 | 0.3705 | 0.9312 | | 0.1196 | 3.56 | 15000 | 0.2890 | 0.9346 | | 0.1159 | 3.68 | 15500 | 0.3449 | 0.9266 | | 0.119 | 3.8 | 16000 | 0.3207 | 0.9335 | | 0.1268 | 3.92 | 16500 | 0.3235 | 0.9312 | | 0.1074 | 4.04 | 17000 | 0.3650 | 0.9335 | | 0.0805 | 4.16 | 17500 | 0.3338 | 0.9381 | | 0.0838 | 4.28 | 18000 | 0.4302 | 0.9209 | | 0.0848 | 4.39 | 18500 | 0.4096 | 0.9323 | | 0.0922 | 4.51 | 19000 | 0.3332 | 0.9369 | | 0.091 | 4.63 | 19500 | 0.3024 | 0.9438 | | 0.0977 | 4.75 | 20000 | 0.2674 | 0.9495 | | 0.0897 | 4.87 | 20500 | 0.3993 | 0.9300 | | 0.1013 | 4.99 | 21000 | 0.3227 | 0.9289 | | 0.0671 | 5.11 | 21500 | 0.3374 | 0.9427 | | 0.0671 | 5.23 | 22000 | 0.4108 | 0.9278 | | 0.0652 | 5.34 | 22500 | 0.3550 | 0.9381 | | 0.0664 | 5.46 | 23000 | 0.3398 | 0.9358 | | 0.0742 | 5.58 | 23500 | 0.3286 | 0.9381 | | 0.0758 | 5.7 | 24000 | 0.3276 | 0.9312 | | 0.075 | 5.82 | 24500 | 0.3202 | 0.9369 | | 0.0686 | 5.94 | 25000 | 0.3481 | 0.9415 | | 0.0729 | 6.06 | 25500 | 0.3816 | 0.9335 | | 0.0568 | 6.18 | 26000 | 0.3132 | 0.9381 | | 0.0529 | 6.29 | 26500 | 0.3757 | 0.9300 | | 0.0506 | 6.41 | 27000 | 0.3396 | 0.9381 | | 0.0476 | 6.53 | 27500 | 0.3642 | 0.9404 | | 0.0555 | 6.65 | 28000 | 0.3430 | 0.9404 | | 0.0574 | 6.77 | 28500 | 0.3401 | 0.9392 | | 0.0524 | 6.89 | 29000 | 0.3378 | 0.9346 | | 0.0492 | 7.01 | 29500 | 0.3833 | 0.9381 | | 0.039 | 7.13 | 30000 | 0.3347 | 0.9346 | | 0.0411 | 7.24 | 30500 | 0.4404 | 0.9335 | | 0.0412 | 7.36 | 31000 | 0.3618 | 0.9381 | | 0.0477 | 7.48 | 31500 | 0.3806 | 0.9381 | | 0.0435 | 7.6 | 32000 | 0.3912 | 0.9335 | | 0.0443 | 7.72 | 32500 | 0.3900 | 0.9392 | | 0.0421 | 7.84 | 33000 | 0.4152 | 0.9369 | | 0.0495 | 7.96 | 33500 | 0.3832 | 0.9289 | | 0.0293 | 8.08 | 34000 | 0.4427 | 0.9346 | | 0.0253 | 8.19 | 34500 | 0.4425 | 0.9381 | | 0.0407 | 8.31 | 35000 | 0.4102 | 0.9358 | | 0.0311 | 8.43 | 35500 | 0.4447 | 0.9369 | | 0.0291 | 8.55 | 36000 | 0.4612 | 0.9346 | | 0.035 | 8.67 | 36500 | 0.4241 | 0.9346 | | 0.0381 | 8.79 | 37000 | 0.4198 | 0.9312 | | 0.0234 | 8.91 | 37500 | 0.4345 | 0.9369 | | 0.0311 | 9.03 | 38000 | 0.4558 | 0.9312 | | 0.028 | 9.14 | 38500 | 0.4245 | 0.9381 | | 0.0213 | 9.26 | 39000 | 0.4462 | 0.9381 | | 0.0276 | 9.38 | 39500 | 0.4210 | 0.9381 | | 0.0183 | 9.5 | 40000 | 0.4310 | 0.9404 | | 0.0184 | 9.62 | 40500 | 0.4437 | 0.9404 | | 0.0296 | 9.74 | 41000 | 0.4311 | 0.9392 | | 0.019 | 9.86 | 41500 | 0.4244 | 0.9415 | | 0.0245 | 9.98 | 42000 | 0.4270 | 0.9415 | ### Framework versions - Transformers 4.21.3 - Pytorch 1.7.1 - Datasets 1.18.3 - Tokenizers 0.11.6