bert-base-uncased-finetuned-glue-sst2
Use for sentiment analysis.
This model is a fine-tuned version of bert-base-uncased on the glue sst2 dataset. The model achieves 91.39%
accuracy on the validation dataset.
Model description
bert-base-uncased is a pretrained English language model. bert-base-uncased-finetuned-glue-sst2
adds a 2-class classification head for predicting positive
and negative
sentiment.
Training and evaluation data
The model has been trained on 10K training samples, even though the glue sst2 dataset contains 67.3K samples. This was done to decrease training time.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 5e-05, 'decay_steps': 3750, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
- training_precision: float32
Training results
- Accuracy (training):
94.33%
- Accuracy (validation):
91.39%
Framework versions
- Transformers 4.35.2
- TensorFlow 2.15.0
- Datasets 2.16.1
- Tokenizers 0.15.0
- Downloads last month
- 2