metadata
license: apache-2.0
base_model: bert-base-uncased
tags:
- generated_from_keras_callback
model-index:
- name: bert-base-uncased-finetuned-glue-sst2
results: []
bert-base-uncased-finetuned-glue-sst2
Use for sentiment analysis.
This model is a fine-tuned version of bert-base-uncased on the glue sst2 dataset. The model achieves 91.39%
accuracy on the validation dataset.
Model description
bert-base-uncased is a pretrained English language model. bert-base-uncased-finetuned-glue-sst2
adds a 2-class classification head for predicting positive
and negative
sentiment.
Training and evaluation data
The model has been trained on 10K training samples, even though the glue sst2 dataset contains 67.3K samples. This was done to decrease training time.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 5e-05, 'decay_steps': 3750, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
- training_precision: float32
Training results
- Accuracy (training):
94.33%
- Accuracy (validation):
91.39%
Framework versions
- Transformers 4.35.2
- TensorFlow 2.15.0
- Datasets 2.16.1
- Tokenizers 0.15.0