metadata
tags:
- generated_from_keras_callback
model-index:
- name: US_politicians_covid_skepticism
results: []
This model is a fine-tuned version of vinai/bertweet-covid19-base-uncased on a dataset of 10k tweets about COVID-19 policies from US legislators in the House and Senate.
The model is intended to identify skepticism of COVID-19 policies (i.e. masks, social distancing, lockdowns, vaccines etc.).
It's a pretty simple task but I used a grid search to optimize hyperparameters. The model uses the following hyperparamters:
Optimized Hyperparameters
- The best learning rate is: 9.928559980965476e-06
- The best weight decay is: 0.003083325125091835
- The best epoch is : 5
- The best train split is : 0.2864649363822965
Training
- Train Loss: 0.1007
- Train Sparse Categorical Accuracy: 0.9591
- Validation Loss: 0.0913
- Validation Sparse Categorical Accuracy: 0.9627
- Optimizer: Adam
- Starting Learn rate: 5e-07