z-dickson/US_politicians_covid_skepticism

This model is a fine-tuned version of vinai/bertweet-covid19-base-uncased on a dataset of 10k tweets about COVID-19 policies from US legislators in the House and Senate.

The model is intended to identify skepticism of COVID-19 policies (i.e. masks, social distancing, lockdowns, vaccines etc.).

It's a pretty simple task but I used a grid search to optimize hyperparameters. The model uses the following hyperparamters:

Optimized Hyperparameters

• The best learning rate is: 9.928559980965476e-06
• The best weight decay is: 0.003083325125091835
• The best epoch is : 5
• The best train split is : 0.2864649363822965

Training

• Train Loss: 0.1007
• Train Sparse Categorical Accuracy: 0.9591
• Validation Loss: 0.0913
• Validation Sparse Categorical Accuracy: 0.9627
• Optimizer: Adam
• Starting Learn rate: 5e-07