tunarebus/distilbert-base-uncased-finetuned-tweet_pemilu2024

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 2.5045
• Validation Loss: 2.4250
• Epoch: 24

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'module': 'transformers.optimization_tf', 'class_name': 'WarmUp', 'config': {'initial_learning_rate': 2e-05, 'decay_schedule_fn': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': -931, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'warmup_steps': 1000, 'power': 1.0, 'name': None}, 'registered_name': 'WarmUp'}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: mixed_float16

Training results

Train Loss	Validation Loss	Epoch
4.9932	4.6882	0
4.6678	4.3954	1
4.4076	4.1306	2
4.1610	3.8876	3
3.9036	3.6392	4
3.6549	3.4398	5
3.4439	3.2595	6
3.3055	3.0760	7
3.1307	2.8918	8
2.9749	2.8005	9
2.8441	2.7331	10
2.7790	2.6070	11
2.6932	2.5804	12
2.6181	2.4717	13
2.5241	2.4091	14
2.4900	2.4074	15
2.5065	2.4340	16
2.4976	2.4425	17
2.4894	2.4307	18
2.5294	2.4356	19
2.4649	2.4002	20
2.5011	2.4172	21
2.5083	2.4533	22
2.4822	2.4224	23
2.5045	2.4250	24

Framework versions

• Transformers 4.35.2
• TensorFlow 2.15.0
• Datasets 2.15.0
• Tokenizers 0.15.0