tunarebus/distilbert-base-uncased-finetuned-tweetdinastipolitik

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 2.3297
• Validation Loss: 2.3462
• Epoch: 34

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'module': 'transformers.optimization_tf', 'class_name': 'WarmUp', 'config': {'initial_learning_rate': 2e-05, 'decay_schedule_fn': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': -958, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'warmup_steps': 1000, 'power': 1.0, 'name': None}, 'registered_name': 'WarmUp'}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: mixed_float16

Training results

Train Loss	Validation Loss	Epoch
4.9633	4.7385	0
4.7409	4.4192	1
4.5418	4.2812	2
4.3390	4.1513	3
4.1633	3.9455	4
3.9945	3.7719	5
3.8723	3.6903	6
3.6898	3.5449	7
3.5524	3.3732	8
3.3717	3.1554	9
3.2974	3.1077	10
3.1591	2.9884	11
3.0308	2.9435	12
2.9841	2.8299	13
2.8901	2.7394	14
2.7851	2.6936	15
2.7536	2.6745	16
2.6951	2.5683	17
2.6414	2.5420	18
2.5789	2.5220	19
2.4913	2.4737	20
2.4694	2.4265	21
2.3930	2.3917	22
2.4195	2.4069	23
2.3199	2.3916	24
2.3211	2.3084	25
2.3548	2.3648	26
2.3584	2.4071	27
2.3544	2.3168	28
2.3077	2.4003	29
2.3576	2.3952	30
2.3485	2.3161	31
2.3258	2.3230	32
2.3272	2.3355	33
2.3297	2.3462	34

Framework versions

• Transformers 4.35.2
• TensorFlow 2.15.0
• Datasets 2.16.1
• Tokenizers 0.15.0