bert-salvadoran-news

This model is a fine-tuned version of distilbert-base-multilingual-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 1.4210
• Validation Loss: 1.5456
• Epoch: 19

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
2.1106	1.9331	0
1.9666	1.7924	1
1.8643	1.7777	2
1.8090	1.7018	3
1.7668	1.7372	4
1.7337	1.6726	5
1.6948	1.6624	6
1.6584	1.6863	7
1.6232	1.6174	8
1.5996	1.6399	9
1.5805	1.5340	10
1.5622	1.6202	11
1.5124	1.6176	12
1.5150	1.5644	13
1.5151	1.6002	14
1.4730	1.5746	15
1.4538	1.5536	16
1.4507	1.5537	17
1.4476	1.5450	18
1.4210	1.5456	19

Framework versions

• Transformers 4.24.0
• TensorFlow 2.9.2
• Datasets 2.7.1
• Tokenizers 0.13.2