finetuned__bert-base-spanish-wwm-cased__augmented-ultrasounds

This model is a fine-tuned version of dccuchile/bert-base-spanish-wwm-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5757

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 256
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.9979	238	1.0145
1.2754	2.0	477	0.8811
1.2754	2.9979	715	0.7855
0.8342	4.0	954	0.7440
0.8342	4.9979	1192	0.7016
0.7436	6.0	1431	0.6743
0.7436	6.9979	1669	0.6591
0.6965	8.0	1908	0.6459
0.6965	8.9979	2146	0.6354
0.6619	10.0	2385	0.6243
0.6619	10.9979	2623	0.6177
0.6386	12.0	2862	0.6086
0.6386	12.9979	3100	0.6027
0.623	14.0	3339	0.5828
0.623	14.9979	3577	0.5869
0.6103	16.0	3816	0.5951
0.6103	16.9979	4054	0.5889
0.6032	18.0	4293	0.5870
0.6032	18.9979	4531	0.5818
0.5967	19.9581	4760	0.5757

Framework versions

• Transformers 4.40.0
• Pytorch 2.2.1+cu121
• Datasets 2.19.0
• Tokenizers 0.19.1