UDA-LIDI/barto_emergency_multi_purpose

barto_prompts

This model is a fine-tuned version of vgaraujov/bart-base-spanish. It achieves the following results on the evaluation set:

• Loss: 0.5242
• Rouge1: 77.7794
• Rouge2: 62.5213
• Rougel: 77.3853
• Rougelsum: 77.2245
• Gen Len: 11.6686

Model description

This checkpoint uses BARTO as base model and different prefix to achieve different tasks in emergency transcribed calls:

• "Resume la emergencia: ": For text summarization
• "Extrae las palabras clave de la emergencia: ": For keyword extraction
• "La palabra que mejor representa la emergencia es: ": Gives a word that represents the text
• "Clasifica la emergencia en [CLAVE ROJA, CLAVE NARANJA, CLAVE AMARILLA, CLAVE VERDE]: ": For text classification

Intended uses & limitations

Under privacy agreement.

Training and evaluation data

Training data used has been provided by the ECU 911 service under a strict confidentiality agreement.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• num_devices: 3
• total_train_batch_size: 48
• total_eval_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.3631	1.0	92	0.6643	66.5661	49.8557	66.156	66.0723	10.7803
0.607	2.0	184	0.5528	72.4516	55.3295	72.0424	71.9591	10.8390
0.4994	3.0	276	0.5330	74.2798	56.9793	73.6683	73.6271	10.9072
0.4215	4.0	368	0.5246	75.5697	58.5086	75.1434	75.0331	11.5663
0.3744	5.0	460	0.5302	75.9054	60.4386	75.4245	75.294	11.6496
0.3392	6.0	552	0.5238	76.8758	61.7901	76.4882	76.444	11.7254
0.3014	7.0	644	0.5302	76.8835	61.9104	76.4603	76.3661	11.6117
0.2807	8.0	736	0.5239	77.4479	62.0839	77.0472	76.8683	11.5417
0.265	9.0	828	0.5210	77.5274	62.249	77.1446	76.9984	11.5890
0.2594	10.0	920	0.5242	77.7794	62.5213	77.3853	77.2245	11.6686

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0