text-to-sparql-t5-base-2024-10-01_04-05

This model is a fine-tuned version of t5-base on the arrow dataset. It achieves the following results on the evaluation set:

• Loss: 0.1612
• Gen Len: 19.0
• Bertscorer-p: 0.6042
• Bertscorer-r: 0.1007
• Bertscorer-f1: 0.3406
• Sacrebleu-score: 6.3972
• Sacrebleu-precisions: [93.50202971813725, 87.89528553225993, 83.9093099978942, 81.08246812206387]
• Bleu-bp: 0.0740

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Gen Len	Bertscorer-p	Bertscorer-r	Bertscorer-f1	Sacrebleu-score	Sacrebleu-precisions	Bleu-bp
0.1434	1.0	4772	0.1290	19.0	0.5779	0.0743	0.3142	5.8962	[92.35991566894258, 84.39366674829903, 78.94400227401933, 75.86961452759951]	0.0713
0.0942	2.0	9544	0.1177	19.0	0.5888	0.0849	0.3250	6.1087	[92.5606800784706, 85.52426907082315, 80.69350019995765, 77.57006871168893]	0.0728
0.0653	3.0	14316	0.1173	19.0	0.6046	0.1056	0.3434	6.3214	[93.2540100046867, 86.96274167420529, 82.274102896671, 78.77417998317914]	0.0742
0.0483	4.0	19088	0.1232	19.0	0.5986	0.0961	0.3355	6.2622	[93.15494173500215, 86.84532601814729, 82.2615628114192, 79.1214879303522]	0.0735
0.0334	5.0	23860	0.1311	19.0	0.6023	0.0994	0.3390	6.3073	[93.43068494727854, 87.49234763885077, 83.1708833292281, 80.1232645304334]	0.0734
0.0235	6.0	28632	0.1357	19.0	0.6001	0.0980	0.3372	6.3131	[93.21137315406656, 87.16716210233382, 82.85332802379921, 79.83819964161484]	0.0737
0.0168	7.0	33404	0.1473	19.0	0.6041	0.1033	0.3419	6.4057	[93.29664975783108, 87.43513246633191, 83.24213326488467, 80.18603064651553]	0.0746
0.0119	8.0	38176	0.1505	19.0	0.6012	0.0990	0.3382	6.3570	[93.1113662456946, 87.19629610143632, 83.0426651081239, 80.06573325445343]	0.0742
0.0088	9.0	42948	0.1542	19.0	0.6055	0.1041	0.3430	6.4203	[93.41891452713682, 87.77185624336455, 83.69605828507379, 80.74261780654649]	0.0744
0.0071	10.0	47720	0.1612	19.0	0.6042	0.1007	0.3406	6.3972	[93.50202971813725, 87.89528553225993, 83.9093099978942, 81.08246812206387]	0.0740

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0
• Datasets 3.0.0
• Tokenizers 0.19.1