text-to-sparql-t5-small-qald9

This model is a fine-tuned version of yazdipour/text-to-sparql-t5-small-qald9 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0298
• Gen Len: 19.0
• P: 0.6865
• R: 0.0002
• F1: 0.3220
• Bleu-score: 6.6739
• Bleu-precisions: [94.676960585128, 94.0891045434495, 93.39122045344911, 92.54922831293241]
• Bleu-bp: 0.0712

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Gen Len	P	R	F1	Bleu-score	Bleu-precisions	Bleu-bp
No log	1.0	25	0.6491	19.0	0.6071	-0.0436	0.2622	6.3628	[87.18847352024922, 77.80117944397641, 73.71559633027523, 69.33534743202416]	0.0829
No log	2.0	50	0.2150	19.0	0.6560	-0.0205	0.2968	6.5917	[91.88755020080322, 88.9808362369338, 87.72597526165556, 86.21593291404612]	0.0743
No log	3.0	75	0.1237	19.0	0.6573	-0.0200	0.2976	6.5917	[91.88755020080322, 88.9808362369338, 87.72597526165556, 86.21593291404612]	0.0743
No log	4.0	100	0.0818	19.0	0.6534	-0.0173	0.2974	6.6884	[91.3738019169329, 88.48484848484848, 87.24007561436673, 85.74401664932363]	0.0758
No log	5.0	125	0.0670	19.0	0.6789	-0.0100	0.3129	6.5857	[93.91727493917276, 92.47359154929578, 91.14533205004813, 89.54352441613588]	0.0718
No log	6.0	150	0.0569	19.0	0.6819	-0.0045	0.3173	6.6856	[93.8989898989899, 92.7224901359053, 91.37517968375658, 89.75171685155837]	0.0727
No log	7.0	175	0.0505	19.0	0.6830	-0.0040	0.3181	6.6731	[94.09146094698502, 93.0171277997365, 91.79068650984158, 90.31233456855479]	0.0723
No log	8.0	200	0.0459	19.0	0.6838	-0.0045	0.3182	6.6530	[94.20583468395462, 93.13984168865436, 91.92307692307692, 90.45599151643691]	0.0720
No log	9.0	225	0.0422	19.0	0.6796	0.0003	0.3190	6.7182	[93.93694421988683, 93.0701754385965, 92.09012464046022, 90.96194503171247]	0.0726
No log	10.0	250	0.0393	19.0	0.6826	-0.0030	0.3184	6.6807	[94.05099150141643, 93.06104523495827, 91.93470955352856, 90.57702488088935]	0.0723
No log	11.0	275	0.0385	19.0	0.6822	-0.0029	0.3183	6.6874	[94.01294498381877, 93.02019315188763, 91.89059500959694, 90.52910052910053]	0.0724
No log	12.0	300	0.0355	19.0	0.6851	-0.0007	0.3209	6.6882	[94.44444444444444, 93.70598591549296, 92.82964388835418, 91.7728237791932]	0.0718
No log	13.0	325	0.0343	19.0	0.6854	-0.0005	0.3211	6.6868	[94.48275862068965, 93.79128137384411, 92.97063071738084, 91.98088157195964]	0.0717
No log	14.0	350	0.0327	19.0	0.6859	0.0000	0.3216	6.6888	[94.56168831168831, 93.92070484581498, 93.15992292870905, 92.2422954303932]	0.0716
No log	15.0	375	0.0321	19.0	0.6845	-0.0012	0.3203	6.7110	[94.25333872925941, 93.41238471673255, 92.41478636581853, 91.21228163049233]	0.0723
No log	16.0	400	0.0309	19.0	0.6873	0.0015	0.3231	6.6845	[94.79674796747967, 94.3071491615181, 93.72586872586872, 93.02449414270501]	0.0711
No log	17.0	425	0.0307	19.0	0.6873	0.0006	0.3226	6.6690	[94.7946319642131, 94.26048565121413, 93.62626750362143, 92.86094832179009]	0.0710
No log	18.0	450	0.0302	19.0	0.6868	-0.0000	0.3220	6.6671	[94.71544715447155, 94.1306266548985, 93.43629343629344, 92.59850905218318]	0.0711
No log	19.0	475	0.0299	19.0	0.6862	-0.0000	0.3217	6.6820	[94.60008120178644, 93.96209784045836, 93.20481927710843, 92.29133439659755]	0.0715
0.2513	20.0	500	0.0298	19.0	0.6865	0.0002	0.3220	6.6739	[94.676960585128, 94.0891045434495, 93.39122045344911, 92.54922831293241]	0.0712

Framework versions

• Transformers 4.39.3
• Pytorch 2.1.2
• Datasets 2.18.0
• Tokenizers 0.15.2