Hugging Face's logo

chainyo
/
t5-base-sede-txt2sql

Text2Text Generation Transformers PyTorch TensorBoard
t5 generated_from_trainer Inference Endpoints text-generation-inference
Model card Files Metrics Community
2
Edit model card

t5-base-sede-txt2sql

This model is a fine-tuned version of google/t5-v1_1-base on the sede dataset. It achieves the following results on the evaluation set:

  • Loss: 1.1577
  • Bleu Score: 0.5923
  • Parsable Queries Accuracy: 0.0
  • Partial Match F1: 0.0
  • Partial Match F1 No Values: 0.0
  • Partial Match Em: 0.0
  • Partial Match No Values Em: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 100

Training results

Training Loss Epoch Step Validation Loss Bleu Score Parsable Queries Accuracy Partial Match F1 Partial Match F1 No Values Partial Match Em Partial Match No Values Em
No log 1.0 95 13.2410 0.0069 0.0 0.0 0.0 0.0 0.0
No log 2.0 190 7.6317 0.0134 0.0 0.0 0.0 0.0 0.0
No log 3.0 285 6.0919 0.0058 0.0 0.0 0.0 0.0 0.0
No log 4.0 380 5.4922 0.0021 0.0 0.0 0.0 0.0 0.0
No log 5.0 475 4.7151 0.0009 0.0 0.0 0.0 0.0 0.0
12.0698 6.0 570 4.1412 0.0003 0.0 0.0 0.0 0.0 0.0
12.0698 7.0 665 3.6398 0.0003 0.0 0.0 0.0 0.0 0.0
12.0698 8.0 760 3.2643 0.0009 0.0 0.0 0.0 0.0 0.0
12.0698 9.0 855 3.0544 0.0013 0.0 0.0 0.0 0.0 0.0
12.0698 10.0 950 2.8015 0.0043 0.0 0.0 0.0 0.0 0.0
4.696 11.0 1045 2.5552 0.0789 0.0 0.0 0.0 0.0 0.0
4.696 12.0 1140 2.3535 0.1036 0.0 0.0 0.0 0.0 0.0
4.696 13.0 1235 2.2132 0.0050 0.0 0.0 0.0 0.0 0.0
4.696 14.0 1330 2.1084 0.1333 0.0 0.0 0.0 0.0 0.0
4.696 15.0 1425 2.0117 0.2972 0.0 0.0 0.0 0.0 0.0
3.1348 16.0 1520 1.9333 0.2481 0.0 0.0 0.0 0.0 0.0
3.1348 17.0 1615 1.8395 0.4149 0.0 0.0 0.0 0.0 0.0
3.1348 18.0 1710 1.7661 0.5439 0.0 0.0 0.0 0.0 0.0
3.1348 19.0 1805 1.7101 0.6001 0.0 0.0 0.0 0.0 0.0
3.1348 20.0 1900 1.6562 0.6219 0.0 0.0 0.0 0.0 0.0
3.1348 21.0 1995 1.6073 0.5865 0.0 0.0 0.0 0.0 0.0
2.4276 22.0 2090 1.5773 0.5683 0.0 0.0 0.0 0.0 0.0
2.4276 23.0 2185 1.5478 0.5408 0.0 0.0 0.0 0.0 0.0
2.4276 24.0 2280 1.5190 0.5749 0.0 0.0 0.0 0.0 0.0
2.4276 25.0 2375 1.4927 0.5818 0.0 0.0 0.0 0.0 0.0
2.4276 26.0 2470 1.4671 0.5673 0.0 0.0 0.0 0.0 0.0
2.076 27.0 2565 1.4499 0.5616 0.0 0.0 0.0 0.0 0.0
2.076 28.0 2660 1.4275 0.6041 0.0 0.0 0.0 0.0 0.0
2.076 29.0 2755 1.4096 0.5764 0.0 0.0 0.0 0.0 0.0
2.076 30.0 2850 1.3983 0.5862 0.0 0.0 0.0 0.0 0.0
2.076 31.0 2945 1.3812 0.5982 0.0 0.0 0.0 0.0 0.0
1.8828 32.0 3040 1.3679 0.5927 0.0 0.0 0.0 0.0 0.0
1.8828 33.0 3135 1.3548 0.5916 0.0 0.0 0.0 0.0 0.0
1.8828 34.0 3230 1.3461 0.5769 0.0 0.0 0.0 0.0 0.0
1.8828 35.0 3325 1.3353 0.5871 0.0 0.0 0.0 0.0 0.0
1.8828 36.0 3420 1.3293 0.5687 0.0 0.0 0.0 0.0 0.0
1.7602 37.0 3515 1.3195 0.5689 0.0 0.0 0.0 0.0 0.0
1.7602 38.0 3610 1.3109 0.5949 0.0 0.0 0.0 0.0 0.0
1.7602 39.0 3705 1.3049 0.5619 0.0 0.0 0.0 0.0 0.0
1.7602 40.0 3800 1.2953 0.5872 0.0 0.0 0.0 0.0 0.0
1.7602 41.0 3895 1.2907 0.6014 0.0 0.0 0.0 0.0 0.0
1.7602 42.0 3990 1.2831 0.5917 0.0 0.0 0.0 0.0 0.0
1.6652 43.0 4085 1.2757 0.5718 0.0 0.0 0.0 0.0 0.0
1.6652 44.0 4180 1.2692 0.5707 0.0 0.0 0.0 0.0 0.0
1.6652 45.0 4275 1.2642 0.5758 0.0 0.0 0.0 0.0 0.0
1.6652 46.0 4370 1.2619 0.6012 0.0 0.0 0.0 0.0 0.0
1.6652 47.0 4465 1.2527 0.5749 0.0 0.0 0.0 0.0 0.0
1.6009 48.0 4560 1.2496 0.5722 0.0 0.0 0.0 0.0 0.0
1.6009 49.0 4655 1.2447 0.5633 0.0 0.0 0.0 0.0 0.0
1.6009 50.0 4750 1.2411 0.5615 0.0 0.0 0.0 0.0 0.0
1.6009 51.0 4845 1.2356 0.5691 0.0 0.0 0.0 0.0 0.0
1.6009 52.0 4940 1.2322 0.5636 0.0 0.0 0.0 0.0 0.0
1.5481 53.0 5035 1.2285 0.5724 0.0 0.0 0.0 0.0 0.0
1.5481 54.0 5130 1.2255 0.5771 0.0 0.0 0.0 0.0 0.0
1.5481 55.0 5225 1.2201 0.5827 0.0 0.0 0.0 0.0 0.0
1.5481 56.0 5320 1.2181 0.5928 0.0 0.0 0.0 0.0 0.0
1.5481 57.0 5415 1.2152 0.5599 0.0 0.0 0.0 0.0 0.0
1.5082 58.0 5510 1.2123 0.5779 0.0 0.0 0.0 0.0 0.0
1.5082 59.0 5605 1.2083 0.5609 0.0 0.0 0.0 0.0 0.0
1.5082 60.0 5700 1.2070 0.5654 0.0 0.0 0.0 0.0 0.0
1.5082 61.0 5795 1.2036 0.5566 0.0 0.0 0.0 0.0 0.0
1.5082 62.0 5890 1.2011 0.5569 0.0 0.0 0.0 0.0 0.0
1.5082 63.0 5985 1.1993 0.5567 0.0 0.0 0.0 0.0 0.0
1.4799 64.0 6080 1.1958 0.5619 0.0 0.0 0.0 0.0 0.0
1.4799 65.0 6175 1.1950 0.5691 0.0 0.0 0.0 0.0 0.0
1.4799 66.0 6270 1.1914 0.5572 0.0 0.0 0.0 0.0 0.0
1.4799 67.0 6365 1.1879 0.5635 0.0 0.0 0.0 0.0 0.0
1.4799 68.0 6460 1.1866 0.5654 0.0 0.0 0.0 0.0 0.0
1.4475 69.0 6555 1.1850 0.5575 0.0 0.0 0.0 0.0 0.0
1.4475 70.0 6650 1.1833 0.5507 0.0 0.0 0.0 0.0 0.0
1.4475 71.0 6745 1.1820 0.5493 0.0 0.0 0.0 0.0 0.0
1.4475 72.0 6840 1.1786 0.5525 0.0 0.0 0.0 0.0 0.0
1.4475 73.0 6935 1.1789 0.5615 0.0 0.0 0.0 0.0 0.0
1.4233 74.0 7030 1.1770 0.5603 0.0 0.0 0.0 0.0 0.0
1.4233 75.0 7125 1.1749 0.5699 0.0 0.0 0.0 0.0 0.0
1.4233 76.0 7220 1.1754 0.5730 0.0 0.0 0.0 0.0 0.0
1.4233 77.0 7315 1.1735 0.5798 0.0 0.0 0.0 0.0 0.0
1.4233 78.0 7410 1.1716 0.5771 0.0 0.0 0.0 0.0 0.0
1.4101 79.0 7505 1.1699 0.5800 0.0 0.0 0.0 0.0 0.0
1.4101 80.0 7600 1.1675 0.5736 0.0 0.0 0.0 0.0 0.0
1.4101 81.0 7695 1.1661 0.5845 0.0 0.0 0.0 0.0 0.0
1.4101 82.0 7790 1.1659 0.5974 0.0 0.0 0.0 0.0 0.0
1.4101 83.0 7885 1.1664 0.5825 0.0 0.0 0.0 0.0 0.0
1.4101 84.0 7980 1.1647 0.5871 0.0 0.0 0.0 0.0 0.0
1.3965 85.0 8075 1.1639 0.5772 0.0 0.0 0.0 0.0 0.0
1.3965 86.0 8170 1.1628 0.5826 0.0 0.0 0.0 0.0 0.0
1.3965 87.0 8265 1.1615 0.5960 0.0 0.0 0.0 0.0 0.0
1.3965 88.0 8360 1.1616 0.5908 0.0 0.0 0.0 0.0 0.0
1.3965 89.0 8455 1.1613 0.5775 0.0 0.0 0.0 0.0 0.0
1.3835 90.0 8550 1.1604 0.5917 0.0 0.0 0.0 0.0 0.0
1.3835 91.0 8645 1.1597 0.5732 0.0 0.0 0.0 0.0 0.0
1.3835 92.0 8740 1.1594 0.5767 0.0 0.0 0.0 0.0 0.0
1.3835 93.0 8835 1.1584 0.5719 0.0 0.0 0.0 0.0 0.0
1.3835 94.0 8930 1.1581 0.5700 0.0 0.0 0.0 0.0 0.0
1.3766 95.0 9025 1.1583 0.5845 0.0 0.0 0.0 0.0 0.0
1.3766 96.0 9120 1.1578 0.5808 0.0 0.0 0.0 0.0 0.0
1.3766 97.0 9215 1.1578 0.5889 0.0 0.0 0.0 0.0 0.0
1.3766 98.0 9310 1.1577 0.5851 0.0 0.0 0.0 0.0 0.0
1.3766 99.0 9405 1.1578 0.5923 0.0 0.0 0.0 0.0 0.0
1.3726 100.0 9500 1.1577 0.5923 0.0 0.0 0.0 0.0 0.0

Framework versions

  • Transformers 4.18.0
  • Pytorch 1.11.0+cu113
  • Datasets 2.1.0
  • Tokenizers 0.12.1
Downloads last month
3

Dataset used to train chainyo/t5-base-sede-txt2sql

Evaluation results

Metadata error: specify a dataset to view leaderboard