FLAN-T5 Base Text to SQL Model

This model was fine-tuned on Google's FLAN-T5 base using SParC, Spider, and CoSQL datasets.

Purpose of this model is to create SQL queries from natural-language text.

In order to achieve accuracte results, database schema was incorporated to the prompt during training.

GitHub repository can be found here.

Requirements

pip install transformers==4.38.2
pip install torch==2.2.2

Usage

Please exercise caution when formatting the input.

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("alpecevit/flan-t5-base-text2sql")
model = AutoModelForSeq2SeqLM.from_pretrained("alpecevit/flan-t5-base-text2sql")

input_text = """
transform question and schema to SQL query. question: Who are the top 5 most paid employess by first name, last name, and salary ? schema: employee(salary, bdate, dno, ssn, fname, sex, superssn, address, minit, lname), department(dnumber, mgrstartdate, dname, mgrssn), dept_locations(dnumber, dlocation), project(pnumber, dnum, pname, plocation), works_on(pno, hours, essn), dependent(bdate, essn, dependent_name, sex, relationship).
"""

token_input = tokenizer(input_text, return_tensors="pt").input_ids
output = model.generate(token_input, max_new_tokens=128)
query = tokenizer.decode(output[0], skip_special_tokens=True)

print("Predicted Query:", query)

Output:

SELECT fname, lname, salary FROM employee ORDER BY salary DESC LIMIT 5

Evaluation

The fine-tuned model was evaluated using the combination of test splits of the above datasets. ROUGE metrics were utilized for the assessment, and the results are outlined below.

{'rouge1': 0.8740305983060861, 'rouge2': 0.7763397400315798, 'rougeL': 0.8449832130213266, 'rougeLsum': 0.8447120646910007}