Package: text-to-sql
Text to SQLThis Python package converts natural language queries into PostgreSQL commands. It uses OpenAI's GPT-3.5-turbo model to understand and translate the user's input into SQL queries. The package also includes functionalities for testing the accuracy of the model and generating mock data.Quick Installationpip3installtext_to_sql# pip3 for MacOS/Linux and pip for WindowsGetting StartedHere is the python code for generating the PostgreSQL query using GPT-3.5 Turbo 16k modelfromtext_to_sqlimportcreate_model,get_sql_query,execute_command# set OPENAI_API_KEY in the environment variablemodel=create_model()model.load_schema_from_file('<file_path>')llm_output=get_sql_query(model,'<natural_language_query>')print('SQL Query: ',llm_output.message)ifllm_output.is_final_output:print('SQL Output: ',execute_command(llm_output.message))More examples are availablehereUsageCreating a ModelTo create a model, use thecreate_modelfunction. This function initializes the language model GPT 3.5 Turbo 16k with the OpenAI API key and sets the temperature to 0.fromtext_to_sqlimportcreate_model# !Important OPENAI_API_KEY should be added in the environment variable before calling this functionmodel=create_model()Loading a SchemaTo load a schema from a file, use theload_schema_from_filemethod. This method takes the absolute path of the schema file as an argument.model.load_schema_from_file('/path/to/schema.txt')Predicting SQL QueriesTo predict a SQL query from a natural language input, use thepredictmethod. This method takes a string as an argument and returns aModelOutputobject. TheModelOutputobject has two properties:messageandis_final_output. Themessageproperty contains the predicted SQL query, and theis_final_outputproperty is a boolean that indicates whether the predicted SQL query is the final output.user_input="How many total visitors have visited hardy.net domain?"output=model.predict(user_input)print("Query: ",output.message)print("Is final output: ",output.is_final_output)Running Test SuitesTo run test suites, use therun_test_suitesfunction. This function lists all the test files for the user to select and runs the selected test file.fromsrcimportrun_test_suitesrun_test_suites()Creating Mock DataTo create mock data, use thecreate_mock_datafunction. This function lists all the data structure files for the user to select and asks how many number of fake data has to be generated. It then generates the specified number of fake data based on the selected data structure.fromsrcimportcreate_mock_datacreate_mock_data()TestingThe package includes a test suite for thewebsite_aggregatesschema. The test suite is defined in thewebsite_aggregate_test.pyfile. Each test case in the test suite is a dictionary with the following keys:input: The natural language input.sql_output: The expected SQL output.description: A description of the test case (optional).To run the test suite, use therun_test_suitesfunction and select thewebsite_aggregate_test.pyfile.Database Connectordatabase_connector.pycontains 7 functions that makes interacting with PostgreSQL databases using the psycopg2 Python module easier. The functions currently present include functions for:Creating a tableDeleting a tableSelecting records from the tableInserting new records into an existing tableDeleting records from an existing tableChecking if a table existsExecuting user-inputted commands yourself (recommended)Pre-requisitesBefore using these functions, ensure that you complete the following beforehand:Install Python 3Refer to platform-specific instructions for installing Python on your machine.Install Psycopg2 moduleIf you don't already have it, runpip install psycopg2orpip install psycopg2-binary. You may have to replacepipwithpip3if the former does not work.Obtain access to a PostgreSQL serverThese functions were designed specifically around Supabase's hosted database. You may need to alter the connection details to fit your needs.Obtain connection detailsIf you are using a PostgreSQL database hosted by Supabase, you'll need the database name, username, password, hostname, and port (optional) to connect to the database. It is recommended that you keep these details secret by placing them in a hidden .env file in the same directory.Import the functionsImport the functions like you would import anything else in Python, by placing this line at the top of your file:import database_connector. If you are experiencing difficulties with importing, try this instead:importos,syssys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)),'..'))fromdatabase_connectorimport*Executing user commandsexecute_command(command)This function executes a SQL command passed to it as a string. Use this whenever possible.Parameters:command: A string that contains a valid SQL statement to be executed on the database.Returns:A list of tuples that contains the result of the query, or None if the query does not return any rows.Exceptions:If something goes wrong, such as a syntax error or a connection error, it prints the exception and returns None.Example:# Create a new table named table_1 with columns name, age, and countryexecute_command("CREATE TABLE table_1 (name varchar(255), age int, country varchar(3))")# Select the names from table_1 where age is greater than 30result=execute_command("SELECT name FROM table_1 WHERE age > 30")print(result)# Output: []Creating a tablecreate_table(table_name, *columns)This function creates a named table with specified columns.Parameters:table_name: A string containing the name of the table to be created.columns: A string containing names of columns, `` (space), the data type of the column, separated by,(comma space).Returns:None.Exceptions:If something goes wrong, such as a table already exists or an invalid data type, it prints the exception and does not create the table.Example:# Create a table named table_2 with columns name, age, and countrycreate_table("table_2","name varchar(255), age int, country varchar(3)")Deleting a tabledrop_table(table_name)This function deletes the table matching the table name.Parameters:table_name: A string containing the name of the table to be deleted.Returns:None.Exceptions:If something goes wrong, such as a table does not exist or a permission error, it prints the exception and does not drop the table.Example:# Delete table named table_1drop_table("table_1")Selecting recordsselect(table_name, column_names, selectors=None)This function selects records from an existing table with optional selectors.Parameters:table_name: A string containing the name of the table to select from.column_names: A list of strings containing the names of columns to retrieve.selectors: Optional dictionary where the values are either values to match against or tuples of operators and values.Returns:A list of dictionaries representing the records that match the selectors, or all records if no selectors are provided.Exceptions:If something goes wrong, such as a table or a column does not exist or an invalid operator or value, it prints the exception and returns None.Examples:# Select records from the name and country column from table_2 where age is less than 30columns=["name","country"]selectors={"age":("<",30)}result=select("table_2",columns,selectors)print(result)# Output: [{'name':'a name', 'country':'ABC'}, ...]# Select records from the name and age column from table_2 where the country is the UScolumns=["name","age"]selectors={"country":"US"}result=select("test1",columns,selectors)print(result)# Output: [{'name':'a name', 'age':'number'}, ...]Inserting recordsinsert_records(table_name, records)This function inserts new records into an existing table.Parameters:table_name: A string containing the name of the table to insert into.records: A list of dictionaries where the key is the column name and the value is the value for that column. Each dictionary in the list is a new entry.Returns:None.Exceptions:If something goes wrong, such as a table or a column does not exist or a value does not match the data type, it prints the exception and does not insert the records.Example:# Insert two new records into table_2records=[{"name":"John","age":"29","country":"US"},{"name":"Jane","age":"24","country":"CA"}]insert_records("test1",records)Dropping recordsdelete_records(table_name, selectors=None)This function deletes records from an existing table using the selectors, or all records if none are specified.Parameters:table_name: A string containing the name of the table to drop records from.selectors: Optional dictionary where the values are either values to match against or tuples of operators and values.Returns:Integer representing the number of records deleted, or zero if no records match the selectors.Exceptions:If something goes wrong, such as a table or a column does not exist or an invalid operator or value, it prints the exception and returns None.Example:# Delete records from table_2 where age is less than 20selectors={"age":("<",20)}delete_records("table_2",selectors)# Delete records where name is Johnselectors={"name":"John"}delete_records("table_2",selectors)# Delete all records from table_2delete_records("table_2")Existing tableexists(table_name)This function checks if a table with the given table name exists and returns a boolean.Parameters:table_name: A string containing the name of the table to check.Returns:True if a table with the name is found, False otherwise.Exceptions:If something goes wrong, such as a connection error or an invalid table name, it prints the exception and returns False.Example:# Check if table_1 existsprint(exists("table_1"))# Output: False# Check if table_2 existsprint(exists("table_2"))# Output: TrueDevelopersVignesh Prakash -pranomvignesh@gmail.comSullivan Dovie -sullivandovie@gmail.comElhanan Wong -wong.elhanan@gmail.comAyana Gaur -ayanagaur2@gmail.comMikhael Gonzalez -mikhael.gonzalez@gmail.comElizabeth Petit -elizavetapetit@gmail.com