*   [](/v0.2/)
*   [How-to guides](/v0.2/docs/how_to/)
*   How to do query validation as part of SQL question-answering

On this page

How to do query validation as part of SQL question-answering
============================================================

Perhaps the most error-prone part of any SQL chain or agent is writing valid and safe SQL queries. In this guide we'll go over some strategies for validating our queries and handling invalid queries.

We will cover:

1.  Appending a "query validator" step to the query generation;
2.  Prompt engineering to reduce the incidence of errors.

Setup[​](#setup "Direct link to Setup")
---------------------------------------

First, get required packages and set environment variables:

    %pip install --upgrade --quiet  langchain langchain-community langchain-openai

    # Uncomment the below to use LangSmith. Not required.# import os# os.environ["LANGCHAIN_API_KEY"] = getpass.getpass()# os.environ["LANGCHAIN_TRACING_V2"] = "true"

The below example will use a SQLite connection with Chinook database. Follow [these installation steps](https://database.guide/2-sample-databases-sqlite/) to create `Chinook.db` in the same directory as this notebook:

*   Save [this file](https://raw.githubusercontent.com/lerocha/chinook-database/master/ChinookDatabase/DataSources/Chinook_Sqlite.sql) as `Chinook_Sqlite.sql`
*   Run `sqlite3 Chinook.db`
*   Run `.read Chinook_Sqlite.sql`
*   Test `SELECT * FROM Artist LIMIT 10;`

Now, `Chinhook.db` is in our directory and we can interface with it using the SQLAlchemy-driven `SQLDatabase` class:

    from langchain_community.utilities import SQLDatabasedb = SQLDatabase.from_uri("sqlite:///Chinook.db")print(db.dialect)print(db.get_usable_table_names())print(db.run("SELECT * FROM Artist LIMIT 10;"))

**API Reference:**[SQLDatabase](https://api.python.langchain.com/en/latest/utilities/langchain_community.utilities.sql_database.SQLDatabase.html)

    sqlite['Album', 'Artist', 'Customer', 'Employee', 'Genre', 'Invoice', 'InvoiceLine', 'MediaType', 'Playlist', 'PlaylistTrack', 'Track'][(1, 'AC/DC'), (2, 'Accept'), (3, 'Aerosmith'), (4, 'Alanis Morissette'), (5, 'Alice In Chains'), (6, 'Antônio Carlos Jobim'), (7, 'Apocalyptica'), (8, 'Audioslave'), (9, 'BackBeat'), (10, 'Billy Cobham')]

Query checker[​](#query-checker "Direct link to Query checker")
---------------------------------------------------------------

Perhaps the simplest strategy is to ask the model itself to check the original query for common mistakes. Suppose we have the following SQL query chain:

*   OpenAI
*   Anthropic
*   Azure
*   Google
*   Cohere
*   FireworksAI
*   Groq
*   MistralAI
*   TogetherAI

    pip install -qU langchain-openai

    import getpassimport osos.environ["OPENAI_API_KEY"] = getpass.getpass()from langchain_openai import ChatOpenAIllm = ChatOpenAI(model="gpt-3.5-turbo-0125")

    pip install -qU langchain-anthropic

    import getpassimport osos.environ["ANTHROPIC_API_KEY"] = getpass.getpass()from langchain_anthropic import ChatAnthropicllm = ChatAnthropic(model="claude-3-sonnet-20240229")

    pip install -qU langchain-openai

    import getpassimport osos.environ["AZURE_OPENAI_API_KEY"] = getpass.getpass()from langchain_openai import AzureChatOpenAIllm = AzureChatOpenAI(    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],    azure_deployment=os.environ["AZURE_OPENAI_DEPLOYMENT_NAME"],    openai_api_version=os.environ["AZURE_OPENAI_API_VERSION"],)

    pip install -qU langchain-google-vertexai

    import getpassimport osos.environ["GOOGLE_API_KEY"] = getpass.getpass()from langchain_google_vertexai import ChatVertexAIllm = ChatVertexAI(model="gemini-pro")

    pip install -qU langchain-cohere

    import getpassimport osos.environ["COHERE_API_KEY"] = getpass.getpass()from langchain_cohere import ChatCoherellm = ChatCohere(model="command-r")

    pip install -qU langchain-fireworks

    import getpassimport osos.environ["FIREWORKS_API_KEY"] = getpass.getpass()from langchain_fireworks import ChatFireworksllm = ChatFireworks(model="accounts/fireworks/models/mixtral-8x7b-instruct")

    pip install -qU langchain-groq

    import getpassimport osos.environ["GROQ_API_KEY"] = getpass.getpass()from langchain_groq import ChatGroqllm = ChatGroq(model="llama3-8b-8192")

    pip install -qU langchain-mistralai

    import getpassimport osos.environ["MISTRAL_API_KEY"] = getpass.getpass()from langchain_mistralai import ChatMistralAIllm = ChatMistralAI(model="mistral-large-latest")

    pip install -qU langchain-openai

    import getpassimport osos.environ["TOGETHER_API_KEY"] = getpass.getpass()from langchain_openai import ChatOpenAIllm = ChatOpenAI(    base_url="https://api.together.xyz/v1",    api_key=os.environ["TOGETHER_API_KEY"],    model="mistralai/Mixtral-8x7B-Instruct-v0.1",)

    from langchain.chains import create_sql_query_chainchain = create_sql_query_chain(llm, db)

**API Reference:**[create\_sql\_query\_chain](https://api.python.langchain.com/en/latest/chains/langchain.chains.sql_database.query.create_sql_query_chain.html)

And we want to validate its outputs. We can do so by extending the chain with a second prompt and model call:

    from langchain_core.output_parsers import StrOutputParserfrom langchain_core.prompts import ChatPromptTemplatesystem = """Double check the user's {dialect} query for common mistakes, including:- Using NOT IN with NULL values- Using UNION when UNION ALL should have been used- Using BETWEEN for exclusive ranges- Data type mismatch in predicates- Properly quoting identifiers- Using the correct number of arguments for functions- Casting to the correct data type- Using the proper columns for joinsIf there are any of the above mistakes, rewrite the query.If there are no mistakes, just reproduce the original query with no further commentary.Output the final SQL query only."""prompt = ChatPromptTemplate.from_messages(    [("system", system), ("human", "{query}")]).partial(dialect=db.dialect)validation_chain = prompt | llm | StrOutputParser()full_chain = {"query": chain} | validation_chain

**API Reference:**[StrOutputParser](https://api.python.langchain.com/en/latest/output_parsers/langchain_core.output_parsers.string.StrOutputParser.html) | [ChatPromptTemplate](https://api.python.langchain.com/en/latest/prompts/langchain_core.prompts.chat.ChatPromptTemplate.html)

    query = full_chain.invoke(    {        "question": "What's the average Invoice from an American customer whose Fax is missing since 2003 but before 2010"    })print(query)

    SELECT AVG(i.Total) AS AverageInvoiceFROM Invoice iJOIN Customer c ON i.CustomerId = c.CustomerIdWHERE c.Country = 'USA'AND c.Fax IS NULLAND i.InvoiceDate >= '2003-01-01' AND i.InvoiceDate < '2010-01-01'

Note how we can see both steps of the chain in the [Langsmith trace](https://smith.langchain.com/public/8a743295-a57c-4e4c-8625-bc7e36af9d74/r).

    db.run(query)

    '[(6.632999999999998,)]'

The obvious downside of this approach is that we need to make two model calls instead of one to generate our query. To get around this we can try to perform the query generation and query check in a single model invocation:

    system = """You are a {dialect} expert. Given an input question, create a syntactically correct {dialect} query to run.Unless the user specifies in the question a specific number of examples to obtain, query for at most {top_k} results using the LIMIT clause as per {dialect}. You can order the results to return the most informative data in the database.Never query for all columns from a table. You must query only the columns that are needed to answer the question. Wrap each column name in double quotes (") to denote them as delimited identifiers.Pay attention to use only the column names you can see in the tables below. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.Pay attention to use date('now') function to get the current date, if the question involves "today".Only use the following tables:{table_info}Write an initial draft of the query. Then double check the {dialect} query for common mistakes, including:- Using NOT IN with NULL values- Using UNION when UNION ALL should have been used- Using BETWEEN for exclusive ranges- Data type mismatch in predicates- Properly quoting identifiers- Using the correct number of arguments for functions- Casting to the correct data type- Using the proper columns for joinsUse format:First draft: <<FIRST_DRAFT_QUERY>>Final answer: <<FINAL_ANSWER_QUERY>>"""prompt = ChatPromptTemplate.from_messages(    [("system", system), ("human", "{input}")]).partial(dialect=db.dialect)def parse_final_answer(output: str) -> str:    return output.split("Final answer: ")[1]chain = create_sql_query_chain(llm, db, prompt=prompt) | parse_final_answerprompt.pretty_print()

    ================================[1m System Message [0m================================You are a [33;1m[1;3m{dialect}[0m expert. Given an input question, create a syntactically correct [33;1m[1;3m{dialect}[0m query to run.Unless the user specifies in the question a specific number of examples to obtain, query for at most [33;1m[1;3m{top_k}[0m results using the LIMIT clause as per [33;1m[1;3m{dialect}[0m. You can order the results to return the most informative data in the database.Never query for all columns from a table. You must query only the columns that are needed to answer the question. Wrap each column name in double quotes (") to denote them as delimited identifiers.Pay attention to use only the column names you can see in the tables below. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.Pay attention to use date('now') function to get the current date, if the question involves "today".Only use the following tables:[33;1m[1;3m{table_info}[0mWrite an initial draft of the query. Then double check the [33;1m[1;3m{dialect}[0m query for common mistakes, including:- Using NOT IN with NULL values- Using UNION when UNION ALL should have been used- Using BETWEEN for exclusive ranges- Data type mismatch in predicates- Properly quoting identifiers- Using the correct number of arguments for functions- Casting to the correct data type- Using the proper columns for joinsUse format:First draft: <<FIRST_DRAFT_QUERY>>Final answer: <<FINAL_ANSWER_QUERY>>================================[1m Human Message [0m=================================[33;1m[1;3m{input}[0m

    query = chain.invoke(    {        "question": "What's the average Invoice from an American customer whose Fax is missing since 2003 but before 2010"    })print(query)

    SELECT AVG(i."Total") AS "AverageInvoice"FROM "Invoice" iJOIN "Customer" c ON i."CustomerId" = c."CustomerId"WHERE c."Country" = 'USA'AND c."Fax" IS NULLAND i."InvoiceDate" BETWEEN '2003-01-01' AND '2010-01-01';

    db.run(query)

    '[(6.632999999999998,)]'

Human-in-the-loop[​](#human-in-the-loop "Direct link to Human-in-the-loop")
---------------------------------------------------------------------------

In some cases our data is sensitive enough that we never want to execute a SQL query without a human approving it first. Head to the [Tool use: Human-in-the-loop](/v0.2/docs/how_to/tools_human/) page to learn how to add a human-in-the-loop to any tool, chain or agent.

Error handling[​](#error-handling "Direct link to Error handling")
------------------------------------------------------------------

At some point, the model will make a mistake and craft an invalid SQL query. Or an issue will arise with our database. Or the model API will go down. We'll want to add some error handling behavior to our chains and agents so that we fail gracefully in these situations, and perhaps even automatically recover. To learn about error handling with tools, head to the [Tool use: Error handling](/v0.2/docs/how_to/tools_error/) page.

[Edit this page](https://github.com/langchain-ai/langchain/edit/master/docs/docs/how_to/sql_query_checking.ipynb)

* * *

#### Was this page helpful?

  

#### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchain/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

How to better prompt when doing SQL question-answering

](/v0.2/docs/how_to/sql_prompting/)[

Next

How to stream runnables

](/v0.2/docs/how_to/streaming/)

*   [Setup](#setup)
*   [Query checker](#query-checker)
*   [Human-in-the-loop](#human-in-the-loop)
*   [Error handling](#error-handling)