StructuredChatAgent uses SQLDatabaseToolkit tools in wrong order
Labels: name: area: agent, name: auto:bug

### System Info

Langchain version - 0.0.235
Python version - 3.10

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Hello,

I'm trying to use a Structured Chat agent with SQL tools as well as a VectorStoreQA tool in order to retrieve data from a Postgres database and our Pinecone store. For the Agent's chat model, I'm using Azure OpenAI's gpt-3.5-turbo version 0301.

I have tried providing the tools via the SQLDatabaseToolkit's get_tools() function as well as importing the individual tools and modifying their descriptions, but I get the same issue with both approaches. Despite the descriptions and formatting instructions provided below, the agent has a lot of trouble with querying the DB. 

It almost always starts off with 'sql_db_query' to construct a query that errors out, which then prompts the agent to check the list of tables and then their schemas. From here, it can usually arrive at the correct query - but this is not the correct behavior. Other times, it will get stuck in a loop of constructing a query, getting an error, and then either checking the query or just constructing a new one. The errors are due to the schema that it hallucinates because it didn't use the 'sql_db_list_tables' and 'sql_db_schema' tools first.

In the code excerpt below, you can see the formatting instructions provided as well as the descriptions that I'm giving the SQL tools.

Has anybody else had issues with SQL database tools and the StructuredChat agent? Should I be using a different type of agent for this?

``````
FORMAT_INSTRUCTIONS = """Use a json blob to specify a tool by providing an action key (tool name) and an 
    action_input key (tool input).

For SQL queries, ALWAYS use the available tools in this order:
     1. sql_db_list_tables
     2. sql_db_schema
     3. sql_db_query_checker
     4. sql_db_query

Valid "action" values: "Final Answer" or {tool_names}

Provide only ONE action per $JSON_BLOB, as shown:

```
{{{{
  "action": $TOOL_NAME,
  "action_input": $INPUT
}}}}
```

Follow this format:

Question: input question to answer
Thought: consider previous and subsequent steps
Action:
```
$JSON_BLOB
```
Observation: action result
... (repeat Thought/Action/Observation N times)
Thought: I know what to respond
Action:
```
{{{{
  "action": "Final Answer",
  "action_input": "Final response to human"
}}}}
```"""

vectorstore_info = VectorStoreInfo(
    name="incident_resolution_instructions",
    description="MOP Documents that help users resolve incidents for different devices and causes",
    vectorstore=pineconeStore,
)

llm = AzureChatOpenAI(temperature=0, verbose=True,
                    deployment_name='chatgpt-35', model_name="gpt-35-turbo", max_tokens=4000)

query_sql_database_tool_description = (
            "ONLY use this tool AFTER using 'sql_db_schema' and 'sql_db_query_checker'."
            "Input to this tool is a detailed and correct SQL query, output is a "
            "result from the database. If the query is not correct, an error message "
            "will be returned. If an error is returned, rewrite the query, check the "
            "query, and try again. If you encounter an issue with Unknown column "
            "'xxxx' in 'field list', use 'sql_db_schema' to query the correct table "
            "fields."
        )
info_sql_database_tool_description = (
            """
            ALWAYS use this tool second AFTER 'sql_db_list_tables'.
            Input to this tool is a comma-separated list of tables, output is the 
            schema and sample rows for those tables. 
            Be sure that the tables actually exist by calling 'sql_db_list_tables'
            first! Example Input: table1, table2, table3
            """
        )
list_sql_database_tool_description = (
    "ALWAYS use this tool first. Input to this tool is an empty string '', output is the list of PostgreSQL tables in the database."
)
query_checker_sql_database_tool_description = (
    """
    ALWAYS Use this tool third AFTER 'sql_db_list_tables' and 'sql_db_schema'.
    ALWAYS use this tool to double check if your query is correct before executing it.
    ALWAYS use this tool BEFORE executing a query with 'sql_db_query'
    """
)

tools = [
            ListSQLDatabaseTool(db=db, description=list_sql_database_tool_description), 
            InfoSQLDatabaseTool(db=db, description=info_sql_database_tool_description), 
            QuerySQLDataBaseTool(db=db, description=query_sql_database_tool_description), 
            QuerySQLCheckerTool(db=db, description=query_checker_sql_database_tool_description, llm=AzureOpenAI(temperature=0, verbose=True,
                        deployment_name='chatgpt-35', model_name="gpt-35-turbo", max_tokens=4000)), 
            VectorStoreQATool(vectorstore=vectorstore_info.vectorstore, llm=AzureOpenAI(temperature=0, verbose=True,
                        deployment_name='chatgpt-35', model_name="gpt-35-turbo", max_tokens=4000), name="incident_resolution_steps", description="Documentation detailing steps to resolve an incident for various device types such as Routers and Switches.") 
         ]

    agent_chain = initialize_agent(tools, llm, agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True, memory=memory, agent_kwargs={
        'prefix': PREFIX, 
        'format_instructions': FORMAT_INSTRUCTIONS,
        'suffix': SUFFIX,
        'input_variables': ["input", "chat_history", "agent_scratchpad"]
    })

    response = agent_chain.run(input=event['question'])
``````

### Expected behavior

The agent should:

- Follow the order provided in the default descriptions of each SQLDatabase Tool
- Follow instructions for tools provided in the formatting instructions or prompt suffix