How to handle multiple queries when doing query analysis
Source: langchain

---sidebar_position: 4---# How to handle multiple queries when doing query analysisSometimes, a query analysis technique may allow for multiple queries to be generated. In these cases, we need to remember to run all queries and then to combine the results. We will show a simple example (using mock data) of how to do that.## Setup#### Install dependencies```python# %pip install -qU langchain langchain-community langchain-openai langchain-chroma```#### Set environment variablesWe'll use OpenAI in this example:```pythonimport getpassimport osos.environ["OPENAI_API_KEY"] = getpass.getpass()# Optional, uncomment to trace runs with LangSmith. Sign up here: https://smith.langchain.com.# os.environ["LANGCHAIN_TRACING_V2"] = "true"# os.environ["LANGCHAIN_API_KEY"] = getpass.getpass()```### Create IndexWe will create a vectorstore over fake information.```pythonfrom langchain_chroma import Chromafrom langchain_openai import OpenAIEmbeddingsfrom langchain_text_splitters import RecursiveCharacterTextSplittertexts = ["Harrison worked at Kensho", "Ankush worked at Facebook"]embeddings = OpenAIEmbeddings(model="text-embedding-3-small")vectorstore = Chroma.from_texts(    texts,    embeddings,)retriever = vectorstore.as_retriever(search_kwargs={"k": 1})```## Query analysisWe will use function calling to structure the output. We will let it return multiple queries.```pythonfrom typing import List, Optionalfrom langchain_core.pydantic_v1 import BaseModel, Fieldclass Search(BaseModel):    """Search over a database of job records."""    queries: List[str] = Field(        ...,        description="Distinct queries to search for",    )``````pythonfrom langchain_core.output_parsers.openai_tools import PydanticToolsParserfrom langchain_core.prompts import ChatPromptTemplatefrom langchain_core.runnables import RunnablePassthroughfrom langchain_openai import ChatOpenAIoutput_parser = PydanticToolsParser(tools=[Search])system = """You have the ability to issue search queries to get information to help answer user information.If you need to look up two distinct pieces of information, you are allowed to do that!"""prompt = ChatPromptTemplate.from_messages(    [        ("system", system),        ("human", "{question}"),    ])llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)structured_llm = llm.with_structured_output(Search)query_analyzer = {"question": RunnablePassthrough()} | prompt | structured_llm```    /Users/harrisonchase/workplace/langchain/libs/core/langchain_core/_api/beta_decorator.py:86: LangChainBetaWarning: The function `with_structured_output` is in beta. It is actively being worked on, so the API may change.      warn_beta(We can see that this allows for creating multiple queries```pythonquery_analyzer.invoke("where did Harrison Work")```    Search(queries=['Harrison work location'])```pythonquery_analyzer.invoke("where did Harrison and ankush Work")```    Search(queries=['Harrison work place', 'Ankush work place'])## Retrieval with query analysisSo how would we include this in a chain? One thing that will make this a lot easier is if we call our retriever asyncronously - this will let us loop over the queries and not get blocked on the response time.```pythonfrom langchain_core.runnables import chain``````python@chainasync def custom_chain(question):    response = await query_analyzer.ainvoke(question)    docs = []    for query in response.queries:        new_docs = await retriever.ainvoke(query)        docs.extend(new_docs)    # You probably want to think about reranking or deduplicating documents here    # But that is a separate topic    return docs``````pythonawait custom_chain.ainvoke("where did Harrison Work")```    [Document(page_content='Harrison worked at Kensho')]```pythonawait custom_chain.ainvoke("where did Harrison and ankush Work")```    [Document(page_content='Harrison worked at Kensho'),     Document(page_content='Ankush worked at Facebook')]```python```