make my program generate search queries using a llm for the retrival. make it generate 2 search queries. and then use a ai to rank the results to make sure they are relevant. 
here is some documentation that will maybe help you:
MultiVector Retriever

It can often be beneficial to store multiple vectors per document. There are multiple use cases where this is beneficial. LangChain has a base MultiVectorRetriever which makes querying this type of setup easy. A lot of the complexity lies in how to create the multiple vectors per document. This notebook covers some of the common ways to create those vectors and use the MultiVectorRetriever.

The methods to create multiple vectors per document include:

    Smaller chunks: split a document into smaller chunks, and embed those (this is ParentDocumentRetriever).
    Summary: create a summary for each document, embed that along with (or instead of) the document.
    Hypothetical questions: create hypothetical questions that each document would be appropriate to answer, embed those along with (or instead of) the document.

Note that this also enables another method of adding embeddings - manually. This is great because you can explicitly add questions or queries that should lead to a document being recovered, giving you more control.

from langchain.retrievers.multi_vector import MultiVectorRetriever

API Reference:

    MultiVectorRetriever

from langchain.storage import InMemoryByteStore
from langchain_chroma import Chroma
from langchain_community.document_loaders import TextLoader
from langchain_openai import OpenAIEmbeddings
from langchain_text_splitters import RecursiveCharacterTextSplitter

API Reference:

    InMemoryByteStore
    TextLoader
    OpenAIEmbeddings
    RecursiveCharacterTextSplitter

loaders = [
    TextLoader("../../paul_graham_essay.txt"),
    TextLoader("../../state_of_the_union.txt"),
]
docs = []
for loader in loaders:
    docs.extend(loader.load())
text_splitter = RecursiveCharacterTextSplitter(chunk_size=10000)
docs = text_splitter.split_documents(docs)

Smaller chunks

Often times it can be useful to retrieve larger chunks of information, but embed smaller chunks. This allows for embeddings to capture the semantic meaning as closely as possible, but for as much context as possible to be passed downstream. Note that this is what the ParentDocumentRetriever does. Here we show what is going on under the hood.

# The vectorstore to use to index the child chunks
vectorstore = Chroma(
    collection_name="full_documents", embedding_function=OpenAIEmbeddings()
)
# The storage layer for the parent documents
store = InMemoryByteStore()
id_key = "doc_id"
# The retriever (empty to start)
retriever = MultiVectorRetriever(
    vectorstore=vectorstore,
    byte_store=store,
    id_key=id_key,
)
import uuid

doc_ids = [str(uuid.uuid4()) for _ in docs]

# The splitter to use to create smaller chunks
child_text_splitter = RecursiveCharacterTextSplitter(chunk_size=400)

sub_docs = []
for i, doc in enumerate(docs):
    _id = doc_ids[i]
    _sub_docs = child_text_splitter.split_documents([doc])
    for _doc in _sub_docs:
        _doc.metadata[id_key] = _id
    sub_docs.extend(_sub_docs)

retriever.vectorstore.add_documents(sub_docs)
retriever.docstore.mset(list(zip(doc_ids, docs)))

# Vectorstore alone retrieves the small chunks
retriever.vectorstore.similarity_search("justice breyer")[0]

Document(page_content='Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \n\nOne of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court.', metadata={'doc_id': '2fd77862-9ed5-4fad-bf76-e487b747b333', 'source': '../../state_of_the_union.txt'})

# Retriever returns larger chunks
len(retriever.invoke("justice breyer")[0].page_content)

9875

The default search type the retriever performs on the vector database is a similarity search. LangChain Vector Stores also support searching via Max Marginal Relevance so if you want this instead you can just set the search_type property as follows:

from langchain.retrievers.multi_vector import SearchType

retriever.search_type = SearchType.mmr

len(retriever.invoke("justice breyer")[0].page_content)

API Reference:

    SearchType

9875

Summary

Oftentimes a summary may be able to distill more accurately what a chunk is about, leading to better retrieval. Here we show how to create summaries, and then embed those.

import uuid

from langchain_core.documents import Document
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI

API Reference:

    Document
    StrOutputParser
    ChatPromptTemplate
    ChatOpenAI

chain = (
    {"doc": lambda x: x.page_content}
    | ChatPromptTemplate.from_template("Summarize the following document:\n\n{doc}")
    | ChatOpenAI(max_retries=0)
    | StrOutputParser()
)

summaries = chain.batch(docs, {"max_concurrency": 5})

# The vectorstore to use to index the child chunks
vectorstore = Chroma(collection_name="summaries", embedding_function=OpenAIEmbeddings())
# The storage layer for the parent documents
store = InMemoryByteStore()
id_key = "doc_id"
# The retriever (empty to start)
retriever = MultiVectorRetriever(
    vectorstore=vectorstore,
    byte_store=store,
    id_key=id_key,
)
doc_ids = [str(uuid.uuid4()) for _ in docs]

summary_docs = [
    Document(page_content=s, metadata={id_key: doc_ids[i]})
    for i, s in enumerate(summaries)
]

retriever.vectorstore.add_documents(summary_docs)
retriever.docstore.mset(list(zip(doc_ids, docs)))

# # We can also add the original chunks to the vectorstore if we so want
# for i, doc in enumerate(docs):
#     doc.metadata[id_key] = doc_ids[i]
# retriever.vectorstore.add_documents(docs)

sub_docs = vectorstore.similarity_search("justice breyer")

sub_docs[0]

Document(page_content="The document is a speech given by President Biden addressing various issues and outlining his agenda for the nation. He highlights the importance of nominating a Supreme Court justice and introduces his nominee, Judge Ketanji Brown Jackson. He emphasizes the need to secure the border and reform the immigration system, including providing a pathway to citizenship for Dreamers and essential workers. The President also discusses the protection of women's rights, including access to healthcare and the right to choose. He calls for the passage of the Equality Act to protect LGBTQ+ rights. Additionally, President Biden discusses the need to address the opioid epidemic, improve mental health services, support veterans, and fight against cancer. He expresses optimism for the future of America and the strength of the American people.", metadata={'doc_id': '56345bff-3ead-418c-a4ff-dff203f77474'})

retrieved_docs = retriever.invoke("justice breyer")

len(retrieved_docs[0].page_content)

9194

Hypothetical Queries

An LLM can also be used to generate a list of hypothetical questions that could be asked of a particular document. These questions can then be embedded

functions = [
    {
        "name": "hypothetical_questions",
        "description": "Generate hypothetical questions",
        "parameters": {
            "type": "object",
            "properties": {
                "questions": {
                    "type": "array",
                    "items": {"type": "string"},
                },
            },
            "required": ["questions"],
        },
    }
]

from langchain.output_parsers.openai_functions import JsonKeyOutputFunctionsParser

chain = (
    {"doc": lambda x: x.page_content}
    # Only asking for 3 hypothetical questions, but this could be adjusted
    | ChatPromptTemplate.from_template(
        "Generate a list of exactly 3 hypothetical questions that the below document could be used to answer:\n\n{doc}"
    )
    | ChatOpenAI(max_retries=0, model="gpt-4").bind(
        functions=functions, function_call={"name": "hypothetical_questions"}
    )
    | JsonKeyOutputFunctionsParser(key_name="questions")
)

API Reference:

    JsonKeyOutputFunctionsParser

chain.invoke(docs[0])

["What was the author's first experience with programming like?",
 'Why did the author switch their focus from AI to Lisp during their graduate studies?',
 'What led the author to contemplate a career in art instead of computer science?']

hypothetical_questions = chain.batch(docs, {"max_concurrency": 5})

# The vectorstore to use to index the child chunks
vectorstore = Chroma(
    collection_name="hypo-questions", embedding_function=OpenAIEmbeddings()
)
# The storage layer for the parent documents
store = InMemoryByteStore()
id_key = "doc_id"
# The retriever (empty to start)
retriever = MultiVectorRetriever(
    vectorstore=vectorstore,
    byte_store=store,
    id_key=id_key,
)
doc_ids = [str(uuid.uuid4()) for _ in docs]

question_docs = []
for i, question_list in enumerate(hypothetical_questions):
    question_docs.extend(
        [Document(page_content=s, metadata={id_key: doc_ids[i]}) for s in question_list]
    )

retriever.vectorstore.add_documents(question_docs)
retriever.docstore.mset(list(zip(doc_ids, docs)))

sub_docs = vectorstore.similarity_search("justice breyer")

sub_docs

[Document(page_content='Who has been nominated to serve on the United States Supreme Court?', metadata={'doc_id': '0b3a349e-c936-4e77-9c40-0a39fc3e07f0'}),
 Document(page_content="What was the context and content of Robert Morris' advice to the document's author in 2010?", metadata={'doc_id': 'b2b2cdca-988a-4af1-ba47-46170770bc8c'}),
 Document(page_content='How did personal circumstances influence the decision to pass on the leadership of Y Combinator?', metadata={'doc_id': 'b2b2cdca-988a-4af1-ba47-46170770bc8c'}),
 Document(page_content='What were the reasons for the author leaving Yahoo in the summer of 1999?', metadata={'doc_id': 'ce4f4981-ca60-4f56-86f0-89466de62325'})]

retrieved_docs = retriever.invoke("justice breyer")

len(retrieved_docs[0].page_content)

9194


Contextual compression

One challenge with retrieval is that usually you don't know the specific queries your document storage system will face when you ingest data into the system. This means that the information most relevant to a query may be buried in a document with a lot of irrelevant text. Passing that full document through your application can lead to more expensive LLM calls and poorer responses.

Contextual compression is meant to fix this. The idea is simple: instead of immediately returning retrieved documents as-is, you can compress them using the context of the given query, so that only the relevant information is returned. “Compressing” here refers to both compressing the contents of an individual document and filtering out documents wholesale.

To use the Contextual Compression Retriever, you'll need:

    a base retriever
    a Document Compressor

The Contextual Compression Retriever passes queries to the base retriever, takes the initial documents and passes them through the Document Compressor. The Document Compressor takes a list of documents and shortens it by reducing the contents of documents or dropping documents altogether.

Get started

# Helper function for printing docs


def pretty_print_docs(docs):
    print(
        f"\n{'-' * 100}\n".join(
            [f"Document {i+1}:\n\n" + d.page_content for i, d in enumerate(docs)]
        )
    )

Using a vanilla vector store retriever

Let's start by initializing a simple vector store retriever and storing the 2023 State of the Union speech (in chunks). We can see that given an example question our retriever returns one or two relevant docs and a few irrelevant docs. And even the relevant docs have a lot of irrelevant information in them.

from langchain_community.document_loaders import TextLoader
from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings
from langchain_text_splitters import CharacterTextSplitter

documents = TextLoader("../../state_of_the_union.txt").load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
texts = text_splitter.split_documents(documents)
retriever = FAISS.from_documents(texts, OpenAIEmbeddings()).as_retriever()

docs = retriever.invoke("What did the president say about Ketanji Brown Jackson")
pretty_print_docs(docs)

API Reference:

    TextLoader
    FAISS
    OpenAIEmbeddings
    CharacterTextSplitter

Document 1:

Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. 

Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. 

One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. 

And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.
----------------------------------------------------------------------------------------------------
Document 2:

A former top litigator in private practice. A former federal public defender. And from a family of public school educators and police officers. A consensus builder. Since she’s been nominated, she’s received a broad range of support—from the Fraternal Order of Police to former judges appointed by Democrats and Republicans. 

And if we are to advance liberty and justice, we need to secure the Border and fix the immigration system. 

We can do both. At our border, we’ve installed new technology like cutting-edge scanners to better detect drug smuggling.  

We’ve set up joint patrols with Mexico and Guatemala to catch more human traffickers.  

We’re putting in place dedicated immigration judges so families fleeing persecution and violence can have their cases heard faster. 

We’re securing commitments and supporting partners in South and Central America to host more refugees and secure their own borders.
----------------------------------------------------------------------------------------------------
Document 3:

And for our LGBTQ+ Americans, let’s finally get the bipartisan Equality Act to my desk. The onslaught of state laws targeting transgender Americans and their families is wrong. 

As I said last year, especially to our younger transgender Americans, I will always have your back as your President, so you can be yourself and reach your God-given potential. 

While it often appears that we never agree, that isn’t true. I signed 80 bipartisan bills into law last year. From preventing government shutdowns to protecting Asian-Americans from still-too-common hate crimes to reforming military justice. 

And soon, we’ll strengthen the Violence Against Women Act that I first wrote three decades ago. It is important for us to show the nation that we can come together and do big things. 

So tonight I’m offering a Unity Agenda for the Nation. Four big things we can do together.  

First, beat the opioid epidemic.
----------------------------------------------------------------------------------------------------
Document 4:

Tonight, I’m announcing a crackdown on these companies overcharging American businesses and consumers. 

And as Wall Street firms take over more nursing homes, quality in those homes has gone down and costs have gone up.  

That ends on my watch. 

Medicare is going to set higher standards for nursing homes and make sure your loved ones get the care they deserve and expect. 

We’ll also cut costs and keep the economy going strong by giving workers a fair shot, provide more training and apprenticeships, hire them based on their skills not degrees. 

Let’s pass the Paycheck Fairness Act and paid leave.  

Raise the minimum wage to $15 an hour and extend the Child Tax Credit, so no one has to raise a family in poverty. 

Let’s increase Pell Grants and increase our historic support of HBCUs, and invest in what Jill—our First Lady who teaches full-time—calls America’s best-kept secret: community colleges.

Adding contextual compression with an LLMChainExtractor

Now let's wrap our base retriever with a ContextualCompressionRetriever. We'll add an LLMChainExtractor, which will iterate over the initially returned documents and extract from each only the content that is relevant to the query.

from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import LLMChainExtractor
from langchain_openai import OpenAI

llm = OpenAI(temperature=0)
compressor = LLMChainExtractor.from_llm(llm)
compression_retriever = ContextualCompressionRetriever(
    base_compressor=compressor, base_retriever=retriever
)

compressed_docs = compression_retriever.invoke(
    "What did the president say about Ketanji Jackson Brown"
)
pretty_print_docs(compressed_docs)

API Reference:

    ContextualCompressionRetriever
    LLMChainExtractor
    OpenAI

/Users/harrisonchase/workplace/langchain/libs/langchain/langchain/chains/llm.py:316: UserWarning: The predict_and_parse method is deprecated, instead pass an output parser directly to LLMChain.
  warnings.warn(
/Users/harrisonchase/workplace/langchain/libs/langchain/langchain/chains/llm.py:316: UserWarning: The predict_and_parse method is deprecated, instead pass an output parser directly to LLMChain.
  warnings.warn(
/Users/harrisonchase/workplace/langchain/libs/langchain/langchain/chains/llm.py:316: UserWarning: The predict_and_parse method is deprecated, instead pass an output parser directly to LLMChain.
  warnings.warn(
/Users/harrisonchase/workplace/langchain/libs/langchain/langchain/chains/llm.py:316: UserWarning: The predict_and_parse method is deprecated, instead pass an output parser directly to LLMChain.
  warnings.warn(
``````output
Document 1:

I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson.

More built-in compressors: filters
LLMChainFilter

The LLMChainFilter is slightly simpler but more robust compressor that uses an LLM chain to decide which of the initially retrieved documents to filter out and which ones to return, without manipulating the document contents.

from langchain.retrievers.document_compressors import LLMChainFilter

_filter = LLMChainFilter.from_llm(llm)
compression_retriever = ContextualCompressionRetriever(
    base_compressor=_filter, base_retriever=retriever
)

compressed_docs = compression_retriever.invoke(
    "What did the president say about Ketanji Jackson Brown"
)
pretty_print_docs(compressed_docs)

API Reference:

    LLMChainFilter

/Users/harrisonchase/workplace/langchain/libs/langchain/langchain/chains/llm.py:316: UserWarning: The predict_and_parse method is deprecated, instead pass an output parser directly to LLMChain.
  warnings.warn(
/Users/harrisonchase/workplace/langchain/libs/langchain/langchain/chains/llm.py:316: UserWarning: The predict_and_parse method is deprecated, instead pass an output parser directly to LLMChain.
  warnings.warn(
/Users/harrisonchase/workplace/langchain/libs/langchain/langchain/chains/llm.py:316: UserWarning: The predict_and_parse method is deprecated, instead pass an output parser directly to LLMChain.
  warnings.warn(
/Users/harrisonchase/workplace/langchain/libs/langchain/langchain/chains/llm.py:316: UserWarning: The predict_and_parse method is deprecated, instead pass an output parser directly to LLMChain.
  warnings.warn(
``````output
Document 1:

Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. 

Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. 

One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. 

And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.

EmbeddingsFilter

Making an extra LLM call over each retrieved document is expensive and slow. The EmbeddingsFilter provides a cheaper and faster option by embedding the documents and query and only returning those documents which have sufficiently similar embeddings to the query.

from langchain.retrievers.document_compressors import EmbeddingsFilter
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings()
embeddings_filter = EmbeddingsFilter(embeddings=embeddings, similarity_threshold=0.76)
compression_retriever = ContextualCompressionRetriever(
    base_compressor=embeddings_filter, base_retriever=retriever
)

compressed_docs = compression_retriever.invoke(
    "What did the president say about Ketanji Jackson Brown"
)
pretty_print_docs(compressed_docs)

API Reference:

    EmbeddingsFilter
    OpenAIEmbeddings

Document 1:

Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. 

Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. 

One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. 

And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.
----------------------------------------------------------------------------------------------------
Document 2:

A former top litigator in private practice. A former federal public defender. And from a family of public school educators and police officers. A consensus builder. Since she’s been nominated, she’s received a broad range of support—from the Fraternal Order of Police to former judges appointed by Democrats and Republicans. 

And if we are to advance liberty and justice, we need to secure the Border and fix the immigration system. 

We can do both. At our border, we’ve installed new technology like cutting-edge scanners to better detect drug smuggling.  

We’ve set up joint patrols with Mexico and Guatemala to catch more human traffickers.  

We’re putting in place dedicated immigration judges so families fleeing persecution and violence can have their cases heard faster. 

We’re securing commitments and supporting partners in South and Central America to host more refugees and secure their own borders.
----------------------------------------------------------------------------------------------------
Document 3:

And for our LGBTQ+ Americans, let’s finally get the bipartisan Equality Act to my desk. The onslaught of state laws targeting transgender Americans and their families is wrong. 

As I said last year, especially to our younger transgender Americans, I will always have your back as your President, so you can be yourself and reach your God-given potential. 

While it often appears that we never agree, that isn’t true. I signed 80 bipartisan bills into law last year. From preventing government shutdowns to protecting Asian-Americans from still-too-common hate crimes to reforming military justice. 

And soon, we’ll strengthen the Violence Against Women Act that I first wrote three decades ago. It is important for us to show the nation that we can come together and do big things. 

So tonight I’m offering a Unity Agenda for the Nation. Four big things we can do together.  

First, beat the opioid epidemic.

Stringing compressors and document transformers together

Using the DocumentCompressorPipeline we can also easily combine multiple compressors in sequence. Along with compressors we can add BaseDocumentTransformers to our pipeline, which don't perform any contextual compression but simply perform some transformation on a set of documents. For example TextSplitters can be used as document transformers to split documents into smaller pieces, and the EmbeddingsRedundantFilter can be used to filter out redundant documents based on embedding similarity between documents.

Below we create a compressor pipeline by first splitting our docs into smaller chunks, then removing redundant documents, and then filtering based on relevance to the query.

from langchain.retrievers.document_compressors import DocumentCompressorPipeline
from langchain_community.document_transformers import EmbeddingsRedundantFilter
from langchain_text_splitters import CharacterTextSplitter

splitter = CharacterTextSplitter(chunk_size=300, chunk_overlap=0, separator=". ")
redundant_filter = EmbeddingsRedundantFilter(embeddings=embeddings)
relevant_filter = EmbeddingsFilter(embeddings=embeddings, similarity_threshold=0.76)
pipeline_compressor = DocumentCompressorPipeline(
    transformers=[splitter, redundant_filter, relevant_filter]
)

API Reference:

    DocumentCompressorPipeline
    EmbeddingsRedundantFilter
    CharacterTextSplitter

compression_retriever = ContextualCompressionRetriever(
    base_compressor=pipeline_compressor, base_retriever=retriever
)

compressed_docs = compression_retriever.invoke(
    "What did the president say about Ketanji Jackson Brown"
)
pretty_print_docs(compressed_docs)

Document 1:

One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. 

And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson
----------------------------------------------------------------------------------------------------
Document 2:

As I said last year, especially to our younger transgender Americans, I will always have your back as your President, so you can be yourself and reach your God-given potential. 

While it often appears that we never agree, that isn’t true. I signed 80 bipartisan bills into law last year
----------------------------------------------------------------------------------------------------
Document 3:

A former top litigator in private practice. A former federal public defender. And from a family of public school educators and police officers. A consensus builder
----------------------------------------------------------------------------------------------------
Document 4:

Since she’s been nominated, she’s received a broad range of support—from the Fraternal Order of Police to former judges appointed by Democrats and Republicans. 

And if we are to advance liberty and justice, we need to secure the Border and fix the immigration system. 

We can do both

Help us out by providing feedback on this documentation page:


Self-querying
info

Head to Integrations for documentation on vector stores with built-in support for self-querying.

A self-querying retriever is one that, as the name suggests, has the ability to query itself. Specifically, given any natural language query, the retriever uses a query-constructing LLM chain to write a structured query and then applies that structured query to its underlying VectorStore. This allows the retriever to not only use the user-input query for semantic similarity comparison with the contents of stored documents but to also extract filters from the user query on the metadata of stored documents and to execute those filters.

Get started

For demonstration purposes we'll use a Chroma vector store. We've created a small demo set of documents that contain summaries of movies.

Note: The self-query retriever requires you to have lark package installed.

%pip install --upgrade --quiet  lark langchain-chroma

from langchain_chroma import Chroma
from langchain_core.documents import Document
from langchain_openai import OpenAIEmbeddings

docs = [
    Document(
        page_content="A bunch of scientists bring back dinosaurs and mayhem breaks loose",
        metadata={"year": 1993, "rating": 7.7, "genre": "science fiction"},
    ),
    Document(
        page_content="Leo DiCaprio gets lost in a dream within a dream within a dream within a ...",
        metadata={"year": 2010, "director": "Christopher Nolan", "rating": 8.2},
    ),
    Document(
        page_content="A psychologist / detective gets lost in a series of dreams within dreams within dreams and Inception reused the idea",
        metadata={"year": 2006, "director": "Satoshi Kon", "rating": 8.6},
    ),
    Document(
        page_content="A bunch of normal-sized women are supremely wholesome and some men pine after them",
        metadata={"year": 2019, "director": "Greta Gerwig", "rating": 8.3},
    ),
    Document(
        page_content="Toys come alive and have a blast doing so",
        metadata={"year": 1995, "genre": "animated"},
    ),
    Document(
        page_content="Three men walk into the Zone, three men walk out of the Zone",
        metadata={
            "year": 1979,
            "director": "Andrei Tarkovsky",
            "genre": "thriller",
            "rating": 9.9,
        },
    ),
]
vectorstore = Chroma.from_documents(docs, OpenAIEmbeddings())

API Reference:

    Document
    OpenAIEmbeddings

Creating our self-querying retriever

Now we can instantiate our retriever. To do this we'll need to provide some information upfront about the metadata fields that our documents support and a short description of the document contents.

from langchain.chains.query_constructor.base import AttributeInfo
from langchain.retrievers.self_query.base import SelfQueryRetriever
from langchain_openai import ChatOpenAI

metadata_field_info = [
    AttributeInfo(
        name="genre",
        description="The genre of the movie. One of ['science fiction', 'comedy', 'drama', 'thriller', 'romance', 'action', 'animated']",
        type="string",
    ),
    AttributeInfo(
        name="year",
        description="The year the movie was released",
        type="integer",
    ),
    AttributeInfo(
        name="director",
        description="The name of the movie director",
        type="string",
    ),
    AttributeInfo(
        name="rating", description="A 1-10 rating for the movie", type="float"
    ),
]
document_content_description = "Brief summary of a movie"
llm = ChatOpenAI(temperature=0)
retriever = SelfQueryRetriever.from_llm(
    llm,
    vectorstore,
    document_content_description,
    metadata_field_info,
)

API Reference:

    AttributeInfo
    SelfQueryRetriever
    ChatOpenAI

Testing it out

And now we can actually try using our retriever!

# This example only specifies a filter
retriever.invoke("I want to watch a movie rated higher than 8.5")

[Document(page_content='Three men walk into the Zone, three men walk out of the Zone', metadata={'director': 'Andrei Tarkovsky', 'genre': 'thriller', 'rating': 9.9, 'year': 1979}),
 Document(page_content='A psychologist / detective gets lost in a series of dreams within dreams within dreams and Inception reused the idea', metadata={'director': 'Satoshi Kon', 'rating': 8.6, 'year': 2006})]

# This example specifies a query and a filter
retriever.invoke("Has Greta Gerwig directed any movies about women")

[Document(page_content='A bunch of normal-sized women are supremely wholesome and some men pine after them', metadata={'director': 'Greta Gerwig', 'rating': 8.3, 'year': 2019})]

# This example specifies a composite filter
retriever.invoke("What's a highly rated (above 8.5) science fiction film?")

[Document(page_content='A psychologist / detective gets lost in a series of dreams within dreams within dreams and Inception reused the idea', metadata={'director': 'Satoshi Kon', 'rating': 8.6, 'year': 2006}),
 Document(page_content='Three men walk into the Zone, three men walk out of the Zone', metadata={'director': 'Andrei Tarkovsky', 'genre': 'thriller', 'rating': 9.9, 'year': 1979})]

# This example specifies a query and composite filter
retriever.invoke(
    "What's a movie after 1990 but before 2005 that's all about toys, and preferably is animated"
)

[Document(page_content='Toys come alive and have a blast doing so', metadata={'genre': 'animated', 'year': 1995})]

Filter k

We can also use the self query retriever to specify k: the number of documents to fetch.

We can do this by passing enable_limit=True to the constructor.

retriever = SelfQueryRetriever.from_llm(
    llm,
    vectorstore,
    document_content_description,
    metadata_field_info,
    enable_limit=True,
)

# This example only specifies a relevant query
retriever.invoke("What are two movies about dinosaurs")

[Document(page_content='A bunch of scientists bring back dinosaurs and mayhem breaks loose', metadata={'genre': 'science fiction', 'rating': 7.7, 'year': 1993}),
 Document(page_content='Toys come alive and have a blast doing so', metadata={'genre': 'animated', 'year': 1995})]

Constructing from scratch with LCEL

To see what's going on under the hood, and to have more custom control, we can reconstruct our retriever from scratch.

First, we need to create a query-construction chain. This chain will take a user query and generated a StructuredQuery object which captures the filters specified by the user. We provide some helper functions for creating a prompt and output parser. These have a number of tunable params that we'll ignore here for simplicity.

from langchain.chains.query_constructor.base import (
    StructuredQueryOutputParser,
    get_query_constructor_prompt,
)

prompt = get_query_constructor_prompt(
    document_content_description,
    metadata_field_info,
)
output_parser = StructuredQueryOutputParser.from_components()
query_constructor = prompt | llm | output_parser

API Reference:

    StructuredQueryOutputParser
    get_query_constructor_prompt

Let's look at our prompt:

print(prompt.format(query="dummy question"))

Your goal is to structure the user's query to match the request schema provided below.

<< Structured Request Schema >>
When responding use a markdown code snippet with a JSON object formatted in the following schema:

```json
{
    "query": string \ text string to compare to document contents
    "filter": string \ logical condition statement for filtering documents
}

The query string should contain only text that is expected to match the contents of documents. Any conditions in the filter should not be mentioned in the query as well.

A logical condition statement is composed of one or more comparison and logical operation statements.

A comparison statement takes the form: comp(attr, val):

    comp (eq | ne | gt | gte | lt | lte | contain | like | in | nin): comparator
    attr (string): name of attribute to apply the comparison to
    val (string): is the comparison value

A logical operation statement takes the form op(statement1, statement2, ...):

    op (and | or | not): logical operator
    statement1, statement2, ... (comparison statements or logical operation statements): one or more statements to apply the operation to

Make sure that you only use the comparators and logical operators listed above and no others. Make sure that filters only refer to attributes that exist in the data source. Make sure that filters only use the attributed names with its function names if there are functions applied on them. Make sure that filters only use format YYYY-MM-DD when handling date data typed values. Make sure that filters take into account the descriptions of attributes and only make comparisons that are feasible given the type of data being stored. Make sure that filters are only used as needed. If there are no filters that should be applied return "NO_FILTER" for the filter value.

<< Example 1. >> Data Source:

{
    "content": "Lyrics of a song",
    "attributes": {
        "artist": {
            "type": "string",
            "description": "Name of the song artist"
        },
        "length": {
            "type": "integer",
            "description": "Length of the song in seconds"
        },
        "genre": {
            "type": "string",
            "description": "The song genre, one of "pop", "rock" or "rap""
        }
    }
}

User Query: What are songs by Taylor Swift or Katy Perry about teenage romance under 3 minutes long in the dance pop genre

Structured Request:

{
    "query": "teenager love",
    "filter": "and(or(eq(\"artist\", \"Taylor Swift\"), eq(\"artist\", \"Katy Perry\")), lt(\"length\", 180), eq(\"genre\", \"pop\"))"
}

<< Example 2. >> Data Source:

{
    "content": "Lyrics of a song",
    "attributes": {
        "artist": {
            "type": "string",
            "description": "Name of the song artist"
        },
        "length": {
            "type": "integer",
            "description": "Length of the song in seconds"
        },
        "genre": {
            "type": "string",
            "description": "The song genre, one of "pop", "rock" or "rap""
        }
    }
}

User Query: What are songs that were not published on Spotify

Structured Request:

{
    "query": "",
    "filter": "NO_FILTER"
}

<< Example 3. >> Data Source:

{
    "content": "Brief summary of a movie",
    "attributes": {
    "genre": {
        "description": "The genre of the movie. One of ['science fiction', 'comedy', 'drama', 'thriller', 'romance', 'action', 'animated']",
        "type": "string"
    },
    "year": {
        "description": "The year the movie was released",
        "type": "integer"
    },
    "director": {
        "description": "The name of the movie director",
        "type": "string"
    },
    "rating": {
        "description": "A 1-10 rating for the movie",
        "type": "float"
    }
}
}

User Query: dummy question

Structured Request:

And what our full chain produces:


```python
query_constructor.invoke(
    {
        "query": "What are some sci-fi movies from the 90's directed by Luc Besson about taxi drivers"
    }
)

StructuredQuery(query='taxi driver', filter=Operation(operator=<Operator.AND: 'and'>, arguments=[Comparison(comparator=<Comparator.EQ: 'eq'>, attribute='genre', value='science fiction'), Operation(operator=<Operator.AND: 'and'>, arguments=[Comparison(comparator=<Comparator.GTE: 'gte'>, attribute='year', value=1990), Comparison(comparator=<Comparator.LT: 'lt'>, attribute='year', value=2000)]), Comparison(comparator=<Comparator.EQ: 'eq'>, attribute='director', value='Luc Besson')]), limit=None)

The query constructor is the key element of the self-query retriever. To make a great retrieval system you'll need to make sure your query constructor works well. Often this requires adjusting the prompt, the examples in the prompt, the attribute descriptions, etc. For an example that walks through refining a query constructor on some hotel inventory data, check out this cookbook.

The next key element is the structured query translator. This is the object responsible for translating the generic StructuredQuery object into a metadata filter in the syntax of the vector store you're using. LangChain comes with a number of built-in translators. To see them all head to the Integrations section.

from langchain.retrievers.self_query.chroma import ChromaTranslator

retriever = SelfQueryRetriever(
    query_constructor=query_constructor,
    vectorstore=vectorstore,
    structured_query_translator=ChromaTranslator(),
)

API Reference:

    ChromaTranslator

retriever.invoke(
    "What's a movie after 1990 but before 2005 that's all about toys, and preferably is animated"
)

[Document(page_content='Toys come alive and have a blast doing so', metadata={'genre': 'animated', 'year': 1995})]

Help us out by providing feedback on this documentation page:



and here is my code:
import os
import json
import shutil
import gradio as gr
from itertools import chain
from langchain_community.embeddings.sentence_transformer import SentenceTransformerEmbeddings
from langchain_unstructured import UnstructuredLoader
from langchain_core.messages import HumanMessage, SystemMessage, AIMessage
from langchain_text_splitters import CharacterTextSplitter
from langchain_community.vectorstores import Chroma
from langchain_community.retrievers import ArxivRetriever
from langchain_community.tools import DuckDuckGoSearchRun
from langchain_community.document_loaders import DirectoryLoader
from langchain_ollama import ChatOllama

# For LangSmith tracing
from langsmith import traceable
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_API_KEY"] = "<TRUFFLEHOG_REDACTED_LANGSMITH>"  # Replace with your actual API key
os.environ["LANGCHAIN_PROJECT"] = "l"  # Optional: "default" is used if not set

# Initialize embeddings and vector store
embeddings = SentenceTransformerEmbeddings(model_name="all-MiniLM-L6-v2")
persist_directory = "./chroma_db"
vectorstore = Chroma(persist_directory=persist_directory, embedding_function=embeddings)

# Ensure the 'uploads' directory exists
if not os.path.exists("uploads"):
    os.makedirs("uploads")

# Function to load or initialize knowledge base JSON
def load_knowledge_base_json():
    if os.path.exists("knowledge_base.json"):
        with open("knowledge_base.json", "r") as f:
            knowledge_base_data = json.load(f)
    else:
        knowledge_base_data = {"files": [], "urls": []}
    return knowledge_base_data

knowledge_base_data = load_knowledge_base_json()

# Function to save knowledge base JSON
def save_knowledge_base_json():
    with open("knowledge_base.json", "w") as f:
        json.dump(knowledge_base_data, f)

# Function to get processed files
def get_processed_files():
    files = os.listdir("uploads")
    return "### Processed Files:\n" + "\n".join(files) if files else "No files processed yet."

# Function to display processed files and URLs
def display_processed_items():
    global knowledge_base_data
    files_text = get_processed_files()
    urls = knowledge_base_data.get("urls", [])
    display_text = files_text + "\n\n### Processed URLs:\n"
    if urls:
        display_text += "\n".join(urls)
    else:
        display_text += "No URLs processed yet."
    return display_text

# Function to update the knowledge base
@traceable
def update_knowledge_base(uploaded_files, website_urls):
    global knowledge_base_data
    if not uploaded_files and not website_urls:
        return "No new documents or URLs to process.", display_processed_items()

    processed_files = []
    skipped_files = []
    docs = []
    new_urls = []

    # Process uploaded files
    if uploaded_files:
        for file in uploaded_files:
            file_path = file.name
            destination_path = os.path.join("uploads", os.path.basename(file_path))
            file_name = os.path.basename(file_path)

            if os.path.exists(destination_path) or file_name in knowledge_base_data["files"]:
                skipped_files.append(file_name)
            else:
                try:
                    shutil.move(file_path, destination_path)
                    processed_files.append(file_name)
                except Exception as e:
                    print(f"Error moving file {file_name}: {str(e)}")

    # Load documents from uploaded files
    if processed_files:
        loader = DirectoryLoader("uploads", glob="**/*")
        documents = loader.load()
        docs.extend(documents)

    # Load documents from URLs
    if website_urls:
        urls = website_urls.split(',')
        for url in urls:
            url = url.strip()
            if url and url not in knowledge_base_data["urls"]:
                loader = UnstructuredLoader(web_url=url)
                try:
                    loaded_docs = loader.load()
                    docs.extend(loaded_docs)
                    new_urls.append(url)
                except Exception as e:
                    print(f"Error loading URL {url}: {str(e)}")
            else:
                skipped_files.append(url)

    # Split documents and add to vector store
    if docs:
        text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=200, length_function=len)
        split_docs = text_splitter.split_documents(docs)
        vectorstore.add_documents(split_docs)
        vectorstore.persist()
        # Update knowledge base data and save to JSON
        knowledge_base_data["files"].extend(processed_files)
        knowledge_base_data["urls"].extend(new_urls)
        save_knowledge_base_json()
        status_message = f"Processed {len(processed_files)} new files and {len(new_urls)} new URLs."
        if skipped_files:
            status_message += f"\nSkipped existing files/URLs: {', '.join(skipped_files)}"
    else:
        status_message = "No new documents or URLs were added."
    return status_message, display_processed_items()

# Function to clear database and uploaded files
def clear_database():
    global vectorstore, knowledge_base_data

    try:
        vectorstore._client._client.close()
    except AttributeError:
        pass

    shutil.rmtree("uploads", ignore_errors=True)
    os.makedirs("uploads", exist_ok=True)

    shutil.rmtree(persist_directory, ignore_errors=True)

    vectorstore = Chroma(persist_directory=persist_directory, embedding_function=embeddings)

    # Clear the knowledge base data
    knowledge_base_data = {"files": [], "urls": []}
    # Remove the JSON file
    if os.path.exists("knowledge_base.json"):
        os.remove("knowledge_base.json")

    return "Knowledge base and uploaded files cleared.", display_processed_items()

# Function to process uploaded files in the chat and include their content
def process_uploaded_files(files):
    content = ""
    if files:
        for file in files:
            if file.name.endswith((".pdf", ".txt", ".docx")):
                loader = UnstructuredLoader(file_path=file.name)
                try:
                    docs = loader.load()
                    content += "\n".join([doc.page_content for doc in docs])
                except Exception as e:
                    print(f"Error processing file {file.name}: {str(e)}")
    return content

# Function to count tokens
def count_tokens(text):
    return len(text.split())  # Simple token count; adjust with tokenizer if needed

# Initialize Llama model
llm = ChatOllama(
    model="llama3.1",
    temperature=0,
    base_url="http://localhost:11434",
    streaming=True
)

# Maximum token limit
MAX_TOKENS = 128000  # Example maximum token limit

# Chat with Llama (normal chat) function
@traceable
def chat_with_llama(message, history, uploaded_files):
    # Process uploaded files and include their content
    file_content = process_uploaded_files(uploaded_files)
    # Convert history to list of messages
    messages = [SystemMessage(content="You are a helpful assistant. The user is able to upload files that get processed as text for you. You always answer to the users question by using the context or your knowlage in the same language the user wants you to talk in.")]
    if history:
        for user_msg, assistant_msg in history:
            if user_msg:
                messages.append(HumanMessage(content=user_msg))
                if assistant_msg is not None:
                    messages.append(AIMessage(content=assistant_msg))
    # Append file content to user message if any
    if file_content:
        message += f"\n\nContent files Uploaded:\n{file_content}"
        message += "\n\nWith that knowlage answer to the question as best as possible: "
    messages.append(HumanMessage(content=message))
    # Append the Question at the end again
    # Check if the total tokens exceed the maximum allowed tokens
    total_tokens = sum(count_tokens(msg.content) for msg in messages)
    if total_tokens > MAX_TOKENS:
        # Truncate the history to fit within the token limit
        while total_tokens > MAX_TOKENS and len(messages) > 2:
            # Remove the oldest user-assistant message pair
            messages.pop(1)
            messages.pop(1)
            total_tokens = sum(count_tokens(msg.content) for msg in messages)
        if total_tokens > MAX_TOKENS:
            return "Error: The conversation exceeds the maximum token limit. Please start a new chat."
    # Stream the response from the model
    try:
        response_stream = llm.stream(messages)
        partial_response = ""
        for chunk in response_stream:
            partial_response += chunk.content
            yield partial_response
    except Exception as e:
        yield f"Error: {str(e)}"

# Function to handle knowledge base Q&A using similarity search
@traceable
def knowledge_base_chat(message, history):
    # Search in the vector store using similarity search
    results = vectorstore.similarity_search(message, k=3)
    if not results:
        yield "No relevant information found in the knowledge base."
        return
    context = "\n\n".join([doc.page_content for doc in results])
    # Build the conversation history
    messages = [SystemMessage(content="You are a helpful assistant that answers questions based on the provided context.")]
    if history:
        for user_msg, assistant_msg in history:
            if user_msg:
                messages.append(HumanMessage(content=user_msg))
                if assistant_msg is not None:
                    messages.append(AIMessage(content=assistant_msg))
    # Add the latest message with context
    prompt = f"Context:\n{context}\n\nQuestion:\n{message}\n\nAnswer:"
    messages.append(HumanMessage(content=prompt))
    # Stream the response from the model
    try:
        response_stream = llm.stream(messages)
        partial_response = ""
        for chunk in response_stream:
            partial_response += chunk.content
            # Use yield to stream the response
            yield partial_response
    except Exception as e:
        yield f"Error: {str(e)}"

# Function for the Internet-Enabled LLM
@traceable
def internet_access_llm(message, history, use_wikipedia):
    # Generate a query from the user input
    query = message
    # Search the internet
    search_tool = DuckDuckGoSearchRun()
    search_results = search_tool.invoke(query)
    # Optionally search Wikipedia
    if use_wikipedia:
        wikipedia_tool = DuckDuckGoSearchRun(backend="wikipedia")
        wiki_results = wikipedia_tool.invoke(query)
        search_results += "\n\n" + wiki_results
    # Build the conversation history
    messages = [SystemMessage(content="You get information from the internet and answer the user's question with that information. Provide an accurate and detailed answer to the user's question using the search results.")]
    if history:
        for user_msg, assistant_msg in history:
            if user_msg:
                messages.append(HumanMessage(content=user_msg))
                if assistant_msg is not None:
                    messages.append(AIMessage(content=assistant_msg))
    # Add the latest message with search results
    prompt = f"Search Results:\n{search_results}\n\nQuestion:\n{message}\n\nAnswer:"
    messages.append(HumanMessage(content=prompt))
    # Stream the response from the model
    try:
        response_stream = llm.stream(messages)
        partial_response = ""
        for chunk in response_stream:
            partial_response += chunk.content
            yield partial_response
    except Exception as e:
        yield f"Error: {str(e)}"

# Function to retrieve arXiv papers and update knowledge base
@traceable
def arxiv_research(query, max_papers):
    global knowledge_base_data
    retriever = ArxivRetriever(load_max_docs=int(max_papers), get_full_documents=True)
    docs = retriever.get_relevant_documents(query)
    # Add documents to the vector store
    if docs:
        text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=200, length_function=len)
        split_docs = text_splitter.split_documents(docs)
        vectorstore.add_documents(split_docs)
        vectorstore.persist()
        # Update knowledge base data and save to JSON
        arxiv_titles = [doc.metadata.get('Title', 'Unknown Title') for doc in docs]
        knowledge_base_data["files"].extend(arxiv_titles)
        save_knowledge_base_json()
        status_message = f"Added {len(docs)} papers to the knowledge base."
    else:
        status_message = "No documents were added from arXiv."
    return status_message, display_processed_items()

# Function to handle arXiv Knowledge Base Q&A using similarity search
@traceable
def ask_arxiv_knowledge_base(message, history):
    # Search in the vector store using similarity search
    results = vectorstore.similarity_search(message, k=3)
    if not results:
        yield "No relevant information found in the arXiv knowledge base."
        return
    context = "\n\n".join([doc.page_content for doc in results])
    # Build the conversation history
    messages = [SystemMessage(content="You are a helpful assistant that answers questions based on arXiv research papers.")]
    if history:
        for user_msg, assistant_msg in history:
            if user_msg:
                messages.append(HumanMessage(content=user_msg))
                if assistant_msg is not None:
                    messages.append(AIMessage(content=assistant_msg))
    # Add the latest message with context
    prompt = f"Context:\n{context}\n\nQuestion:\n{message}\n\nAnswer:"
    messages.append(HumanMessage(content=prompt))
    # Stream the response from the model
    try:
        response_stream = llm.stream(messages)
        partial_response = ""
        for chunk in response_stream:
            partial_response += chunk.content
            yield partial_response
    except Exception as e:
        yield f"Error: {str(e)}"

# Gradio interface
with gr.Blocks() as demo:
    gr.Markdown("# Multi-Tab Gradio App with Llama and Similarity Search")

    # Chat with Llama Tab
    with gr.Tab("Chat with Llama"):
        gr.Markdown("## Chat Interface")
        with gr.Row():
            upload_files = gr.File(
                label="Upload PDFs, DOCX or TXT files",
                file_types=[".pdf", ".txt", ".docx"],
                file_count="multiple",
                visible=True
            )
        gr_chat = gr.ChatInterface(
            fn=chat_with_llama,
            title="Chat with Llama",
            fill_height=True,  # Use fill_height instead of height
            additional_inputs=[upload_files]
        )

    # Knowledge Base Tab
    with gr.Tab("Knowledge Base"):
        gr.Markdown("## Knowledge Base Management and Chat")
        with gr.Row():
            with gr.Column(scale=1):
                gr.Markdown("### Update Knowledge Base")
                kb_upload_files = gr.File(
                    label="Upload Files",
                    file_types=[".pdf", ".txt", ".docx"],
                    file_count="multiple"
                )
                kb_website_urls = gr.Textbox(
                    label="Website URLs",
                    placeholder="Enter comma-separated URLs"
                )
                kb_update_button = gr.Button("Update Knowledge Base")
                kb_clear_button = gr.Button("Clear Database")
                kb_update_status = gr.Markdown()
                kb_update_button.click(
                    update_knowledge_base,
                    inputs=[kb_upload_files, kb_website_urls],
                    outputs=[kb_update_status, kb_update_status]
                )
                kb_clear_button.click(
                    clear_database,
                    outputs=[kb_update_status, kb_update_status]
                )
                gr.Markdown("### Processed Files and URLs")
                processed_items_display = gr.Markdown(display_processed_items())
                # Update the display when the knowledge base is updated or cleared
                kb_update_button.click(
                    update_knowledge_base,
                    inputs=[kb_upload_files, kb_website_urls],
                    outputs=[kb_update_status, processed_items_display]
                )
                kb_clear_button.click(
                    clear_database,
                    outputs=[kb_update_status, processed_items_display]
                )
            with gr.Column(scale=2):
                gr.Markdown("### Chat with Knowledge Base")
                gr_chat_kb = gr.ChatInterface(
                    fn=knowledge_base_chat,
                    title="Knowledge Base Chat",
                    fill_height=True  # Use fill_height instead of height
                )

    # LLM with Internet Access Tab
    with gr.Tab("LLM with Internet Access"):
        gr.Markdown("## Internet-Enabled LLM")
        use_wikipedia = gr.Checkbox(label="Use Wikipedia")
        gr_chat_internet = gr.ChatInterface(
            fn=internet_access_llm,
            title="LLM with Internet Access",
            additional_inputs=[use_wikipedia],
            fill_height=True  # Use fill_height instead of height
        )

    # arXiv Research Papers Tab
    with gr.Tab("arXiv Research Papers"):
        gr.Markdown("## Retrieve arXiv Papers and Chat")
        with gr.Row():
            with gr.Column(scale=1):
                gr.Markdown("### Retrieve arXiv Papers")
                arxiv_query = gr.Textbox(
                    label="Search Query",
                    placeholder="Enter your search query for arXiv papers"
                )
                max_papers = gr.Slider(
                    label="Number of Papers",
                    minimum=1,
                    maximum=10,
                    step=1,
                    value=5
                )
                arxiv_button = gr.Button("Retrieve Papers")
                arxiv_result = gr.Markdown()
                arxiv_button.click(
                    arxiv_research,
                    inputs=[arxiv_query, max_papers],
                    outputs=[arxiv_result, arxiv_result]
                )
                gr.Markdown("### Processed Files and URLs")
                processed_items_display_arxiv = gr.Markdown(display_processed_items())
                # Update the display when new papers are added
                arxiv_button.click(
                    arxiv_research,
                    inputs=[arxiv_query, max_papers],
                    outputs=[arxiv_result, processed_items_display_arxiv]
                )
            with gr.Column(scale=2):
                gr.Markdown("### Chat with arXiv Knowledge Base")
                gr_chat_arxiv = gr.ChatInterface(
                    fn=ask_arxiv_knowledge_base,
                    title="arXiv Knowledge Base Chat",
                    fill_height=True  # Use fill_height instead of height
                )

    # Apply CSS styling to adjust chatbox sizes if needed
    # Uncomment and adjust the CSS if 'fill_height' doesn't work as expected
    """
    demo.css = '''
    .gradio-chatbot {
        height: 600px !important;
    }
    '''
    """

    # Launch the app
    demo.launch()