Pass filters and other search_kwargs through retrieval chain at query time (#9195)
Labels: name: area: vector store, name: auto:improvement

**Description:**

Adds the ability to pass filters and other search_kwargs to a vector store when *calling* a conversational retriever . (#9195)

Existing functionality allows setting the search_kwargs only when *constructing* the retriever - which limits the ability to create a single retriever then alter filters during chat sessions.

My use case (backed by the others in the Issue) is to be able to ingest documents into a vector database with different 'tags', then during a long-running RAG0-based chat session, specify the tags to filter ingested documents on with each question.

The existing support for search_kwargs is still useful - args like 'k', the number of results, are likely to be definitely determined at construction time. Hence this fix *merges* the search_kwargs provided with the query with any that were set during construction. That is: query-time values *override* construction-time values, but only those with the same key. Hence you can set 'k', 'score_threshold', etc, during construction, then pass in a custom 'filter' with the query. 

Query-time search_kwargs are provided in the `inputs` parameter, but are not listed in the input_keys, since they are not mandatory. This preserves compatibility, and makes sense, given the search_kwargs, very much are inputs.

The documentation is updated with an example on the vector_db_qa.mdx page.


Details on the rationale behind this PR:

1. Many of the vector store implementations take “filter” as a parameter in their similarity_search (chroma, elastic, pgvector, open_search), and some, like weaviate, take more complex kwargs, accepting in them “where_filter”.
 We need a way to pass down arguments for these parameters from the original __call__ method on the base chain, in such a way as they get to - and only to - the vector stores that can use them.

2. The generic VectorStoreRetriever has a search_kwargs field (Pydantic) that is intended to hold just this sort of parameter, and passes it into similarty_search() (and other search methods). But is limited in that it can only be set when the VectorRetriever is constructed.  

3. The abstract BaseRetriever._get_relevant_documents doesn’t take extra args, but BaseRetriever allows for the concrete implementations in different retrievers to have extra args (which only just slightly freaks me out as an old OO Java programmer :)). BaseRetriever detects this and sets _expects_other_args to True which causes it to pass on kwargs from its get_relevant_documents() method to the specific retrievers _get_relevant_documents implementation. 

So… to get our filter down where it’s needed, we need to get it in a more dynamic search_kwargs dictionary, and get that merged with the one passed in from VectorStoreRetriever, without interfering with the base retriever.

For this we need one change to VectoreStoreRetriever:
- add a search_kwargs dictionary to its _get_relevant_documents method
- merge that with the base search kwargs (dictionary update - to override, treating the construction-time search_kwargs as defaults

We have to change one more class though, to get the search_kwargs into the get_relevant_documents incoming kwargs parameter: we can do that in ConversationalRetrievalChain._get_docs which currently does not call it with any kwargs. But  _get_docs just happens to get the inputs dictionary from which history and question are taken, which is passed in from the original __call__. Hence we:
- Modifiy ConversationalRetrievalChain._get_docs to retrieve the search_kwargs, if there are any, from the inputs[] list and add them to the kwargs going into get_relevant_documents

So, we can put search_kwargs in inputs- because they are just as much inputs as the question and chat_history, then we can extract them in _get_docs and pass them into  get_relevant_documents, and we get them where they need to go with only two changes.

The modifications needed for base retrievers is slightly different, but similar enough.

**Issue:**
Passing filter through ConversationalRetrievalChain to the underlying vector store  #9195


**Tag maintainer:** @baskaryan