Pprados/rag vectorstore
Labels: name: area: vector store, name: auto:enhancement, name: size:XXL

**Description:**
The `ParentDocumentRetriever` class provides the `add_documents()` methods, which may seem unconventional for a retriever.

This deviation arises from the need to establish and maintain connections between document fragments and their various iterations. Consequently, the `parent_splitter` and `child_splitter` were introduced. As elaborated in this [pull request](https://github.com/langchain-ai/langchain/pull/11968), it's crucial not to confine oneself to text splitters exclusively. Instead, a preference is given to working with *transformations*, from which text splitters inherit.

However, my [initial proposal](https://github.com/langchain-ai/langchain/pull/11968) proved insufficient to fully leverage this concept with all the advanced features of Langchain. An [issue](https://github.com/langchain-ai/langchain/issues/11983) has been raised to address this limitation.

To enhance text selection, the proposed idea by `ParentDocumentRetriever` is as follows:
- Convert each text fragment into multiple versions to enhance proximity between questions and potential answers.
- Select corresponding fragments in the vector store.
- Return the original fragment for injection into the prompt.

For this purpose, maintaining a link between the original document, each document chunk, and every transformation 
applied to each chunk is essential. 

**My Proposition:**
The `RagVectorStore` class serves as a vector store wrapper with rules for transforming a document 
into a chunk and each chunk into different versions. It closely resembles `ParentRetriever` but employs 
a `VectorStore` instead of a `Retriever`.

This approach is advantageous due to its simplicity and compatibility with all advanced features of Langchain. 
Feel free to explore the benefits of this solution in [this notebook](https://github.com/pprados/langchain-rag/blob/master/docs/integrations/vectorstores/rag_vectorstore.ipynb).

Compatibility includes:
- [Indexing API](https://python.langchain.com/docs/modules/data_connection/indexing)
- Advanced retrievers like [MultiQueryRetriever](https://python.langchain.com/docs/modules/data_connection/retrievers/MultiQueryRetriever), [Self Querying](https://python.langchain.com/docs/modules/data_connection/retrievers/self_query), etc.

The implementation required several changes and additional pull requests, each of which can be validated independently before merging into this one:
- [Adds an in-memory implementation of RecordStore](https://github.com/langchain-ai/langchain/pull/13200)
- [Add SQLDocStore](https://github.com/langchain-ai/langchain/pull/13181)
- [Add a Wrapper vectorstore, compatible with SelfQueryRetriever](https://github.com/langchain-ai/langchain/pull/13190)
- [Add *LCEL* Transformers](https://github.com/langchain-ai/langchain/pull/13819)

**Issues:**
- [Pull Request: 11968](https://github.com/langchain-ai/langchain/pull/11968)
- [Issue: 11983](https://github.com/langchain-ai/langchain/issues/11983)

**Dependencies:** 
Any dependencies required for this change.

**Tag Maintainers:**
@baskaryan @hwchase17

**Twitter Handle:**
Twitter account: pprados