{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to combine results from multiple retrievers\n",
    "\n",
    "The [EnsembleRetriever](https://api.python.langchain.com/en/latest/retrievers/langchain.retrievers.ensemble.EnsembleRetriever.html) supports ensembling of results from multiple retrievers. It is initialized with a list of [BaseRetriever](https://api.python.langchain.com/en/latest/retrievers/langchain_core.retrievers.BaseRetriever.html) objects. EnsembleRetrievers rerank the results of the constituent retrievers based on the [Reciprocal Rank Fusion](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf) algorithm.\n",
    "\n",
    "By leveraging the strengths of different algorithms, the `EnsembleRetriever` can achieve better performance than any single algorithm. \n",
    "\n",
    "The most common pattern is to combine a sparse retriever (like BM25) with a dense retriever (like embedding similarity), because their strengths are complementary. It is also known as \"hybrid search\". The sparse retriever is good at finding relevant documents based on keywords, while the dense retriever is good at finding relevant documents based on semantic similarity.\n",
    "\n",
    "## Basic usage\n",
    "\n",
    "Below we demonstrate ensembling of a [BM25Retriever](https://api.python.langchain.com/en/latest/retrievers/langchain_community.retrievers.bm25.BM25Retriever.html) with a retriever derived from the [FAISS vector store](https://api.python.langchain.com/en/latest/vectorstores/langchain_community.vectorstores.faiss.FAISS.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install --upgrade --quiet  rank_bm25 > /dev/null"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.retrievers import EnsembleRetriever\n",
    "from langchain_community.retrievers import BM25Retriever\n",
    "from langchain_community.vectorstores import FAISS\n",
    "from langchain_openai import OpenAIEmbeddings\n",
    "\n",
    "doc_list_1 = [\n",
    "    \"I like apples\",\n",
    "    \"I like oranges\",\n",
    "    \"Apples and oranges are fruits\",\n",
    "]\n",
    "\n",
    "# initialize the bm25 retriever and faiss retriever\n",
    "bm25_retriever = BM25Retriever.from_texts(\n",
    "    doc_list_1, metadatas=[{\"source\": 1}] * len(doc_list_1)\n",
    ")\n",
    "bm25_retriever.k = 2\n",
    "\n",
    "doc_list_2 = [\n",
    "    \"You like apples\",\n",
    "    \"You like oranges\",\n",
    "]\n",
    "\n",
    "embedding = OpenAIEmbeddings()\n",
    "faiss_vectorstore = FAISS.from_texts(\n",
    "    doc_list_2, embedding, metadatas=[{\"source\": 2}] * len(doc_list_2)\n",
    ")\n",
    "faiss_retriever = faiss_vectorstore.as_retriever(search_kwargs={\"k\": 2})\n",
    "\n",
    "# initialize the ensemble retriever\n",
    "ensemble_retriever = EnsembleRetriever(\n",
    "    retrievers=[bm25_retriever, faiss_retriever], weights=[0.5, 0.5]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='I like apples', metadata={'source': 1}),\n",
       " Document(page_content='You like apples', metadata={'source': 2}),\n",
       " Document(page_content='Apples and oranges are fruits', metadata={'source': 1}),\n",
       " Document(page_content='You like oranges', metadata={'source': 2})]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "docs = ensemble_retriever.invoke(\"apples\")\n",
    "docs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Runtime Configuration\n",
    "\n",
    "We can also configure the individual retrievers at runtime using [configurable fields](/docs/how_to/configure). Below we update the \"top-k\" parameter for the FAISS retriever specifically:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.runnables import ConfigurableField\n",
    "\n",
    "faiss_retriever = faiss_vectorstore.as_retriever(\n",
    "    search_kwargs={\"k\": 2}\n",
    ").configurable_fields(\n",
    "    search_kwargs=ConfigurableField(\n",
    "        id=\"search_kwargs_faiss\",\n",
    "        name=\"Search Kwargs\",\n",
    "        description=\"The search kwargs to use\",\n",
    "    )\n",
    ")\n",
    "\n",
    "ensemble_retriever = EnsembleRetriever(\n",
    "    retrievers=[bm25_retriever, faiss_retriever], weights=[0.5, 0.5]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='I like apples', metadata={'source': 1}),\n",
       " Document(page_content='You like apples', metadata={'source': 2}),\n",
       " Document(page_content='Apples and oranges are fruits', metadata={'source': 1})]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "config = {\"configurable\": {\"search_kwargs_faiss\": {\"k\": 1}}}\n",
    "docs = ensemble_retriever.invoke(\"apples\", config=config)\n",
    "docs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that this only returns one source from the FAISS retriever, because we pass in the relevant configuration at run time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}