seansullivan/Langchain-Docs-June-26-2024 huggingface dataset:
# rag-ollama-multi-query

This template performs RAG using Ollama and OpenAI with a multi-query retriever. 

The multi-query retriever is an example of query transformation, generating multiple queries from different perspectives based on the user's input query. 

For each query, it retrieves a set of relevant documents and takes the unique union across all queries for answer synthesis.

We use a private, local LLM for the narrow task of query generation to avoid excessive calls to a larger LLM API.

See an example trace for Ollama LLM performing the query expansion [here](https://smith.langchain.com/public/8017d04d-2045-4089-b47f-f2d66393a999/r).

But we use OpenAI for the more challenging task of answer syntesis (full trace example [here](https://smith.langchain.com/public/ec75793b-645b-498d-b855-e8d85e1f6738/r)).

## Environment Setup

To set up the environment, you need to download Ollama. 

Follow the instructions [here](https://python.langchain.com/docs/integrations/chat/ollama). 

You can choose the desired LLM with Ollama. 

This template uses `zephyr`, which can be accessed using `ollama pull zephyr`.

There are many other options available [here](https://ollama.ai/library).

Set the `OPENAI_API_KEY` environment variable to access the OpenAI models.

## Usage

To use this package, you should first install the LangChain CLI:

```shell
pip install -U langchain-cli
```

To create a new LangChain project and install this package, do:

```shell
langchain app new my-app --package rag-ollama-multi-query
```

To add this package to an existing project, run:

```shell
langchain app add rag-ollama-multi-query
```

And add the following code to your `server.py` file:

```python
from rag_ollama_multi_query import chain as rag_ollama_multi_query_chain

add_routes(app, rag_ollama_multi_query_chain, path="/rag-ollama-multi-query")
```

(Optional) Now, let's configure LangSmith. LangSmith will help us trace, monitor, and debug LangChain applications. You can sign up for LangSmith [here](https://smith.langchain.com/). If you don't have access, you can skip this section

```shell
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=<your-api-key>
export LANGCHAIN_PROJECT=<your-project>  # if not specified, defaults to "default"
```

If you are inside this directory, then you can spin up a LangServe instance directly by:

```shell
langchain serve
```

This will start the FastAPI app with a server running locally at [http://localhost:8000](http://localhost:8000)

You can see all templates at [http://127.0.0.1:8000/docs](http://127.0.0.1:8000/docs)
You can access the playground at [http://127.0.0.1:8000/rag-ollama-multi-query/playground](http://127.0.0.1:8000/rag-ollama-multi-query/playground)

To access the template from code, use:

```python
from langserve.client import RemoteRunnable

runnable = RemoteRunnable("http://localhost:8000/rag-ollama-multi-query")
```
{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "681a5d1e",
   "metadata": {},
   "source": [
    "## Connect to template\n",
    "\n",
    "In `server.py`, set -\n",
    "```\n",
    "add_routes(app, chain_ext, path=\"/rag_ollama_multi_query\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "8d61a866-f91f-41ec-a840-270b0c9c895c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'The various types of agent memory mentioned in the context are:\\n\\n1. Explicit / declarative memory: This refers to memory of facts and events, including episodic memory (events and experiences) and semantic memory (facts and concepts).\\n\\n2. Implicit / procedural memory: This type of memory is unconscious and involves skills and routines that are performed automatically, like riding a bike or typing on a keyboard.\\n\\n3. Short-term memory: This is the in-context learning utilized by the model to learn.\\n\\n4. Long-term memory: This provides the agent with the capability to retain and recall information over extended periods, often by leveraging an external vector store and fast retrieval.\\n\\n5. Sensory memory: This is the earliest stage of memory that retains impressions of sensory information (visual, auditory, etc) after the original stimuli have ended. It includes subcategories like iconic memory (visual), echoic memory (auditory), and haptic memory (touch).'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langserve.client import RemoteRunnable\n",
    "\n",
    "rag_app_ollama = RemoteRunnable(\"http://0.0.0.0:8001/rag_ollama_multi_query\")\n",
    "rag_app_ollama.invoke(\"What are the different types of agent memory?\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
from langchain.retrievers.multi_query import MultiQueryRetriever
from langchain_community.chat_models import ChatOllama, ChatOpenAI
from langchain_community.document_loaders import WebBaseLoader
from langchain_community.embeddings import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate, PromptTemplate
from langchain_core.pydantic_v1 import BaseModel
from langchain_core.runnables import RunnableParallel, RunnablePassthrough
from langchain_text_splitters import RecursiveCharacterTextSplitter

# Load
loader = WebBaseLoader("https://lilianweng.github.io/posts/2023-06-23-agent/")
data = loader.load()

# Split
text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)
all_splits = text_splitter.split_documents(data)

# Add to vectorDB
vectorstore = Chroma.from_documents(
    documents=all_splits,
    collection_name="rag-private",
    embedding=OpenAIEmbeddings(),
)


QUERY_PROMPT = PromptTemplate(
    input_variables=["question"],
    template="""You are an AI language model assistant. Your task is to generate five
    different versions of the given user question to retrieve relevant documents from
    a vector database. By generating multiple perspectives on the user question, your
    goal is to help the user overcome some of the limitations of the distance-based
    similarity search. Provide these alternative questions separated by newlines.
    Original question: {question}""",
)

# Add the LLM downloaded from Ollama
ollama_llm = "zephyr"
llm = ChatOllama(model=ollama_llm)

# Run
retriever = MultiQueryRetriever.from_llm(
    vectorstore.as_retriever(), llm, prompt=QUERY_PROMPT
)  # "lines" is the key (attribute name) of the parsed output

# RAG prompt
template = """Answer the question based only on the following context:
{context}
Question: {question}
"""
prompt = ChatPromptTemplate.from_template(template)

# RAG
model = ChatOpenAI()
chain = (
    RunnableParallel({"context": retriever, "question": RunnablePassthrough()})
    | prompt
    | model
    | StrOutputParser()
)


# Add typing for input
class Question(BaseModel):
    __root__: str


chain = chain.with_types(input_type=Question)
{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "681a5d1e",
   "metadata": {},
   "source": [
    "## Run Template\n",
    "\n",
    "In `server.py`, set -\n",
    "```\n",
    "add_routes(app, chain_rag_conv, path=\"/rag-multi-modal-mv-local\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d774be2a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langserve.client import RemoteRunnable\n",
    "\n",
    "rag_app = RemoteRunnable(\"http://localhost:8001/rag-multi-modal-mv-local\")\n",
    "rag_app.invoke(\" < keywords here > \")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
import os
from pathlib import Path

from langchain_community.vectorstores import Chroma
from langchain_nomic import NomicMultimodalEmbeddings

# Load images
img_dump_path = Path(__file__).parent / "docs/"
rel_img_dump_path = img_dump_path.relative_to(Path.cwd())
image_uris = sorted(
    [
        os.path.join(rel_img_dump_path, image_name)
        for image_name in os.listdir(rel_img_dump_path)
        if image_name.endswith(".jpg")
    ]
)

# Index
vectorstore = Path(__file__).parent / "chroma_db_multi_modal"
re_vectorstore_path = vectorstore.relative_to(Path.cwd())

# Load embedding function
print("Loading embedding function")
embedding = NomicMultimodalEmbeddings(
    vision_model="nomic-embed-vision-v1", text_model="nomic-embed-text-v1"
)

# Create chroma
vectorstore_mmembd = Chroma(
    collection_name="multi-modal-rag",
    persist_directory=str(Path(__file__).parent / "chroma_db_multi_modal"),
    embedding_function=embedding,
)

# Add images
print("Embedding images")
vectorstore_mmembd.add_images(uris=image_uris)
{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "681a5d1e",
   "metadata": {},
   "source": [
    "## Run Template\n",
    "\n",
    "In `server.py`, set -\n",
    "```\n",
    "add_routes(app, chain_rag_conv, path=\"/rag-multi-modal-local\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d774be2a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langserve.client import RemoteRunnable\n",
    "\n",
    "rag_app = RemoteRunnable(\"http://localhost:8001/rag-multi-modal-local\")\n",
    "rag_app.invoke(\" < keywords here > \")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
# RAG with Multiple Indexes (Routing)

A QA application that routes between different domain-specific retrievers given a user question.

## Environment Setup

This application queries PubMed, ArXiv, Wikipedia, and [Kay AI](https://www.kay.ai) (for SEC filings).

You will need to create a free Kay AI account and [get your API key here](https://www.kay.ai). 
Then set environment variable:

```bash
export KAY_API_KEY="<YOUR_API_KEY>"
```

## Usage

To use this package, you should first have the LangChain CLI installed:

```shell
pip install -U langchain-cli
```

To create a new LangChain project and install this as the only package, you can do:

```shell
langchain app new my-app --package rag-multi-index-router
```

If you want to add this to an existing project, you can just run:

```shell
langchain app add rag-multi-index-router
```

And add the following code to your `server.py` file:
```python
from rag_multi_index_router import chain as rag_multi_index_router_chain

add_routes(app, rag_multi_index_router_chain, path="/rag-multi-index-router")
```

(Optional) Let's now configure LangSmith. 
LangSmith will help us trace, monitor and debug LangChain applications. 
You can sign up for LangSmith [here](https://smith.langchain.com/). 
If you don't have access, you can skip this section


```shell
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=<your-api-key>
export LANGCHAIN_PROJECT=<your-project>  # if not specified, defaults to "default"
```

If you are inside this directory, then you can spin up a LangServe instance directly by:

```shell
langchain serve
```

This will start the FastAPI app with a server is running locally at 
[http://localhost:8000](http://localhost:8000)

We can see all templates at [http://127.0.0.1:8000/docs](http://127.0.0.1:8000/docs)
We can access the playground at [http://127.0.0.1:8000/rag-multi-index-router/playground](http://127.0.0.1:8000/rag-multi-index-router/playground)  

We can access the template from code with:

```python
from langserve.client import RemoteRunnable

runnable = RemoteRunnable("http://localhost:8000/rag-multi-index-router")
```
# RAG with Multiple Indexes (Fusion)

A QA application that queries multiple domain-specific retrievers and selects the most relevant documents from across all retrieved results.

## Environment Setup

This application queries PubMed, ArXiv, Wikipedia, and [Kay AI](https://www.kay.ai) (for SEC filings).

You will need to create a free Kay AI account and [get your API key here](https://www.kay.ai).
Then set environment variable:

```bash
export KAY_API_KEY="<YOUR_API_KEY>"
```

## Usage

To use this package, you should first have the LangChain CLI installed:

```shell
pip install -U langchain-cli
```

To create a new LangChain project and install this as the only package, you can do:

```shell
langchain app new my-app --package rag-multi-index-fusion
```

If you want to add this to an existing project, you can just run:

```shell
langchain app add rag-multi-index-fusion
```

And add the following code to your `server.py` file:
```python
from rag_multi_index_fusion import chain as rag_multi_index_fusion_chain

add_routes(app, rag_multi_index_fusion_chain, path="/rag-multi-index-fusion")
```

(Optional) Let's now configure LangSmith. 
LangSmith will help us trace, monitor and debug LangChain applications. 
You can sign up for LangSmith [here](https://smith.langchain.com/). 
If you don't have access, you can skip this section


```shell
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=<your-api-key>
export LANGCHAIN_PROJECT=<your-project>  # if not specified, defaults to "default"
```

If you are inside this directory, then you can spin up a LangServe instance directly by:

```shell
langchain serve
```

This will start the FastAPI app with a server is running locally at 
[http://localhost:8000](http://localhost:8000)

We can see all templates at [http://127.0.0.1:8000/docs](http://127.0.0.1:8000/docs)
We can access the playground at [http://127.0.0.1:8000/rag-multi-index-fusion/playground](http://127.0.0.1:8000/rag-multi-index-fusion/playground)  

We can access the template from code with:

```python
from langserve.client import RemoteRunnable

runnable = RemoteRunnable("http://localhost:8000/rag-multi-index-fusion")
```
from operator import itemgetter

import numpy as np
from langchain.retrievers import (
    ArxivRetriever,
    KayAiRetriever,
    PubMedRetriever,
    WikipediaRetriever,
)
from langchain.utils.math import cosine_similarity
from langchain_community.chat_models import ChatOpenAI
from langchain_community.embeddings import OpenAIEmbeddings
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.pydantic_v1 import BaseModel
from langchain_core.runnables import (
    RunnableParallel,
    RunnablePassthrough,
)

pubmed = PubMedRetriever(top_k_results=5).with_config(run_name="pubmed")
arxiv = ArxivRetriever(top_k_results=5).with_config(run_name="arxiv")
sec = KayAiRetriever.create(
    dataset_id="company", data_types=["10-K"], num_contexts=5
).with_config(run_name="sec_filings")
wiki = WikipediaRetriever(top_k_results=5, doc_content_chars_max=2000).with_config(
    run_name="wiki"
)

embeddings = OpenAIEmbeddings()


def fuse_retrieved_docs(input):
    results_map = input["sources"]
    query = input["question"]
    embedded_query = embeddings.embed_query(query)
    names, docs = zip(
        *((name, doc) for name, docs in results_map.items() for doc in docs)
    )
    embedded_docs = embeddings.embed_documents([doc.page_content for doc in docs])
    similarity = cosine_similarity(
        [embedded_query],
        embedded_docs,
    )
    most_similar = np.flip(np.argsort(similarity[0]))[:5]
    return [
        (
            names[i],
            docs[i],
        )
        for i in most_similar
    ]


def format_named_docs(named_docs):
    return "\n\n".join(
        f"Source: {source}\n\n{doc.page_content}" for source, doc in named_docs
    )


system = """Answer the user question. Use the following sources to help \
answer the question. If you don't know the answer say "I'm not sure, I couldn't \
find information on {{topic}}."

Sources:

{sources}"""
prompt = ChatPromptTemplate.from_messages([("system", system), ("human", "{question}")])

retrieve_all = RunnableParallel(
    {"ArXiv": arxiv, "Wikipedia": wiki, "PubMed": pubmed, "SEC 10-K Forms": sec}
).with_config(run_name="retrieve_all")


class Question(BaseModel):
    __root__: str


answer_chain = (
    {
        "question": itemgetter("question"),
        "sources": lambda x: format_named_docs(x["sources"]),
    }
    | prompt
    | ChatOpenAI(model="gpt-3.5-turbo-1106")
    | StrOutputParser()
).with_config(run_name="answer")
chain = (
    (
        RunnableParallel(
            {"question": RunnablePassthrough(), "sources": retrieve_all}
        ).with_config(run_name="add_sources")
        | RunnablePassthrough.assign(sources=fuse_retrieved_docs).with_config(
            run_name="fuse"
        )
        | RunnablePassthrough.assign(answer=answer_chain).with_config(
            run_name="add_answer"
        )
    )
    .with_config(run_name="QA with fused results")
    .with_types(input_type=Question)
)
import os

from langchain_community.document_loaders import PyPDFLoader
from langchain_community.embeddings import OpenAIEmbeddings
from langchain_community.vectorstores import MongoDBAtlasVectorSearch
from langchain_text_splitters import RecursiveCharacterTextSplitter
from pymongo import MongoClient

MONGO_URI = os.environ["MONGO_URI"]

# Note that if you change this, you also need to change it in `rag_mongo/chain.py`
DB_NAME = "langchain-test-2"
COLLECTION_NAME = "test"
ATLAS_VECTOR_SEARCH_INDEX_NAME = "default"
EMBEDDING_FIELD_NAME = "embedding"
client = MongoClient(MONGO_URI)
db = client[DB_NAME]
MONGODB_COLLECTION = db[COLLECTION_NAME]

if __name__ == "__main__":
    # Load docs
    loader = PyPDFLoader("https://arxiv.org/pdf/2303.08774.pdf")
    data = loader.load()

    # Split docs
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)
    docs = text_splitter.split_documents(data)

    # Insert the documents in MongoDB Atlas Vector Search
    _ = MongoDBAtlasVectorSearch.from_documents(
        documents=docs,
        embedding=OpenAIEmbeddings(disallowed_special=()),
        collection=MONGODB_COLLECTION,
        index_name=ATLAS_VECTOR_SEARCH_INDEX_NAME,
    )
{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "681a5d1e",
   "metadata": {},
   "source": [
    "## Connect to template\n",
    "\n",
    "In `server.py`, set -\n",
    "```\n",
    "add_routes(app, chain_ext, path=\"/rag_mongo\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d774be2a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langserve.client import RemoteRunnable\n",
    "\n",
    "rag_app_pinecone = RemoteRunnable(\"http://0.0.0.0:8001/rag_mongo\")\n",
    "rag_app_pinecone.invoke(\"How does agent memory work?\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
import os

from langchain_community.chat_models import ChatOpenAI
from langchain_community.document_loaders import PyPDFLoader
from langchain_community.embeddings import OpenAIEmbeddings
from langchain_community.vectorstores import MongoDBAtlasVectorSearch
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.pydantic_v1 import BaseModel
from langchain_core.runnables import (
    RunnableLambda,
    RunnableParallel,
    RunnablePassthrough,
)
from langchain_text_splitters import RecursiveCharacterTextSplitter
from pymongo import MongoClient

# Set DB
if os.environ.get("MONGO_URI", None) is None:
    raise Exception("Missing `MONGO_URI` environment variable.")
MONGO_URI = os.environ["MONGO_URI"]

DB_NAME = "langchain-test-2"
COLLECTION_NAME = "test"
ATLAS_VECTOR_SEARCH_INDEX_NAME = "default"

client = MongoClient(MONGO_URI)
db = client[DB_NAME]
MONGODB_COLLECTION = db[COLLECTION_NAME]

# Read from MongoDB Atlas Vector Search
vectorstore = MongoDBAtlasVectorSearch.from_connection_string(
    MONGO_URI,
    DB_NAME + "." + COLLECTION_NAME,
    OpenAIEmbeddings(disallowed_special=()),
    index_name=ATLAS_VECTOR_SEARCH_INDEX_NAME,
)
retriever = vectorstore.as_retriever()

# RAG prompt
template = """Answer the question based only on the following context:
{context}
Question: {question}
"""
prompt = ChatPromptTemplate.from_template(template)

# RAG
model = ChatOpenAI()
chain = (
    RunnableParallel({"context": retriever, "question": RunnablePassthrough()})
    | prompt
    | model
    | StrOutputParser()
)


# Add typing for input
class Question(BaseModel):
    __root__: str


chain = chain.with_types(input_type=Question)


def _ingest(url: str) -> dict:
    loader = PyPDFLoader(url)
    data = loader.load()

    # Split docs
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)
    docs = text_splitter.split_documents(data)

    # Insert the documents in MongoDB Atlas Vector Search
    _ = MongoDBAtlasVectorSearch.from_documents(
        documents=docs,
        embedding=OpenAIEmbeddings(disallowed_special=()),
        collection=MONGODB_COLLECTION,
        index_name=ATLAS_VECTOR_SEARCH_INDEX_NAME,
    )
    return {}


ingest = RunnableLambda(_ingest)