seansullivan/Langchain-Docs-June-26-2024 huggingface dataset:
{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "16f2c32e",
   "metadata": {},
   "source": [
    "## Document Loading\n",
    "\n",
    "Load a blog post on agents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "c9fadce0",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import WebBaseLoader\n",
    "\n",
    "loader = WebBaseLoader(\"https://lilianweng.github.io/posts/2023-06-23-agent/\")\n",
    "text = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4086be03",
   "metadata": {},
   "source": [
    "## Run Template\n",
    "\n",
    "In `server.py`, set -\n",
    "```\n",
    "add_routes(app, chain_ext, path=\"/extraction_openai_functions\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ed507784",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langserve.client import RemoteRunnable\n",
    "\n",
    "oai_function = RemoteRunnable(\"http://0.0.0.0:8001/extraction_openai_functions\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68046695",
   "metadata": {},
   "source": [
    "The function wille extract paper titles and authors from an input."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "6dace748",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'title': 'Chain of Thought', 'author': 'Wei et al. 2022'},\n",
       " {'title': 'Tree of Thoughts', 'author': 'Yao et al. 2023'},\n",
       " {'title': 'LLM+P', 'author': 'Liu et al. 2023'}]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "oai_function.invoke({\"input\": text[0].page_content[0:4000]})"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "langserve",
   "language": "python",
   "name": "langserve"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
import json
from typing import List, Optional

from langchain.utils.openai_functions import convert_pydantic_to_openai_function
from langchain_community.chat_models import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.pydantic_v1 import BaseModel

template = """A article will be passed to you. Extract from it all papers that are mentioned by this article. 

Do not extract the name of the article itself. If no papers are mentioned that's fine - you don't need to extract any! Just return an empty list.

Do not make up or guess ANY extra information. Only extract what exactly is in the text."""  # noqa: E501

prompt = ChatPromptTemplate.from_messages([("system", template), ("human", "{input}")])


# Function output schema
class Paper(BaseModel):
    """Information about papers mentioned."""

    title: str
    author: Optional[str]


class Info(BaseModel):
    """Information to extract"""

    papers: List[Paper]


# Function definition
model = ChatOpenAI()
function = [convert_pydantic_to_openai_function(Info)]
chain = (
    prompt
    | model.bind(functions=function, function_call={"name": "Info"})
    | (
        lambda x: json.loads(x.additional_kwargs["function_call"]["arguments"])[
            "papers"
        ]
    )
)

# chain = prompt | model.bind(
#     functions=function, function_call={"name": "Info"}
# ) | JsonKeyOutputFunctionsParser(key_name="papers")
# extraction-anthropic-functions

This template enables [Anthropic function calling](https://python.langchain.com/docs/integrations/chat/anthropic_functions). 

This can be used for various tasks, such as extraction or tagging.

The function output schema can be set in `chain.py`. 

## Environment Setup

Set the `ANTHROPIC_API_KEY` environment variable to access the Anthropic models.

## Usage

To use this package, you should first have the LangChain CLI installed:

```shell
pip install -U langchain-cli
```

To create a new LangChain project and install this as the only package, you can do:

```shell
langchain app new my-app --package extraction-anthropic-functions
```

If you want to add this to an existing project, you can just run:

```shell
langchain app add extraction-anthropic-functions
```

And add the following code to your `server.py` file:
```python
from extraction_anthropic_functions import chain as extraction_anthropic_functions_chain

add_routes(app, extraction_anthropic_functions_chain, path="/extraction-anthropic-functions")
```

(Optional) Let's now configure LangSmith. 
LangSmith will help us trace, monitor and debug LangChain applications. 
You can sign up for LangSmith [here](https://smith.langchain.com/). 
If you don't have access, you can skip this section


```shell
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=<your-api-key>
export LANGCHAIN_PROJECT=<your-project>  # if not specified, defaults to "default"
```

If you are inside this directory, then you can spin up a LangServe instance directly by:

```shell
langchain serve
```

This will start the FastAPI app with a server is running locally at 
[http://localhost:8000](http://localhost:8000)

We can see all templates at [http://127.0.0.1:8000/docs](http://127.0.0.1:8000/docs)
We can access the playground at [http://127.0.0.1:8000/extraction-anthropic-functions/playground](http://127.0.0.1:8000/extraction-anthropic-functions/playground)  

We can access the template from code with:

```python
from langserve.client import RemoteRunnable

runnable = RemoteRunnable("http://localhost:8000/extraction-anthropic-functions")
```

By default, the package will extract the title and author of papers from the information you specify in `chain.py`. This template will use `Claude2` by default. 

---
{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "4ae4b789",
   "metadata": {},
   "source": [
    "## Document Loading\n",
    "\n",
    "Load a blog post on agents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "5d6bd62e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import WebBaseLoader\n",
    "\n",
    "loader = WebBaseLoader(\"https://lilianweng.github.io/posts/2023-06-23-agent/\")\n",
    "text = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e21575d",
   "metadata": {},
   "source": [
    "## Run Template\n",
    "\n",
    "In `server.py`, set -\n",
    "```\n",
    "add_routes(app, chain_ext, path=\"/extraction-anthropic-functions\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5fd794ec-a002-490e-8eb9-06ce3e6c2f14",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langserve.client import RemoteRunnable\n",
    "\n",
    "anthropic_function_model = RemoteRunnable(\n",
    "    \"http://localhost:8001/extraction-anthropic-functions\"\n",
    ")\n",
    "anthropic_function_model.invoke(text[0].page_content[0:1500])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "langserve",
   "language": "python",
   "name": "langserve"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
from typing import List, Optional

from langchain.utils.openai_functions import convert_pydantic_to_openai_function
from langchain_core.output_parsers.openai_functions import JsonKeyOutputFunctionsParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.pydantic_v1 import BaseModel
from langchain_experimental.llms.anthropic_functions import AnthropicFunctions

template = """A article will be passed to you. Extract from it all papers that are mentioned by this article. 

Do not extract the name of the article itself. If no papers are mentioned that's fine - you don't need to extract any! Just return an empty list.

Do not make up or guess ANY extra information. Only extract what exactly is in the text."""  # noqa: E501

prompt = ChatPromptTemplate.from_messages([("system", template), ("human", "{input}")])


# Function output schema
class Paper(BaseModel):
    """Information about papers mentioned."""

    title: str
    author: Optional[str]


class Info(BaseModel):
    """Information to extract"""

    papers: List[Paper]


# Function definition
model = AnthropicFunctions()
function = [convert_pydantic_to_openai_function(Info)]

chain = (
    prompt
    | model.bind(functions=function, function_call={"name": "Info"})
    | JsonKeyOutputFunctionsParser(key_name="papers")
)
# elastic-query-generator

This template allows interacting with Elasticsearch analytics databases in natural language using LLMs. 

It builds search queries via the Elasticsearch DSL API (filters and aggregations). 

## Environment Setup

Set the `OPENAI_API_KEY` environment variable to access the OpenAI models.

### Installing Elasticsearch

There are a number of ways to run Elasticsearch. However, one recommended way is through Elastic Cloud.

Create a free trial account on [Elastic Cloud](https://cloud.elastic.co/registration?utm_source=langchain&utm_content=langserve).

With a deployment, update the connection string.

Password and connection (elasticsearch url) can be found on the deployment console.

Note that the Elasticsearch client must have permissions for index listing, mapping description, and search queries.

### Populating with data

If you want to populate the DB with some example info, you can run `python ingest.py`.

This will create a `customers` index. In this package, we specify indexes to generate queries against, and we specify `["customers"]`. This is specific to setting up your Elastic index.

## Usage

To use this package, you should first have the LangChain CLI installed:

```shell
pip install -U langchain-cli
```

To create a new LangChain project and install this as the only package, you can do:

```shell
langchain app new my-app --package elastic-query-generator
```

If you want to add this to an existing project, you can just run:

```shell
langchain app add elastic-query-generator
```

And add the following code to your `server.py` file:
```python
from elastic_query_generator.chain import chain as elastic_query_generator_chain

add_routes(app, elastic_query_generator_chain, path="/elastic-query-generator")
```

(Optional) Let's now configure LangSmith. 
LangSmith will help us trace, monitor and debug LangChain applications. 
You can sign up for LangSmith [here](https://smith.langchain.com/). 
If you don't have access, you can skip this section

```shell
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=<your-api-key>
export LANGCHAIN_PROJECT=<your-project>  # if not specified, defaults to "default"
```

If you are inside this directory, then you can spin up a LangServe instance directly by:

```shell
langchain serve
```

This will start the FastAPI app with a server is running locally at 
[http://localhost:8000](http://localhost:8000)

We can see all templates at [http://127.0.0.1:8000/docs](http://127.0.0.1:8000/docs)
We can access the playground at [http://127.0.0.1:8000/elastic-query-generator/playground](http://127.0.0.1:8000/elastic-query-generator/playground)  

We can access the template from code with:

```python
from langserve.client import RemoteRunnable

runnable = RemoteRunnable("http://localhost:8000/elastic-query-generator")
```
from elasticsearch import Elasticsearch

# Setup Elasticsearch
# This shows how to set it up for a cloud hosted version

# Password for the 'elastic' user generated by Elasticsearch
ELASTIC_PASSWORD = "..."

# Found in the 'Manage Deployment' page
CLOUD_ID = "..."

# Create the client instance
db = Elasticsearch(cloud_id=CLOUD_ID, basic_auth=("elastic", ELASTIC_PASSWORD))

customers = [
    {"firstname": "Jennifer", "lastname": "Walters"},
    {"firstname": "Monica", "lastname": "Rambeau"},
    {"firstname": "Carol", "lastname": "Danvers"},
    {"firstname": "Wanda", "lastname": "Maximoff"},
    {"firstname": "Jennifer", "lastname": "Takeda"},
]
for i, customer in enumerate(customers):
    db.create(index="customers", document=customer, id=i)
from elasticsearch import Elasticsearch
from langchain.output_parsers.json import SimpleJsonOutputParser
from langchain_community.chat_models import ChatOpenAI
from langchain_core.pydantic_v1 import BaseModel

from .elastic_index_info import get_indices_infos
from .prompts import DSL_PROMPT

# Setup Elasticsearch
# This shows how to set it up for a cloud hosted version

# Password for the 'elastic' user generated by Elasticsearch
ELASTIC_PASSWORD = "..."

# Found in the 'Manage Deployment' page
CLOUD_ID = "..."

# Create the client instance
db = Elasticsearch(cloud_id=CLOUD_ID, basic_auth=("elastic", ELASTIC_PASSWORD))

# Specify indices to include
# If you want to use on your own indices, you will need to change this.
INCLUDE_INDICES = ["customers"]

# With the Elasticsearch connection created, we can now move on to the chain

_model = ChatOpenAI(temperature=0, model="gpt-4")

chain = (
    {
        "input": lambda x: x["input"],
        # This line only get index info for "customers" index.
        # If you are running this on your own data, you will want to change.
        "indices_info": lambda _: get_indices_infos(
            db, include_indices=INCLUDE_INDICES
        ),
        "top_k": lambda x: x.get("top_k", 5),
    }
    | DSL_PROMPT
    | _model
    | SimpleJsonOutputParser()
)


# Nicely typed inputs for playground
class ChainInputs(BaseModel):
    input: str
    top_k: int = 5


chain = chain.with_types(input_type=ChainInputs)
from typing import List


def _list_indices(database, include_indices=None, ignore_indices=None) -> List[str]:
    all_indices = [index["index"] for index in database.cat.indices(format="json")]

    if include_indices:
        all_indices = [i for i in all_indices if i in include_indices]
    if ignore_indices:
        all_indices = [i for i in all_indices if i not in ignore_indices]

    return all_indices


def get_indices_infos(
    database,
    sample_documents_in_index_info=5,
    include_indices=None,
    ignore_indices=None,
) -> str:
    indices = _list_indices(
        database, include_indices=include_indices, ignore_indices=ignore_indices
    )
    mappings = database.indices.get_mapping(index=",".join(indices))
    if sample_documents_in_index_info > 0:
        for k, v in mappings.items():
            hits = database.search(
                index=k,
                query={"match_all": {}},
                size=sample_documents_in_index_info,
            )["hits"]["hits"]
            hits = [str(hit["_source"]) for hit in hits]
            mappings[k]["mappings"] = str(v) + "\n\n/*\n" + "\n".join(hits) + "\n*/"
    return "\n\n".join(
        [
            "Mapping for index {}:\n{}".format(index, mappings[index]["mappings"])
            for index in mappings
        ]
    )
from langchain_core.prompts.prompt import PromptTemplate

PROMPT_SUFFIX = """Only use the following Elasticsearch indices:
{indices_info}

Question: {input}
ESQuery:"""

DEFAULT_DSL_TEMPLATE = """Given an input question, create a syntactically correct Elasticsearch query to run. Always limit your query to at most {top_k} results, unless the user specifies in their question a specific number of examples they wish to obtain, or unless its implied that they want to see all. You can order the results by a relevant column to return the most interesting examples in the database.

Unless told to do not query for all the columns from a specific index, only ask for a the few relevant columns given the question.

Pay attention to use only the column names that you can see in the mapping description. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which index. Return the query as valid json.

Use the following format:

Question: Question here
ESQuery: Elasticsearch Query formatted as json
"""  # noqa: E501

DSL_PROMPT = PromptTemplate.from_template(DEFAULT_DSL_TEMPLATE + PROMPT_SUFFIX)
# Contributing

Thanks for taking the time to contribute a new template!
We've tried to make this process as simple and painless as possible.
If you need any help at all, please reach out!

To contribute a new template, first fork this repository.
Then clone that fork and pull it down locally.
Set up an appropriate dev environment, and make sure you are in this `templates` directory.

Make sure you have `langchain-cli` installed.

```shell
pip install -U langchain-cli
```

You can then run the following command to create a new skeleton of a package.
By convention, package names should use `-` delimiters (not `_`).

```shell
langchain template new $PROJECT_NAME
```

You can then edit the contents of the package as you desire.
Note that by default we expect the main chain to be exposed as `chain` in the `__init__.py` file of the package.
You can change this (either the name or the location), but if you do so it is important to update the `tool.langchain`
part of `pyproject.toml`.
For example, if you update the main chain exposed to be called `agent_executor`, then that section should look like:

```text
[tool.langserve]
export_module = "..."
export_attr = "agent_executor"
```

Make sure to add any requirements of the package to `pyproject.toml` (and to remove any that are not used).

Please update the `README.md` file to give some background on your package and how to set it up.

If you want to change the license of your template for whatever, you may! Note that by default it is MIT licensed.

If you want to test out your package at any point in time, you can spin up a LangServe instance directly from the package.
See instructions [here](LAUNCHING_PACKAGE.md) on how to best do that.
# Launching LangServe from a Package

You can also launch LangServe directly from a package, without having to pull it into a project.
This can be useful when you are developing a package and want to test it quickly.
The downside of this is that it gives you a little less control over how the LangServe APIs are configured,
which is why for proper projects we recommend creating a full project.

In order to do this, first change your working directory to the package itself.
For example, if you are currently in this `templates` module, you can go into the `pirate-speak` package with:

```shell
cd pirate-speak
```

Inside this package there is a `pyproject.toml` file.
This file contains a `tool.langchain` section that contains information on how this package should be used.
For example, in `pirate-speak` we see:

```text
[tool.langserve]
export_module = "pirate_speak.chain"
export_attr = "chain"
```

This information can be used to launch a LangServe instance automatically.
In order to do this, first make sure the CLI is installed:

```shell
pip install -U langchain-cli
```

You can then run:

```shell
langchain template serve
```

This will spin up endpoints, documentation, and playground for this chain.
For example, you can access the playground at [http://127.0.0.1:8000/playground/](http://127.0.0.1:8000/playground/)

![Screenshot of the LangServe Playground web interface with input and output fields.](playground.png "LangServe Playground Interface")
# csv-agent

This template uses a [csv agent](https://python.langchain.com/docs/integrations/toolkits/csv) with tools (Python REPL) and memory (vectorstore) for interaction (question-answering) with text data.

## Environment Setup

Set the `OPENAI_API_KEY` environment variable to access the OpenAI models.

To set up the environment, the `ingest.py` script should be run to handle the ingestion into a vectorstore.

## Usage

To use this package, you should first have the LangChain CLI installed:

```shell
pip install -U langchain-cli
```

To create a new LangChain project and install this as the only package, you can do:

```shell
langchain app new my-app --package csv-agent
```

If you want to add this to an existing project, you can just run:

```shell
langchain app add csv-agent
```

And add the following code to your `server.py` file:
```python
from csv_agent.agent import agent_executor as csv_agent_chain

add_routes(app, csv_agent_chain, path="/csv-agent")
```

(Optional) Let's now configure LangSmith. 
LangSmith will help us trace, monitor and debug LangChain applications. 
You can sign up for LangSmith [here](https://smith.langchain.com/). 
If you don't have access, you can skip this section


```shell
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=<your-api-key>
export LANGCHAIN_PROJECT=<your-project>  # if not specified, defaults to "default"
```

If you are inside this directory, then you can spin up a LangServe instance directly by:

```shell
langchain serve
```

This will start the FastAPI app with a server is running locally at 
[http://localhost:8000](http://localhost:8000)

We can see all templates at [http://127.0.0.1:8000/docs](http://127.0.0.1:8000/docs)
We can access the playground at [http://127.0.0.1:8000/csv-agent/playground](http://127.0.0.1:8000/csv-agent/playground)  

We can access the template from code with:

```python
from langserve.client import RemoteRunnable

runnable = RemoteRunnable("http://localhost:8000/csv-agent")
```
from langchain.indexes import VectorstoreIndexCreator
from langchain_community.document_loaders import CSVLoader
from langchain_community.vectorstores import FAISS

loader = CSVLoader("/Users/harrisonchase/Downloads/titanic.csv")

docs = loader.load()
index_creator = VectorstoreIndexCreator(vectorstore_cls=FAISS)

index = index_creator.from_documents(docs)

index.vectorstore.save_local("titanic_data")