Deploy a Langchain agent as an API endpoint in 10 minutes
Published: 2023-09-20 15:06:05

> Github repository at the end of the article

So you coded a cool Langchain agent in Python. Congrats! In this tutorial, we will walk you through the process of making it an OpenAPI endpoint, which can be deployed and called as an API, allowing you to seamlessly integrate it into your product or workflows. All that in only 10 minutes. Let’s get started!

---

## Installation and setup
You need to have Python 3.8 or higher installed.

### (Optional) Create a new virtual environment
```
python3 -m venv venv
source venv/bin/activate
```

### Install the required packages

- `langchain` for the orchestration of the agent
- `openai` for the OpenAI API
- `phospho` for the deployment of the agent
- `python-dotenv` for the environment variables (like api keys)

```
pip install --upgrade langchain openai phospho python-dotenv
pip freeze > requirements.txt
```
### Store your OpenAI API key
In this example, we are going to use OpenAI as our LLM provider for our Langchain agent. But you can use any LLM provider you wish.

```
OPENAI_API_KEY=your-api-key
```
Create an `.env` file and store your OpenAI API key in it. For extra security, you can create a new OpenAI key for this project.

## Build the agent logic

### Create a new langchain agent

Create a `main.py` python file at the route of the project. You must name it `main.py` since phospho will look for this file to initialize the agent.

Let’s load the environment variables from the `.env` file :

```
# main.py

# Load environment variables

from dotenv import load_dotenv
load_dotenv()
```

Now, let’s create a basic chatbot using the OpenAI API. As you can imagine, the process would be roughly the same for more complex agents.

```
# main.py

# Load environment variables

from dotenv import load_dotenv
load_dotenv()

# use langchain to easily interact with the OpenAI API

from langchain.schema import (
    AIMessage,
    HumanMessage,
    SystemMessage
)
from langchain.chat_models import ChatOpenAI

chat = ChatOpenAI(model_name="gpt-3.5-turbo",temperature=0.3)
messages = [
    SystemMessage(content="You are a funny and helpful assistant."),
    HumanMessage(content="Explain to me quantum physics."),
]
response=chat(messages)

print(response.content,end='\n')
```

Try it locally :

```
python main.py
```

Now, let’s package this logic into a function that we can use in our agent :

```
# main.py

# Load environment variables

from dotenv import load_dotenv
load_dotenv()

# import schema for chat messages and ChatOpenAI in order to query chatmodels GPT-3.5-turbo or GPT-4

from langchain.schema import (
    AIMessage,
    HumanMessage,
    SystemMessage
)
from langchain.chat_models import ChatOpenAI

chat = ChatOpenAI(model_name="gpt-3.5-turbo",temperature=0.3)

def my_agent(input):
    messages = [
        SystemMessage(content="You are a funny and helpful assistant."),
        HumanMessage(content=input),
    ]
    
    response=chat(messages)

    return response.content
```

## Deploy the agent

Having the agent run locally is cool. But now, let’s deploy it so everyone can use it.

To do so, we are going to package our langchain agent into a phospho agent. [phospho](https://phospho.app) is an agnostic _Plateform as a Service_ to deploy and manage autonomous agents. They also provide an SDK to help developers go from prototype to production with their agents.

Here, we only want to create a basic conversational agent. So we only need to create a chat route.

```
# main.py

from langchain.schema import (
    AIMessage,
    HumanMessage,
    SystemMessage
)
from langchain.chat_models import ChatOpenAI
from phospho import Agent, Message

# Load environment variables

from dotenv import load_dotenv
load_dotenv()

# Define our langchain agent
chat = ChatOpenAI(model_name="gpt-3.5-turbo",temperature=0.3)

def my_agent(input):
    messages = [
        SystemMessage(content="You are a funny and helpful assistant."),
        HumanMessage(content=input),
    ]
    
    response=chat(messages)

    return response.content

# Package it into a phospho agent

agent = Agent()

@agent.chat()
def my_chat(message):
    # Use our agent logic to generate a response
    response = my_agent(message.content)
    # Return the response in a Message object
    return Message(response)
```

That’s it!

With this code, we have created a new agent with a single route `/chat` that will call our `my_agent` function and send the response back to the user. Note that you need to send an object of type `Message` as a response (view the [documentation](https://docs.phospho.app) for more info).

## Create a new Github repository​

At the root of your project, create a new Github repository and push your code to the `main` branch. Do not forget to add a `.gitignore` file to ignore the `.env` file (there is your API key inside) and the `venv` folder :

```
# Ignore virtualenv
venv/

# Ignore .env
.env

# Ignore pycache
__pycache__/
```

## Create a new phospho project

Go to your phospho account [dashboard](https://phospho.app/dashboard) and create a new project.

For the project name :
- respect naming conventions, use just alphanumeric characters (no spaces, no special characters, no emojis, etc…)
- if the name is already taken, choose a new one. Come on, be creative!

Select the github repository you want to link to your phospho project.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kuoyz2opiv8p1a6occns.png)

## Setup the environment variables

Remember, we have stored our OpenAI API key in a `.env` file. But this file is not available on the server since we didn't push it to Github. Our code won't work if the API key is not available as an environment variable to the langchain agent.

On the phospho webapp, go to your project page and click on the **Manage Env Variables** tab. Then, click on the Environment variables tab and add a new variable called `OPENAI_API_KEY` with your OpenAI API key as a value.

At the next restart of your agent, it will be able to use the OpenAI API (to force a restart, you can trigger a new deployment).


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/88gjwnfv6ukap21dqbcq.png)

## Deploy the agent

Once project is created, just push some code to the `main` branch or go to the project page and trigger the deployment manually using the button.

After a one or two minutes, your agent should be deployed and ready to use. You have the endpoint URL on your project page.

## Test your agent

Your agent is now deployed as an OpenAPI endpoint. You can find the URL of the API endpoint of your agent on the project page. You can test it directly using the API or by going in your browser to the URL and adding `/docs` at the end.

Unless you go viral, you should be able to use the free plan (no credit card required to start).


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qsze6ze0ypeya0msluuw.png)

## Github repo with the source code

You can find all the source code of this tutorial in [this github repo](https://github.com/phospho-app/agent-basic-chatgpt)

Congratulations! You have successfully deployed a simple conversational agent using Langchain, OpenAI and phospho. Feel free to explore and enhance the capabilities of your agent further.

Want to give anonymous feedback? Fill in the google form [here](https://docs.google.com/forms/d/e/1FAIpQLSf22VB-VYCtsI5xouTpW6vmNc2oqbVJ89tGr6e4YLfjSC8ATg/viewform).