How to Deploy LangChain App 🐦🔗 as an API
Tags: ai, machinelearning, nlp, tutorial
Published: 2023-06-11 18:59:34

## Introduction

Recently, I had an idea to add my own AI Assistant to my website - [www.akshaymakes.com](http://www.akshaymakes.com). I wanted it to answer questions about my projects and myself and talk a lot about machine learning and artificial intelligence in general. My website is built on SvelteKit and I could have chosen to use OpenAI’s API directly from the front end. But, I wanted the assistant to be scalable. That is, in the future, I wanted it to be able to browse through my blogs, projects, and other material on my website and answer questions much better. So for this purpose and to keep the website simple, I created a LangChain application using FastAPI which would integrate with my website with REST API. I wanted the experience to be similar to ChatGPT i.e. it should be able to remember the context of the conversation and continue the conversation naturally. I deployed the application on Deta Space which was quite simple to do. Let me take you step by step through the process. We will keep the application simple right now. In future blogs, I will explain how you can add your website as context using Vector Databases like Weaviate or Pinecone to make the chat assistant more knowledgeable about you.

So in this tutorial I will show you how to create an API for getting OpenAI outputs using LangChain, FastAPI and Deta Space. Let’s begin.

## Setting Up

1. Start with a new Python project in a new directory. In our example, let us call the project directory as `LangChainAPI.`
    
2. Create a directory in `LangChainAPI` called `app` and new file `.env`
    
3. Inside the app folder, create an empty `__init__.py` file and a new `main.py` and `conversation.py` file.
    
4. In this new directory, initiate a virtual environment with the following terminal command.
    
    ```bash
    python -m venv venv
    ```
    
5. This is how the project structure will look like
    
    ```
    
    ├── app│ 
    │   ├── __init__.py
    │   ├── main.py
    │   ├── conversation.py
    ├── .venv| 
    └── .gitignore
    └── .env
    
    ```
    
6. Activate the environment.
    
    For Windows
    
    ```bash
    venv\Scripts\activate.bat
    ```
    
    For MacOS/Linux
    
    ```bash
    source venv/bin/activate
    ```
    
7. Install the dependencies.
    
    ```bash
    pip install langchain fastapi "uvicorn[standard]" openai python-dotenv
    ```
    
8. Install Deta Space CLI
    
    For Windows
    
    ```bash
    iwr <https://deta.space/assets/space-cli.ps1> -useb | iex
    ```
    
    For MacOS/Linux
    
    ```bash
   curl -fsSL get.deta.dev/space-cli.sh | sh
    ```
    
9. Initiate Git Repository and commit
    
    ```bash
    git init
    git add .
    git commit -m "First Commit"
    ```
    
10. Create an account on [https://deta.space/signup](https://deta.space/signup) and get your access token from the settings.
    
    ![Access Token](https://res.cloudinary.com/dltwftrgc/image/upload/v1686508933/Blogs/langchain-api/access_token_ugisfi.png)
    
11. Login to Deta Space in the CLI. It will ask for the access token. Paste it.
    
    ```bash
    space login
    ```
    
That is all for the setup. Now let us create the API.

## Creating the API

In the app folder open `conversation.py`. This is where we will write the logic for LangChain.
    
 ```python
    from langchain import OpenAI, ConversationChain, LLMChain, PromptTemplate
    
    load_dotenv()
    
    def conversation(human_input):
    	template = """Assistant is a large language model trained by  OpenAI.
    	
    	Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.
    	
    	Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.
    	
    	Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.
    	
    	{history}
    	Human: {human_input}
    	Assistant:"""
    	
    	prompt = PromptTemplate(
    	    input_variables=["history", "human_input"],
    	    template=template
    	)
    	
    	chatgpt_chain = LLMChain(
    	    llm=OpenAI(temperature=0),
    	    prompt=prompt,
    	    verbose=True,
    	)
    	
        output = chatgpt_chain.predict(human_input=human_input)
        return output


 ```
    
In the `main.py` file.
    
```python
from fastapi import FastAPI
from langcorn import create_service
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
from app.conversation import conversation

class Input(BaseModel):
	human_input: str

class Output(BaseModel):
	output: str

app=FastAPI()

@app.post("/conversation")
async def input(input: Input):
	output = Output(output=conversation(input.human_input))
	return output

origins = [
	"<http://localhost>",
	"<http://localhost:5173>",
		"...Your Domains..."
]

app.add_middleware(
	CORSMiddleware,
	allow_origins=origins,
	allow_credentials=True,
	allow_methods=["*"],
	allow_headers=["*"],
)
```
    
In the `origins` you can add other domains that will be making requests to your api.
    
## Run the API Server Locally

In the terminal use this command to start the terminal locally. It will start on `localhost:8000`.

```bash
uvicorn app.main:app --reload
```

To test your API, go to `localhost:8000/docs` in your browser. This should open Swagger Docs.

![Swagger](https://res.cloudinary.com/dltwftrgc/image/upload/v1686508933/Blogs/langchain-api/swagger_pqcp14.png)

You can input your prompt and check if you are getting the response. Once this works, feel free to play around with LangChain. You can add memory, context etc. At this moment, Deta Space does not support local Vector Databases. So you will have to use remote Vector Databases if you need to save your context files and embeddings.

## Deploy to Deta Space

Once you are happy with the API, commit the changes to git

```bash
git add .
git commit -m "API Works"
```

Initiate Deta Space:

```bash
space new
```

This will create a new `SpaceFile` in the project. Open this file and make and overwrite this to it.

```
# Spacefile Docs: <https://go.deta.dev/docs/spacefile/v0>
v: 0
micros:
  - name: LangChainAPI
	src: ./
	engine: python3.9
	primary: true
	run: uvicorn app.main:app
	presets: 
	  env: 
		- name: OPENAI_API_KEY
		  description: Secret message only available to this Micro
		  default: "OpenAPI Key"
	  api_keys: true
```

Save the file and run this command in the terminal.

```
space push
```

This will create an instance of your API on the Deta Space Dashboard. In this case, it is named “gpt_server”. In your case, it will be “LangChainAPI”.

Go to the instance settings and add your OpenAI API Key from the “Configurations” tab. Then, go to the domains tab and get the base URL of your API. You can test it out in the browser first using the Swagger Docs and then use it in your application as REST API.

![Deta Demo](https://res.cloudinary.com/dltwftrgc/image/upload/v1686508934/Blogs/langchain-api/deta_demo_xpjewt.gif)

---


Want to connect?

🌍[My Website](http://www.akshaymakes.com)  
🐦[My Twitter](https://twitter.com/akshayballal95)  
👨[My LinkedIn](https://www.linkedin.com/in/akshay-ballal/)