Project: YugabyteDB (App Definition and Development / Database)
File: ai-langchain-openai.md

title: How to Develop LLM Apps with LangChain, OpenAI and YugabyteDB headerTitle: Build LLM applications with LangChain, OpenAI, and YugaybteDB linkTitle: LangChain and OpenAI description: Learn to build context-aware LLM applications using LangChain and OpenAI. image: /images/tutorials/ai/icons/langchain-icon.svg headcontent: Use YugaybteDB as the database backend for LLM applications menu: preview_tutorials: identifier: tutorials-ai-langchain-openai parent: tutorials-ai weight: 60 type: docs This tutorial demonstrates how to use to build applications with LLM integration. LangChain is a framework for developing applications powered by language models. Follow the guide to learn how to build a scalable LLM application using the chain capabilities of LangChain in conjunction with an LLM. The application can query a YugabyteDB database from natural language. Python 3 Docker Download the application and provide settings specific to your deployment: Clone the repository. ```sh git clone https://github.com/YugabyteDB-Samples/yugabytedb-langchain-openai-shoe-store-search.git ``` Install the application dependencies. Dependencies can be installed in a virtual environment, or globally on your machine. Option 1 (recommended): Install Dependencies from requirements.txt* in virtual environment: ```sh python3 -m venv yb-langchain-env source yb-langchain-env/bin/activate pip install -r requirements.txt ``` Option 2: Install Dependencies Globally: ```sh pip install langchain pip install psycopg2 pip install langchain_openai pip install langchain_experimental pip install flask pip install python-dotenv ``` Create an and store its value in a secure location. This will be used to connect the application to the LLM to generate SQL queries and an appropriate response from the database. Configure the application environment variables in `{project_directory/.env}`. Start a 3-node YugabyteDB cluster in Docker (or feel free to use another deployment option): ```sh mkdir ~/ybdockerdata docker network create custom-network docker run -d --name yugabytedb-node1 --net custom-network \ -p 15433:15433 -p 7001:7000 -p 9001:9000 -p 5433:5433 \ -v ~/ybdockerdata/node1:/home/yugabyte/yb_data --restart unless-stopped \ yugabytedb/yugabyte:{{< yb-version version="preview" format="build">}} \ bin/yugabyted start \ --basedir=/home/yugabyte/ybdata --background=false docker run -d --name yugabytedb-node2 --net custom-network \ -p 15434:15433 -p 7002:7000 -p 9002:9000 -p 5434:5433 \ -v ~/ybdockerdata/node2:/home/yugabyte/yb_data --restart unless-stopped \ yugabytedb/yugabyte:{{< yb-version version="preview" format="build">}} \ bin/yugabyted start --join=yugabytedb-node1 \ --basedir=/home/yugabyte/ybdata --background=false docker run -d --name yugabytedb-node3 --net custom-network \ -p 15435:15433 -p 7003:7000 -p 9003:9000 -p 5435:5433 \ -v ~/ybdockerdata/node3:/home/yugabyte/yb_data --restart unless-stopped \ yugabytedb/yugabyte:{{< yb-version version="preview" format="build">}} \ bin/yugabyted start --join=yugabytedb-node1 \ --basedir=/home/yugabyte/ybdata --background=false ``` The database connectivity settings are provided in the `{project_dir}/.env` file and do not need to be changed if you started the cluster using the preceding command. Navigate to the YugabyteDB UI to confirm that the database is up and running, at <http://127.0.0.1:15433>. This application requires an e-commerce database with a product catalog and inventory information. This schema includes `products`, `users`, `purchases`, and `product_inventory`. It also creates a read-only user role to prevent any destructive actions while querying the database directly from LangChain. The `pg_trgm` PostgreSQL extension is installed to execute similarity searches on alphanumeric text. Copy the schema to the first node's Docker container. ```sh docker cp {project_dir}/sql/schema.sql yugabytedb-node1:/home ``` Copy the seed data file to the Docker container. ```sh docker cp {projectdir}/sql/generateddata.sql yugabytedb-node1:/home ``` Execute the SQL files against the database. ```sh docker exec -it yugabytedb-node1 bin/ysqlsh -h yugabytedb-node1 -c '\i /home/schema.sql' docker exec -it yugabytedb-node1 bin/ysqlsh -h yugabytedb-node1 -c '\i /home/generated_data.sql' ``` The Flask server for this application exposes a REST endpoint which returns values from the database. Start the server. ```sh python3 app.py ``` ```output Running on http://127.0.0.1:8080 ``` Send a POST request to the `/queries` endpoint with a relevant prompt. For instance: ```sh curl -X POST http://localhost:8080/yourendpoint -H "Content-Type: application/json" -d '{"userprompt":"Find me shoes that are in stock and available in size 15."}' ``` ```output.json [ { "color": [ "yellow", "blue" ], "description": null, "name": "Efficient Jogger 1", "price":

"quantity": 22, "width": [ "narrow", "wide" ] }, { "color": [ "blue", "yellow" ], "description": null, "name": "Efficient Jogger 8", "price": "143.63", "quantity": 85, "width": [ "wide", "narrow" ] }, ... ] ``` The Python application relies on a custom prompt to provide additional context to the LLM, in order to generate more relevant responses. For instance, sample queries are provided to help suggest SQL syntax to OpenAI: ```output ... Query the database using PostgreSQL syntax. The color and width columns are array types. The name column is of type VARCHAR. An example query using an array columns would be: SELECT * FROM products, unnest(color) as col WHERE col::text % SOME_COLOR; or SELECT * FROM products, unnest(width) as wid WHERE wid::text % SOME_WIDTH; An example query using the name column would be: select * from products where name ILIKE '%input_text%'; ... ``` After setting the context, the user input can be used to complete the prompt. ```output ... Generate a PostgreSQL query using the input: {input_text}. Answer should be in the format of a JSON object. This object should have the key "query" with the SQL query and "query_response" as a JSON array of the query response. ``` After generating a prompt, the LLM and YugabyteDB database work in conjunction to generate a response. The application connects to the database, defining the columns to use when executing queries. The OpenAI LLM is instantiated and added to the `SQLDatabaseChain`. This chain is then invoked, generating and executing a SQL query and subsequently returning the data in the format defined in our prompt. ```python prompt = customprompt(userprompt) sqldatabase = SQLDatabase.fromuri(LOCALDBURLSTRING, includetables=["products", "users", "purchases", "productinventory"]) llm = OpenAI(temperature=0, max_tokens=-1) dbchain = SQLDatabaseChain(llm=llm, database=sqldatabase, verbose=True, usequerychecker=True,returnintermediatesteps=True) try: nwans = dbchain.invoke(prompt) result = nw_ans["result"] result_json = json.loads(result) return resultjson["queryresponse"] except (Exception, psycopg2.Error) as error: print(error) ``` ```sh Entering new SQLDatabaseChain chain... Query the database using PostgreSQL syntax. Use the shoecolor enum to query the color. Do not query this column with any values not found in the shoecolor enum. Use the shoewidth enum to query the width. Do not query this column with any values not found in the shoewidth enum. The color and width columns are array types. The name column is of type VARCHAR. An example query using an array columns would be: SELECT * FROM products, unnest(color) as col WHERE col::text % SOME_COLOR; or SELECT * FROM products, unnest(width) as wid WHERE wid::text % SOME_WIDTH; An example query using the name column would be: select * from products where name ILIKE '%input_text%'; It is not necessary to search on all columns, only those necessary for a query. Generate a PostgreSQL query using the input: How many narrow shoes come in pink? How about blue?. Answer should be in the format of a JSON object. This object should have the key "query" with the SQL query and "query_response" as a JSON array of the query response. SQLQuery:SELECT COUNT() FROM product_inventory WHERE color = 'pink' AND width = 'narrow' UNION ALL SELECT COUNT() FROM product_inventory WHERE color = 'blue' AND width = 'narrow' SQLResult: [(4,), (5,)] Answer:{"query": "SELECT COUNT() FROM product_inventory WHERE color = 'pink' AND width = 'narrow' UNION ALL SELECT COUNT() FROM productinventory WHERE color = 'blue' AND width = 'narrow'", "queryresponse": [{"count": 4}, {"count": 5}]} Finished chain. ``` LangChain provides a powerful toolkit to application developers seeking LLM integrations. By connecting to YugabyteDB, users can access data directly, without the need for replication. For more information about LangChain, see the . If you would like to learn more on integrating OpenAI with YugabyteDB, check out the tutorial.