A Step-by-Step Guide for Deploying LLMs with MindsDB and OpenAI
Tags: webdev, openai, database, programming
Published: 2023-07-11 13:00:05

## Introduction

A large language model (LLM) is a machine learning model trained on vast amounts of data and can perform natural language processing (NLP) tasks such as analyzing and summarizing text, interpreting languages, answering questions based on sentiments, text generation, and more.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yfi9o5p4b8fqh2mj0lcu.png)

The model generates coherent and contextually relevant human-like responses based on the data it was trained on and the input it receives.

[MindsDB](https://mindsdb.com/) allows you to create models that utilize the OpenAI GPT-4 features and deploy them so that you are ready to perform NLP tasks within your database.

The [MindsDB Generative AI Tables](https://docs.mindsdb.com/what-is-mindsdb) concept makes this possible.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9mqp7m5ykj0uij75fd29.png)

> **AI Tables refer to machine learning models stored within a database as virtual tables.**

In this article, you will learn how to deploy LLMs with MindsDB and OpenAI.

We will leverage the OpenAI engine made available by MindsDB to create and deploy models that can rephrase text, determine word count, and analyze & respond to customers’ reviews.

Additionally, MindsDB allows you to fine-tune some LLMs with specialized or task-specific datasets to improve their performance and applicability to specific tasks.

---

## Pre-Requisites

To follow along with this tutorial, you need a [MindsDB cloud account](https://cloud.mindsdb.com/).

If you prefer a local instance, set up MindsDB via [Docker](https://docs.mindsdb.com/setup/self-hosted/docker) or [pip](https://docs.mindsdb.com/setup/self-hosted/pip/source).

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qnb6jo895chqrxj0y3o1.png)

---

## Connecting to a Database

Let’s start by connecting MindsDB to a public MySQL demo database.

We will use a few tables inside the MySQL public database to demonstrate how to deploy LLMs.

```sql
CREATE DATABASE project_llm
WITH ENGINE = 'mysql',
PARAMETERS = {
    "user": "user",
    "password": "MindsDBUser123!",
    "host": "db-demo-data.cwoyhfn6bzs0.us-east-1.rds.amazonaws.com",
    "port": "3306",
    "database": "public"
};
```
We have successfully connected our database to MindsDB with the name project_llm.

Let's take a look at its tables.

Run the following query to see the tables:

```sql
SHOW FULL TABLES FROM project_llm;
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6r47ywiccrjwbxt121ya.png)


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/itc5nqlxl6lobn6p2ziv.png)

---

## Rephrasing Text

Recall we mentioned that for this tutorial, we will create and deploy 3 different models to rephrase text, determine word count, and analyze & respond to customers’ reviews.

In this section, we will create a model for rephrasing text. We will call it the `rephraser_model`. This model will rephrase email content more politely.

```sql
CREATE MODEL rephraser_model
PREDICT paraphrase
USING
    engine = 'openai',
    prompt_template = 'rephrase content in a more polite way. {{email_content}}'
    api_key = 'YOUR_OPENAI_API_KEY';

```
The Model uses the OpenAI engine. `prompt_template` stores a message to be answered by the model.

If you are on the MindsDB cloud, the `api_key` parameter is optional but mandatory for local usage and MindsDB pro.

For convenience and to avoid repetition, you can create a MindsDB ML engine to hold your API key. This way, you don’t have to enter the API key each time you want to create a model.

```sql
CREATE ML_ENGINE openai2
FROM openai
USING
    api_key = 'YOUR_OPENAI_API_KEY';

```
Check the status of your model. You can only use a model after it has completed generating (this process does not take time to complete).

```sql
DESCRIBE rephraser_model;
```
Now that our Model is ready. Let’s ask it to rephrase some texts for us.

```sql
SELECT email_content, paraphrase
FROM rephraser_model
WHERE email_content = "Jeff, get over here!";
```
**Output**:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/le2wq5ht1wqh7sjhtli5.png)


Let’s do a batch prediction by joining our `rephraser_model` to the `email` table and rephrase some email content more politely.

```sql
SELECT input.email_content, output.paraphrase
FROM project_llm.emails AS input
JOIN rephraser_model AS output
LIMIT 3;

```
**Output**:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/chgo1dwung6yht8b4kjc.png)

---

## Determining Word Count

Following the same process, let’s create and deploy a model to determine the word count of comments.

### Create a model

```sql

CREATE MODEL word_count_model
PREDICT count
USING
    engine = 'openai2',
    prompt_template = 'describe the word count of the comments strictly as "2  words", "3 words" "4 words", and so on.
    "this is great" : 3 words
    "I want this good" : 4 words
    "I want bread right now" : 5 words
    "please take note and effect changes": 6 words
    "{{comment}}.":';
```
Optionally, you can add the `max_tokens` and `temperature` parameters to `USING`. `max_tokens` defines the maximum token of the prediction. `temperature` defines the riskiness of the answer provided by the model.

Check the status of your model, and then go ahead to test it out.

```sql
SELECT comment, count
FROM word_count_model
WHERE comment = 'I like the fact that things are getting better with me daily';
```
**Output**:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0q3dsi9ujkxzh0d8jwj9.png)

Let’s do a batch prediction by joining our model to the `user_comments` table.

```sql
SELECT input.comment, output.count
FROM project_llm.user_comments AS input
JOIN word_count_model AS output
LIMIT 3;

```
**Output**:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w4srz0ttzsiwlyte8ger.png)

---

## Analyze and Respond to Customers’ Reviews

Let's create a model to analyze and respond to customers' reviews.

The model will first analyze and determine if a review is positive or negative.

- If negative, the model responds with an apology and promises to make their experience better next time.

- If positive, it will respond with a thank you note and ask them for a referral.

```sql
CREATE MODEL auto_response_model
PREDICT response
USING
    engine = 'openai',
    prompt_template = 'Give a one sentence response to customers based on the review type.
    "I love this product": "positive"
    "It is not okay" : "negative"

    If the review is negative, respond with an apology and promise to make it better.
    If the review is positive, respond with a thank you and ask them to refer us to their   friends.
    {{review}}';


```
Let’s query the model with some synthetic data.


```sql
SELECT review, response
FROM auto_response_model
WHERE review = "I am not happy about this product";


```
**Output**:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ugo3hpkd9pgefoqqm5ki.png)

Make a batch prediction.


```sql
SELECT input.review, output.response
FROM project_llm.amazon_reviews AS input
JOIN auto_response_model AS output
LIMIT 2;


```
**Output**:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gkz2skbudgk9zh4b7ksp.png)

It is important to note that the models we have built are production ready.

> **MindsDB simplifies the process of building and deploying machine learning models by designing them to be production ready by default so that you don’t need to use an ML Ops flow.**

It makes using machine learning in your database easier and faster without requiring extensive ML expertise.

---

## Conclusion

[MindsDB](https://mindsdb.com/) allows you to use pre-trained OpenAI natural language processing (NLP) models inside your database to perform NLP tasks easily and derive valuable information from text data with only a few SQL queries or by using REST endpoints.

The MindsDB OpenAI integration enables developers to leverage natural language queries to analyze data and efficiently gain a deeper understanding of their data.

Introducing ML capabilities into your database enables you to make data-driven decisions and scale your product by making insightful predictions.

Sign up for a [free demo account](https://cloud.mindsdb.com/) to get a hands-on experience with NLP capabilities with MindsDB.

and mindsDB is one of the most promising artificial intelligence startups in 2023.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h03722avae8c4i6eu20e.png)

That was the end of this blog. Today, I hope you learned something new.

If you did, please like/share it so that others can see it as well.

> Thank you for being a regular reader; you’re a big part of why I’ve been able to share my life/career experiences with you.

Follow MindsDB on [Twitter](https://twitter.com/MindsDB) for the most recent updates, and if you have any questions or problems, you can join their [Slack](https://mindsdbcommunity.slack.com/join/shared_invite/zt-1w5znlu7o-B0FU6z1k2oYm3k0v6Hzx~A#/shared-invite/email) server to discuss and get help.

Follow me on [Twitter](https://twitter.com/TheAnkurTyagi) to get more writing and software engineering articles.