Build A Dual-Purpose App: Text-to-Image and Custom Chatbot Using Comet, GPT-3.5, DALL-E 2, and Streamlit
Tags: python, webdev, ai
Published: 2024-05-29 12:05:46

<h2 class="graf graf--h3" style="padding-left: 40px;">Overview</h2>
<p class="graf graf--p">In this guide, we will explore how to create a dual-purpose application: a chatbot powered by custom dataset and a text-to-image generator, using OpenAI’s GPT-3.5 turbo and DALL-E 2 models, along with Comet and Streamlit.</p>
<img class="size-full wp-image-96752 aligncenter" src="https://www.hitsubscribe.com/wp-content/uploads/2024/04/All-technologies.png" alt="" width="667" height="607" />
<p class="graf graf--p">Now let’s take a brief look at the infrastructures we will be using.</p>

<h2 class="graf graf--h3">Comet</h2>
<p class="graf graf--p"><a class="markup--anchor markup--p-anchor" href="https://www.comet.com/" target="_blank" rel="noopener" data-href="https://www.comet.com/">Comet</a> is a platform that offers real-time experiment tracking with additional collaboration features. With Comet you can log your object detection models (YOLO, Tensorflow), large language models, regression and classification models and the like, with their various parameters.  It also gives you the capability to monitor the training and prompting of all of these models and provides you with the option to share your logged projects publicly or privately with your team.</p>
<p class="graf graf--p">One advantage Comet has over similar platforms is its ability to easily integrate with your existing infrastructure and tools so you can manage, visualize, and optimize models from training runs to production monitoring.  <img class="wp-image-96753 size-full aligncenter" src="https://www.hitsubscribe.com/wp-content/uploads/2024/04/COMET-HOMEPAGE-e1713469576872.png" alt="" width="900" height="421" /></p>

<h2 class="graf graf--h3">GPT-3.5 Turbo Model</h2>
<p class="graf graf--p">According to OpenAI, the GPT-3.5 Turbo is a model that improves on GPT-3.5 and can understand as well as generate natural language or code. With the help of user feedback, OpenAI has improved the GPT-3.5 Turbo language model, making it more proficient at understanding and following instructions. Being a fine-tuned model by OpenAI, it has been given examples of inputs and expected outputs to train (fine-tune) it for a particular task. OpenAI created GPT-3.5 Turbo as an expansion of their well-liked GPT-3 model. The GPT-3.5-Turbo-Instruct is available in three model sizes: 1.3B, 6B, and 175B parameters.</p>

<h2 class="graf graf--h3">DALL-E 2</h2>
<p class="graf graf--p">DALL·E 2 is an AI system that can create realistic images and art from a description in natural language. Below is an image generated by this app by running a prompt “<strong class="markup--strong markup--p-strong">A cup pouring fire as a portal to another dimension.</strong>”</p>
<img class="aligncenter wp-image-96754 size-full" src="https://www.hitsubscribe.com/wp-content/uploads/2024/04/cup-to-another-dimension-e1713469651769.png" alt="" width="850" height="850" />
<h2 class="graf graf--h3">Streamlit</h2>
<p class="graf graf--p"><a class="markup--anchor markup--p-anchor" href="https://streamlit.io/" target="_blank" rel="noopener" data-href="https://streamlit.io/">Streamlit</a> is a platform that enables you to build web applications that can be hosted in the cloud in just minutes. It helps you build interactive dashboards, generate reports, or create chat applications. Once you’ve created an app, you can use the Community Cloud platform to deploy, manage, and share your application.</p>
<p class="graf graf--p">This <a class="markup--anchor markup--p-anchor" href="https://onyejiakutheophilus-dataprofessionals-ap-prediction-page-jdvug7.streamlit.app/" target="_blank" rel="noopener" data-href="https://onyejiakutheophilus-dataprofessionals-ap-prediction-page-jdvug7.streamlit.app/">application</a> is an example of deploying with Streamlit.</p>

<h2>Prerequisites</h2>
In this section, we will quickly take a look at some of the tools you will need to successfully follow along with these steps and ultimately build your own application.
<ul>
 	<li><strong>Python</strong>: A high level programming language for many use cases. For this project, we will be using <strong>Python version 3.9.</strong> This project will still work with any older version of Python. Proceed <a href="https://www.python.org/downloads/">here</a> to download any version of Python from the list of operating systems available. Ensure to add Python to your PC environment variable by following this <a href="https://phoenixnap.com/kb/add-python-to-path">guide</a>.</li>
 	<li><strong>pip: </strong>A package installer used in python. It is very important to have pip running in your PC for you to be able to flow along with this project. See this <a href="https://phoenixnap.com/kb/install-pip-windows">guide</a> on how to install pip and add it to your PC path.</li>
 	<li><strong>Pycharm IDE</strong>: <a href="https://www.jetbrains.com/pycharm/">Pycharm</a>  is the integrated development environment we will be using to build the application. It is simply where we will be writing our code. It is easy to install and saves you a lot of coding time, by assisting with code completion, code navigation, code refactoring and debugging. The community edition of this software is free! Once you create and give a name to any new project, it provides you with a Python virtual environment (venv) that enables the installation of libraries specifically for that project as opposed to sharing them with all users of the computer.</li>
 	<li><strong>Dataset</strong>: The dataset we will be using in this project for training the LLM can be found <a href="https://github.com/prust/wikipedia-movie-data/blob/master/movies-2020s.json">here</a>. Taking a closer look at the dataset structure, as seen in the figure below for the first two movies from the dataset, we will need only the movie's "<strong>title</strong>", "<strong>year</strong>", "<strong>genre</strong>" and the "<strong>extract</strong>". This structure of the dataset is very important to take into consideration; when we get to the coding part of this project, we will look into that.</li>
</ul>
<h3 class="graf graf--h3"><img class="aligncenter wp-image-96807 size-full" src="https://www.hitsubscribe.com/wp-content/uploads/2024/04/dataset-image-e1713470396898.png" alt="" width="950" height="457" /></h3>
<h3 class="graf graf--h3">Now Let’s Get Started!</h3>
<p class="graf graf--p">To achieve our objective, we will be following just 5 simple steps.</p>

<h2 class="graf graf--h3">Step 1: Create a Comet account to log your LLM</h2>
<p class="graf graf--p">Now, if you haven't already, go to <a class="markup--anchor markup--p-anchor" href="https://www.comet.com/" target="_blank" rel="noopener" data-href="https://www.comet.com/">Comet</a> and create a new account. After successfully creating your account, head on to <a class="markup--anchor markup--p-anchor" href="https://www.comet.com/account-settings/apiKeys" target="_blank" rel="noopener" data-href="https://www.comet.com/account-settings/apiKeys">API key section </a>to get a copy of your comet API key.</p>
<img class="aligncenter wp-image-96755 size-full" src="https://www.hitsubscribe.com/wp-content/uploads/2024/04/Comet-Api-key-e1713470412659.png" alt="" width="950" height="441" />
<p class="graf graf--p">Here, you can generate an API key for all your projects. You will need the API key for this part of the project. Copy and save it somewhere.</p>

<h2 class="graf graf--h3">Step 2: Create an OpenAI account to access OpenAI API</h2>
<p class="graf graf--p">If you are new to OpenAI, create your OpenAI account <a class="markup--anchor markup--p-anchor" href="https://chat.openai.com/" target="_blank" rel="noopener" data-href="https://chat.openai.com/">here</a>. Once you’ve successfully created an account, go on to <a class="markup--anchor markup--p-anchor" href="https://platform.openai.com/api-keys" target="_blank" rel="noopener" data-href="https://platform.openai.com/api-keys">API key section</a> by using the same credentials you used when creating your account. On the left panel of the screen, click on “<strong class="markup--strong markup--p-strong">API Keys</strong>” and then proceed to click on “<strong class="markup--strong markup--p-strong">Create new secret key</strong>”. This is shown below:</p>
<img class="aligncenter wp-image-96756 size-full" src="https://www.hitsubscribe.com/wp-content/uploads/2024/04/openai-apikeys-e1713469875863.png" alt="" width="950" height="479" />
<p class="graf graf--p">Next, you get a pop-up, as shown below, asking you to give a name for your secret key. Proceed to give it any name and click the “<strong>Create secret key</strong>” option.</p>
<img class="aligncenter wp-image-96757 size-full" src="https://www.hitsubscribe.com/wp-content/uploads/2024/04/input-a-test-key-e1713469912610.png" alt="" width="950" height="477" />
<p class="graf graf--p">Once done, you get the response to save your key. Ensure to copy and save your API key somewhere as you might loose it if you do not copy it instantly.</p>
<img class="size-full wp-image-96758 aligncenter" src="https://www.hitsubscribe.com/wp-content/uploads/2024/04/api-key.png" alt="" width="1210" height="685" />

&nbsp;
<blockquote class="graf graf--blockquote">Note: This API key is very important to save immediately. Once you close this pop up, you will not be able to get the key again and you will then need to create a new key from scratch. It is therefore necessary you copy it and save it somewhere on your PC. An MS word will do just fine.</blockquote>
<h3 class="graf graf--h3">Building the Application</h3>
<p class="graf graf--p">Now its time to build the application. You can use any IDE of your choice (I used Pycharm). We will need the following libraries for the successful development of this application:</p>

<ul class="postList">
 	<li class="graf graf--li"><strong class="markup--strong markup--li-strong">comet-llm</strong>: This is a tool that will be used to log and visualize our LLM prompts.</li>
 	<li class="graf graf--li"><strong class="markup--strong markup--li-strong">openai</strong>: This is the tool with which we will be using the GPT 3.5 turbo and DALL-E 2 API’s.</li>
 	<li class="graf graf--li"><strong class="markup--strong markup--li-strong">Streamlit</strong>: An open-source framework used for building data science and machine learning applications.</li>
 	<li><strong>json</strong>: Python module for encoding and decoding JSON data.</li>
 	<li><strong>urllib.request</strong>: Python module for making HTTP requests and working with URLs.</li>
</ul>
<h2 class="graf graf--h3">Step 3: Install all Dependencies</h2>
You first create a new project in your Pycharm IDE and give it any name. This way, you automatically have an environment to start coding with your Python interpreter and other packages.
<p class="graf graf--p">Now, in your IDE terminal, run the following commands to install all the dependencies:</p>

<pre class="lang:python decode:true ">pip install openai streamlit comet_llm</pre>
<p class="graf graf--p">Once done successfully, you will need to configure your API key from OpenAI.</p>

<h2 class="graf graf--h3">Step 4: Configure your OpenAI API key</h2>
<p class="graf graf--p">Inside your IDE directory, create a new folder called “.<strong class="markup--strong markup--p-strong">streamlit”</strong> and create a new file, “<strong class="markup--strong markup--p-strong">secrets.toml”</strong> file inside it. It will look like this snippet shown below:</p>
<img class="size-full wp-image-96759 aligncenter" src="https://www.hitsubscribe.com/wp-content/uploads/2024/04/SECRET-FILE.png" alt="" width="700" height="133" />
<p class="graf graf--p">Now open the “<strong class="markup--strong markup--p-strong">secrets..toml” </strong>file and add the following line of text:</p>

<pre class="lang:default decode:true ">MY_KEY = "Now copy your Openai API key you copied before now and paste it here to replace this."</pre>
<p class="graf graf--p">Make sure to replace <strong class="markup--strong markup--p-strong">“Now copy your OpenAI API key you copied before and paste it here.”</strong> with your actual OpenAI API key. After adding this line, save the file.</p>

<h2 class="graf graf--h3">Step 5: Write your Code</h2>
<p class="graf graf--p">Now create a new python script and give it any name. For this, I named mine “<strong class="markup--strong markup--p-strong">dualapp”</strong>. Below is the code to build this dual-purpose app with inline explanation for each line of code.</p>

<pre class="lang:python decode:true ">import streamlit as st
import openai
from openai import OpenAI
import comet_llm
import json
from urllib.request import urlopen

# Initialize OpenAI client
client = OpenAI(api_key=st.secrets["MY_KEY"])


# Load the JSON content of movies from the provided URL
response = urlopen("https://raw.githubusercontent.com/prust/wikipedia-movie-data/master/movies-2020s.json")

# Limit to the first 100 items
train_data = json.loads(response.read())[:100]

# Extract relevant information for training
movie_info = """
Hi! I am a chatbot designed to assist you. 
Here are some movies you might find interesting:
"""

for entry in train_data:
    title = entry.get('title', '')
    year = entry.get('year', '')
    genres = ", ".join(entry.get('genres', []))
    extract = entry.get('extract', 'No extract available')
    movie_info += f"- {title} ({year}) - Genres: {genres}\n"
    movie_info += f"  Extract: {extract}\n"


# Instruction for the model
instruction = """
You are strictly going to answer questions based on the movies provided to you. Do not discuss any other information that
has nothing to do with the movies provided to you. 
I want you to take note of the year, title, genre, and extract of the movies and be able to answer questions on them.
"""

# Combine movie_info and instruction for the system message
system_message = instruction + "\n\n" + movie_info

selection = st.sidebar.selectbox("Chat Bot to Text to Image", ("Custom Chat Bot", "Text to Image"))

if selection == "Custom Chat Bot":
    # Initialize Streamlit UI
    st.title("This is a chatbot about Theo")

    # Initialize chat history
    if "messages" not in st.session_state:
        st.session_state.messages = []

    # Display chat history
    for message in st.session_state.messages:
        if message["role"] == "user":
            st.markdown(f"**You:** {message['content']}")
        elif message["role"] == "assistant":
            st.markdown(f"**💼:** {message['content']}")

    # User input for new chat
    prompt = st.text_input("📝", key="user_input_" + str(len(st.session_state.messages)))

    if prompt:
        st.session_state.messages.append({"role": "user", "content": prompt})

        # Formulate message for OpenAI API
        messages = [{"role": "system", "content": system_message}]
        for message in st.session_state.messages:
            messages.append({"role": message["role"], "content": message["content"]})

        full_response = ""
        for response in client.chat.completions.create(
                messages=messages,
                model="gpt-3.5-turbo",
                stream=True,
        ):
            full_response += (response.choices[0].delta.content or "")
        st.session_state.messages.append({"role": "assistant", "content": full_response})
        st.markdown(f"**💼:** {full_response}")

        # Display user input field for next chat
        st.text_input("📝", key="user_input_" + str(len(st.session_state.messages)))

        # log LLM prompt on comet
        comet_llm.log_prompt(
            api_key="9HibPMbc18shhthis_is_my_api_key",
            prompt=prompt,
            output=full_response,
            metadata={
                "model": "gpt-3.5-turbo"
            }
        )

else:
    # Initialize OpenAI client
    client = OpenAI(api_key=st.secrets["MY_KEY"])

    # Streamlit UI for Text to Image
    st.title("DALL-E-2 Text-to-Image Generation")

    # User input for text prompt
    text_prompt = st.text_input("Enter a text prompt")

    if text_prompt:
        # Use the OpenAI API to generate image from text prompt
        response = client.images.generate(
            model="dall-e-2",
            prompt=text_prompt,
            size="1024x1024",
            quality="standard",
            n=1,
        )

        # Get the generated image URL from the OpenAI response
        image_url = response.data[0].url

        # Display generated image
        st.image(image_url, caption="Generated Image", use_column_width=True)
</pre>
<p class="graf graf--p">Key take-aways from the code above:</p>

<ul class="postList">
 	<li class="graf graf--li">Initialize OpenAI client using the API key you copied from your OpenAI account.</li>
 	<li class="graf graf--li">With the variable <code class="markup--code markup--li-code">system_message</code> we are able to teach or give instruction to our model about any information.</li>
 	<li class="graf graf--li">Initialize the chat history.</li>
 	<li class="graf graf--li">We display the chat history.</li>
 	<li class="graf graf--li">We also provide a new chat for user input right away.</li>
 	<li class="graf graf--li">We formulate the message for OpenAI, then iteratively generate completions from a chat client using a GPT-3.5 Turbo model based on the provided messages.</li>
 	<li class="graf graf--li">We log the LLM prompt on Comet using the API key from Comet.</li>
</ul>
<h2 class="graf graf--h3">Run your App!</h2>
<p class="graf graf--p">Run the command below to run your app. The name I gave to this app is “<strong class="markup--strong markup--p-strong">dualapp” </strong>as mentioned before.<strong class="markup--strong markup--p-strong"> </strong></p>

<pre class="lang:python decode:true ">streamlit run dualapp.py</pre>
<p class="graf graf--p">Bravo! You’ll get the response shown below:</p>
<img class="size-full wp-image-96761 aligncenter" src="https://www.hitsubscribe.com/wp-content/uploads/2024/04/APP-RAN-SUCCESSFULLY.png" alt="" width="791" height="178" />
<p class="graf graf--p">Click on the link in the output message to view your app.</p>
<p class="graf graf--p">This is the home page of the app<img class="alignnone wp-image-96781 size-full" src="https://www.hitsubscribe.com/wp-content/uploads/2024/04/home-page-1-e1713470271585.png" alt="" width="950" height="418" /></p>
<p class="graf graf--p">Below is a Prompt using the chat bot</p>
&nbsp;
<p class="graf graf--p"><img class="aligncenter wp-image-96782 size-full" src="https://www.hitsubscribe.com/wp-content/uploads/2024/04/Prompt-in-app-and-a-corresponding-log-on-comet-e1713470299390.png" alt="" width="950" height="418" />Below is the corresponding LLM log on comet. Visit <a class="markup--anchor markup--p-anchor" href="https://www.comet.com/theophilus/llm-general/prompts" target="_blank" rel="noopener" data-href="https://www.comet.com/theophilus/llm-general/prompts"><strong class="markup--strong markup--p-strong">here</strong></a> to view this page. Make sure to click on “Columns” in order to select the variables of the table you want to see as shown in the figure below:</p>
&nbsp;
<p class="graf graf--p"><img class="aligncenter wp-image-96779 size-full" src="https://www.hitsubscribe.com/wp-content/uploads/2024/04/Log-on-Comet-1-e1713470314669.png" alt="" width="950" height="373" />Now lets explore the app:</p>
[video width="1280" height="644" mp4="https://www.hitsubscribe.com/wp-content/uploads/2024/04/YouCut_20240409_113404658.mp4"][/video]
<h3 class="graf graf--h3">Summary</h3>
<p class="graf graf--p">To successfully create this dual purpose app that integrates both text-to-image and custom chatbot, we followed the following steps:</p>

<ul class="postList">
 	<li class="graf graf--li"><strong class="markup--strong markup--li-strong">Step 1</strong>: Create a Comet account to log your LLM.</li>
 	<li class="graf graf--li"><strong class="markup--strong markup--li-strong">Step 2</strong>: Create an OpenAI account to access your OpenAI API keys.</li>
 	<li class="graf graf--li"><strong class="markup--strong markup--li-strong">Step 3</strong>: Install all dependencies.</li>
 	<li class="graf graf--li"><strong class="markup--strong markup--li-strong">Step 4</strong>: Configure your OpenAI API key.</li>
 	<li class="graf graf--li"><strong class="markup--strong markup--li-strong">Step 5</strong>: Write your code.</li>
</ul>
<p class="graf graf--p">Thank you for your time!</p>
<strong>Credit: Dataset from <a href="https://github.com/prust">Peter Rust</a> </strong>