Creating a Generative AI Chatbot using GPT-2 and Flask
Published: 2024-07-09 11:36:00

Generative chatbots are revolutionizing human-computer interaction by allowing machines to not only respond to predefined commands but also generate contextually relevant and coherent responses. This technological advancement has found applications in a variety of sectors, from online customer service to virtual assistance on educational and business platforms. The ability of chatbots to understand natural language and generate near-human responses is transforming the way we interact with technology, offering scalable and efficient solutions for automated communication

## **Chatbot implementation in Flask**

**Initial configuration:**

In this snippet, the necessary libraries are imported and the GPT-2 model and tokenizer are loaded.

```python

from flask import Flask, request, jsonify
from transformers import GPT2LMHeadModel, GPT2Tokenizer

print("Starting application...")

app = Flask(__name__)

# Cargar el modelo y el tokenizer
print("Loading model and tokenizer...")
tokenizer = GPT2Tokenizer.from_pretrained("distilgpt2")
model = GPT2LMHeadModel.from_pretrained("distilgpt2")

print("Model and tokenizer loaded.")

```
 - Flask and other libraries are imported to configure the web server.
 - GPT2LMHeadModel and GPT2Tokenizer are imported from transformers to 
   load the GPT-2 model and its tokenizer.
 - A Flask instance (app) is initialized to create the web application.

**Function to generate responses:**

This snippet defines the generate_response function that uses the GPT-2 model to generate responses based on a given prompt.

```python
def generate_response(prompt):
    print(f"Generating response for prompt: {prompt}")
    inputs = tokenizer.encode(prompt, return_tensors="pt")
    outputs = model.generate(
        inputs, 
        max_length=50,  
        num_beams=5,    
        early_stopping=True,
        no_repeat_ngram_size=2,  
        temperature=0.7,  
        top_k=50,         
        top_p=0.95        
    )
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    print(f"Generated response: {response}")
    return response
```
 - generate_response takes a prompt as input and encodes it using the 
   tokenizer.
 - model.generate generates a response based on the prompt using 
   parameters such as max_length to limit the length, num_beams to 
   improve quality, temperature to control creativity, among others.
 - The result is decoded using the tokenizer to obtain the final 
   response.
**Flask API for the chatbot:**

In this snippet, we configure a POST /chatbot route that accepts JSON requests with a prompt, generates a response using the generate_response function, and returns the generated response as JSON.
```python
@app.route('/chatbot', methods=['POST'])
def chatbot():
    try:
        print("Received request.")
        body = request.get_json()
        prompt = body.get('prompt', '')
        response = generate_response(prompt)
        return jsonify({'response': response})
    except Exception as e:
        print(f"Error: {str(e)}")
        return jsonify({'message': 'Internal server error', 'error': str(e)}), 500

if __name__ == '__main__':
    print("Starting Flask server...")
    app.run(debug=True)
```
 - @app.route('/chatbot', methods=['POST']) defines a /chatbot route that   
   accepts POST methods to receive JSON requests.
 - chatbot() is the function that handles POST requests, gets the JSON 
   request body prompt, generates a response using generate_response, and 
   returns the generated response as JSON.
 - In case of errors, they are captured and handled by returning an error 
   message with status code 500.

## **Testing with CURL**
 - First question:
```bash
curl -X POST http://127.0.0.1:5000/chatbot -H "Content-Type: application/json" -d "{\"prompt\": \"Can you tell me about the weather today?\"}"
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4szdlhn1by6assk6ludy.png)

 - Second question:
```bash
curl -X POST http://127.0.0.1:5000/chatbot -H "Content-Type: application/json" -d "{\"prompt\": \"hello?\"}""
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/djlafbtu5h8wvgrbzkta.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ko6r14qvmtvytifthois.png)

- Third question:
```bash
curl -X POST http://127.0.0.1:5000/chatbot -H "Content-Type: application/json" -d "{\"prompt\": \"how are you?\"}"
```
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3bv4g2guik8kaguw91ql.png)

## **Next Steps:**

We should always try to deploy this type of projects in the cloud, whether AWS or Azure, in this case I leave an illustrative diagram of what the deployment would look like using AWS lambda, AWS Api Gateway and AWS S3.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pyzcnhwoiio1woged3b7.png)

## **Conclusion:**

The combination of GPT-2 and Flask allows the development of chatbots for educational purposes, capable of dealing naturally and effectively with users. These technological advances are rapidly transforming how we think about human-computer interaction, offering scalable and efficient solutions for a variety of applications, from customer service to AI-assisted education, as long as we use the most advanced models such as GPT-4 or GPT. -4o. As we continue to explore and refine these tools, it's clear that generative chatbots represent a significant step toward smarter, more adaptive systems in the digital future.