Stream OpenAI's ChatGPT responses directly in the browser.
Tags: openai, chatgpt, browser, api
Published: 2023-06-20 17:40:00

In this blog post you will learn how to stream OpenAI API responses directly in the browser, without the need for a backend as a middle-man between OpenAI and the browser application and without leaking your OpenAI API key to the world.

## Stream the response through [Signway](https://www.signway.io)

Signway is an open source and managed solution for generating pre-signed URLs, like the ones [AWS S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ShareObjectPreSignedURL.html) uses for sharing objects, but in this case for proxying correctly signed requests to destination APIs, and stream the response back. Its core is open source, so you can also host it yourself https://github.com/gabotechs/signway.

## For following this tutorial you will need
- [Python +3.9](https://www.python.org/downloads/) or greater installed in you system
- An [OpenAI Api key](https://platform.openai.com/account/api-keys)

## Create a Signway Application

- Go to [Signway's web page](https://www.signway.io) and log into the console.
- Once you log in you will be prompted to create an Application.
- After giving a nice name to the Application, you will see a pop-up sowing the newly created Application ID and Secret. Copy them and store them in a safe place, we will need them later.
- Once the Application is created, go to "Custom Headers" and add the "Authorization" header that will be sent to OpenAI.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kbk5roc08o536byx4uu4.jpeg)

## Code a function that generates pre-signed URLs for Signway in Python 🐍

We will be using Python for this, but other languages could be used for generating pre-signed URLs.

First, create a virtual environment and install the Signway SDK:

```bash
python3 -m venv venv
source venv/bin/activate
pip install signway-sdk
```

Then create a script that generates pre-signed URLs. We will call this script `sign.py`:

```python
# sign.py
from signway_sdk import sign_url
import os

print(sign_url(
  id=os.environ['SW_ID'],
  secret=os.environ['SW_SECRET'],
  host="https://api.signway.io",
  proxy_url="https://api.openai.com/v1/completions",
  expiry=900,
  method="POST"
))
```

Export the application secrets saved from the earlier step:

```bash
export SW_ID=""
export SW_SECRET=""
```

> Remember to populate this variables with the Application ID and Application Secret that the Signway page gave you.

Executing this script gives us a URL that looks like this:

```bash
$ python3 sign.py

https://api.signway.io/?X-Sw-Algorithm=SW1-HMAC-SHA256&X-Sw-Credential=app-id%2F20230619&X-Sw-Date=20230619T195222Z&X-Sw-Expires=300&X-Sw-Proxy=https%3A%2F%2Fapi.openai.com%2Fv1%2Fcompletions&X-Sw-SignedHeaders=&X-Sw-Body=false&X-Sw-Signature=4aa60955f477eeb2bd556c41ecce3508d02b5b83bec9c0c188737998f73c5176
```

That is a signed URL, it can be safely transferred to clients  as it does not contain any sensible information. In this case, we have set an expiration time of 15 mins, after that time the URL will be useless.

## Create the frontend code that will stream the response

For this we will use a very simple example without using any framework:

> Remeber to execute `python3 sign.py` and copy-paste the generated URL in the `const URL = ""` variable

```html
<script>
    // Execute 'python3 sign.py' and paste the result URL
    const URL = ""

    fetch(URL, {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({
            model: "text-davinci-003",
            stream: true,
            prompt: "Say this is a test"
        }),
    }).then(async res => {
        const p = document.querySelector('#response')
        const reader = res.body.getReader()
        const decoder = new TextDecoder()
        while (true) {
            const { done, value } = await reader.read()
            p.innerText += decoder.decode(value)
            if (done) break
        }
    })
</script>
<html>
    <p id="response"/>
</html>
```

If you open this file in your browser you will see how the response from Open AI is streamed.

## What's next?

- You can do more things with Signway, like providing extra security by signing more things in the request, like the body or some headers. There is a cool tutorial about this [here](https://gabotechs.github.io/signway/openai.html)
- The natural thing to do is to generate pre-signed URLs from a trusted source, like your backend, you can use the excellent [FastAPI](https://fastapi.tiangolo.com/lo/) framework for that.
- Make sure to pass by the [Github repo](https://github.com/gabotechs/signway) and give it a star if you like it.
- Feel free to play around the free tier of [Signway Managed](https://www.signway.io), but join the waitlist if you are willing to take it to the next level.