How ChatGPT Uses Server-Sent Events to Stream Real-Time Conversation
Tags: chatgpt, sse, firstpost
Published: 2023-06-28 07:21:49

Whether you're a developer or not, you're probably using ChatGPT in your daily work. But have you ever wondered how ChatGPT manages to deliver real-time conversations that feel so incredibly human?

In this article, I'll show you how the ChatGPT API uses SSE (Server-Sent Events) to stream responses in real time as they're being generated. Additionally, I'll guide you through the process of creating your own basic SSE endpoint using Node.js and Express.js.

### So what is SSE?

> In simple words, It's one request and a very long response.

SSE is one of my favorite backend communication patterns. It allows a server to push data to a client as soon as it becomes available, without the client having to continuously ask or [poll](https://www.geeksforgeeks.org/what-is-long-polling-and-short-polling/) the server for new data. This allows for real-time updates to be delivered to the client without the need for constant requests.

In the case of ChatGPT, when you send a message, the server may immediately start generating a response using machine learning algorithms. As soon as the server generates a new piece of text, it can send it to the client using SSE, which allows the client to render the response as soon as it arrives.

**It's worth noting that SSE is a unidirectional protocol, meaning that the client cannot send data back to the server using the same connection.**

### Implementation

Now that you're familiar with SSE, Let's try to implement a basic SSE endpoint using Node. JS & Express. JS.

* To get started, Set up your Node.js project and install the necessary dependenciess
    

```bash
$ mkdir sse-example
$ cd sse-example
$ npm init -y
$ npm install express
```

* Now, create a `index.js` file and initialize an express app. Then define a route to handle SSE connection.
    

```js
const express = require('express');
const app = express();

app.get('/', (req, res) => {  
	res.send(`  
	<!DOCTYPE html>  
	<html>  
		<body>  
			<h1>SSE Example:</h1>  
			<div id="result"></div>  
			<script>  
				var source = new EventSource("/stream");  
				source.onmessage = function(event) {  
				document.getElementById("result").innerHTML += event.data + "<br>";  
				};  
			</script>  
		</body>  
	</html>  
`)  
})

app.get('/stream', (req, res) => {
  // Required header: It tells client that the response will be an event stream
  res.setHeader('Content-Type', 'text/event-stream');
  sendCount(res);
});

let counter = 0;
function sendCount(res) {
  setInterval(() => {
    res.write(`data: Count: [${counter++}]\n\n`);
  }, 1000);
}
```
Let me break down the code for you:
- First there's a GET endpoint `("/")`, that serves an HTML page.
  - The HTML page includes some markup and a script tag.
  - Within the script tag, an `EventSource` object is created with the URL `/stream` to establish a connection between the client and the SSE endpoint.
  - The `onmessage` event listener is set on the `source` object, for capturing events received from server.
  - Inside the callback, we insert the data to the `div` to display received message in HTML page.
- Then there's GET endpoint `("/stream")`, which will be used to stream real-time data.
  - I have created a function called `sendData` that takes the response object and sends data every 1 second using `res.write`. This mimics a real-world scenario where GPT generates responses through its machine learning algorithms.
  - It is worth mentioning that you have to send the word `data:` in every event because it determines the boundary of an event. By starting event data with word `data:` and ending with `\n\n`, allows clients to capture and parse the events correctly.
            
After setting up these two endpoints, start the server and listen on port `8080`.
    
```js
const server = app.listen(8080, () => {
  console.log('Server is listening on port 8080!');
});
```

Now, Open your browser, and head over to [http://localhost:8080](http://localhost:8080).
    
You should see some text being rendered on the page every 1 second, representing the events received from the server.

![Screenshot](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8zobl2mxaxketdpvt0gh.gif)    

This is just a basic implementation to get you started. You can build upon this foundation to create more complex SSE applications tailored to your specific needs.

---

Thanks for reading!