Thoughts on Server-Sent Events, HTTP/2, and Envoy
By: Kaitlin Moreno
Tags: Server Sent Events, Http2, JavaScript, Microservices, Envoy Proxy

Editor’s note: The following blog post is part of a series of reflections and observations made by individual GreyMatter.io team members while conducting research and development in support of the Envoy Proxy open source community. This post has also been published at https://dzone.com/articles/thoughts-on-server-sent-events-http2-and-envoy-1.

In a distributed system, moving data efficiently between services is no small task. It can be especially tricky for a front-end web application that relies on polling data from many backend services.

I recently explored solutions to this problem for Grey Matter — specifically, how we could reduce traffic to some of the most requested services in our network. Our web app had the following characteristics:

Polls services for updates every 5 seconds

Doesn’t need bidirectional communication

Doesn’t need to support legacy browsers

Communicates with services behind an Envoy-based proxy

I looked into two server-push solutions — WebSockets and Server-Sent Events (SSE). In the end, I realized WebSockets were overkill for our use case and decided to focus on SSE, mainly because it’s just plain old HTTP and lets us take advantage of some really cool features of HTTP/2.

A quick primer on SSE

SSE is part of the HTML5 EventSource standard and basically just gives us an API for managing and parsing events from long-running HTTP connections.

To implement an SSE client, instantiate the EventSource object and start listening for events. When this code is loaded in the browser, it will open up a TCP connection and send an HTTP request to the server, telling it to push events down the line when it has them. We can even listen for custom events, like dad jokes:

const source = new EventSource(`https://localhost:7080/events`, {

withCredentials: true

});

​

source.addEventListener(“dadJoke”, function(event) {

console.log(event.data)

});

If the connection drops, EventSource will send an error event and automatically try to reconnect!

The server implementation is also very straightforward. For a server to become an SSE server, it needs to

Set the appropriate headers ( text/event-stream )

) Keep track of connected clients

Maintain a history of messages so clients can catch up on missed messages (Optional)

Send messages in a specific format — a block of text terminated by a pair of newlines:

id: 150

event: dadJoke

retry: 10000

data: They’re making a movie about clocks. It’s about time.

That’s pretty much it! There are plenty of SSE server packages out there to make things even easier, or you can roll your own.

Issues with SSE

There are a few common drawbacks you’ll read about with SSE:

No native support for binary types

Unilateral communication

No IE support

No way to add headers with the EventSource object

Max of 6 client connections from a single host

The first 4 points weren’t so much of an issue for us. We don’t need to send anything but JSON to the browser, the client doesn’t need to push data to the server, there are polyfills for IE, and we can attach any headers or route-specific data using our proxy.

The last one is a big deal. One of the biggest perceived limitations of SSE is the browser’s max parallel connection limit, which is set to 6 per domain in most modern browsers following the HTTP/1.1 spec. I read that HTTP/2 solves this issue, so I dove in a bit further to understand how.

From client-pull to server-push

In HTTP (<1.0) the client asks the server for a resource and the server responds. It’s a 1:1 deal.

When HTTP/1.1 came along it introduced the concept of “pipelining”, where multiple HTTP requests are sent on a single TCP connection. That way a client can request a bunch of resources from the server at the same time but only has to open up one connection.

There was a huge problem with the implementation though — the server had to respond in the order it received requests from the client, because there was no other way to identify which request went with which response. This leads to something called the head-of-line blocking problem (HOL), where if the client sends requests A, B, and C, but request A requires a lot of server resources, B and C are blocked until A finishes.

Along comes HTTP/2, which was developed to overcome performance problems with HTTP/1.x. It gives us compressed HTTP headers, request prioritization, and most importantly allows us to multiplex multiple requests over a single TCP connection. Wait, didn’t we already have that last one with HTTP/1.1? Yes and no; because of issues like the HOL problem, there was a lack of reliable pipelining support in browsers, so the web development community went with homegrown solutions for reusing connections instead — domain sharding, concatenation, spriting, and inlining assets.

HTTP/2 is a real multiplexing solution because of how the underlying “framing layer” works. Instead of HTTP/1.x’s plaintext data, HTTP/2 messages are broken down into smaller parts, like headers and request/response bodies and then encoded. These small, encoded chunks are called “frames” and include an ID so they can be associated with a particular message. That means they don’t need to be sent all at once — frames that belong to different messages can be interleaved. When these frames reach their destination they can be reassembled and decoded into a standard HTTP message by HTTP/2 compatible servers.

3 different requests multiplexed over a single TCP connection

As long as connections are behind the same hostname, turning on HTTP/2 will give us many requests as we want over the same connection!

There is another implication here for SSE — since an SSE stream is just a long-running HTTP request, we can have as many individual SSE streams as we want over the same connection. We can send messages from server <-> client AND client <-> server.