An architectural overview for WebRTC — A protocol for implementing video conferencing
By: Eytan Manor
Tags: Nodejs, WebRTC, Zoom, JavaScript, Remote Working

I didn’t know much of video conferencing when I only got started. After investigating the matter thoroughly, I came across WebRTC — a protocol which is responsible for Real Time Communication (thus, RTC).

WebRTC is not necessarily intended for video conferencing, but was definitely built with that in mind. By today’s standards, a latency of less than a second is considered to be real time. WebRTC is the fastest solution as for today, and to top it all off, it’s open-source, which makes the technology free of charge. Any other solution falls behind in terms of latency, but keep in mind that they weren’t built to give us real time performance, and they serve different purposes. Below is a latency comparison diagram, just so you can understand how fast WebRTC really is:

When I started to get into WebRTC, I realized that it’s architecturally complex. To make it work, I needed to setup several applications. It doesn’t sum up with a single methodology like REST or web sockets, many components are involved. When I followed tutorials, I did exactly as I was instructed to, but quickly fell into realization that things just don’t add up. As soon as I tried to make a slight change into the system, like adding the ability to have a call with more than 2 people, things started to fall apart. Knowing what I know now, I came into a very important conclusion: you have to understand how WebRTC operates on the architectural level, moreso than its API.

And so, in this article, I would like to talk about how WebRTC works, have an overview about the components involved, and how they communicate with each other. I will not talk about anything specific to its API, as I feel like there are plenty tutorials and docs about it, yet a handful (good) articles about its architecture.

WebRTC overview

WebRTC is a protocol that was designed to enable direct communication between browsers. It includes a set of classes and methods to standardize the process, and it is available ever since Chrome 23:

See: RTCPeerConnection, the most primitive WebRTC class

Beside standardizing the communication process, the browsers give you an easy and secure access to the hardware, which is complementary to WebRTC. You can stream your screen, your microphone, and your camera; which will normally require you to install external plug-ins or binaries, and can get quite complicated, considering that each OS and hardware require different (and complicated!) configs. Originally I was trying to implement screen sharing with ffmpeg; I did manage to make it work, but ran into many compatibility issues.

We’re really blessed to have that out of the box, but I should probably talk about the media part in another article.

Peer connections

WebRTC is based on p2p architecture (peer to peer); the participants of the call are responsible for transferring data from one end to another, without relying on a middleman (for the most part, I’ll cover that later). If one participant disconnects for whatever reason, the others will keep broadcasting data; unlike traditional communication, where data is no longer streamed if connection to the server is lost. In addition, peers are geographically much closer to one another, so the data doesn’t have a long distance to travel.

Accordingly, when I enter a conversation, I have to represent each peer with a dedicated instance. So given that I have a total of 4 peers in the conversation, including myself, I’ll have 3 peer instances, where each one is directly linked to a different browser; this is how the mesh would look like:

A system of 4 peers

Signaling server

As the call goes, I’ll have to keep track of people who join or leave the conversation, and create or dispose connections respectively. To keep track of these events, we need to have a signaling server.

A signaling server is dedicated to establish the initial connection between 2 or more peers who would like to communicate. Once the connection has been established, you wouldn’t need to use it for the on-going communication. You might use it however if you would like to signal additional events, e.g. a peer has disconnected; it’s up to you.

The signaling server can be implemented in many ways, all you need is a bridge between peer A and peer B. You can use anything from REST, to theoretically copy-pasting via email, but normally you would like to use web sockets for this kind of scenario, because communication can be spontaneously initiated at any time:

When I join a conversation, I broadcast it with the signaling server so everyone can know about it

SDP

Once we know that someone has joined the conversation, we need exchange information about each other’s systems inorder to establish a connection. This information is based on a protocol called SDP (session description protocol), and it includes details about its belonging peer e.g. what agent is it using, what hardware does it support, what type of media would it like to exchange, etc. The SDP config is a simple key-value object:

An SDP config can either represent an answer, or an offer. Whenever we would like to initiate a connection establishment we make an offer, and in return, we should get an answer. Offer / answer are bi-directional, what I mean by that is that it doesn’t matter which side initiates the connection, the outcome will be the same.

However, it is important to keep track of what end does the SDP config in question represents: us, or the other peer. When initializing a peer instance, we would need 2 things: a local description, and a remote description. A local description, represents us, and a remote description, represents the other end. Together, we can successfully establish a connection:

Connecting 2 peers with their SDPs

ICE candidates

A peer might have many communication transports, not just one. Someone might have multiple private IPs/ports, and/or multiple public IPs/ports, and/or various protocols, and/or one or more reverse proxies, etc. As soon as we create an SDP offer, WebRTC will try to find every possible communication transport to the browser, which is known as ICE candidate (interactive connection establishment):

An actual RTCIceCandidate instance

An ICE candidate is just another key-value pair that should be added to the SDP. We can either wait for WebRTC to find every possible candidate and send a complete SDP, or we can send each detected ICE candidate with the signaling server and gradually extend the SDP; both options are valid. WebRTC should know how to alternate between ICEs and pick the most viable option.

By default, WebRTC will give preference to ICEs which are based on UDP (User Datagram Protocol). Unlike TCP (Transmission Control Protocol, the traditional one used by HTTP), where packets are not streamed unless prior packets are 100% sent, UDP will keep streaming packets regardless of the state of prior packets, making the communication much faster.

As soon as we create an SDP, WebRTC starts looking for ICE candidates

NAT

Today, most machines aren’t connected directly to the global network, and they most likely go through a NAT layer (Network Address Translation). Your machine’s private IP/port will literally be translated to a different public IP/port when transporting through the router.

Since WebRTC strives to achieve as directly as possible connection between 2 parties, the fact that either of them goes through a proxy arises some complications in the process, such that we should be aware of. Let’s have a look at the different NAT configs, and see how we can establish a direct connection using them (I took the definitions directly from dh2i.com, I couldn’t put it any better myself):