Scaling OpenVidu
By: Rodrigo Botti
Tags: Scalability, Video Streaming Server, AWS, WebRTC, Openvidu

Architecture

First of all, OpenVidu offers a "premium" paid version called OpenVidu Pro. This is the version we have deployed and that our telemedicine solution is using. We chose to use the paid version for two main reasons:

it offers detailed monitoring of the video call sessions which is great for troubleshooting and analysis

it offers a means for scaling it's media nodes — the server nodes responsible for streaming the video content

OpenVidu Cluster Architecture from OpenVidu Pro Scalability page

Deployment

We deployed the OpenVidu Pro Cluster to our AWS account following their guidelines. It uses a parameterized CloudFormation Stack that takes in some configuration values — at Nexa we are very familiar with CloudFormation so this was a great fit to how we do things when it comes to infrastructure.

Auto Scaling Problem

If you read OpenVidu Pro's feature list you will notice that scalability is a manual effort and that elasticity is still a work in progress.

Manual Scaling in OpenVidu Inspector from OpenVidu Scalability page

"So what gives? Are you telling me I read up until this point just for you to tell me easily found information on how to press a button at a web control panel ?! Where's the automation?" — an understandably frustrated reader.

Calm down fellow reader. I am going to show you how you can automate this process. I was just giving you some context. Just keep on reading it, it will pay off.

One thing I omitted up until this point is that OpenVidu Pro has a REST API and it will play a major role in automating the auto scaling process: we are able to launch and drop media nodes using this API.

Solution

Context

Let’s just list some things we know that will help us decide on a final solution to our auto scaling mechanism.

OpenVidu Pro has a REST API that is able to launch and drop media nodes giving us the ability to scale nodes arbitrarily

All video sessions will always have only two connected users: the patient and the doctor

Video streaming is a CPU intensive process i.e. mainly CPU usage in the media nodes increases as the load — connected users which translates to sessions created — increases

OpenVidu Server — master node — balances the sessions uniformly across it’s media nodes distributing the load

Our telemedicine solution has fixed “office hours” from 8am to 10pm — this is a business rule

From that we have:

We need to scale the media nodes according to their CPU usage

CPU usage will be uniform across all media nodes i.e. the average CPU usage will be practically the same as the CPU usage on any given node

Finally we have what we need to formulate a solution.

Upscaling

Upscaling high-level architecture diagram

CloudWatch Alarm which observes the CPU usage of the media nodes by their AMI

Since the usage is uniform across the nodes, by observing the average usage of the nodes we are indirectly observing the usage in each node

It is easier than monitoring each instance individually — would require an alarm per instance or polling telemetry data from the launched nodes which are dynamic

SNS Topic as the alarm event destination

By using a topic we can have multiple consumers

Lambda subscribing to the topic

Receives the alarm event and calls OpenVidu Pro's REST API to launch a new media node

Should only launch a new media node if there are no media nodes in a launching status — verified by listing media node data using the REST API

Note: to be able to use CloudWatch Alarms to monitor by AMI, the instances need to be launched with detailed CloudWatch monitoring enabled. In order to do that we need to modify the launch script that is used by the OpenVidu Server master node when launching media nodes. More on that in Part 2.

Downscaling

Given that "office hours" are fixed we can trigger a downscale to a single node after hours.

It's not the ideal solution when it comes to cost management — since it could lead to having idle and/or underutilized media nodes during the day which we would still be paying for pointlessly — but it is by far the easiest one to implement.

Downscaling high-level architecture diagram

CloudWatch Event triggered periodically

Lambda triggered from the aforementioned event

Lists media nodes using the REST API

Orders them by number of sessions descending

Drops all except the first — most loaded — using the strategy when-no-sessions

The Code

In Part 2 we'll take a look at some of the source code of the autoscaling solution: