# A Cloud-Based Distributed Architecture to Accelerate Video Encoders
## 3 Proposed Architecture
### Virtual Infrastructure  
The logical view of the architecture is shown in Figure 1. Two networks are defined: a self service job network used to send and receive messages that encapsulate video encode jobs and a provider media network used to transfer video data.  
Besides, three types of VMs have been defined: a _Master Video_ (master), _Worker Video Encoders_ (workers), and a _Media Storage_ (media). The roles of these VMs are the following:  
* Master VM has installed a message broker, RabbitMQ ([[https://www.rabbitmq.com/](https://www.rabbitmq.com/)]([https://www.rabbitmq.com/](https://www.rabbitmq.com/))) in our deployment, and a Java Runtime Environment (JRE) ([[https://www.oracle.com/java](https://www.oracle.com/java)]([https://www.oracle.com/java](https://www.oracle.com/java))). This node hosts a Java application that allows the user to specify: the video to be encoded, the encoder, the location of the video in media, the bit rates, the chunk size, or the total number of frames, among others. The number of chunks is calculated, and for each chunk a message with all the information to encode that chunk (chunk number, bitrates, location of the video, where to put the encoded videos, the encoder, etc.) is sent to the message broker.
* Each worker VM has installed a JRE and ffmpeg[22] with support for H.265/HEVC using the x265 encoder [23] and for VP9 using the libvpx-vp9 encoder [24]. These nodes run a Java application that subscribes to the queue where video job messages are stored. When the application in a worker node receives a message to encode a particular chunk, it downloads the video chunk from the media and starts an encoder process using the configuration specified in the message. Finally, the encoded chunk is uploaded to the media.
* Media VM contains the videos to be processed, provides chunks of these videos, and stores the encoded chunks.  
With respect to the networking, the _Media Network_ uses Single Root I/O Virtualization (SR-IOV) in order to achieve near native performance in the transference of video chunks. Before starting a new VM connected to the _Media Network_, a new OpenStack neutron port is created in this network. This port creates a new Virtual Function in a SR-IOV capable network interface at the compute node. This port is assigned at boot time to the VM. Each worker is connected to two networks: the _Media Network_, to access the media VM, and the _Job Network_ to connect to the master VM. The media VM is  
Figure 1: Logical view of the proposed virtual infrastructure.  
connected to the _Media Network_ and to the _Job Network_. The master VM only requires a connection to the _Job Network_.  
Regarding storage, when the media or a worker is started, a new disk volume is created in the compute node where the VM is allocated and, using virsh in the compute node, the disk is attached to the virtual machine. In this way, we achieve locality between data and computation.  
All the VMs started in the infrastructure are configured using a cloud-init (a multi-distribution package that handles early initialization of the cloud instance) script provided at boot time. These scripts format the attached disk volume, initialize directories, obtain and install compressed file artifacts from a web server, and finally run the application. VMs have access to the Internet through the external network, but only master and media can be reached from outside.