# WebRTC

## Summary


WebRTC is a technology that brings real-time communications (RTC) capabilities to the web by natively making these part of a web browser. It also adopts open patent-free components to make this technology available to everyone. For both developers and users, WebRTC lowers the barrier to entry to develop and experience RTC in apps. WebRTC is expected to enable innovative RTC apps for the web.

WebRTC signalling requires a server to set up the connection. Once set up, media and other data can be exchanged directly between peers. Only in some cases when a direct peer-to-peer connection is not possible, another server is needed to relay the media.

## Discussion

### Why was WebRTC invented?

Voice and video calling is something that only telecom companies provided in the past. Later, with VoIP, these calls could be delivered over the internet by even non-telecom companies. Skype, Viber, WhatsApp and WeChat are some examples of voice or video calling services on the internet. But the problem was that to use them users needed to download apps or install plug-ins in their web browsers. For developers, it was difficult to add real-time capabilities to their products and maintain them for different platforms, devices and browsers. Licensing fees meant that only large corporations could afford them. 

WebRTC solves this by embedding real-time communication capabilities into web browsers. Since web users already have browsers, they need not install anything extra. Since WebRTC is part of the browser, developers need not build RTC capabilities from scratch. Developers can focus on building innovative applications powered by WebRTC technology. More importantly, building apps with RTC capabilities is no longer limited to VoIP experts. All web developers can get into it.

WebRTC is open and uses no proprietary or patented components. Thus, even small firms or individuals can get into building WebRTC-based solutions.


### How has been the adoption of WebRTC?

As of January 2018, WebRTC is supported by many major browsers: Chrome, Firefox, Edge, Safari, iOS Safari, Samsung Internet. Object RTC API is supported only by Edge. Opera supports WebRTC. On others such as Internet Explorer, or on older versions of browsers, plugins such as OpenTok can be used. 

It's been said that WebRTC is already in the background and vendors are focusing on delivering effective use cases. This suggests that WebRTC as a technology is stable and mature. Among real-world solutions, RBS is using it to give personalized customer service. Fluke is using it for their remote teams and service staff to collaborate. Esurance uses it to expedite claims processing. At scale, Google Meet and Hangouts is serving 4.7 million minutes of audio/video per day. On Facebook Messenger, 400 million uses voice/video chat a month. Discord is serving 9.5 billion messages a month. Amazon Chime for videoconferencing serves 24.8 million minutes a month. Peer5 is making 1 billion WebRTC connections a day. 


### What are some use cases of WebRTC?

Anything that can benefit from media-rich content, can leverage WebRTC: telehealth, remote classroom, online tutoring, remote interviews, online customer service, online collaboration, cloud-based video conferencing, webinars with audience interaction, adding voice/video capability to traditional social/chat apps, multiplayer gaming, and more. 

The use of the low-latency data channel leads to more use cases. By allowing camera access, you can upload your profile picture to a social website. You could sign into a service biometrically. You could record an interview and send the video to a recruitment company. Two devices can get connected directly without needing to trust a server somewhere. Low-latency broadcasting, peer-assisted data delivery, a web-based torrent client and a streaming media server are more use cases with the data channel. 

WebRTC can be part of embedded devices such as set-top boxes, smart TVs, IP cameras, media-streaming dongles, etc. In other words, WebRTC has a place in the Internet of Things. Data channel can be used for alerts, collect sensor readings or control devices. 

RFC 7478 identifies typical WebRTC use cases and translates them into requirements. 


### What are the some benefits of using WebRTC?

WebRTC is an open standardized browser-based technology free of patents. Developers can easily enhance their traditional web apps with RTC capabilities. Browsers that support WebRTC will include APIs to manage peer-to-peer connections, transport layer, media engines and codecs, thus removing most of the complexities for web developers. It's also easy to interwork with widely deployed SIP/H.323 systems by adding a gateway or supporting WebRTC. 

WebRTC video quality is better than Flash, besides the obvious cost savings in terms of licensing fees. Latency is reduced. Connection times are faster when JS WebSockets are used. Since media is served via browser, HTML5 can be used for customized UI designs. 

Encryption is mandated for all communications including signalling. TLS, DTLS and SRTP are used to secure communications. Since it's part of a browser, the installation process is secure. Access to camera or microphone requires explicit user permission. 


### What's the protocol stack of WebRTC?

WebRTC uses UDP at the transport layer due to its real-time requirements. SRTP and SCTP are used at a higher layer to multiplex streams. For security, DTLS is used. Because peers can sit behind firewalls and NATs, ICE/STUN/TURN are needed. 

For signalling, TLS and TCP are used. To exchange session parameters, SDP is used. 


### What are the standards that define WebRTC?

WebRTC is standardized by two bodies: 

    + **IETF**: IETF specifies formats, protocols and security aspects that enable peer-to-peer WebRTC communications. This is the transport part, what goes "on the wire." Visit WebRTC status page for current status.
    + **W3C**: W3C specifies browser APIs that developers can use from their apps. This is the access and usage part. Visit WebRTC status page for current status.As of January 2018, WebRTC 1.0 standard isn't out yet. Browsers currently implement the proposed APIs with special prefixes. Meanwhile, Object RTC, which is being referred to as WebRTC 1.1, uses JS objects to provide low-level control of media parameters. While such control is possible with SDP, it would require browser changes. 

WebRTC does not standardize the signalling. Vendors are free to choose any signalling and there are plenty of options: Socket.io, WebSocket, XHR/XMLHttpRequest , SIP, XMPP or even custom signalling. Thus, WebRTC does not replace SIP, which is a common myth. In fact, RFC 7118 talks about SIP over WebSocket, which could be used by WebRTC.

While signalling is not standardized, there's an IETF draft titled JavaScript Session Establishment Protocol that provides APIs to control the signalling.


### Could you describe a typical call flow in WebRTC?

Participating peers exchange signalling data via a server to setup a direct peer-to-peer connection. Once such a connection is setup, real-time media traffic is exchanged directly without involving the server. Since peers often sit behind a firewall or a NAT device, they need to know the public IP address and port of the other peer for direct media exchange. This is done using the *ICE framework*. 

ICE uses a *STUN* server to figure out the public address and port of peers. ICE first tries for UDP. If that fails, it attempts for HTTP over TCP. If that fails, it attempts HTTPS over TCP. If STUN succeeds, the media route is direct peer-to-peer. If direct peer-to-peer is not possible, ICE involves a *TURN* server. In this case, media is sent to the TURN server, which simply relays it to the other peer. STUN servers are typically used for asymmetric NAT and TURN servers for symmetric NAT. 


### As a developer, what do I need to build a WebRTC-enabled app?

Your app should call essential WebRTC APIs: 

    + `MediaStream`: Access content streams from camera and microphone. A `MediaStream` object can contain one or more `MediaStreamTrack` objects. For example, audio is a track and video is another track synchronized to the audio. API `getUserMedia()` informs what the app requires, obtains user permissions and then accesses media streams.
    + `RTCPeerConnection`: Gather and exchange local media capabilities and resolve addresses before sending and receiving audio/video content.
    + `RTCDataChannel`: Send and receive non-media content, if your app deals with arbitrary data.You should decide on a method of signalling suited for your app. You will need a server to handle this signalling to set up peer-to-peer connections. You will also need to implement the ICE framework along with STUN and TURN servers. In the case of multiparty calls, you will also need SFUs and MCUs. If this sounds too complex, you should prefer a platform provider to take care of most server-side stuff: Tokbox, Twilio, Agora.io, Xirsys, Temasys, Kandy, etc. 


### What metrics should I look at to assess performance of a WebRTC application?

The following are some common metrics: 

    + **Bandwidth**: End-to-end bandwidth matters. Symptoms of low bandwidth include video freezing, frame-rate drops, choppy audio and call drops.
    + **Latency**: A latency that's below 100 ms is usually not noticeable. High latency affects video more severely than audio. Symptoms of a high latency connection include lag in playback, video freezing and frame-rate drops.
    + **Loss**: Packets are sometimes dropped or lost, particularly on wireless links. Symptoms include video freezing, frame-rate drops and choppy audio. In RTC, loss is often preferred over playback lag due to NACKs and retransmissions. With video, NACKs may be used and when those fail a full frame is requested, which incurs a high bandwidth cost. On a congested network, lowering the bitrate is a suitable approach.
    + **Jitter**: This is common in high-latency networks or when packets traverse a number of hops. Symptoms are similar to packet loss but these symptoms are alleviated when the jitter buffer adapts to network conditions.

### Are there any problems or challenges with WebRTC?

There have been some challenges with WebRTC. Many enterprises are still using legacy browsers including Internet Explorer, which don't support WebRTC. Multiparty video is a drain on battery and taxes processing power available on laptops or mobiles. Enterprise deployments are complex and often involve multiple servers that scale very differently. 

From a standardization perspective, developers are not happy that Object RTC is being proposed while WebRTC 1.0 hasn't been finalized yet. However, ORTC is an evolution of WebRTC 1.0 and parts of ORTC are already into WebRTC 1.0. 

Another challenge may be that different browsers support different video codecs, leading to interoperability problems. 

## Milestones

2009

The idea for WebRTC is born in Google to bring real-time capabilities to the web with built-in support from browsers. Alternatives at the time were Flash, which was low quality and needed server-side license; or plug-ins that needed to be developed for various browsers/platforms. Moreover, users needed to download and install plugins or custom apps. 

May  
2010

Real-time communications need audio/video codecs. In the spirit of openness on the web, Google releases WebM as a patent-free alternative to H.26x codecs. WebM can be traced back to the work of On2 Technologies that Google acquired in 2009. WebRTC uses codecs and media frameworks that are free of patents and royalties. 

Oct  
2010

A one-day workshop is organized to kickoff the standardization process. Standardization involves IETF to define the formats and protocols, and W3C to define APIs that web apps can use on the browser side. 

May  
2011

Google open sources WebRTC under a permissive BSD license. Thus, the term "WebRTC" often refer to the standard and also the Google's open source project hosted at webrtc.org. 

Nov  
2011

Chrome browser adds WebRTC support. 

2013

Firefox starts supporting WebRTC. Chrome and Firefox on Android also start supporting WebRTC. 

Nov  
2017

W3C releases WebRTC 1.0: Real-time Communication Between Browsers as a Candidate Recommendation. 

2020

In the year of the COVID-19 pandemic, the use of WebRTC from within Chrome browser sees 100X increase. During this year, Chrome also becomes 30% more battery friendly for video calling. 

Jan  
2021

W3C publishes WebRTC 1.0: Real-time Communication Between Browsers as a W3C Recommendation. Beyond what's been standardized, improvements continue. Video codec **AV1** that saves up to 50% bandwidth is becoming available in WebRTC implementations. **WebRTC NV** is looking into supplementary APIs to enable new use cases. These are expected to complement 5G and WebAssembly technologies.