Interpreting HTML5 Video Events
Tags: javascript, streaming, typescript, html5video
Published: 2023-03-13 12:05:55

When building a HTML5 web video player, it tends to need a user interface, quality of service tracking, user interaction tracking, and sometimes ad tracking. To achieve this, the [HTML5 Media Event](https://html.spec.whatwg.org/multipage/media.html#mediaevents) standard needs to be interpreted.

After building multiple web video players, and having worked on more than one iteration of an event interpreter, something didn't feel right. Things weren't kept [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself).

When repetition strikes, it's usually a good idea to do something about it.

## Creating a Media Event Filter

Enter [@eyevinn/media-event-filter](https://www.npmjs.com/package/@eyevinn/media-event-filter), a filter that interprets HTML5 media events in a way that makes sense.

> This filter aims to provide a single source of truth that can be used across player engines and native browser playback.

Whether using one or multiple player engines, like [Shaka Player](https://github.com/shaka-project/shaka-player/), [HLS.js](https://github.com/video-dev/hls.js/), or [dash.js](https://github.com/Dash-Industry-Forum/dash.js/), the filter will output a standardised event sequence. It does so without relying on events output by the player engine, giving applications a single, player agnostic, source of truth for video playback state updates.

## Example Implementation

```typescript
import {
  getMediaEventFilter,
  FilteredMediaEvent,
} from "@eyevinn/media-event-filter";

const videoElement = document.createElement("video");

const mediaEventFilter = getMediaEventFilter({
  videoElement,
  callback: (event: FilteredMediaEvent) => {
    switch (event) {
      case FilteredMediaEvent.LOADED:
        // handle loaded
        break;
      case FilteredMediaEvent.BUFFERING:
        // handle buffering
        break;
      case FilteredMediaEvent.BUFFERED:
      // handle buffered
      // ...
      default:
        break;
    }
  },
});

// Create a player, using your engine of choice
const player = createPlayer(videoElement);

// Load a manifest and play
player.load(url).then(player.play);

// Call when done
mediaEventFilter.teardown();
```
## Live Example

See a barebones [React + Shaka + Media Event Filter](https://codepen.io/atlimar/pen/wvEmpXM) example over at codepen.

## Production Readiness

While making the filter open source is recent, it has been battle tested for years. With millions of playback sessions, multiple third party tracking systems have been validated using the filter.

## Event Sequence

The sequence follows the [Eyevinn Player Analytics Specification](https://github.com/Eyevinn/player-analytics-specification), which maps directly to many popular tracking service provider SDKs.

Sample event sequence comparison, using `<video autoplay>` with Shaka Player in Firefox, excluding `timeupdate`:

Native | Filtered | Comment
---------- | ---------- | ----------
ratechange | — | Shaka sets playback rate to 0 to control buffering
progress | — |
durationchange | — |
resize | — |
loadedmetadata | — |
ratechange | — |
progress | — |
loadeddata | — |
canplay | — |
play | — |
playing | — |
canplaythrough | loaded | video is ready to start playing
— | playing | video is playing, play event missing due to autoplay
progress | — |
... | — |
progress | — |
pause | pause | manual pause
progress | — |
seeking | seeking | seeking while paused
ratechange | — | Shaka sets playback rate to 0 to control buffering
play | play | Play requested during a seek
waiting | — |
progress | — |
progress | — |
progress | — |
seeked | — |
canplay | — |
playing | — |
canplaythrough | — | video element thinks it can start playing, but playback rate is 0
progress | — |
ratechange | seeked | shaka returns playback rate to normal
— | playing | video is playing again after previous pause
progress | — |
progress | — |
pause | pause |
progress | — |
play | play |
playing | playing |
progress | — |
progress | — |
seeking | seeking | seeking while playing
waiting | — |
ratechange | — |
progress | — |
progress | — |
seeked | — |
canplay | — |
playing | — |
progress | — |
canplaythrough | — |
progress | — |
ratechange | seeked | seek finished, video is rolling
progress | — |
progress | — |
progress | — |
progress | — |
ratechange | buffering | buffer empty, shaka sets playbackrate to 0
progress | — |
progress | — |
progress | — |
canplaythrough | — | video element thinks it can start playing, but playback rate is 0
progress | — |
ratechange | buffered | shaka returns playback rate to normal
progress | — |
... | — |
progress | — |
seeking | seeking |
waiting | — |
ratechange | — |
progress | — |
... | — |
progress | — |
seeked | — |
canplay | — |
playing | — |
canplaythrough | — |
progress | — |
ratechange | seeked |
progress | — |
progress | — |
progress | — |
durationchange | — |
progress | — |
seeking | seeking | seeking within buffer, no ratechange
waiting | — |
seeked | seeked | seek finished
canplay | — |
playing | — |
canplaythrough | — |
pause | pause |
ended | ended | end of stream reached
emptied | — |

Download it at [npm](https://www.npmjs.com/package/@eyevinn/media-event-filter).

Contribute at [github](https://github.com/Eyevinn/media-event-filter).