How LyricsX keeps track of progress of media players
Tags: javascript, swift, typescript, lyrics
Published: 2021-02-10 01:51:04

---
title: How LyricsX keeps track of progress of media players
published: true
date: 2021-01-06 05:50:57 UTC
tags: javascript,swift,typescript,Lyrics
canonical_url: https://blog.1a23.com/2021/01/06/how-lyricsx-keeps-track-of-progress-of-media-players/?utm_source=rss&utm_medium=rss&utm_campaign=how-lyricsx-keeps-track-of-progress-of-media-players
cover_image: https://blog.1a23.com/wp-content/uploads/sites/2/2021/01/LyricsX-progress-1.png
---

[LyricsX](https://github.com/ddddxxx/LyricsX) is an open source software for macOS to download and display lyrics of current playing track on Music (previously iTunes), Spotify, Audirvana, Vox, Swinsian, or the _Now Playing_ indicator in the OS. It gets time-tagged lyrics files from local storage or internet, and then display the lyrics in sync with the player.

As a crucial component of [the development of Lyricova Jukebox](https://twitter.com/i/events/1301509405490511872), I have researched multiple implementations of real time lyrics display programs, and I found the mechanism behind LyricsX particularly interesting. Here in this article, I’d share with you how LyricsX tracking the player progress in a unique and resource-saving way.

## Common practices

Before we go to LyricsX, let’s first take a look at how the majority of other open source software work on this. In short, there are 2 major ways of doing it: rely on the player to send out update events, or query the player for progress at a regular time interval. Both ways are not ideal when it comes to the extreme cases where the user adjusts the player progress a lot, and the lyrics having a lot of lines in a short period of time.

The first way, **relying on the player to send out progress update events**, is not reliable because the update interval totally depends on the platform. Some platforms may issue updates several times a second, but some may just send once a couple of seconds. In the worst case, some lines might just be skipped over due to the long update interval. Even worse some might just lack of such feature exposed to the public, rendering this way unusable in those cases.

The second way, **query the player for progress on a regular time interval**, could indeed solve the problems mentioned above, but just in an ideal system where the query itself is as simple as reading a variable. In contrary, it is almost never the case in real life. Usually to query the progress, you had to go through some kind of <abbr title="remote procedure call">RPC</abbr>, or running a tiny script. Either way it is, it already sounds a lot complicated and resource-consuming, in a worse case, some may even need to send a request through the internet for the playing progress. Depends on the precision, you might want to run the query multiple times a second. But if a single query can already take half a second, it is almost impossible to achieve a better precision this way.

## How did LyricsX do it?

The way LyricsX solved all the problems above is really cleaver. Instead of keep asking for the player for the current progress, it maintains a timer inside itself, and try to sync it with the player by using only a few basic events.

### State

The program maintains a global state of the player, with the following properties:

- **Player status** : Obviously, this can be either _playing_ or _paused_. For the _Stop_ state of some players, there is no track standby for playing, hence no timeline for lyrics, and thus out of our consideration.
- When the state is _playing_:
  - **Internal `t_0` time** : This is calculated by `current time in internal clock - player progress`, serving as a reference point of the current play session. The _internal clock_ can be anything that is fast enough to retrieve, like the system clock of the real world time, or a clock to measure code performance. We only care about the relative difference here.
  - **Playback speed** : 1.0 for normal speed, 0.5 for half speed, 2.0 for twice the speed, so on and so forth. 
- When the state is _paused_:
  - **Playback progress** : the progress of the player when it is being paused.

The global state is updated in the following player events:

- Status change (on play, on pause, on stop)
- Manual time change (on seek)
- Rate change (on rate change)

You can see the event set we have is pretty basic, and mostly essential for such a software to run.

### Sync the UI

With the player state in hand, it is much simpler to get the current player progress through just a simple system call and a calculation.

```
progress = ( t() - t_0 ) / rate
```

where `t_0` is a function to get the current time in the _internal clock_.

Apart from that, since we now maintain a copy of the player state, we can simply assume that the player will continue the playback in the indicated speed as long as the state doesn’t change. With that, we can put more optimizations in place, such as generating animation sequence timelines ahead. All those preparation work only need to be done when the player state changes, which is much less frequent the update of playback progress.

## In _Lyricova Jukebox_

In my attempt to port this logic in _Lyricova Jukebox_, I registered the `play`, `pause`, `seeked` and `ratechange` events, and for the internal clock, we used `performance.now()` for its high precision and the convenience to use in `requestAnimationFrame()`.

For the timeline, LyricsX used a seemingly reliable internal method `DispatcherQueue.schedule()` provided by Apple. However, the scheduling function in web – `window.setTimeout()` – does not have the honor to have a high enough precision for us. We then resorted to have a `requestAnimationFrame` loop running when the media is playing. Thus, to start a loop when the player state turns to _Start_, and stop the loop when it turns to _Stop_.

In the `requestAnimatioFrame` loop, we check for the current time against the end time of the current keyframe. Since the loop is run every single frame in the browser, we keep the logic in the loop simple, and only advance a keyframe when the current frame ends. The frame pointer will only be updated in the hooks mentioned above to make sure the pointer is always pointing to the right frame.

* * *

If you are interested in the implementation details, you can take a look at the source code of LyricsX and its submodule MusicPlayer which is responsible for translating events from different players.

Alternatively, if you are more comfortable with React and TypeScript, I have ported this logic to my Lyricova Jukebox, which operates on the `<audio />` tag of the browser.

- [ddddxxx/LyricsX](https://github.com/ddddxxx/LyricsX)
- [ddddxxx/MusicPlayer](https://github.com/ddddxxx/MusicPlayer)
- [blueset/project-lyricova: `usePlayerState()`](https://github.com/blueset/project-lyricova/blob/9a46256088e688a0cb9366e66c1f21220c717c43/packages/jukebox/src/frontendUtils/hooks.ts#L152)

## Acknowledgement

Thanks a lot to [ddddxxx](https://github.com/ddddxxx), the main maintainer of LyricsX, for the brilliant idea and implementation of the software.

The post [How LyricsX keeps track of progress of media players](https://blog.1a23.com/2021/01/06/how-lyricsx-keeps-track-of-progress-of-media-players/) appeared first on [1A23 Blog](https://blog.1a23.com).