Drum Patterns from Latent Space
By: Aleksey Tikhonov
Tags: Drums, Neural Networks, Visualization, Percussion, Machine Learning

Drum Patterns from Latent Space

Percussion Beats And Where To Find Them

TL;DR: I collected a large dataset of drum patterns, then used a neural network approach to map them into a latent explorable space with some recognizable genre areas. Try the interactive exploration tool or download several thousand of unique generated beats.

Context Overview

In the recent years there have been many projects dedicated to the neural network-generated music (including drum patterns). Some of such project use an explicit construction of a latent space in which each point corresponds to a melody. This space can then be used both to study and classify musical structures, as well as to generate new melodies with specified characteristics. Some others used less complex techniques, such as “language model” approaches. However, I was unable to find an overall representation of typical beat patterns mapped to 2D space, so I decided to create one myself.

Below, I listed relevant projects that I managed to found and analyze before I started working on my own:

LSTMetallica — author used a language model approach to predict the next step in a beat.

neuralbeats — another language model project, very similar to LSTMetallica.

Magenta VAE— Google’s Magenta is a great source of interesting models and projects on music generation and augmentation. Particularly, in 2016 they released the drum_rnn model, and in March 2018 they published the music_vae model. Since then a lot of projects used these models.

model, and in March 2018 they published the model. Since then a lot of projects used these models. For example, last year Tero Parviainen created a really great online drum beat generator based on drum_rnn+Magenta.js+TensorFlow.js+Tone.js.

Beat Blender was another Magenta-based project (first presented at the NIPS 2017). It’s quite similar to what I wanted to do, however, the authors hadn’t built the overview map of different genres, but only an interpolation space between pairs of patterns.

Last but not least, there was my other project, Neural Network RaspberryPi Music box, which used the VAE space to generate an endless stream of piano-like music.

Dataset Building

Most of the projects I found used small datasets of manually selected and cleaned beat patterns, e.g. GrooveMonkee free loop pack, free drum loops collection or aq-Hip-Hop-Beats-60–110-bpm.

It wasn’t enough for my goals so I decided to automatically extract beat patterns from huge MIDI collections available online. In total I’ve collected ~200K MIDI files, then kept only those with a nontrivial 9th channel (percussion channel according to the MIDI standard) so there were approximately 90K tracks left. Next, I did some additional filtering basically in the same fashion as implemeted in the neuralbeats and LSTMetallica projects (I used a 4/4 time signature, and applied quantization and simplification to the subset of instruments).

Then I split tracks into separate chunks based on long pauses, and searched for patterns of length 32 steps that were repeated at least 3 times in a row — in order to speed up the process I used hashing and a few simple greedy heuristics. Finally, I discarded trivial patterns with too low entropy and checked the uniqueness of each pattern in all possible phase shifts. Ultimately, I ended up with 33K unique patterns in my collection.

some examples of distilled patterns

I used a simple scheme to encode each pattern: a pattern has 32 time ticks and there are 14 possible percussion instruments (after simplification), so each pattern could be described by 32 integers in the range from 0 to 16383.

You can download the dataset here in TSV format:

First column holds the pattern code (32 comma-separated integers).

Second column is the point of this pattern in the latent 4D space (4 comma-separated float values), see details below.

Third column is the t-SNE mapping from the latent space into 2D projection (2 comma-separated float values), see details below.

Neural Network

I used the pytorch framework to build a network with a 3-layered FCN encoder mapping the beat matrix (32*14 bits) into 4D latent space, and a decoder with the same size as the encoder. The first hidden layer has 64 neurons, the second one has — 32. I used ReLU between the layers and a sigmoid to map the decoder output back into the bit mask.