A Data Scientist’s Approach to Visual Audio Comparison
By: Quinn Dizon
Tags: Music, Data Science, Visualization, Audio

Loading the Data

Getting audio data into Librosa is actually quite simple, but the analysis and frequency conversion can take a bit of code. For best analytical results we need to know the sample rate of each of our files, and it’s also best if those sample rates match between files. Here, they are all 44.1 kHz (a pretty standard sample rate).

My AudioAnalyzer class takes care of the heavy lifting for getting the audio into librosa and transforming it into the frequency domain. Again, if you’re interested in seeing the full code for any of this, go check out my project repository.

Initialize the AudioAnalyzer instances

input_sr stands for input sample rate, and is set to 44100 for each file. We also have fft_size set to 44100. FFT is basically a way of taking time domain data and creating a specified number of bins in which to place frequency information. In matching our FFT size to our sample rate, we are making it so that each of these bins is exactly 1 Hz. If you’re interested in reading more on FFT, check out this page.

Behind the scenes, we’re loading the audio, passing it through FFT processing, and then averaging the amplitudes of each frequency bin to get a single spectrum for each file.

The AudioAnalyzer class can already produce a simple spectrogram for us.

plotting individual spectrums

As you can see from the code, I have specified a minimum and maximum frequency. This is just so that we can zero in on a specific range where we will be able to see everything a little more clearly. Also, notice in the graphs below that the y-axis (amplitude) has been scaled for each spectrogram to be between 0 and 1. When comparing these spectrograms, it is helpful to have everything on the same scale.

From these charts, you probably think that they look rather similar. That’s because they do. Comparing the frequency domains side by side like this is pretty hard to do (and is why I am writing this article!). Luckily, I have another class that can help us to better visualize and compare the differences.

Comparing the Audio

I handle the comparison between two audio files with another class — SpectrumCompare . This class takes in two of our previously created AudioAnalyzer instances and runs some calculations in the background to help us compare them.

Let’s take a look.

Initializing SpectrumCompare class

Simple as that! We can now call a few different plotting methods on our orig_to_09 instance to compare the original 1965 version against the remastered 2009 version.

First, let’s plot the spectrograms from these two files on top of each other.

Plot basic spectrum group

With this visual, it is already much easier to compare the frequency spectrums of these two versions.

From a quick glance, it appears that the remastered version has higher amplitudes across all frequencies. But is the difference in amplitude consistent across all ranges? If not, where are the greater differences? Luckily, SpectrumCompare can help us answer these questions!

Plot full spectrum group with threshold

By passing a few other arguments in our plot_spectrum_group , we can see how the amplitude changes over the various frequency bins. The green line here shows the difference in scaled amplitudes, and the red line is the threshold. When the green line is above the red line, it means that the remastered version has a higher amplitude in that frequency bin. When it is below(which we don’t see in this particular case), it means that the original has a higher amplitude at that frequency.

Note: the order in which you add your AudioAnalyzer instances to the SpectrumCompare initialization will determine which side of the red line means more amplitude for which spectrum. The first one in gets the lower half and the second one gets the upper half. See the project repo for more information.

We can confirm our last observation that the remastered version tends to have greater amplitude across the plotted spectral range. We can even see where many peaks form and where they start to flatten out after about 700 Hz.

While this graph certainly lets us visualize more about this comparison, I think it can be simplified a bit and still give all the same general information. For this, we can use the SpectrumCompare.plot_spectrum_heatmap method.

Alternative heatmap plot

Great! Now this is much more pleasing to look at, and delivers all the same information. Our red and green lines have been replaced by the heatmap background. Through this heatmap we can see that there are a few particular hotspots. If we want to get a little more precise about our analysis, we can use the frange parameter to zoom in on a particular range. Here it looks like most of the hotspots occur between 100–500 Hz (a pretty meaty range for a lot of male vocal and instrumental music).

Heatmap from 20–500 Hz

At this closer scale, we can see that somewhere around 180 Hz there is a pretty significant hotspot indicating that the remastered version has higher amplitude at that frequency. That range corresponds to around F3, which is a comfortable male vocal range and right in the middle of the guitar range. This also happens to be the key for this song (F major). Furthermore, we can see these hotspots begin to fade as we get closer to 500 Hz.

So one conclusion we can make from this visual analysis is that the remastered version seems to place emphasis on bringing out this range around 180 Hz through 500 Hz. These hotspots are relatively contained, and we don’t see any particularly wide regions where the amplitude has been boosted significantly in the remaster. So these are some subtle changes that will likely reflect a little more emphasis on the main vocal and instrumental lines.

With this quick rundown of the process out of the way, lets look at some other comparisons between these different versions.