I’m Sorry for Your Loss: Spectrally-Based Audio Distances Are Bad at Pitch
Abstract
Growing research demonstrates that synthetic failure modes imply poor generalization.
We compare commonly used audio-to-audio losses on a synthetic benchmark, measuring the pitch distance between two stationary sinusoids.
The results are surprising: many have poor sense of pitch direction.
These shortcomings are exposed using simple rank assumptions.
Our task is trivial for humans but difﬁcult for these audio distances, suggesting signiﬁcant progress can be made in self-supervised audio learning by improving current losses.
Introduction
Rather than a physical quantity contained in the signal, pitch is a percept that is strongly correlated with the fundamental frequency of a sound.
While physically elusive, pitch is a natural dimension to consider when comparing sounds.
Children as young as 8 months old are sensitive to melodic contour [65], implying that an early and innate sense of relative pitch is important to parsing our auditory experience.
This basic ability is distinct from the acute pitch sensitivity of professional musicians, which improves as a function of musical training [36].
Pitch is important to both music and speech signals.
Typically, speech pitch is more narrowly circumscribed than music, having a range of 70–1000Hz, compared to music which spans roughly the range of a grand piano, 30–4000Hz [31].
Pitch in speech signals also ﬂuctuates more rapidly [6].
In tonal languages, pitch has a direct inﬂuence on the meaning of words [8, 73, 76].
For non-tone languages such as English, pitch is used to infer (supra-segmental) prosodic and speaker-dependent features [19, 21, 76].
In this work, we demonstrate pitch-based failure modes for audio-to-audio distances commonly used as loss functions.
Negative results have potentially deleterious consequences on this benchmark: gradient-descent takes a circuitous route for some losses, implying that generalization speed and ﬁnal accuracy are impaired.
Worse yet, other loss functions get stuck in varying scopes of local minima.
Their search spaces contain both ﬁne
-grained oscillations and broader plateaus
1.1 Perceptual Audio-to-Audio Distances
Audio-to-audio distances attempt to quantify what a human judge would consider the similarity between sounds.
Therefore, we argue that they are fundamentally perceptual in nature.
The notion of a distance between two sounds has a long history of investigation in perceptual research [59, 60].
While early work focused on loudness and pitch distances in simple tones, a more recent thread in experimental psychology explores the auditory distances between instrument timbres, sounds that are more abstractly connected [26, 45, 47].
In this line of research, subjects are asked to rate the distance between multiple pairs of recorded instrument notes.
The differences are used to ﬁt to a timbre space in which all instruments are situated.
The dimensions accounting for the greatest variance in this space are then correlated to acoustic descriptors of the stimuli.
One particularly strong acoustical correlate is the spectral centroid, a measure that appears independently in many timbre studies [44].
Spectral centroid expresses the weighted mean of the spectral frame normalized by frame energy—in a frame containing a single sinusoid, the spectral centroid will be its frequency.
It is a strong acoustical correlate of perceived “brightness” [53].
1.2 Automatic Audio-to-Audio Distances
Many important audio modeling settings rely upon a perceptually-coherent distance between two audio representations, both in generative and constrastive setttings.
Generative audio losses can be viewed as divergences between audio representations.
Thus, audio generation tasks use audio-toaudio distances: a) neural synthesis, in particular conditional spectrogram inversion [17, 34, 46, 66]; b) audio source separation [29, 33, 52, 58]; c) denoising [23, 43]; and d) upsampling [75], not to mention audio self-supervised learning in general.
LeCun [40] argues that most human and animal learning is self-supervised, whether generative or contrastive.
Contrastive losses are based upon knowing that the representation distance between the target and a similar prediction is lower than the representation distance between the target and a dissimilar prediction.
The contrastive approach involves fewer parametric assumptions than the generative one, and so we use it in our dissimilarity-ranking experiments (see Section 3.3).
1.3 Learning Representations of Pitch
Knowing that pitch is an important dimension for representation learning, there are several options for inducing pitch in learned representations: data, features, and loss functions.
Engel et al.
[15] supplement their data with synthesized audio of a known pitch to aid in selfsupervised pitch learning.
One may use explicit pitch- features from a pitch extractor, such as CREPE [35] or SPICE [24] as input [27, 42] or conditioning information [34, 66, 73].
Nonetheless, learning new data and incorporating better features is gated by the underlying loss function.
Audio loss construction typically involves some prior knowledge from digital signal processing, for example in their choice of time-frequency representation.
What is surprising is that these wellmotivated distances perform poorly in our simple pitch experiments.
This negative result explains why methods using spectral losses nonetheless end up using pitch-tracker models for conditioning [16] or reconstruction [15].
Alternately or in addition to DSP-motivated losses, knowledge from auxiliary models can be distilled using feature-matching losses (Section 2.1).
Distances based on Spectral Representations
Auditory neural networks commonly use spectral representations of sound.
One can generate a (static) spectrum from a time-domain signal with the Fourier transform in Equation 1:
[Equation]
where x ( t ) is the signal, t is time, ω is frequency and j = √− 1 is the imaginary unit.
The Fourier transform exchanges all time information for frequency information, which is not always desirable for analysis purposes.
In order to keep both, the signal can be windowed into small segments or “frames.”
Taking a Fourier transform of each frame results in a two dimensional function called the short-time Fourier transform (STFT), shown in Equation 2, where τ is the time index and w ( t ) is the windowing function.
As with the Fourier transform, the STFT produces a complex-valued function.
Its real-valued modulus is sufﬁcient for many applications; this magnitude-only version is commonly called the “spectrogram,” and indicates the amplitude of frequency components over time.
The spectrogram and its variants (e.g., the Mel spectrogram) are among the most popular audio representations for signal analysis.
All time-frequency representations suffer from the same fundamental trade-off, which is a direct consequence of the uncertainty principle [20]: the more that is known about a signal’s location in time, the less is known about its frequency, and vice versa.
Shorter STFT frames gain temporal accuracy, but cut off frequencies that require a longer time extent to fully oscillate (Figure 1).
Window functions w ( t ) are designed to fade frames in and out, and introduce their own spectral proﬁle into the signal.
The choice of window is a trade-off between reducing the spectral noise ﬂoor (lower “side-lobes”) and gaining spectral precision (narrower “main lobes”) [28].
We use a Hann window throughout.
2.1 Differentiable Audio Distances
We are interested in differentiable audio distances that are well-behaved and easy to optimize.
Audioto-audio distances are typically the (cid:96) 1 or (cid:96) 2 between representations of the target audio and the predicted audio.
While the raw waveform can be used to this end [34, 46, 66], a more common approach is to use the (cid:96) 1 or (cid:96) 2 distance between the spectrogram or log-magnitude spectrogram [12, 55, 69, 71].
A particular loss might have certain weaknesses in certain situations, such as gradient instability.
Multiple losses may be mixed to work in tandem, and hopefully add up to more than the sum of their parts.
The multi-scale spectrogram (MSS) captures difference in audio phenomena at multiple scales of time and frequency in an attempt to side-step the time-frequency trade-off discussed in the previous section.
Interestingly, this roughly echoes a strategy used in the primary auditory cortex [7].
As before, multi-scale losses are typically computed as (cid:96) 1 or (cid:96) 2 over linear- and log-spectrograms [5, 13, 15, 16, 68, 70, 72].
Alternately, one may use neural auditory distances.
In the “knowledge-distillation” or “teacherstudent” framework, one net trains the other [32].
A speciﬁc formulation of this approach, sometimes called “perceptual loss,” or “feature-matching,” considers the internal activations of parts or the whole of a neural net [15, 37, 55, 74].
Even speciﬁcally for pitch-learning, there are numerous examples of a feature-matching losses: a typical speech VQ-VAE is pitch-insensitive [67], but one can add a pitch reconstruction loss [21] or learn an additional pitch-only VQ-VAE [76].
Experimental Design
Our synthetic benchmark suggest that losses based upon common spectral representations, as well as many off-the-shelf neural representations, have difﬁculties tracking pitch orientation.
Our tool for this inquiry is a plain model of pure stationary sine waves: a well-behaved audio-to-audio loss should have high pitch-orientation accuracy, both on a ﬁne scale as well as a coarse one.
This approach is similar in spirit to [39], which uses gradient-orientation as an evaluation criterion on a toy spheres task (albeit in the setting of adversarial learning).
3.1 Pure Sinuisoids
Pure sine waves have three underlying factors of variation: amplitude ( A ), frequency ( ω ), and initial phase ( φ ):
[Equation]
Given a target sine wave, we seek an audio distance that permits a sine-wave model to generate a predicted sine wave with the same frequency and level as the target.
Note that while linear amplitude and frequency are used to generate the signal, these quantities are more meaningfully described by their logarithmic equivalents [60].
Thus, in our benchmark, sinusoids are generated with a uniform distribution in the pitch space (or logarithmic distribution in frequency) between 30–4000Hz.
The signal level (hereafter, “level”) is sampled from a normalized uniform decibel scale 0db through -25dB.
We calculated normalized level of a sinusoid as
[Equation]
such that a maximum amplitude A of 1 corresponds to a signal level L A of 0dB. The initial phase φ of isolated sinusoids is not perceptible.
We randomize it to remove a confounding variable.
[Figure]
2b shows the MSS
) distance.
3.2 Tuning A Sine: Thought Experiment
Imagine that an experimenter asks you to adjust the pitch of one sine tone oscillator to match another.
If the pitches start far apart, you would have a conﬁdent sense of the direction to tune in, up or down, and have gradually less conﬁdence as you approach the target pitch.
Figure 2a shows the spectrogram-based distance, as predicted pitch sweeps across the spectrum.
The observed pitch sensitivity is in many ways the opposite of a natural pitch sense: at great distances it has no sense of pitch orientation whatsoever (a vanishing gradient), and it locks-in with extreme conﬁdence at ﬁne pitch distances.
This result stands to reason, given the coordinate-for-coordinate comparison implicit in norm-based distance calculations—an effect that is only exaggerated in spectrograms with large frame size, and therefore ﬁne spectral resolution (Figure 1, left).
The MSSbased distance (Figure 2b) somewhat alleviates this problem, but still suffers from a sharp gradient near the target and a vanishing gradient in high predicted pitch extrema.
Finally, our distances are measured on a log-linear frequency scale.
However, the spectrogram operates on a linear scale, so lower target pitches have slightly wider regions with informative gradients.
3.3 Gradient-Sign (Dissimilarity) Ranking Accuracy
In a well-behaved distance, a target sinusoid should be closer to a predicted candidate sinusoid than to a perturbed candidate sinuisoid that is further away by construction.
This argument motivates an evaluation using a dissimilarity-ranking accuracy [1, 50], i.e. whether the gradient is pointed in the correct direction.
We randomly sample a target sinusoid with level A and pitch ω , and a prediction sinusoid with level A (cid:48) and pitch ω (cid:48) (where A (cid:54) =
A (cid:48) and ω (cid:54) = ω (cid:48) ).
We also construct a third perturbed sinuisoid.
The perturbed signal level is the same as the prediction signal level.
The perturbed signal pitch ω (cid:48)(cid:48) := ω (cid:48)
+ ε · sgn( ω (cid:48) − ω ) is, by construction, ε further away from the target ω than the prediction signal pitch ω (cid:48) .
From these signals we compute two distances: the distance from the target to the prediction, and the distance from the target to perturbation.
In a well behaved distance, the prediction should be closer to the target than the perturbation is.
This condition gives us a 0/1 error.
Repeated and averaged over 1000 trials, high accuracies indicate that gradients usually point in the right direction.
We reproduce the same process using perturbations in level with a ﬁxed pitch, using
A (cid:48)(cid:48) :
= A (cid:48) + ε · sgn( A (cid:48) − A ) .
Formally:
[Equation]
In addition to numeric computations using explicit values of ε , a gradient may be solved and computed analytically using automatic differentiation.
This is the equivalent of letting this value go to zero in
Equations 5 and 6.
We deﬁne (cid:15) :
= ε → 0 .
We make no assumptions about pitch- and level- distance, besides our choice of range (30Hz-4KHz, -25dB to 0dB) and that sampling occurs logarithmically with respect to A and ω (Equation 3).
Results
We present quantitative results in Table 1.
Each column indicates the average accuracy for the analytic, ﬁne- and coarse-numeric conditions, with respect to changing pitch and changing level.
The sign-accuracy of analytic gradients ( (cid:15) ≈ 0 ) demonstrates whether the distance is capable of breaking out of very ﬁne-grained local minima.
While in practice gradient descent uses the analytic gradient, we also present numerically-computed gradients to characterize local minima at broader resolutions.
The ﬁne perturbation values (30 cents or 2dB) are meant to be plausibly detectable by a discerning human listener.
The coarse perturbation values (600 cents or 10dB) are meant to be obvious to a human listener with normal hearing, and to give an indication of broad-scale local minima in the search space.
For qualitative elucidation, we provide coarse heatmaps based on a ﬁxed target pitch and level in
Figure 3.
Many distances are sensitive to the choice of target signal (Figures 2a and 2b), so heatmaps based on a ﬁxed target do not necessarily illustrate the entire search space.
Supplementary Figuresand 6 depict coarse heatmaps for other target pitches and levels.
4.1 Spectral Audio Distances
The spectrogram (the modulus of Eq. 2) is a starting point for many kinds of spectral analysis.
A spectrogram-based distance can roughly model pitch in the coarse condition (69.5%), but otherwise the gradients score near chance (i.e., they are not pointed in any meaningful direction).
Levelperturbed gradients score near random in both coarse and ﬁne conditions.
In the case of both pitch and level, the analytical gradient is in agreement with the ﬁne perturbation conditions.
Interestingly, the log-magnitude spectrogram has some level sensitivity, but is worse at pitch tracking than its linear equivalent.
Mel spectrograms are a popular variant of the spectrogram based on the Mel scale [60].
By pooling frequency bins into progressively wider frequency spans, this representation roughly emulates the logscaled nature of human frequency perception.
As with the spectrogram, the Mel spectrogram-based distance performs near chance (if slightly above) in all conditions.
Mel-frequency cepstral coefﬁcients (MFCCs) are a popular feature for speech and music applications based on cepstral analysis [49].
In [69], the authors argue that MFCCs are somewhat pitch invariant.
Fittingly, the MFCC representation struggles with pitch, having an accuracy slightly worse than the spectrogram, though it has the best level score among the spectral representations we tested.
The MFCC analytic gradient computed on level is entirely at odds with the ﬁne perturbation results, scoring nearly at chance.
This phenomenon is reﬂected in the extra ﬁne-grained vertical streaks in the
MFCC heatmap (Figure 3e).
At a broader resolution the curves look smooth; at super-ﬁne resolutions they are rocky.
Multi-scale spectral loss (MSS) compares spectrograms at multiple scales—in our work, we use the six window sizes shown in Figure 1 for MSS.
In effect, this mitigates the poor behavior of the spectrogram-based pitch distance (Figure 2).
Indeed, the pitch-perturbed gradient is correctly oriented 77% of the time.
Still the gradient is not entirely smooth (Figure 4a).
MSS has poor analytic level gradients (55%).
The heatmap in Figure 4b shows the level-gradient always drives the level down and away from the target signal, even when the prediction level is lower than than the target.
Only
[Table]
using stochasticity
[Figure] [Figure]
when pitch is locked-in are the level gradients correctly oriented.
Hence, MSS level accuracies are 55%.
The log MSS outperforms the log-spectrogram, improving mainly on level but not pitch.
The log-scaled spectral centroid recovers the pitch precisely on pure sinusoids.
As per its normalization factor, it is insensitive to level.
However, its analytic gradients are highly chaotic for both level and pitch, scoring at chance.
4.2 Neural Audio Distances
Large-scale neural audio models are far more expressive than traditional spectral representations.
Neural models have evinced uncanny ability on other zero-shot problem domains, such as uncovering analogical reasoning when trained to predict a missing word [54].
We test a variety of models under our evaluation methodology, using their ﬁnal layer as an audio representation.
NSynth Wavenet is trained on musical notes from the NSynth dataset [17].
It has a strong coarse level accuracy (93%) and coarse pitch accuracy (86%).
At ﬁner pitch distances the pitch accuracy suffers (59%).
This is perhaps because the dataset comprises single notes, each spaced one semitone (100 cents) apart.
VGGish [30] and OpenL3 [10] are large-scale nets trained on a variety of audio samples taken from internet video, the latter incorporating multi-modal information from the video feed.
Interestingly, both are insensitive to level, scoring just slightly above chance.
OpenL3 is highly pitch accurate at a broad scale (99%), but scores near random (59%) at a ﬁne scale.
We auditioned many pre-trained models of wav2vec 2.0 [2], a large-scale neural net trained completely on speech; we achieved the best results using their large (LV-60) no ﬁne-tuning model.
This model has good coarse-pitch accuracy (83%), despite having never listened to music.
Overall, we were surprised to ﬁnd some deﬁciencies of these models on our benchmark, which speaks to the difﬁculty of extracting a general-purpose low-dimensional representation of audio.
Poor results of these representations is not meant to impugn whether pitch can be extracted from these representations using one or more layers.
Fine-tuning a large-scale neural model on a simple task will likely succeed, even in a one-shot learning scenario [74].
Discussion
Pitch and level are both natural dimensions to consider when comparing sounds.
To a listener with typical hearing, making small adjustments based on pitch or level is as simple as tuning a guitar, or turning down the radio.
So why is it that common spectral-based loss functions have difﬁculty with our benchmark?
While some distances fare relatively well on a strictly pitch-based perturbation, almost all do poorly when introducing any perturbations of level.
A further mystery still, why is the search space apparently bumpy at very ﬁne resolutions?
In any norm-based distance, one makes the implicit assumption of orthogonality; i.e., that the only important difference between two vectors lies in the difference of corresponding values.
In the case of the spectrogram, the distance asks the question “do I need more of frequency x , or less?”.
But these differences contain no explicit spectral orientation—in other words, they have no sense of their spectral neighbors, and by consequence can’t answer a question that is simple for humans: “is pitch x higher or lower than pitch y ?”.
MSS appears to mitigate this problem by taking advantage of the spectral smearing of smaller window sizes (see Figure 1).
However, its analytic gradient remains unimpressive in the pitch condition, and nearly random in the level condition.
Many of the analytic gradients perform poorly.
Even the 77% accurate MSS gradient is not smooth (Figure 4b).
We do not completely understand the nature of this phenomenon, however in our investigations we have found a few patterns: unwindowed STFT distances result in a much bumpier gradient, and Hann windowing smooths the search space.
The use of multiple analysis scales further smooths the search space, however these distances still contain many very ﬁne-grained local minima.
Finally, pitch-recovery is an intrinsically difﬁcult regression problem.
Sinuisoidal activations can compactly construct functions with inﬁnite VC-dimension [22], and backprop through a periodic function is difﬁcult to train.
Does my non-music model really need robust pitch sensitivity?
A pessimistic perspective might be that if a particular task is almost solved, it is not useful to pursue broader machine understanding.
If so, what could motivate the speech, NLP, and other machine listening communities to include broad-scale pitch perception in their training regimes?
Task-speciﬁc approaches can produce one-trick ponies with unexpected failure modes.
Sturm [61] describes Clever Hans, a horse that could ostensibly do basic arithmetic, fooling even a zoologist.
In fact, Clever Hans had no mathematical understanding at all, and instead relied upon subconscious visual cues from his interrogator.
Bello [3] describes an analogous failure-mode: a model that predicted the tango genre exceptionally well, but only because all the tango recordings in its corpus were old and characteristically band-limited.
This phenomenon is similar to certain patch-based face recognition approaches, where a face is recognized when presented with an adversarial example where the eyes and mouth are switched.
Even a few pixel change can trick a neural network [62, 63].
There is a growing understanding that uncovering unusual failure modes is not butterﬂy hunting, but indeed is deeply connected with generalization.
Adversarial examples occur in nature [38].
Gilmer et al.
[25] demonstrate a tight correlation between vision performance on a toy adversarial sphere dataset with generalization on natural tasks.
Spurious correlation [48] and under-speciﬁcation [11] also lead to poor generalization.
Evaluation should not only quantify performance on a speciﬁc task, but aid the improvement of systems in general [61].
Training on broad open-domain data, like the recently released FSD50K corpus [18], exposes the model to the full gamut of natural phenomena.
Sufﬁciently diverse synthesized data might encompass real-world data as a special case [64].
Multi-task learning demonstrates that sample complexity reduces, training accelerates, and generalization improves by including data that surfaces more information than is present in the target task’s data [9, 56].
Incorporating more training data may lead to a better result, but there might be a ceiling to what data alone can bring to the table [4].
In NLP, the nuances of a question like “is an orange more like a baseball or more like a banana?”
cannot be fully comprehended by a model trained purely on text, even the entire web.
Learning the size of objects using pure text analysis requires signiﬁcant gymnastics [14]; vision demonstrates physical size more easily.
This school of thought suggests that listening to all the audio on the web can only learn so much; true general purpose learning can only happen through grounding in an experiential notion of sound, i.e., one that is perceptually, and perhaps even cognitively, based.
By extension, this implies incorporating more modalities than just hearing [10].
More controversial still is the argument that passively consuming the internet’s entirety of media data in all modalities (text, vision, and audio) provides insufﬁcient context, and the broader contextual scopes of embodiment and social interaction are needed for deeper understanding [4].
Conclusion
Learning is only as good as the underlying optimization criterion.
Our synthetic benchmark exposes a surprising gap between automatic audio-to-audio distances and human perception.
In contrast to the human experience of sound, spectrally-based audio representations—as well as several off-the-shelf neural representations—have difﬁculties with basic pitch orientation.
When signal level is included as a dimension of variability, gradient orientation accuracy decreases.
The confounding effects of level are perplexing, and further work is necessary to identify the cause.
Our negative results demonstrate that even for a low-dimensional problem (i.e., pitch and level), distances based upon these audio representations are perceptually unintuitive and bad for learning.
Their search spaces have local minima at ﬁne and coarse resolutions.
These issues are surely compounded in higher-dimensional problems, and raise the spectre that they cannot model asyet-unknown underlying factors of variation.
The question remains: what is necessary to induce general-purpose audio representations similar to human auditory perception?
Broader Impact
We present this work as an invitation to researchers to scrutinize auditory representations at large, and to hold loss functions to a higher standard, one closer to what is perceptually salient.
We do not expect any negative outcomes from this work.
Models of auditory perception are implicitly based on prototypical notion of “healthy” hearing.
We acknowledge that hearing ability varies widely from listener to listener, and that this line of research may be considered exclusionary from the perspective of the hearing impaired.
To this extent, research in this direction may leverage data that are not wholly representative of the population.
We should be cognizant of this bias and—indeed—it should be possible for machines to model different people’s pitch perception.
Acknowledgments and Disclosure of Funding
The authors would like to thank the following readers for their feedback: Alex Nisnevich, Alexandre
Passos, Edouard Oyallon, Felipe Tobar, Harri Taylor, Jesse Engel, Lamtharn (Hanoi) Hantrakul,
Nicolas Pinto, Oleg Polosin, Pranay Manocha, Richard Socher, Richard Zhang, as well as the workshop reviewers and chairs.
A Supplementary Material A.1 Ideal Models of Pitch Distance
There are at least two orthogonal factors when in comes to a representation of pure pitch distance:
Distance determined by the ratio of the fundamental frequencies of the notes in question.
For example, the ratio of 660:440Hz is equivalent to the ratio 750:500Hz, and therefore their pitch distances are the same.
However, this rule is complicated by the fact that human hearing is not equally sensitive to pitch throughout the spectrum.
We typically hear melodic information in the range from 30Hz—4KHz [51, 60]
Notes separated by one or more octaves (frequency ratio of powers of two, e.g. A440Hz and A880Hz) share a “pitch class” and are equivalent in this pitch dimension. This
phenomenon leads to helical representations of pitch, which curls up and around once an octave
To the knowledge of the authors, the relative magnitude of these two dimensions in the perception of pitch distance is an open question.
For simplicity, we model only the ﬁrst dimension.
Under this assumption, the ideal perceived pitch distance is strictly monotonic for ﬁxed level [60]:
[Equation]
if ω < ω
[Equation]
<
ω
[Equation]
or ω > ω [Equation] > [Equation]
7) d
[Equation]
A
[Equation]
ω )
[Equation]
x
[Equation]
A
[Equation]
ω
[Equation]
)) <
d ( x ( A
[Equation]
ω ) , x ( A (cid:48)(cid:48) , ω
[Equation]
if A < A (cid:48) <
A (cid:48)(cid:48) or A > A (cid:48) >
A (cid:48)(cid:48)
(8) which lead to Equations 5 and 6.
A.2 Hyperparameters
All spectral distances are implemented in PyTorch using a sampling rate of 44.1KHz and Hann windowing.
Other windowing methods available in scipy performed equivalently or worse.
Although hyperparameter selection on evaluation data is against the spirit of zero-shot learning, we tuned hyperparameters to concisely demonstrate upper-bounds on performance.
These are the hyperparameters used in our results section:
Spectrogram (cid:96) 1 distance, | Spectrogram ( x ) | , nfft=2048, frame_overlap=0.
log(Spectrogram) (cid:96) 2 distance, log( | Spectrogram ( x ) | + 1e-4 ) , nfft=2048, frame_overlap=0.
Mel (cid:96) 1 distance, nfft=1024, frame_overlap=0.5, nmels=1024, fmin=30, fmax=
MFCC (cid:96) 1 distance, nfft=1024, frame_overlap=0.5, nmels=128, nmfcc=128, norm=
None, fmin=30, fmax=
MSSTFT and log MSSTFT
Computed as Spectrogram and log(Spectrogram) respectively, with nffts=[2048, 1024, 512, 256, 128, 64]
Spectral Centroid (cid:96) 1 distance, log 2 spectral centroid, with nfft=2048, frame_overlap=0.75, power=1.
[Figure] [Figure]