Package: tfr
Spectral audio feature extraction usingtime-frequency
reassignment.Besides normal spectrograms it allows to compute reassigned
spectrograms, transform them (eg. to log-frequency scale) and requantize
them (eg. to musical pitch bins). This is useful to obtain good features
for audio analysis or machine learning on audio data.A reassigned spectrogram often provides more precise localization of
energy in the time-frequency plane than a plain spectrogram. Roughly
said in the reassignment method we use the phase (which is normally
discarded) and move the samples on the time-frequency plane to a more
suitable place computed from derivatives of the phase.This library supports reassignment in both frequency and time (both are
optional). As well it does requantization from the input overlapping
grid to an non-overlapping output grid.It is a good building block to computechromagram
features(aka pitch
class profiles) where pitch is transformed into pitch class by ignoring
the octave. See alsoharmonic pitch class
profiles.Installationpip install tfrOr for development (all code changes will be available):git clone https://github.com/bzamecnik/tfr.git
pip install -e tfrUsageSplit audio signal to framesYou can read time-domain signal from an audio file (using thesoundfilelibrary) and split it into frames for spectral processing.import tfr
signal_frames = tfr.SignalFrames('audio.flac')SignalFramesinstance contains the signal split into frames and some
metadata useful for further processing.The signal values are normalized to [0.0, 1.0] and the channels are
converted to mono.It is possible to provide the signal a numpy array as well.import tfr
x = np.sin(2 * np.pi * 10 * np.linspace(0, 1, 1000))
signal_frames = tfr.SignalFrames(x)Minimal example - pitchgram from audio fileimport tfr
x_pitchgram = tfr.pitchgram(tfr.SignalFrames('audio.flac'))From audio frames it computes a reassigned pitchgram of shape(frame_count, bin_count)with values being log-magnitudes in dBFS[-120.0,0.0]. Sensible parameters are used by default, but you can
change them if you wish.Reassigned spectrogramLike normal one but sharper and requantized.import tfr
x_spectrogram = tfr.reassigned_spectrogram(tfr.SignalFrames('audio.flac'))Signal frames with specific parametersframe_size- affects the FFT size - trade-off between frequency
and time resolution, good to use powers of two, eg. 4096hop_size- affects the overlap between frames since a window
edges fall to zero, eg. half of frame_size (2048)import tfr
signal_frames = tfr.SignalFrames('audio.flac', frame_size=1024, hop_size=256)General spectrogram APIThepitchgramandreassigned_spectrogramfunctions are just
syntax sugar for theSpectrogramclass. You can use it directly to
gain more control.General usage:x_spectrogram = tfr.Spectrogram(signal_frames).reassigned()From one Spectrogram instance you can efficiently compute reassigned
spectrograms with various parameters.s = tfr.Spectrogram(signal_frames)
x_spectrogram_tf = s.reassigned(output_frame_size=4096)
x_spectrogram_f = s.reassigned(output_frame_size=512)Different window function (by default we use Hann window):import scipy
x_spectrogram = tfr.Spectrogram(signal_frames, window=scipy.blackman).reassigned()Different output frame size (by default we make it the same as input hop
size):x_spectrogram = tfr.Spectrogram(signal_frames).reassigned(output_frame_size=512)Disable reassignment of time and frequency separately:s = tfr.Spectrogram(signal_frames)
x_spectrogram = s.reassigned(reassign_time=False, reassign_frequency=False)
x_spectrogram_t = s.reassigned(reassign_frequency=False)
x_spectrogram_f = s.reassigned(reassign_time=False)
x_spectrogram_tf = s.reassigned()Disable decibel transform of output values:x_spectrogram = tfr.Spectrogram(signal_frames).reassigned(magnitudes='power')Magnitudes in the spectrogram can be transformed at the end in multiple
ways given by themagnitudesparameter:linear- energy spectrumpower- power spectrumpower_db- power spectrum in decibels, range: [-120, 0]power_db_normalized- power spectrum in decibels normalized to
range: [0, 1]this is useful as a featureUse some specific transformation of the output values.LinearTransform(default) is just for normal spectrogram,PitchTransformis for pitchgram. Or you can write your own.x_spectrogram = tfr.Spectrogram(signal_frames).reassigned(transform=LinearTransform())x_pitchgram = tfr.Spectrogram(signal_frames).reassigned(transform=PitchTransform())class LogTransform():
  def __init__(self, bin_count=100)
    self.bin_count = bin_count

  def transform_freqs(self, X_inst_freqs, sample_rate):
      X_y = np.log10(np.maximum(sample_rate * X_inst_freqs, eps))
      bin_range = (0, np.log10(sample_rate))
      return X_y, self.bin_count, bin_range

x_log_spectrogram = tfr.Spectrogram(signal_frames).reassigned(transform=LogTransform())Pitchgram parametersIn pitchgram the frequencies are transformed into pitches in some tuning
and then quantized to bins. You can specify the tuning range of pitch
bins and their subdivision.tuning- instance ofTuningclass, transforms between pitch
and frequencybin_rangeis in pitches where 0 = 440 Hz (A4), 12 is A5, -12 is
A3, etc.bin_division- bins per each pitchExtract features via CLI# basic STFT spectrogram
python -m tfr.spectrogram_features audio.flac spectrogram.npz
# reassigned STFT spectrogram
python -m tfr.spectrogram_features audio.flac -t reassigned reassigned_spectrogram.npz
# reassigned pitchgram
python -m tfr.spectrogram_features audio.flac -t pitchgram pitchgram.npzLook for other options:python -m tfr.spectrogram_features --helpscikit-learn transformerIn order to extract pitchgram features within a sklearn pipeline, we can
usePitchgramTransformer:import soundfile as sf
x, fs = sf.read('audio.flac')

from tfr.signal import to_mono
from tfr.sklearn import PitchgramTransformer
ct = PitchgramTransformer(sample_rate=fs)
x_pitchgram = ct.transform(x)

# output:
#  - shape: (frame_count, bin_count)
#   - values in dBFB normalized to [0.0, 1.0]StatusCurrently it’s alpha. I’m happy to extract it from some other project
into a separate repo and package it. However, the API must be completely
redone to be more practical and obvious.AboutAuthor: Bohumír Zámečník ([@bzamecnik](http://twitter.com/bzamecnik))License: MITSupport the projectNeed some consulting or coding work regarding audio processing, machine
learning or big data? Drop me a message viaemailorLinkedIn. Or just
say hello :).LiteratureA Unified Theory of Time-Frequency
Reassignment- Kelly R. Fitz,
Sean A. Fulop, Digital Signal Processing 30 September 2005Algorithms for computing the time-corrected instantaneous frequency
(reassigned) spectrogram, with
applications- Sean A. Fulop, Kelly Fitz, Journal of Acoustical Society of
America, Jan 2006Time Frequency Reassignment: A Review and
Analysis- Stephen W. Hainsworth, Malcolm D. Macleod, Technical Report,
Cambridge University Engineering Dept.Improving the Readability of Time-Frequency and Time-Scale
Representations by the Reassignment
Method- Francois Auger, Patrick Flandrin, IEEE Transactions on Signal
Processing, vol. 43, no. 5, May 1995Time–frequency reassignment: from principles to
algorithms- P. Flandrin, F. Auger, E. Chassande-Mottin, CRC Press 2003Time-frequency toolbox for Matlab, user’s guide and reference
guide-
F.Auger, P.Flandrin, P.Goncalves, O.Lemoine