Utilities for `FeatureExtractors`

This page lists all the utility functions that can be used by the audio `FeatureExtractor` in order to compute special features from a raw audio using common algorithms such as _Short Time Fourier Transform_ or _log mel spectrogram_.

Most of those are only useful if you are studying the code of the audio processors in the library.

## [](#transformers.audio_utils.hertz_to_mel)Audio Transformations

#### transformers.audio\_utils.hertz\_to\_mel

[](#transformers.audio_utils.hertz_to_mel)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/audio_utils.py#L25)

( freq: typing.Union\[float, numpy.ndarray\]mel\_scale: str = 'htk' ) → `float` or `np.ndarray`

Parameters

-   [](#transformers.audio_utils.hertz_to_mel.freq)**freq** (`float` or `np.ndarray`) — The frequency, or multiple frequencies, in hertz (Hz).
-   [](#transformers.audio_utils.hertz_to_mel.mel_scale)**mel\_scale** (`str`, _optional_, defaults to `"htk"`) — The mel frequency scale to use, `"htk"` or `"slaney"`.

Returns

`float` or `np.ndarray`

The frequencies on the mel scale.

Convert frequency from hertz to mels.

#### transformers.audio\_utils.mel\_to\_hertz

[](#transformers.audio_utils.mel_to_hertz)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/audio_utils.py#L59)

( mels: typing.Union\[float, numpy.ndarray\]mel\_scale: str = 'htk' ) → `float` or `np.ndarray`

Parameters

-   [](#transformers.audio_utils.mel_to_hertz.mels)**mels** (`float` or `np.ndarray`) — The frequency, or multiple frequencies, in mels.
-   [](#transformers.audio_utils.mel_to_hertz.mel_scale)**mel\_scale** (`str`, _optional_, `"htk"`) — The mel frequency scale to use, `"htk"` or `"slaney"`.

Returns

`float` or `np.ndarray`

The frequencies in hertz.

Convert frequency from mels to hertz.

#### transformers.audio\_utils.mel\_filter\_bank

[](#transformers.audio_utils.mel_filter_bank)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/audio_utils.py#L115)

( num\_frequency\_bins: intnum\_mel\_filters: intmin\_frequency: floatmax\_frequency: floatsampling\_rate: intnorm: typing.Optional\[str\] = Nonemel\_scale: str = 'htk' ) → `np.ndarray` of shape (`num_frequency_bins`, `num_mel_filters`)

Creates a frequency bin conversion matrix used to obtain a mel spectrogram. This is called a _mel filter bank_, and various implementation exist, which differ in the number of filters, the shape of the filters, the way the filters are spaced, the bandwidth of the filters, and the manner in which the spectrum is warped. The goal of these features is to approximate the non-linear human perception of the variation in pitch with respect to the frequency.

Different banks of mel filters were introduced in the literature. The following variations are supported:

-   MFCC FB-20: introduced in 1980 by Davis and Mermelstein, it assumes a sampling frequency of 10 kHz and a speech bandwidth of `[0, 4600]` Hz.
-   MFCC FB-24 HTK: from the Cambridge HMM Toolkit (HTK) (1995) uses a filter bank of 24 filters for a speech bandwidth of `[0, 8000]` Hz. This assumes sampling rate ≥ 16 kHz.
-   MFCC FB-40: from the Auditory Toolbox for MATLAB written by Slaney in 1998, assumes a sampling rate of 16 kHz and speech bandwidth of `[133, 6854]` Hz. This version also includes area normalization.
-   HFCC-E FB-29 (Human Factor Cepstral Coefficients) of Skowronski and Harris (2004), assumes a sampling rate of 12.5 kHz and speech bandwidth of `[0, 6250]` Hz.

This code is adapted from _torchaudio_ and _librosa_. Note that the default parameters of torchaudio’s `melscale_fbanks` implement the `"htk"` filters while librosa uses the `"slaney"` implementation.

#### transformers.audio\_utils.optimal\_fft\_length

[](#transformers.audio_utils.optimal_fft_length)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/audio_utils.py#L193)

( window\_length: int )

Finds the best FFT input size for a given `window_length`. This function takes a given window length and, if not already a power of two, rounds it up to the next power or two.

The FFT algorithm works fastest when the length of the input is a power of two, which may be larger than the size of the window or analysis frame. For example, if the window is 400 samples, using an FFT input size of 512 samples is more optimal than an FFT size of 400 samples. Using a larger FFT size does not affect the detected frequencies, it simply gives a higher frequency resolution (i.e. the frequency bins are smaller).

#### transformers.audio\_utils.window\_function

[](#transformers.audio_utils.window_function)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/audio_utils.py#L206)

( window\_length: intname: str = 'hann'periodic: bool = Trueframe\_length: typing.Optional\[int\] = Nonecenter: bool = True )

Parameters

-   [](#transformers.audio_utils.window_function.window_length)**window\_length** (`int`) — The length of the window in samples.
-   [](#transformers.audio_utils.window_function.name)**name** (`str`, _optional_, defaults to `"hann"`) — The name of the window function.
-   [](#transformers.audio_utils.window_function.periodic)**periodic** (`bool`, _optional_, defaults to `True`) — Whether the window is periodic or symmetric.
-   [](#transformers.audio_utils.window_function.frame_length)**frame\_length** (`int`, _optional_) — The length of the analysis frames in samples. Provide a value for `frame_length` if the window is smaller than the frame length, so that it will be zero-padded.
-   [](#transformers.audio_utils.window_function.center)**center** (`bool`, _optional_, defaults to `True`) — Whether to center the window inside the FFT buffer. Only used when `frame_length` is provided.

Returns an array containing the specified window. This window is intended to be used with `stft`.

The following window types are supported:

-   `"boxcar"`: a rectangular window
-   `"hamming"`: the Hamming window
-   `"hann"`: the Hann window

#### transformers.audio\_utils.spectrogram

[](#transformers.audio_utils.spectrogram)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/audio_utils.py#L267)

( waveform: ndarraywindow: ndarrayframe\_length: inthop\_length: intfft\_length: typing.Optional\[int\] = Nonepower: typing.Optional\[float\] = 1.0center: bool = Truepad\_mode: str = 'reflect'onesided: bool = Truepreemphasis: typing.Optional\[float\] = Nonemel\_filters: typing.Optional\[numpy.ndarray\] = Nonemel\_floor: float = 1e-10log\_mel: typing.Optional\[str\] = Nonereference: float = 1.0min\_value: float = 1e-10db\_range: typing.Optional\[float\] = Nonedtype: dtype = <class 'numpy.float32'> )

Calculates a spectrogram over one waveform using the Short-Time Fourier Transform.

This function can create the following kinds of spectrograms:

-   amplitude spectrogram (`power = 1.0`)
-   power spectrogram (`power = 2.0`)
-   complex-valued spectrogram (`power = None`)
-   log spectrogram (use `log_mel` argument)
-   mel spectrogram (provide `mel_filters`)
-   log-mel spectrogram (provide `mel_filters` and `log_mel`)

How this works:

1.  The input waveform is split into frames of size `frame_length` that are partially overlapping by \`frame\_length
    -   hop\_length\` samples.
2.  Each frame is multiplied by the window and placed into a buffer of size `fft_length`.
3.  The DFT is taken of each windowed frame.
4.  The results are stacked into a spectrogram.

We make a distinction between the following “blocks” of sample data, each of which may have a different lengths:

-   The analysis frame. This is the size of the time slices that the input waveform is split into.
-   The window. Each analysis frame is multiplied by the window to avoid spectral leakage.
-   The FFT input buffer. The length of this determines how many frequency bins are in the spectrogram.

In this implementation, the window is assumed to be zero-padded to have the same size as the analysis frame. A padded window can be obtained from `window_function()`. The FFT input buffer may be larger than the analysis frame, typically the next power of two.

Note: This function is not optimized for speed yet. It should be mostly compatible with `librosa.stft` and `torchaudio.functional.transforms.Spectrogram`, although it is more flexible due to the different ways spectrograms can be constructed.

#### transformers.audio\_utils.power\_to\_db

[](#transformers.audio_utils.power_to_db)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/audio_utils.py#L455)

( spectrogram: ndarrayreference: float = 1.0min\_value: float = 1e-10db\_range: typing.Optional\[float\] = None ) → `np.ndarray`

Parameters

-   [](#transformers.audio_utils.power_to_db.spectrogram)**spectrogram** (`np.ndarray`) — The input power (mel) spectrogram. Note that a power spectrogram has the amplitudes squared!
-   [](#transformers.audio_utils.power_to_db.reference)**reference** (`float`, _optional_, defaults to 1.0) — Sets the input spectrogram value that corresponds to 0 dB. For example, use `np.max(spectrogram)` to set the loudest part to 0 dB. Must be greater than zero.
-   [](#transformers.audio_utils.power_to_db.min_value)**min\_value** (`float`, _optional_, defaults to `1e-10`) — The spectrogram will be clipped to this minimum value before conversion to decibels, to avoid taking `log(0)`. The default of `1e-10` corresponds to a minimum of -100 dB. Must be greater than zero.
-   [](#transformers.audio_utils.power_to_db.db_range)**db\_range** (`float`, _optional_) — Sets the maximum dynamic range in decibels. For example, if `db_range = 80`, the difference between the peak value and the smallest value will never be more than 80 dB. Must be greater than zero.

the spectrogram in decibels

Converts a power spectrogram to the decibel scale. This computes `10 * log10(spectrogram / reference)`, using basic logarithm properties for numerical stability.

The motivation behind applying the log function on the (mel) spectrogram is that humans do not hear loudness on a linear scale. Generally to double the perceived volume of a sound we need to put 8 times as much energy into it. This means that large variations in energy may not sound all that different if the sound is loud to begin with. This compression operation makes the (mel) spectrogram features match more closely what humans actually hear.

Based on the implementation of `librosa.power_to_db`.

#### transformers.audio\_utils.amplitude\_to\_db

[](#transformers.audio_utils.amplitude_to_db)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/audio_utils.py#L506)

( spectrogram: ndarrayreference: float = 1.0min\_value: float = 1e-05db\_range: typing.Optional\[float\] = None ) → `np.ndarray`

Parameters

-   [](#transformers.audio_utils.amplitude_to_db.spectrogram)**spectrogram** (`np.ndarray`) — The input amplitude (mel) spectrogram.
-   [](#transformers.audio_utils.amplitude_to_db.reference)**reference** (`float`, _optional_, defaults to 1.0) — Sets the input spectrogram value that corresponds to 0 dB. For example, use `np.max(spectrogram)` to set the loudest part to 0 dB. Must be greater than zero.
-   [](#transformers.audio_utils.amplitude_to_db.min_value)**min\_value** (`float`, _optional_, defaults to `1e-5`) — The spectrogram will be clipped to this minimum value before conversion to decibels, to avoid taking `log(0)`. The default of `1e-5` corresponds to a minimum of -100 dB. Must be greater than zero.
-   [](#transformers.audio_utils.amplitude_to_db.db_range)**db\_range** (`float`, _optional_) — Sets the maximum dynamic range in decibels. For example, if `db_range = 80`, the difference between the peak value and the smallest value will never be more than 80 dB. Must be greater than zero.

the spectrogram in decibels

Converts an amplitude spectrogram to the decibel scale. This computes `20 * log10(spectrogram / reference)`, using basic logarithm properties for numerical stability.

The motivation behind applying the log function on the (mel) spectrogram is that humans do not hear loudness on a linear scale. Generally to double the perceived volume of a sound we need to put 8 times as much energy into it. This means that large variations in energy may not sound all that different if the sound is loud to begin with. This compression operation makes the (mel) spectrogram features match more closely what humans actually hear.