## 5 Perceptual Audio Features Extraction Techniques
### Multiscale Spectro-Temporal-Based Perceptual Features  
There are different approaches in the bibliography that use the concept of spectro-temporal response field and that incorporate two-stage processes inspired in the auditory system. Those approaches rely on the fact that measurements in the primary auditory cortex of different animals revealed its spectro-temporal organization, _i.e._, the receptive fields are selective to modulations in the time-frequency domain. In the following paragraphs, proposals that incorporate spectro-temporal analysis of audio signals at different (temporal and/or frequency) scales are reviewed.  
* **Multiscale spectro-temporal modulations:** it consists of two basic stages, as defined by [PERSON] _et al._[208]. An early stage models the transformation of the acoustic signal into an internal neural representation referred to as an auditory spectrogram (using bank of 128 constant-Q bandpass filters with center frequencies equally spaced on a logarithmic frequency axis). Subsequently a central stage analyzes the spectrogram to estimate the content of its spectral and temporal modulations using a bank of modulation-selective filters (equivalent to a two-dimensional affine wavelet transform of the auditory spectrogram, with a spectro-temporal mother wavelet resembling a two-dimensional spectro-temporal Gabor function) mimicking those described in a model of the mammalian primary auditory cortex. In [208], [PERSON] _et al._ use multiscale spectro-temporal modulations to discriminate speech from nonspeeh consisting of animal vocalizations, music, and environmental sounds. Moreover, these features have been applied to music genre classification ([PERSON] _et al._[209]) and voice activity detection ([PERSON] _et al._[210]).
* **Computational models for auditory receptive fields:** in [206,207], [PERSON] and [PERSON] describe a theoretical and methodological framework to define computational models for auditory receptive fields. The proposal is also based on a two-stage process: (i) a first layer of frequency selective temporal receptive fields where the input signal is represented as multi-scale spectrograms, which can be specifically configured to simulate the physical resonance system in the cochlea spectrogram; (ii) a second layer of spectro-temporal receptive fields which consist of kernel-based 2D processing units in order to capture relevant auditory changes in both time and frequency dimensions (after logarithmic representation in both amplitude and frequency axes, and ignoring phase information), including from separable to non-separable (introducing an specific glissando parameter) spectro-temporal patterns. The presented model is closely related to biological receptive fields (_i.e._, those that can be physiologically measured from neurons, in the inferior colliculus and the primary auditory cortex, as reported by [PERSON] _et al._ in [211]). This work gives an interesting perspective unifying in one theory a way to axiomatically derive representations like Gammatone (see [PERSON] and [PERSON] [212]) or Gabor filterbanks ([PERSON] _et al._[130]), regarding the causality of the filters used in the first audio analysis stage (see Section 5.6). A set of new auditory features are proposed respecting auditory invariances, being the result of the output 2D spectrogram after the kernel-based processing, using different operators like: spectro-temporal smoothings, onset and offset detections, spectral sharpenings, ways for capturing frequency variations over time and glissando estimation.