## 5 Perceptual Audio Features Extraction Techniques
### Frequency Domain Perceptual Features
#### 5.2.1 Modulation-Based Perceptual Frequency Features  
Modulation-based perceptual frequency features represent the low-frequency (e.g., around 20 Hz) modulation content present in audio signals, which produce both amplitude and frequency variations. These variations are easily observed in audio signals that incorporate beats and rhythm (e.g., rhythmic patterns in music, audio signals coming from industrial machineries, speech signals, _etc._). This modulation information can reflect structural evolution along time of the frequency content of a sound and can be measured separately for each frequency band. The following paragraphs describe the most relevant modulation frequency features found in the literature based on a perceptual-based approach, including those reviewed by [PERSON] _et al._[17] and some recent contribution to the field:  
* **4 Hz modulation energy:** is defined with the aim of capturing the most relevant hearing sensation of fluctuation in terms of amplitude- and frequency-modulated sounds (see [PERSON] [164]). The authors propose a model of fluctuation strength, based on a psychoacoustical magnitude, namely the temporal masking pattern. This feature can be computed filtering each subband of a signal spectral analysis by a 4 Hz band-pass filter along time and it has been used for music/speech discrimination (see [PERSON] and [PERSON] [61]). * **Computer model of amplitude-modulation sensitivity of single units in the inferior culiculus:** the work by [PERSON] and [PERSON] [165] introduces a computer model of a neural circuit that replicates amplitude-modulation sensitivity of cells in the central nucleous of the inferior culliculus (ICC) is presented, allowing for the encoding of signal periodicity as a rate-based code. * **Joint acoustic and modulation frequency features:** these are time-invariant representations that model the non-stationary behavior of an audio signal ([PERSON] and [PERSON] [166]). Modulation frequencies for each frequency band are extracted from demodulation of the Bark-scaled spectrogram using the Wavelet transform (see Section 4.3). These features have been used for audio fingerprinting by [PERSON] and [PERSON] [166], and they are similar to rhythm pattern feature (related to rhythm in music). * **Auditory filter bank temporal envelopes:** or AFTE for short, this is another attempt to capture modulation information related to sound [167]. Modulation information is here obtained through bandpass filtering the output bands of a logarithmic-scale filterbank of 4 th-order Gammatone bandpass filters. These features have been used for audio classification and musical genre classification by [PERSON] and [PERSON] [167], and by [PERSON] _et al._[13]. * **Modulation spectrogram:** also referred to as MS, this feature displays and encodes the signal in terms of the distribution of slow modulations across time and frequency, as defined by [PERSON] and [PERSON] [168]. In particular, it was defined to represent modulation frequencies in the speech signal between 0 and 8 Hz, with a peak sensitivity at 4 Hz, corresponding closely to the long-term modulation spectrum of speech. The MS is computed in critical-band-wide channels and incorporates a simple automatic gain control, and emphasizes spectro-temporal peaks. MS has been used for robust speech recognition (see [PERSON] _et al._[169], or [PERSON] _et al._[170]), music classification ([PERSON] _et al._[77]), or content-based audio identification incorporating a Wavelet transform ([PERSON] _et al._[171]). Recently, the MS features have been separated through a tensor factorization model, which represents each component as modulation spectra being activated across different subbands at each time frame, being applied for monaural speech separation purposes in the work by [PERSON] and [PERSON] [172] and for the classification of pathological infant cries ([PERSON] and [PERSON] [173]).