# Psychaocoustic Models for Perceptual Audio Coding--A Tutorial Review
## 5 Some Recent Developments  
So far, this tutorial has focused exclusively on perceptual models that have been used for coding applications. These were subject to severe constraints regarding permissible computational and memory complexity, as well as processing delay (i.e., algorithmic delay until the model output is available for the coding process, which usually has to be in the same processing block). Beyond these specific codec models, a plethora of psychoacoustic models has been developed since the beginning to calculate psychoacoustic attributes such as loudness [25; 74], pitch [25], sharpness [25; 75], roughness [25], listener envelopment (LEV) [76], etc. Many models are (at least partially) based on the physiology of the HAS, including the auditory pathway, the cochlea, and the hair cells. Some focus on more exact prediction of the masking behavior [77]. Others form a perceptual measurement system that attempts to objectively predict the audio quality as assessed in subjective listening tests [9; 78; 79]. While none of these models was suitable as part of a first-generation perceptual audio codec, some of them are on their way to being adapted to audio coding, given the increased availability of computational resources nowadays. Specifically, such sophisticated and computationally complex models can be used for offline encoding, i.e., for non-real-time encoding of audio material with file-level access (allowing unlimited look ahead into the signal's \"future\" and multi-pass encoding). This section briefly reviews one example of an advanced model that has been utilized to investigate what benefit it can bring to audio coding. The publication [11] describes a sophisticated psychoacoustic model for the perceptually optimal control of a particular coding tool. The psychoacoustic model part is based on an excitation model by [PERSON] [33], which has a strong focus on correct modeling of temporal masking behavior. The model comprises the following processing steps:* The audio signal is mapped to the cochlea domain by a _fourth order gamma-tone filterbank_, which produces 42 bandpass filter outputs that are equally spaced on the ERB scale. * The behavior of the inner hair cells is modeled by _half-wave rectification_ and subsequent _low-pass processing_ of the filter outputs. This can be seen as a demodulation process of each of the individual bandpass signals. * As a next step, the temporal adaptation of the auditory system (that accounts for phenomena like post masking) is represented by a set of _adaptation loops_. * A modulation filterbank then spectrally analyzes, for each bandpass output, the signal's temporal amplitude modulation for modulation center frequencies between 0 and 243 Hz. * Finally, some internal noise is added to represent the threshold of sensitivity for detection of changes to the signal, as they can occur due to various processing or coding options. This processing derives a so-called _Internal Representation (IR)_ of an audio signal that can be calculated for both an original (reference) signal and the encoded or decoded (processed) signal. The accumulated squared difference of the IRs between original and processed signal forms a metric for the audibility of the differences between both signals. Further relevant perceptual effects are considered by appropriate weighting of the difference of the two internal representations:  
* In order to account for the phenomenon of _Comodulation Masking Release (CMR)_[80; 81], the degree of comodulate-ion between the analyzed band and its surrounding bands is estimated and the IR difference is scaled up correspondingly. CMR describes the phenomenon that thresholds for masking at one frequency band can drop dramatically due to the presence of other (adjacent or non-adjacent) frequency bands that have the same temporal modulation pattern. This effect may play a significant role in the perception of speech signals. An earlier model for consideration of CMR in audio coding was proposed in a previous study [82].