# Speaker Diarization: A Review of Objectives and Methods
## 2 Description of the Acoustics of Speech Signals
### Aspects of Speech to Help Estimate Time Stamps  
In terms of the temporal waveform of speech, the signal consists mostly of voiced sounds, which appear as patterns that roughly repeat with every closure of the vocal cords; this abrupt action generates a wide spectral band of energy that acts as a rough \"impulse\" to excite the VT filter. As the VT has a mostly tubular structure that is much more narrow (e.g., 1-2 cm diameter) than its length (average of 17 cm for men), the spectrum of the VT output is that of a series of resonances. For a 17 cm VT, these resonances, called _formants_, are spaced roughly at averages of 500, 1500, 2500, and 3500 Hz. These values derive directly from the speed of sound (about 340 m/s), in a quarter-wavelength resonator, which is an approximate model of a VT closed at the glottis and open at the lips [23]. In strong voiced sounds called _sonorants_, which dominate most of speech, the intense lower formants slowly rise and fall with VT shape changes. These resonance movement contours, which last from 60 ms up to a few seconds (mostly 100-300 ms), are distinctive for both ASR and SD. However, few modern classification algorithms exploit such analysis, as attempts to track formants were largely discarded from consideration decades ago (with some exceptions: [24]).  
Voiced non-noisy sections of speech, thus, have distinctive, regular patterns for individual speakers that can be exploited when judging continuity in audio signals. Both resonances and the vocal cord vibration rate (the latter called the _fundamental frequency_, F0) change relatively slowly, as speakers utter a sequence of linguistic units called _phonemes_, which are the constituents of words. Typically, phonemes average about 80 ms in duration, while F0 in men averages about 100 Hz (F0 depends on vocal cord size and can be up to 1000 Hz in small infants). Thus, a typical voiced phoneme has about eight pitch periods, which slowly vary as F0 changes and as the VT moves from phoneme to phoneme (motion called _coarticulation_).  
For SD, abrupt audio changes are often relevant. Except when the VT excitation changes between voiced and unvoiced (which occurs, on average, 0-2 times per second), the speech spectrum is often fairly stable, with incremental changes from period to period. The other source of abrupt spectral change in speech occurs for nasal consonants, e.g., when the velum lowers and there is a complete closure in the oral portion of the VT (or the reverse change at the end of such consonants). These abrupt changes in speech within an individual's voice may not occur for long stretches of speech (e.g., in a famous example used in testing speech coders decades ago: \"May we all learn a yellow lion roar\"). On the other hand, some unusual utterances (e.g., rapidly repeating a syllable such as /ta/) could have several abrupt changes per second.  
In conversations with multiple speakers, each person would have a pattern of harmonics (spaced at multiples of F0, which changes slowly), modulated by formants that also mostly change slowly, characterized by their VT shape. One could exploit the resulting general continuity of both F0 and formants (within one speaker's voice) to help track individual voices [25]. A commonly used (older) alternative SD approach is to estimate sets of speech features every 10 ms (at a very common 100 Hz _frame_ rate), and then cluster frames with related features into groups of frames that potentially represent a speaker turn. We discuss the advantages and disadvantages of these different approaches below.