## 3 Automatic Language Identification Using Speech Rhythm Features
### Automatic Extraction of Rhythm Metrics
#### 3.3.1 Detection of Consonantal and Vocalic Intervals  
The interval detection involves the detection of silence intervals, as silences preceding and following an utterance should not be included in calculating rhythm metrics. The silence intervals are detected by Voice Activity Detection (VAD). Among features of speech signals, the sum of the signal energy in a frame is widely used as a criterion for VAD. We estimate the energy for every frame of 20 ms and determine whether each frame is silence or not by comparing it with a pre-defined threshold. When a frame is detected as silence, it does not participate in the following procedure: the detection of consonantal or vocalic intervals. For the interval detection, two GMMs for consonants and vowels are used to detect consonantal and vocalic intervals.  
To construct these GMMs, a phonemically demarcated speech corpus is required. For this work, we used a database (called 'PRAWN_DB') provided in [32] that contains phonemically demarcated read speech data with 20 native English speakers. It could be awkward to use only English data as a training set for interval detection. Since a study argued rhythmic continuum [28], English has been an extremely stress-based language showing high fluctuations of vocalic and consonantal intervals (as aforementioned in Section 3.2). As English retains extreme values in speech rhythm metrics, the difference of languages from English could provide a good criterion of language discrimination.  
The process of model construction begins with extracting MFCCs from speech signals. Speech samples are segmented to a specific size (20 ms) with overlapped samples (10 ms). Then, a window function (Hamming) is used to highlight the intermediate portion of samples. Segmented samples are converted into a frequency-domain using STFT. Finally, coefficients are obtained from the results by discrete cosine transform. Although dynamic features of delta or accelerated coefficients are used in standard speech recognition, only 12 coefficients and one normalized energy value in log-scale are used to reduce the dimension of each GMM.  
The complete procedure for constructing models for interval detection is shown in Figure 2. First, the respective non-silence regions of each speech file of PRAWN_DB is segmented into one of 44 English phonemes with phoneme-based demarcation. Then, each segment is labeled CON (consonant) or VOW (vowel), and MFCCs are extracted. Next, two GMMs are trained with 80% of the data, reserving 20% of the data for the validation set. In the recognition test for the initial GMMs, the accuracy was about 70%. We found that the low performance is caused by vowel-like consonants. Specific groups of consonants, such as liquids (/l, r/), glides (/w, y/), and nasals (/m, n/) preserve vocalic characteristics  
Figure 1: Overall procedure of the automatic extraction of rhythm metrics.  
and tend to be recognized as vowels. Therefore, we included the consonant groups in the vowel model and achieved 90% correctness. Figure 3 represents a sample result of interval detection conducted for unknown speech data. The consonant and vowel detection approach proposed herein requires no transcriptions and speech segment information. The approach automatically detects and identifies consonantal and vocalic regions, only depending on pre-trained GMMs, without knowledge of the language data such as language types or contents of speech, thus allowing extracting speech rhythm features from unknown language data.