Discrimination between speech and music based on a low frequency modulation feature

The possibility to discriminate between speech and music signals by using a feature based on low frequency modulation has been investigated. Three different low frequency modulation parameters have been extracted and tested concerning the ability of discrimination. The low frequency modulation amplitudes calculated over 20 critical bands and their standard deviations were found to be good features for this discrimination task even with VQ models. They were also found to be less sensitive to channel quality and model size than MFCC features. 1. Background and review In preprocessors for speech recognition there is often a need to classify and segment the signal before the transcription. Many attempts have been made to discriminate between speech and music or other sound classes [1-4]. Most of these are based on knowledge of the speech production and perception or a combination of the two. The relation between these is described by Greenberg [5] where he in particular points out the important aspects of • the spectral energy distribution • the sound pressure • rapid changes in spectral energy over 100 ms interval • microand macro-modulations These findings are explained from a production point of view by Stevens [6], showing that the larger organs used in the speech production, i.e. the soft palate, the tongue, the jaws and lips need approximately 50-300 ms to complete their extreme movements (from one extreme point to another and back again). This corresponds to a maximum frequency of 320 Hz. Typically the spectral characteristics are used in different kinds of cepstral coefficient analysis and the added information in the rapid change is used in the delta and deltadelta features. These are used in many speech and speaker recognisers and therefore even in the preprocessing part performing the segmentation [1,2]. Some of these aspects can even be found in the time domain, used by Saunders [3] for speech/music discrimination. To catch the shape of the spectrum he used the Zero Crossing Rate, suggested by B. Kedem [7], rather than the cepstral features. The micro-modulations are used in different pitch tracking algorithms [8] and the macro-modulation can be found in the low frequency modulation amplitude. (It will be referred to as the LF modulation or LF component.) Greenberg uses this feature as a preprocessor for a speech recogniser [9]. This technique filters out the low and high frequency noise since the speech information is generally following the syllable rhythm of approximately 4 Hz. This fact is also used as a channel equalisation and noise reduction method, the RASTA processing [10]. Bacon and Viermeister have shown that normal-hearing persons are sensitive to LF modulation, especially in the range of 2-15 Hz [11]. Compare the findings of Stevens [6] above. This LF modulation of speech, among other features, was used by Scheirer & Slaney in a speech/music discriminator used on broadcasting recordings [4]. They found that the normalised 4 Hz component was specifically higher in speech than in music. The component for each of 40 perceptual channels were calculated and added. This feature was found to be a good discriminator. In the same report they also used the fact that music has a beat or rhythm that follows all the frequency bands synchronously. A score for synchronous events in the different bands over 5 seconds was calculated. These features alone were two of the best discriminators in that report. 2. The low frequency modulation The aim of this study has been to examine this LF modulation component. This includes the extraction of a feature representing this aspect and examination of its ability to discriminate between speech and music by itself or in combination with conventional cepstral features. Discriminating between speech and music by the LF component causes problems, however, since music also has a strong LF component in all frequency bands that could be very close to 4 Hz. For example, the rhythm of 60 per second will produce a 4 Hz component on the 16 notes. The difference between speech and music is that in music the modulation is more synchronous over all bands or a wider frequency range, while it differs more in speech. However, there could be some bands, even close to each other, that do not follow the same modulation pattern in music. We found a difference in low frequency modulation behaviour on the following issues: • Speech had a high correlation, especially between adjacent bands, while music had a rather low correlation between all bands, see example in Fig 1. • Both the LF modulation amplitude and its standard deviation varied more for speech than for music, especially in bands 5-10 (or 12), 400 – 1300 (1700) Hz, corresponding to the movements of the first and partly the second formant. This behaviour can be noticed in Fig 2. The first observation could be explained by the fact that almost all frequencies are modulated by the same signal (or movement) in speech while music contains several modulation sources, i.e. many instruments.