Using Sequential Patterns as Short-Term and Long-Term Features for Audio Genre Classification

This extended abstract describes a submission to the Music Information Retrieval Evaluation eXchange (MIREX) 2009 for the audio genre classification task. We improved the approach [1] by mining sequential patterns from audio transcriptions according to each genre. Moreover, we consider these patterns as either short-term features or long-term features for further classifiers design. 1. SYSTEM DESCRIPTION The system consists of four stages, as diagrammed in Figure 1. The first stage is based on [2], which aims to tokenize a song into a string of hidden Markov model (HMM) indices (these models are called acoustic segment models, ASM). Then we mine sequential patterns for each genre in the next stage. The third stage attempts to generate two unit-song matrices which denote the occurring frequency of the specific combinations of ASM units for each song; the first unit-song matrix is the same as [2], and we will describe how to generate the second one in the following sections. In the final stage, these two matrices are normalized and utilized for the classifiers design. We submit four versions of algorithms to evaluate our performance. 1.1 Tokenizing Audio as Transcription The tokenization stage which can be found in [2] tokenizes a song into a string of ASM indices. The idea is analogous to tokenizing an utterance into a string of words or phonemes in the field of automatic speech recognition (ASR). First, we perform a maximum-likelihood segmentation algorithm [3] to tokenize the training corpus into acoustic segments. Next, by quantizing the segment centroids, we build a vocabulary of ASMs, which consists of small acoustic tokens. Then, a string of symbols (e.g., the ASM indices that best represent segments of a song) can represent a temporal transcript for a song. Besides, we need to refine these acoustic segment models to capture more exact spectral information. Specifically, these initial transcripts will be considered as a reference to re-train the acoustic models via Baum-Welch estimation, and then Viterbi decoding utilizes the updated ASMs to create new transcriptions. This iterative process is repeated until convergence. In this system, we model each ASM as a 3-state HMM (8-component GMM with a diagonal covariance matrix in each state). In the generation of initial transcriptions, we only use the first 8 MFCCs (frame size is 30ms and no overlap) to capture the slowly changing spectral shape of songs. In the procedure of re-estimating acoustic models and creating new transcriptions, we use 39 MFCCs (frame size is 30ms, and overlap is 10ms) instead. Figure 1. Diagram of our system. 1.2 Mining Sequential Patterns from Each Genre Given a database of music transcriptions and a minimum support (minSup) threshold which is used to determine whether a mined pattern is frequent or not, we can apply ApriorAll [4] algorithm to find the sequential patterns. Table 1 shows an example of music transcriptions. Song Id Transcription 1 (12, 23, 45, 34, 40, 12) 2 (12, 34, 23, 34, 10) 3 (23, 45, 25, 23) 4 (45, 20, 23, 43, 34, 25) 5 (12, 23, 34, 25) Table 1. An example of transcriptions within the same audio genre (number denotes the ASM index). If minSup is set to 0.6 (which denotes a mined sequential pattern needs to exist in at least 3 times out of these 5 transcriptions), we can mine sequential patterns as (45), (23, 25) and (12, 23, 34). In our experiments, the minSup was set empirically to 0.8. Note that the items in sequential patterns need not to be consecutive but the order must be preserved. Moreover, a pattern is said to be a sequential pattern if it is not contained in any other pattern, i.e., pattern is said to be contained in pattern ( if there exists integers i1 < i2 < . < in such that , , ., . Thus, (12), (23), (25), (34), (12, 23), (12, 34) and (23, 34) are not sequential patterns, since they are contained in those three mined sequential patterns. Please refer to ( n a a a . 2 1 )