Experiments with a new method for fast unsupervised speaker adaptation in continuous speech recognition (2009)

In this paper, a new method for improving continuos speech recognition (CSR) using speaker adaptation is introduced. The method is designed to give a positive contribution to the recognition even when only a small sample of speech is available. The proposed method is unsupervised in the sense that no information about the speaker is provided except the utterance to be recognised. The recognition is based on an artificial neural network (ANN) working on the phoneme level. The hypothesised speaker characteristics as well as a parametrisation of the acoustic segment are input to the ANN. This unified approach makes it possible to search for the optimal speaker parameters and the optimal phoneme sequence in the same optimisation procedure. The effect of speaker adaptation is achieved by assuming that the speaker parameters are constant over the utterance. I N T R O D U C T I O N It is well established that speaker adaptation can contribute considerably to the performance of automatic continuos speech recognition systems (CSR) (Blomberg 1990), (Huang & Lee 1991). However, the procedures used, often require long adaptation time. Different kinds of supervision is also often required. In some of the possible CSR-applications, these requirements are unwanted. One example is systems for information retrieval with a large number of users (Blomberg et al 1993). The proposed method is an attempt to avoid these requirements and to give a significant adaptation effect even when the first utterance of a new speaker is recognised. To perform unsupervised speaker adaptation it is necessary to formalise knowledge o f speaker differences. In this paper the characteristics of a speaker are described by a set of speaker parameters. These parameters w i l l always be related to the speaker's formant space (for example mean F l ) . The problem could be divided into two sub-problems: (i) Given an utterance, estimate the speaker parameters, (ii) Given the utterance and the estimated parameters, find the phoneme sequence. Unfortunately, parameters like mean F l cannot be estimated using statistical methods unless we have a large ensemble o f utterances. This would make the algorithm too slow for our purposes. O n the other hand, i f the correct phoneme sequence was known, then it would be possible to estimate the speaker parameters from a small number o f phoneme samples by measuring the deviations from some standard phoneme formant values. O f course this procedure includes supervision and w i l l not be used here. Fonetik -94 125 The proposed solution is to perform (i) and (ii) simultaneously. The algorithm tries to find the optimal combination of speaker parameters and phoneme sequence with respect to some error measurement. T H E A L G O R I T H M i) Assume that a database organised such that sequences of phoneme segments constituting utterances can be extracted, is available. Each segment is parametrisised to give a representation of its acoustical properties ii) Assume that it is possible to estimate speaker parameters for each speaker. ii i) Assume that a speaker-sensitive A N N with both the acousticaland speaker parameters as input have been trained. iv) Assume that an n-best list of phoneme sequences for the input utterance is given. The algorithm w i l l reorder this n-best list (Hefherington 1993). This list w i l l have been generated by some speaker independent recognition module. v) The task is to find the optimal speaker parameters (giving the minimum ANN-error) for each sequence and reorder the list according to this error measurement. To perform the optimisation efficiently, some properties of A N N ' s and the back propagation algorithm are exploited. In the forward phase of the back propagation algorithm, the output activities of the phonemes are computed as a function of the acousticaland speaker parameters. The error is then the sum of the square of the deviations from the target-values. Typically, the target value for the correct phoneme is 0.9 and for all other phonemes 0.1. In this agorifhm, the error w i l l be minimised by choosing optimal speaker parameters. In the backward phase, derivatives of the error with respect to activations and weights are computed. Here, only the derivatives with respect to the speaker parameters are used. Thus, after the forward and backward phase, the error and the gradient of the objective function are avilable. Given this information it is possible to use a gradient descent method to find the optimal parameters. In this study a Quasi-Newton method was used (Luenberger 1984). The optimisation is constrained by the condition that the speaker parameters are constant during the utterance. This is equivalent to speaker adaptation. T H E E X P E R I M E N T S The T IMIT database is both large enough and has sufficiently many speakers to test the suggested algorithm. In a study by Carlson & Glass (1990) five speaker parameters was measured. The same speakerand acoustical representation are used in this study. The '• E ° ' a speaker parameters are mean formant frequencies and amplitudes of the first three formants and standard deviation values for the first two formants. The acoustical parameters are the first three formant frequencies and amplitudes in the beginning, middle and end of the vowel, the transition speed of the b a s e l i n e , s p e a k e r i n d e p e d e n t s p e a k e r s e n s i t i v e c o n t e x t a n d s p e a k e r s e n s i t i v e Figure I. The classification performance in percentage correct classification. The supplied speaker parameters are used where needed. 126 Working Papers 43, Dept of Linguistics and Phonetics, Lund, Sweden formants in the three parts and the duration of the vowel (28 parameters). Phoneme sequences are restricted to the sequences of the vowels only. Utterances from 350 speakers were used for training the A N N s and 50 speakers was used for testing. A speaker independent A N N with only the acoustical parameters as input was trained with the training utterances. Together with a simplified version of the A * n-best searchalgorithm (Zue et al 1991) a list of vowel-sequence hypotheses for each test utterance was created. A speaker sensitive A N N with both the acoustical parameters and the speaker parameters as input was used for the actual speaker adaptation. In some of the experiments a representation of the phonetic context was also input to the A N N s . Classification results for the different A N N s fed with the correct speaker parameters are shown in Figure 1. The results are similar to those reported by Carlson & Glass (1990). Training o f both A N N s was performed using the standard back propagation algorithm (Rumelhart et al 1986). A l l A N N s have 50 hidden units and 16 output units. The performance of the adaptation depends on the number of vowels in the utterance and the quality of the n-best list. By using the T first vowel tokens from each speaker in the test set, the sequence length can be systematically varied. Performance with varying T is shown in Figure 2. The quality o f the n-best list is varied by randomly choosing Q per cent of the segments and impose the correct classification for them. Forcing a % Error % Improvement •sp. indep Jsp. adapt. • improvement 30