Parti le Methods for Bayesian Modelling and Enhan ementof

This paper applies time-varying autoregressive (TVAR) models with sto hastially evolving parameters to the problem of spee h modelling and enhan ement. The sto hasti evolution models for the TVAR parameters are Markovian di usion proesses. The main aim of the paper is to perform on-line estimation of the lean spee h and model parameters, and to determine the adequa y of the hosen statisti al models. EÆ ient parti le methods are developed to solve the optimal ltering and xed-lag smoothing problems. The algorithms ombine sequential importan e sampling (SIS), a sele tion step and Markov hain Monte Carlo (MCMC) methods. They employ several varian e redu tion strategies to make the best use of the statistial stru ture of the model. It is also shown how model adequa y may be determined by ombining the parti le lter with frequentist methods. The modelling and enhan ement performan e of the models and estimation algorithms are evaluated in simulation studies on both syntheti and real spee h data sets. SP Edi s: 1-ENHA 1 I Introdu tion A widely used and popular model for the spee h produ tion system is the autoregressive (AR) pro ess  This model exploits the lo al orrelations in a time series by forming the predi tion for the urrent sample as a linear ombination of the immediately pre eding samples. In pra ti e lean spee h signals are rarely available, the spee h being ontaminated by some ba kground or appli ation-spe i noise pro ess. Fortunately, most of these may be adequately modelled as a slowly time-varying white Gaussian or Gaussian mixture pro ess that additively ombines with the lean spee h signal. This is the approa h taken with su ess in e.g. [13, 22℄, and is hen e also adopted here. The main short oming of the AR spee h produ tion model is obvious. Asso iated with the AR oeÆ ients is an arti ulatory on guration that remains xed throughout the analysis interval. In reality, however, the vo al tra t is ontinually hanging, sometimes slowly, sometimes rapidly (e.g. during plosive sounds and spee h transitions). To partly re on ile the time-varying hara ter of the vo al tra t with the time invarian e of the model, spee h is normally pro essed in short (possibly overlapping) segments or frames, during ea h of whi h the signal is assumed to be stationary. However, sin e the framing is de ned a priori with no relation to the phoneti information, non-stationary frames are still likely to o ur, even for very short analysis intervals. In these ir umstan es non-stationary models may provide more true-to-life approximations of the behaviour of the vo al tra t. One su h model is the time-varying AR (TVAR) pro ess. Models within this general lass have been applied in the ontext of spee h modelling and enhan ement before in e.g. [8, 15, 16, 23℄. The TVAR pro ess is a generalisation of the standard AR pro ess where the model parameters are allowed to vary with time. In [30℄ a TVAR spee h produ tion model with sto hasti ally evolving parameters is adopted, and shown to outperform standard AR pro ess models in terms of obje tive spee h modelling and enhan ement riteria. This model is also adopted here. In [30℄ the spee h signal is still pro essed on a frame-by-frame basis, and even though the nonstationary nature of the model allows for longer analysis intervals, undesired blo king artifa ts still remain, and dis ontinuities at the boundaries annot be ompletely eliminated. Also, the iterative nature of the bat h estimation algorithms makes them unsuitable for real-time or near real-time 2 implementations. In most spee h appli ations the samples be ome available sequentially, making them more suited for on-line estimation methods. The development of su h strategies is the main fo us of this paper. The TVAR spee h and noise pro ess model fa ilitates a state-spa e representation. Within a sequential framework general re ursive expressions may be derived for the ltering and xed-lag smoothing distributions, from whi h estimates of the lean spee h signal and model parameters may be obtained. The integrations ne essary to ompute these distributions and the subsequent estimates admit losed-form analyti al solutions in only a small number of spe ialised ases, in luding the elebrated Kalman lter for linear Gaussian state-spa e models. For general state-spa e models, of whi h the one studied here is an example, approximate methods must be employed. Classi al methods to obtain approximations to the desired distributions in lude analyti al approximations, su h as the extended Kalman lter [1℄ and the Gaussian sum lter [2℄, and deterministi numeri al integration te hniques (see e.g. [6℄). The extended Kalman lter and Gaussian sum lter are omputationally heap, but fail in diÆ ult ir umstan es. The numeri al integration te hniques, on the other hand, are only feasible in low-dimensional state-spa es. Another approximation strategy is that of sequential Monte Carlo integration, also ommonly known as parti le methods. These methods were rst introdu ed in automati ontrol at the end of the 1960's [17℄, but due to the primitive omputers available at the time, were largely forgotten. In the beginning of the 1990's the great in rease in omputational power allowed a rebirth of this eld. The rst operational parti le lter, the soalled bootstrap lter, was proposed in  Following this seminal paper, parti le methods have re eived a lot of interest in the engineering and statisti al ommunities (see [10, 25℄ for an introdu tion and [9℄ for a summary of the state-of-the-art). Within the sequential Monte Carlo integration framework the distributions of interest are represented by a large number of samples, alled parti les. As will be evident later, these parti les and their asso iated importan e weights evolve randomly in time a ording to a simulation-based rule. This is equivalent to a dynami grid approximation of the target distributions, where the regions of higher probability are allo ated proportionally more grid positions. Using these parti les Monte Carlo estimates of the quantities of interest may be obtained, with the a ura y of these estimates being independent of the dimension of the state-spa e. This method is easier to 3 implement than lassi al numeri al methods, and allows omplex non-linear and non-Gaussian estimation problems to be solved eÆ iently in an on-line manner. This paper applies parti le te hniques to obtain ltered and xed-lag smoothed estimates of the lean spee h signal and model parameters, when modelling spee h as the output of a TVAR pro ess with sto hasti ally evolving parameters, observed in slowly time-varying additive white Gaussian noise. The algorithms developed here are not just a straightforward appli ation of the basi methods, but are designed to make eÆ ient use of the stru ture of the model, and in orporate various varian e redu tion strategies. Related te hniques have been applied before in the ontext of dis rete state estimation for jump Markov linear systems in [11℄. Furthermore, the ltering strategy developed here is straightforwardly ombined with frequentist methods to perform model validation . The remainder of the paper is organised as follows. The model spe i ation and estimation obje tives are stated in Se tion II. In Se tion III sequential parti le methods are developed to solve the ltering problem and determine the model adequa y. After having shown that a dire t extension of the lter to xed-lag smoothing is ineÆ ient, Se tion IV develops an eÆ ient parti le xed-lag smoothing algorithm, based on the introdu tion of MCMC steps. Se tion V presents and dis usses simulation results on syntheti and real spee h data sets, and some on lusions are rea hed in Se tion VI.  ; (1) where at ( t) , (a1;t ( t) ; : : : ; ak;t ( t)) are the TVAR oeÆ ients, 2et ( t) is the varian e of the TVAR innovation sequen e, and N (0; 1) denotes the standard normal distribution. The signal is assumed to be submerged in additive white Gaussian noise, so that the observed value at time t > 0 be omes yt = xt + nt ( t)nt; nt iid N (0; 1) ; (2) where fntg is a white noise pro ess independent of fetg, and 2nt ( t) is the varian e of the observation noise. Conditionally on f tg the signal model is linear, fa ilitating a onditionally Gaussian statespa e (CGSS) representation. More pre isely, de ning the ve tors t , (xt; : : : ; xt k+1), yt , (yt), vt , (et) and wt , (nt), and the system matri es At ( t) , 2664 aTt ( t) Ik 1 0k 1 13775 Bt ( t) , 2664 et ( t) 0k 1 13775 (3) Ct ( t) = C , 1 01 k 1 Dt ( t) , nt ( t) ; (4) the signal model of (1) and (2) is readily expressed in the CGSS form given by t = At ( t) t 1 +Bt ( t)vt; vt iid N (0nv 1; Inv) (5) yt = Ct ( t) t +Dt ( t)wt; wt iid N (0nw 1; Inw) ; (6) where t 2 Rn is the system state, yt 2 Rny is the observation, and vt 2 Rnv and wt 2 Rnw are the system disturban es at time t, respe tively, and N ( ; ) denotes the Gaussian distribution with mean and ovarian e matrix . It is further assumed that Dt ( t)DTt ( t) > 0, for all t > 0, 0 N (m0 ( 0) ;P0 ( 0)), with P0 ( 0) a positive de nite matrix, and that 0, vt and wt are mutually independent for all t > 0. 5 The model order k