nils-herrmann/clean_scirepeval huggingface dataset:
Subspace constrained Gaussian mixture models for speech recognition

A standard approach to automatic speech recognition uses hidden Markov models whose state dependent distributions are Gaussian mixture models. Each Gaussian can be viewed as an exponential model whose features are linear and quadratic monomials in the acoustic vector. We consider here models in which the weight vectors of these exponential models are constrained to lie in an affine subspace shared by all the Gaussians. This class of models includes Gaussian models with linear constraints placed on the precision (inverse covariance) matrices (such as diagonal covariance, maximum likelihood linear transformation, or extended maximum likelihood linear transformation), as well as the LDA/HLDA models used for feature selection which tie the part of the Gaussians in the directions not used for discrimination. In this paper, we present algorithms for training these models using a maximum likelihood criterion. We present experiments on both small vocabulary, resource constrained, grammar-based tasks, as well as large vocabulary, unconstrained resource tasks to explore the rather large parameter space of models that fit within our framework. In particular, we demonstrate significant improvements can be obtained in both word error rate and computational complexity.
Adaptation of front end parameters in a speech recognizer

In this paper we consider the problem of adapting parameters of the algorithm used for extraction of features. Typical speech recognition systems use a sequence of modules to extract features which are then used for recognition. We present a method to adapt the parameters in these modules under a variety of criteria, e.g maximum likelihood, maximum mutual information. This method works under the assumption that the functions that the modules implement are differentiable with respect to their inputs and parameters. We use this framework to optimize a linear transform preceding the linear discriminant analysis (LDA) matrix and show that it gives significantly better performance than a linear transform after the LDA matrix with small amounts of data. We show that linear transforms can be estimated by directly optimizing likelihood or the MMI objective without using auxiliary functions. We also apply the method to optimize the Mel bins, and the compression power in a system that uses power law compression.
Modeling inverse covariance matrices by basis expansion

This paper proposes a new covariance modeling technique for Gaussian mixture models. Specifically the inverse covariance (precision) matrix of each Gaussian is expanded in a rank-1 basis i.e., /spl Sigma//sub j//sup -1/=P/sub j/=/spl Sigma//sub k=1//sup D//spl lambda//sub k//sup j/a/sub k/a/sub k//sup T/, /spl lambda//sub k//sup j//spl isin//spl Ropf/,a/sub k//spl isin//spl Ropf//sup d/. A generalized EM algorithm is proposed to obtain maximum likelihood parameter estimates for the basis set {a/sub k/a/sub k//sup T/}/sub k=1//sup D/ and the expansion coefficients {/spl lambda//sub k//sup j/}. This model, called the extended maximum likelihood linear transform (EMLLT) model, is extremely flexible: by varying the number of basis elements from D=d to D=d(d+1)/2 one gradually moves from a maximum likelihood linear transform (MLLT) model to a full-covariance model. Experimental results on two speech recognition tasks show that the EMLLT model can give relative gains of up to 35% in the word error rate over a standard diagonal covariance model, 30% over a standard MLLT model.
CEPSTRUM DOMAIN LAPLACE DENOISING

This paper introduces the Laplace algorithm for de-noising in the cepstrum domain with applications to speech recognition. Our method uses Gaussian mixture priors for clean speech and noise cepstra and assumes that speech and noise mix linearly in the spectrum domain. The Laplace algorithm involves two steps (a) computing the posterior mode of the observed noisy cepstra and (b) Gaussian approximation of the posterior around the mode. We show that the Algonquin algorithm is a special case of our approach where a Newton method is used for (a). Interestingly, this observation also proves that the Algonquin algorithm does not converge in general. We propose the use of the BFGS method for (a) which also allows us to efficiently apply the Laplace algorithm in the cepstral domain. De-noising in the cepstral domain gives more than 31% relative reduction in word error rate on average on the Aurora 2 task.
Task adaptation of acoustic and language models based on large quantities of data

We investigate use of large amounts, over 1500 hours, of untranscribed data recorded from a deployed conversational system to improve the acoustic and language models. The system that we considered allows users to perform transactions on their retirement accounts. Using all the untranscribed data we get over 19% relative improvement in word error rate over a baseline system. In contrast, a system built using 70 hours of transcribed data results in over 31% relative improvement.
Phaselets of Framelets 2 . 1 Notation

Phaselets are a set of dyadic wavelets that are related in a particular way such that the associated redundant wavelet transform is nearly shift-invariant [1]. Framelets are a set of functions that generalize the notion of a single dyadic wavelet in the sense that dyadic dilates and translates of these functions form a frame in L(IR) . This paper generalizes the notion of phaselets to framelets. Sets of framelets that only differ in their Fourier transform phase are constructed such that the resulting redundant wavelet transform is approximately shift-invariant. Explicit constructions of phaselets are given for frames with two and three framelet generators. The results in this paper generalize the construction of Hilbert transform pairs of framelets .
Discriminative estimation of subspace precision and mean (SPAM) models

The SPAM model was recently proposed as a very general method for modeling Gaussians with constrained means and covariances. It has been shown to yield significant error rat e improvements over other methods of constraining covarianc es such as diagonal, semi-tied covariances, and extended maxi mum likelihood linear transformations. In this paper we address the problem of discriminative estimation of SPAM mode l parameters, in an attempt to further improve its performanc e. We present discriminative estimation under two criteria: m aximum mutual information (MMI) and an “error-weighted” train ing. We show that both these methods individually result in over 20% relative reduction in word error rate on a digit task over maximum likelihood (ML) estimated SPAM model parameters. We also show that a gain of as much as 28% relative can be achieved by combining these two discriminative estimati on techniques. The techniques developed in this paper also app ly directly to an extension of SPAM called subspace constraine d exponential models.
Lowpass delay filters with flat magnitude and group delay constraints

This paper describes the design of finite impulse response (FIR) delay filters that minimize a squared error and have prescribed number of zeros at /spl omega/=/spl pi/ and prescribed magnitude and group delay flatness at /spl omega/=0. An important special case is the design of least squared error lowpass filters with prescribed flatness constraints and zeros at /spl omega/=/spl pi/. Even though the flatness constraints are in general nonlinear functions of the filter coefficients, we show the remarkable fact that for a subclass of the filters a simple orthogonal projection of least squared error filters onto a special linear subspace determined via Baher (1982) filters gives the solution. The paper also introduces the notion of delay filters that are high-order approximations to the ideal delay and establishes their equivalence to Baher filters. This connection gives novel elementary derivations of Baher filters and their properties. Matlab programs are provided at the end of the paper for the design of filters described in this paper.
Large vocabulary conversational speech recognition with a subspace constraint on inverse covariance matrices

This paper applies the recently proposed SPAM models for acoustic modeling in a Speaker Adaptive Training (SAT) context on large vocabulary conversational speech databases, including the Switchboard database. SPAM models are Gaussian mixture models in which a subspace constraint is placed on the precision and mean matrices (although this paper focuses on the case of unconstrained means). They include diagonal covariance, full covariance, MLLT, and EMLLT models as special cases. Adaptation is carried out with maximum likelihood estimation of the means and feature-space under the SPAM model. This paper shows the first experimental evidence that the SPAM models can achieve significant word-error-rate improvements over state-of-the-art diagonal covariance models, even when those diagonal models are given the benefit of choosing the optimal number of Gaussians (according to the Bayesian Information Criterion). This paper also is the first to apply SPAM models in a SAT context. All experiments are performed on the IBM “Superhuman” speech corpus which is a challenging and diverse conversational speech test set that includes the Switchboard portion of the 1998 Hub5e evaluation data set.
Maximum likelihood training of subspaces for inverse covariance modeling

Speech recognition systems typically use mixtures of diagonal Gaussians to model the acoustics. Using Gaussians with a more general covariance structure can give improved performance; EM-LLT and SPAM models give improvements by restricting the inverse covariance to a linear/affine subspace spanned by rank one and full rank matrices respectively. We consider training these subspaces to maximize likelihood. For EMLLT ML training the subspace results in significant gains over the scheme proposed by Olsen and Gopinath  2002). For SPAM ML training of the subspace slightly improves performance over the method reported by Axelrod, Gopinath and Olsen  2002). For the same subspace size an EMLLT model is more efficient computationally than a SPAM model, while the SPAM model is more accurate. This paper proposes a hybrid method of structuring the inverse covariances that both has good accuracy and is computationally efficient.
Dimensional reduction, covariance modeling, and computational complexity in ASR systems

We study acoustic modeling for speech recognition using mixtures of exponential models with linear and quadratic features tied across all context dependent states. These models are one version of the SPAM models introduced by Axelrod, Gopinath and Olsen (see  2002). They generalize diagonal covariance, MLLT, EMLLT, and full covariance models. Reduction of the dimension of the acoustic vectors using LDA/HDA projections corresponds to a special case of reducing the exponential model feature space. We see, in one speech recognition task, that SPAM models on an LDA projected space of varying dimensions achieve a significant fraction of the WER improvement in going from MLLT to full covariance modeling, while maintaining the low computational cost of the MLLT models. Further, the feature precomputation cost can be minimized using the hybrid feature technique of Visweswariah, Olsen, Gopinath and Axelrod (see ICASSP 2003); and the number of Gaussians one needs to compute can be greatly reducing using hierarchical clustering of the Gaussians (with fixed feature space). Finally, we show that reducing the quadratic and linear feature spaces separately produces models with better accuracy, but comparable computational complexity, to LDA/HDA based models.
Least squared error FIR filters with flat amplitude or group delay constraints

Finite impulse response lowpass filters with prescribed number of zeros at /spl omega/=/spl pi/ and either prescribed group delay flatness or prescribed amplitude flatness and linear phase are designed by minimizing a squared error. A simple orthogonal projection of least squared error filters (that are often available in closed form) onto a linear subspace determined via Thiran filters (for group delay flatness) and via Herrmann filters (for amplitude flatness) gives the desired filters.
The phaselet transform-an integral redundancy nearly shift-invariant wavelet transform

This paper introduces an approximately shift invariant redundant dyadic wavelet transform - the phaselet transform - that includes the popular dual-tree complex wavelet transform of Kingsbury (see Phil.  1999) as a special case. The main idea is to use a finite set of wavelets that are related to each other in a special way - and hence called phaselets - to achieve approximate shift-redundancy; the bigger the set, the better the approximation. A sufficient condition on the associated scaling filters to achieve this is that they are fractional shifts of each other. Algorithms for the design of phaselets with a fixed number vanishing moments is presented - building on the work of Selesnick (see  for the design of wavelet pairs for Kingsbury's dual-tree complex wavelet transform. Construction of two-dimensional (2-D) directional bases from tensor products of one-dimensional (1-D) phaselets is also described. Phaselets as a new approach to redundant wavelet transforms and their construction are both novel and should be interesting to the reader, independent of the approximate shift invariance property that this paper argues they possess.
Acoustic modeling with mixtures of subspace constrained exponential models

Gaussian distributions are usually parameterized with their natural parameters: the mean µ and the covariance Σ. They can also be re-parameterized as exponential models with canonical parameters P =Σ −1 and ψ = Pµ . In this paper we consider modeling acoustics with mixtures of Gaussians parameterized with canonical parameters where the parameters are constrained to lie in a shared affine subspace. This class of models includes Gaussian models with various constraints on its parameters: diagonal covariances, MLLT models, and the recently proposed EMLLT and SPAM models. We describe how to perform maximum likelihood estimation of the subspace and parameters within a fixed subspace. In speech recognition experiments, we show that this model improves upon all of the above classes of models with roughly the same number of parameters and with little computational overhead. In particular we get 30-40% relative improvement over LDA+MLLT models when using roughly the same number of parameters.
MAXIMUM LIKELIHOOD TRAINING OF BASES FOR RAPID ADAPTATION

Adaptation using linear transforms is well known to significantly improve the performance of speech recognition systems. Restricting the transforms to lie in a subspace obtained at training time has been shown to be useful in achieving good performance even with few seconds of adaptation data. This paper addresses the issue of how the subspace is learned. Algorithms for maximum likelihood (ML) estimation of the basis (which defines the subspace) are given. Our experimental results suggest that ML estimation of bases provides better error rate performance over bases learned using Principal Component Analysis (PCA). Moreover, the gains are greater for smaller bases, which are practically important since they can be used at test time with smaller memory and computational overhead. We also explore adaptation with joint mean-feature space transforms.
Adaptation experiments on the SPINE database with the Extended Maximum Likelihood Linear Transformation (EMLLT) model

This paper applies the recently proposed Extended Maximum Likelihood Linear Transformation (EMLLT) model for inverse covariances in a Speaker Adaptive Training (SAT) context. The paper adapts standard algorithms for maximum likelihood estimation of linear transforms for mean, variance and feature space adaptation respectively, to the EMLLT model. Experimental results showing word-error-rate improvements are reported on the SPINE2 database. The system described here is the best-performing system submitted by IBM in the SPINE2 evaluation conducted by NIST in October 2001.
Digit recognition in noisy environments via a sequential GMM/SVM system

This paper exploits the fact that when GMM and SVM classifiers with roughly the same level of performance exhibit uncorrelated errors they can be combined to produce a better classifier. The gain accrues from combining the descriptive strength of GMM models with the discriminative power of SVM classifiers. This idea, first exploited in the context of speaker recognition [1, 2], is applied to speech recognition - specifically to a digit recognition task in a noisy environment - with significant gains in performance.
Large vocabulary conversational speech recognition with the extended maximum likelihood linear transformation (EMLLT) model

This paper applies the recently proposed Extended Maximum Likelihood Linear Transformation (EMLLT) model in a Speaker Adaptive Training (SAT) context on the Switchboard database. Adaptation is carried out with maximum likelihood estimation of linear transforms for the means, precisions (inverse covariances) and the feature-space under the EMLLT model. This paper shows the first experimental evidence that significant word-error-rate improvements can be achieved with the EMLLT model (in both VTL and VTL+SAT training contexts) over a state-of-the-art diagonal covariance model in a difficult large-vocabulary conversational speech recognition task. The improvements were of the order of 1% absolute in multiple scenarios.
A robust high accuracy speech recognition system for mobile applications

This paper describes a robust, accurate, efficient, low-resource, medium-vocabulary, grammar-based speech recognition system using hidden Markov models for mobile applications. Among the issues and techniques we explore are improving robustness and efficiency of the front-end, using multiple microphones for removing extraneous signals from speech via a new multichannel CDCN technique, reducing computation via silence detection, applying the Bayesian information criterion (BIC) to build smaller and better acoustic models, minimizing finite state grammars, using hybrid maximum likelihood and discriminative models, and automatically generating baseforms from single new-word utterances.
Short-time Gaussianization for robust speaker verification

In this paper, a novel approach for robust speaker verification, namely short-time Gaussianization, is proposed. Short-time Gaussianization is initiated by a global linear transformation of the features, followed by a short-time windowed cumulative distribution function (CDF) matching. First, the linear transformation in the feature space leads to local independence or decorrelation. Then the CDF matching is applied to segments of speech localized in time and tries to warp a given feature so that its CDF matches normal distribution. It is shown that one of the recent techniques used for speaker recognition, feature warping [l] can be formulated within the framework of Gaussianization. Compared to the baseline system with cepstral mean subtraction (CMS), around 20% relative improvement in both equal error rate(EER) and minimum detection cost function (DCF) is obtained on NIST 2001 cellular phone data evaluation.
Structuring linear transforms for adaptation using training time information

Linear transforms are often used for adaptation to test data in speech recognition systems. However, when used with small amounts of test data, these techniques provide limited improvements if any. This paper proposes a two-step Bayesian approach where a) the transforms lie in a subspace obtained at training time and b) the expansion coefficients of the transform are obtained using MAP. Estimation algorithms are given for adaptation transforms for means, covariances, and feature spaces. Experimental results indicate that our method gives a significant improvement in performance over other methods.
Modeling with a subspace constraint on inverse covariance matrices

We consider a family of Gaussian mixture models for use in HMM based speech recognition system. These “SPAM” models have state independent choices of subspaces to which the precision (inverse covariance) matrices and means are restricted to belong. They provide a flexible tool for robust, compact, and fast acoustic modeling. The focus of this paper is on the case where the means are unconstrained. The models in the case already generalize the recently introduced EMLLT models, which themselves interpolate between MLLT and full covariance models. We describe an algorithm to train both the state-dependent and state-independent parameters. Results are reported on one speech recognition task. The SPAM models are seen to yield significant improvements in accuracy over EMLLT models with comparable model size and runtime speed. We find a relative reduction in error rate over an MLLT model can be obtained while decreasing the acoustic modeling time by .
Rapid adaptation with linear combinations of rank-one matrices

Linear transforms are often used to adapt the acoustic models in speech recognition systems. When there is very little (5–10 sees.) acoustic data adaptation suffers from unreliable parameter estimation. Typically this problem is handled by imposing a diagonal or block diagonal structure on the transform. This paper proposes using transforms that are linear combinations of rank-one matrices. This approach is applied to the adaptation of the Gaussian means, Gaussian covariances and the acoustic features. Experimental results with varying amounts of adaptation data indicate that for the same number of parameters, our new parameterization performs significantly better than simpler transform parameterizations (diagonal and/or block-diagonal).
EFFICIENT HIERARCHICAL LABELER ALGORITHM FOR GAUSSIAN LIK ELIHOODS COMPUTATION IN RESOURCE CONSTRAINED SPEECH RECOGNITION SY STEMS

This paper presents a new time/memory-efficient algorithm for the evaluation of state likelihoods in an HMM-based speech recognizer where the states are modeled by Gaussian Mixtures. We first present a fast hierarchical labeling scheme and then an improved version, which is specifically geared toward use in recognizers that use asynchronous sear ch (e.g., stack search) as opposed to synchronous (Viterbi) se arch. The improved labeling algorithm improves the speed of the hierarchical labeler by more than 30%.
An EM algorithm for convolutive independent component analysis

We address the problem of blind separation of convolutive mixtures of spatially and temporally independent sources modeled with mixtures of Gaussians. We present an EM algorithm to compute maximum likelihood estimates of both the separating filters and the source density parameters, whereas, in the state-of-the-art, separating filters are usually estimated with gradient descent techniques. The use of the EM algorithm, as opposed to the usual gradient descent techniques, does not require the empirical tuning of a learning rate, and thus can be expected to provide a more stable convergence. Besides, we show how multichannel autoregressive spectral estimation techniques can be used in order to initialize the EM algorithm properly. We demonstrate the efficiency of our EM algorithm together with the proposed initialization scheme by reporting on simulations with artificial mixtures.