nils-herrmann/clean_scirepeval huggingface dataset:
A robust compensation strategy for extraneous acoustic variations in spontaneous speech recognition

We propose a robust compensation strategy to deal effectively with extraneous acoustic variations for spontaneous speech recognition. This strategy extends speaker adaptive training, and uses hidden Markov models (HMM) parameter transformations to normalize the extraneous variations in the training data according to a set of predefined conditions. A "compact" model and the associated prior probability density functions (PDFs) of transformation parameters are estimated using the maximum likelihood criterion. In the testing phase, the generic model and the prior PDFs are used to search for the unknown word sequence based on Bayesian prediction classification (BPC). The proposed strategy is evaluated in the switchboard task, and is used to deal with three types of extraneous variations and mismatch in conversational speech recognition: pronunciation variations, inter-speaker variability, and telephone handset mismatch. Experimental results show that moderate word error rate reduction is achieved in comparison with a well-trained baseline HMM system under identical experimental conditions.
Noise from corrupted speech log mel-spectral energies

In this paper we present a new statistical model, which describes the corruption to speech recognition Mel-frequency spectral features caused by additive noise. This model explicitly represents the effect of unknown phase together with the unobserved clean speech and noise as three hidden variables. We use this model to produce noise robust features for automatic speech recognition. The model is constructed in the log Mel-frequency feature domain. In addition to being linearly related to MFCC recognition parameters, we gain the advantage of low dimensionality and independence of the corruption across feature dimensions. We illustrate the surprising result that, even when the true noise Mel-frequency spectral feature is known, the traditional spectral subtraction formula is flawed. We show the new model can be used to derive a spectral subtraction formula which produces superior error rate results, and is less sensitive to tuning parameters. Finally, we present results demonstrating that the new model is more general than spectral subtraction, and can take advantage of a prior noise estimate to produce robust features, rather than relying on point estimates of noise.
Evaluation of the SPLICE algorithm on the Aurora2 database

This paper describes recent improvements to SPLICE, Stereo-based Piecewise Linear Compensation for Environments, which produces an estimate of cepstrum of undistorted speech given the observed cepstrum of distorted speech. For distributed speech recognition applications, SPLICE can be placed at the server, thus limiting the processing that would take place at the client. We evaluated this algorithm on the Aurora2 task, which consists of digit sequences within the TIDigits database that have been digitally corrupted by passing them through a linear filter and/or by adding different types of realistic noises at SNRs ranging from 20dB to -5dB. For clean acoustic models, we achieve a 67.39% average decrease in word error rate over all test sets. For retrained multi-style acoustic models, the average decrease is 27.87%. The average relative word error rate reduction is 47.63%.
A maximum a posteriori approach to speaker adaptation using the trended hidden Markov model

A formulation of the maximum a posteriori (MAP) approach to speaker adaptation is presented with use of the trended or nonstationary-state hidden Markov model (HMM), where the Gaussian means in each HMM state are characterized by time-varying polynomial trend functions of the state sojourn time. Assuming uncorrelatedness among the polynomial coefficients in the trend functions, we have obtained analytical results for the MAP estimates of the parameters including time-varying means and time-invariant precisions. We have implemented a speech recognizer based on these results in speaker adaptation experiments using the TI46 corpora. The experimental evaluation demonstrates that the trended HMM, with use of either the linear or the quadratic polynomial trend function, consistently outperforms the conventional, stationary-state HMM. The evaluation also shows that the unadapted, speaker-independent models are outperformed by the models adapted by the MAP procedure under supervision with as few as a single adaptation token. Further, adaptation of polynomial coefficients alone is shown to be better than adapting both polynomial coefficients and precision matrices when fewer than four adaptation tokens are used, while the reverse is found with a greater number of adaptation tokens.
An EKF-based algorithm for learning statistical hidden dynamic model parameters for phonetic recognition

Presents a parameter estimation algorithm based on the extended Kalman filter (EKF) for the statistical coarticulatory hidden dynamic model (HDM). We show how the EKF parameter estimation algorithm unifies and simplifies the estimation of both the state and parameter vectors. Experiments based on N-best rescoring demonstrate superior performance of the (context-independent) HDM over a triphone baseline HMM in the TIMIT phonetic recognition task. We also show that the HDM is capable of generating speech vectors close to those from the corresponding real data.
A new method for speech denoising and robust speech recognition using probabilistic models for clean speech and for noise

We present a new method for speech denoising and robust speech recognition. Using the framework of probabilistic models allows us to integrate detailed speech models and models of realistic non-stationary noise signals in a principled manner. The framework transforms the denoising problem into a problem of Bayes-optimal signal estimation, producing minimum mean square error estimators of desired features of clean speech from noisy data. We describe a fast and efficient implementation of an algorithm that computes these estimators. The effectiveness of this algorithm is demonstrated in robust speech recognition experiments, using the Wall Street Journal speech corpus and Microsoft Whisper large-vocabulary continuous speech recognizer. Results show significantly lower word error rates than those under noisy-matched condition. In particular, when the denoising algorithm is applied to the noisy training data and subsequently the recognizer is retrained, very low error rates are obtained.
Efficient on-line acoustic environment estimation for FCDCN in a continuous speech recognition system

There exists a number of cepstral de-noising algorithms which perform quite well when trained and tested under similar acoustic environments, but degrade quickly under mismatched conditions. We present two key results that make these algorithms practical in real noise environments, with the ability to adapt to different acoustic environments over time. First, we show that it is possible to leverage the existing de-noising computations to estimate the acoustic environment on-line and in real time. Second, we show that it is not necessary to collect large amounts of training data in each environment-clean data with artificial mixing is sufficient. When this new method is used as a pre-processing stage to a large vocabulary speech recognition system, it can be made robust to a wide variety of acoustic environments. With synthetic training data, we are able to reduce the word error rate by 27%.
Recursive noise estimation using iterative stochastic approximation for stereo-based robust speech recognition

We present an algorithm for recursive estimation of parameters in a mildly nonlinear model involving incomplete data. In particular, we focus on the time-varying deterministic parameters of additive noise in the nonlinear model. For the nonstationary noise that we encounter in robust speech recognition, different observation data segments correspond to different noise parameter values. Hence, recursive estimation algorithms are more desirable than batch algorithms, since they can be designed to adaptively track the changing noise parameters. One such design based on the iterative stochastic approximation algorithm in the recursive-EM framework is described. This new algorithm jointly adapts time-varying noise parameters and the auxiliary parameters introduced to give a linear approximation of the nonlinear model. We present stereo-based robust speech recognition results for the AURORA task, which demonstrate the effectiveness of the new algorithm compared with a more traditional, MMSE noise estimation technique under otherwise identical experimental conditions.
Efficient decoding strategy for conversational speech recognition using state-space models for vocal-tract-resonance dynamics

In this paper, we present an efficient strategy for likelihood computation and decoding in a continuous speech recognizer using underlying state-space dynamic models for the hidden speech dynamics. The state-space models have been constructed in a special way so as to be suitable for the conversational or casual style of speech where phonetic reduction abounds. The interacting multiple model (IMM) state estimation algorithm for switching state-space models is first introduced, which uses a merging strategy derived from Bayes’s rule to meet the challenge of exponential growth in the switching combination. Then one specific dynamic-programming based decoding algorithm, incorporating the merging strategy, are derived, implemented, and evaluated. It successfully overcomes the exponential growth in the original search paths by using the path-merging strategy. Evaluation experiments on conversational speech using the Switchboard corpus demonstrate that the use of the new decoding strategy is capable of reducing the recognizer’s word error rate compared with the baseline recognizers, including the HMM system and the state-space dynamic model using the HMM-produced phonetic boundaries, under identical test conditions.
High-performance robust speech recognition using stereo training data

We describe a novel technique of SPLICE (Stereo-based Piecewise Linear Compensation for Environments) for high performance robust speech recognition. It is an efficient noise reduction and channel distortion compensation technique that makes effective use of stereo training data. We present a new version of SPLICE using the minimum-mean-square-error decision, and describe an extension by training clusters of hidden Markov models (HMMs) with SPLICE processing. Comprehensive results using a Wall Street Journal large vocabulary recognition task and with a wide range of noise types demonstrate the superior performance of the SPLICE technique over that under noisy matched conditions (19% word error rate reduction). The new technique is also shown to consistently outperform the spectral-subtraction noise reduction technique, and is currently being integrated into the Microsoft MiPad, a new generation PDA prototype.
A functional articulatory dynamic model for speech production

Introduces a statistical speech production model. The model synthesizes natural speech by modeling some key dynamic properties of vocal articulators in a linear/nonlinear state-space framework. The goal-oriented movements of the articulators (tongue tip, tongue dorsum, upper lip, lower lip, and jaw) are described in a linear dynamic state equation. The resulting articulatory trajectories, combined with the effects of the velum and larynx, are nonlinearly mapped into the acoustic feature space (MFCCs). The key challenges in this model are the development of a nonlinear parameter estimation methodology, and the incorporation of appropriate prior assumptions to assert in the articulatory dynamic structure. Such a model can also be directly applied to speech recognition to better account for coarticulation and phonetic reduction phenomena with considerably fewer parameters than HMM based approaches.
A Bayesian approach to the verification problem: applications to speaker verification

In this paper, we study the general verification problem from a Bayesian viewpoint. In the Bayesian approach, the verification decision is made by evaluating Bayes factors against a critical threshold. The calculation of the Bayes factors in turn requires the computation of several Bayesian predictive densities. As a case study, we apply the method to speaker verification based on the Gaussian mixture model (GMM). We propose an efficient algorithm to calculate the Bayes factors for the GMM, where the Viterbi approximation is adopted in the computation of joint Bayesian predictive densities. We evaluate the proposed method for the NIST98 speaker verification evaluation data. Experimental results show that the new Bayesian approach achieves moderate improvements over a well-trained baseline system using the conventional likelihood ratio test.
Speech Denoising and Dereverberation Using Probabilistic Models

This paper presents a unified probabilistic framework for denoising and dereverberation of speech signals. The framework transforms the denoising and dereverberation problems into Bayes-optimal signal estimation. The key idea is to use a strong speech model that is pre-trained on a large data set of clean speech. Computational efficiency is achieved by using variational EM, working in the frequency domain, and employing conjugate priors. The framework covers both single and multiple microphones. We apply this approach to noisy reverberant speech signals and get results substantially better than standard methods.
A path-stack algorithm for optimizing dynamic regimes in a statistical hidden dynamic model of speech

In this paper we report our recent research whose goal is to improve the performance of a novel speech recognizer based on an underlying statistical hidden dynamic model of phonetic reduction in the production of conversational speech. We have developed a path-stack search algorithm which efficiently computes the likelihood of any observation utterance while optimizing the dynamic regimes in the speech model. The effectiveness of the algorithm is tested on the speech data in the Switchboard corpus, in which the optimized dynamic regimes computed from the algorithm are compared with those from exhaustive search. We also present speech recognition results on the Switchboard corpus that demonstrate improvements of the recognizer’s performance compared with the use of the dynamic regimes heuristically set from the phone segmentation by a state-of-the-art hidden Markov model (HMM) system.
Spontaneous speech recognition using a statistical coarticulatory model for the vocal-tract-resonance dynamics.

A statistical coarticulatory model is presented for spontaneous speech recognition, where knowledge of the dynamic, target-directed behavior in the vocal tract resonance is incorporated into the model design, training, and in likelihood computation. The principal advantage of the new model over the conventional HMM is the use of a compact, internal structure that parsimoniously represents long-span context dependence in the observable domain of speech acoustics without using additional, context-dependent model parameters. The new model is formulated mathematically as a constrained, nonstationary, and nonlinear dynamic system, for which a version of the generalized EM algorithm is developed and implemented for automatically learning the compact set of model parameters. A series of experiments for speech recognition and model synthesis using spontaneous speech data from the Switchboard corpus are reported. The promise of the new model is demonstrated by showing its consistently superior performance over a state-of-the-art benchmark HMM system under controlled experimental conditions. Experiments on model synthesis and analysis shed insight into the mechanism underlying such superiority in terms of the target-directed behavior and of the long-span context-dependence property, both inherent in the designed structure of the new dynamic model of speech.
Large-vocabulary speech recognition under adverse acoustic environments

We report our recent work on noise-robust large-vocabulary speech recognition. Three key innovations are developed and evaluated in this work: 1) a new model learning paradigm that comprises a noise-insertion process followed by noise reduction; 2) a noise adaptive training algorithm that integrates noise reduction into probabilistic multi-style system training; and 3) a new algorithm (SPLICE) for noise reduction that makes no assumptions about noise stationarity. Evaluation on a large-vocabulary speech recognition task demonstrates significant and consistent error rate reduction using these techniques. The resulting error rate is shown to be lower than that achieved by the matched-noisy condition for both stationary and nonstationary natural, as well as simulated, noises.
Annotation and Use of Speech Production Corpus for Building Language-Universal Speech Recognizers

A corpus linguistic study is reported in this paper, guided by articulatory phonology and by general phonetic principles of speech production. A direct application of this study is the construction of Hidden Markov Model topologies for automatic speech recognition, taking into account integrated multilingualism with the consideration of the common physiological organs and processes involved in the production of speech sounds from the world’s languages. We demonstrate in this study that incorporation of speech production principles can provide effective constraints on pronunciation modeling for the purpose of building language-universal speech recognizers.
HMM adaptation using vector taylor series for noisy speech recognition

In this paper we address the problem of robustness of speech recognition systems in noisy environments. The goal is to estimate the parameters of a HMM that is matched to a noisy environment, given a HMM trained with clean speech and knowledge of the acoustical environment. We propose a method based on truncated vector Taylor series that approximates the performance of a system trained with that corrupted speech. We also provide insight on the approximations used in the model of the environment and compare them with the lognormal approximation in PMC.
Speaker clustering apparatus based on feature quantities of vocal-tract configuration and speech recognition apparatus therewith

A speaker clustering apparatus generates HMMs for clusters based on feature quantities of a vocal-tract configuration of speech waveform data, and a speech recognition apparatus provided with the speaker clustering apparatus. In response to the speech waveform data of N speakers, an estimator estimates feature quantities of vocal-tract configurations, with reference to correspondence between vocal-tract configuration parameters and Formant frequencies predetermined based on a predetermined vocal tract model of a standard speaker. Further, a clustering processor calculates speaker-to-speaker distances between the N speakers based on the feature quantities of the vocal-tract configurations of the N speakers as estimated, and clusters the vocal-tract configurations of the N speakers using a clustering algorithm based on calculated speaker-to-speaker distances, thereby generating K clusters. Then the clustering processor trains an initial HMM based on the speech waveform data of speakers respectively belonging to the generated K clusters, thereby generating K hidden Markov models corresponding to the K clusters.
Data-driven model construction for continuous speech recognition using overlapping articulatory features

A new, data-driven approach to deriving overlapping articulatory-feature based HMMs for speech recognition is presented in this paper. This approach uses speech data from University of Wisconsin's Microbeam X-ray Speech Production Database. Regression tree models were created for constructing HMMs. Use of actual articulatory data improves upon our previous rule-based feature overlapping system. The regression trees allow construction of the HMM topology for an arbitrary utterance given its phonetic transcription and some prosodic information. Experimental results in ASR show preliminary success of this approach.
A robust training strategy against extraneous acoustic variations for spontaneous speech recognition

In the paper, we propose a robust training strategy to deal with extraneous acoustic variations for conversational speech recognition. This strategy generalizes speaker adaptive training, where HMM parameter transformations are used to normalize the extraneous variations in the training data according to a set of pre-defined conditions. Then a compact model and the associated prior p.d.f.’s of transformation parameters are estimated using the maximum likelihood criterion. In the testing phase, the compact model and the prior p.d.f.’s are used to search for the unknown word sequence based on Bayesian Prediction Classification. The proposed strategy is evaluated in a Switchboard task to deal with pronunciation variations in spontaneous speech recognition. Preliminary results show moderate word error rate reduction over a well-trained baseline system under identical experimental conditions.
Statistical estimation of articulatory trajectories from the speech signal using dynamical and phonological constraints

In speech science and technology, the acoustic-to-articulatory mapping is known as a difficult problem due to its non-linear and one-to-many characteristics. Over the years, different optimization techniques have been proposed to solve this problem. One of these methods is based on the extended Kalman filtering and smoothing. Although the application of this technique to vowels was promising, its extension to all classes of speech sounds has not been successful. This thesis focuses on developing and improving a statistical method of estimating the articulatory trajectories from the speech signal based on the extended Kalman filtering and smoothing. 
In this study, we proposed a new way of constraining the acoustic-to-articulatory inversion by imposing high-level phonological constraints in addition to the dynamical ones. These phonological constraints were imposed by constructing different dynamical models with separate acoustic observation functions for each coproduction unit of speech consisting of two consecutive phones. Each observation sub-function was approximated in small regions by piecewise linear functions using articulatory-acoustic look-up tables. The estimation of the model parameters was based on a direct maximum-likelihood method using training articulatory-acoustic trajectories. An integrated method has been proposed in this study for the recognition of coproduction units and segmentation of the speech signal based on maximum-likelihood of the acoustic observations given different coproduction models. The likelihood of the acoustic observations given every phonological coproduction model was computed using the innovation sequences from the extended Kalman filter. The smoothed articulatory states of the corresponding model with the highest likelihood were used as the best estimate of the articulatory trajectories in every segment. Good estimation results for all classes of speech sounds have been obtained in different experiments using both synthesized and real human data.
A robust speech understanding system using conceptual relational grammar

We describe a robust speech understanding system based on our newly developed approach to spoken language processing. We show that a robust NLU system can be rapidly developed using a relatively simple speech recognizer to provide sufficient information for database retrieval by spoken language. Our experimental system consists of three components: a speech recognizer based on HMM, a natural language parser based on conceptual relational grammar and a data retrieval system based on the ATIS database. With the use of the robust parsing strategy, database query tasks can be successfully performed.