nils-herrmann/clean_scirepeval huggingface dataset:
A factorial HMM aproach to robust isolated digit recognition in background music

This paper presents a novel solution to the problem of isolated digit recognition in background music. A Factorial Hidden Markov Model (FHMM) architecture is proposed to accurately model the simultaneous occurrence of two independent processes, such as an utterance of a digit and an excerpt of music. The FHMM is implemented with its equivalent HMM by extending Nadas’ MIXMAX algorithm to a mixture of Gaussians PDF. At around 0 dB SNR, the proposed system shows an average relative reduction in word error rate of 57% in the recognition of isolated digits in background music.
Formant tracking by mixture state particle filter

This paper presents a mixture state particle filter method for formant tracking during both vowels and consonants. We show that the mixture state particle filter model is able to incorporate prior information about phoneme class into the system, which helps the system to find global optimal solutions. Formant frequencies are defined as eigenfrequencies of the vocal tract in this paper, and by exploring this fact using spectral estimation techniques, the observation PDF of the particle filter can be simplified. We show that by using this likelihood function in the importance weights, the system is able to track the formants using a small number of particles.
Modeling and recognition of phonetic and prosodic factors for improvements to acoustic speech recognition models

This paper examines the usefulness of including prosodic and phonetic context information in the phoneme model of a speech recognizer. This is done by creating a series of prosodic and phonetic models and then comparing the mutual information between the observations and each possible context variable. Prosodic variables show improvement less often than phone context variables, however, prosodic variables generally show a larger increase in mutual information. A recognizer with allophones defined using the maximum mutual information prosodic and phonetic variables outperforms a recognizer with allophones defined exclusively using phonetic variables.
Automatic detection of contrast for speech understanding

Contrast is a very popular phenomenon in spoken language, and carries very important information to help understanding contents and structures of spoken language. In this paper, we propose an idea of automatic contrast detection as an effort for better speech understanding. We study the automatic tagging of three specific types of contrast: symmetric contrast, contrastive focus, and contrastive topic. We label the three types of contrasted words as contrast (C), and other words as noncontrast (¬C). The classification of contrast events is based on prosodic, spectral, and part-of-speech (POS) information sources. The integration of different knowledge sources is realized by a time-delay recursive neural network (TDRNN). The approach we proposed was testified on 235 spontaneous utterances consisting of 3500 words (samples). The contrast detection was speaker independent. The tests yielded an average of 87.9% classification rate.
Strong-sense class-dependent features for statistical recognition

In statistical classification and recognition problems with many classes, it is commonly the case that different classes exhibit wildly different properties. In this case it is unreasonable to expect to be able to summarize these properties by using features designed to represent all the classes. In contrast, features should be designed to represent subsets that exhibit common properties without regard to any class outside this subset. The value of these features for classes outside the subset may be meaningless, or simply undefined. The main problem, due to the statistical nature of the recognizer, is how to compare likelihoods conditioned on different sets of features to decode an input pattern. This paper introduces a class-dependent feature design approach that can be integrated with any probabilistic model. This approach avoids the need of having a conditional probabilistic model for each class and feature type pair, and therefore decreases the computational and storage requirements of using heterogeneous features. This paper presents an algorithm to calculate the class-dependent features that minimize an estimate of the relative entropy between the conditional probabilistic model and the actual conditional probability density function (PDF) of the features of each class. An approach to a hidden Markov model (HMM) automatic speech recognition (ASR) system is applied. A nonlinear class-dependent volume-preserving transformation of the features is used to minimize the objective function. Using this approach, 2% improvement in phoneme recognition accuracy is achieved compared to the baseline system. The approach also shows improvement in recognition accuracy compared to previous class-dependent linear features transformation.
A FACTORIAL HMM APPROACH TO ROBUST ISOLATED DIGIT RECOGNITION IN NON-STATIONARY NOISE

This paper presents a novel solution to the problem of isolated digit recognition in non-stationary noise. A Factorial Hidden Markov Model (FHMM) architecture is proposed that accurately models the simultaneous occurrence of two independent processes, i.e. an utterance of a digit and a clip of non-stationary noise. The FHMM is implemented with its equivalent HMM by extending Nadas’ MIXMAX algorithm to a mixture of Gaussians PDF. The proposed system is tested on the recognition of two digits spoken simultaneously by different talkers as well as on the recognition of isolated digits mixed with background music. At 0 dB SNR, the simultaneous recognition accuracy is improved by 105% over baseline. The system also shows a relative improvement of up to 170% over baseline in the recognition of isolated digits immersed in music at 0 dB SNR.
Particle filtering approach to Bayesian formant tracking

This paper proposes a formant tracker capable of computing the maximum a posteriori probability formant frequencies (eigenfrequencies of the vocal tract) during periods of consonant closure. Two specific novel algorithms are proposed. First, an exponentially weighted autoregressive (EWAR) spectral model is proposed. The EWAR model is capable of modeling the peak amplitudes, bandwidths, and frequencies in an ARMA spectral model without any explicit model of the spectral zeros. Instead of explicit zero models, the amplitudes of spectral peaks are adjusted by exponential coupling weights. It is demonstrated that the parameters of the EWAR model may be efficiently computed from the observed speech cepstrum. Second, the smoothness of formant frequency trajectories is modeled using a linear dynamic systems model with a nonlinear output map, and maximum a posteriori probability tracking of dynamic formant frequencies is demonstrated using a particle filtering approach.
Acoustic segmentation using switching state Kalman filter

Segmenting the acoustic signal in the TIMIT database by a switching state Kalman filter model is reported in this paper. According to the assumption that the high dimensional acoustic feature vector of the LSF (line spectrum frequency) of the speech signal is probably embedded in a low dimensional space, a two dimensional vector is used to represent the continuous state vector in this model. The parameters of the model are initialized by PPCA (probabilistic principal component analysis) and first order vector autoregression, and are re-estimated by the EM algorithm. We show that this model can be used to classify vowels, nasals, frication and silence by an approximate Viterbi inference.
MENTAL STATE DETECTION OF DIALOGUE SYSTEM USERS VIA SPOKEN LANGUAGE

This paper presents an approach to simulate the mental activities of children during their interaction with computers through their spoken language. The mental activities are categorized into three states: confidence, confusion and frustration. Two knowledge sources are used in the detection. One is prosody, which indicates utterance type and user’s attitude. The other is embedded key words/phrases which help interpret the utterances. Moreover, it is found that children’s speech exhibits very different acoustic characteristics from adults. Given the uniqueness of children’s speech, this paper applies a vocal-tract-length-normalization (VTLN)-based technique to compensate for both inter-speaker variability and intraspeaker variability in children’s speech. The detected key words/phrases are then integrated with prosodic information as the cues for the MAP decision of mental states. Tests on a set of 50 utterances collected from the project experiment showed the classification accuracy was 74%.
Analysis of the three-dimensional tongue shape using a three-index factor analysis model.

Three-dimensional tongue shape during vowel production is analyzed using the three-mode PARAFAC (parallel factors) model. Three-dimensional MRI images of five speakers (9 vowels) are analyzed. Sixty-five virtual fleshpoints (13 segments along the rostral-caudal dimension and 5 segments along the right-left direction) are chosen based on the interpolated tongue shape images. Methods used to adjust the alignment of MRI images, to set up the fleshpoints, and to measure the position of the fleshpoints are presented. PARAFAC analysis of this 3D coordinate data results in a stable two-factor solution that explains about 70% of the variance.
The Effect of Accent on Acoustic Cues to Stop Voicing and Place of Articulation in Radio News Speech

In previous research evidence for the effects of stress and accent on phonetic variation is based on laboratory speech. In the present paper, we report on a study of the effects of accent on the acoustic cues for stop voicing and place of articulation in the speech of four announcers from the Boston University Radio News corpus. The results show that there are significant effects of accent on VOT, F0 and Closure Duration for voicing contrasts and significant effects on VOT and Closure Duration for place contrasts. In addition, comparison of the patterns of accentual effects reveals that the effect on voicing cues has a pattern of paradigmatic strengthening and combined strengthening, resulting in enhancement of voicing contrasts while syntagmatic strengthening appears to be the main effect on acoustic cues for place of articulation.
Open‐loop dereverberation of multichannel room impulse responses

We are developing the audio display for a CAVE‐type virtual reality theater, a 3‐m cube with displays covering all six rigid faces. The user’s headgear continuously reports ear positions so headphones would be possible, but we nevertheless prefer loudspeakers because this enhances the sense of total immersion. Because sounds produced at the loudspeakers are distorted by the room impulse responses, we therefore face the problem of controlling the sound at the listener’s two ears. Our proposed solution consists of open‐loop acoustic point control, i.e., dereverberation. The room impulse responses from each loudspeaker to each ear of the listener are inverted using multichannel inversion methods, to create exactly the desired sound field at the listener’s ears. Because the actual room impulse responses cannot be measured in real time (as the listener walks around), instead the impulse responses simulated by the image‐source method is used. A new evaluation criterion is proposed to quantitatively evaluate bot...
Vowel category dependence of the relationship between palate height, tongue height, and oral area.

This article evaluates intertalker variance of oral area, logarithm of the oral area, tongue height, and formant frequencies as a function of vowel category. The data consist of coronal magnetic resonance imaging (MRI) sequences and acoustic recordings of 5 talkers, each producing 11 different vowels. Tongue height (left, right, and midsagittal), palate height, and oral area were measured in 3 coronal sections anterior to the oropharyngeal bend and were subjected to multivariate analysis of variance, variance ratio analysis, and regression analysis. The primary finding of this article is that oral area (between palate and tongue) showed less intertalker variance during production of vowels with an oral place of articulation (palatal and velar vowels) than during production of vowels with a uvular or pharyngeal place of articulation. Although oral area variance is place dependent, percentage variance (log area variance) is not place dependent. Midsagittal tongue height in the molar region was positively correlated with palate height during production of palatal vowels, but not during production of nonpalatal vowels. Taken together, these results suggest that small oral areas are characterized by relatively talker-independent vowel targets and that meeting these talker-independent targets is important enough that each talker adjusts his or her own tongue height to compensate for talker-dependent differences in constriction anatomy. Computer simulation results are presented to demonstrate that these results may be explained by an acoustic control strategy: When talkers with very different anatomical characteristics try to match talker-independent formant targets, the resulting area variances are minimized near the primary vocal tract constriction.
Approximately independent factors of speech using nonlinear symplectic transformation

This paper addresses the problem of representing the speech signal using a set of features that are approximately statistically independent. This statistical independence simplifies building probabilistic models based on these features that can be used in applications like speech recognition. Since there is no evidence that the speech signal is a linear combination of separate factors or sources, we use a more general nonlinear transformation of the speech signal to achieve our approximately statistically independent feature set. We choose the transformation to be symplectic to maximize the likelihood of the generated feature set. In this paper, we describe applying this nonlinear transformation to the speech time-domain data directly and to the Mel-frequency cepstrum coefficients (MFCC). We discuss also experiments in which the generated feature set is transformed into a more compact set using a maximum mutual information linear transformation. This linear transformation is used to generate the acoustic features that represent the distinctions among the phonemes. The features resulted from this transformation are used in phoneme recognition experiments. The best results achieved show about 2% improvement in recognition accuracy compared to results based on MFCC features.
Bayesian learning for models of human speech perception

Human speech recognition error rates are 30 times lower than machine error rates. Psychophysical experiments have pinpointed a number of specific human behaviors that may contribute to accurate speech recognition, but previous attempts to incorporate such behaviors into automatic speech recognition have often failed because the resulting models could not be easily trained from data. This paper describes Bayesian learning methods for computational models for human speech perception. Specifically, the linked computational models proposed in this paper seek to imitate the following human behaviors: independence of distinctive feature errors, perceptual magnet effect, the vowel sequence illusion, sensitivity to energy onsets and offsets, and redundant use of asynchronous acoustic correlates. The proposed models differ from many previous computational psychological models in that the desired behavior is learned from data, using a constrained optimization algorithm (the EM algorithm), rather than being coded into the model as a series of fixed rules.
PROSODY AS A CONDITIONING VARIABLE IN SPEECH RECOGNITION

In this paper, we demonstrate two different methods for improving the accuracy and correctness of the standard HMM speech recognizer. The first method involves incorporating prosody into the acoustic model. The second method modifies the HMM so that it can make use of explicit duration probability densities to improve recognition rates. We prove that both of these methods are effective by running two different experiments using the Hidden Markov Toolkit (HTK). The first experiment determines the correctness and accuracy of a model by performing word recognition. The second experiment determines how well a model picks up on the prosody.
Maximum conditional mutual information projection for speech recognition

Linear discriminant analysis (LDA) in its original modelfree formulation is best suited to classification problems with equal-covariance classes. Heteroscedastic discriminant analysis (HDA) removes this equal covariance constraint, and therefore is more suitable for automatic speech recognition (ASR) systems. However, maximizing HDA objective function does not correspond directly to minimizing the recognition error. In its original formulation, HDA solves a maximum likelihood estimation problem in the original feature space to calculate the HDAtransformationmatrix. Sincethedimensionoftheoriginal feature space in ASR problems is usually high, the estimation of the HDA transformation matrix becomes computationally expensive and requires a large amount of training data.  We start with showing that the calculation of the LDA projection matrix is a maximum mutual information estimation probleminthelower-dimensionalspacewithsomeconstraintsonthe modelofthejointconditionalandunconditionalprobabilitydensity functions (PDF) of the features, and then, by relaxing these constraints,wedevelopadimensionalityreductionapproachthat maximizestheconditionalmutualinformationbetweentheclass identity and the feature vector in the lower-dimensional space given the recognizer model. Using this approach, we achieved 1% improvement in phoneme recognition accuracy compared to the baseline system. Improvement in recognition accuracy compared to both LDA and HDA approaches is also achieved .
PROSODY DEPENDENT SPEECH RECOGNITION ON RADIO NEWS

Does prosody help word recognition? Humans listening to natural prosody, as opposed to monotone or foreign prosody, are able to understand the content with lower cognitive load and higher accuracy . For automatic Large Vocabulary Continuous Speech Recognition (LVCSR), the answer is not that straightforward. Even though successful word recognition and successful prosody recognition have been demonstrated independently in many academic and commercial applications, no result has been reported in the literature that shows improved word recognition on a large-vocabulary continuous speech recognition task with the help of prosody. In 1997, Kompe  presented a theoretical proof stating that prosody can never improve word recognition accuracy unless the recognizer uses prosody dependent models. In this paper, we propose a novel probabilistic framework in which word and phoneme are dependent on prosody in a way that improves word recognition. We propose the use of prosody-dependent allophones based on the “hidden mode variable” theory of Ostendorf et al , but with prosody dependence carefully restricted to a subset of distributions that are known to be most sensitive to prosodic context. Specifically, we propose to model prosody dependence of the phoneme duration probability density functions (PDFs), the acoustic-prosodic observation PDFs and the language model, and to ignore prosody dependence of the acoustic-phonetic observation PDFs. In so doing, we create effective models of the most striking and most often reported prosody-dependent allophonic variation, without significantly increasing the parameter count of the speech recognizer.
The effect of accent on the acoustic cues to stop voicing in Radio News speech

Data from three acoustic cues to stop voicing is analyzed for the effect of phrasal accent in a corpus of read radio news speech from a single speaker of American English. The results show that VOT, Closure Duration and F0 are significant cues to voicing for stops in this corpus, though the acoustic patterns vary by place of articulation. There is a significant effect of phrasal accent on each cue, with increased values for both voiced and voiceless stops, described here as a pattern of syntagmatic strengthening. Patterns of paradigmatic strengthening, with enhanced voicing contrast under accent, are observed only in sporadic cases. The results indicate that accent effects in radio news speech are similar to those reported for “laboratory speech” from prior phonetics studies.
NON-LINEAR INDEPENDENT COMPONENT ANALYSIS FOR SPEECH RECOGNITION

This paper addresses the problem of representing the speech signal using a set of features that are approximately statistically independent. This statistical independence simplifies building probabilistic models based on these features that can be used in applications like speech recognition. Since there is no evidence that the speech signal is a linear combination of separate factors or sources, we use a more general non-linear transformation of the speech signal to achieve our approximately statistically independent feature set. We choose the transformation to be symplectic to maximize the likelihood of the generated feature set. In this paper, we describe applying this nonlinear transformation to the speech time-domain data directly and to the Melfrequency cepstrum coefficients (MFCC). The features resulted from this transformation are used in phoneme recognition experiments. The best results achieved show about 2% improvement in recognition accuracy compared to results based on MFCC features.
Speech Coding: Fundamentals and Applications

In this chapter, we present an overview of the most widely used algorithms, standards, and applications of wideband and narrowband speech coding. Algorithms for speech coding are classified into four broad headings: (1) waveform coding techniques (including PCM, companded PCM, and DPCM), which are typically used for landline telephony, internet telephony, and secure military communications; (2) subband coding, including perceptually transparent multi-rate and embedded coding which is mainly used for internet and digital audio applications; (3) linear predictive analysis by synthesis coding (LPC-AS) algorithms, including multipulse LPC, CELP, SELP, VSELP, and low-delay CELP, which are typically used for digital cellular and telephony; and (4) LPC vocoders, including advanced vocoder algorithms (e.g., MELP, MBE, and PWI) are used for applications such as secure telephony and satellite telephony. Applications in areas such as voiceover IP (VoIP) and digital cellular are emerging and require a speech coder to gracefully adapt to rapidly changing channel conditions—a need that is met by embedded and multirate speech coders associated with joint source-channel coding algorithms. Measures of speech coder perceptual quality include subjective measures of intelligibility (DRT and DALT) and naturalness (MOS and DAM), as well as objective measures such as segmental SNR, Bark spectral distortion, PSQM, and PESQ. Speech coding standards are set by organizations including the ITU (for landline telephony), MPEG (for multimedia applications), ETSI (for European digital cellular), TIA (for U.S. digital cellular), and DDVPC (for United States military applications). 
 
 
 
 
speech coding; 
PCM; 
subband coding; 
CELP; 
LPC; 
digital cellular; 
multimedia; 
voiceover IP; 
mean opinion score (MOS)