nils-herrmann/clean_scirepeval huggingface dataset:
Feature Pruning for Fast Likelihood Evaluation of Automatic Speech Recognition

This work presents feature pruning, a simple yet effective technique to reduce the likelihood computation in ASR systems that use continuous density HMMs. Our technique, under certain conditions, only evaluates the likelihoods of a fraction of feature elements, and approximates those of the remaining (pruned) ones by prediction. The order in which feature elements are evaluated is obtained by a data-driven approach to minimize computation. With this order, feature pruning can speed up the likelihood evaluation by a factor of 1.33 and reduce its power consumption by 27% for an isolated word recognition task. For a continuous speech recognition system using either monophone or triphone models, the speedup and power reduction of the likelihood evaluation are 1.50 and 35% respectively.
On Virtual Evidence and Soft Evidence in Bayesian Networks

We provide an in-depth description of virtual evidence and its application to Bayesian networks. As we will see, virtual evidence significantly extends the power of Bayesia n networks, and makes them much more widely applicable. It does this by giving Bayesian networks the ability to represent aspects from both directed and undirected graphical models. We describe how both evidence and virtual evidence can be seen as having the evidence nodes in the Bayesian network possess virtual children who are always observed to have value unity, and where the conditional probability table of these children imbue their parents either with a sin gle value (hard evidence) or with a collection of values (soft evidenced), the latter case allowing each such value possessing a weight which gets applied to the probability score. This soft evidence is seen as a straightforward gener alization of evidence, where the soft evidence phenomena is interpreted as information provided from outside the context of the process modeled by the Bayesian network (or alternatively, as an extension to the event space that is par titioned normally by the collection of random variables in the Bayesian network). We discuss Pearl’s example of soft ev idence, interpret hybrid ANN/HMM (artificial-neural network/hidden Markov model) systems as soft evidence, and conclude that while the form of soft evidence depends only on ratios of scores and not on the scores’ absolute value s, the actual score values may be obtained from any information source so desired. We find that some information sources, however, might be more intuitively justifiable than others.
Graphical model approach to pitch tracking

Many pitch trackers based on dynamic programming require meticulous design of local cost and transition cost functions. The forms of these functions are often empirically determined and their parameters are tuned accordingly. Parameter tuning usually requires great effort without a guarantee of optimal performance. This work presents a graphical model framework to automatically optimize pitch tracking parameters in the maximum likelihood sense. Therein, probabilistic dependencies between pitch, pitch transition and acoustical observations are expressed using the language of graphical models, and probabilistic inference is accomplished using the Graphical Model Toolkit (GMTK). Experiments show that this framework not only expedites the design of a pitch tracker, but also yields remarkably good performance for both pitch estimation and voicing decision.
DBN based multi-stream models for audio-visual speech recognition

In this paper, we propose a model based on dynamic Bayesian networks (DBN) to integrate information from multiple audio and visual streams. We also compare the DBN based system (implemented using the Graphical Model Toolkit (GMTK)) with a classical HMM (implemented in the Hidden Markov Model Toolkit (HTK)) for both the single and two stream integration problems. We also propose a new model (mixed integration) to integrate information from three or more streams derived from different modalities and compare the new model's performance with that of a synchronous integration scheme. A new technique to estimate stream confidence measures for the integration of three or more streams is also developed and implemented. Results from our implementation using the Clemson University Audio Visual Experiments (CUAVE) database indicate an absolute improvement of about 4% in word accuracy in the -4 to 10db average case when making use of two audio and one video streams for the mixed integration models over the sychronous models.
Codebook design for ASR systems using custom arithmetic units

Custom arithmetic is a novel and successful technique to reduce the computation and resource utilization of ASR systems running on mobile devices. It represents all floating-point numbers by integer indices and substitutes a sequence of table lookups for all arithmetic operations. The first and crucial step in custom arithmetic design is to quantize system variables, preferably to low precision. This paper explores several techniques to quantize variables with high entropy, including a reordering of Gaussian computation and a normalization of Viterbi search. Furthermore, a discriminatively inspired distortion measure is investigated for scalar quantization to better maintain recognition accuracy. Experiments on an isolated word recognition show that each system variable can be scalar quantized to less than 8 bits using a standard quantization method, except for the alpha probability in Viterbi search which requires 10 bits. However, using our normalization and discriminative distortion measure, the, forward probability can be quantized to 9 bits, thereby halving the corresponding lookup table size. This greatly reduces the memory bandwidth and enables the implementation of custom arithmetic on ASR systems.
Optimal sub-graphical models

We investigate the problem of reducing the complexity of a graphical model (G,PG) by finding a subgraph H of G, chosen from a class of subgraphs H, such that H is optimal with respect to KL-divergence. We do this by first defining a decomposition tree representation for G, which is closely related to the junction-tree representation for G. We then give an algorithm which uses this representation to compute the optimal H ∈ H. Gavril  and Tarjan  have used graph separation properties to solve several combinatorial optimization problems when the size of the minimal separators in the graph is bounded. We present an extension of this technique which applies to some important choices of H even when the size of the minimal separators of G are arbitrarily large. In particular, this applies to problems such as finding an optimal subgraphical model over a (k - 1)-tree of a graphical model over a k-tree (for arbitrary k) and selecting an optimal subgraphical model with (a constant) d fewer edges with respect to KL-divergence can be solved in time polynomial in |V(G)| using this formulation.
Statistical Models for Empirical Search-Based Performance Tuning

Achieving peak performance from the computational kernels that dominate application performance often requires extensive machine-dependent tuning by hand. Automatic tuning systems have emerged in response, and they typically operate by (1) generating a large number of possible, reasonable implementations of a kernel, and (2) selecting the fastest implementation by a combination of heuristic modeling, heuristic pruning, and empirical search (i.e. actually running the code). This paper presents quantitative data that motivate the development of such a search-based system, using dense matrix multiply as a case study. The statistical distributions of performance within spaces of reasonable implementations, when observed on a variety of hardware platforms, lead us to pose and address two general problems which arise during the search process. First, we develop a heuristic for stopping an exhaustive compiletime search early if a near-optimal implementation is found. Secondly, we show how to construct run-time decision rules, based on run-time inputs, for selecting from among a subset of the best implementations when the space of inputs can be described by continuously varying features. We address both problems by using statistical modeling techniques that exploit the large amount of performance data collected during the search. We demonstrate these methods on actual performance data collected by the PHiPAC tuning system for dense matrix multiply. We close with a survey of recent projects that use or otherwise advocate an empirical search-based approach to code generation and algorithm selection, whether at the level of computational kernels, compiler and run-time systems, or problem-solving environments. Collectively, these efforts suggest a number of possible software architectures for constructing platform-adapted libraries and applications.
Optimization on Seperator Trees

Given a chordal graph G, and a class of subgraphs H of G, we investigate the problem of finding an element H ∈ H which minimizes a function f : H → R. We show that when f satisfies a decomposability criterion, and when H admits what we call an efficiently decomposable configuration space with respect to f , then this optimization problem can be solved in polynomial time. We give a dynamic programming formulation for solving this problem, which uses a representation of of G called a seperator-tree. The seperator-clique tree is very closely related to the clique-tree representation for G, and we give a procedure for constructing a seperator-clique tree from a clique-tree. Gavril  and Tarjan [2] have used graph separation properties to solve several combinatorial optimization problems when the size of the minimal separators in the graph is bounded. Our technique is an extension of theirs, and it does not require the minimal separators of G to be of bounded size. We give examples to show that this framework can be used to solve a number of problems in machine learning and statistics that require the reduction of complexity of graphical models.
PAC-learning Bounded Tree-width Graphical Models

We show that the class of strongly connected graphical models with tree-width at most k can be properly efficiently PAC-learnt with respect to the Kullback-Leibler Divergence. Previous approaches to this problem, such as those of Chow (), and Hoffgen () have shown that this class is PAC-learnable by reducing it to a combinatorial optimization problem. However, for k > 1, this problem is NP-complete (), and so unless P=NP, these approaches will take exponential amounts of time. Our approach differs significantly from these, in that it first attempts to find approximate conditional independencies by solving (polynomially many) submodular optimization problems, and then using a dynamic programming formulation to combine the approximate conditional independence information to derive a graphical model with underlying graph of the tree-width specified. This gives us an efficient (polynomial time in the number of random variables) PAC-learning algorithm which requires only polynomial number of samples of the true distribution, and only polynomial running time.
WHAT HMMS CAN'T DO

Hidden Markov models (HMMs) are the predominant methodology for automatic speech recognition (ASR) systems. Ever since their inception, it has been said that HMMs are an inadequate statistical model for such purposes. Results over the years have shown, however, that HMM-based ASR performance continually improves given enough training data and engineering effort. In this paper, we argue that there are, in theory at least, no theoretical limitations to the class of probability distributions representable by HMMs. In search of a model to supersede the HMM for ASR, therefore, we should search for models with better parsimony, computational properties, noise insensitivity, and that better utilize high-level knowledge sources.
A Low-Resource ASR Back-End Based on Custom Arithmetic

Most contemporary ASR systems running on desktops use continuous-density HMMs (CHMM) with oating-p oint representations. It is important to reduce their memory and power requirements so that they can be more aordable for portable devices. In this paper, we pro- pose a novel speech recognition back-end based on custom arithmetic, where all oating-p oint variables are represented by integer indices and all arithmetic operations are replaced by a sequence of table lookups. One critical issue associated with table lookups is what we call an accumulative variable whose dynamic range is either large or unpredictable. Such a variable would introduce much distortion if quantized to low precision, so that the table lookup would incur a great loss of information. We therefore explore dieren t quantiza- tion structures dealing with this problem in likelihood eval- uation, and present a normalization method for the Viterbi search to make the range of the forward probabilities pre- dictable. Furthermore, we investigate several optimization algorithms on system-wide bit-width allocation. The best algorithm uses 80 Kbytes of tables to achieve all back-end operations with only a slight degradation in system perfor- mance. As a side eect, the oine storage for parameters is reduced by 80%, and the memory requirement for online computation is reduced by nearly 70%.
Necessary Intransitive Likelihood-Ratio Classifiers

In pattern classification tasks, errors are introduced because of differences between the true model and the one obtained via model estimation. Using likelihood-ratio based classification, it is possible to correct for this discrepancy by finding class-pair specific terms to adjust the likelihood ratio directly, and that can make class-pair preference relationships intransitive. In this work, we introduce new methodology that makes necessary corrections to the likelihood ratio, specifically those that are necessary to achieve perfect classification (but not perfect likelihood-ratio correction which can be overkill). The new corrections, while weaker than previously reported such adjustments, are analytically challenging since they involve discontinuous functions, therefore requiring several approximations. We test a number of these new schemes on an isolated-word speech recognition task as well as on the UCI machine learning data sets. Results show that by using the bias terms calculated in this new way, classification accuracy can substantially improve over both the baseline and over our previous results.
PRUNING IN LIKELIHOOD EVALUATION OF HMM-BASED SPEECH RECOGNITION

In this work, we present a novel technique to reduce the likelihood computation in ASR systems that use continuous density HMMs. The proposed method, under certain conditions, only evaluates the component likelihoods of certain features, and approximates those of the remaining features by prediction. We investigate two feature clustering approaches associated with the pruning technique. While the simple sequential clustering works remarkably well, a datadriven approach performs even better in its attempt to save computation while maintaining baseline performance. With the second approach, we can achieve a saving of 45% in the likelihood evaluation for isolated word recognition, and 50% for continuous speech recognition using either monophone or triphone models. The technique is easily incorporated into any recognizer and costs only a negligible additional overhead.
DBN based multi-stream models for speech

We propose dynamic Bayesian network (DBN) based synchronous and asynchronous multi-stream models for noise-robust automatic speech recognition. In these models, multiple noise-robust features are combined into a single DBN to obtain better performance than any single feature system alone. Results on the Aurora 2.0 noisy speech task show significant improvements of our synchronous model over both single stream models and over a ROVER based fusion method.
Feature pruning in likelihood evaluation of HMM-based speech recognition

In this work, we present a simple yet effective technique to reduce the likelihood computation in ASR systems that use continuous density HMM. In a variety of speech recognition tasks, likelihood evaluation accounts for a significant portion of the total computational load. Our proposed method, under certain conditions, only evaluates the component likelihoods of certain features, and approximates those of the remaining (pruned) features by prediction. We investigate two feature clustering approaches associated with our pruning technique. While a simple sequential clustering works remarkably well, a data-driven approach performs even better in its attempt to save computation while maintaining baseline recognition accuracy. With the second approach, we can speed up the likelihood evaluation by 33% and reduce its power consumption by 27% for an isolated word recognition task. For a continuous speech recognition system using either monophone or triphone models, the speedup and power reduction of the likelihood evaluation are 50% and 35% respectively.
Selectively Computing Dynamic Features in the Likelihood Computation of ASR Systems

This paper proposes a novel technique to reduce the likelihood computation in ASR systems that use continuous density HMMs. Based on the nature of dynamic features and the numerical properties of Gaussian mixture distributions, we approximate the observation likelihood computation to achieve a speedup. Although the technique does not show appreciable benet in an isolated word task, it yields signicant improvements in continuous speech recognition. For example, 50% of the computation can be saved on the TIMIT database with only a negligible degradation in system performanc
On Triangulating Dynamic Graphical Models

This paper introduces improved methodology to triangulate dynamic graphical models and dynamic Bayesian networks (DBNs). In this approach, a standard DBN template can be modified so the repeating and unrolled graph section may dissect the original DBN time slice and may also span (and partially intersect) many such slices. We introduce the notion of a "boundary" which divides a graph into multi-slice partitions each of which has an interface, and define the "boundary algorithm", a method to find the best boundary (and corresponding interface) between partitions in such models. We prove that, after using this algorithm, the sizes of the best forwardand backward- interface (and also the corresponding fill-ins) are identical. The boundary algorithm allows for constrained elimination orders (and therefore graph triangulations) that are impossible using standard slice-by-slice constrained elimination. We describe the above using the Graphical Model ToolKit (GMTK)'s notion of dynamic graphical model, slightly generalizing standard DBN templates. We report triangulation results on hand-concocted graphs, novel speech recognition DBNs, and random graphs, and find that the boundary algorithm can significantly improve both tree width and graph weight.
Factored Language Models and Generalized Parallel Backoff

We introduce factored language models (FLMs) and generalized parallel backoff (GPB). An FLM represents words as bundles of features (e.g., morphological classes, stems, data-driven clusters, etc.), and induces a probability model covering sequences of bundles rather than just words. GPB extends standard backoff to general conditional probability tables where variables might be heterogeneous types, where no obvious natural (temporal) backoff order exists, and where multiple dynamic backoff strategies are allowed. These methodologies were implemented during the JHU 2002 workshop as extensions to the SRI language modeling toolkit. This paper provides initial perplexity results on both CallHome Arabic and on Penn Treebank Wall Street Journal articles. Significantly, FLMs with GPB can produce bigrams with significantly lower perplexity, sometimes lower than highly-optimized baseline trigrams. In a multi-pass speech recognition context, where bigrams are used to create first-pass bigram lattices or N-best lists, these results are highly relevant.
Low-resource noise-robust feature post-processing on Aurora 2.0

We present a highly effective and extremely simple noiserobust front end based on novel post-processing of standard MFC C features. It performs remarkably well on the Aurora 2.0 noisydigits database without requiring any increase in model complexity. Compared to the Aurora 2.0 baseline system, our technique improves the average word error rate by 45% in the multicondition training case, (matched training/testing conditions) and 60% in the clean training case (mismatched training/testing conditions) — this is an improvement that rivals some of the best known results on this database. Our method, moreover, improves the performances in all cases, regardless of clean or noisy speech, matched or mis-matched environments. Our technique is entirely general because it makes no assumptions about the existence, type, or level of noise in the speech signal. Moreover, its simplicity means that it should be easy to integrate with other techniques in order to yield further improvements.
Towards Simple Methods of Noise Robustness

We introduce an effective and simple noise-robust feature processing technique which achieves very good results on the Aurora noisy-digits database. This technique does not require knowledge of the noise type and level. Also, it does not require any increase in modeling parameters. It performs well both on matched and mis-matched training and testing environments. In comparison to the Aurora baseline results, it improves relative performance by 45% in the case of multi-condition training and 60% in the case of clean training. The improvement is most profound in the noisiest of cases. Our feature processing technique can be easily integrated into other noise-robust feature processing schemes and noise-robust speech models to possibly yield further improvements. In other words, the simplicity of our technique suggests that it might be generally applicable.
Generalized rules for combination and joint training of classifiers

Classifier combination has repeatedly been shown to provide significant improvements in performance for a wide range of classification tasks. In this paper, we focus on the problem of combining probability distributions generated by different classifiers. Specifically, we present a set of new combination rules that generalize the most commonly used combination functions, such as the mean, product, min, and max operations. These new rules have continuous and differentiable forms, and can thus not only be used for combination of independently trained classifiers, but also as objective functions in a joint classifier training scheme. We evaluate both of these schemes by applying them to the combination of phone classifiers in a speech recognition system. We find a significant performance improvement over previously used combination schemes when jointly training and combining multiple systems using a generalization of the product rule.
Mixed-memory Markov models for Automatic Language Identification

Automatic language identification (LID) continues to play an integral part in many multilingual speech applications. The most widespread approach to LID is the phonotactic approach, which performs language classification based on the probabilities of phone sequences extracted from the test signal. These probabilities are typically computed using statistical phone n-gram models. In this paper we investigate the approximation of these standard n-gram models by mixed-memory Markov models with application to both a phone-based and an articulatory feature-based LID system. We demonstrate significant improvements in accuracy with a substantially reduced set of parameters on a 10-way language identification task.