nils-herrmann/clean_scirepeval huggingface dataset:
Noise Robust Discriminative Models

For classification problems, it is important that the classifier is trained with data which is likely to appear in the future. Discriminative models, because of their nature to focus on the boundary between classes rather than data itself, usually do not have the capability to deal with noisy training data. We propose the use of generative models as filters to make discriminative models more robust against noise. Firstly the distribution of the training data is estimated, then examples which do not satisfy some criterion, like having low likelihood, will be considered as outliers and discarded before training discriminative models. The idea was tested on a noisy data set from the UCI Machine Learning Repository.
Nonlinear Analysis of Cognitive and Motor-related EEG Signals

After the recent development of the theory of nonlinear dynamical systems and deterministic chaos and the introduction of one, in theory, simple method for computing the phase space, many researchers started to analyze electroencephalographic (EEG) signals in terms of nonlinear dynamics, with the belief that new information not accessible by linear analysis could be obtained. However, after an initial enthusiasm, many results obtained with short-length and noisy data have come under question. After a short theoretical introduction of the field and the explanation of a practical method to compute reasonable embedding parameters, we present an overview of the results obtained and the problems encountered by the application of nonlinear analysis to cognitive functions.
Modeling human interaction in meetings

The paper investigates the recognition of group actions in meetings by modeling the joint behaviour of participants. Many meeting actions, such as presentations, discussions and consensus, are characterised by similar or complementary behaviour across participants. Recognising these meaningful actions is an important step towards the goal of providing effective browsing and summarisation of processed meetings. A corpus of meetings was collected in a room equipped with a number of microphones and cameras. The corpus was labeled in terms of a predefined set of meeting actions characterised by global behaviour. In experiments, audio and visual features for each participant are extracted from the raw data and the interaction of participants is modeled using HMM-based approaches. Initial results on the corpus demonstrate the ability of the system to recognise the set of meeting actions.
EVALUATION OF FORMANT-LIKE FEATURES FOR ASR IDIAP-RR 02-04 EVALUATION OF FORMANT-LIKE FEATURES FOR ASR

This paper investigates possibilities to automatically find a low-dimensional, formant-related physical representation of the speech signal, which is suitable for automatic speech recognition (ASR). This aim is motivated by the fact that formants have been shown to be discriminant features for ASR. Combinations of automatically extracted formant-like features and ‘conventional’, noiserobust, state-of-the-art features (such as MFCCs including spectral subtraction and cepstral mean subtraction) have previously been shown to be more robust in adverse conditions than state-of-the-art features alone. However, it is not clear how these automatically extracted formant-like features behave in comparison with true formants. The purpose of this paper is to investigate two methods to automatically extract formant-like features, and to compare these features to hand-labeled formant tracks as well as to standard MFCCs in terms of their performance on a vowel classification task.  We would like to thank Prof. James Hillenbrand for making the AEV database available to us and for his swift reply to all our enquiries. The development of the database was supported by a research grant from the American National Institutes of Health (NIDCD 1-R01-DC01661). Febe de Wet's visit to IDIAP was made possible by the I.B.M. Frye grant.  Weber was funded through the Swiss National Science Foundation, project FN 2000-059169.99/1.
The MNIST Database of Handwritten upper-case letters

The MNIST database of Handwritten upper-case letters is a subset derived from NIST Special Database 19 Handprinted Forms and Characters Database, and developed at IDIAP. It comprises the image files of handwritten upper-case letters, which have been size-normalized and centered in a fixed-size image. Generally the image files were written in SVM-Torch format, it's convenient for the Torch users, it's also easy to transfer it into other format. There is software attached with INIST, which used to get visiable image files.
An Asynchronous Hidden Markov Model for Audio-Visual Speech Recognition

This paper presents a novel Hidden Markov Model architecture to model the joint probability of pairs of asynchronous sequences describing the same event. It is based on two other Markovian models, namely Asynchronous Input/Output Hidden Markov Models and Pair Hidden Markov Models. An EM algorithm to train the model is presented, as well as a Viterbi decoder that can be used to obtain the optimal state sequence as well as the alignment between the two sequences. The model has been tested on an audio-visual speech recognition task using the M2VTS database and yielded robust performances under various noise conditions.
Evaluation of formant-like features for ASR

This paper investigates possibilities to automatically find a low-dimensional, formant-related physical representation of the speech signal, which is suitable for automatic speech recognition (ASR). This aim is motivated by the fact that formants have been shown to be discriminant features for ASR. Combinations of automatically extracted formant-like features and `conventional', noise-robust, state-of-the-art features (such as MFCCs including spectral subtraction and cepstral mean subtraction) have previously been shown to be more robust in adverse conditions than state-of-the-art features alone. However, it is not clear how these automatically extracted formant-like features behave in comparison with true formants. The purpose of this paper is to investigate two methods to automatically extract formant-like features, and to compare these features to hand-labeled formant tracks as well as to standard MFCCs in terms of their performance on a vowel classification task.
Towards Robust and Adaptive Speech Recognition Models towards Robust and Adaptive Speech Recognition Models

In this paper, we discuss a family of new Automatic Speech Rec ognition (ASR) approaches, which somewhat deviate from the usual ASR approaches but whi ch have recently been shown to be more robust to nonstationary noise, without requiring specific a daptation or “multi-style” training. More specifically, we will motivate and briefly describe new approaches b a ed on multi-stream and subband ASR. These approaches extend the standard hidden Markov model (HMM) ba sed pproach by assuming that the different (frequency) streams representing the speech signal are pro c ssed by different (independent) “experts”, each expert focusing on a different characteristic of the signal , and that the different stream likelihoods (or posteriors) are combined at some (temporal) stage to yield a globa l recognition output. As a further extension to multi-stream ASR, we will finally introduce a new approach, r eferred to as HMM2, where the HMM emission probabilities are estimated via state specific feature based HMMs responsible for merging the stream information and modeling their possible correlation.
Towards Robust and Adaptive Speech Recognition Models

In this paper, we discuss a family of new Automatic Speech Recognition (ASR) approaches, which somewhat deviate from the usual ASR approaches but which have recently been shown to be more robust to nonstationary noise, without requiring specific adaptation or “multi-style” training. More specifically, we will motivate and briefly describe new approaches based on multi-stream and subband ASR. These approaches extend the standard hidden Markov model (HMM) based approach by assuming that the different (frequency) streams representing the speech signal are processed by different (independent) “experts”, each expert focusing on a different characteristic of the signal, and that the different stream likelihoods (or posteriors) are combined at some (temporal) stage to yield a global recognition output. As a further extension to multi-stream ASR, we will finally introduce a new approach, referred to as HMM2, where the HMM emission probabilities are estimated via state specific feature based HMMs responsible for merging the stream information and modeling their possible correlation.
Offline cursive word recognition using continuous density hidden Markov models trained with PCA or ICA features

This work presents an offline cursive word recognition system dealing with single writer samples. The system is based on a continuous density hidden Markov model trained using either the raw data, or data transformed using principal component analysis or independent component analysis. Both techniques significantly improved the recognition rate of the system. Preprocessing, normalization and feature extraction are described as well as the training technique adopted. Several experiments were performed using a publicly available database. The accuracy obtained is the highest presented in the literature over the same data.
Improving face verification using skin color information

The performance of face verification systems has steadily improved over the last few years, mainly focusing on models rather than on feature processing. State-of-the-art methods often use the gray-scale face image as input. We propose to use an additional feature of the face image: the skin color The new feature set is tested on a benchmark database, namely XM2VTS, using a simple discriminant artificial neural network. Results show that the skin color information improves the performance.
A State-of-the-art Neural Network for Robust Face Verification

The performance of face verification systems has steadily improved over the last few years, mainly focusing on models rather than on feature processing. State-of-the-art methods often use the gray-scale face image as input. In this paper, we propose to use an additional feature to the face image: the skin color. The new feature set is tested on a benchmark database, namely XM2VTS, using a simple discriminant artificial neural network. Results show that the skin color information improves the performance and that the proposed model achieves robust state-of-the-art results.
A multi-sample multi-source model for biometric authentication

In this study, two techniques that can improve the authentication process are examined: (i) multiple samples and (ii) multiple biometric sources. We propose the fusion of multiple samples obtained from multiple biometric sources at the score level. By using the average operator, both the theoretical and empirical results show that integrating as many samples and as many biometric sources as possible can improve the overall reliability of the system. This strategy is called the multi-sample multi-source approach. This strategy was tested on a real-life database using neural networks trained in one-versus-all configuration.
Writer adaptation techniques in off-line cursive word recognition

This work presents the application of HMM adaptation techniques to the problem of off-line cursive script recognition. Instead of training a new model for each writer one first creates a unique model with a mixed database and then adapts it for each different writer using his own small dataset. Experiments on a publicly available benchmark database show that an adapted system has an accuracy higher than 80% even when less than 30 word samples are used during adaptation, while a system trained using the data of the single writer only needs at least 200 words (the estimate is a lower bound) in order to achieve the same performance as the adapted models.
IDIAP Martigny-Valais-Suisse CONFIDENCE MEASURES FOR MULTIMODAL IDENTITY VERIFICATION

Multimodal fusion for identity verification has already shown great improvement compared to unimodal algorithms. In this paper, we propose to integrate confidence measures during the fusion process. We present a comparison of three different methods to generate such confidence information from unimodal identity verification systems. These methods can be used either to enhance the performance of a multimodal fusion algorithm or to obtain a confidence level on the decisions taken by the system. All the algorithms are compared on the same benchmark database, namely XM2VTS, containing both speech and face information. Results show that some confidence measures did improve statistically significantly the performance, while other measures produced reliable confidence levels over the fusion decisions.
Hidden Markov Models and other Finite State Automata for Sequence Processing

learning ; speech Reference EPFL-CHAPTER-82756 URL: http://publications.idiap.ch/downloads/reports/2001/rr01-37.pdf Related documents: http://publications.idiap.ch/index.php/publications/showcite/bourlard:2001:idiap-01-37 Record created on 2006-03-10, modified on 2017-05-10
Transforming the feature vectors to improve HMM based cursive word recognition systems

Although many Offline Cursive Word Recognition systems are based on HMMs, no attention was ever paid, to our knowledge, to the fact that the feature vectors are typically not in the most suitable form for modeling. They are most of the time correlated and embedded in a space of dimension higher than their Intrinsic Dimension. This leads to several problems and has a negative influence on the performance. By applying some transforms it is possible to solve, or at least to attenuate, such problems resulting in data easier to model and in systems with higher recognition rate. In this work, we used Principal Component Analysis (linear and nonlinear) and Independent Component Analysis. A reduction of the error rate by up to 30.3% (over single writer data) and 16.2% (over multiple writer samples) is shown to be achieved.
A comparative study of adaptation methods for speaker verification

learning ; speech Note: IDIAP-RR 01-34 Reference EPFL-CONF-82827 URL: http://publications.idiap.ch/downloads/reports/2002/mariethoz-icslp2002.pdf Related documents: http://publications.idiap.ch/index.php/publications/showcite/mariethoz:2001:idiap-01-34 Record created on 2006-03-10, modified on 2017-05-10
Increasing speech recognition robustness with HMM2

The purpose of this paper is to investigate the behavior of HMM2 models for the recognition of noisy speech. It has previously been shown that HMM2 is able to model dynamically important structural information inherent in the speech signal, often corresponding to formant positions/tracks. As formant regions are known to be robust in adverse conditions, HMM2 seems particularly promising for improving speech recognition robustness. Here, we review different variants of the HMM2 approach with respect to their application to noise-robust automatic speech recognition. It is shown that HMM2 has the potential to tackle the problem of mismatch between training and testing conditions, and that a multi-stream combination of (already noise-robust) cepstral features and formant-like features (extracted by HMM2) improves the noise robustness of a state-of-the-art automatic speech recognition system.
A MULTI-SAMPLE MULTI-SOURCE MODEL FO R B I 0 M ET R I C AUT H ENT I CAT I 0 N

In this study, two techniques that can improve the authentication process are examined: (i) multiple samples and (ii) multiple biometric sources. We propose the fusion of multiple sam- ples obtained from multiple biometric sources at the score level. By using the average operator, both the theoretical and empirical re- sults show that integrating as many samples and as many biometric sources as possible can improve the overall reliability of the system. This strategy is called multi-sample multi-source approach. This strategy was tested on a real-life database using neural networks trained in one-versus-all configuration.
IDIAP HMM/HMM2 System: Theoretical Basis and Software Specifications

State-of-the-art Automatic Speech Recognition (ASR) systems make extensive use of Hidden Markov Models (HMMs), characterized by flexible statistical modeling, powerful optimization (training) techniques and efficient recognition algorithms. When allowed by the software implementation, their flexibility can also be fully exploited in research, by testing various topologies, acoustic units, parameterization schemes, etc. Unfortunately, these HMM systems still suffer from an excessive sensitivity to the variability generally observed in real acoustic environments, including speaker, channel and noise characteristics. In an attempt to tackle this problem, IDIAP recently introduced a new form of HMM, referred to as HMM2, exhibiting numerous potential advantages, which could result in improved robustness of current speech recognition systems. HMM2 can be described as a mixture of HMMs where the HMM emission probabilities (usually estimated by Gaussian Mixtures or a neural network) are themselves estimated by state-specific HMMs working along the acoustic features. Among other properties, it is believed that such HMM2 approach could better model the time/frequency speech flow, including better modeling of the feature correlation. After a brief reminder of the HMM theory, this report first introduces the theoretical basis of HMM2, including their parameterization schemes and the estimation of their parameters through a generalized form of the Expectation-Maximization (EM) training algorithm. It is also the goal of the present report to describe the functionalities and specifications of a new software able to handle, in a flexible way, different forms of HMM and HMM2 topologies and training schemes.
HMM2- Extraction of Formant Features and their Use for Robust ASR

As recently introduced, an HMM2 can be considered as a particular case of an HMM mixture in which the HMM emission probabilities (usually estimated through Gaussian mixtures or an artificial neural network) are modeled by state-dependent, feature-based HMM (referred to as frequency HMM). A general EM training algorithm for such a structure has already been developed. Although there are numerous motivations for using such a structure, and many possible ways to exploit it, this paper will mainly focus on one particular instantiation of HMM2 in which the frequency HMM will be used to extract formant structure information, which will then be used as additional acoustic features in a standard Automatic Speech Recognition (ASR) system. While the fact that this architecture is able to automatically extract meaningful formant information is interesting by itself, empirical results will also show the robustness of these features to noise, and their potential to enhance state-of-the-art noise-robust HMM-based ASR.
Speech recognition using advanced HMM2 features

HMM2 is a particular hidden Markov model where state emission probabilities of the temporal (primary) HMM are modeled through (secondary) state-dependent frequency-based HMMs (see Weber, K. et al.,  2000). As we show in another paper (see Weber et al.,  2001), a secondary HMM can also be used to extract robust ASR features. Here, we further investigate this novel approach towards using a full HMM2 as feature extractor, working in the spectral domain, and extracting robust formant-like features for a standard ASR system. HMM2 performs a nonlinear, state-dependent frequency warping, and it is shown that the resulting frequency segmentation actually contains particularly discriminant features. To improve the HMM2 system further, we complement the initial spectral energy vectors with frequency information. Finally, adding temporal information to the HMM2 feature vector yields further improvements. These conclusions are experimentally validated on the Numbers95 database, where word error rates of 15%, using only a 4-dimensional feature vector (3 formant-like parameters and one time index) were obtained.
HMM2- extraction of formant structures and their use for robust ASR

As recently introduced in , an HMM2 can be considered as a particular case of an HMM mixture in which the HMM emission probabilities (usually estimated through Gaussian mixtures or an artificial neural network) are modeled by statedependent, feature-based HMM (referred to as frequency HMM). A general EM training algorithm for such a structure has been developed . Although there are numerous motivations for using such a structure, and many possible ways to exploit it, this paper will mainly focus on one particular instantiation of HMM2 in which the frequency HMM will be used to extract formant structure information, which will then be used as additional acoustic features in a standard Automatic Speech Recognition (ASR) system. While the fact that this architecture is able to automatically extract meaningful formant information is interesting by itself, empirical results also show the robustness of these features to noise, and their potential to enhance state-of-the-art noise-robust HMM-based ASR.
A Pragmatic View of the Application of HMM2 for ASR

This report investigates the HMM2 approach recently introduced in the framework of automatic speech recognition. HMM2 can be seen as a mixture of HMMs, where a conventional primary HMM (processing a time series of speech data) is supported on a lower level by a secondary HMM, working along the frequency dimension of a temporal segment of speech. The application of HMM2 to the speech signal is motivated by numerous potential advantages. However, speech recognition results did not show the expected performance improvements. In this paper, the HMM2 approach is pragmatically analyzed and evaluated on speech data, revealing some problems and suggesting potential solutions.
Increasing Speech Recognition Noise Robustness with HMM2

The purpose of this paper is to investigate the behavior of HMM2 models for the recognition of noisy speech. It has previously been shown that HMM2 is able to model dynamically important structural information inherent in the speech signal, often corresponding to formant positions/tracks. As formant regions are known to be robust in adverse conditions, HMM2 seems particularly promising for improving speech recognition robustness. Here, we review different variants of the HMM2 approach with respect to their application to noise-robust automatic speech recognition. It is shown that HMM2 has the potential to tackle the problem of mismatch between training and testing conditions, and that a multi-stream combination of (already noise-robust) cepstral features and formant-like features (extracted by HMM2) improves the noise robustness of a state-of-the-art automatic speech recognition system.
Learning the decision function for speaker verification

Explores the possibility of replacing the usual thresholding decision rule of log likelihood ratios used in speaker verification systems by more complex and discriminant decision functions based for instance on linear regression models or support vector machines. Current speaker verification systems, based on generative models such as HMMs or Gaussian mixture models, can indeed easily be adapted to use such decision functions. Experiments on both text dependent and text independent tasks always yielded performance improvements and sometimes significantly.
HMM2- a novel approach to HMM emission probability estimation

In this paper, we discuss and investigate a new method to estimate local emission probabilities in the framework of hidden Markov models (HMM). Each feature vector is considered to be a sequence and is supposed to be modeled by yet another HMM. Therefore, we call this approach `HMM2'. There is a variety of possible topologies of such HMM2 systems, e.g. incorporating trellis or ergodic HMM structures. Preliminary HMM2 speech recognition experiments on cepstral and spectral features yielded worse results than state-of-the-art systems. However, we believe that HMM2 systems have a lot of potential advantages and are therefore worth investigating further.