nils-herrmann/clean_scirepeval huggingface dataset:
Spectral Subband Centroids as Complementary Features for Speaker Authentication

Most conventional features used in speaker authentication are based on estimation of spectral envelopes in one way or another, e.g., Mel-scale Filterbank Cepstrum Coefficients (MFCCs), Linear-scale Filterbank Cepstrum Coefficients (LFCCs) and Relative Spectral Perceptual Linear Prediction (RASTA-PLP). In this study, Spectral Subband Centroids (SSCs) are examined. These features are the centroid frequency in each subband. They have properties similar to formant frequencies but are limited to a given subband. Empirical experiments carried out on the NIST2001 database using SSCs, MFCCs, LFCCs and their combinations by concatenation suggest that SSCs are somewhat more robust compared to conventional MFCC and LFCC features as well as being partially complementary.
Machine Learning for Multimodal Interaction, First International Workshop,MLMI 2004, Martigny, Switzerland, June 21-23, 2004, Revised Selected Papers

MLMI 2004.- Accessing Multimodal Meeting  Systems, Problems and Possibilities.- Browsing Recorded Meetings with  Meeting Modelling in the Context of Multimodal Research.- Artificial Companions.- Zakim - A Multimodal Software System for Large-Scale Teleconferencing.- Towards Computer Understanding of Human Interactions.- Multistream Dynamic Bayesian Network for Meeting Segmentation.- Using Static Documents as Structured and Thematic Interfaces to Multimedia Meeting Archives.- An Integrated Framework for the Management of Video  The NITE XML Toolkit Meets the ICSI Meeting Corpus: Import, Annotation, and Browsing.- S-SEER: Selective Perception in a Multimodal Office Activity Recognition System.- Mapping from Speech to Images Using Continuous State Space Models.- An Online Algorithm for Hierarchical Phoneme Classification.- Towards Predicting Optimal Fusion Candidates: A Case Study on Biometric Authentication Tasks.- Mixture of SVMs for Face Class Modeling.- AV16.3: An Audio-Visual Corpus for Speaker Localization and Tracking.- The 2004 ICSI-SRI-UW Meeting Recognition System.- On the Adequacy of Baseform Pronunciations and Pronunciation Variants.- Tandem Connectionist Feature Extraction for Conversational Speech  Long-Term Temporal Features for Conversational Speech Recognition.- Speaker Indexing in Audio Archives Using Gaussian Mixture Scoring Simulation.- Speech Transcription and Spoken Document Retrieval in  A Mixed-Lingual Phonological Component Which Drives the Statistical Prosody Control of a Polyglot TTS Synthesis System.- Shallow Dialogue Processing Using Machine Learning Algorithms (or Not).- ARCHIVUS: A System for Accessing the Content of Recorded Multimodal Meetings.- Piecing Together the Emotion Jigsaw.- Emotion Analysis in Man-Machine Interaction Systems.- A Hierarchical System for Recognition, Tracking and Pose Estimation.- Automatic Pedestrian Tracking Using Discrete Choice Models and Image Correlation Techniques.- A Shape Based, Viewpoint Invariant Local Descriptor.
Multimodal Authentication Using Asynchronous HMMs

It has often been shown that using multiple modalities to authenticate the identity of a person is more robust than using only one. Various combination techniques exist and are often performed at the level of the output scores of each modality system. In this paper, we present a novel HMM architecture able to model the joint probability distribution of pairs of asynchronous sequences (such as speech and video streams) describing the same event. We show how this model can be used for audio-visual person authentication. Results on the M2VTS database show robust performances of the system under various audio noise conditions, when compared to other state-of-the-art techniques.
On automatic annotation of meeting databases

In this paper, we discuss meetings as an application domain for multimedia content analysis. Meeting databases are a rich data source suitable for a variety of audio, visual and multi-modal tasks, including speech recognition, people and action recognition, and information retrieval. We specifically focus on the task of semantic annotation of audio-visual (AV) events, where annotation consists of assigning labels (event names) to the data. In order to develop an automatic annotation system in a principled manner, it is essential to have a well-defined task, a standard corpus and an objective performance measure. In this work we address each of these issues to automatically annotate events based on participant interactions.
An Investigation of Spectral Subband Centroids for Speaker Authentication

Most conventional features used in speaker authentication are based on estimation of spectral envelopes in one way or another, in the form of cepstrums, e.g., Mel-scale Filterbank Cepstrum Coefficients (MFCCs), Linear-scale Filterbank Cepstrum Coefficients (LFCCs) and Relative Spectral Perceptual Linear Prediction (RASTA-PLP). In this study, Spectral Subband Centroids (SSCs) are examined. These features are the centroid frequency in each subband. They have properties similar to the formant frequency but are limited to a given subband. Preliminary empirical findings, on a subset of the XM2VTS database, using Analysis of Variance and Linear Discriminant Analysis suggest that, firstly, a certain number of centroids (up to about 16) are necessary to cover enough information about the speaker's identity; and secondly, that SSCs could provide complementary information to the conventional MFCCs. Theoretical findings suggest that mean-subtracted SSCs are more robust to additive noise. Further empirical experiments carried out on the more realistic NIST2001 database using SSCs, MFCCs (respectively LFCCs) and their combinations by concatenation suggest that SSCs are indeed robust and complementary features to conventional MFCC (respectively LFCCs) features often used in speaker authentication.
Non-Linear Variance Reduction Techniques in Biometric Authentication

In this paper, several approaches that can be used to improve biometric authentication applications are proposed. The idea is inspired by the ensemble approach, i.e., the use of several classifiers to solve a problem. Compared to using only one classifier, the ensemble of classifiers has the advantage of reducing the overall variance of the system. Instead of using multiple classifiers, we propose here to examine other possible means of variance reduction (VR), namely through the use of multiple synthetic samples, different extractors (features) and biometric modalities. The scores are combined using the average operator, Multi-Layer Perceptron and Support Vector Machines. It is found empirically that VR via modalities is the best technique, followed by VR via extractors, VR via classifiers and VR via synthetic samples. This order of effectiveness is due to the corresponding degree of independence of the combined objects (in decreasing order). The theoretical and empirical findings show that the combined experts via VR techniques always perform better than the average of their participating experts. Furthermore, in practice, most combined experts perform better than any of their participating experts.
Comparison of four machine learning algorithms for spatial data analysis

This chapter proposes a clear methodology on how to use machine learning algorithms for spatial data analysis in order to avoid any bias and eventually obtain fair estimation of their performance on new data. Four different machine learning algorithms are presented, namely multilayer perceptrons (MLP), mixture of experts (ME), support vector regression (SVR) and a local version of the latter (local SVR). Evaluation criteria adapted to geostatistical problems are also presented in order to compare adequately different models on the same dataset. Finally, an experimental comparison is given on the SIC97 dataset as well as an analysis of the results.
Robust Features for Frontal Face Authentication in Difficult Image Conditions

In this paper we extend the recently proposed DCT-mod2 feature extraction technique (which utilizes polynomial coefficients derived from 2D DCT coefficients obtained from horizontally & vertically neighbouring blocks) via the use of various windows and diagonally neighbouring blocks. We also propose enhanced PCA, where traditional PCA feature extraction is combined with DCT-mod2. Results using test images corrupted by a linear and a non-linear illumination change, white Gaussian noise and compression artefacts, show that use of diagonally neighbouring blocks and windowing is detrimental to robustness against illumination changes while being useful for increasing robustness against white noise and compression artefacts. We also show that the enhanced PCA technique retains all the positive aspects of traditional PCA (that is, robustness against white noise and compression artefacts) while also being robust to illumination changes; moreover, enhanced PCA outperforms PCA with histogram equalisation pre-processing.
Offline recognition of large vocabulary cursive handwritten text

This paper presents a system for the offline recognitionof cursive handwritten lines of text. The system is based oncontinuous density HMMs and Statistical Language Models.The system recognizes data produced by a single writer.No a-priori knowledge is used about the content of the textto be recognized. Changes in the experimental setup withrespect to the recognition of single words are highlighted.The results show a recognition rate of ~85% with a lexiconcontaining 50'000 words. The experiments were performedover a publicly available database.
Variance Reduction Techniques in Biometric Authentication

In this paper, several approaches that can be used to improve biometric authentication applications are proposed. The idea is inspired by the ensemble approach, i.e., the use of several classifiers to solve a problem. Compared to using only one classifier, the ensemble of classifiers has the advantage of reducing the overall variance of the system. Instead of using multiple classifiers, we propose here to examine other possible means of variance reduction (VR), namely through the use of multiple real samples, synthetic samples, different extractors (features) and biometric modalities. It is found empirically that VR via modalities is the best technique, followed by VR via real samples, VR via extractors, VR via classifiers and VR via synthetic samples. This order of effectiveness is due to the corresponding degree of independence of the combined objects (in decreasing order). The theoretical and empirical findings show that the combined experts via VR techniques always perform better than the average of their participating experts. Furthermore, in practice, most combined experts perform better than any of their participating experts
From Samples to Objects in Kernel Methods

This paper presents a general method for incorporating prior knowledge into kernel methods. It applies when the prior knowledge can be formalized by the description of an object around each sample of the training set, assuming that all points in the given object share the same desired class. Two implementation techniques of this method, based on analytical kernel jittering and the vicinal risk minimization principle, are considered. Empirical results on one artificial dataset and one real dataset based on EEG signals demonstrate the performance of the proposed method.
ADAPTED GENERATIVE MODELS FOR FACE VERIFICATION

It has been shown previously that systems based on local features and relatively complexgenerative models, namely 1D Hidden Markov Models (HMMs) and pseudo-2D HMMs, are suitable forface recognition (here we mean both identiﬁcation and veriﬁcation). Recently a simpler generative model,namely the Gaussian Mixture Model (GMM), was also shown to perform well. In this paper we ﬁrst proposeto increase the performance of the GMM approach (without sacriﬁcing its simplicity) through the use oflocal features with embedded positional information; we show that the performance obtained is comparableto 1D HMMs. Secondly, we evaluate different training techniques for both GMM and HMM based systems.We show that the traditionally used Maximum Likelihood (ML) training approach has problems estimatingrobust model parameters when there is only a few training images available; we propose to tackle thisproblem through the use of Maximum a Posteriori (MAP) training, where the lack of data problem canbe effectively circumvented; we show that models estimated with MAP are signiﬁcantly more robust and areable to generalize to adverse conditions present in the BANCA database.
Face Verification Competition on the XM2VTS Database

In the year 2000 a competition was organised to collect face verification results on an identical, publicly available data set using a standard evaluation protocol. The database used was the Xm2vts database along with the Lausanne protocol . Four different institutions submitted results on the database which were subsequently published in [13]. Three years later, a second contest using the same dataset and protocol was organised as part of AVBPA 2003. This time round seven seperate institutions submitted results to the competition. This paper presents the results of the competition and shows that verification results on this protocol have increased in performance by a factor of 3.
Multi-modal audio-visual event recognition for football analysis

The recognition of events within multi-modal data is a challenging problem. In this paper we focus on the recognition of events by using both audio and video data. We investigate the use of data fusion techniques in order to recognise these sequences within the framework of hidden Markov models (HMM) used to model audio and video data sequences. Specifically we look at the recognition of play and break sequences in football and the segmentation of football games based on these two events. Recognising relatively simple semantic events such as this is an important step towards full automatic indexing of such video material. These experiments were done using approximately 3 hours of data from two games of the Euro96 competition. We propose that modelling the audio and video streams separately for each sequence and fusing the decisions from each stream should yield an accurate and robust method of segmenting multi-modal data.
Phoneme-grapheme based speech recognition system

State-of-the-art ASR systems typically use phonemes as the subword units. We investigate a system where the word models are defined in-terms of two different subword units, i.e., phonemes and graphemes. We train models for both the subword units, and then perform decoding using either both or just one subword unit. We have studied this system for American English where there is weak correspondence between grapheme and phoneme. We carried out the study in the framework of a state-of-the-art hybrid HMM/ANN system. The results show that there is good potential in using graphemes as auxiliary subword units.
Con dence Measures for Multimodal Identity Veri cation ?

Multimodal fusion for identity veri cation has already shown great improvement compared to unimodal algorithms. In this paper, we propose to integrate con dence measures during the fusion process. We present a comparison of three di erent methods to generate such con dence information from unimodal identity veri cation systems. These methods can be used either to enhance the performance of a multimodal fusion algorithm or to obtain a con dence level on the decisions taken by the system. All the algorithms are compared on the same benchmark database, namely XM2VTS, containing both speech and face information. Results show that some con dence measures did improve statistically signi cantly the performance, while other measures produced reliable con dence levels over the fusion decisions.
The BANCA Database and Evaluation Protocol

In this paper we describe the acquisition and content of a new large, realistic and challenging multi-modal database intended for training and testing multi-modal verification systems. The BANCA database was captured in four European languages in two modalities (face and voice). For recording, both high and low quality microphones and cameras were used. The subjects were recorded in three different scenarios, controlled, degraded and adverse over a period of three months. In total 208 people were captured, half men and half women. In this paper we also describe a protocol for evaluating verification algorithms on the database. The database will be made available to the research community through http://www.ee.surrey.ac.uk/Research/VSSP/banca.
Scalability Analysis of Audio-Visual Person Identity Verification

In this work, we present a multimodal identity verification system based on the fusion of the face image and the text independent speech data of a person. The system conciliates the monomodal face and speaker verification algorithms by fusing their respective scores. In order to assess the authentication system at different scales, the performance is evaluated at various sizes of the face and speech user template. The user template size is a key parameter when the storage space is limited like in a smart card. Our experimental results show that the multimodal fusion allows to reduce significantly the user template size while keeping a satisfactory level of performance. Experiments are performed on the newly recorded multimodal database BANCA.
Martigny-Valais-Suisse TOWARDS ROBUST AND ADAPTIVE SPEECHRECOGNITION MODELS

In this paper, we discuss a family of new Automatic Speech Recognition (ASR) approaches, which somewhat deviate from the usual ASR approaches but which have recently been shown to be more robust to nonstationary noise, without requiring specific adaptation or “multi-style” training. More specifically, we will motivate and briefly describe new approaches based on multi-stream and subband ASR. These approaches extend the standard hidden Markov model (HMM) based approach by assuming that the different (frequency) streams representing the speech signal are processed by different (independent) “experts”, each expert focusing on a different characteristic of the signal, and that the different stream likelihoods (or posteriors) are combined at some (temporal) stage to yield a global recognition output. As a further extension to multi-stream ASR, we will finally introduce a new approach, referred to as HMM2, where the HMM emission probabilities are estimated via state specific feature based HMMs responsible for merging the stream information and modeling their possible correlation.
Evaluation of formant-like features for automatic speech recognition 1

This study investigates possibilities to find a low-dimensional, formant-related physical representation of speech signals, which is suitable for automatic speech recognition. This aim is motivated by the fact that formants are known to be discriminant features for speech recognition. Combinations of automatically extracted formant-like features and state-of-the-art, noise-robust features have previously been shown to be more robust in adverse conditions than state-of-the-art features alone. However, it is not clear
Speech & Face Based Biometric Authentication at Idiap Speech & Face Based Biometric Authentication at Idiap

We present an overview of recent research at IDIAP on speech & face based biometric authentication. This report covers user-customised passwords, adaptation techniques, confidence measures (for use in fusion of audio & visual scores), face verification in difficult image conditions, as well as other related research issues. We also overview the Torch machine-learning library, which has aided in the implementation of the above mentioned techniques.
Client Dependent GMM-SVM Models for Speaker Verification

Generative Gaussian Mixture Models (GMMs) are known to be the dominant approach for modeling speech sequences in text independent speaker verification applications because of their scalability, good performance and their ability in handling variable size sequences. On the other hand, because of their discriminative properties, models like Support Vector Machines (SVMs) usually yield better performance in static classification problems and can construct flexible decision boundaries. In this paper, we try to combine these two complementary models by using Support Vector Machines to postprocess scores obtained by the GMMs. A cross-validation method is also used in the baseline system to increase the number of client scores in the training phase, which enhances the results of the SVM models. Experiments carried out on the XM2VTS and PolyVar databases confirm the interest of this hybrid approach.
Face Verification Using Synthesized Non-Frontal Models

{THIS} {REPORT} {HAS} {BEEN} {SUPERSEDED} {BY} {IDIAP-RR} 04-04. { this report we address the problem of non-frontal face verification when only a frontal training image is available (e.g. a passport photograph) by augmenting a client's frontal face model with artificially synthesized models for non-frontal views. In the framework of a {G}aussian {M}ixture {M}odel ({GMM}) based classifier, two techniques are proposed for the synthesis: {UBM}diff and {L}in{R}eg. {B}oth techniques rely on prior information and learn how face models for the frontal view are related to face models at a non-frontal view. {T}he synthesis and augmentation approach is evaluated by applying it to two face verification systems: { ({PCA}) based and {DCT}mod2 based; the two systems are a representation of holistic and non-holistic approaches, respectively. {R}esults from experiments on the {FERET} database suggest that in almost all cases, frontal model augmentation has beneficial effects for both systems; they also suggest that the {L}in{R}eg technique (which is based on multivariate regression of classifier parameters) is more suited to the {PCA} based system and that the {UBM}diff technique (which is based on differences between two general face models) is more suited to the {DCT}mod2 based system. The results also support the view that the standard {DCT}mod2/{GMM} system (trained on frontal faces) is less affected by out-of-plane rotations than the corresponding {PCA}/{GMM} system; moreover, the {DCT}mod2/{GMM} system using augmented models is, in almost all cases, more robust than the corresponding {PCA}/{GMM} system.
Meeting Data Collection Specifications

This document specifies a process for collecting a new corpus of meetings in the IDIAP Smart Meeting Room. This document is a working draft that is expected to be updated and augmented throughout the data collection process. This follows from an earlier data collection effort that resulted in a corpus of 60 scripted meetings (30 train, 30 test), each of 5 minutes duration (now available at \textsf{mmm.idiap.ch}). The current data collection effort aims to address some of the limitations of the previous corpus, as well as to cater for a richer variety of research tasks.
Improving face authentication using virtual samples

We present a simple yet effective way of improving a face verification system by generating multiple virtual samples from the unique image corresponding to an access request. These images are generated using simple geometric transformations. This method is often used during training to improve the accuracy of a neural network model by making it robust against minor translation, scale and orientation changes. Our main contribution is to introduce such a method during testing. By generating N images from one single image and propagating them to a trained network model, one obtains N scores. By merging these scores using a simple mean operator, we show that the variance of merged scores is decreased by a factor between 1 and N. An experiment is carried out on the XM2VTS database which achieves new state-of-the-art performances.