nils-herrmann/clean_scirepeval huggingface dataset:
Kalman-filter based join cost for unit-selection speech synthesis

We introduce a new method for computing join cost in unitselection speech synthesis which uses a linear dynamical model (also known as a Kalman filter) to model line spectral frequency trajectories. The model uses an underlying subspace in which it makes smooth, continuous trajectories. This subspace can be seen as an analogy for underlying articulator movement. Once trained, the model can be used to measure how well concatenated speech segments join together. The objective join cost is based on the error between model predictions and actual observations. We report correlations between this measure and mean listener scores obtained from a perceptual listening experiment. Our experiments use a state-of-the art unit-selection text-to-speech system: rVoice from Rhetorical Systems Ltd.
Clustering Linear Dynamic Models for the use of context-dependent models on Speech Recognition

Linear Dynamic Models (LDM) have already been shown to be able to model speech and give high performance on phone recognition tasks. In this project, an approach of using LDMs with triphones as context-dependent models is presented. These contextdependent phones allow the models to deal with coarticulation. Since the number of possible triphones is very large, clustering is performed to determine which models can share parameters. The algorithm presented uses a binary decision tree where decisions are based on phonetic questions. An alternative approach to the likelihood calculation is presented for driving the clustering, and some distance measures are presented and analysed. For dealing with the noise distributions and the initial state, Bhattacharyya distance and Kullback-Lieber divergence are proposed, and for the matrices some definitions are discussed based on the Singular Value Decomposition and a Polar Decomposition of the matrices. Experiments have been performed in order to evaluate the performance of the proposed system. The task chosen was a phone classification on the TIMIT database. The best system on the TIMIT standard test set and without language model give an accuracy of 59.43%. The baseline with monophones was 56.94% which means and absolute improvemenet of 2.49%. This performanced was achieved by clustering the observation noise of the models using Bhattacharyya distance.
Discriminative methods for improving named entity extraction on speech data

In this paper we present a method of discriminatively training language models for spoken language understanding; we show improvements in named entity F-scores on speech data using these improved language models. A comparison between theoretical probabilities associated with manual markup and the actual probabilities of output markup is used to identify probabilities requiring adjustment. We present results which support our hypothesis that improvements in F-scores are possible by using either previously used training data or held out development data to improve discrimination amongst a set of N-gram language models.
Estimation of voice source and vocal tract characteristics based on multi-frame analysis

This paper presents a new approach for estimating voice source and vocal tract filter characteristics of voiced speech. When it is required to know the transfer function of a system in signal processing, the input and output of the system are experimentally observed and used to calculate the function. However, in the case of source-filter separation we deal with in this paper, only the output (speech) is observed and the characteristics of the system (vocal tract) and the input (voice source) must simultaneously be estimated. Hence the estimate becomes extremely difficult, and it is usually solved approximately using oversimplified models. We demonstrate that these characteristics are separable under the assumption that they are independently controlled by different factors. The separation is realised using an iterative approximation along with the Multi-frame Analysis method, which we have proposed to find spectral envelopes of voiced speech with minimum interference of the harmonic structure.
Estimating the spectral envelope of voiced speech using multi-frame analysis

This paper proposes a novel approach for estimating the spectral envelope of voiced speech independently of its harmonic structure. Because of the quasi-periodicity of voiced speech, its spectrum indicates harmonic structure and only has energy at frequencies corresponding to integral multiples of .I t is hence impossible to identify transfer characteristics between the adjacent harmonics. In order to resolve this problem, Multiframe Analysis (MFA) is introduced. The MFA estimates a spectral envelope using many portions of speech which are vocalised using the same vocal-tract shape. Since each of the portions usually has a different and ensuing different harmonic structure, a number of harmonics can be obtained at various frequencies to form a spectral envelope. The method thereby gives a closer approximation to the vocal-tract transfer function.
Transforming voice quality

Voice transformation is the process of transforming the characteristics of speech uttered by a source speaker, such that a listener would believe the speech was uttered by a target speaker. In this paper we address the problem of transforming voice quality. We do not attempt to transform prosody. Our system has two main parts corresponding to the two components of the source-filter model of speech production. The first component transforms the spectral envelope as represented by a linear prediction model. The transformation is achieved using a Gaussian mixture model, which is trained on aligned speech from source and target speakers. The second part of the system predicts the spectral detail from the transformed linear prediction coefficients. A novel approach is proposed, which is based on a classifier and residual codebooks. On the basis of a number of performance metrics it outperforms existing systems.
The Keyword Lexicon - An accent-independent lexicon for automatic speech recognition

Recent work at the Centre for Speech Technology Research (CSTR) at the University of Edinburgh has developed an accent-independent lexicon for speech synthesis (the Unisyn project). The main purpose of this lexicon is to avoid the problems and cost of writing a new lexicon for every new accent needed for synthesis. Only recently , a first attempt has been made to use the Keyword Lexicon for automatic speech recognition.
Named entity extraction from word lattices

We present a method for named entity extraction from word lattices produced by a speech recogniser. Previous work by others on named entity extraction from speech has used either a manual transcript or 1-best recogniser output. We describe how a single Viterbi search can recover both the named entity sequence and the corresponding word sequence from a word lattice, and further that it is possible to trade off an increase in word error rate for improved named entity extraction.
New objective distance measures for spectral discontinuities in concatenative speech synthesis

The quality of unit selection based concatenative speech synthesis mainly depends on how well two successive units can be joined together to minimise the audible discontinuities. The objective measure of discontinuity used when selecting units is known as the join cost. The ideal join cost measures perceived discontinuity, based on easily measurable spectral properties of the units being joined, in order to ensure smooth and natural-sounding synthetic speech. In this paper we describe a perceptual experiment conducted to measure the correlation between subjective human perception and various objective spectrally-based measures proposed in the literature. Also we report new objective distance measures derived from various distance metrics based on these spectral features, which have good correlation with human perception to concatenation discontinuities. Our experiments used a state-of-the art unit-selection text-to-speech system: rVoice from Rhetorical Systems Limited.
Framewise phone classification using support vector machines

We describe the use of SupportVector Machines for phonetic classification on the TIMIT corpus. Unlike previous work, in which entire phonemes are classified, our system operates in a framewise manner and is intended for use as the front-end of a hybrid system similar toABBOT. We therefore avoid the problems of classifying variable-length vectors. Our framelevel phone classification accuracy on the complete TIMIT test set is competitive with other results from the literature. In addition, we address the serious problem of scaling Support Vector Machines by using the Kernel Fisher Discriminant.
Speech recognition in the articulatory domain: investigating an alternative to acoustic HMMs

The hidden Markov model (HMM) has proven to be the model which made large-vocabulary automatic speech recognition (ASR) possible. The HMM is robust, versatile and has at its disposal a host of efficient algorithms to deal with training, speaker adaptation and recognition. However there is nothing uniquely speech orientated about the HMM. In fact, certain assumptions are made of speech which are known to be untrue. For example, speech is modelled as a piecewise stationary process when we know it to be continuous. Also co-articulation, which should be a rich source of information, simply provides unwanted variation. This variation is generally taken into account by modelling every phone in every context which in turn leads to problems of data sparcity, making elaborate parameter tying schemes necessary.
ASR - articulatory speech recognition

The hidden Markov model (HMM) has proven to be the model which has made large-vocabulary automatic speech recognition (ASR) possible. The HMM is robust, versatile and has at its disposal a host of efficient algorithms which deal with training, speaker adaptation and recognition. However, there is nothing uniquely speech orientated about the HMM. In fact, certain assumptions are made of speech which are known to be untrue. For example, speech is modelled as a piecewise stationary process when we know it to be continuous. Also, co-articulation, which should be a rich source of information, simply provides unwanted variation. This variation is generally taken into account by modelling every phone in every context which in turn leads to problems of data sparcity, making elaborate parameter tying schemes necessary. Speech is generally modelled in a parametrised version of the acoustic domain, which is natural given that this is the data we have most ready access to. Any practical speech recogniser must of course take acoustic waveforms as input, however to take these in isolation from the production mechanism which created them ignores a rich source of prior knowledge. We propose that modelling speech in the articulatory domain using linear dynamic models (see section 4) will address some of these issues. The data here consists of trajectories which evolve smoothly over time, namely coordinates of points on the articulators. Effects such as coarticulation and assimilation are most simply described in articulatory terms, as opposed to in acoustic terms where they are confounded with the representation. Models that work in the articulatory domain are therefore able to explicitly model these phenomena. We have access to real articulatory data, collected by Alan Wrench at Queen Margaret College, Edinburgh (see  for further details). This has been used to train neural networks to recover articulatory traces from the acoustics. In our experiments we have used both real and automatically recovered articulation.
Detection of phonological features in continuous speech using neural networks

We report work on the first component of a two-stage speech recognition architecture based onphonological features rather than phones. This paper reports experiments on three phonological feature systems: (1) the Sound Pattern of English (SPE) system which uses binary features, (2) amulti-valued (MV) feature system which uses traditional phonetic categories such as manner, place, etc., and (3)Government Phonology (GP) which uses a set of structured primes. All experiments used recurrent neural networks to perform feature detection. In these networks the input layer is a standard framewise cepstral representation, and the output layer represents the values of the features. The system effectively produces a representation of the most likely phonological features for each input frame. All experiments were carried out on the TIMIT speaker-independent database. The networks performed well in all cases, with the average accuracy for a single feature ranging from 86% and 93%. We describe these experiments in detail, and discuss the justification and potential advantages of using phonological features rather than phones for the basis of speech recognition.
An automatic speech recognition system using neural networks and linear dynamic models to recover and model articulatory traces

We describe a speech recognition system which uses articulatory parameters as basic features and phone-dependent linear dynamic models. The system first estimates articulatory trajectories from the speech signal. Estimations of x and y coordinates of 7 actual articulator positions in the midsagittal plane are produced every 2 milliseconds by a recurrent neural network, trained on real articulatory data. The output of this network is then passed to a set of linear dynamic models, which perform phone recognition.
Dynamical system modelling of articulator movement.

We describe the modelling of articulatory movements using (hidden) dynamical system models trained on Electro-Magnetic Articulograph (EMA) data. These models can be used for automatic speech recognition and to give insights into articulatory behaviour. They belong to a class of continuous-state Markov models, which we believe can offer improved performance over conventional Hidden Markov Models (HMMs) by better accounting for the continuous nature of the underlying speech production process - that is, the movements of the articulators. To assess the performance of our models, a simple speech recognition task was used, on which the models show promising results.
Intonation and Dialog Context as Constraints for Speech Recognition

This paper describes a way of using intonation and dialog context to improve the performance of an automatic speech recognition (ASR) system. Our experiments were run on the DCIEM Maptask corpus, a corpus of spontaneous task-oriented dialog speech. This corpus has been tagged according to a dialog analysis scheme that assigns each utterance to one of 12 “move types,” such as “acknowledge,” “query-yes/no” or “instruct.” Most ASR systems use a bigram language model to constrain the possible sequences of words that might be recognized. Here we use a separate bigram language model for each move type. We show that when the “correct” move-specific language model is used for each utterance in the test set, the word error rate of the recognizer drops. Of course when the recognizer is run on previously unseen data, it cannot know in advance what move type the speaker has just produced. To determine the move type we use an intonation model combined with a dialog model that puts constraints on possible sequences of move types, as well as the speech recognizer likelihoods for the different move-specific models. In the full recognition system, the combination of automatic move type recognition with the move specific language models reduces the overall word error rate by a small but significant amount when compared with a baseline system that does not take intonation or dialog acts into account. Interestingly, the word error improvement is restricted to “initiating” move types, where word recognition is important. In “response” move types, where the important information is conveyed by the move type itself—for example, positive versus negative response—there is no word error improvement, but recognition of the response types themselves is good. The paper discusses the intonation model, the language models, and the dialog model in detail and describes the architecture in which they are combined.
Speech recognition via phonetically featured syllables

Speech can be naturally described by phonetic features, such as a set of acoustic phonetic features or a set of articulatory features. This thesis establi shes the effectiveness of using phonetic features in phoneme recognition by comparing a recogniser based on them to a recogniser using an established parametrisation as a baseline. The usefulness of phonetic features serves as the foundation for the subsequent modelling of syllables. Syllables are subject to fewer of the context-sensitivity effects that hamper phone-based speech recognition. I investigate the different questions involved in creating syllable models. After training a feature-based syllable recogniser, I compare the feature based syllables against a baseline. To conclude, the feature based syllable models are compared against the baseline phoneme models in word recognition. With the resultant feature-syllable models performing well in word recognition, the featuresyllables show their future potential for large vocabulary automatic speech recognition.
Using information above the word level for automatic speech recognition

This thesis introduces a general method for using information at the utterance level and across utterances for automatic speech recognition. The method involves classi cation of utterances into types. Using constraints at the utterance level via this classi cation method allows information sources to be exploited which cannot necessarily be used directly for word recognition. The classi cation power of three sources of information is investigated: the language model in the speech recogniser, dialogue context and intonation. The method is applied to a challenging task: the recognition of spontaneous dialogue speech. The results show success in automatic utterance type classi cation, and subsequent word error rate reduction over a baseline system, when all three information sources are probabilistically combined.
Speech synthesis using non-uniform units in the Verbmobil project

IN THE VERBMOBIL PROJECT Simon Kingy Thomas Portele Florian H ofer Institut f ur Kommunikationsforschung und Phonetik (IKP), Universit at Bonn Poppelsdorfer Allee 47, D-53115 Bonn, Germany http://www.ikp.uni-bonn.de ynow at the Centre for Speech Technology Research, University of Edinburgh, 80, South Bridge, Edinburgh EH1 1HN, GB http://www.cstr.ed.ac.uk email: Simon.King@ed.ac.uk ABSTRACT We describe a concatenative speech synthesiser for British English which uses the HADIFIX  inventory structure originally developed for German by Portele. An inventory of non-uniform units was investigated with the aimof improving segmental quality compared to diphones. A combination of soft (diphone) and hard concatenation was used, which allowed a dramatic reduction in inventory size. We also present a unit selection algorithm which selects an optimum sequence of units from this inventory for a given phoneme sequence. The work described is part of the concept-to-speech synthesiser for the language and speech project Verbmobil  which is funded by the German Ministry of Science (BMBF).
Using intonation to constrain language models in speech recognition

This paper describes a method for using intonation to reduce word error rate in a speech recognition system designed to recognise spontaneous dialogue speech. We use a form of dialogue analysis based on the theory of conversational games. Different move types under this analysis conform to different language models. Different move types are also characterised by different intonational tunes. Our overall recognition strategy is first to predict from intonation the type of game move that a test utterance represents, and then to use a bigram language model for that type of move during recognition.
Using prosodic information to constrain language models for spoken dialogue

We present work intended to improve speech recognition performance for computer dialogue by taking into account the way that dialogue context and intonational tune interact to limit the possibilities for what an utterance might be. We report on the extra constraint achieved in a bigram language model, expressed in terms of entropy, by using separate submodels for different sorts of dialogue acts, and trying to predict which submodel to apply by analysis of the intonation of the sentence being recognised.
Finding intonational boundaries using acoustic cues related to the voice source.

Acoustic cues related to the voice source, including harmonic structure and spectral tilt, were examined for relevance to prosodic boundary detection. The measurements considered here comprise five categories: duration, pitch, harmonic structure, spectral tilt, and amplitude. Distributions of the measurements and statistical analysis show that the measurements may be used to differentiate between prosodic categories. Detection experiments on the Boston University Radio Speech Corpus show equal error detection rates around 70% for accent and boundary detection, using only the acoustic measurements described, without any lexical or syntactic information. Further investigation of the detection results shows that duration and amplitude measurements, and, to a lesser degree, pitch measurements, are useful for detecting accents, while all voice source measurements except pitch measurements are useful for boundary detection.
A Hybrid Model for Spontaneous Speech Understanding

This paper presents a hybrid model to understand spontaneous speech by combination of speech connotation and denotation analyses. The testbed of the approach is an intelligent tutoring system (ITS) collected by Wizard-of-Oz (WoZ) simulations. The children users are inexperienced and their utterances are often dysfluent and have loose grammar structure. To robustly understand spontaneous speech in the tutorial environment, we categorize the user utterances into 30 tutoring events, which can reflect the content meaning of utterances in a broad and shallow way. The objective of this study is to classify utterances into the target events. The tutoring event classification integrates speech connotation analysis and speech denotation analysis. The speech connotation analysis intends to model the cognitive states of students by three classes: confidence, puzzlement, and hesitation. The speech denotation analysis intends to compute the event-utterance similarity based on the TF•IDF vector of some pragmatically and semantically salient words embedded in the utterances. We define salient words by those words that contain novel information neither presupposed by the interlocutor nor denoted in the precedent part of the utterance. We used speech and transcribed text for experiments, and achieved 75.5% accuracy when the salient words were manually annotated. The accuracy reduced by 15.4% relative when the salient words were automatically extracted.
Distinctive feature based SVM discriminant features for improvements to phone recognition on telephone band speech

Support vector machines (SVM’s) can be trained to classify manner transitions between phones and to identify the place of articulation of any given phone with high accuracy. The discriminant outputs of these SVM’s can be used as input features for a standard ASR system. There is a signiﬁcant improvement in correctness and accuracy using these SVM discriminant features when compared to an MFCC based recognizer of equal parameters.