nils-herrmann/clean_scirepeval huggingface dataset:
Semantic Tagging using a Probabilistic Context Free Grammar

This paper describes a statistical model for extraction of events at the sentence level, or "semantic tagging", typically the first level of processing in Information Extraction systems. We illustrate the approach using a management succession task, tagging sentences with three slots involved in each succession event: the post, person coming into the post, and person leaving the post. The approach requires very limited resources: a part-of-speech tagger; a morphological analyzer; and a set of training examples that have been labeled with the three slots and the indicator (verb or noun) used to express the event. Training on 560 sentences, and testing on 356 sentences, shows the accuracy of the approach is 77.5% (if partial slot matches are deemed incorrect) or 87.8% (if partial slot matches are deemed correct).
Three Generative, Lexicalised Models for Statistical Parsing

In this paper we first propose a new statistical parsing model, which is a generative model of lexicalised context-free grammar. We then extend the model to include a probabilistic treatment of both subcategorisation and wh-movement. Results on Wall Street Journal text show that the parser performs at 88.1/87.5% constituent precision/recall, an average improvement of 2.3% over (Collins 96).
A New Statistical Parser Based on Bigram Lexical Dependencies

This paper describes a new statistical parser which is based on probabilities of dependencies between head-words in the parse tree. Standard bigram probability estimation techniques are extended to calculate probabilities of dependencies between pairs of words. Tests using Wall Street Journal data show that the method performs at least as well as SPATTER (Magerman 95; Jelinek et al. 94), which has the best published results for a statistical parser on this task. The simplicity of the approach means the model trains on 40,000 sentences in under 15 minutes. With a beam search strategy parsing speed can be improved to over 200 sentences a minute with negligible loss in accuracy.
University of Pennsylvania: Description of the University of Pennsylvania System Used for MUC-6

Breck Baldwin and Jeff Reynar informally began the University of Pennsylvania's MUC-6 coreference effort in January of 1995. For the first few months, tools were built and the system was extended at weekly 'hack sessions.' As more people began attending these meetings and contributing to the project, it grew to include eight graduate students. While the effort was still informal, Mark Wasson, from Lexis-Nexis, became an advisor to the project. In July, the students proposed to the faculty that we formally participate in the coreference task. By that time, we had developed some of the system's infrastructure and had implemented a simplistic coreference resolution system which resolved proper nouns by means of string matching. After much convincing, the faculty agreed at the end of July that we could formally participate in MUC-6. We then began an intensive effort with full-time participation from Baldwin and Reynar, and part-time efforts from the other authors. In August we were given permission from Yael Ravin of IBM's Information Retrieval group to use the IBM Name Extraction Module . We were also given access to a large acronym dictionary which Peter Flynn maintains for a world wide web site in Iceland (http://curia.ucc.ie/info/net/acronyms/acro.html).
Prepositional Phrase Attachment through a Backed-off Model

Recent work has considered corpus-based or statistical approaches to the problem of prepositional phrase attachment ambiguity. Typically, ambiguous verb phrases of the form v np1 p np2 are resolved through a model which considers values of the four head words (v, n1, p and n2). This paper shows that the problem is analogous to n-gram language models in speech recognition, and that one of the most common methods for language modeling, the backed-off estimate, is applicable. Results on Wall Street Journal data of 84.5% accuracy are obtained using this method. A surprising result is the importance of low-count events — ignoring events which occur less than 5 times in training data reduces performance to 81.6%.
Development of a conversational telephone speech recognizer for Levantine Arabic

Manylanguages,includingArabic, are characterizedbyawidevariety of different dialectsthat often differ strongly from eachother. When developing speech technology for dialect-rich languages, the portability andreusability of data,algorithms, andsystemcomponents becomes extremely important. In this paper, we describe the development of a large-vocabulary speech recognition system for Levantine Arabic, which was a new dialectal recognition task for our existing system. We discuss the dialect-specific modeling choices (grapheme vs. phoneme based acoustic models, automatic vowelization techniques, and morphological languagemodels) and investigate to what extent techniquespreviously tested on other languagesare portable to the present task. We present stateof-the-art recognition results on the 2004 Levantine Arabic Rich Transcription evaluation.
Automatic Diacritization of Arabic for Acoustic Modeling in Speech Recognition

Automatic recognition of Arabic dialectal speech is a challenging task because Arabic dialects are essentially spoken varieties. Only few dialectal resources are available to date; moreover, most available acoustic data collections are transcribed without diacritics. Such a transcription omits essential pronunciation information about a word, such as short vowels. In this paper we investigate various procedures that enable us to use such training data by automatically inserting the missing diacritics into the transcription. These procedures use acoustic information in combination with different levels of morphological and contextual constraints. We evaluate their performance against manually diacritized transcriptions. In addition, we demonstrate the effect of their accuracy on the recognition performance of acoustic models trained on automatically diacritized training data.
An efficient repair procedure for quick transcriptions

We describe an efficient procedure for automatic repair of quickly transcribed (QT) speech. QT speech, typically closed captioned data from television broadcasts, usually has a significant number of deletions and misspellings, and has a characteristic absence of disfluencies such as filled pauses (for example, um, uh). Errors of these kinds often throw an acoustic model training program out of alignment and make it hard for it to resynchronize. At best the erroneous utterance is discarded and does not benefit the training procedure. At worst, it could misalign and end up sabotaging the training data. The procedure we propose in this paper aims to cleanse such quick transcriptions so that they align better with the acoustic evidence and thus provide for better acoustic models for automatic speech recognition (ASR). Results from comparing our transcripts with those from careful transcriptions on the same corpus, and from comparable state-of-the-art methods are also presented and discussed.
Voicing feature integration in SRI's decipher LVCSR system

We augment the Mel cepstral (MFCC) feature representation with voicing features from an independent front end. The voicing feature front end parameters are optimized for recognition accuracy. The voicing features computed are the normalized autocorrelation peak and a newly proposed entropy of the high-order cepstrum. We explored several alternatives to integrate the voicing features into SRI's DECIPHER system. Promising early results were obtained in a simple system concatenating the voicing features with MFCC features and optimizing the voicing feature window duration. Best results overall came from a more complex system combining a multiframe voicing feature window with the MFCC plus third differential features using linear discriminant analysis and optimizing the number of voicing feature frames. The best integration approach from the single-pass system experiments was implemented in a multi-pass system for large vocabulary testing on the Switchboard database. An average WER reduction of 2% relative was obtained on the NIST Hub-5 dev2001 and eval2002 databases.
Morphology-based language modeling for arabic speech recognition

Language modeling is a difficult problem for languages with rich morphology. In this paper we investigate the use of morphology-based language models at different stages in a speech recognition system for conversational Arabic. Class-based and single-stream factored language models using morphological word representations are applied within an N-best list rescoring framework. In addition, we explore the use of factored language models in first-pass recognition, which is facilitated by two novel procedures: the data-driven optimization of a multi-stream language model structure, and the conversion of a factored language model to a standard word-based model. We evaluate these techniques on a large-vocabulary recognition task and demonstrate that they lead to perplexity and word error rate reductions.
SRI'S 2004 BROADCAST NEWS SPEECH TO TEXT SYSTEM

We describe the system that we used to automatically transcribeBroadcastNewsSpeechin the2004NIST evaluationssponsored by the DARPA EARS program. We also describe some of our efforts leading up to the NIST evaluation, in particular, concerning data selection and model training. Since one of the hallmarksofthe 2004evaluationwasthe availabilityof relatively large amounts of quickly transcribed data, we describe our techniques, and those of others in adapting this data for model training, and experiments attempting to evaluate the suitability of this data for training purposes. We concludeby presenting results and also discussing whether or not increases in amounts of training data proportionately benefit the research community.
Cross-dialectal acoustic data sharing for Arabic speech recognition

The automatic recognition of Arabic dialectal speech is a challenging task since Arabic dialects are essentially spoken varieties, for which only sparse resources (transcriptions and standardized acoustic data) are available to date. In this paper we describe the use of acoustic data from modern standard Arabic (MSA) to improve the recognition of Egyptian conversational Arabic (ECA). The cross-dialectal use of data is complicated by the fact that MSA is written without short vowels and other diacritics and thus has incomplete phonetic information. This problem is addressed by automatically vowelizing MSA data before combining it with ECA data. We described the vowelization procedure as well as speech recognition experiments and show that our technique yields improvements over our baseline system.
Development of phrase translation systems for handheld computers: from concept to field

We describe the development and conceptual evolution of handheld spoken phrase translation systems, beginning with an initial unidirectional system for translation of English phrases, and later extending to a limited bidirectional phrase translation system between English and Pashto, a major language of Afghanistan. We review the challenges posed by such projects, such as the constraints imposed by the computational platform, to the limitations of the phrase translation approach when dealing with naive respondents. We discuss our proposed solutions and their experimental evaluation.
Prosodic knowledge sources for automatic speech recognition

In this work, different prosodic knowledge sources are integrated into a state-of-the-art large vocabulary speech recognition system. Prosody manifests itself on different levels in the speech signal: within the words as a change in phone durations and pitch, in between the words as a variation in the pause length, and beyond the words, correlating with higher linguistic structures and nonlexical phenomena. We investigate three models, each exploiting a different level of prosodic information, in rescoring N-best hypotheses according to how well recognized words correspond to prosodic features of the utterance. Experiments on the Switchboard corpus show word accuracy improvements with each prosodic knowledge source. A further improvement is observed with the combination of all models, demonstrating that they each capture somewhat different prosodic characteristics of the speech signal.
Novel approaches to Arabic speech recognition: report from the 2002 Johns-Hopkins Summer Workshop

Although Arabic is currently one of the most widely spoken languages in the world, there has been relatively little speech recognition research on Arabic compared to other languages. Moreover, most previous work has concentrated on the recognition of formal rather than dialectal Arabic. This paper reports on our project at the 2002 Johns Hopkins Summer Workshop, which focused on the recognition of dialectal Arabic. Three problems were addressed: (a) the lack of short vowels and other pronunciation information in Arabic texts; (b) the morphological complexity of Arabic; and (c) the discrepancies between dialectal and formal Arabic. We present novel approaches to automatic vowel restoration, morphology-based language modeling and the integration of out-of-corpus language model data, and report significant word error rate improvements on the LDC Arabic CallHome task.
Large-vocabulary audio-visual speech recognition: a summary of the Johns Hopkins Summer 2000 Workshop

We report a summary of the Johns Hopkins Summer 2000 Workshop on audio-visual automatic speech recognition (ASR) in the large-vocabulary, continuous speech domain. Two problems of audio-visual ASR were mainly addressed: visual feature extraction and audio-visual information fusion. First, image transform and model-based visual features were considered, obtained by means of the discrete cosine transform (DCT) and active appearance models, respectively. The former were demonstrated to yield superior automatic speech reading. Subsequently, a number of feature fusion and decision fusion techniques for combining the DCT visual features with traditional acoustic ones were implemented and compared. Hierarchical discriminant feature fusion and asynchronous decision fusion by means of the multi-stream hidden Markov model consistently improved ASR for both clean and noisy speech. Compared to an equivalent audio-only recognizer, introducing the visual modality reduced ASR word error rate by 7% relative in clean speech, and by 27% relative at an 8.5 dB SNR audio condition.
Integration of multiple knowledge sources in speech recognition using minimum error training

Modern automatic speech recognition systems employ two statistical models: a language model and an acoustic model. These two models are estimated independently of each other, most often using a maximum likelihood criterion. They are then combined to compute the a posteriori probability of a word sequence given an acoustic signal. During combination, a static tunable parameter is used to scale the score of one of the models relative to the other. 
In this work a general formulation is presented for combining scores from several models—knowledge sources—into a single log-linear model to compute sentence probabilities. The parameters of the new model are the weights of the log-linear combination. The combination can be performed either statically, with constant weights, as is the case in the traditional way the acoustic and language models are combined, or dynamically, where the parameters may vary for different segments of a hypothesis. In the dynamic combination the weights aim to capture the dynamic change of confidence on each of the models combined. In order to achieve robust estimation of the parameters, each segment is automatically assigned to one of a small number of categories, or classes, and a single set of parameters is used for each segment class. Different techniques are described to estimate the parameters in order to achieve minimum word error rate. 
Three applications of this approach are presented: (1) The combination of several acoustic models trained using speech from resource-rich languages in order to obtain a recognition system for a language with little acoustic training data. The segments for which a set of parameters is defined correspond to hypothesized phones and the classes for the dynamic combination are chosen using phonological knowledge. (2) The dynamic combination of the baseline acoustic and language models. Different ways are investigated for clustering word links in a recognition lattice, and the language model weight and insertion penalty parameters are estimated for each cluster. Features previously used to predict confidence (correctness) of the recognized words are utilized here to define the link clusters. This way of dynamically modifying the language model weight may be interpreted as acoustic sensitive language modeling. (3) The integration in the model of side-information, available during a first pass recognition. The new model is used to rescore the hypotheses.
Towards language independent acoustic modeling

We describe procedures and experimental results using speech from diverse source languages to build an ASR system for a single target language. This work is intended to improve ASR in languages for which large amounts of training data are not available. We have developed both knowledge-based and automatic methods to map phonetic units from the source languages to the target language. We employed HMM adaptation techniques and discriminative model combination to combine acoustic models from the individual source languages for recognition of speech in the target language. Experiments are described in which Czech Broadcast News is transcribed using acoustic models trained from small amounts of Czech read speech augmented by English, Spanish, Russian, and Mandarin acoustic models.
Use of word level side information to improve speech recognition

Word level information obtained from the output of a speech recognizer has been used in the past to extract confidence features for the hypothesized words. This work describes a post-recognition process which treats these word-level features as independent knowledge sources and combines them in one log linear model for the posterior probability of a word sequence. This model is used for rescoring the hypotheses. The parameters of the model are optimized using a discriminative model combination approach, where a simplex optimization method, known as amoeba search, is used to minimize the non-smooth function of empirical error rate on training data. The method is evaluated on the SWITCHBOARD database. After training 20 new parameters, we obtain a significant word error rate reduction over the baseline system. A correlation measure between features and word accuracy is defined to help analyze and explain the results.
Minimum risk acoustic clustering for multilingual acoustic model combination

In this paper we describe procedures for combining multiple acoustic models, obtained using training corpora from different languages, in order to improve ASR performance in languages for which large amounts of training data are not available. We treat these models as multiple sources of information whose scores are combined in a log-linear model to compute the hypothesis likelihood. The model combination can either be performed in a static way, with constant combination weights, or in a dynamic way, with parameters that can vary for different segments of a hypothesis. The aim is to optimize the parameters so as to achieve minimum word error rate. In order to achieve robust parameter estimation in the dynamic combination case, the parameters are defined to be piecewise constant on different phonetic classes that form a partition of the space of hypothesis segments. The partition is defined, using phonological knowledge, on segments that correspond to hypothesized phones. We examine different ways to define such a partition, including an automatic approach that gives a binary tree structured partition which tries to achieve the minimum WER with the minimum number of classes.
Regret-based Utility Elicitation in Constraint-based Decision Problems

We propose new methods of preference elicitation for constraint-based optimization problems based on the use of minimax regret. Specifically, we assume a constraintbased optimization problem (e.g., product configuration) in which the objective function (e.g., consumer preferences) are unknown or imprecisely specified. Assuming a graphical utility model, we describe several elicitation strategies that require the user to answer only binary (bound) queries on the utility model parameters. While a theoretically motivated algorithm can provably reduce regret quickly (in terms of number of queries), we demonstrate that, in practice, heuristic strategies perform much better, and are able to find optimal (or near-optimal) configurations with far fewer queries.
Tangent-corrected embedding

Images and other high-dimensional data can frequently be characterized by a low dimensional manifold (e.g. one that corresponds to the degrees of freedom of the camera). Recently, nonlinear manifold learning techniques have been used to map images to points in a lower dimension space, capturing some of the dynamics of the camera or the subjects. In general, these methods do not take advantage of any prior understanding of the dynamics we might have, relying instead on local Euclidean distances that can be misleading in image space. In practice, we frequently have some prior knowledge regarding the transformations that relate images (e.g. rotation, translation, etc). We present a method for augmenting existing embedding techniques with additional information derived from known transformations, either in the form of tangent spaces that locally characterize the manifold or distances derived from reconstruction errors. The extra information is incorporated directly into the cost function of the embedding technique. The techniques we augment are largely attractive because there is a closed form solution for their cost optimization. Our approach likewise produces a closed form solution for the augmented cost function. Experiments demonstrate the effectiveness of the approach on a variety of image data.
Improved estimation for unsupervised part-of-speech tagging

We demonstrate that a simple hidden Markov model can achieve state of the art performance in unsupervised part-of-speech tagging, by improving aspects of standard Baum-Welch (EM) estimation. One improvement uses word similarities to smooth the lexical tag /spl rarr/ word probability estimates, which avoids over-fitting the lexical model. Another improvement constrains the model to preserve a specified marginal distribution over the hidden tags, which avoids over-fitting the tag /spl rarr/ tag transition model. Although using more contextual information than an HMM remains desirable, improving basic estimation still leads to significant improvements and remains a prerequisite for training more complex models.
Learning Coordination Classifiers

We present a new approach to ensemble classification that requires learning only a single base classifier. The idea is to learn a classifier that simultaneously predicts pairs of test labels--as opposed to learning multiple predictors for single test labels-- then coordinating the assignment of individual labels by propagating beliefs on a graph over the data. We argue that the approach is statistically well motivated, even for independent identically distributed (iid) data. In fact, we present experimental results that show improvements in classification accuracy over single-example classifiers, across a range of iid data sets and over a set of base classifiers. Like boosting, the technique increases representational capacity while controlling variance through a principled form of classifier combination.