nils-herrmann/clean_scirepeval huggingface dataset:
A computational model for detecting and perceiving dynamic properities of speech

We investigate many aspects of a computational framework for detecting phonetically salient dynamic properties of the speech signal from continuous speech input. As a prototypical example of this problem, we consider frameworks for the detection of the transition from closure to burst for stop consonants in continuous speech. We formulate the problem as an optimal non‐inear filtering problem. This involves the mapping of the dynamic acoustic data into a low dimension Riemannian manifold in an infinite dimensional Hilbert space. This leads to many kinds of metrics that can now be imposed on the dynamic speech signal and we explore these metrics in the context of the perceptual magnet effect. An extensive discussion of the robustness of the strategy, the nature of the errors made, the relationship to acoustic phonetis and distinctive features and to categorical perception will be conducted. Related problems that have a similar structure include nasal detection using transitions from vowels into adjacent nas...
Evolution of universal grammar.

Universal grammar specifies the mechanism of language acquisition. It determines the range of grammatical hypothesis that children entertain during language learning and the procedure they use for evaluating input sentences. How universal grammar arose is a major challenge for evolutionary biology. We present a mathematical framework for the evolutionary dynamics of grammar learning. The central result is a coherence threshold, which specifies the condition for a universal grammar to induce coherent communication within a population. We study selection of grammars within the same universal grammar and competition between different universal grammars. We calculate the condition under which natural selection favors the emergence of rule-based, generative grammars that underlie complex language.
The intera tion of stability and weakness inAdaBoostSamuel

We provide an analysis of AdaBoost within the framework of algorithmic stability. In particular, we show that AdaBoost is a stabilitypreserving operation: if the “input” (the weak learner) to AdaBoost is stable, then the “output” (the strong learner) is almost-everywhere stable. Because classifier combination schemes such as AdaBoost have greatest effect when the weak learner is weak, we discuss weakness and its implications. We also show that the notion of almost-everywhere stability is sufficient for good bounds on generalization error. These bounds hold even when the weak learner has infinite VC dimension.
AN ACTIVE LEARNING FORMULATION FOR INSTANCE SELECTION WITH APPLICATIONS TO OBJECT DETECTION

In certain real-world learning scenarios where there are enormous amounts of training data, the training process can become computationally intractable. Researchers have attempted to address this problem by performing instance selection, which is to automatically identify and preserve a sufficiently small but highly informative data sample for training. In most classical formulations of example-based learning, the learner passively receives randomly drawn training examples from which it recovers the unknown target function. Active Learning describes a different example-based learning paradigm where the learner explicitly seeks for new training examples of high utility, and can thus be viewed as a form of instance selection. This chapter presents a Bayesian formulation for active learning within a classical function approximation learning framework, and shows how one can derive precise example selection algorithms for learning some simple target function classes more accurately with less training data. We then present a real-world learning scenario on object (face) detection, and show how our active learning formulation leads to a useful instance selection heuristic for identifying and retaining high utility training data.
The evolutionary dynamics of grammar acquisition.

Grammar is the computational system of language. It is a set of rules that specifies how to construct sentences out of words. Grammar is the basis of the unlimited expressibility of human language. Children acquire the grammar of their native language without formal education simply by hearing a number of sample sentences. Children could not solve this learning task if they did not have some pre-formed expectations. In other words, children have to evaluate the sample sentences and choose one grammar out of a limited set of candidate grammars. The restricted search space and the mechanism which allows to evaluate the sample sentences is called universal grammar. Universal grammar cannot be learned; it must be in place when the learning process starts. In this paper, we design a mathematical theory that places the problem of language acquisition into an evolutionary context. We formulate equations for the population dynamics of communication and grammar learning. We ask how accurate children have to learn the grammar of their parents' language for a population of individuals to evolve and maintain a coherent grammatical system. It turns out that there is a maximum error tolerance for which a predominant grammar is stable. We calculate the maximum size of the search space that is compatible with coherent communication in a population. Thus, we specify the conditions for the evolution of universal grammar.
Perspectives from the informational complexity of learning

We discuss two seemingly disparate problems of learning from examples within the framework of statistical learning theory. The first involves real-valued function learning using neural networks and an analysis of this has two interesting aspects (1) it shows how the generalization ability of a learner is bounded both by finite data and limited representational capacity (2) it shifts attention away from asymptotics to learning with finite resources. The perspective that this yields is then brought to bear on the second problem of learning natural language grammars to articulate some issues that computational linguistics needs to deal with.
Multiple classifiers by constrained minimization

The paper describes an approach to combining multiple classifiers in order to improve classification accuracy. Since individual classifiers in the ensemble should somehow be uncorrelated to yield higher classification accuracy than a single classifier, we propose to train classifiers by minimizing the correlation between their classification errors. A simple combination strategy for three classifiers is then proposed and its achievable error rate is analyzed and compared to individual single classifier performance. The proposed approach has been evaluated on artificial data and a nasal/oral vowel classification task. Theoretical analyses and experimental results illustrate the effectiveness of the proposed approach.
Distinctive feature detection using support vector machines

An important aspect of distinctive feature based approaches to automatic speech recognition is the formulation of a framework for robust detection of these features. We discuss the application of the support vector machines (SVM) that arise when the structural risk minimization principle is applied to such feature detection problems. In particular, we describe the problem of detecting stop consonants in continuous speech and discuss an SVM framework for detecting these sounds. In this paper we use both linear and nonlinear SVMs for stop detection and present experimental results to show that they perform better than a cepstral features based hidden Markov model (HMM) system, on the same task.
Feature based representation for audio-visual speech recognition

In this paper, we consider the interaction of acoustic and visual stimuli at the subphonemic level of the distinctive feature. We argue that this provides a natural intermediate level for audio-visual integration and discuss the visual and acoustic feature detection problems that are associated with this task. 1.  USING AUDIO-VISUAL CUES While compelling psychophysical evidence exists [9] suggesting that the auditory and visual modalities are strongly linked in word and syllable recognition, it is generally unclear how to embody such audio-visual links in a coherent and insightful computational framework. In speech recognition systems that are based on HMMs, the core recognition engine (an HMM) is constrained to receive inputs that are a sequence of vectors in a finitedimensional space. As a result, most approaches using such systems end up simply concatenating (at each point in time) the video frame and audio frame into one audio-visual frame and performing recognition with such combined inputs. Alternatively, one uses essentially two HMM systems — one based entirely on video inputs and another entirely on audio inputs and combines likelihood scores from the two in a late integration strategy. In this paper, we consider the possibility of an intermediate level at which the interaction of visual and acoustic cues might occur. We argue that the distinctive feature provides a reasonable intermediate level at which the cues can be integrated prior to lexical access and describe the visual and auditory components of such a feature based strategy for audio-visual recognition. 1.1. Distinctive Features An alternative framework for speech recognition has been pursued in Niyogi et al [6, 5] that utilize the notion of distinctive features . Such a perspective has its roots in phonological theory that suggests that phonemes are not the atomic units of which syllables, words and other linguistic objects are composed but in fact are themselves decomposable into primitives called distinctive features. Each distinctive feature is conceptually a binary valued variable and shown in table 1 are some phonemes and their distinctive features. The distinctive features have typically been viewed as phonological oppositions that separate minimal pairs of confusible phonemes in a language. Thus the minimal pairs p,b, t,d etc. are separated by the feature voice with /p/ and /t/ being unvoiced /voice/ and /b/ and /d/ being voiced /+voice/. The distinctive features may also be viewed as defining a natural phonological class, e.g., labial sounds p,b,m form a class with feature value /+labial/ Feature p b t s m n u Consonantal + + + + + + Labial + + + Alveolar + + + Nasal + + Voicing + + + + Continuant + + Table 1: Phonemes and their associated distinctive feature values. Auditory Cues Visual Cues DISTINCTIVE FEATURE
Generalization bounds for function approximation from scattered noisy data

We consider the problem of approximating functions from scattered data using linear superpositions of non-linearly parameterized functions. We show how the total error (generalization error) can be decomposed into two parts: an approximation part that is due to the finite number of parameters of the approximation scheme used; and an estimation part that is due to the finite number of data available. We bound each of these two parts under certain assumptions and prove a general bound for a class of approximation schemes that include radial basis functions and multilayer perceptrons.
Investigating the Sample Complexity of Active Learning Schemes

In the classical learning framework of the previous chapter (akin to PAC) examples were randomly drawn and presented to the learner. In this chapter, we consider the possibility of a more active learner who is allowed to choose his/her own examples. Our investigations can be divided into two natural parts. The first, is in a function approximation setting, and develops an adaptive sampling strategy (equivalent to adaptive approximation) motivated from the standpoint of optimal recovery (Micchelli and Rivlin, 1976). We provide a general formulation of the problem. This can be regarded as sequential optimal recovery. We demonstrate the application of this general formulation to two special cases of functions on the real line 1) monotonically increasing functions and 2) functions with bounded derivative. An extensive investigation of the sample complexity of approximating these functions is conducted yielding both theoretical and empirical results on test functions. Our theoretical results (stated in PAC-style), along with the simulations demonstrate the superiority of our active scheme over both passive learning as well as classical optimal recovery. The second part of this chapter is in a concept learning framework and discusses the idea of ∈-focusing: a scheme where the active learner can iteratively draw examples from smaller and smaller regions of the input space thereby gaining vast improvements in sample complexity.
On the Relationship between Hyothesis Complexity, Sample Complexity and Generalization Error for Neural Networks

Feedforward networks are a class of approximation techniques that can be used to learn to perform some tasks from a finite set of examples. The question of the capability of a network to generalize from a finite training set to unseen data is clearly of crucial importance. In this chapter, we bound the generalization error of a class of Radial Basis Functions, for certain well defined function learning tasks, in terms of the number of parameters and number of examples. We show that the total generalization error is partly due to the insufficient representational capacity of the network (because of the finite size of the network being used) and partly due to insufficient information about the target function because of the finite number of samples. Prior research has looked at representational capacity or sample complexity in isolation. In the spirit of A. Barron, H. White and S. Geman we develop a framework to look at both. While the bound that we derive is specific for Radial Basis Functions, a number of observations deriving from it apply to any approximation technique. Our result also sheds light on ways to choose an appropriate network architecture for a particular problem and the kinds of problems that can be effectively solved with finite resources, i.e., with finite number of parameters and finite amounts of data.
The voicing feature for stop consonants: acoustic phonetic analyses and automatic speech recognition experiments

We examine the distinctive feature [voice] that separates the voiced from the unvoiced sounds for the case of stop consonants. We conduct acoustic-phonetic analyses on a large database and demonstrate the superior separability using a temporal measure (voice onset time; VOT) rather than spectral measures. We describe several algorithms to estimate the VOT automatically from continuous speech and compare them on a speech recognition problem to reduce error rates by as much as 53 % over a baseline HMM based system.
The Logical Problem of Language Change: A Case Study of European Portuguese

In this article we present new results of a novel computational approach to the interaction of two important cognitive-linguistic phenomena: (1) language learning; and (2) language change over time (diachronic linguistics). We exploit the insight that while language learning takes place at the individual level, language change is more properly regarded as an ensemble property that takes place at the level of populations of language learners. We show by analytical and computer simulation methods that language learning can be regarded as the driving force behind a dynamical systems account of language change. We apply this model to the specific case of historical change from Classical Portuguese to European Portuguese, demonstrating how a particular language learning model coupled with data on the differences between Classical and European Portuguese leads to specific predictions for possible language-change envelopes. The main investigative message of this paper is to show how this methodology can be applied to a specific case, that of Portuguese. The main moral underscores the individual/population difference; we show that simply because an individual will choose a particular grammar does not mean that all other grammars will be eliminated. 1. Language Change: An Apparent Paradox
Incorporating voice onset time to improve letter recognition accuracies

We consider the possibility of incorporating distinctive features into a statistically based speech recognizer. We develop a two pass strategy for recognition with a standard HMM based first pass followed by a second pass that performs an alternative analysis to extract class-specific features. For the voiced/voiceless distinction on stops for an alphabet recognition task, we show that a linguistically motivated acoustic feature exists (the VOT), provides superior separability to standard spectral measures, and can be automatically extracted from the signal to reduce error rates by 48.7% over state of the art HMM systems.
Language Learning Problems in the Principles and Parameters Framework

This chapter considers a learning problem in which the hypothesis class is a class of parameterized grammars. After a brief introduction to the “principles and parameters” framework of modern linguistic theory, we consider a specific learning problem previously analyzed in a seminal work by Gibson and Wexler (1994). With our informational-complexity point of view developed in this book, we reanalyze their learning problem. This puts particular emphasis on the sample complexity of learning, in contrast to previous research in the inductive inference, or Gold frameworks (see Osherson and Weinstein, 1986). We show how to formally characterize this problem in particular, and a class of learning problems in finite parameter spaces in general, as a Markov structure. Important new language learning results follow directly: we explicitly compute sample complexity bounds under different distributional assumptions, learning regimes, and grammatical parameterizations. Briefly, we may view this as a precise way to model the “poverty of stimulus” children face in language acquisition. Our reanalysis alters several conclusions made by Gibson and Wexler. We therefore consider this chapter as a useful application of learning-theoretic notions to natural languages, and their acquisition. Finally, we describe several directions for further research.
Incorporating prior information in machine learning by creating virtual examples

One of the key problems in supervised learning is the insufficient size of the training set. The natural way for an intelligent learner to counter this problem and successfully generalize is to exploit prior information that may be available about the domain or that can be learned from prototypical examples. We discuss the notion of using prior knowledge by creating virtual examples and thereby expanding the effective training-set size. We show that in some contexts this idea is mathematically equivalent to incorporating the prior knowledge as a regularizer, suggesting that the strategy is well motivated. The process of creating virtual examples in real-world pattern recognition tasks is highly nontrivial. We provide demonstrative examples from object recognition and speech recognition to illustrate the idea.
The informational complexity of learning - perspectives on neural networks and generative grammar

From the Publisher: 
Among other topics, The Informational Complexity of Learning: Perspectives on Neural Networks and Generative Grammar brings together two important but very different learning problems within the same analytical framework. The first concerns the problem of learning functional mappings using neural networks, followed by learning natural language grammars in the principles and parameters tradition of Chomsky. These two learning problems are seemingly very different. Neural networks are real-valued, infinite-dimensional, continuous mappings. On the other hand, grammars are boolean-valued, finite-dimensional, discrete (symbolic) mappings. Furthermore the research communities that work in the two areas almost never overlap. The book's objective is to bridge this gap. It uses the formal techniques developed in statistical learning theory and theoretical computer science over the last decade to analyze both kinds of learning problems. By asking the same question - how much information does it take to learn - of both problems, it highlights their similarities and differences. Specific results include model selection in neural networks, active learning, language learning and evolutionary models of language change.
Evolutionary Consequences of Language Learning

Linguists‘ intuitions about language change can be captured by adynamical systems model derived from the dynamics of language acquisition.Rather than having to posit a separate model for diachronic change, as hassometimes been done by drawing on assumptions from population biology (cf.Cavalli-Sforza and Feldman, 1973; 1981; Kroch, 1990), this new modeldispenses with these independent assumptions by showing how the behavior ofindividual language learners leads to emergent, global populationcharacteristics of linguistic communities over several generations. As thesimplest case, we formalize the example of two grammars and show that eventhis situation leads directly to a nonlinear (quadratic) dynamical system.We study this one parameter model in a variety of situations for differentkinds of acquisition algorithms and maturational times, showing howdifferent learning theories can have very different evolutionaryconsequences. This allows us to formulate an evolutionary criterion for theadequacy of grammatical and learning theories. An application of thecomputational model to the historical loss of Verb Second from Old French toModern French is described showing how otherwise adequate grammaticaltheories might fail the evolutionary criterion.
Comparing support vector machines with Gaussian kernels to radial basis function classifiers

The support vector (SV) machine is a novel type of learning machine, based on statistical learning theory, which contains polynomial classifiers, neural networks, and radial basis function (RBF) networks as special cases. In the RBF case, the SV algorithm automatically determines centers, weights, and threshold that minimize an upper bound on the expected test error. The present study is devoted to an experimental comparison of these machines with a classical approach, where the centers are determined by X-means clustering, and the weights are computed using error backpropagation. We consider three machines, namely, a classical RBF machine, an SV machine with Gaussian kernel, and a hybrid system with the centers determined by the SV method and the weights trained by error backpropagation. Our results show that on the United States postal service database of handwritten digits, the SV machine achieves the highest recognition accuracy, followed by the hybrid system. The SV approach is thus not only theoretically well-founded but also superior in a practical application.
Learning from triggers

In this article we provide a refined analysis of learning in finite parameter spaces using the Triggering Learning Algorithm (TLA) of Gibson and Wexler (1994). We show that the behavior of the TLA can be modeled exactly as a Markov chain. This Markov model allows us to (1) describe formally the conditions for learnability in such spaces, (2) uncover problematic states in addition to the local maxima described by Gibson and Wexler, and (3) characterize convergence times for the learning algorithms quantitatively. In addition, we present arguments questioning the psychological plausibility of the TLA as a learning algorithm