nils-herrmann/clean_scirepeval huggingface dataset:
USING UPPER BOUNDS ON ATTAINABLEDISCRIMINATION TO SELECT DISCRETEVALUED

Selection of features that will permit accurate pattern clas-siication is, in general, a diicult task. However, if a particular data set is represented by discrete valued features, it becomes possible to determine empirically the contribution that each feature makes to the discrimination between classes. We describe how to calculate the maximum discrimination possible in a two alternative forced choice decision problem, when discrete valued features are used to represent a given data set. (In this paper, we measure discrimination in terms of the area under the receiver operating characteristic (ROC) curve.) Since this bound corresponds to the upper limit of classiication achievable by any classiier (with that given data representation), we can use it to assess whether recognition errors are due to a lack of separability in the data or shortcomings in the classiication technique. In comparison to the training and testing of artiicial neural networks, the empirical bound on discrimination can be eeciently calculated , allowing an experimenter to decide whether subsequent development of neural network models is warranted. We extend the discrimination bound method so that we can estimate both the maximum and average discrimination we can expect on unseen test data. These estimation techniques are the basis of a backwards elimination algorithm that can be used to rank features in order of their discriminative power. We use two problems to demonstrate this feature selection process: classiication of the Mushroom Database, and a real-world, pregnancy related medical risk prediction task | assessment of risk of perinatal death.
Geometrical techniques for finding the embedding dimension of time series

We present a technique to determine the embedding dimension of deterministic time series. The technique is based mainly on the assumption that if a continuous deterministic time series, with first derivative and underlying model f/sub S/(x/sub t-1/, ..., x/sub t-m/)=x/sub t/ is represented on its convenient embedding dimension /spl Rfr//sup m/, then any two neighbouring input observations x/sub m//sup i/ and x/sub m//sup j/, should correspond to similar output observations x/sub t//sup i/ and x/sub  j/. Thus the ratio between the input and output distances of neighbouring observations is used as a criterion to determine the time series embedding dimension. The performance of this technique is illustrated and compared using synthetic time series (Logistic map, Henon map and Mackey Glass function), as well as the Laser data from the Santa Fe competition. Our technique applied to the Laser data, in combination with a Limited Resource Allocating Neural Network (LRAN), has proved to be as successful as the techniques proposed by the winners of the competition on the 100-step prediction of this chaotic time series.
Pruning with Replacement on Limited Resource Allocating Networks by F-Projections

The principle of F-projection, in sequential function estimation, provides a theoretical foundation for a class of gaussian radial basis function networks known as the resource allocating networks (RAN). The ad hoc rules for adaptively changing the size of RAN architectures can be justified from a geometric growth criterion defined in the function space. In this paper, we show that the same arguments can be used to arrive at a pruning with replacement rule for RAN architectures with a limited number of units. We illustrate the algorithm on the laser time series prediction problem of the Santa Fe competition and show that results similar to those of the winners of the competition can be obtained with pruning and replacement.
Sequential Tracking in Pricing Financial Options using Model Based and Neural Network Approaches

This paper shows how the prices of option contracts traded in financial markets can be tracked sequentially by means of the Extended Kalman Filter algorithm. I consider call and put option pairs with identical strike price and time of maturity as a two output nonlinear system. The Black-Scholes approach popular in Finance literature and the Radial Basis Functions neural network are used in modelling the nonlinear system generating these observations. I show how both these systems may be identified recursively using the EKF algorithm. I present results of simulations on some FTSE 100 Index options data and discuss the implications of viewing the pricing problem in this sequential manner.
What patient information allows us to make accurate predictions of outcome?

Only some of the information contained in a medical record will be useful to the prediction of patient outcome. The authors describe a novel method for selecting those outcome predictors which allow them to reliably discriminate between adverse and benign end results. Using the area under the receiver operating characteristic as a nonparametric measure of discrimination, the authors show how to calculate the maximum discrimination attainable with a given set of discrete valued features. This upper limit forms the basis of their feature selection algorithm. They use the algorithm to select features (from maternity records) relevant to the prediction of failure to progress in labour. The results of this analysis motivate investigation of those predictors of failure to progress relevant to parous and nulliparous-sub-populations.
Limits on the Discrimination Possible with Discrete Valued Data, with Application to Medical Risk Prediction Limits on the Discrimination Possible with Discrete Valued Data, with Application to Medical Risk Prediction

We describe an upper bound on the accuracy (in the ROC sense) attainable in two-alternative forced choice risk prediction, for a speciic set of data represented by discrete features. By accuracy, we mean the probability that a risk prediction system will correctly rank a randomly chosen high risk case and a randomly chosen low risk case. We also present methods for estimating the maximum accuracy we can expect to attain using a given set of discrete features to represent data sampled from a given population. These techniques allow an experimenter to calculate the maximum performance that could be achieved, without having to resort to applying speciic risk prediction methods. Furthermore, these techniques can be used to rank discrete features in order of their eeect on maximum attainable accuracy.
On the Practical Applicability of VC Dimension Bounds

This article addresses the question of whether some recent Vapnik-Chervonenkis (VC) dimension-based bounds on sample complexity can be regarded as a practical design tool. Specifically, we are interested in bounds on the sample complexity for the problem of training a pattern classifier such that we can expect it to perform valid generalization. Early results using the VC dimension, while being extremely powerful, suffered from the fact that their sample complexity predictions were rather impractical. More recent results have begun to improve the situation by attempting to take specific account of the precise algorithm used to train the classifier. We perform a series of experiments based on a task involving the classification of sets of vowel formant frequencies. The results of these experiments indicate that the more recent theories provide sample complexity predictions that are significantly more applicable in practice than those provided by earlier theories; however, we also find that the recent theories still have significant shortcomings.
Vocal tract modelling with recurrent neural networks

The speech production system is modelled using true glottal excitation as the source and a recurrent neural network to represent the vocal tract. The hidden nodes have multiple delays of one and two samples, making the network equivalent to a parallel formant synthesiser in the linear regions of the hidden node sigmoids. An ARX model identification is carried out to initialise the neural network parameters. These parameters are re-estimated in an analysis-by-synthesis framework to minimise the synthesis (output) error. Unlike other analysis-by-synthesis speech production models such as CELP, the source and filter in this approach are decoupled, enabling manipulation of the source time-scale to achieve high quality pitch changes.
Semisupervised alignment of manifolds

In this paper, we study a family of semisupervised learning algorithms for “aligning” different data sets that are characterized by the same underlying manifold. The optimizations of these algorithms are based on graphs that provide a discretized approximation to the manifold. Partial alignments of the data sets—obtained from prior knowledge of their manifold structure or from pairwise correspondences of subsets of labeled examples— are completed by integrating supervised signals with unsupervised frameworks for manifold learning. As an illustration of this semisupervised setting, we show how to learn mappings between different data sets of images that are parameterized by the same underlying modes of variability (e.g., pose and viewing angle). The curse of dimensionality in these problems is overcome by exploiting the low dimensional structure of image manifolds.
Learning Harmonic Relationships in Digital Audio with Dirichlet-Based Hidden Markov Models

Harmonic analysis is a standard musicological tool for understanding many pieces of Western classical music and making comparisons among them. Traditionally, this analysis is done on paper scores, and most past research in machine-assisted analysis has begun with digital representations of them. Human music students are also taught to hear their musical analyses, however, in both musical recordings and performances. Our approach attempts to teach machines to do the same, beginning with a corpus of recorded Mozart symphonies. The audio files are first transformed into an ordered series of normalized pitch class profile (PCP) vectors. Simplified rules of tonal harmony are encoded in a transition matrix. Classical music tends to change key more frequently than popular music, and so these rules account not only for chords, as most previous work has done, but also for the keys in which they function. A hidden Markov model (HMM) is used with this transition matrix to train Dirichlet distributions for major and minor keys on the PCP vectors. The system tracks chords and keys successfully and shows promise for a real-time implementation.
Graph-based methods for unsupervised and semi-supervised learning

In many applications of machine learning, it is cheap to collect unlabeled examples but expensive to collect labeled ones. Often, the unlabeled examples can provide useful auxiliary information by revealing that the data has a simple underlying structure (M. Belkin and P. Niyogi, 2004). Of particular interest is the case when seemingly high dimensional data can be described in terms of a small number of degrees of freedom. How can we search for low dimensional structure in high dimensional data (C. J. C. Burges, 2005)? If the data is mainly confined to a low dimensional subspace, then simple linear methods can be used to discover the subspace and estimate its dimensionality. More generally, though, if the data lies on (or near) a low dimensional manifold, then its structure may be highly nonlinear, and linear methods are bound to fail. The talk would have four aims: (i) to provide an overview of graph-based spectral methods for nonlinear dimensionality reduction; (ii) to illustrate how unlabeled examples can be used, in conjunction with labeled examples, to improve classification when the data lies on or near on a low dimensional manifold; (iii) to suggest how low dimensional manifolds may arise from particular types of acoustic feature extraction; (iv) to speculate how recent methods in unsupervised and semi-supervised learning could be generalized to problems involving sequences, as occur in speech and language processing
Distance Metric Learning for Large Margin Nearest Neighbor Classification

The accuracy of k-nearest neighbor (kNN) classification depends significantly on the metric used to compute distances between different examples. In this paper, we show how to learn a Mahalanobis distance metric for kNN classification from labeled examples. The Mahalanobis metric can equivalently be viewed as a global linear transformation of the input space that precedes kNN classification using Euclidean distances. In our approach, the metric is trained with the goal that the k-nearest neighbors always belong to the same class while examples from different classes are separated by a large margin. As in support vector machines (SVMs), the margin criterion leads to a convex optimization based on the hinge loss. Unlike learning in SVMs, however, our approach requires no modification or extension for problems in multiway (as opposed to binary) classification. In our framework, the Mahalanobis distance metric is obtained as the solution to a semidefinite program. On several data sets of varying size and difficulty, we find that metrics trained in this way lead to significant improvements in kNN classification. Sometimes these results can be further improved by clustering the training examples and learning an individual metric within each cluster. We show how to learn and combine these local metrics in a globally integrated manner.
Nonlinear Dimensionality Reduction by Semidefinite Programming and Kernel Matrix Factorization

We describe an algorithm for nonlinear dimensionality reduction based on semidefinite programming and kernel matrix factorization. The algorithm learns a kernel matrix for high dimensional data that lies on or near a low dimensional manifold. In earlier work, the kernel matrix was learned by maximizing the variance in feature space while preserving the distances and angles between nearest neighbors. In this paper, adapting recent ideas from semi-supervised learning on graphs, we show that the full kernel matrix can be very well approximated by a product of smaller matrices. Representing the kernel matrix in this way, we can reformulate the semidefinite program in terms of a much smaller submatrix of inner products between randomly chosen landmarks. The new framework leads to order-of-magnitude reductions in computation time and makes it possible to study much larger problems in manifold learning.
Visualization of low Dimensional Structure in tonal pitch Space

In his 2001 monograph Tonal Pitch Space, Fred Lerdahl defined a distance function over tonal and post-tonal harmonies distilled from years of research on music cognition. Although this work references the toroidal structure commonly associated with harmonic space, it stops short of presenting an explicit embedding of this torus. It is possible to use statistical techniques to recreate such an embedding from the distance function, yielding a more complex structure than the standard toroidal model has heretofore assumed. Nonlinear techniques can reduce the dimensionality of this structure and be tuned to emphasize global or local anatomy. The resulting manifolds highlight the relationships inherent in the tonal system and offer a basis for future work in machine-assisted analysis and music theory. Comments Presented at the International Computer Music Conference (ICMC 2005 : free sound), Paper ID 1200, held in Barcelona, September 5-9, 2005. http://www.icmc2005.org/index.php?selectedPage=119 This conference paper is available at ScholarlyCommons: http://repository.upenn.edu/cis_papers/193 VISUALIZATION OF LOW DIMENSIONAL STRUCTURE IN TONAL PITCH SPACE J. Ashley Burgoyne and Lawrence K. Saul Department of Computer and Information Science University of Pennsylvania {burgoyne,lsaul}@cis.upenn.edu
Markov Processes on Curves

We study the classification problem that arises when two variables—one continuous (x), one discrete (s)—evolve jointly in time. We suppose that the vector x traces out a smooth multidimensional curve, to each point of which the variable s attaches a discrete label. The trace of s thus partitions the curve into different segments whose boundaries occur where s changes value. We consider how to learn the mapping between the trace of x and the trace of s from examples of segmented curves. Our approach is to model the conditional random process that generates segments of constant s along the curve of x. We suppose that the variable s evolves stochastically as a function of the arc length traversed by x. Since arc length does not depend on the rate at which a curve is traversed, this gives rise to a family of Markov processes whose predictions are invariant to nonlinear warpings (or reparameterizations) of time. We show how to estimate the parameters of these models—known as Markov processes on curves (MPCs)—from labeled and unlabeled data. We then apply these models to two problems in automatic speech recognition, where x are acoustic feature trajectories and s are phonetic alignments.
Hierarchical Distributed Representations for Statistical Language Modeling

Statistical language models estimate the probability of a word occurring in a given context. The most common language models rely on a discrete enumeration of predictive contexts (e.g., n-grams) and consequently fail to capture and exploit statistical regularities across these contexts. In this paper, we show how to learn hierarchical, distributed representations of word contexts that maximize the predictive value of a statistical language model. The representations are initialized by unsupervised algorithms for linear and nonlinear dimensionality reduction , then fed as input into a hierarchical mixture of experts, where each expert is a multinomial distribution over predicted words . While the distributed representations in our model are inspired by the neural probabilistic language model of Bengio et al. , our particular architecture enables us to work with significantly larger vocabularies and training corpora. For example, on a large-scale bigram modeling task involving a sixty thousand word vocabulary and a training corpus of three million sentences, we demonstrate consistent improvement over class-based bigram models [10, 13]. We also discuss extensions of our approach to longer multiword contexts.
Modeling distances in large-scale networks by matrix factorization

In this paper, we propose a model for representing and predicting distances in large-scale networks by <i>matrix factorization</i>. The model is useful for network distance sensitive applications, such as content distribution networks, topology-aware overlays, and server selections. Our approach overcomes several limitations of previous coordinates-based mechanisms, which cannot model sub-optimal routing or asymmetric routing policies. We describe two algorithms --- singular value decomposition (SVD) and nonnegative matrix factorization (NMF)---for representing a matrix of network distances as the product of two smaller matrices. With such a representation, we build a scalable system--- <i>Internet Distance Estimation Service (IDES)</i>---that predicts large numbers of network distances from limited numbers of measurements. Extensive simulations on real-world data sets show that IDES leads to more accurate, efficient and robust predictions of latencies in large-scale networks than previous approaches.
Unsupervised learning of image manifolds by semidefinite programming

Can we detect low dimensional structure in high dimensional data sets of images and video? The problem of dimensionality reduction arises often in computer vision and pattern recognition. In this paper, we propose a new solution to this problem based on semidefinite programming. Our algorithm can be used to analyze high dimensional data that lies on or near a low dimensional manifold. It overcomes certain limitations of previous work in manifold learning, such as Isomap and locally linear embedding. We illustrate the algorithm on easily visualized examples of curves and surfaces, as well as on actual images of faces, handwritten digits, and solid objects.
Mixed Memory Markov Models: Decomposing Complex Stochastic Processes as Mixtures of Simpler Ones

We study Markov models whose state spaces arise from the Cartesian product of two or more discrete random variables. We show how to parameterize the transition matrices of these models as a convex combination—or mixture—of simpler dynamical models. The parameters in these models admit a simple probabilistic interpretation and can be fitted iteratively by an Expectation-Maximization (EM) procedure. We derive a set of generalized Baum-Welch updates for factorial hidden Markov models that make use of this parameterization. We also describe a simple iterative procedure for approximately computing the statistics of the hidden states. Throughout, we give examples where mixed memory models provide a useful representation of complex stochastic processes.
Nonnegative deconvolution for time of arrival estimation

The interaural time difference (ITD) of arrival is a primary cue for acoustic sound source localization. Traditional estimation techniques for ITD based upon cross-correlation are related to maximum-likelihood estimation of a simple generative model. We generalize the time difference estimation into a deconvolution problem with nonnegativity constraints. The resulting nonnegative least squares optimization can be efficiently solved using a novel iterative algorithm with guaranteed global convergence properties. We illustrate the utility of this algorithm using simulations and experimental results from a robot platform.
Exploratory analysis and visualization of speech and music by locally linear embedding

Many problems in voice recognition and audio processing involve feature extraction from raw waveforms. The goal of feature extraction is to reduce the dimensionality of the audio signal while preserving the informative signatures that, for example, distinguish different phonemes in speech or identify particular instruments in music. If the acoustic variability of a data set is described by a small number of continuous features, then we can imagine the data as lying on a low dimensional manifold in the high dimensional space of all possible waveforms. Locally linear embedding (LLE) is an unsupervised learning algorithm for feature extraction in this setting. In this paper, we present results from the exploratory analysis and visualization of speech and music by LLE.
A Generalized Linear Model for Principal Component Analysis of Binary Data

We investigate a generalized linear model for dimensionality reduction of binary data. The model is related to principal component analysis (PCA) in the same way that logistic regression is related to linear regression. Thus we refer to the model as logistic PCA. In this paper, we derive an alternating least squares method to estimate the basis vectors and generalized linear coefficients of the logistic PCA model. The resulting updates have a simple closed form and are guaranteed at each iteration to improve the model’s likelihood. We evaluate the performance of logistic PCA—as measured by reconstruction error rates—on data sets drawn from four real world applications. In general, we find that logistic PCA is much better suited to modeling binary data than conventional PCA.