nils-herrmann/clean_scirepeval huggingface dataset:
Learning Preference Relations for Information Retrieval

In this paper we investigate the problem of learning a preference relation i~om a given set of ranked documents. We show that the Bayes’s optimal decision function, when applied to learning a preference relation, may violate transitivity. This is undesirable for information retrieval, because it is in conflict with a document ranking based on the user’s preferences. To overcome this problem we present a vector space based method that performs a linear mapping from documeats to scalar utility values and thus guarantees transitivity. The learning of the relation between documeats is formulated as a classification problem on pairs of documents and is solved using the principle of structural risk minimization for good generalization. The approach is extended to polynomial utility functions by using the potential function method (the so called "kernel trick"), which allows to incorporate higher order correlations of features into the utility function at minimal computational costs. The resulting algorithm is tested on aa example with artificial data. The algorithm successfully learns the utility function underlying the training examples and shows good classification performance.
Neural Networks in Economics: Background, Applications and New Developments

Neural Networks – originally inspired from Neuroscience – p rovide powerful models for statistical data analysis. Their most prominent feature is their a b lity to “learn” dependencies based on a finite number of observations. In the context of Neural Ne tworks the term “learning” means that the knowledge acquired from the samples can be gen eralized to as yet unseen observations. In this sense, a Neural Network is often calle d a Learning Machine. As such, Neural Networks might be considered as a metaphor for an agen t who learns dependencies of his environment and thus infers strategies of behavior ba sed on a limited number of observations. In this contribution, however, we want to abstract from the biological origins of Neural Networks and rather present them as a purely mathematical mo del. The paper is organized as follows: In Section 2 we will summar ize the main results of Statistical Learning Theory which provide a basis for understa nding the generalization properties of existing Neural Network learning algorithms. In Section 3 we will introduce basic concepts and techniques of Neural Network Learning. Then, in Section 4, we will give an overview of existing applications of Neural Networks in Economics. R ecently, however, the ideas from Statistical Learning Theory, as introduced in Section 2, ha ve lead the way to the so called Support Vector learning, which will be described in Section 5 for the task of classification. Finally, in Section 6 we leave the review track and present a s tate–of–the–art application of this technique to the problem of learning the preference structu e from a given set of observations. In contrast to cardinal utility theory our approach is based on ordinal utilities and requires as input only objects together with their respective relation (i.e. preference ranking). Throughout the paper, we will support the theoretical arguments by exam ples.
Supervised learning of preference relations

In this paper we investigate the problem of learning a prefer ence relation from a given set of ranked objects. The main difference between preference learning a nd standard classification learning is the demand, that the mapping from objects to ranks has to be transitive and ant isymmetric. To model this problem we present an approach that performs a linear mapping from objects to scal ar utility values and thus guarantees transitivity and antisymmetry. The learning of the preference between objec ts is formulated as a classification problem on pairs of objects and is solved using the principle of structural ri sk minimization. The approach is extended to nonlinear utility functions by using the potential function method (t he so called kernel trick), which allows to incorporate higher order correlations of features into the utility func tion at minimal computational costs.
Classification on Pairwise Proximity Data

We investigate the problem of learning a classification task on data represented in terms of their pairwise proximities. This representation does not refer to an explicit feature representation of the data items and is thus more general than the standard approach of using Euclidean feature vectors, from which pairwise proximities can always be calculated. Our first approach is based on a combined linear embedding and classification procedure resulting in an extension of the Optimal Hyperplane algorithm to pseudo-Euclidean data. As an alternative we present another approach based on a linear threshold model in the proximity values themselves, which is optimized using Structural Risk Minimization. We show that prior knowledge about the problem can be incorporated by the choice of distance measures and examine different metrics W.r.t. their generalization. Finally, the algorithms are successfully applied to protein structure data and to data from the cat's cerebral cortex. They show better performance than K-nearest-neighbor classification.
Unbiased Assesment of Learning Algorithms

In order to rank the performance of machine learning algorithms, many researchers conduct experiments on benchmark data sets. Since most learning algorithms have domain-specific parameters, it is a popular custom to adapt these parameters to obtain a minimal error rate on the test set. The same rate is then used to rank the algorithm, which causes an opt imistic bias. We quantify this bias, showing, in particular, that an algorithm with more parameters wi l l probably be ranked higher than an equally good algorithm with fewer parameters. We demonstrate this result, showing the number of parameters and trials required in order to pretend to outperform C4.5 or FOIL, respectively, for various benchmark problems. We then describe out how unbiased ranking experiments should be conducted.
Unbiased assessment of learning algorithms

In order to rank the performance of machine learning algorithms, many researchers conduct experiments on benchmark data sets. Since most learning algorithms have domain-specific parameters, it is a popular custom to adapt these parameters to obtain a minimal error rate on the test set. The same rate is then used to rank the algorithm, which causes an optimistic bias. We quantify this bias, showing, in particular, that an algorithm with more parameters will probably be ranked higher than an equally good algorithm with fewer parameters. We demonstrate this result, showing the number of parameters and trials required in order to pretend to outperform C4.5 or FOIL, respectively, for various benchmark problems. We then describe out how unbiased ranking experiments should be conducted.
E cient-Subsumption based on GraphAlgorithms ?

The-subsumption problem is crucial to the eeciency of ILP learning systems. We discuss two-subsumption algorithms based on strategies for preselecting suitable matching literals. The class of clauses, for which subsumption becomes polynomial, is a superset of the deter-ministic clauses. We further map the general problem of-subsumption to a certain problem of nding a clique of xed size in a graph, and in return show that a specialization of the pruning strategy of the Car-raghan and Pardalos clique algorithm provides a dramatic reduction of the subsumption search space. We also present empirical results for the mesh design data set.
Efficient Theta-Subsumption Based on Graph Algorithms

The θ-subsumption problem is crucial to the efficiency of ILP learning systems. We discuss two θ-subsumption algorithms based on strategies for preselecting suitable matching literais. The class of clauses, for which subsumption becomes polynomial, is a superset of the deterministic clauses. We further map the general problem of θ-subsumption to a certain problem of finding a clique of fixed size in a graph, and in return show that a specialization of the pruning strategy of the Carraghan and Pardalos clique algorithm provides a dramatic reduction of the subsumption search space. We also present empirical results for the mesh design data set.
Asymptotics of Gaussian Regularized Least Squares

We consider regularized least-squares (RLS) with a Gaussian kernel. We prove that if we let the Gaussian bandwidth σ → ∞ while letting the regularization parameter λ → 0, the RLS solution tends to a polynomial whose order is controlled by the rielative rates of decay of 1/ σ1 and λ: if λ = σ-(2k+1), then, as σ → ∞, the RLS solution tends to the kth order polynomial with minimal empirical error. We illustrate the result with an example.
Towards Component-based Car Detection

Recent results in computer vision have supported the theory that object detectors built in the statistical learning framework can benefit from a two stage learning process, first learning appropriate diagnostic features for the object being trained, and subsequently training an upper-level classifier on the excitation of these part detectors. In this study we develop a hierarchical detection architecture for automobiles. The classifier operates by first locating keypoints in the test image with a well known interest operator. These keypoints are then compared against a corpus of car-specific keypoints learned from the training data. The resulting similarity vector is input into a Support Vector Machine for classification. We compare the performance of our classifier to that of well documented learning algorithms (SVMs and k-Nearest Neighbors) on two separate databases of still images of cars. Our results suggest that part-based detection architectures indeed work well for this class of objects.
In Defense of One-Vs-All Classification

We consider the problem of multiclass classification. Our main thesis is that a simple "one-vs-all" scheme is as accurate as any other approach, assuming that the underlying binary classifiers are well-tuned regularized classifiers such as support vector machines. This thesis is interesting in that it disagrees with a large body of recent published work on multiclass classification. We support our position by means of a critical review of the existing literature, a substantial collection of carefully controlled experimental work, and theoretical arguments.
Speaker recognition using local models

Many of the problems arising in speech processing are characterized by extremely large training and testing sets, constraining the kinds of models and algorithms that lead to tractable implementations. In particular, we would like the amount of processing associated with each test frame to be sublinear (i.e., logarithmic) in the number of training points. In this paper, we consider smoothed kernel regression models at each test frame, using only those training frames that are close to the desired test frame. The problem is made tractable via the use of approximate nearest neighbors techniques. The resulting system is conceptually simple, easy to implement, and fast, with performance comparable to more sophisticated methods. Preliminary results on a NIST speaker recognition task are presented, demonstrating the feasibility of the method.
Chapter 7 Regularized Least-Squares Classification

We consider the solution of binary classification problems via Tikhonov regularization in a Reproducing Kernel Hilbert Space using the square loss, and denote the resulting algorithm Regularized Least-Squares Classification (RLSC). We sketch the historical developments that led to this algorithm, and demonstrate empirically that its performance is equivalent to that of the well-known Support Vector Machine on several datasets. Whereas training an SVM requires solving a convex quadratic program, training RLSC requires only the solution of a single system of linear equations. We discuss the computational tradeoffs between RLSC and SVM, and explore the use of approximations to RLSC in situations where the full RLSC is too expensive. We also develop an elegant leaveone-out bound for RLSC that exploits the geometry of the algorithm, making a connection to recent work in algorithmic stability.
Estimating Dataset Size Requirements for Classifying DNA Microarray Data

A statistical methodology for estimating dataset size requirements for classifying microarray data using learning curves is introduced. The goal is to use existing classification results to estimate dataset size requirements for future classification experiments and to evaluate the gain in accuracy and significance of classifiers built with additional data. The method is based on fitting inverse power-law models to construct empirical learning curves. It also includes a permutation test procedure to assess the statistical significance of classification performance for a given dataset size. This procedure is applied to several molecular classification problems representing a broad spectrum of levels of complexity.
An Analytical Method for Multiclass Molecular Cancer Classification

A treating composition and method for treatment of shock and/or stress in animals. The composition comprises, in a preferred form, equal volume amounts of solutions of sodium acetate and sodium propionate. It may be administered orally, intravenously, subcutaneously, etc. The preferred dosage level is from about 0.25 cc. per pound of body weight to about 0.5 cc. per pound of body weight.
Bayesian Estimation of Transcript Levels Using a General Model of Array Measurement Noise

Gene arrays demonstrate a promising ability to characterize expression levels across the entire genome but suffer from significant levels of measurement noise. We present a rigorous new approach to estimate transcript levels and ratios from one or more gene array experiments, given a model of measurement noise and available prior information. The Bayesian estimation of array measurements (BEAM) technique provides a principled method to identify changes in expression level, combine repeated measurements, or deal with negative expression level measurements. BEAM is more flexible than existing techniques, because it does not assume a specific functional form for noise and prior models. Instead, it relies on computational techniques that apply to a broad range of models. We use Affymetrix yeast chip data to illustrate the process of developing accurate noise and prior models from existing experimental data. The resulting noise model includes novel features such as heavy-tailed additive noise and a gene-specific bias term. We also verify that the resulting noise and prior models fit data from an Affymetrix human chip set.
A bayesian approach to transcript estimation from gene array data: the BEAM technique

We present a new statistically optimal approach to estimate transcript levels and ratios from one or more gene array experiments. The Bayesian Estimation of Array Measurements (BEAM) technique uses a model of measurement noise and prior information to estimate biological expression levels. It provides a principled method to deal with negative expression level measurements, combine multiple measurements, and identify changes in expression level. BEAM is more flexible than existing techniques, because it does not assume a specific functional form for noise and prior models. Rather, it uses a more accurate noise model developed from experimental data, a process we illustrate here using Affymetrix yeast chips.
Prediction of central nervous system embryonal tumour outcome based on gene expression

Embryonal tumours of the central nervous system (CNS) represent a heterogeneous group of tumours about which little is known biologically, and whose diagnosis, on the basis of morphologic appearance alone, is controversial. Medulloblastomas, for example, are the most common malignant brain tumour of childhood, but their pathogenesis is unknown, their relationship to other embryonal CNS tumours is debated, and patients’ response to therapy is difficult to predict. We approached these problems by developing a classification system based on DNA microarray gene expression data derived from 99 patient samples. Here we demonstrate that medulloblastomas are molecularly distinct from other brain tumours including primitive neuroectodermal tumours (PNETs), atypical teratoid/rhabdoid tumours (AT/RTs) and malignant gliomas. Previously unrecognized evidence supporting the derivation of medulloblastomas from cerebellar granule cells through activation of the Sonic Hedgehog (SHH) pathway was also revealed. We show further that the clinical outcome of children with medulloblastomas is highly predictable on the basis of the gene expression profiles of their tumours at diagnosis.
Musical query-by-description as a multiclass learning problem

We present the query-by-description (QBD) component of "Kandem", a time-aware music retrieval system. The QBD system we describe learns a relation between descriptive text concerning a musical artist and their actual acoustic output, making such queries as "Play me something loud with an electronic beat" possible by merely analyzing the audio content of a database. We show a novel machine learning technique based on regularized least-squares classification (RLSC) that can quickly and efficiently learn the non-linear relation between descriptive language and audio features by treating the problem as a large number of possible output classes linked to the same set or input features. We show how the RLSC training can easily eliminate irrelevant labels.
Applications of Regularization

We discuss applications of regularization to bioinformatics and text classification. In both of these very high-dimension domains, we find that using regularization to restore well-posedness to the associated learning problems allows us to overcome the curse of dimensionality and achieve state-of-the-art results. In the bioinformatics domain, we focus on classifying DNA microarray data and finding the relevant genes for the classification function. We also address the issue of statistical significance via permutation tests. In the text classification domain, we show how the use of either the Support Vector Machine or the (much faster) Regularized Least-Squares Classification algorithm produces state-of-the-art results. We find that best results are achieved by using a simple one-vs.-all classification system, and present theoretical arguments to support this finding.
The Audiomomma Music Recommendation System

We design and implement a system that recommends musicians to listeners. The basic idea is to keep track of what artists a user listens to, to nd other users with similar tastes, and to recommend other artists that these similar listeners enjoy. The system utilizes a client-server architecture, a web-based interface, and an SQL database to store and process information. We describe Audiomomma-0.3, a proof-of-concept implementation of the above ideas. This report describes research done within the Center for Biological & Computational Learning in the Department of Brain & Cognitive Sciences and in the Arti cial Intelligence Laboratory at the Massachusetts Institute of Technology. This research was sponsored by grants from: OAEce of Naval Research under contract No. N0001493-1-3085, OAEce of Naval Research (DARPA) under contract No. N00014-00-1-0907, National Science Foundation (ITR) under contract No. IIS-0085836, National Science Foundation (KDI) under contract No. DMS-9872936, and National Science Foundation under contract No. IIS-9800032. Additional support was provided by: Central Research Institute of Electric Power Industry, Center for eBusiness (MIT), Eastman Kodak Company, DaimlerChrysler AG, Compaq, Honda R&D Co., Ltd., Komatsu Ltd., Merrill-Lynch, NEC Fund, Nippon Telegraph & Telephone, Siemens Corporate Research, Inc., and The Whitaker Foundation.
An Empirical Comparison of SNoW and SVMs for Face Detection

Impressive claims have been made for the performance of the SNoW algorithm on face detection tasks by Yang et. al. . In particular, by looking at both their results and those of Heisele et. al. , one could infer that the SNoW system performed substantially better than an SVM-based system, even when the SVM used a polynomial kernel and the SNoW system used a particularly simplistic \primitive" linear representation. We evaluated the two approaches in a controlled experiment, looking directly at performance on a simple, xed-sized test set, isolating out \infrastructure" issues related to detecting faces at various scales in large images. We found that SNoW performed about as well as linear SVMs, and substantially worse than polynomial SVMs.  This report describes research done within the Center for Biological and Computational Learning in the Department of Brain and Cognitive Sciences and at the Arti cial Intelligence Laboratory at the Massachusetts Institute of Technology. This research is sponsored by a grant from the OÆce of Naval Research under contract No.  Additional support is provided by Eastman Kodak Company, Daimler-Benz AG, Siemens Corporate Research Inc., AT&T, Digital Equipment and Central Research Institute of Electric Power Industry.
Improving Multiclass Text Classification with the Support Vector Machine

We compare Naive Bayes and Support Vector Machines on the task of multiclass text classification. Using a variety of approaches to combine the underlying binary classifiers, we find that SVMs substantially outperform Naive Bayes. We present full multiclass results on two well-known text data sets, including the lowest error to date on both data sets. We develop a new indicator of binary performance to show that the SVM’s lower multiclass error is a result of its improved binary performance. Furthermore, we demonstrate and explore the surprising result that one-vs-all classification performs favorably compared to other approaches even though it has no error-correcting properties.
Bayesian Methods for Transcript Level Estimation from Noisy Array Measurements

Gene arrays demonstrate a promising ability to characterize expression levels across the entire genome, but they suffer from significant levels of measurement noise. We present a statistical technique to estimate transcript levels or transcript level ratios from one or more gene array experiments, incorporating a model of measurement noise and prior information about biological expression levels. The Bayesian Estimation of Array Measurements (BEAM) technique provides a principled method to handle issues currently addressed through rough and varying heuristics, such as identification of changes in expression level, combination of repeated measurements, and rectification of negative measurements. More importantly, the BEAM technique produces associated measures of estimation uncertainty (e.g., p-values) which serve to determine the statistical significance of reported results. While our method applies to any gene array technology, we illustrate it using a detailed noise model that we develop for Affymetrix yeast chips. The BEAM technique allows for the design of experiments that maximize the useful information derived from a minimum number of chips. Further, it can be used to extract additional and more statistically rigorous conclusions from existing data.