nils-herrmann/clean_scirepeval huggingface dataset:
Kernels for Multi--task Learning

This paper provides a foundation for multi-task learning using reproducing kernel Hilbert spaces of vector-valued functions. In this setting, the kernel is a matrix-valued function. Some explicit examples will be described which go beyond our earlier results in . In particular, we characterize classes of matrix- valued kernels which are linear and are of the dot product or the translation invariant type. We discuss how these kernels can be used to model relations between the tasks and present linear multi-task learning algorithms. Finally, we present a novel proof of the representer theorem for a minimizer of a regularization functional which is based on the notion of minimal norm interpolation.
A Function Representation for Learning in Banach Spaces

Kernel–based methods are powerful for high dimensional function representation. The theory of such methods rests upon their attractive mathematical properties whose setting is in Hilbert spaces of functions. It is natural to consider what the corresponding circumstances would be in Banach spaces. Led by this question we provide theoretical justifications to enhance kernel–based methods with function composition. We explore regularization in Banach spaces and show how this function representation naturally arises in that problem. Furthermore, we provide circumstances in which these representations are dense relative to the uniform norm and discuss how the parameters in such representations may be used to fit data.
New results on error correcting output codes of kernel machines

We study the problem of multiclass classification within the framework of error correcting output codes (ECOC) using margin-based binary classifiers. Specifically, we address two important open problems in this context: decoding and model selection. The decoding problem concerns how to map the outputs of the classifiers into class codewords. In this paper we introduce a new decoding function that combines the margins through an estimate of their class conditional probabilities. Concerning model selection, we present new theoretical results bounding the leave-one-out (LOO) error of ECOC of kernel machines, which can be used to tune kernel hyperparameters. We report experiments using support vector machines as the base binary classifiers, showing the advantage of the proposed decoding function over other functions of I he margin commonly used in practice. Moreover, our empirical evaluations on model selection indicate that the bound leads to good estimates of kernel parameters.
Regularized multi--task learning

Past empirical work has shown that learning multiple related tasks from data simultaneously can be advantageous in terms of predictive performance relative to learning these tasks independently. In this paper we present an approach to multi--task learning based on the minimization of regularization functionals similar to existing ones, such as the one for Support Vector Machines (SVMs), that have been successfully used in the past for single--task learning. Our approach allows to model the relation between tasks in terms of a novel kernel function that uses a task--coupling parameter. We implement an instance of the proposed approach similar to SVMs and test it empirically using simulated as well as real data. The experimental results show that the proposed method performs better than existing multi--task learning methods and largely outperforms single--task learning using SVMs.
Leave One Out Error, Stability, and Generalization of Voting Combinations of Classifiers

We study the leave-one-out and generalization errors of voting combinations of learning machines. A special case considered is a variant of bagging. We analyze in detail combinations of kernel machines, such as support vector machines, and present theoretical estimates of their leave-one-out error. We also derive novel bounds on the stability of combinations of any classifiers. These bounds can be used to formally show that, for example, bagging increases the stability of unstable learning machines. We report experiments supporting the theoretical findings.
Comparing Convolution Kernels and Recursive Neural Networks for Learning Preferences on Structured Data

Convolution kernels and recursive neural networks (RNN) are both suitable approaches for supervised learning when the input portion of an instance is a discrete structure like a tree or a graph. We report about an empirical comparison between the two architectures in a large scale preference learning problem related to natural language processing, where instances are candidate incremental parse trees. We found that kernels never outperform RNNs, even when a limited number of examples is employed for learning. We argue that convolution kernels may lead to feature space representations that are too sparse and too general because not focused on the specific learning task. The adaptive encoding mechanism in RNNs in this case allows us to obtain better prediction accuracy at smaller computational cost.
Image Representations and Feature Selection for Multimedia Database Search

The success of a multimedia information system depends heavily on the way the data is represented. Although there are "natural" ways to represent numerical data, it is not clear what is a good way to represent multimedia data, such as images, video, or sound. We investigate various image representations where the quality of the representation is judged based on how well a system for searching through an image database can perform-although the same techniques and representations can be used for other types of object detection tasks or multimedia data analysis problems. The system is based on a machine learning method used to develop object detection models from example images that can subsequently be used for examples to detect-search-images of a particular object in an image database. As a base classifier for the detection task, we use support vector machines (SVM), a kernel based learning method. Within the framework of kernel classifiers, we investigate new image representations/kernels derived from probabilistic models of the class of images considered and present a new feature selection method which can be used to reduce the dimensionality of the image representation without significant losses in terms of the performance of the detection-search-system.
On Tuning Hyper-Parameters of Multiclass Margin Classifiers

The choice of hyper-parameters (e.g. kernel parameters) can significantly affect generalization performance of large margin classifiers. In this paper we are concerned with the problem of tuning these values in the case of multi-class problems that have been recast into a set of binary problems. We report several experimental results comparing independent and joint tuning of the hyper-parameters of the binary classifiers. Several different encoding strategies are explored, including error correcting output codes. Tuning was carried out by using a validation set and a newly introduced bound on the leave-one-out error.
From Margins to Probabilities in Multiclass Learning Problems

We study the problem of multiclass classification within the framework of error correcting output codes (ECOC) using margin-based binary classifiers. An important open problem in this context is how to measure the distance between class codewords and the outputs of the classifiers. In this paper we propose a new decoding function that combines the margins through an estimate of their class conditional probabilities. We report experiments using support vector machines as the base binary classifiers, showing the advantage of the proposed decoding function over other functions of the margin commonly used in practice. We also present new theoretical results bounding the leave-one-out error of ECOC of kernel machines, which can be used to tune kernel parameters. An empirical validation indicates that the bound leads to good estimates of kernel parameters and the corresponding classifiers attain high accuracy.
Leave-one-out error and stability of learning algorithms with applications

1,078,620. Thermal regenerators. H. BRANDT, and A. KRITZLER [trading as APPARATEBAU ROTHEMUHLE BRANDT & KRITZLER]. Sept. 28, 1965 [Sept. 29, 1964], No. 41184/65. Heading F4K. In a rotary duct type regenerator-in which a peripheral sealing frame (32) is joined to each rotating duct (20) by an expansion member (33) and also by a number of swivel joints (48) for rotating the frame with the duct, and the frame and its sealing strip (36) are held in sealing contact with the regenerator mass by resilient means, the resilient means comprising a plurality of bolts 40 spaced around the periphery of the duct and attached at one end to the frame (32), each bolt having therearound a spring 43 acting between a fixed point on the bolt and a flange 41 on the duct, the bolt being threaded in the frame and rotatable by a tommy-bar 60 to adjust the pressure of the frame on the regenerator mass-the axial movement of bolt 40, during operation of the regenerator, is limited by an adjustable nut 46 co-operating with a slide 57, 54 ( inserted in a guide-way 55, 56, the slide having such thickness as to adjust the clearance a between nut 46 and slide 57, 54 to a desired value, e.g. 5A0 mm. The gap between nut 46 and flange 41 is set approximately before starting the regenerator, and when the regenerator has attained stable operating temperatures, the regenerator is stopped for the short time necessary to insert a slide of the correct thickness at each bolt.
A Simple Algorithm for Learning Stable Machines

We present an algorithm for learning stable machines which is motivated by recent results in statistical learning theory. The algorithm is similar to Breiman's bagging despite some important differences in that it computes an ensemble combination of machines trained on small random sub-samples of an initial training set. A remarkable property is that it is often possible to just use the empirical error of these combinations of machines for model selection. We report experiments using support vector machines and neural networks validating the theory.
A Short Review of Statistical Learning Theory

Statistical learning theory has emerged in the last few years as a solid and elegant frameworkfor studying the problem of learning from examples. Unlike previous "classical" learning techniques, this theory completely characterizes the necessary and sufficient conditions for a learning algorithm to be consistent. The key quantity is the capacity of the set of hypotheses employed in the learning algorithm and the goal is to control this capacity depending on the given examples. Structural risk minimization (SRM) is the main theoretical algorithm which implements this idea. SRMis inspired and closely related to regularization theory. For practical purposes, however, SRM is a very hard problem and impossible to implement when dealing with a large number of examples. Techniques such as support vector machines and older regularization networks are a viable solution to implement the idea of capacity control. The paper also discusses how these techniques can be formulated as a variational problem in a Hilbert space and show how SRM can be extended in order to implement both classical regularization networks and support vector machines.
Maintenance Training of Electric Power Facilities Using Object Recognition by SVM

We are developing a support system for maintenance training of electric power facilities using augmented reality. To develop in the system, we evaluated the use of Support VectorMachine Classifier (SVM) for object recognition. This paper presents our experimental results of object recognition by combinations of SVMs. The recognition results of over 10,000 images show very high performance rates. The support system that uses the combinations of SVMs works in real time without special marks or sensors.
Learning Preference Relations from Data

A number of learning tasks can be solved robustly using key concepts from statistical learning theory. In this paper we first summarize the main concepts of statistical learning theory, a framework in which certain learning from examples problems, namely classification, regression, and density estimation, have been studied in a principled way. We then show how the key concepts of the theory can be used not only for these standard learning from examples problems, but also for many others. In particular we discuss how to learn functions which model a preference relation. The goal is to illustrate the value of statistical learning theory beyond the standard framework it has been used until now.
Support Vector Machines with Clustering for Training with Very Large Datasets

We present a method for training Support Vector Machines (SVM) classifiers with very large datasets. We present a clustering algorithm that can be used to preprocess standard training data and show how SVM can be simply extended to deal with clustered data, that is effectively training with a set of weighted examples. The algorithm computes large clusters for points which are far from the decision boundary and small clusters for points near the boundary. This implies that when SVMs are trained on the preprocessed clustered data set nearly the same decision boundary is found but the computational time decreases significantly. When the input dimensionality of the data is not large, for example of the order of ten, the clustering algorithm can significantly decrease the effective number of training examples, which is a useful feature for training SVM on large data sets. Preliminary experimental results indicate the benefits of our approach.
Component-based face detection

We present a component-based, trainable system for detecting frontal and near-frontal views of faces in still gray images. The system consists of a two-level hierarchy of Support Vector Machine (SVM) classifiers. On the first level, component classifiers independently detect components Of a face. On the second level, a single classifier checks if the geometrical configuration of the detected components in the image matches a geometrical model of a face. We propose a method for automatically learning components by using 3-D head models, This approach has the advantage that no manual interaction is required for choosing and extracting components. Experiments show that the component-based system is significantly more robust against rotations in depth than a comparable system trained on whole face patterns.
A New Machine Learning Approach to Fingerprint Classification

We present new fingerprint classification algorithms based on two machine learning approaches: support vector machines (SVMs), and recursive neural networks (RNNs). RNNs are trained on a structured representation of the fingerprint image. They are also used to extract a set of distributed features which can be integrated in the SVMs. SVMs are combined with a new error correcting code scheme which, unlike previous systems, can also exploit information contained in ambiguous fingerprint images. Experimental results indicate the benefit of integrating global and structured representations and suggest that SVMs are a promising approach for fingerprint classification.
Categorization by Learning and Combining Object Parts

We describe an algorithm for automatically learning discriminative components of objects with SVM classifiers. It is based on growing image parts by minimizing theoretical bounds on the error probability of an SVM. Component-based face classifiers are then combined in a second stage to yield a hierarchical SVM classifier. Experimental results in face classification show considerable robustness against rotations in depth and suggest performance at significantly better level than other face detection systems. Novel aspects of our approach are: a) an algorithm to learn component-based classification experts and their combination, b) the use of 3-D morphable models for training, and c) a maximum operation on the output of each component classifier which may be relevant for biological models of visual recognition.
Support Vector Machines: Theory and Applications

This chapter presents a summary of the issues discussed during the one day workshop on “Support Vector Machines (SVM) Theory and Applications” organized as part of the Advanced Course on Artificial Intelligence (ACAI ’99) in Chania, Greece [19]. The goal of the chapter is twofold: to present an overview of the background theory and current understanding of SVM, and to discuss the papers presented as well as the issues that arose during the workshop.
WORKSHOP ON SUPPORT VECTOR MACHINES : THEORY AND APPLICATIONS

This paper presents a summary of the issues discussed during the one day workshop on "Support Vector Machines (SVM) Theory and Applications" organized as part of the Advanced Course on Artificial Intelligence (ACAI 99) in Chania, Greece. The goal of the paper is twofold: to present an overview of the background theory and current understanding of SVM, and to discuss the papers presented as well as the issues that arose during the workshop.
Fingerprint Classification with Combinations of Support Vector Machines

We report about some experiments on the fingerprint database NIST-4 using different combinations of Support Vector Machine (SVM) classifiers. Images have been preprocessed using the feature extraction technique as in . Our best classification accuracy is 89.3 percent (with 1.8 percent rejection due to the feature extraction process) and is obtained by an error-correction scheme of SVM classifiers. Our current system does not outperform previously proposed classification methods, but the focus here is on the development of novel algorithmic ideas. In particular, as far as we know, SVM have not been applied before in this area and our preliminary findings clearly suggest that they are an effective and promising approach for fingerprint classification.
People Recognition in Image Sequences by Supervised Learning

We describe a system that learns from examples to recognize people in images taken indoors. Images of people are represented by color-based and shape-based features. Recognition is carried out through combinations of Support Vector Machine classi- ers (SVMs). Di erent types of multiclass strategies based on SVMs are explored and compared to k-Nearest Neighbors classi ers (kNNs). The system works in real time and shows high performance rates for people recognition throughout one day.
On the Noise Model of Support Vector Machines Regression

Support Vector Machines Regression (SVMR) is a learning technique where the goodness of fit is measured not by the usual quadratic loss function (the mean square error), but by a different loss function called the Ɛ-Insensitive Loss Function (ILF), which is similar to loss functions used in the field of robust statistics. The quadratic loss function is well justified under the assumption of Gaussian additive noise. However, the noise model underlying the choice of the ILF is not clear. In this paper the use of the ILF is justified under the assumption that the noise is additive and Gaussian, where the variance and mean of the Gaussian are random variables. The probability distributions for the variance and mean will be stated explicitly. While this work is presented in the framework of SVMR, it can be extended to justify nonquadratic loss functions in any Maximum Likelihood or Maximum AP osteriori approach. It applies not only to the ILF, but to a much broader class of loss functions.
A Note on the Generalization Performance of Kernel Classifiers with Margin

We present distribution independent bounds on the generalization misclassification performance of a family of kernel classifiers with margin. Support Vector Machine classifiers (SVM) stem out of this class of machines. The bounds are derived through computations of the Vγ dimension of a family of loss functions where the SVM one belongs to. Bounds that use functions of margin distributions (i.e. functions of the slack variables of SVM) are derived.
Object recognition and detection by a combination of support vector machine and rotation invariant phase only correlation

This paper proposes an object recognition and detection method by a combination of support vector machine classifier (SVM) and rotation invariant phase-only correlation (RIPOC). SVM is a learning technique that is well founded in statistical learning theory. RIPOC is a position and rotation invariant pattern matching technique. We combined these two techniques to develop an augmented reality system. This system can recognize and detect objects from image sequences without special image marks or sensors and show information about the objects through a head-mounted display. Performance is real time.
Face Detection in Still Gray Images

We present a trainable system for detecting frontal and near-frontal views of faces in still gray images using Support Vector Machines (SVMs). We first consider the problem of detecting the whole face pattern by a single SVM classifier. In this context we compare different types of image features, present and evaluate a new method for reducing the number features and discuss practical issues concerning the parameterization of SVMs and the selection of training data. The second part of the paper describes a component-based method for face detection consisting of a two-level hierarchy of SVM classifiers. On the first level, component classifiers independently detect components of a face, such as the eyes, the nose, and the mouth. On the second level, a single classifier checks if the geometrical configuration of the detected components in the image matches a geometrical model of a face.
People recognition and pose estimation in image sequences

Presents a system which learns from examples to automatically recognize people and estimate their poses in image sequences with the potential application to daily surveillance in indoor environments. The person in the image is represented by a set of features based on color and shape information. Recognition is carried out through a hierarchy of biclass SVM classifiers that are separately trained to recognize people and estimate their poses. The system shows a very high accuracy in people recognition and about 85% level of performance in pose estimation, outperforming in both cases k-nearest neighbors classifiers. The system works in real time.
Regularization Networks and Support Vector Machines

Regularization Networks and Support Vector Machines are techniques for solving certain problems of learning from examples – in particular, the regression problem of approximating a multivariate function from sparse data. Radial Basis Functions, for example, are a special case of both regularization and Support Vector Machines. We review both formulations in the context of Vapnik's theory of statistical learning which provides a general foundation for the learning problem, combining functional analysis and statistics. The emphasis is on regression: classification is treated as a special case.
Image representations for object detection using kernel classifiers

This paper presents experimental comparisons of various image representations for object detection using kernel classifiers. In particular it discusses the use of support vector machines (SVM) for object detection using as image representations raw pixel values, projections onto principal components, and Haar wavelets. General linear transformations of the images through the choice of the kernel of the SVM are considered. Experiments showing the effects of histogram equalization, a non-linear transformation, are presented. Image representations derived from probabilistic models of the class of images considered, through the choice of the kernel of the SVM, are also evaluated. Finally, we present a feature selection method using SVMs, and show experimental results.