nils-herrmann/clean_scirepeval huggingface dataset:
Beyond the Margin

The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba.
[Anastomosis using the Valtrac ring--pro and con].

Authors present their experiences with the gastrointestinal anastomosis construction with the biofragmentable ring (Valtrac). Between May 1995 and June 1999 they used it in the group of 75 patients with mean age 58.4 (range 19-59) years. They used Valtrac most often--in 32 patients--to construct the anastomosis between small and large bowel after right hemicolectomy. One enterocutaneous fistula and one intestinal obstruction due to adhesions occurred in this group. Two patients had the signs of fecal impaction in postoperative period, which disappeared after the ring fragmentation. According to their experiences authors claim, that the anastomosis with Valtrac is a safe procedure with a few number of postoperative complications. Disadvantage in our conditions is its high price.
Advances in Large Margin Classifiers

From the Publisher: 
The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms. 
The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba.
Leave-One-Out Methods

The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba.
Entropy Numbers of Linear Function Classes

This paper collects together a miscellany of results originally motivated by the analysis of the generalization performance of the “maximum-margin” algorithm due to Vapnik and others. The key feature of the paper is its operator-theoretic viewpoint. New bounds on covering numbers for classes related to Maximum Margin classes are derived directly without making use of a combinatorial dimension such as the VC-dimension. Specific contents of the paper include: a new and self-contained proof of Maurey’s theorem and some generalizations with small explicit values of constants; bounds on the covering numbers of maximum margin classes suitable for the analysis of their generalization performance; the extension of such classes to those induced by balls in quasi-Banach spaces (such as norms with ). extension of results on the covering numbers of convex hulls of basis functions to -convex hulls ( ); an appendix containing the tightest known bounds on the entropy numbers of the identity operator between and ( ).
Lernen mit Kernen


Zusammenfassung. Dieser Beitrag erläutert neue Ansätze und Ergebnisse der statistischen Lerntheorie. Nach einer Einleitung wird zunächst das Lernen aus Beispielen vorgestellt und erklärt, dass neben dem Erklären der Trainingdaten die Komplexität von Lernmaschinen wesentlich für den Lernerfolg ist. Weiterhin werden Kern-Algorithmen in Merkmalsräumen eingeführt, die eine elegante und effiziente Methode darstellen, verschiedene Lernmaschinen mit kontrollierbarer Komplexität durch Kernfunktionen zu realisieren. Beispiele für solche Algorithmen sind Support-Vektor-Maschinen(SVM), die Kernfunktionen zur Schätzung von Funktionen verwenden, oder Kern-PCA (principal component analysis), die Kernfunktionen zur Extraktion von nichtlinearen Merkmalen aus Datensätzen verwendet. Viel wichtiger als jedes einzelne Beispiel ist jedoch die Einsicht, dass jeder Algorithmus, der sich anhand von Skalarprodukten formulieren lässt, durch Verwendung von Kernfunktionen nichtlinear verallgemeinert werden kann.Die Signifikanz der Kernalgorithmen soll durch einen kurzen Abriss einiger industrieller und akademischer Anwendungen unterstrichen werden. Hier konnten wir Rekordergebnisse auf wichtigen praktisch relevanten Benchmarks erzielen. We describe recent developments and results of statistical learning theory. In the framework of learning from examples, two factors control generalization ability: explaining the training data by a learning machine of a suitable complexity. We describe kernel algorithms in feature spaces as elegant and efficient methods of realizing such machines. Examples thereof are Support Vector Machines (SVM) and Kernel PCA (Principal Component Analysis). More important than any individual example of a kernel algorithm, however, is the insight that any algorithm that can be cast in terms of dot products can be generalized to a nonlinear setting using kernels.Finally, we illustrate the significance of kernel algorithms by briefly describing industrial and academic applications, including ones where we obtained benchmark record results.
Classification on proximity data with LP-machines

We provide a new linear program to deal with classification of data in the case of data given in terms of pairwise proximities. This allows to avoid the problems inherent in using feature spaces with indefinite metric in support vector machines, since the notion of a margin is purely needed in input space where the classification actually occurs. Moreover in our approach we can enforce sparsity in the proximity representation by sacrificing training error. This turns out to be favorable for proximity data. Similar to /spl nu/-SV methods, the only parameter needed in the algorithm is the (asymptotical) number of data points being classified with a margin. Finally, the algorithm is successfully compared with /spl nu/-SV learning in proximity space and K-nearest-neighbors on real world data from neuroscience and molecular biology.
Advances in Large Margin Classiiers Library of Congress Cataloging-in-publication Data Advances in Large Margin Classiiers / Edited Includes Bibliographical References and Index. Isbn 0-xxx-xxxxx-x (alk. Paper) 1. Machine Learning. 2. Algorithms. 3. Kernel Functions

All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher.
Lernen mit Kernen Support-Vektor-Methoden zur Analyse hochdimensionaler Daten

We describe recent developments and results of statistical learning theory. In the framework of learning from examples, two factors control generalization ability: explaining the training data by a learning machine of a suitable complexity. We describekernel algorithms in feature spaces as elegant and efficient methods of realizing such machines. Examples thereof are Support Vector Machines (SVM) and Kernel PCA (Principal Component Analysis) . More important than any individual example of a kernel algorithm, however, is the insight thatany algorithm that can be cast in terms of dot products can be generalized to a nonlinear setting using kernels. Finally, we illustrate the significance of kernel algorithms by briefly describing industrial and academic applications, including ones where we obtained benchmark record results.
Sample Based GeneralizationBounds 1

It is known that the covering numbers of a function class on a double sample (length 2m, where m is the number of points in the sample) can be used to bound the generalization performance of a classiier by using a margin based analysis. Traditionally this has been done using a \Sauer-like" relationship involving a combinatorial dimension such as the fat-shattering dimension. In this paper we show that one can utilize an analogous argument in terms of the observed covering numbers on a single m-sample (being the actual observed data points). The signiicance of this is that for certain interesting classes of functions, such as support vector machines, one can readily estimate the empirical covering numbers quite well. We show how to do so in terms of the eigenvalues of the Gram matrix created from the data. These covering numbers can be much less than a priori bounds indicate in situations where the particular data received is \easy". The work can be considered an extension of previous results which provided generalization performance bounds in terms of the VC-dimension of the class of hypotheses restricted to the sample, with the considerable advantage that the covering numbers can be readily computed, and they often are small.
v-Arc: Ensemble Learning in the Presence of Outliers

AdaBoost and other ensemble methods have successfully been applied to a number of classification tasks, seemingly defying problems of overfitting. AdaBoost performs gradient descent in an error function with respect to the margin, asymptotically concentrating on the patterns which are hardest to learn. For very noisy problems, however, this can be disadvantageous. Indeed, theoretical analysis has shown that the margin distribution, as opposed to just the minimal margin, plays a crucial role in understanding this phenomenon. Loosely speaking, some outliers should be tolerated if this has the benefit of substantially increasing the margin on the remaining points. We propose a new boosting algorithm which allows for the possibility of a pre-specified fraction of points to lie in the margin area Or even on the wrong side of the decision boundary.
The Entropy Regularization Information Criterion

Effective methods of capacity control via uniform convergence bounds for function expansions have been largely limited to Support Vector machines, where good bounds are obtainable by the entropy number approach. We extend these methods to systems with expansions in terms of arbitrary (parametrized) basis functions and a wide range of regularization methods covering the whole range of general linear additive models. This is achieved by a data dependent analysis of the eigenvalues of the corresponding design matrix.
Entropy Numbers, Operators and Support Vector Kernels

We derive new bounds for the generalization error of feature space machines, such as support vector machines and related regularization networks by obtaining new bounds on their covering numbers. The proofs are based on a viewpoint that is apparently novel in the field of statistical learning theory. The hypothesis class is described in terms of a linear operator mapping from a possibly infinite dimensional unit ball in feature space into a finite dimensional space. The covering numbers of the class are then determined via the entropy numbers of the operator. These numbers, which characterize the degree of compactness of the operator, can be bounded in terms of the eigenvalues of an integral operator induced by the kernel function used by the machine. As a consequence we are able to theoretically explain the effect of the choice of kernel functions on the generalization performance of support vector machines.
Advances in kernel methods: support vector learning

Introduction to support vector learning roadmap. Part 1 Theory: three remarks on the support vector method of function estimation, Vladimir Vapnik generalization performance of support vector machines and other pattern classifiers, Peter Bartlett and John Shawe-Taylor Bayesian voting schemes and large margin classifiers, Nello Cristianini and John Shawe-Taylor support vector machines, reproducing kernel Hilbert spaces, and randomized GACV, Grace Wahba geometry and invariance in kernel based methods, Christopher J.C. Burges on the annealed VC entropy for margin classifiers - a statistical mechanics study, Manfred Opper entropy numbers, operators and support vector kernels, Robert C. Williamson et al. Part 2 Implementations: solving the quadratic programming problem arising in support vector classification, Linda Kaufman making large-scale support vector machine learning practical, Thorsten Joachims fast training of support vector machines using sequential minimal optimization, John C. Platt. Part 3 Applications: support vector machines for dynamic reconstruction of a chaotic system, Davide Mattera and Simon Haykin using support vector machines for time series prediction, Klaus-Robert Muller et al pairwise classification and support vector machines, Ulrich Kressel. Part 4 Extensions of the algorithm: reducing the run-time complexity in support vector machines, Edgar E. Osuna and Federico Girosi support vector regression with ANOVA decomposition kernels, Mark O. Stitson et al support vector density estimation, Jason Weston et al combining support vector and mathematical programming methods for classification, Bernhard Scholkopf et al.
Linear programs for automatic accuracy control in regression.

We have recently proposed a new approach to control the number of basis functions and the accuracy in support vector machines. The latter is transferred to a linear programming setting, which inherently enforces sparseness of the solution. The algorithm computes a nonlinear estimate in terms of kernel functions and an /spl epsiv/>0 with the property that at most a fraction /spl nu/ of the training set has an error exceeding /spl epsiv/. The algorithm is robust to local perturbations of these points' target values. We give an explicit formulation of the optimization equations needed to solve the linear program and point out which modifications of the standard optimization setting are necessary to take advantage of the particular structure of the equations in the regression case.
Accepted for NIPS ’ 99 SV Estimation of a Distribution ’ s Support

Suppose you are given some dataset drawn from an underlying p robability distributionP and you want to estimate a subset S of input space such that the probability that a test point drawn from P lies outside ofS is bounded by some a priori specified 0 < 1. We propose an algorithm which approaches this problem by try ing to estimate a functionf which is positive onS and negative on the complement. The functional form of is given by a kernel expansion in terms of a potentially small subset of the training data; it is regu larized by controlling the length of the weight vector in an associated fea ture space. The algorithm is a natural extension of the support vector al gorithm to the case of unlabelled data.
Input space versus feature space in kernel-based methods

This paper collects some ideas targeted at advancing our understanding of the feature spaces associated with support vector (SV) kernel functions. We first discuss the geometry of feature space. In particular, we review what is known about the shape of the image of input space under the feature space map, and how this influences the capacity of SV methods. Following this, we describe how the metric governing the intrinsic geometry of the mapped surface can be computed in terms of the kernel, using the example of the class of inhomogeneous polynomial kernels, which are often used in SV pattern recognition. We then discuss the connection between feature space and input space by dealing with the question of how one can, given some vector in feature space, find a preimage (exact or approximate) in input space. We describe algorithms to tackle this issue, and show their utility in two applications of kernel methods. First, we use it to reduce the computational complexity of SV decision functions; second, we combine it with the Kernel PCA algorithm, thereby constructing a nonlinear statistical denoising technique which is shown to perform well on real-world data.
Engineering Support Vector Machine Kerneis That Recognize Translation Initialion Sites

In order to extract protein sequences from nucleotide sequences, it is an important step to recognize points at which regions start that code for proteins. These points are called translation initiation sites (TIS).



The task of finding TIS can be modeled as a classification problem. We demonstrate the applicability of support vector machines for this task, and show how to incorporate prior biological knowledge by engineering an appropriate kernel function. With the described techniques the recognition performance can be improved by 26% over leading existing approaches. We provide evidence that existing related methods (e.g. ESTScan) could profit from advanced TIS recognition.
Support Vector Method for Novelty Detection

Suppose you are given some dataset drawn from an underlying probability distribution P and you want to estimate a "simple" subset S of input space such that the probability that a test point drawn from P lies outside of S equals some a priori specified ν between 0 and 1. 
 
We propose a method to approach this problem by trying to estimate a function f which is positive on S and negative on the complement. The functional form of f is given by a kernel expansion in terms of a potentially small subset of the training data; it is regularized by controlling the length of the weight vector in an associated feature space. We provide a theoretical analysis of the statistical performance of our algorithm. 
 
The algorithm is a natural extension of the support vector algorithm to the case of unlabelled data.
Support-Vektor-Methoden zur Analyse hochdimensionaler Daten

We describe recent developments and results of statistical learning theory. In the framework of learning from examples, two factors control generalization ability: explai- ning the training data by a learning machine of a suitable complexity. We describe kernel algorithms in feature spaces as elegant and efficient methods of realizing such machines. Examples thereof are Support Vector Machines (SVM)and Kernel PCA (Principal Component Analysis) . More important than any individual example of a kernel algorithm, however, is the insight that any algorithm that can be cast in terms of dot products can be generalized to a nonlinear setting using kernels. Finally, we illustrate the significance of kernel algorithms by briefly describing industrial and academic applications, in- cluding ones where we obtained benchmark record results.
On a Kernel-Based Method for Pattern Recognition, Regression, Approximation, and Operator Inversion

We present a kernel-based framework for pattern recognition, regression estimation, function approximation, and multiple operator inversion. Adopting a regularization-theoretic framework, the above are formulated as constrained optimization problems. Previous approaches such as ridge regression, support vector methods, and regularization networks are included as special cases. We show connections between the cost function and some properties up to now believed to apply to support vector machines only. For appropriately chosen cost functions, the optimal solution of all the problems described above can be found by solving a simple quadratic programming problem.
Fast Approximation of Support Vector Kernel Expansions, and an Interpretation of Clustering as Approximation in Feature Spaces

Kernel-based learning methods provide their solutions as expansions in terms of a kernel. We consider the problem of reducing the computational complexity of evaluating these expansions by approximating them using fewer terms. As a by-product, we point out a connection between clustering and approximation in reproducing kernel Hilbert spaces generated by a particular class of kernels.
Nonlinear Component Analysis as a Kernel Eigenvalue Problem

A new method for performing a nonlinear form of principal component analysis is proposed. By the use of integral operator kernel functions, one can efficiently compute principal components in high-dimensional feature spaces, related to input space by some nonlinear mapfor instance, the space of all possible five-pixel products in 16 16 images. We give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition.
Kernel PCA and De-Noising in Feature Spaces

Kernel PCA as a nonlinear feature extractor has proven powerful as a preprocessing step for classification algorithms. But it can also be considered as a natural generalization of linear principal component analysis. This gives rise to the question how to use nonlinear features for data compression, reconstruction, and de-noising, applications common in linear PCA. This is a nontrivial task, as the results provided by kernel PCA live in some high dimensional feature space and need not have pre-images in input space. This work presents ideas for finding approximate pre-images, focusing on Gaussian kernels, and shows experimental results using these pre-images in data reconstruction and de-noising on toy examples as well as on real world data.
Kernel PCA pattern reconstruction via approximate pre-images.

Algorithms based on Mercer kernels construct their solutions in terms of expansions in a high-dimensional feature space F. Previous work has shown that all algorithms which can be formulated in terms of dot products in F can be performed using a kernel without explicitly working in F. The list of such algorithms includes support vector machines and nonlinear kernel principal component extraction. So far, however, it did not include the reconstruction of patterns from their largest nonlinear principal components, a technique which is common practice in linear principal component analysis.
Support Vector Machine Reference Manual

The Support Vector Machine (SVM) is a new type of learning machine. The SVM is a general architecture that can be applied to pattern recognition, regression estimation and other problems. The following researchers were involved in the development of the SVM: The Support Vector Machine (SVM) program allows a user to carry out pattern recognition and regression estimation, using support vector techniques on some given data. If you have any questions not answered by the documentation, you can e-mail us at:
From Regularization Operators to Support Vector Kernels

We derive the correspondence between regularization operators used in Regularization Networks and Hilbert Schmidt Kernels appearing in Support Vector Machines. More specifically, we prove that the Green's Functions associated with regularization operators are suitable Support Vector Kernels with equivalent regularization properties. As a by-product we show that a large number of Radial Basis Functions namely conditionally positive definite functions may be used as Support Vector kernels.