nils-herrmann/clean_scirepeval huggingface dataset:
Further results on the margin distribution

A number of results have bounded generalization error of a classifier in terms of its margin on the training points. There has been some debate about whether the minimum margin is the best measure of the distribution of training set margin values with which to estimate the generalization error. Freund and Schapire [7] have shown how a different function of the margin distribution can be used to bound the number of mistakes of an on-line learning algorithm for a perceptron, as well as an expected error bound. Shawe-Taylor and Cristianini [ 131 showed that a slight generalization of their construction can be used to give a pat style bound on the tail of the distribution of the generalization errors that arise from a given sample size when using threshold linear classifiers. We show that in the linear case the approach can be viewed as a change of kernel and that the algorithms arising from the approach are exactly those originally proposed by Cortes and Vapnik . We generalise the basic result to function classes with bounded fat-shattering dimension and the Ii measure for slack variables which gives rise to Vapnik’s box constraint algorithm. Finally, application to regression is considered, which includes standard least squares as a special case.
Project number:

The project “Landscape Analysis of Habitat Supply and Effects on Populations of the Northern Spotted Owl in BC” was initiated in 2003 by the Canadian Spotted Owl Recovery Team (CSORT) to address questions specific to habitat supply and population for recovery planning for the Northern Spotted Owl (SPOW), an Endangered subspecies in Canada, facing extirpation from British Columbia (Committee on the Status of Endangered Wildlife in Canada 2001, Blackburn et al. 2002). The CSORT collaborated with modellers, analysts and topic experts to develop and test management/habitatfocused modelling approaches. The final model framework includes the following components 1) the spatially explicit SPOW habitat classification model, 2) the landscape dynamics model that projects forest growth and stand-replacing natural disturbances, 3) the population model, 4) the spatial timber supply model, 5) the structural (cost-surface) models to capture effects of dispersal and movement patterns on territory placement and connectivity between habitat areas, 6) a Bayesian belief network framework (BBN) for assessment of habitat quality that accounts for cross-scale influences of biological attribute states, their spatial configuration, and risk of loss of habitat to be integrated into a probabilistic model of habitat quality, and 7) a Resource Location Model for proactive habitat reserve selection that integrates changing habitat conditions with population or habitat targets through time.
2 Margin Distribution and Soft Margin

Typical bounds on generalization of Support Vector Machines are based on the minimum distance between training examples and the separating hyperplane. There has been some debate as to whether a more robust function of the margin distribution could provide generalization bounds. Freund and Schapire (1998) have shown how a diierent function of the margin distribution can be used to bound the number of mistakes of an on-line learning algorithm for a perceptron, as well as to give an expected error bound. We show that a slight generalization of their construction can be used to give a pac style bound on the tail of the distribution of the generalization errors that arise from a given sample size. Furthermore, we show that the approach can be viewed as a change of kernel and that the algorithms arising from the approach are exactly those originally proposed by Cortes and Vapnik (1995). Finally, we discuss the relations of this approach with other techniques, such as regularization and shrinkage methods 1 .
A multiplicative updating algorithm for training support vector machine

Support Vector Machines nd maximal margin hyperplanes in a high dimensional feature space, represented as a sparse linear combination of training points. Theoretical results exist which guarantee a high generalization performance when the margin is large or when the representation is very sparse. Multiplicative-Updating algorithms are a new tool for perceptron learning which are guaranteed to converge rapidly when the target concept is sparse. In this paper we present a Multiplicative-Updating algorithm for training Support Vector Machines which combines the generalization power provided by VC theory with the convergence properties of multiplicative algorithms.
Kernel-dependent support vector error bounds

Model selection in support vector machines is usually carried out by minimizing the quotient of the radius of the smallest enclosing sphere of the data and the observed margin on the training set. We provide a new criterion taking the distribution within that sphere into account by considering the eigenvalue distribution of the Gram matrix of the data. Experimental results on real world data show that this new criterion provides a good prediction of the shape of the curve relating generalization error to kernel width.
Bayesian Voting Schemes as Large Margin Classifiers

It is often claimed that one of the main distinctive features of Bayesian Learning Algorithms for neural networks is that they don't simply output one hypothesis, but rather an entire distribution of probability over an hypothesis set: the Bayes posterior. An alternative perspective is that they output a linear combination of classiiers, whose coeecients are given by Bayes theorem. This can be regarded as a hyperplane in a high-dimensional feature space. We provide a novel theoretical analysis of such classiiers, based on data-dependent VC theory, proving that they can be expected to be large margin hyperplanes in a Hilbert space, and hence to have low eeective VC-dimension. This not only explains the remarkable resistance to overrtting exhibited by such classiiers, but also co-locates them in the same class as other systems, such as Support Vector Machines and Adaboost, which have a similar performance.  In recent years, a new method for training neural networks has been proposed and used, mainly due to the work of MacKay and Neal 7, 8, 9]. The systems inspired by this approach are generally know as Bayesian Learning Algorithms and have proven to be quite resistant to overrtting.
Large Margin DAG's for Multiclass Classification

We present a new learning architecture: the Decision Directed Acyclic Graph (DDAG), which is used to combine many two­class classifiers into a multiclass classifier. For an N­class problem, the DDAG contains N(N-1)/2 classifiers, one for each pair of classes. We present a VC analysis of the case when the node classifiers are hyperplanes; the resulting bound on the test error depends on N and on the margin achieved at the nodes, but not on the dimension of the space. This motivates an algorithm, DAGSVM, which operates in a kernel­induced feature space and uses two­class maximal margin hyperplanes at each decision­node of the DDAG. The DAGSVM is substantially faster to train and evaluate than either the standard algorithm or Max Wins, while maintaining comparable accuracy to both of these algorithms.
Large Margin Trees for Induction and Transduction

The problem of controlling the capacity of decision trees is considered for the case where the decision nodes implement linear threshold functions. In addition to the standard early stopping and pruning procedures, we implement a strategy based on the margins of the decision boundaries at the nodes. The approach is motivated by bounds on generalization error obtained in terms of the margins of the individual classifiers. Experimental results are given which demonstrate that considerable advantage can be derived from using the margin information. The same strategy is applied to the problem of transduction, where the positions of the testing points are revealed to the training algorithm. This information is used to generate an alternative training criterion motivated by transductive theory. In the transductive case, the results are not as encouraging, suggesting that little, if any, consistent advantage is culled from using the unlabelled data in the proposed fashion. This conclusion does not contradict theoretical results, but leaves open the theoretical and practical question of whether more effective use can be made of the additional information.
Generalization Performance of Classifiers in Terms of Observed Covering Numbers

It is known that the covering numbers of a function class on a double sample (length 2m) can be used to bound the generalization performance of a classifier by using a margin based analysis. In this paper we show that one can utilize an analogous argument in terms of the observed covering numbers on a single m-sample (being the actual observed data points). The significance of this is that for certain interesting classes of functions, such as support vector machines, there are new techniques which allow one to find good estimates for such covering numbers in terms of the speed of decay of the eigenvalues of a Gram matrix. These covering numbers can be much less than a priori bounds indicate in situations where the particular data received is "easy". The work can be considered an extension of previous results which provided generalization performance bounds in terms of the VC-dimension of the class of hypotheses restricted to the sample, with the considerable advantage that the covering numbers can be readily computed, and they often are small.
Covering numbers for support vector machines

Support vector (SV) machines are linear classifiers that use the maximum margin hyperplane in a feature space defined by a kernel function. Until recently, the only bounds on the generalization performance of SV machines (within Valiant's probably approximately correct framework) took no account of the kernel used except in its effect on the margin and radius. More recently, it has been shown that one can bound the relevant covering numbers using tools from functional analysis. In this paper, we show that the resulting bound can be greatly simplified. The new bound involves the eigenvalues of the integral operator induced by the kernel. It shows that the effective dimension depends on the rate of decay of these eigenvalues. We present an explicit calculation of covering numbers for an SV machine using a Gaussian kernel, which is significantly better than that implied by previous results.
Multiplicative Updatings for Support-vector Learning Produced as Part of the Esprit Working Group in Neural and Computational Learning Ii, Neurocolt2 27150

Support Vector machines nd maximal margin hyperplanes in a high dimensional feature space. Theoretical results exist which guarantee a high generalization performance when the margin is large or when the number of support vectors is small. Multiplicative-Updating algorithms are a new tool for perceptron learning whose theoretical properties are well studied. In this work we present a Multiplicative-Updating algorithm for learning Support Vector machines which exploits the particular structure of high-generalization hypotheses, by achieving fast rate of convergence just in those situations where high generalization can be obtained, namely small number of support vectors or large margin.
Structural Risk Minimization Over Data-Dependent Hierarchies

The paper introduces some generalizations of Vapnik's (1982) method of structural risk minimization (SRM). As well as making explicit some of the details on SRM, it provides a result that allows one to trade off errors on the training sample against improved generalization performance. It then considers the more general case when the hierarchy of classes is chosen in response to the data. A result is presented on the generalization performance of classifiers with a "large margin". This theoretically explains the impressive generalization performance of the maximal margin hyperplane algorithm of Vapnik and co-workers (which is the basis for their support vector machines). The paper concludes with a more general result in terms of "luckiness" functions, which provides a quite general way for exploiting serendipitous simplicity in observed data to obtain better prediction accuracy from small training sets. Four examples are given of such functions, including the Vapnik-Chervonenkis (1971) dimension measured on the sample.
Classification Accuracy Based on Observed Margin

Following recent results  showing the importance of the fat-shattering dimension in explaining the beneficial effect of a large margin on generalization performance, the current paper investigates how the margin on a test example can be used to give greater certainty of correct classification in the distribution independent model. Hence, generalization analysis is possible at three distinct phases, a priori using a standard pac analysis, after training based on properties of the chosen hypothesis [10], and finally in this paper at testing based on properties of the test example. The results also show that even if the classifier does not classify all of the training examples correctly, the fact that a new example has a larger margin than that on the misclassified test examples, can be used to give very good estimates for the generalization performance in terms of the fat-shattering dimension measured at a scale proportional to the excess margin. The estimate relies on a sufficiently large number of the correctly classified training examples having a margin roughly equal to that used to estimate generalization, indicating that the corresponding output values need to be ``well sampled.
Dynamically Adapting Kernels in Support Vector Machines Produced as Part of the Esprit Working Group in Neural and Computational Learning Ii, Neurocolt2 27150

The kernel-parameter is one of the few tunable parameters in Support Vector machines, and controls the complexity of the resulting hypothesis. The choice of its value amounts to model selection, and is usually performed by means of a validation set. We present an algorithm which can automatically perform model selection and learning with no additional computational cost and with no need of a validation set. Theoretical results motivating this approach providing upper bounds on the generalisation error and experimental results connrming its validity are presented.
Robust Bounds on Generalization from the Margin Distribution

A number of results have bounded generalization of a classifier in terms of its margin on the training points. There has been some debate about whether the minimum margin is the best measure of the distribution of training set margin values with which to estimate the generalization. Freund and Schapire have shown how a different function of the margin distribution can be used to bound the number of mistakes of an on-line learning algorithm for a perceptron, as well as an expected error bound. We show that a slight generalization of their construction can be used to give a pac style bound on the tail of the distribution of the generalization errors that arise from a given sample size. Algorithms arising from the approach are related to those of Cortes and Vapnik. We generalise the basic result to function classes with bounded fat- shattering dimension and the 1-norm of the slack variables which gives rise to Vapnik's box constraint algorithm. We also extend the results to the regression case and obtain bounds on the probability that a randomly chosen test point will have error greater than a given value. The bounds apply to the $\epsilon$-insensitive loss function proposed by Vapnik for Support Vector Machine regression. A special case of this bound gives a bound on the probabilities in terms of the least squares error on the training set showing a quadratic decline in probability with margin.
Bayesian Classifiers Are Large Margin Hyperplanes in a Hilbert Space

Bayesian algorithms for Neural Networks are known to produce classiiers which are very resistent to overrt-ting. It is often claimed that one of the main distinctive features of Bayesian Learning Algorithms is that they don't simply output one hypothesis, but rather an entire distribution of probability over an hypothesis set: the Bayes posterior. An alternative perspective is that they output a linear combination of classiiers, whose coeecients are given by Bayes theorem. One of the concepts used to deal with thresholded convex combinations is thèmargin' of the hyperplane with respect to the training sample, which is correlated to the predictive power of the hypothesis itself. We provide a novel theoretical analysis of such clas-siiers, based on Data-Dependent VC theory, proving that they can be expected to be large margin hyper-planes in a Hilbert space. We then present experimental evidence that the predictions of our model are correct , i.e. that bayesian classifers really nd hypotheses which have large margin on the training examples. This not only explains the remarkable resistance to overrtting exhibited by such classiiers, but also co-locates them in the same class of other systems, like Support Vector machines and Adaboost, which have a similar performance.
Dynamically Adapting Kernels in Support Vector Machines

The kernel-parameter is one of the few tunable parameters in Support Vector machines, controlling the complexity of the resulting hypothesis. Its choice amounts to model selection and its value is usually found by means of a validation set. We present an algorithm which can automatically perform model selection with little additional computational cost and with no need of a validation set. In this procedure model selection and learning are not separate, but kernels are dynamically adjusted during the learning process to find the kernel parameter which provides the best possible upper bound on the generalisation error. Theoretical results motivating the approach and experimental results confirming its validity are presented.
Optimizing Classifers for Imbalanced Training Sets

Following recent results[9, 8] showing the importance of the fat-shattering dimension in explaining the beneficial effect of a large margin on generalization performance, the current paper investigates the implications of these results for the case of imbalanced datasets and develops two approaches to setting the threshold. The approaches are incorporated into ThetaBoost, a boosting algorithm for dealing with unequal loss functions. The performance of ThetaBoost and the two approaches are tested experimentally.
Confidence Estimates of Classification Accuracy on New Examples

Following recent results  showing the importance of the fat shattering dimension in explaining the beneficial effect of a large margin on generalization performance, the current paper investigates how the margin on a test example can be used to give greater certainty of correct classification in the distribution independent model. The results show that even if the classifier does not classify all of the training examples correctly, the fact that a new example has a larger margin than that on the misclassified examples, can be used to give very good estimates for the generalization performance in terms of the fat shattering dimension measured at a scale proportional to the excess margin. The estimate relies on a sufficiently large number of the correctly classified training examples having a margin roughly equal to that used to estimate generalization, indicating that the corresponding output values need to be ‘well sampled’. If this is not the case it may be better to use the estimate obtained from a smaller margin.
Fraud detection and management in mobile telecommunications networks

The paper discusses the status of research on detection of fraud undertaken as part of the European Commission-funded ACTS ASPeCT (Advanced security for personal communications technologies) project. A first task has been the identification of possible fraud scenarios and of typical fraud indicators which can be mapped to data in toll tickets. Currently, the project is exploring the detection of fraudulent behaviour based on a combination of absolute and differential usage. Three approaches are being investigated: a rule based approach and two approaches based on neural networks, where both supervised and unsupervised learning are considered. Special attention is being paid to the feasibility of the implementations.
BRUTUS - A Hybrid Detection Tool

ACTS project AC095, Advanced Security for Personal Communications Technologies (ASPeCT), is engaged in the advancement of security issues for the next generation of mobile communications UMTS. One of the work packages within this project is developing fraud detection and management tools. Prototypes of three different fraud detection tools have been developed, and demonstrated, using Rule-Based and Neural Network technologies. Last year ASPeCT introduced new fraud detection concepts for the GSM network building on the experience gained from fraud scenarios encountered on the Vodafone TACS network. Based on these findings we proposed various Rule-Based and Neural Network architectures to implement these ideas as separate fraud detection tools. The goal was to develop techniques that would work under GSM and later migrate to UMTS. This year we report back on the successes of the various methods and propose BRUTUS, a hybrid detection tool, built upon a generalisation of the existing fraud detection techniques. The new system migrates to other areas of electronic commerce and incorporates extra features for the purpose of business marketing and engineering. A common suite of experiments has been performed on the three systems using two sets of data. One dataset contained only fraudulent activity, transcribed into GSM Toll Ticket format from TACS. The second contained the Toll Tickets of new subscribers from the GSM network. We varied parameters that could be tuned in each of the systems in order to determine the number of subscribers raising alarms. Curves were produced showing the trade-off between the percentage of correctly identified fraudsters versus the percentage of new subscribers raising alarms. This ratio is extremely important as even a small percentage of new subscribers amounts to a significant number of individuals. BRUTUS utilises a management module, or Adaptive Critic, providing the user with a comprehensive analysing tool for pseudo real-time detection of behaviour changes. We provide details of the systems architecture of the individual components and the common framework within which they operate. Future direction for the project will be outlined with our current aims and suggestions for areas in which further work needs to be performed.