an iterative algorithm learning the maximal margin classifier abstract a simple learning algorithm for maximal margin classifiers also support vector machines with quadratic cost function is proposed we build our iterative algorithm on top of the schlesinger kozinec algorithm s k algorithm from 1981 which finds a maximal margin hyperplane with a given precision for separable data we suggest a generalization of the s k algorithm i to the non linear case using kernel functions and ii for non separable data the requirement in memory storage is linear to the data this property allows the proposed algorithm to be used for large training problems the resulting algorithm is simple to implement and as the experiments showed competitive to the state of the art algorithms the implementation of the algorithm in matlab is available we tested the algorithm on the problem aiming at recognition poor quality numerals

on consistent nonparametric order determination and chaos summary we give a brief introduction to deterministic chaos and a link between chaotic deterministic models and stochastic time series models we argue that it is often natural to determine the embedding dimension in a noisy environment first in any systematic study of chaos setting the stochastic models within the framework of non linear autoregression we introduce the notion of a generalized partial autocorrelation and an order we approach the estimation of the embedding dimension via order determination of an unknown non linear autoregression by cross validation and give justification by proving its consistency under global boundedness as a by product we provide a theoretical justification of the final prediction error approach of auestad and tj0stheim some illustrations based on the henon map and several real data sets are given the bias of the residual sum of squares as essentially a noise variance estimator is quantified

flexible discriminant analysis by optimal scoring abstract fisher s linear discriminant analysis is a valuable tool for multigroup classification with a large number of predictors one can find a reduced number of discriminant coordinate functions that are optimal for separating the groups with two such functions one can produce a classification map that partitions the reduced space into regions that are identified with group membership and the decision boundaries are linear this article is about richer nonlinear classification schemes linear discriminant analysis is equivalent to multiresponse linear regression using optimal scorings to represent the groups in this paper we obtain nonparametric versions of discriminant analysis by replacing linear regression by any nonparametric regression method in this way any multiresponse regression technique such as mars or neural networks can be postprocessed to improve its classification performance

the ii p method for estimating multivariate functions from noisy data the method for estimating an underlying smooth function of m variables x l xm using noisy data is based on approximating it by a sum of products of the form m m x m the problem is then reduced to estimating the univariate functions in the products a convergent algorithm is described the method keeps tight control on the degrees of freedom used in the fit many examples are given the quality of fit given by the method is excellent usually only a few products are enough to fit even fairly complicated functions the coding into products of univariate functions allows a relatively understandable interpretation of the multivariate fit

a fast iterative nearest point algorithm for support vector machine classifier design in this paper we give a new fast iterative algorithm for support vector machine svm classifier design the basic problem treated is one that does not allow classification violations the problem is converted to a problem of computing the nearest point between two convex polytopes the suitability of two classical nearest point algorithms due to gilbert and mitchell et al is studied ideas from both these algorithms are combined and modified to derive our fast algorithm for problems which require classification violations to be allowed the violations are quadratically penalized and an idea due to cortes and vapnik and friess is used to convert it to a problem in which there are no classification violations comparative computational evaluation of our algorithm against powerful svm methods such as platt s sequential minimal optimization shows that our algorithm is very competitive

universal approximation using radial basis function networks there have been several recent studies concerning feedforward networks and the problem of approximating arbitrary functionals of a finite number of real variables some of these studies deal with cases in which the hidden layer nonlinearity is not a sigmoid this was motivated by successful applications of feedforward networks with nonsigmoidal hidden layer units this paper reports on a related study of radial basis function rbf networks and it is proved that rbf networks having one hidden layer are capable of universal approximation here the emphasis is on the case of typical rbf networks and the results show that a certain class of rbf networks with the same smoothing factor in each kernel node is broad enough for universal approximation

regression modeling in back propagation and projection pursuit learning we study and compare two types of connectionist learning methods for model free regression problems 1 the backpropagation learning bpl and 2 the projection pursuit learning ppl emerged in recent years in the statistical estimation literature both the bpl and the ppl are based on projections of the data in directions determined from interconnection weights however unlike the use of fixed nonlinear activations usually sigmoidal for the hidden neurons in bpl the ppl systematically approximates the unknown nonlinear activations moreover the bpl estimates all the weights simultaneously at each iteration while the ppl estimates the weights cyclically neuron by neuron and layer by layer at each iteration although the bpl and the ppl have comparable training speed when based on a gauss newton optimization algorithm the ppl proves more parsimonious in that the ppl requires a fewer hidden neurons to approximate the true function to further improve the statistical performance of the ppl an orthogonal polynomial approximation is used in place of the supersmoother method originally proposed for nonlinear activation approximation in the ppl

multivariate adaptive regression splines a new method is presented for flexible regression modeling of high dimensional data the model takes the form of an expansion in product spline basis functions where the number of basis functions as well as the parameters associated with each one product degree and knot locations are automatically determined by the data this procedure is motivated by the recursive partitioning approach to regression and shares its attractive properties unlike recursive partitioning however this method produces continuous models with continuous derivatives it has more power and flexibility to model relationships that are nearly additive or involve interactions in at most a few variables in addition the model can be represented in a form that separately identifies the additive contributions and those associated with the different multivariable interactions

tree structured classification via generalized discriminant analysis abstract the problem of constructing classification rules that can be represented as decision trees is considered each object to be classified has an associated x vector containing possibly incomplete covariate information tree construction is based on the information provided in a learning sample of objects with known class identities the x vectors in the learning sample may have missing values as well procedures are proposed for each of the components of classifier construction such as split selection tree size determination treatment of missing values and ranking of variables the main idea is recursive application of linear discriminant analysis with the variables at each stage being appropriately chosen according to the data and the type of splits desired standard statistical techniques used as basic building blocks include analysis of variance linear and canonical discriminant analysis and principal component analysis a new method of tree structured classification is obtained by assem

optimal partitioning for classification and regression trees an iterative algorithm that finds a locally optimal partition for an arbitrary loss function in time linear in n for each iteration is presented the algorithm is a k means like clustering algorithm that uses as its distance measure a generalization of kullback s information divergence moreover it is proven that the globally optimal partition must satisfy a nearest neighbour condition using divergence as the distance measure these results generalize similar results of l breiman et al 1984 to an arbitrary number of classes or regression variables and to an arbitrary number of bills experimental results on a text to speech example are provided and additional applications of the algorithm including the design of variable combinations surrogate splits composite nodes and decision graphs are suggested

regularized discriminant analysis abstract linear and quadratic discriminant analysis are considered in the small sample high dimensional setting alternatives to the usual maximum likelihood plug in estimates for the covariance matrices are proposed these alternatives are characterized by two parameters the values of which are customized to individual situations by jointly minimizing a sample based estimate of future misclassification risk computationally fast implementations are presented and the efficacy of the approach is examined through simulation studies and application to data these studies indicate that in many circumstances dramatic gains in classification accuracy can be achieved

uniform infinite planar triangulations the existence of the weak limit as n of the uniform measure on rooted triangulations of the sphere with n vertices is proved some properties of the limit are studied in particular the limit is a probability measure on random triangulations of the plane

asymptotic global behavior for stochastic approximation and diffusions with slowly decreasing noise effects global minimization via monte carlo the asymptotic behavior of the systems x n 1 x n a n b x n xi n a n sigma x n psi n and dy bar b y dt sqrt a t sigma y dw is studied where psi n is i i d gaussian xi n is a correlated bounded sequence of random variables and a n approx a 0 log a 1 n without xi n such algorithms are versions of the simulated annealing method for global optimization when the objective function values can only be sampled via monte carlo the discrete algorithm is a combination of stochastic approximation and simulated annealing our forms are appropriate the psi n are the annealing variables and xi n is the sampling noise for large a 0 a full asymptotic analysis is presented via the theory of large deviations mean escape time after arbitrary time n from neighborhoods of stable sets of the algorithm mean transition times after arbitrary time n from a neighborhood of one stable set to another a

hierarchical mixtures of experts and the em algorithm we present a tree structured architecture for supervised learning the statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models glim s learning is treated as a maximum likelihood problem in particular we present an expectation maximization em algorithm for adjusting the parameters of the architecture we also develop an on line learning algorithm in which the parameters are updated incrementally comparative simulation results are presented in the robot dynamics domain

modified quadratic discriminant functions and the application to chinese character recognition issues in the quadratic discriminant functions qdf are discussed and two types of modified quadratic disriminant functions mqdf1 mqdf2 which are less sensitive to the estimation error of the covariance matrices are proposed the mqdf1 is a function which employs a kind of a pseudo bayesian estimate of the covariance matrix instead of the maximum likelihood estimate ordinarily used in the qdf the mqdf2 is a variation of the mqdf1 to save the required computation time and storage two discriminant functions were applied to chinese character recognition to evaluate their effectiveness and remarkable improvement was observed in their performance

improving the generalization properties of radial basis function neural networks an important feature of radial basis function neural networks is the existence of a fast linear learning algorithm in a network capable of representing complex nonlinear mappings satisfactory generalization in these networks requires that the network mapping be sufficiently smooth we show that a modification to the error functional allows smoothing to be introduced explicitly without significantly affecting the speed of training a simple example is used to demonstrate the resulting improvement in the generalization properties of the network

a simple lemma on greedy approximation in hilbert space and convergence rates for projection pursuit regression and neural network training a general convergence criterion for certain iterative sequences in hilbert space is presented for an important subclass of these sequences estimates of the rate of convergence are given under very mild assumptions these results establish an 0 1 f4n nonsampling convergence rate for projection pursuit regression and neural network training where n represents the number of ridge functions neurons or coefficients in a greedy basis expansion

a novel geometric approach to binary classification based on scaled convex hulls geometric methods are very intuitive and provide a theoretical foundation to many optimization problems in the fields of pattern recognition and machine learning in this brief the notion of scaled convex hull sch is defined and a set of theoretical results are exploited to support it these results allow the existing nearest point algorithms to be directly applied to solve both the separable and nonseparable classification problems successfully and efficiently then the popular s k algorithm has been presented to solve the nonseparable problems in the context of the sch framework the theoretical analysis and some experiments show that the proposed method may achieve better performance than the state of the art methods in terms of the number of kernel evaluations and the execution time

a practical bayesian framework for backpropagation networks a quantitative and practical bayesian framework is described for learning of mappings in feedforward networks the framework makes possible 1 objective comparisons between solutions using alternative network architectures 2 objective stopping rules for network pruning or growing procedures 3 objective choice of magnitude and type of weight decay terms or additive regularizers for penalizing large weights etc 4 a measure of the effective number of well determined parameters in a model 5 quantified estimates of the error bars on network parameters and on network output and 6 objective comparisons with alternative learning and interpolation models such as splines and radial basis functions the bayesian evidence automatically embodies occam s razor penalizing overflexible and overcomplex models the bayesian approach helps detect poor underlying assumptions in learning models for learning models well matched to a problem a good correlation between generalization ability and the bayesian evidence is obtained

on von neumann regular rings with weak comparability abstract in 1991 k c o meara first defined the notion of weak comparability for regular rings and he showed that every simple directly finite regular ring with weak comparability is unit regular in this paper we investigate properties for regular rings with weak comparability and we show that the strict cancellation property and the strict unperforation property hold for the family of directly finite finitely generated projective modules over these rings

clipping algorithms for solving the nearest point problem over reduced convex hulls the nearest point problem npp i e finding the closest points between two disjoint convex hulls has two classical solutions the gilbert schlesinger kozinec gsk and mitchell dem yanov malozemov mdm algorithms when the convex hulls do intersect npp has to be stated in terms of reduced convex hulls rchs made up of convex pattern combinations whose coefficients are bound by a m 1 value and that are disjoint for suitable m the gsk and mdm methods have recently been extended to solve npp for rchs using the particular structure of the extreme points of a rch while effective their reliance on extreme points may make them computationally costly particularly when applied in a kernel setting in this work we propose an alternative clipped extension of classical mdm that results in a simpler algorithm with the same classification accuracy than that of the extensions already mentioned but also with a much faster numerical convergence

using radial basis functions to approximate a function and its error bounds a novel network called the validity index network vi net is presented the vi net derived from radial basis function networks fits functions and calculates confidence intervals for its predictions indicating local regions of poor fit and extrapolation

flexible parsimonious smoothing and additive modeling a simple method is presented for fitting regression models that are nonlinear in the explanatory variables despite its simplicity or perhaps because of it the method has some powerful characteristics that cause it to be competitive with and often superior to more sophisticated techniques especially for small data sets in the presence of high noise

a geometric approach to support vector machine svm classification the geometric framework for the support vector machine svm classification problem provides an intuitive ground for the understanding and the application of geometric optimization algorithms leading to practical solutions of real world classification problems in this work the notion of reduced convex hull is employed and supported by a set of new theoretical results these results allow existing geometric algorithms to be directly and practically applied to solve not only separable but also nonseparable classification problems both accurately and efficiently as a practical application of the new theoretical results a known geometric algorithm has been employed and transformed accordingly to solve nonseparable problems successfully

local convergence of large random triangulations coupled with an ising model we prove the existence of the local weak limit of the measure obtained by sampling random triangulations of size n decorated by an ising configuration with a weight proportional to the energy of this configuration to do so we establish the algebraicity and the asymptotic behaviour of the partition functions of triangulations with spins for any boundary condition in particular we show that these partition functions all have the same phase transition at the same critical temperature some properties of the limiting object called the infinite ising planar triangulation are derived including the recurrence of the simple random walk at the critical temperature

a general soft method for learning svm classifiers with l 1 norm penalty based on the geometric interpretation of support vector machines svms this paper presents a general technique that allows almost all the existing l 2 norm penalty based geometric algorithms including gilbert s algorithm schlesinger kozinec s sk algorithm and mitchell dem yanov malozemov s mdm algorithm to be softened to achieve the corresponding learning l 1 svm classifiers intrinsically the resulting soft algorithms are to find e optimal nearest points between two soft convex hulls theoretical analysis has indicated that our proposed soft algorithms are essentially generalizations of the corresponding existing hard algorithms and consequently they have the same properties of convergence and almost the identical cost of computation as a specific example the problem of solving n svms by the proposed soft mdm algorithm is investigated and the corresponding solution procedure is specified and analyzed to validate the general soft technique several real classification experiments are conducted with the proposed l 1 norm based mdm algorithms and numerical results have demonstrated that their performance is competitive to that of the corresponding l 2 norm based algorithms such as sk and mdm algorithms

how tight are the vapnik chervonenkis bounds we describe a series of numerical experiments that measure the average generalization capability of neural networks trained on a variety of simple functions these experiments are designed to test the relationship between average generalization performance and the worst case bounds obtained from formal learning theory using the vapnik chervonenkis vc dimension blumer et al 1989 haussler et al 1990 recent statistical learning theories tishby et al 1989 schwartz et al 1990 suggest that surpassing these bounds might be possible if the spectrum of possible generalizations has a gap near perfect performance we indeed find that in some cases the average generalization is significantly better than the vc bound the approach to perfect performance is exponential in the number of examples m rather than the 1 m result of the bound however in these cases we have not found evidence of the gap predicted by the above statistical theories in other cases we do find the 1 m behavior of the vc bound

projection based approximation and a duality with kernel methods projection pursuit regression and kernel regression are methods for estimating a smooth function of several variables from noisy data obtained at scattered sites methods based on local averaging can perform poorly in high dimensions curse of dimensionality intuition and examples have suggested that projection based approaches can provide better fits for what sorts of regression functions is this true when and by how much do projection methods reduce the curse of dimensionality we make a start by focusing on the two dimensional problem and study the l 2 approximation error bias of the two procedures with respect to gaussian measure let ra stand for a certain ppr type approximation and ka for a particular kernel type approximation building on a simple but striking duality for polynomials we show that ra behaves significantly better than the minimax rate of approximation for radial functions while ka performs significantly better than the minimax rate for harmonic functions in fact the rate improvements carry over to large classes ra behaving very well for functions with enough angular smoothness oscillating slowly with angle while ka behaves very well for functions with enough laplacian smoothness oscillations averaging out locally the rate improvements matter they are equivalent to lowering the dimensionality of the problem for example for functions with nice tail behavior ra behaves as if the dimensionality of the problem were 1 5 rather than its nominal value 2 also ra and ka are complementary for a given function if one method offers a dimensionality reduction the other does not

on the interpretation of regression plots abstract a framework is developed for the interpretation of regression plots including plots of the response against selected covariates residual plots added variable plots and detrended added variable plots it is shown that many of the common interpretations associated with these plots can be misleading the framework also allows for the generalization of standard plots and the development of new plotting paradigms a paradigm for graphical exploration of regression problems is sketched

critical ising model on random triangulations of the disk enumeration and local limits we consider boltzmann random triangulations coupled to the ising model on their faces under dobrushin boundary conditions and at the critical point of the model the first part of this paper computes explicitly the partition function of this model by solving its tutte s equation extending a previous result by bernardi and bousquet melou j combin theory ser b 101 5 315 377 2011 to the model with dobrushin boundary conditions we show that the perimeter exponent of the model is 7 3 in contrast to the exponent 5 2 for uniform triangulations in the second part we show that the model has a local limit in distribution when the two components of the dobrushin boundary tend to infinity one after the other the local limit is constructed explicitly using the peeling process along an ising interface moreover we show that the main interface in the local limit touches the infinite boundary almost surely only finitely many times a behavior opposite to that of the bernoulli percolation on uniform maps some scaling limits closely related to the perimeters of finite clusters are also obtained

modelling complexity applications of gibbs sampling in medicine summary we review applications of gibbs sampling in medicine involving longitudinal spatial covariate measurement and survival models applications in immunology pharmacology transplantation cancer screening industrial epidemiology and genetic epidemiology are discussed

on nonparametric discrimination using density differences summary we propose a technique for nonparametric discrimination in which smoothing parameters are chosen jointly according to a criterion based on the difference between two densities the approach is suitable for categorical continuous and mixed data and uses information from both populations to determine the smoothing parameter for any one population in the case of categorical data optimal performance is sometimes achieved using negative smoothing parameters a property which does not emerge if the smoothing parameters are chosen individually

an effective method to determine whether a point is within a convex hull and its generalized convex polyhedron classifier abstract a convex polyhedron classifier that encloses the minority class using a combination of hyperplanes is potentially effective in imbalanced classification to construct an easy to use convex polyhedron classifier this paper first presents a theoretical foundation for determining whether a point is within the convex hull of a finite point set this foundation corresponds to a geometric method in which the result is expressed as a separating hyperplane if the given point and the given convex hull are located on either side of the learned hyperplane this indicates that the point is outside of the convex hull otherwise the conclusion that the point is within the convex hull can be obtained as a generalization of the geometric method a convex polyhedron classifier is further proposed for binary classification if two finite point sets are polyhedrally separable a series of hyperplanes can be learned as a combined piecewise linear classifier which surrounds a point set that is inside using a convex polyhedron and excludes the other point set that is outside experimental results on twelve real world datasets show that the proposed classifier is generally better than the other two piecewise linear classifiers moreover a comparison with several types of support vector machines confirms its competitiveness

a glimpse of the conformal structure of random planar maps we present a way to study the conformal structure of random planar maps the main idea is to explore the map along an sle schramm loewner evolution process of parameter kappa 6 and to combine the locality property of the sle6 together with the spatial markov property of the underlying lattice in order to get a non trivial geometric information we follow this path in the case of the conformal structure of random triangulations with a boundary

estimating the error rate of a prediction rule improvement on cross validation abstract we construct a prediction rule on the basis of some data and then wish to estimate the error rate of this rule in classifying future observations cross validation provides a nearly unbiased estimate using only the original data cross validation turns out to be related closely to the bootstrap estimate of the error rate this article has two purposes to understand better the theoretical basis of the prediction problem and to investigate some related estimators which seem to offer considerably improved estimation in small samples

a ν twin support vector machine ν tsvm classifier and its geometric algorithms in this paper a n twin support vector machine n tsvm is presented improving upon the recently proposed twin support vector machine tsvm this n tsvm introduces a pair of parameters n to control the bounds of the fractions of the support vectors and the error margins the theoretical analysis shows that this n tsvm can be interpreted as a pair of minimum generalized mahalanobis norm problems on two reduced convex hulls rchs based on the well known gilbert s algorithm a geometric algorithm for tsvm ga tsvm and its probabilistic speed up version named pga tsvm are presented computational results on several synthetic as well as benchmark datasets demonstrate the significant advantages of the proposed algorithms in terms of both computation complexity and classification accuracy

products of idempotents in regular rings ii abstract this paper characterizes products of idempotents in von neumann regular rings which are unit regular or right self injective for unit regular rings the minimum number of idempotents needed in such a product is determined thereby generalizing a result of c s ballantine products of idempotent matrices linear algebra appl 19 1978 81 86 in the case of a matrix with entries from a field

uniqueness and universality of the brownian map we consider a random planar map mn which is uniformly distributed over the class of all rooted q angulations with n faces we let mn be the vertex set of mn which is equipped with the graph distance dgr both when q 4 is an even integer and when q 3 there exists a positive constant cq such that the rescaled metric spaces mn cqn 1 4dgr converge in distribution in the gromov hausdorff sense toward a universal limit called the brownian map the particular case of triangulations solves a question of schramm

the structure of countably generated projective modules over regular rings abstract we prove that for every regular ring r there exists an isomorphism between the monoids of isomorphism classes of finitely generated projective right modules over the rings end r r r and rcfm r where the latter denotes the ring of countably infinite row and column finite matrices over r we use this result to give a precise description of the countably generated projective modules over simple regular rings and over regular rings satisfying s comparability

the brownian map is the scaling limit of uniform random plane quadrangulations we prove that uniform random quadrangulations of the sphere with n faces endowed with the usual graph distance and renormalized by n 1 4 converge as n in distribution for the gromov hausdorff topology to a limiting metric space we validate a conjecture by le gall by showing that the limit is up to a scale constant the so called brownian map which was introduced by marckert mokkadem and le gall as the most natural candidate for the scaling limit of many models of random plane maps the proof relies strongly on the concept of geodesic stars in the map which are configurations made of several geodesics that only share a common endpoint and do not meet elsewhere

a geometric nearest point algorithm for the efficient solution of the svm classification task geometric methods are very intuitive and provide a theoretically solid approach to many optimization problems one such optimization task is the support vector machine svm classification which has been the focus of intense theoretical as well as application oriented research in machine learning in this letter the incorporation of recent results in reduced convex hulls rchs to a nearest point algorithm npa leads to an elegant and efficient solution to the svm classification task with encouraging practical results to real world classification problems i e linear or nonlinear and separable or nonseparable

geometric algorithms to large margin classifier based on affine hulls the geometric framework for binary data classification problems provides an intuitive foundation for the comprehension and application of geometric optimization algorithms leading to practical solutions of real world classification problems in this paper some theoretical results on the candidate extreme points of the notion of reduced affine hull rah are introduced these results allow the existing nearest point algorithms to be directly applied to solve both separable and inseparable classification problems based on rahs successfully and efficiently as the practical applications of the new theoretical results the popular gilbert schlesinger kozinec and mitchell dem yanov malozemov algorithms are presented to solve binary classification problems in the context of the rah framework the theoretical analysis and some experiments show that the proposed methods successfully achieve significant performance

decision theoretic generalizations of the pac model for neural net and other learning applications we describe a generalization of the pac learning model that is based on statistical decision theory in this model the learner receives randomly drawn examples each example consisting of an instance x in x and an outcome y in y and tries to find a hypothesis h x a where h in h that specifies the appropriate action a in a to take for each instance x in order to minimize the expectation of a loss l y a here x y and a are arbitrary sets l is a real valued function and examples are generated according to an arbitrary joint distribution on x times y special cases include the problem of learning a function from x into y the problem of learning the conditional probability distribution on y given x regression and the problem of learning a distribution on x density estimation we give theorems on the uniform convergence of empirical loss estimates to true expected loss rates for certain hypothesis spaces h and show how this implies learnability with bounded sample size disregarding computational complexity as an application we give distribution independent upper bounds on the sample size needed for learning with feedforward neural networks our theorems use a generalized notion of vc dimension that applies to classes of real valued functions adapted from pollard s work and a notion of capacity and metric dimension for classes of functions that map into a bounded metric space supersedes 89 30 and 90 52 also in information and computation vol 100 no 1 september 1992

an iterative growing and pruning algorithm for classification tree design a critical issue in classification tree design obtaining right sized trees i e trees which neither underfit nor overfit the data is addressed instead of stopping rules to halt partitioning the approach of growing a large tree with pure terminal nodes and selectively pruning it back is used a new efficient iterative method is proposed to grow and prune classification trees this method divides the data sample into two subsets and iteratively grows a tree with one subset and prunes it with the other subset successively interchanging the roles of the two subsets the convergence and other properties of the algorithm are established theoretical and practical considerations suggest that the iterative free growing and pruning algorithm should perform better and require less computation than other widely used tree growing and pruning algorithms numerical results on a waveform recognition problem are presented to support this view

simple regular rings satisfying weak comparability abstract open problem 3 in goodearl s book on von neumann regular rings asks if a directly finite simple regular ring must be unit regular this paper gives a positive answer in the case where the ring satisfies a very weak comparability condition no directly finite simple regular rings are known to fail this comparability condition

linear convergence rate for the mdm algorithm for the nearest point problem in this paper we will prove a linear convergence rate for the extension of the mitchell dem yanov and malozemov mdm algorithm for solving the nearest point problem npp while linear convergence proofs for the related but different smo method intended for svm training require that the kernel matrix be positive definite no such assumption is needed in npp for mdm moreover we will also show linear convergence for the sequence of mdm vectors to the unique solution vector w of npp and for a quantity that measures the gap in the karush kuhn tucker conditions furthermore even if there might be several multiplier representations for w we will show that any mdm generated multiplier sequence converges linearly to an optimal multiplier this linear convergence is shown to be optimal and it is also numerically illustrated over six datasets we will follow an approach that relies on a geometric point of view that yields a simple path to the proofs highlightslinear convergence rate for the mdm algorithm is proved no particular assumption is needed for the proof a geometric and simple approach is followed this may pave the way for proving similar results for the smo algorithm used to train svms

penalized discriminant analysis fisher s linear discriminant analysis lda is a popular data analytic tool for studying the relationship between a set of predictors and a categorical response in this paper we describe a penalized version of lda it is designed for situations in which there are many highly correlated predictors such as those obtained by discretizing a function or the grey scale values of the pixels in a series of images in cases such as these it is natural efficient and sometimes essential to impose a spatial smoothness constraint on the coefficients both for improved prediction performance and interpretability we cast the classification problem into a regression framework via optimal scoring using this our proposal facilitates the use of any penalized regression technique in the classification setting the technique is illustrated with examples in speech recognition and handwritten character recognition

resistant fits for some commonly used logistic models with medical application logistic regression type models are used in many applications some examples include the classical dose response experiment prospective and retrospective studies of disease incidence with and without matching and the analysis of ordinal data in most instances the model is fitted by the method of maximum likelihood which like least squares is sensitive to atypical observations an alternative to maximum likelihood is proposed and illustrated by examples

density estimation with confidence sets exemplified by superclusters and voids in the galaxies abstract a method is presented for forming both a point estimate and a confidence set of semiparametric densities the final product is a three dimensional figure that displays a selection of density estimates for a plausible range of smoothing parameters the boundaries of the smoothing parameter are determined by a nonparametric goodness of fit test that is based on the sample spacings for each value of the smoothing parameter our estimator is selected by choosing the normal mixture that maximizes a function of the sample spacings a point estimate is selected from this confidence set by using the method of cross validation an algorithm to find the mixing distribution that maximizes the spacings functional is presented these methods are illustrated with a data set from the astronomy literature the measurements are velocities at which galaxies in the corona borealis region are moving away from our galaxy if the galaxies are clustered the velocity density will be multimodal with clusters correspond

on robustness in the logistic regression model summary we investigate robustness in the logistic regression model copas has studied two forms of robust estimator a robust resistant estimate of pregibon and an estimate based on a misclassification model he concluded that robust resistant estimates are much more biased in small samples than the usual logistic estimate is and recommends a bias corrected version of the misclassification estimate we show that there are other versions of robust resistant estimates which have bias often approximately the same as and sometimes even less than the logistic estimate these estimates belong to the mallows class in addition the corrected misclassification estimate is inconsistent at the logistic model we develop a simple consistent modification the modified estimate is a member of the mallows class but unlike most robust estimates it has an interpretable tuning constant the results are illustrated on data sets featuring different kinds of outliers

on a conjecture of huber concerning the convergence of projection pursuit regression we generalize the projection pursuit procedure of friedman and stuetzle abstract version and prove strong convergence this answers a question of huber

multiconlitron a general piecewise linear classifier based on the convexly separable concept we present a solid geometric theory and a new general framework to design piecewise linear classifiers for two arbitrarily complicated nonintersecting classes by using a multiconlitron which is a union of multiple conlitrons that comprise a set of hyperplanes or linear functions surrounding a convex region for separating two convexly separable datasets we propose a new iterative algorithm called the cross distance minimization algorithm cdma to compute hard margin non kernel support vector machines svms via the nearest point pair between two convex polytopes using cdma we derive two new algorithms i e the support conlitron algorithm sca and the support multiconlitron algorithm sma to construct support conlitrons and support multiconlitrons respectively which are unique and can separate two classes by a maximum margin as in an svm comparative experiments show that sma can outperform linear svm on many of the selected databases and provide similar results to radial basis function svm on some of them while sca performs better than linear svm on three out of four applicable databases other experiments show that sma and sca may be further improved to draw more potential in the new research direction of piecewise linear learning

statistical theory of learning curves under entropic loss criterion the present paper elucidates a universal property of learning curves which shows how the generalization error training error and the complexity of the underlying stochastic machine are related and how the behavior of a stochastic machine is improved as the number of training examples increases the error is measured by the entropic loss it is proved that the generalization error converges to h0 the entropy of the conditional distribution of the true machine as h0 m 2t while the training error converges as h0 m 2t where t is the number of examples and m shows the complexity of the network when the model is faithful implying that the true machine is in the model m is reduced to m the number of modifiable parameters this is a universal law because it holds for any regular machine irrespective of its structure under the maximum likelihood estimator similar relations are obtained for the bayes and gibbs learning algorithms these learning curves show the relation among the accuracy of learning the complexity of a model and the number of training examples

some asymptotic results for learning in single hidden layer feedforward network models abstract we investigate the properties of a recursive estimation procedure the method of back propagation for a class of nonlinear regression models single hidden layer feedforward network models recently developed by cognitive scientists the results follow from more general results for a class of recursive m estimators obtained using theorems of ljung 1977 and walk 1977 for the method of stochastic approximation conditions are given ensuring that the back propagation estimator converges almost surely to a parameter value that locally minimizes expected squared error loss provided the estimator does not diverge and that the back propagation estimator is asymptotically normal when centered at this minimizer this estimator is shown to be statistically inefficient and a two step procedure that has efficiency equivalent to that of nonlinear least squares is proposed practical issues are illustrated by a numerical example involving approximation of the henon map