hard or soft classification large margin unified machines margin based classifiers have been popular in both machine learning and statistics for classification problems among numerous classifiers some are hard classifiers while some are soft ones soft classifiers explicitly estimate the class conditional probabilities and then perform classification based on estimated probabilities in contrast hard classifiers directly target the classification decision boundary without producing the probability estimation these two types of classifiers are based on different philosophies and each has its own merits in this article we propose a novel family of large margin classifiers namely large margin unified machines lums which covers a broad range of margin based classifiers including both hard and soft ones by offering a natural bridge from soft to hard classification the lum provides a unified algorithm to fit various classifiers and hence a convenient platform to compare hard and soft classification both theoretical consistency and numerical performance of

compatibility conditions for time dependent partial differential equations and the rate of convergence of chebyshev and fourier spectral methods abstract compatibility conditions for partial differential equations pdes are an infinite set of relations between the initial conditions the pde and the boundary conditions which are necessary and sufficient for the solution to be c that is infinitely differentiable everywhere on the computational domain including the boundaries since the performance of chebyshev spectral and spectral element methods is dramatically reduced when the solution is not c one would expect that the compatibility conditions would be a major theme in the spectral literature instead it has been completely ignored therefore we pursue three goals here first we present a proof of the compatibility conditions in a simplified form that does not require functional analysis second we analyze the connection between the compatibility conditions and the rate of convergence of chebyshev methods lastly we describe strategies for slightly adjusting initial conditions so that the compatibility conditions are satisfied

sum accelerated pseudospectral methods finite differences and sech weighted differences this work continues our previous studies of algorithms for accelerating the convergence of pseudospectral derivative series in order to obtain new differentiation schemes which have the sparsity and low cost of finite differences but the accuracy of spectral methods we develop a general theoretical framework for difference schemes finite differences are close to optimum but can be bettered by a new scheme we have dubbed sech weighted differences through numerical examples we show that sech weighted differences are effective in contrast non linear accelerations like pade approximants and the levin u transform so popular in other applications are inferior for approximating derivatives to linear accelerations like finite differences sech weighted differences and the euler method

a unified formulation and fast accelerated proximal gradient method for classification binary classification is the problem of predicting the class a given sample belongs to to achieve a good prediction performance it is important to find a suitable model for a given dataset however it is often time consuming and impractical for practitioners to try various classification models because each model employs a different formulation and algorithm the difficulty can be mitigated if we have a unified formulation and an efficient universal algorithmic framework for various classification models to expedite the comparison of performance of different models for a given dataset in this paper we present a unified formulation of various classification models including c svm l2 svm svm mm fda mm mpm logistic regression distance weighted discrimination and develop a general optimization algorithm based on an accelerated proximal gradient apg method for the formulation we design various techniques such as backtracking line search and adaptive restarting strategy in order to speed up the practical convergence of our method we also give a theoretical convergence guarantee for the proposed fast apg algorithm numerical experiments show that our algorithm is stable and highly competitive to specialized algorithms designed for specific models e g sequential minimal optimization smo for svm

on the local convergence of semismooth newton methods for linear and nonlinear second order cone programs without strict complementarity the optimality conditions of a nonlinear second order cone program can be reformulated as a nonsmooth system of equations using a projection mapping this allows the application of nonsmooth newton methods for the solution of the nonlinear second order cone program conditions for the local quadratic convergence of these nonsmooth newton methods are investigated related conditions are also given for the special case of a linear second order cone program an interesting and important feature of these conditions is that they do not require strict complementarity of the solution some numerical results are included in order to illustrate the theoretical considerations

a lag averaged generalization of euler s method for accelerating series abstract euler s method is an efficient accelerator for an alternating series that is one whose terms a j are such that sign a j sign a j 1 however its effectiveness deteriorates rapidly as the period p of the oscillations in degree j increases to reduce this nonuniformity we introduce an algorithm which begins with the recursive computation of a triangular array of numbers the diagonal elements define a generalization of euler s method each horizontal row defines a delayed euler method which is even more effective the key element is that the array is generated by repeatedly averaging partial sums of the original series with a lag that is a distance in degree between the partial sums that are combined which optimally is half the period p of the oscillations in the series we illustrate the algorithm with the fourier series for a function with a shock wave jump discontinuity the terms a j 1 j sin jx vary from strictly alternating p 2 at x 0 to monotonic p as x the lag averaged euler scheme with and without delay is much more efficient than its classical counterpart especially near the discontinuities we present an analytical convergence theory for both the standard and generalized euler methods

the maximal data piling direction for discrimination we study a discriminant direction vector that generally exists only in high dimension low sample size settings projections of data onto this direction vector take on only two distinct values one for each class there exist infinitely many such directions in the subspace generated by the data but the maximal data piling vector has the longest distance between the projections this paper investigates mathematical properties and classification performance of this discrimination method

multicategory ψ learning in binary classification margin based techniques usually deliver high performance as a result a multicategory problem is often treated as a sequence of binary classifications in the absence of a dominating class this treatment may be suboptimal and may yield poor performance such as for support vector machines svms we propose a novel multicategory generalization of learning that treats all classes simultaneously the new generalization eliminates this potential problem while at the same time retaining the desirable properties of its binary counterpart we develop a statistical learning theory for the proposed methodology and obtain fast convergence rates for both linear and nonlinear learning examples we demonstrate the operational characteristics of this method through a simulation our results indicate that the proposed methodology can deliver accurate class prediction and is more robust against extreme observations than its svm counterpart

on reject and refine options in multicategory classification abstractin many real applications of statistical learning a decision made from misclassification can be too costly to afford in this case a reject option which defers the decision until further investigation is conducted is often preferred in recent years there has been much development for binary classification with a reject option yet little progress has been made for the multicategory case in this article we propose margin based multicategory classification methods with a reject option in addition and more importantly we introduce a new and unique refine option for the multicategory problem where the class of an observation is predicted to be from a set of class labels whose cardinality is not necessarily one the main advantage of both options lies in their capacity of identifying error prone observations moreover the refine option can provide more constructive information for classification by effectively ruling out implausible classes efficient implementations have been developed

another look at distance weighted discrimination summary distance weighted discrimination dwd is a modern margin based classifier with an interesting geometric motivation it was proposed as a competitor to the support vector machine svm despite many recent references on dwd dwd is far less popular than the svm mainly because of computational and theoretical reasons we greatly advance the current dwd methodology and its learning theory we propose a novel thrifty algorithm for solving standard dwd and generalized dwd and our algorithm can be several hundred times faster than the existing state of the art algorithm based on second order cone programming in addition we exploit the new algorithm to design an efficient scheme to tune generalized dwd furthermore we formulate a natural kernel dwd approach in a reproducing kernel hilbert space and then establish the bayes risk consistency of the kernel dwd by using a universal kernel such as the gaussian kernel this result solves an open theoretical problem in the dwd literature a comparison study on 16 benchmark data sets shows that data driven generalized dwd consistently delivers higher classification accuracy with less computation time than the svm

solution path for pin svm classifiers with positive and negative tau values applying the pinball loss in a support vector machine svm classifier results in pin svm the pinball loss is characterized by a parameter tau its value is related to the quantile level and different tau values are suitable for different problems in this paper we establish an algorithm to find the entire solution path for pin svm with different tau values this algorithm is based on the fact that the optimal solution to pin svm is continuous and piecewise linear with respect to tau we also show that the nonnegativity constraint on tau is not necessary i e tau can be extended to negative values first in some applications a negative tau leads to better accuracy second tau 1 corresponds to a simple solution that links svm and the classical kernel rule the solution for tau 1 can be obtained directly and then be used as a starting point of the solution path the proposed method efficiently traverses tau values through the solution path and then achieves good performance by a suitable tau in particular tau 0 corresponds to c svm meaning that the traversal algorithm can output a result at least as good as c svm with respect to validation error

generalized self concordant functions a recipe for newton type methods we study the smooth structure of convex functions by generalizing a powerful concept so called self concordance introduced by nesterov and nemirovskii in the early 1990s to a broader class of convex functions which we call generalized self concordant functions this notion allows us to develop a unified framework for designing newton type methods to solve convex optimization problems the proposed theory provides a mathematical tool to analyze both local and global convergence of newton type methods without imposing unverifiable assumptions as long as the underlying functionals fall into our class of generalized self concordant functions first we introduce the class of generalized self concordant functions which covers the class of standard self concordant functions as a special case next we establish several properties and key estimates of this function class which can be used to design numerical methods then we apply this theory to develop several newton type methods for solving a class of smooth convex optimization problems involving generalized self concordant functions we provide an explicit step size for a damped step newton type scheme which can guarantee a global convergence without performing any globalization strategy we also prove a local quadratic convergence of this method and its full step variant without requiring the lipschitz continuity of the objective hessian mapping then we extend our result to develop proximal newton type methods for a class of composite convex minimization problems involving generalized self concordant functions we also achieve both global and local convergence without additional assumptions finally we verify our theoretical results via several numerical examples and compare them with existing methods

complementarity and nondegeneracy in semidefinite programming primal and dual nondegeneracy conditions are defined for semidefinite programming given the existence of primal and dual solutions it is shown that primal nondegeneracy implies a unique dual solution and that dual nondegeneracy implies a unique primal solution the converses hold if strict complementarity is assumed primal and dual nondegeneracy assumptions do not imply strict complementarity as they do in lp the primal and dual nondegeneracy assumptions imply a range of possible ranks for primal and dual solutions x and z this is in contrast with lp where nondegeneracy assumptions exactly determine the number of variables which are zero it is shown that primal and dual nondegeneracy and strict complementarity all hold generically numerical experiments suggest probability distributions for the ranks of x and z which are consistent with the nondegeneracy conditions

full stability of locally optimal solutions in second order cone programs the paper presents complete characterizations of lipschitzian full stability of locally optimal solutions to second order cone programs socps expressed entirely in terms of their initial data these characterizations are obtained via appropriate versions of the quadratic growth and strong second order sufficient conditions under the corresponding constraint qualifications we also establish close relationships between full stability of local minimizers for socps and strong regularity of the associated generalized equations at nondegenerate points our approach is mainly based on advanced tools of second order variational analysis and generalized differentiation

partial smoothness tilt stability and generalized hessians we compare two recent variational analytic approaches to second order conditions and sensitivity analysis for nonsmooth optimization we describe a broad setting where computing the generalized hessian of mordukhovich is easy in this setting the idea of tilt stability introduced by poliquin and rockafellar is equivalent to a classical smooth second order condition

incorporating prior probabilities into high dimensional classifiers in standard parametric classifiers or classifiers based on nonparametric methods but where there is an opportunity for estimating population densities the prior probabilities of the respective populations play a key role however those probabilities are largely ignored in the construction of high dimensional classifiers partly because there are no likelihoods to be constructed or bayes risks to be estimated nevertheless including information about prior probabilities can reduce the overall error rate particularly in cases where doing so is most important i e when the classification problem is particularly challenging and error rates are not close to zero in this paper we suggest a general approach to reducing error rate in this way by using a method derived from breiman s bagging idea the potential improvements in performance are identified in theoretical and numerical work the latter involving both applications to real data and simulations the method is simple and explicit to apply and does not involve choice of any tuning parameters

convexity classification and risk bounds many of the classification algorithms developed in the machine learning literature including the support vector machine and boosting can be viewed as minimum contrast methods that minimize a convex surrogate of the 0 1 loss function the convexity makes these algorithms computationally efficient the use of a surrogate however has statistical consequences that must be balanced against the computational virtues of convexity to study these issues we provide a general quantitative relationship between the risk as assessed using the 0 1 loss and the risk as assessed using any nonnegative surrogate loss function we show that this relationship gives nontrivial upper bounds on excess risk under the weakest possible condition on the loss function that it satisfies a pointwise form of fisher consistency for classification the relationship is based on a simple variational transformation of the loss function that is easy to compute in many applications we also present a refined version of this result in the

bidirectional discrimination with application to data visualization linear classifiers are very popular but can have limitations when classes have distinct subpopulations general nonlinear kernel classifiers are very flexible but do not give clear interpretations and may not be efficient in high dimensions we propose the bidirectional discrimination classification method which generalizes linear classifiers to two or more hyperplanes this new family of classification methods gives much of the flexibility of a general nonlinear classifier while maintaining the interpretability and much of the parsimony of linear classifiers they provide a new visualization tool for high dimensional low sample size data although the idea is generally applicable we focus on the generalization of the support vector machine and distance weighted discrimination methods the performance and usefulness of the proposed method are assessed using asymptotics and demonstrated through analysis of simulated and real data our method leads to better classification performance in high dimensional situations where subclusters are present in the data

machine learning with data dependent hypothesis classes we extend the vc theory of statistical learning to data dependent spaces of classifiers this theory can be viewed as a decomposition of classifier design into two components the first component is a restriction to a data dependent hypothesis class and the second is empirical risk minimization within that class we define a measure of complexity for data dependent hypothesis classes and provide data dependent versions of bounds on error deviance and estimation error we also provide a structural risk minimization procedure over data dependent hierarchies and prove consistency we use this theory to provide a framework for studying the trade offs between performance and computational complexity in classifier design as a consequence we obtain a new family of classifiers with dimension independent performance bounds and efficient learning procedures

applications of second order cone programming in a second order cone program socp a linear function is minimized over the intersection of an affine set and the product of second order quadratic cones socps are nonlinear convex problems that include linear and convex quadratic programs as special cases but are less general than semidefinite programs sdps several efficient primaldual interior point methods for socp have been developed in the last few years after reviewing the basic theory of socps we describe general families of problems that tan be recast as socps these include robust linear programming and robust leastsquares problems problems involving sums or maxima of norms or with convex hyperbolic constraints we discuss a variety of engineering applications such as filter design antenna array weight design truss design and grasping forte optimization in robotics we describe an efficient primaldual interior point method for solving socps which shares many of the features of primaldual interior point methods for linear program

second order optimality conditions based on parabolic second order tangent sets in this paper we discuss second order optimality conditions in optimization problems subject to abstract constraints our analysis is based on various concepts of second order tangent sets and parametric duality we introduce a condition called second order regularity under which there is no gap between the corresponding second order necessary and second order sufficient conditions we show that the second order regularity condition always holds in the case of semidefinite programming

acceleration of algebraically converging fourier series when the coefficients have series in powers of 1 n if the coefficients in a fourier cosine series f x f email protected n 0 a ncos nx decrease as a small negative power of n then one may need millions of terms to sum the series to high accuracy we show that if the a n are known analytically and have a power series in 1 n then it is straightforward to approximate f x as a series of what we shall the lanczos krylov lk functions we describe the similar methodology for sine series general fourier series are merely the sum of a cosine series with a sine series and thus are implicitly handled too for cosine coefficients that involve only even powers of n and sine coefficients that are functions of odd powers of n the lk functions may be expressed in terms of bernoulli polynomials the lk functions for cosine coefficients involving odd powers of n and for sine coefficients in even powers of n are not known explicitly these are also known as clausen functions we provide rapidly convergent series to compute these clausen functions to high accuracy our method includes the endpoint subtraction ideas of lanczos and krylov but is more general the sum n 1 1 n 1 1 email protected cos nx where l 0 is a constant arises in phase transitions in absorbed monolayers on metal surfaces it is easily summed by our method which correctly incorporates the logarithmic singularities at x email protected

perturbation analysis of second order cone programming problems we discuss first and second order optimality conditions for nonlinear second order cone programming problems and their relation with semidefinite programming problems for doing this we extend in an abstract setting the notion of optimal partition then we state a characterization of strong regularity in terms of second order optimality conditions this is the first time such a characterization is given for a nonpolyhedral conic problem

weighted distance weighted discrimination and its asymptotic properties while distance weighted discrimination dwd is an appealing approach to classification in high dimensions it was designed for balanced datasets in the case of unequal costs biased sampling or unbalanced data there are major improvements available using appropriately weighted versions of dwd wdwd a major contribution of this paper is the development of optimal weighting schemes for various nonstandard classification problems in addition we discuss several alternative criteria and propose an adaptive weighting scheme awdwd and demonstrate its advantages over nonadaptive weighting schemes under some situations the second major contribution is a theoretical study of weighted dwd both high dimensional low sample size asymptotics and fisher consistency of dwd are studied the performance of weighted dwd is evaluated using simulated examples and two real data examples the theoretical results are also confirmed by simulations

second order cone programming second order cone programming socp problems are convex optimization problems in which a linear function is minimized over the intersection of an affine linear manifold with the cartesian product of second order lorentz cones linear programs convex quadratic programs and quadratically constrained convex quadratic programs can all be formulated as socp problems as can many other problems that do not fall into these three categories these latter problems model applications from a broad range of fields from engineering control and finance to robust optimization and combinatorial optimization on the other hand semidefinite programming sdp that is the optimization problem over the intersection of an affine set and the cone of positive semidefinite matrices includes socp as a special case therefore socp falls between linear lp and quadratic qp programming and sdp like lp qp and sdp problems socp problems can be solved in polynomial time by interior point methods the computational effort per iteration required by these methods to solve socp problems is greater than that required to solve lp and qp problems but less than that required to solve sdp s of similar size and structure because the set of feasible solutions for an socp problem is not polyhedral as it is for lp and qp problems it is not readily apparent how to develop a simplex or simplex like method for socp while socp problems can be solved as sdp problems doing so is not advisable both on numerical grounds and computational complexity concerns for instance many of the problems presented in the survey paper of vandenberghe and boyd vb96 as examples of sdps can in fact be formulated as socps and should be solved as such in 2 3 below we give socp formulations for four of these examples the convex quadratically constrained quadratic programming qcqp problem problems involving fractional quadratic functions rutcor rutgers university e mail alizadeh rutcor rutgers edu research supported in part by the u s national science foundation grant ccr 9901991 ieor columbia university e mail gold ieor columbia edu research supported in part by the department of energy grant de fg02 92er25126 national science foundation grants dms 94 14438 cda 97 26385 and dms 01 04282

scale adjustments for classifiers in high dimensional low sample size settings distance based classifiers are generally considered to be effective at discriminating between populations that differ in location indeed nearest neighbour methods and the support vector machine are frequently used in very high dimensional problems involving gene expression data where it is believed that elevated levels of expression convey much of the information for classification however one problem inherent to distance based classifiers is that scale differences can mask location differences in consequence such classifiers can have poor performance if the information for classification accumulates through a large number of relatively small location differences in data components rather than via large differences in this paper we show that a simple adjustment for scale applicable to a variety of distance based classifiers can remedy the problem for some classifiers such as those based on the support vector machine or the centroid method scale corrections are important primarily in the case of small training sample sizes however for other classifiers including those based on nearest neighbour and average distance methods scale adjustments are helpful more generally

asymptotic behavior of support vector machine for spiked population model for spiked population model we investigate the large dimension n and large sample size m asymptotic behavior of the support vector machine svm classification method in the limit of n m at fixed m n we focus on the generalization performance by analytically evaluating the angle between the normal direction vectors of svm separating hyperplane and corresponding bayes optimal separating hyperplane this is an analogous result to the one shown in paul 2007 and nadler 2008 for the angle between the sample eigenvector and the population eigenvector in random matrix theorem we provide not just bound but sharp prediction of the asymptotic behavior of svm that can be determined by a set of nonlinear equations based on the analytical results we propose a new method of selecting tuning parameter which significantly reduces the computational cost a surprising finding is that svm achieves its best performance at small value of the tuning parameter under spiked population model these results are confirmed to be correct by comparing with those of numerical simulations on finite size systems we also apply our formulas to an actual dataset of breast cancer and find agreement between analytical derivations and numerical computations based on cross validation

distance weighted discrimination high dimension low sample size statistical analysis is becoming increasingly important in a wide range of applied contexts in such situations the popular support vector machine suffers from data piling at the margin which can diminish generalizability this leads naturally to the development of distance weighted discrimination which is based on second order cone programming a modern computationally intensive optimization method

numerical comparisons of nonlinear convergence accelerators as part of a continuing program of numerical tests of convergence accelerators we have compared the iterated aitken s a2 method wynn s algorithm brezinski s 0 algorithm and levin s u transform on a broad range of test problems linearly convergence alternating monotone and irregular sign series logarithmically convergent series power method and bernoulli method sequences alternating and monotone asymptotic series and some perturba tion series arising in applications in each category either the algorithm or the u transform gives the best results of the four methods tested in some cases differences among methods are slight and in others they are quite striking 1 introduction we reported in 19 on a numerical test program that compared eleven different scalar convergence accelerators on a broad range of infinite series problems in an attempt to find an across the board accelerator that would handle both linear and logarithmic convergence and both alternating and monotone series or sequences the best candidates to emerge from that study were levin s u transform 14 and brezinski s 0 algorithm 6 we also provided some theoretical support for that conclusion in terms of the wide range of series summed exactly by these methods and by proving that 02 and the levin transforms all accelerate linear convergence the other levin transforms t and v produced essentially equivalent numerical results to u except that t fails on logarithmic convergence we have continued the test program started with 19 to include monotone linearly convergent series series with irregular sign patterns specifically fourier series and sequences whose terms do not depend explicitly on the previous term and or the index e g bernoulli s method since convergence accelerators are often known to transform divergent series to convergent ones we have also included in our test program divergent power series asymptotic series and a few perturbation series the methods considered in this paper are the u and 0 algorithms noted above and

analysis of principal nested spheres a general framework for a novel non geodesic decomposition of high dimensional spheres or high dimensional shape spaces for planar landmarks is discussed the decomposition principal nested spheres leads to a sequence of submanifolds with decreasing intrinsic dimensions which can be interpreted as an analogue of principal component analysis in a number of real datasets an apparent one dimensional mode of variation curving through more than one geodesic component is captured in the one dimensional component of principal nested spheres while analysis of principal nested spheres provides an intuitive and flexible decomposition of the high dimensional sphere an interesting special case of the analysis results in finding principal geodesics similar to those from previous approaches to manifold principal component analysis an adaptation of our method to kendall s shape space is discussed and a computational algorithm for fitting principal nested spheres is proposed the result provides a coordinate system to visualize the data structure and an intuitive summary of principal modes of variation as exemplified by several datasets

a statistical approach to set classification by feature selection with applications to classification of histopathology images summary set classification problems arise when classification tasks are based on sets of observations as opposed to individual observations in set classification a classification rule is trained with n sets of observations where each set is labeled with class information and the prediction of a class label is performed also with a set of observations data sets for set classification appear for example in diagnostics of disease based on multiple cell nucleus images from a single tissue relevant statistical models for set classification are introduced which motivate a set classification framework based on context free feature extraction by understanding a set of observations as an empirical distribution we employ a data driven method to choose those features which contain information on location and major variation in particular the method of principal component analysis is used to extract the features of major variation multidimensional scaling is used to represent features as vector valued points on which conventional classifiers can be applied the proposed set classification approaches achieve better classification results than competing methods in a number of simulated data examples the benefits of our method are demonstrated in an analysis of histopathology images of cell nuclei related to liver cancer

solving semidefinite quadratic linear programs using sdpt3 this paper discusses computational experiments with linear optimization problems involving semidefinite quadratic and linear cone constraints sqlps many test problems of this type are solved using a new release of sdpt3 a matlab implementation of infeasible primal dual path following algorithms the software developed by the authors uses mehrotra type predictor corrector variants of interior point methods and two types of search directions the hkm and nt directions a discussion of implementation details is provided and computational results on problems from the sdplib and dimacs challenge collections are reported

second order variational analysis in second order cone programming the paper conducts a second order variational analysis for an important class of nonpolyhedral conic programs generated by the so called second order lorentz ice cream cone mathcal q from one hand we prove that the indicator function of mathcal q is always twice epi differentiable and apply this result to characterizing the uniqueness of lagrange multipliers together with an error bound estimate in the general second order cone programming setting involving twice differentiable data on the other hand we precisely calculate the graphical derivative of the normal cone mapping to mathcal q under the metric subregularity constraint qualification and then give an application of the latter result to a complete characterization of isolated calmness for perturbed variational systems associated with second order cone programs the obtained results seem to be the first in the literature in these directions for nonpolyhedral problems without imposing any nondegeneracy assumptions

a survey of the s lemma in this survey we review the many faces of the s lemma a result about the correctness of the s procedure the basic idea of this widely used method came from control theory but it has important consequences in quadratic and semidefinite optimization convex geometry and linear algebra as well these were all active research areas but as there was little interaction between researchers in these different areas their results remained mainly isolated here we give a unified analysis of the theory by providing three different proofs for the s lemma and revealing hidden connections with various areas of mathematics we prove some new duality results and present applications from control theory error estimation and computational geometry

variational analysis and full stability of optimal solutions to constrained and minimax problems abstract the main goal of this paper is to develop applications of advanced tools of first order and second order variational analysis and generalized differentiation to the fundamental notion of full stability of local minimizers of general classes of constrained optimization and minimax problems in particular we derive second order characterizations of full stability and investigate its relationships with other notions of stability for parameterized conic programs and minimax problems furthermore the developed variational approach allows us to largely unify and provide new self contained proofs of some quite recent results in this direction for problems of constrained optimization with c 2 smooth data

flexible high dimensional classification machines and their asymptotic properties classification is an important topic in statistics and machine learning with great potential in many real applications in this paper we investigate two popular large margin classification methods support vector machine svm and distance weighted discrimination dwd under two contexts the high dimensional low sample size data and the imbalanced data a unified family of classification machines the flexible assortment machine flame is proposed within which dwd and svm are special cases the flame family helps to identify the similarities and differences between svm and dwd it is well known that many classifiers overfit the data in the high dimensional setting and others are sensitive to the imbalanced data that is the class with a larger sample size overly influences the classifier and pushes the decision boundary towards the minority class svm is resistant to the imbalanced data issue but it overfits high dimensional data sets by showing the undesired data piling phenomenon the dwd method was proposed to improve svm in the high dimensional setting but its decision boundary is sensitive to the imbalanced ratio of sample sizes our flame family helps to understand an intrinsic connection between svm and dwd and provides a trade off between sensitivity to the imbalanced data and overfitting the high dimensional data several asymptotic properties of the flame classifiers are studied simulations and real data applications are investigated to illustrate theoretical findings

trigonometric approximation of periodic functions belonging to lip ω t p class analysis of signals or time functions are of great importance because it convey information or attributes of some phenomenon in this paper we consider the functions signals defined in terms of their integral modulus of continuity and approximate them by matrix means of their trigonometric fourier series the error of approximation obtained in this paper depends on the integral modulus of continuity of the signal we also discuss an example to show the application of the result in the situation when the fourier series of the signal has gibbs phenomenon we also discuss some corollaries of our theorems

variable selection for model based high dimensional clustering and its application to microarray data variable selection in high dimensional clustering analysis is an important yet challenging problem in this article we propose two methods that simultaneously separate data points into similar clusters and select informative variables that contribute to the clustering our methods are in the framework of penalized model based clustering unlike the classical l1 norm penalization the penalty terms that we propose make use of the fact that parameters belonging to one variable should be treated as a natural group numerical results indicate that the two new methods tend to remove noninformative variables more effectively and provide better clustering results than the l1 norm approach

theoretical measures of relative performance of classifiers for high dimensional data with small sample sizes summary we suggest a technique related to the concept of detection boundary that was developed by ingster and by donoho and jin for comparing the theoretical performance of classifiers constructed from small training samples of very large vectors the resulting classification boundaries are obtained for a variety of distance based methods including the support vector machine distance weighted discrimination and kth nearest neighbour classifiers for thresholded forms of those methods and for techniques based on donoho and jin s higher criticism approach to signal detection assessed in these terms standard distance based methods are shown to be capable only of detecting differences between populations when those differences can be estimated consistently however the thresholded forms of distance based classifiers can do better and in particular can correctly classify data even when differences between distributions are only detectable not estimable other methods including higher criticism classifiers can on occasion perform better still but they tend to be more limited in scope requiring substantially more information about the marginal distributions moreover as tail weight becomes heavier the classification boundaries of methods designed for particular distribution types can converge to and achieve the boundary for thresholded nearest neighbour approaches for example although higher criticism has a lower classification boundary and in this sense performs better in the case of normal data the boundaries are identical for exponentially distributed data when both sample sizes equal 1

geometric representation of high dimension low sample size data summary high dimension low sample size data are emerging in various areas of science we find a common structure underlying many such data sets by using a non standard type of asymptotics the dimension tends to while the sample size is fixed our analysis shows a tendency for the data to lie deterministically at the vertices of a regular simplex essentially all the randomness in the data appears only as a random rotation of this simplex this geometric representation is used to obtain several new statistical insights

classification methods with reject option based on convex risk minimization in this paper we investigate the problem of binary classification with a reject option in which one can withhold the decision of classifying an observation at a cost lower than that of misclassification since the natural loss function is non convex so that empirical risk minimization easily becomes infeasible the paper proposes minimizing convex risks based on surrogate convex loss functions a necessary and sufficient condition for infinite sample consistency both risks share the same minimizer is provided moreover we show that the excess risk can be bounded through the excess surrogate risk under appropriate conditions these bounds can be tightened by a generalized margin condition the impact of the results is illustrated on several commonly used surrogate loss functions

pseudopower expansion of solutions of generalized equations and constrained optimization problems we show that the solution of a strongly regular generalized equation subject to a scalar perturbation expands in pseudopower series in terms of the perturbation parameter i e the expansion of orderk is the solution of generalized equations expanded to orderk and thus depends itself on the perturbation parameter in the polyhedral case this expansion reduces to a usual taylor expansion these results are applied to the problem of regular perturbation in constrained optimization we show that if the strong regularity condition is satisfied the property of quadratic growth holds and at least locally the solutions of the optimization problem and of the associated optimality system coincide if in addition the number of inequality constraints is finite the solution and the lagrange multiplier can be expanded in taylor series if the data are analytic the solution and the multiplier are analytic functions of the perturbation parameter

towards a holistic cortical thickness descriptor heat kernel based grey matter morphology signatures in this paper we propose a heat kernel based regional shape descriptor that may be capable of better exploiting volumetric morphological information than other available methods thereby improving statistical power on brain magnetic resonance imaging mri analysis the mechanism of our analysis is driven by the graph spectrum and the heat kernel theory to capture the volumetric geometry information in the constructed tetrahedral meshes in order to capture profound brain grey matter shape changes we first use the volumetric laplace beltrami operator to determine the point pair correspondence between white grey matter and csf grey matter boundary surfaces by computing the streamlines in a tetrahedral mesh secondly we propose multi scale grey matter morphology signatures to describe the transition probability by random walk between the point pairs which reflects the inherent geometric characteristics thirdly a point distribution model is applied to reduce the dimensionality of the grey matter morphology signatures and generate the internal structure features with the sparse linear discriminant analysis we select a concise morphology feature set with improved classification accuracies in our experiments the proposed work outperformed the cortical thickness features computed by freesurfer software in the classification of alzheimer s disease and its prodromal stage i e mild cognitive impairment on publicly available data from the alzheimer s disease neuroimaging initiative the multi scale and physics based volumetric structure feature may bring stronger statistical power than some traditional methods for mri based grey matter morphology analysis

a method of analytic centers for quadratically constrained convex quadratic programs an interior point method is developed for maximizing a concave quadratic function under convex quadratic constraints the algorithm constructs a sequence of nested convex sets and finds their approximate centers using a partial newton step given the first convex set and its approximate center the total arithmetic operations required to converge to an approximate solution are of order o sqrt m m n n 2 ln varepsilon where m is the number of constraints n is the number of variables and varepsilon is determined by the desired tolerance of the optimal value and the size of the first convex set a method to initialize the algorithm is also proposed so that the algorithm can start from an arbitrary perhaps infeasible point

second order cone programming approaches for handling missing and uncertain data we propose a novel second order cone programming formulation for designing robust classifiers which can handle uncertainty in observations similar formulations are also derived for designing regression functions which are robust to uncertainties in the regression setting the proposed formulations are independent of the underlying distribution requiring only the existence of second order moments these formulations are then specialized to the case of missing values in observations for both classification and regression problems experiments show that the proposed formulations outperform imputation

a proof that the discrete singular convolution dsc lagrange distributed approximating function ldaf method is inferior to high order finite differences finite differences approximate the mth derivative of a function u x by a series j n nd j m u x j where the x j are the grid points the closely related discrete singular convolution dsc and lagrange distributed approximating function ldaf methods treated here as a single algorithm approximate derivatives in the same way as finite differences but with different numerical weights that depend upon a free parameter a by means of fourier analysis and error theorems we show that the dsc is worse than the standard finite differences in differentiating exp ikx for all k when a a f d where a f d 1 n 1 with n as the stencil width is the value of the dsc parameter that makes its weights most closely resemble those of finite differences for a

object oriented data analysis of cell images this article discusses a study of cell images in cell culture biology from an object oriented point of view the motivation of this research is to develop a statistical approach to cell image analysis that better supports the automated development of stem cell growth media a major hurdle in this process is the need for human expertise based on studying cells under the microscope to make decisions about the next step of the cell culture process we aim to use digital imaging technology coupled with statistical analysis to tackle this important problem the discussion in this article highlights a common critical issue choice of data objects instead of conventionally treating either the individual cells or the wells a container in which the cells are grown as data objects a new type of data object is proposed that is the union of a well with its corresponding set of cells the image data analysis suggests that the cell well unions can be a better choice of data objects than the cells or the wells al

generalized differentiation of piecewise linear functions in second order variational analysis abstract the paper is devoted to a comprehensive second order study of a remarkable class of convex extended real valued functions that is highly important in many aspects of nonlinear and variational analysis specifically those related to optimization and stability this class consists of lower semicontinuous functions with possibly infinite values on finite dimensional spaces which are labeled as piecewise linear ones and can be equivalently described via the convexity of their epigraphs in this paper we calculate the second order subdifferentials generalized hessians of arbitrary convex piecewise linear functions together with the corresponding geometric objects entirely in terms of their initial data the obtained formulas allow us in particular to justify a new exact equality type second order sum rule for such functions in the general nonsmooth setting

the strong second order sufficient condition and constraint nondegeneracy in nonlinear semidefinite programming and their implications for a locally optimal solution to the nonlinear semidefinite programming problem under robinson s constraint qualification the following conditions are proved to be equivalent the strong second order sufficient condition and constraint nondegeneracy the nonsingularity of clarke s jacobian of the karush kuhn tucker system the strong regularity of the karush kuhn tucker point and others

the high dimension low sample size geometric representation holds under mild conditions summary high dimension low small sample size datasets have different geometrical properties from those of traditional low dimensional data in their asymptotic study regarding increasing dimensionality with a fixed sample size hall et al 2005 showed that each data vector is approximately located on the vertices of a regular simplex in a high dimensional space a perhaps unappealing aspect of their result is the underlying assumption which requires the variables viewed as a time series to be almost independent we establish an equivalent geometric representation under much milder conditions using asymptotic properties of sample covariance matrices we discuss implications of the results such as the use of principal component analysis in a high dimensional space extension to the case of nonindependent samples and also the binary classification problem

on the aubin property of critical points to perturbed second order cone programs two gaps were found in the proof of the main theorems theorems 21 and 26 of the paper on the aubin property of critical points to perturbed second order cone programs siam j optim 21 2011 3 pp 798 823 by j v outrata and h ramirez c in this note both these gaps will be filled as to the second one a new technical result will be employed which may possibly be used also in other situations

hyperdisk based large margin classifier we introduce a large margin linear binary classification framework that approximates each class with a hyperdisk the intersection of the affine support and the bounding hypersphere of its training samples in feature space and then finds the linear classifier that maximizes the margin separating the two hyperdisks we contrast this with support vector machines svms which find the maximum margin separator of the pointwise convex hulls of the training samples arguing that replacing convex hulls with looser convex class models such as hyperdisks provides safer margin estimates that improve the accuracy on some problems both the hyperdisks and their separators are found by solving simple quadratic programs the method is extended to nonlinear feature spaces using the kernel trick and multi class problems are dealt with by combining binary classifiers in the same ways as for svms experiments on a range of data sets show that the method compares favourably with other popular large margin classifiers

optimal filter and mollifier for piecewise smooth spectral data we discuss the reconstruction of piecewise smooth data from its pseudo spectral information spectral projections enjoy superior resolution provided the function is globally smooth while the presence of jump discontinuities is responsible for spurious o 1 gibbs oscillations in the neighborhood of edges and an overall deterioration of the convergence rate to the unacceptable first order classical filters and mollifiers are constructed to have compact support in the fourier frequency and physical time spaces respectively and are dilated by the projection order or the width of the smooth region to maintain this compact support in the appropriate region here we construct a noncompactly supported filter and mollifier with optimal joint time frequency localization for a given number of vanishing moments resulting in a new fundamental dilation relationship that adaptively links the time and frequency domains not giving preference to either space allows for a more balanced error decomposition which when minimized yields an optimal filter and mollifier that retain the robustness of classical filters yet obtain true exponential accuracy