hard or soft classification large margin unified machines margin based classifiers have been popular in both machine learning and statistics for classification problems among numerous classifiers some are hard classifiers while some are soft ones soft classifiers explicitly estimate the class conditional probabilities and then perform classification based on estimated probabilities in contrast hard classifiers directly target the classification decision boundary without producing the probability estimation these two types of classifiers are based on different philosophies and each has its own merits in this article we propose a novel family of large margin classifiers namely large margin unified machines lums which covers a broad range of margin based classifiers including both hard and soft ones by offering a natural bridge from soft to hard classification the lum provides a unified algorithm to fit various classifiers and hence a convenient platform to compare hard and soft classification both theoretical consistency and numerical performance of

adaptive weighted learning for unbalanced multicategory classification summary in multicategory classification standard techniques typically treat all classes equally this treatment can be problematic when the dataset is unbalanced in the sense that certain classes have very small class proportions compared to others the minority classes may be ignored or discounted during the classification process due to their small proportions this can be a serious problem if those minority classes are important in this article we study the problem of unbalanced classification and propose new criteria to measure classification accuracy moreover we propose three different weighted learning procedures two one step weighted procedures as well as one adaptive weighted procedure we demonstrate the advantages of the new procedures using multicategory support vector machines through simulated and real datasets our results indicate that the proposed methodology can handle unbalanced classification problems effectively

the effects of random and discrete sampling when estimating continuous time diffusions high frequency financial data are not only discretely sampled in time but the time separating successive observations is often random we analyze the consequences of this dual feature of the data when estimating a continuous time model in particular we measure the additional effects of the randomness of the sampling intervals over and beyond those due to the discreteness of the data we also examine the effect of simply ignoring the sampling randomness we find that in many situations the randomness of the sampling has a larger impact than the discreteness of the data

fast rates for support vector machines using gaussian kernels for binary classification we establish learning rates up to the order of n 1 for support vector machines svms with hinge loss and gaussian rbf kernels these rates are in terms of two assumptions on the considered distributions tsybakov s noise assumption to establish a small estimation error and a new geometric noise condition which is used to bound the approximation error unlike previously proposed concepts for bounding the approximation error the geometric noise assumption does not employ any smoothness assumption 1 introduction in recent years support vector machines svms have been the subject of many theoretical considerations despite this effort their learning performance on restricted classes of distributions is still widely unknown in particular it is unknown under which nontrivial circumstances svms can guarantee fast learning rates the aim of this work is to use concepts like tsybakov s noise assumption and local rademacher averages to establish learning rates up to the order of n 1 for nontrivial distributions in addition to these concepts that are used to deal with the stochastic part of the analysis we also introduce a geometric assumption for distributions that allows us to estimate the approximation properties of gaussian rbf kernels unlike many other concepts introduced for bounding the approximation error our geometric assumption is not in terms of smoothness but describes the concentration and the noisiness of the data generating distribution near the decision boundary let us formally introduce the statistical classification problem to this end let us fix a subset x r d we write y 1 1 given a finite training set

enhancement of cooperation in prisoner s dilemma game on weighted lattices abstract we introduce the vertex weight into the spatial prisoners dilemma game to investigate the evolution of cooperation each player on a square lattice is assigned to a particular weight followed by three types of distributions which include the exponential power law and uniform ones compared with the traditional version we find that the cooperation level is markedly enhanced under the weighted square lattice for most ranges of b the highest cooperation level can be obtained under the uniform distribution while power law distribution usually leads to the lowest cooperation the distributed weight can produce a heavy heterogeneity among the individuals payoff some cooperators with higher weight will foster the cooperative clusters and even spread the cooperation strategy around the clusters while defectors have no such advantages in addition we still investigate the impact of the amplitude of undulation of weight distribution on the cooperation and the non monotonic behavior about b is observed finally the influence of noise on the cooperation is also studied for these types of distribution of weight to some extent our weighted scheme can characterize the difference or diversity of players which will be beneficial to further understand the role of individuals during the evolution of cooperation

multicategory support vector machines two category support vector machines svm have been very popular in the machine learning community for classification problems solving multicategory problems by a series of binary classifiers is quite common in the svm paradigm however this approach may fail under various circumstances we propose the multicategory support vector machine msvm which extends the binary svm to the multicategory case and has good theoretical properties the proposed method provides a unifying framework when there are either equal or unequal misclassification costs as a tuning criterion for the msvm an approximate leave one out cross validation function called generalized approximate cross validation is derived analogous to the binary case the effectiveness of the msvm is demonstrated through the applications to cancer classification using microarray data and cloud classification with satellite radiance profiles

effects of strategy migration direction and noise in the evolutionary spatial prisoner s dilemma spatial games are crucial for understanding patterns of cooperation in nature and to some extent society they are known to be more sensitive to local symmetries than e g spin models this paper concerns the evolution of the prisoner s dilemma game on regular lattices with three different types of neighborhoods char22 the von neumann moore and kagom e types we investigate two kinds of dynamics for the players to update their strategies that can be unconditional cooperator or defector depending on the payoff difference an individual can adopt the strategy of a random neighbor a voter model like dynamics vmld or impose its strategy on a random neighbor i e invasion process like dynamics ipld in particular we focus on the effects of noise in combination with the strategy dynamics on the evolution of cooperation we find that vmld compared to ipld better supports the spreading and sustaining of cooperation we see that noise has nontrivial effects on the evolution of cooperation maximum cooperation density can be realized either at a medium noise level in the limit of zero noise or in both these regions the temptation to defect and the local interaction structure determine the outcome especially in the low noise limit the local interaction plays a crucial role in determining the fate of cooperators we elucidate these both by numerical simulations and mean field cluster approximation methods

the maximal data piling direction for discrimination we study a discriminant direction vector that generally exists only in high dimension low sample size settings projections of data onto this direction vector take on only two distinct values one for each class there exist infinitely many such directions in the subspace generated by the data but the maximal data piling vector has the longest distance between the projections this paper investigates mathematical properties and classification performance of this discrimination method

memory based snowdrift game on networks we present a memory based snowdrift game mbsg taking place on networks we found that when a lattice is taken to be the underlying structure the transition of spatial patterns at some critical values of the payoff parameter is observable for both four and eight neighbor lattices the transition points as well as the styles of spatial patterns can be explained by local stability analysis in sharp contrast to previously reported results cooperation is promoted by the spatial structure in the mbsg interestingly we found that the frequency of cooperation of the mbsg on a scale free network peaks at a specific value of the payoff parameter this phenomenon indicates that properly encouraging selfish behaviors can optimally enhance the cooperation the memory effects of individuals are discussed in detail and some nonmonotonous phenomena are observed on both lattices and scale free networks our work may shed some new light on the study of evolutionary games over networks

role of population density and increasing neighborhood in the evolution of cooperation on diluted lattices abstract we investigate the evolution of cooperative behaviors with increasing neighborhood size on diluted lattices for three typical pairwise game models which include prisoner s dilemma snowdrift and stag hunt games all numerical results indicate that cooperation can persist or emerge around the optimal population density which is dictated by the percolation threshold on the square lattice meanwhile the neighborhood size determines the interaction ranges of focal players and then dominates the percolation threshold and extensive numerical simulations demonstrate that the intermediate neighborhood size is the most beneficial to the evolution of cooperation in the current lattice setup the current findings can help to deeply understand the sustenance and emergence of collective cooperation in many natural social and economic systems

statistical performance of support vector machines the support vector machine svm algorithm is well known to the computer learning community for its very good practical results the goal of the present paper is to study this algorithm from a statistical perspective using tools of concentration theory and empirical processes our main result builds on the observation made by other authors that the svm can be viewed as a statistical regularization procedure from this point of view it can also be interpreted as a model selection principle using a penalized criterion it is then possible to adapt general methods related to model selection in this framework to study two important points 1 what is the minimum penalty and how does it compare to the penalty actually used in the svm algorithm 2 is it possible to obtain oracle inequalities in that setting for the specific loss function used in the svm algorithm we show that the answer to the latter question is positive and provides relevant insight to the former our result shows that it is possible to obtain fast rates of convergence for svms

statistical behavior and consistency of classification methods based on convex risk minimization we study how closely the optimal bayes error rate can be approximately reached using a classification algorithm that computes a classifier by minimizing a convex upper bound of the classification error function the measurement of closeness is characterized by the loss function used in the estimation we show that such a classification scheme can be generally regarded as a nonmaximum likelihood conditional in class probability estimate and we use this analysis to compare various convex loss functions that have appeared in the literature furthermore the theoretical insight allows us to design good loss functions with desirable properties another aspect of our analysis is to demonstrate the consistency of certain classification methods using convex risk minimization this study sheds light on the good performance of some recently proposed linear classification methods including boosting and support vector machines it also shows their limitations and suggests possible improvements

the evolutionary public goods game on scale free networks with heterogeneous investment abstract the public goods game pgg is generally considered as a suitable paradigm to explain ubiquitous cooperative behavior in this study we investigated the evolutionary pgg on scale free networks and studied the effect of individual heterogeneity by setting the cooperator x an investment value correlated to its degree as i x n k x j k j where k x is the degree of x j runs over all players and is a tunable parameter it is shown that the cooperation level is remarkably promoted by negative values of whereas it is highly depressed by positive values of moreover the effect of environmental noise has also been investigated our result may sharpen the understanding of cooperation induced by the individual diversity

statistical significance of clustering for high dimension low sample size data clustering methods provide a powerful tool for the exploratory analysis of high dimension low sample size hdlss data sets such as gene expression microarray data a fundamental statistical issue in clustering is which clusters are really there as opposed to being artifacts of the natural sampling variation we propose sigclust as a simple and natural approach to this fundamental statistical problem in particular we define a cluster as data coming from a single gaussian distribution and formulate the problem of assessing statistical significance of clustering as a testing procedure this gaussian null assumption allows direct formulation of p values that effectively quantify the significance of a given clustering hdlss covariance estimation for sigclust is achieved by a combination of invariance principles together with a factor analysis model the properties of sigclust are studied simulated examples as well as an application to a real cancer microarray data set show that the proposed method

convexity classification and risk bounds many of the classification algorithms developed in the machine learning literature including the support vector machine and boosting can be viewed as minimum contrast methods that minimize a convex surrogate of the 0 1 loss function the convexity makes these algorithms computationally efficient the use of a surrogate however has statistical consequences that must be balanced against the computational virtues of convexity to study these issues we provide a general quantitative relationship between the risk as assessed using the 0 1 loss and the risk as assessed using any nonnegative surrogate loss function we show that this relationship gives nontrivial upper bounds on excess risk under the weakest possible condition on the loss function that it satisfies a pointwise form of fisher consistency for classification the relationship is based on a simple variational transformation of the loss function that is easy to compute in many applications we also present a refined version of this result in the

pricing forward start variance swaps with stochastic volatility abstract in this paper a general approach is presented to price forward start variance swaps with discrete sampling times based on the heston 1993 s two factor stochastic volatility model using this approach we work out two analytical closed form formulae for the price of forward start variance swap with the realized variance being defined by the actual return realized variance and the log return realized variance respectively the main features of this new approach and the developed formulae as a result include 1 treating the pricing problem of variance swaps with different definitions of discretely sampled realized variance in a highly unified way 2 easily obtaining analytical closed form solutions for forward start variance swaps with two popularly used definitions of discretely sampled realized variance 3 enabling the investigation of some important properties of the forward start variance swaps utilizing the elegant and simple form of the obtained solutions with these advantages we believe that the approach can be applied to price variance swaps based on other stochastic volatility models as well

phase diagrams for an evolutionary prisoner s dilemma game on two dimensional lattices the effects of payoffs and noise on the maintenance of cooperative behavior are studied in an evolutionary prisoner s dilemma game with players located on the sites of different two dimensional lattices this system exhibits a phase transition from a mixed state of cooperators and defectors to a homogeneous one where only the defectors remain alive using monte carlo simulations and the generalized mean field approximations we have determined the phase boundaries critical points separating the two phases on the plane of the temperature noise and temptation to choose defection in the zero temperature limit the cooperation can be sustained only for those connectivity structures where three site clique percolation occurs

cooperation in the noisy case prisoner s dilemma game on two types of regular random graphs we have studied an evolutionary prisoner s dilemma game with players located on two types of random regular graphs with a degree of 4 the analysis is focused on the effects of payoffs and noise temperature on the maintenance of cooperation when varying the noise level and or the highest payoff the system exhibits a second order phase transition from a mixed state of cooperators and defectors to an absorbing state where only defectors remain alive for the random regular graph and bethe lattice the behavior of the system is similar to those found previously on the square lattice with nearest neighbor interactions although the measure of cooperation is enhanced by the absence of loops in the connectivity structure for low noise the optimal connectivity structure is built up from randomly connected triangles

weighted distance weighted discrimination and its asymptotic properties while distance weighted discrimination dwd is an appealing approach to classification in high dimensions it was designed for balanced datasets in the case of unequal costs biased sampling or unbalanced data there are major improvements available using appropriately weighted versions of dwd wdwd a major contribution of this paper is the development of optimal weighting schemes for various nonstandard classification problems in addition we discuss several alternative criteria and propose an adaptive weighting scheme awdwd and demonstrate its advantages over nonadaptive weighting schemes under some situations the second major contribution is a theoretical study of weighted dwd both high dimensional low sample size asymptotics and fisher consistency of dwd are studied the performance of weighted dwd is evaluated using simulated examples and two real data examples the theoretical results are also confirmed by simulations

distance weighted discrimination high dimension low sample size statistical analysis is becoming increasingly important in a wide range of applied contexts in such situations the popular support vector machine suffers from data piling at the margin which can diminish generalizability this leads naturally to the development of distance weighted discrimination which is based on second order cone programming a modern computationally intensive optimization method

solving semidefinite quadratic linear programs using sdpt3 this paper discusses computational experiments with linear optimization problems involving semidefinite quadratic and linear cone constraints sqlps many test problems of this type are solved using a new release of sdpt3 a matlab implementation of infeasible primal dual path following algorithms the software developed by the authors uses mehrotra type predictor corrector variants of interior point methods and two types of search directions the hkm and nt directions a discussion of implementation details is provided and computational results on problems from the sdplib and dimacs challenge collections are reported

flexible high dimensional classification machines and their asymptotic properties classification is an important topic in statistics and machine learning with great potential in many real applications in this paper we investigate two popular large margin classification methods support vector machine svm and distance weighted discrimination dwd under two contexts the high dimensional low sample size data and the imbalanced data a unified family of classification machines the flexible assortment machine flame is proposed within which dwd and svm are special cases the flame family helps to identify the similarities and differences between svm and dwd it is well known that many classifiers overfit the data in the high dimensional setting and others are sensitive to the imbalanced data that is the class with a larger sample size overly influences the classifier and pushes the decision boundary towards the minority class svm is resistant to the imbalanced data issue but it overfits high dimensional data sets by showing the undesired data piling phenomenon the dwd method was proposed to improve svm in the high dimensional setting but its decision boundary is sensitive to the imbalanced ratio of sample sizes our flame family helps to understand an intrinsic connection between svm and dwd and provides a trade off between sensitivity to the imbalanced data and overfitting the high dimensional data several asymptotic properties of the flame classifiers are studied simulations and real data applications are investigated to illustrate theoretical findings

robust truncated hinge loss support vector machines the support vector machine svm has been widely applied for classification problems in both machine learning and statistics despite its popularity however svm has some drawbacks in certain situations in particular the svm classifier can be very sensitive to outliers in the training sample moreover the number of support vectors svs can be very large in many applications to circumvent these drawbacks we propose the robust truncated hinge loss svm rsvm which uses a truncated hinge loss the rsvm is shown to be more robust to outliers and to deliver more accurate classifiers using a smaller set of svs than the standard svm our theoretical results show that the rsvm is fisher consistent even when there is no dominating class a scenario that is particularly challenging for multicategory classification similar results are obtained for a class of margin based classifiers

infinitely imbalanced logistic regression in binary classification problems it is common for the two classes to be imbalanced one case is very rare compared to the other in this paper we consider the infinitely imbalanced case where one class has a finite sample size and the other class s sample size grows without bound for logistic regression the infinitely imbalanced case often has a useful solution under mild conditions the intercept diverges as expected but the rest of the coefficient vector approaches a non trivial and useful limit that limit can be expressed in terms of exponential tilting and is the minimum of a convex objective function the limiting form of logistic regression suggests a computational shortcut for fraud detection problems

learning from examples as an inverse problem many works related learning from examples to regularization techniques for inverse problems emphasizing the strong algorithmic and conceptual analogy of certain learning algorithms with regularization algorithms in particular it is well known that regularization schemes such as tikhonov regularization can be effectively used in the context of learning and are closely related to algorithms such as support vector machines nevertheless the connection with inverse problem was considered only for the discrete finite sample problem and the probabilistic aspects of learning from examples were not taken into account in this paper we provide a natural extension of such analysis to the continuous population case and study the interplay between the discrete and continuous problems from a theoretical point of view this allows to draw a clear connection between the consistency approach in learning theory and the stability convergence property in ill posed inverse problems the main mathematical result of the paper is a new probabilistic bound for the regularized least squares algorithm by means of standard results on the approximation term the consistency of the algorithm easily follows

coevolutionary games on networks we study agents on a network playing an iterated prisoner s dilemma against their neighbors the resulting spatially extended coevolutionary game exhibits stationary states which are nash equilibria after perturbation of these equilibria avalanches of mutations reestablish a stationary state scale free avalanche distributions are observed that are in accordance with calculations from the nash equilibria and a confined branching process the transition from subcritical to critical avalanche dynamics can be traced to a change in the degeneracy of the cooperative macrostate and is observed for many variants of this game

on ψ learning the concept of large margins have been recognized as an important principle in analyzing learning methodologies including boosting neural networks and support vector machines svms however this concept alone is not adequate for learning in nonseparable cases we propose a learning methodology called learning that is derived from a direct consideration of generalization errors we provide a theory for learning and show that it essentially attains the optimal rates of convergence in two learning examples finally results from simulation studies and from breast cancer classification confirm the ability of learning to outperform svm in generalization

the high dimension low sample size geometric representation holds under mild conditions summary high dimension low small sample size datasets have different geometrical properties from those of traditional low dimensional data in their asymptotic study regarding increasing dimensionality with a fixed sample size hall et al 2005 showed that each data vector is approximately located on the vertices of a regular simplex in a high dimensional space a perhaps unappealing aspect of their result is the underlying assumption which requires the variables viewed as a time series to be almost independent we establish an equivalent geometric representation under much milder conditions using asymptotic properties of sample covariance matrices we discuss implications of the results such as the use of principal component analysis in a high dimensional space extension to the case of nonindependent samples and also the binary classification problem