maximum volume clustering a new discriminative clustering approach the large volume principle proposed by vladimir vapnik which advocates that hypotheses lying in an equivalence class with a larger volume are more preferable is a useful alternative to the large margin principle in this paper we introduce a new discriminative clustering model based on the large volume principle called maximum volume clustering mvc and then propose two approximation schemes to solve this mvc model a soft label mvc method using sequential quadratic programming and a hard label mvc method using semi definite programming respectively the proposed mvc is theoretically advantageous for three reasons the optimization involved in hard label mvc is convex and under mild conditions the optimization involved in soft label mvc is akin to a convex one in terms of the resulting clusters secondly the soft label mvc method possesses a clustering error bound thirdly mvc includes the optimization problems of a spectral clustering two relaxed k means clustering and an information maximization clustering as special limit cases when its regularization parameter goes to infinity experiments on several artificial and benchmark data sets demonstrate that the proposed mvc compares favorably with state of the art clustering methods

boosting the margin a new explanation for the effectiveness of voting methods one of the surprising recurring phenomena observed in experiments with boosting is that the test error of the generated classifier usually does not increase as its size becomes very large and often is observed to decrease even after the training error reaches zero in this paper we show that this phenomenon is related to the distribution of margins of the training examples with respect to the generated voting classification rule where the margin of an example is simply the difference between the number of correct votes and the maximum number of votes received by any incorrect label we show that techniques used in the analysis of vapnik s support vector classifiers and of neural networks with small weights can be applied to voting methods to relate the margin distribution to the test error we also show theoretically and experimentally that boosting is especially effective at increasing the margins of the training examples finally we compare our explanation to those based on the bias variance decomposition

ambiguity based multiclass active learning most existing works on active learning al focus on binary classification problems which limit their applications in various real world scenarios one solution to multiclass al mal is evaluating the informativeness of unlabeled samples by an uncertainty model and selecting the most uncertain one for query in this paper an ambiguity based strategy is proposed to tackle this problem by applying a possibility approach first the possibilistic memberships of unlabeled samples in the multiple classes are calculated from the one against all based support vector machine model then by employing fuzzy logic operators these memberships are aggregated into a new concept named k order ambiguity which estimates the risk of labeling a sample among k classes afterward the k order ambiguities are used to form an overall ambiguity measure to evaluate the uncertainty of the unlabeled samples finally the sample with the maximum ambiguity is selected for query and a new mal strategy is developed experiments demonstrate the feasibility and effectiveness of the proposed method

statistical analysis of some multi category large margin classification methods the purpose of this paper is to investigate statistical properties of risk minimization based multi category classification methods these methods can be considered as natural extensions of binary large margin classification we establish conditions that guarantee the consistency of classifiers obtained in the risk minimization framework with respect to the classification error examples are provided for four specific forms of the general formulation which extend a number of known methods using these examples we show that some risk minimization formulations can also be used to obtain conditional probability estimates for the underlying problem such conditional probability information can be useful for statistical inferencing tasks beyond classification

exploiting coding theory for classification we propose a multiclass to binary decomposition strategy founded on coding theory we use low density parity check codes a very effective family of binary block codes exploiting the algebraic properties of the code we handle both coding and decoding two decoding rules are proposed that provide many advantages over known strategies several experiments are performed that show significant performance improvements a powerful strategy for the classification of multiple classes is to create a classifier ensemble that decomposes the polychotomy into several dichotomies the central issue when designing a multiclass to binary decomposition scheme is the definition of both the coding matrix and the decoding algorithm in this study we propose a new classification system based on low density parity check codes which is a very effective class of binary block codes the main idea is to exploit the algebraic properties of the codes to generate the codewords for the coding matrix and to define two decoding approaches which allow us to detect and recover possible errors or rejects produced by the dichotomizers experiments based on benchmark datasets demonstrated that the proposed approach provides a statistically significant improvement in terms of the classification performance compared with state of the art decomposition strategies

consistency of spectral clustering consistency is a key property of all statistical procedures analyzing randomly sampled data surprisingly despite decades of work little is known about consistency of most clustering algorithms in this paper we investigate consistency of the popular family of spectral clustering algorithms which clusters the data with the help of eigenvectors of graph laplacian matrices we develop new methods to establish that for increasing sample size those eigenvectors converge to the eigenvectors of certain limit operators as a result we can prove that one of the two major classes of spectral clustering normalized clustering converges under very general conditions while the other unnormalized clustering is only consistent under strong additional assumptions which are not always satisfied in real data we conclude that our analysis provides strong evidence for the superiority of normalized spectral clustering

generalization error bounds for bayesian mixture algorithms bayesian approaches to learning and estimation have played a significant role in the statistics literature over many years while they are often provably optimal in a frequentist setting and lead to excellent performance in practical applications there have not been many precise characterizations of their performance for finite sample sizes under general conditions in this paper we consider the class of bayesian mixture algorithms where an estimator is formed by constructing a data dependent mixture over some hypothesis space similarly to what is observed in practice our results demonstrate that mixture approaches are particularly robust and allow for the construction of highly complex estimators while avoiding undesirable overfitting effects our results while being data dependent in nature are insensitive to the underlying model assumptions and apply whether or not these hold at a technical level the approach applies to unbounded functions constrained only by certain moment conditions finally the bounds derived can be directly applied to non bayesian mixture approaches such as boosting and bagging

multi class boosting with asymmetric binary weak learners we introduce a multi class generalization of adaboost with binary weak learners we use a vectorial codification to represent class labels and a multi class exponential loss function to evaluate classifier responses this representation produces a set of margin values that provide a range of punishments for failures and rewards for successes moreover the stage wise optimization of this model introduces an asymmetric boosting procedure whose costs depend on the number of classes separated by each weak learner in this way the boosting algorithm takes into account class imbalances when building the ensemble the experiments performed compare this new approach favorably to adaboost mh gentleboost and the samme algorithms

a unified view on multi class support vector classification a unified view on multi class support vector machines svms is presented covering most prominent variants including the one vs all approach and the algorithms proposed by weston watkins crammer singer lee lin wahba and liu yuan the unification leads to a template for the quadratic training problems and new multi class svm formulations within our framework we provide a comparative analysis of the various notions of multi class margin and margin based loss in particular we demonstrate limitations of the loss function considered for instance in the crammer singer machine we analyze fisher consistency of multi class loss functions and universal consistency of the various machines on the one hand we give examples of svms that are in a particular hyperparameter regime universally consistent without being based on a fisher consistent loss these include the canonical extension of svms to multiple classes as proposed by weston watkins and vapnik as well as the one vs all approach on the other hand it is demonstrated that machines based on fisher consistent loss functions can fail to identify proper decision boundaries in low dimensional feature spaces we compared the performance of nine different multi class svms in a thorough empirical study our results suggest to use the weston watkins svm which can be trained comparatively fast and gives good accuracies on benchmark functions if training time is a major concern the one vs all approach is the method of choice

multicategory support vector machines two category support vector machines svm have been very popular in the machine learning community for classification problems solving multicategory problems by a series of binary classifiers is quite common in the svm paradigm however this approach may fail under various circumstances we propose the multicategory support vector machine msvm which extends the binary svm to the multicategory case and has good theoretical properties the proposed method provides a unifying framework when there are either equal or unequal misclassification costs as a tuning criterion for the msvm an approximate leave one out cross validation function called generalized approximate cross validation is derived analogous to the binary case the effectiveness of the msvm is demonstrated through the applications to cancer classification using microarray data and cloud classification with satellite radiance profiles

fast sdp relaxations of graph cut clustering transduction and other combinatorial problems the rise of convex programming has changed the face of many research fields in recent years machine learning being one of the ones that benefitted the most a very recent developement the relaxation of combinatorial problems to semi definite programs sdp has gained considerable attention over the last decade helmberg 2000 de bie and cristianini 2004a although sdp problems can be solved in polynomial time for many relaxations the exponent in the polynomial complexity bounds is too high for scaling to large problem sizes this has hampered their uptake as a powerful new tool in machine learning in this paper we present a new and fast sdp relaxation of the normalized graph cut problem and investigate its usefulness in unsupervised and semi supervised learning in particular this provides a convex algorithm for transduction as well as approaches to clustering we further propose a whole cascade of fast relaxations that all hold the middle between older spectral relaxations and the new sdp relaxation allowing one to trade off computational cost versus relaxation accuracy finally we discuss how the methodology developed in this paper can be applied to other combinatorial problems in machine learning and we treat the max cut problem as an example

linear time maximum margin clustering maximum margin clustering mmc is a newly proposed clustering method which has shown promising performance in recent studies it extends the computational techniques of support vector machine svm to the unsupervised scenario traditionally mmc is formulated as a nonconvex integer programming problem which makes it difficult to solve several methods have been proposed in the literature to solve the mmc problem based on either semidefinite programming sdp or alternating optimization however these methods are still time demanding when handling large scale data sets which limits its application in real world problems in this paper we propose a cutting plane maximum margin clustering cpmmc algorithm it first decomposes the nonconvex mmc problem into a series of convex subproblems by making use of the constrained concave convex procedure cccp then for each subproblem our algorithm adopts the cutting plane algorithm to solve it moreover we show that the cpmmc algorithm takes o sn time to converge with guaranteed accuracy where n is the number of samples in the data set and s is the sparsity of the data set i e the average number of nonzero features of the data samples we also derive the multiclass version of our cpmmc algorithm experimental evaluations on several real world data sets show that cpmmc performs better than existing mmc methods both in efficiency and accuracy

pairwise support vector machines and their application to large scale problems pairwise classification is the task to predict whether the examples a b of a pair a b belong to the same class or to different classes in particular interclass generalization problems can be treated in this way in pairwise classification the order of the two input examples should not affect the classification result to achieve this particular kernels as well as the use of symmetric training sets in the framework of support vector machines were suggested the paper discusses both approaches in a general way and establishes a strong connection between them in addition an efficient implementation is discussed which allows the training of several millions of pairs the value of these contributions is confirmed by excellent results on the labeled faces in the wild benchmark

interfacial waves and deformations in pre stressed elastic media the propagation of interfacial stoneley waves along the boundary between two half spaces of pre stressed incompressible isotropic elastic material is examined the underlying deformation in each half space corresponds to a pure homogeneous strain with one principal axis of strain normal to the interface and the others having a common orientation the secular equation governing the wave speed for propagation along a principal axis is obtained in respect of general strain energy functions detailed analysis of the secular equation reveals general sufficient conditions for the existence of a wave and in particular cases necessary and sufficient conditions for the existence of a unique interfacial wave it is also shown that when an interfacial wave exists its speed is greater than that of the least of the rayleigh wave speeds for the separate half spaces paralleling a result from the linear theory for the special case of quasi static interfacial deformations corresponding to vanishing wave speed an existence criterion is found moreover it is shown that inequalities that exclude surface deformations in each half space also exclude interfacial deformations dependence of the above results on the underlying homogeneous deformations and on material parameters is illustrated by numerical results for the neo hookean strain energy function

a multiclass classification method based on decoding of binary classifiers in this letter we present new methods of multiclass classification that combine multiple binary classifiers misclassification of each binary classifier is formulated as a bit inversion error with probabilistic models by making an analogy to the context of information transmission theory dependence between binary classifiers is incorporated into our model which makes a decoder a type of boltzmann machine we performed experimental studies using a synthetic data set data sets from the uci repository and bioinformatics data sets and the results show that the proposed methods are superior to the existing multiclass classification methods

normalized cuts and image segmentation we propose a novel approach for solving the perceptual grouping problem in vision rather than focusing on local features and their consistencies in the image data our approach aims at extracting the global impression of an image we treat image segmentation as a graph partitioning problem and propose a novel global criterion the normalized cut for segmenting the graph the normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups we show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion we applied this approach to segmenting static images as well as motion sequences and found the results to be very encouraging

a view of margin losses as regularizers of probability estimates regularization is commonly used in classifier design to assure good generalization classical regularization enforces a cost on classifier complexity by constraining parameters this is usually combined with a margin loss which favors large margin decision rules a novel and unified view of this architecture is proposed by showing that margin losses act as regularizers of posterior class probabilities in a way that amplifies classical parameter regularization the problem of controlling the regularization strength of a margin loss is considered using a decomposition of the loss in terms of a link and a binding function the link function is shown to be responsible for the regularization strength of the loss while the binding function determines its outlier robustness a large class of losses is then categorized into equivalence classes of identical regularization strength or outlier robustness it is shown that losses in the same regularization class can be parameterized so as to have tunable regularization strength this parameterization is finally used to derive boosting algorithms with loss regularization boostlr three classes of tunable regularization losses are considered in detail canonical losses can implement all regularization behaviors but have no flexibility in terms of outlier modeling shrinkage losses support equally parameterized link and binding functions leading to boosting algorithms that implement the popular shrinkage procedure this offers a new explanation for shrinkage as a special case of loss based regularization finally tunable losses enable the independent parameterization of link and binding functions leading to boosting algorithms of great exibility this is illustrated by the derivation of an algorithm that generalizes both adaboost and logitboost behaving as either one when that best suits the data to classify various experiments provide evidence of the benefits of probability regularization for both classification and estimation of posterior class probabilities

on the design of an ecoc compliant genetic algorithm genetic algorithms ga have been previously applied to error correcting output codes ecoc in state of the art works in order to find a suitable coding matrix nevertheless none of the presented techniques directly take into account the properties of the ecoc matrix as a result the considered search space is unnecessarily large in this paper a novel genetic strategy to optimize the ecoc coding step is presented this novel strategy redefines the usual crossover and mutation operators in order to take into account the theoretical properties of the ecoc framework thus it reduces the search space and lets the algorithm to converge faster in addition a novel operator that is able to enlarge the code in a smart way is introduced the novel methodology is tested on several uci datasets and four challenging computer vision problems furthermore the analysis of the results done in terms of performance code length and number of support vectors shows that the optimization process is able to find very efficient codes in terms of the trade off between classification performance and the number of classifiers finally classification performance per dichotomizer results shows that the novel proposal is able to obtain similar or even better results while defining a more compact number of dichotomies and svs compared to state of the art approaches highlightsa novel genetic algorithm to optimize the ecoc coding step is presented the crossover and mutation operators are redefined taking into account the ecoc properties a new operator that is able to extend the ecoc code is developed we introduce a novel regularization parameter that is able to control the number of dichotomies

a computationally efficient scheme for feature extraction with kernel discriminant analysis the kernel discriminant analysis kda an extension of the linear discriminant analysis lda and null space based lda into the kernel space generally provides good pattern recognition pr performance for both small sample size sss and non sss pr problems due to the eigen decomposition technique adopted however the original scheme for the feature extraction with the kda suffers from a high complexity burden in this paper we derive a transformation of the kda into a linear equation problem and propose a novel scheme for the feature extraction with the kda the proposed scheme is shown to provide us with a reduction of complexity without degradation of pr performance in addition to enhance the pr performance further we address the incorporation of regularization into the proposed scheme highlightswe propose a complexity reduced scheme for the kernel discriminant analysis the proposed scheme does not cause degradation of recognition accuracy performance the core is the transformation of the original problem into a linear equation problem we further address the incorporation of regularization into the proposed scheme

a unified view of class selection with probabilistic classifiers the possibility of selecting a subset of classes instead of one unique class for assignation is of great interest in many decision making systems selecting a subset of classes instead of singleton allows to reduce the error rate and to propose a reduced set to another classifier or an expert this second step provides additional information and therefore increases the quality of the result in this paper a unified view of the problem of class selection with probabilistic classifiers is presented the proposed framework based on the evaluation of the probabilistic equivalence allows to retrieve class selective frameworks that have been proposed in the literature we also describe an approach in which the decision rules are compared by the help of a normalized area under the error selection curve it allows to get a relative independence of the performance of a classifier without reject option and thus a reliable class selection decision rule evaluation the power of this generic proposition is demonstrated by evaluating and comparing it to several state of the art methods on nine real world datasets and four different probabilistic classifiers a unified framework of class selection by the use of probabilistic equivalence is presented a learning procedure for adapting the probabilistic equivalence is proposed a normalized performance measure for class selection is described experiments on real world datasets using different classifiers show the statistical significance of the proposition

complexity reduced scheme for feature extraction with linear discriminant analysis owing to the singularity of the within class scatter linear discriminant analysis lda becomes ill posed for small sample size sss problems null space based lda nlda which is an extension of lda provides good discriminant performances for sss problems yet as the original scheme for the feature extractor fe of nlda suffers from a complexity burden a few modified schemes have since been proposed for complexity reduction in this brief by transforming the problem of finding the fe of nlda into a linear equation problem a novel scheme is derived offering a further reduction of the complexity

rademacher penalties and structural risk minimization we suggest a penalty function to be used in various problems of structural risk minimization this penalty is data dependent and is based on the sup norm of the so called rademacher process indexed by the underlying class of functions sets the standard complexity penalties used in learning problems and based on the vc dimensions of the classes are conservative upper bounds in a probabilistic sense uniformly over the set of all underlying distributions for the penalty we suggest thus for a particular distribution of training examples one can expect better performance of learning algorithms with the data driven rademacher penalties we obtain oracle inequalities for the theoretical risk of estimators obtained by structural minimization of the empirical risk with rademacher penalties the inequalities imply some form of optimality of the empirical risk minimizers we also suggest an iterative approach to structural risk minimization with rademacher penalties in which the hierarchy of classes is not given in advance but is determined in the data driven iterative process of risk minimization we prove probabilistic oracle inequalities for the theoretical risk of the estimators based on this approach as well

mercer kernel based clustering in feature space the article presents a method for both the unsupervised partitioning of a sample of data and the estimation of the possible number of inherent clusters which generate the data this work exploits the notion that performing a nonlinear data transformation into some high dimensional feature space increases the probability of the linear separability of the patterns within the transformed space and therefore simplifies the associated data structure it is shown that the eigenvectors of a kernel matrix which defines the implicit mapping provides a means to estimate the number of clusters inherent within the data and a computationally simple iterative procedure is presented for the subsequent feature space partitioning of the data

robust adaptive observers for nonlinear systems with bounded disturbances since existing adaptive observers for nonlinear systems may generate unbounded parameter estimates in the presence of bounded disturbances robust adaptive observers are presented which prevent parameter estimate drift in addition the input to state stability of the error dynamics with respect to disturbances and parameter time derivatives is guaranteed by generalizing a persistency of excitation result asymptotic convergence of state estimation errors is still achieved for systems in adaptive observer form when disturbances are not present by a suitable extension of barbalat s lemma

reduced nonlinear observer for bounded estimation of the static friction model with the stribeck effect in this paper a nonlinear observer to estimate the parameters of the static friction model with the stribeck effect is presented this system cannot be expressed as a linear function of the model parameters however a nonlinear disturbance term can be added to the linear approximation then under the persistency excitation assumption we prove explicitly that the estimation error is ultimately bounded simulation results demonstrate the effectiveness of the proposed method compared with a coulomb estimator

trace norm regularization reformulations algorithms and multi task learning we consider a recently proposed optimization formulation of multi task learning based on trace norm regularized least squares while this problem may be formulated as a semidefinite program sdp its size is beyond general sdp solvers previous solution approaches apply proximal gradient methods to solve the primal problem we derive new primal and dual reformulations of this problem including a reduced dual formulation that involves minimizing a convex quadratic function over an operator norm ball in matrix space this reduced dual problem may be solved by gradient projection methods with each projection involving a singular value decomposition the dual approach is compared with existing approaches and its practical effectiveness is illustrated on simulations and an application to gene expression pattern analysis

gaussian kernel optimization for pattern classification this paper presents a novel algorithm to optimize the gaussian kernel for pattern classification tasks where it is desirable to have well separated samples in the kernel feature space we propose to optimize the gaussian kernel parameters by maximizing a classical class separability criterion and the problem is solved through a quasi newton algorithm by making use of a recently proposed decomposition of the objective criterion the proposed method is evaluated on five data sets with two kernel based learning algorithms the experimental results indicate that it achieves the best overall classification performance compared with three competing solutions in particular the proposed method provides a valuable kernel optimization solution in the severe small sample size scenario

review paper characterization and stability of two phase piecewise homogeneous deformations many solid materials exhibit stress induced phase transformations such phenomena can be modelled with the aid of the nonlinear elasticity theory with appropriate choices of the strain energy function it is known that if a two phase deformation with gradient f in a finite elastic body is a local energy minimizer then given any point p of the surface of discontinuity the piecewise homogeneous deformation corresponding to the two values f p of f p is a global energy minimizer thus instability of the latter state would imply instability of the former state in this paper we investigate the stability properties of such piecewise homogeneous deformations more precisely we are concerned with two joined half spaces that correspond to two different phases of the same material we first show how such a two phase deformation can be constructed then the stability of the piecewise homogeneous deformation is investigated with the aid of two test criteria one is a kinetic stability criterion based on a quasi static approach and on the growth decay behaviour of the interface in the undeformed configuration when it is perturbed the other referred to as the energy criterion is used to determine whether the deformation is a minimizer of the total energy with respect to perturbations of the interface in both the current and undeformed configurations we clarify the differences between the two criteria and provide a compact formula which can be used to establish the stability instability of any two phase piecewise homogeneous deformations

classification by pairwise coupling we discuss a strategy for polychotomous classification that involves estimating class probabilities for each pair of classes and then coupling the estimates together the coupling model is similar to the bradley terry method for paired comparisons we study the nature of the class probability estimates that arise and examine the performance of the procedure in real and simulated data sets classifiers used include linear discriminants nearest neighbors adaptive nonlinear methods and the support vector machine

classification using geometric level sets a variational level set method is developed for the supervised classification problem nonlinear classifier decision boundaries are obtained by minimizing an energy functional that is composed of an empirical risk term with a margin based loss and a geometric regularization term new to machine learning the surface area of the decision boundary this geometric level set classifier is analyzed in terms of consistency and complexity through the calculation of its e entropy for multicategory classification an efficient scheme is developed using a logarithmic number of decision functions in the number of classes rather than the typical linear number of decision functions geometric level set classification yields performance results on benchmark data sets that are competitive with well established methods

randomboost simplified multiclass boosting through randomization we propose a novel boosting approach to multiclass classification problems in which multiple classes are distinguished by a set of random projection matrices in essence the approach uses random projections to alleviate the proliferation of binary classifiers typically required to perform multiclass classification the result is a multiclass classifier with a single vector valued parameter irrespective of the number of classes involved two variants of this approach are proposed the first method randomly projects the original data into new spaces while the second method randomly projects the outputs of learned weak classifiers these methods are not only conceptually simple but also effective and easy to implement a series of experiments on synthetic machine learning and visual recognition data sets demonstrate that our proposed methods could be compared favorably with existing multiclass boosting algorithms in terms of both the convergence rate and classification accuracy

gensvm a generalized multiclass support vector machine traditional extensions of the binary support vector machine svm to multiclass problems are either heuristics or require solving a large dual optimization problem here a generalized multiclass svm is proposed called gensvm in this method classification boundaries for a k class problem are constructed in a k 1 dimensional space using a simplex encoding additionally several different weightings of the misclassification errors are incorporated in the loss function such that it generalizes three existing multiclass svms through a single optimization problem an iterative majorization algorithm is derived that solves the optimization problem without the need of a dual formulation this algorithm has the advantage that it can use warm starts during cross validation and during a grid search which significantly speeds up the training phase rigorous numerical experiments compare linear gensvm with seven existing multiclass svms on both small and large data sets these comparisons show that the proposed method is competitive with existing methods in both predictive accuracy and training time and that it significantly outperforms several existing methods on these criteria

feature scaling for kernel fisher discriminant analysis using leave one out cross validation kernel fisher discriminant analysis kfd is a successful approach to classification it is well known that the key challenge in kfd lies in the selection of free parameters such as kernel parameters and regularization parameters here we focus on the feature scaling kernel where each feature individually associates with a scaling factor a novel algorithm named fs kfd is developed to tune the scaling factors and regularization parameters for the feature scaling kernel the proposed algorithm is based on optimizing the smooth leave one out error via a gradient descent method and has been demonstrated to be computationally feasible fs kfd is motivated by the following two fundamental facts the leave one out error of kfd can be expressed in closed form and the step function can be approximated by a sigmoid function empirical comparisons on artificial and benchmark data sets suggest that fs kfd improves kfd in terms of classification accuracy

an exponentially stable adaptive friction compensator this note presents a novel adaptive compensation scheme for coulomb friction in a servocontrol system an adaptive observer for estimating the unknown coulomb friction coefficient is also derived on the basis of the lyapunov technique in addition a linearizing control law is developed to compensate for the friction force and obtain the tracking objective the proposed adaptive compensation guarantees an exponential convergence for state errors and parameter error and known adaptive schemes guarantee only an asymptotic or stable convergence simulation results demonstrate the effectiveness of the proposed method for a single mass servocontrol system

considerations of the existence of interfacial stoneley waves in bonded anisotropic elastic half spaces the questions of uniqueness and existence of subsonic stoneley waves in bonded anisotropic linear elastic half spaces are settled by using the notion of the interface impedance tensor which is a simple linear combination of the hermitian surface impedance tensors of the separate half spaces a definite existence criterion is presented in a form that proves most useful in numerical searches for stoneley waves in the sense that such searches need not be conducted when a stoneley wave mode

the analysis of contingency tables preface contingency tables and the chi square test 2 x 2 contingency tables r x c contingency tables multidimensional tables log linear models for contingency table linear logistic models contingency tables with ordered categories some special types of contingency tables appendices percentage points of the chi squared distribution computer software for fitting log linear and logistic models reference index

a non parametric approach to extending generic binary classifiers for multi classification ensemble methods which combine generic binary classifier scores to generate a multi classification output are commonly used in state of the art computer vision and pattern recognition systems that rely on multi classification in particular we consider the one vs one decomposition of the multi class problem where binary classifier models are trained to discriminate every class pair we describe a robust multi classification pipeline which at a high level involves projecting binary classifier scores into compact orthogonal subspaces followed by a non linear probabilistic multi classification step using kernel density estimation kde we compare our approach against state of the art ensemble methods dcs drcw on 16 multi class datasets we also compare against the most commonly used ensemble methods vote nest on 6 real world computer vision datasets finally we measure the statistical significance of our approach using non parametric tests experimental results show that our approach gives a statistically significant improvement in multi classification performance over state of the art highlightsensemble methods combine binary classifiers to yield a multi classification output one vs one ensemble binary classifiers trained to discriminate each class pair we propose a robust non parametric probabilistic one vs one ensemble method kdemrp kdemrp improves classification performance over state of the art dcs drcw kdemrp improvements are statistically significant

the concave convex procedure the concave convex procedure cccp is a way to construct discrete time iterative dynamical systems that are guaranteed to decrease global optimization and energy functions monotonically this procedure can be applied to almost any optimization problem and many existing algorithms can be interpreted in terms of it in particular we prove that all expectation maximization algorithms and classes of legendre minimization and variational bounding algorithms can be reexpressed in terms of cccp we show that many existing neural network and mean field theory algorithms are also examples of cccp the generalized iterative scaling algorithm and sinkhorn s algorithm can also be expressed as cccp by changing variables cccp can be used both as a new way to understand and prove the convergence of existing optimization algorithms and as a procedure for generating new algorithms

optimization approaches to supervised classification abstract the supervised classification problem one of the oldest and most recurrent problems in applied data analysis has always been analyzed from many different perspectives when the emphasis is placed on its overall goal of developing classification rules with minimal classification cost supervised classification can be understood as an optimization problem on the other hand when the focus is in modeling the uncertainty involved in the classification of future unknown entities it can be formulated as a statistical problem other perspectives that pay particular attention to pattern recognition and machine learning aspects of supervised classification have also a long history that has lead to influential insights and different methodologies in this review two approaches to supervised classification strongly related to optimization theory will be discussed and compared in particular we will review methodologies based on mathematical programming models that optimize observable criteria linked to the true objective of misclassification error or cost minimization and approaches derived from the minimization of known bounds on the true misclassification error the former approach is known as the mathematical programming approach to supervised classification while the latter is in the origin of the well known classification support vector machines throughout the review two group as well as general multi group problems will be considered and the review will conclude with a discussion of the most promising research directions in this area

the polarization vectors at the interface and the secular equation for stoneley waves in monoclinic bimaterials stoneley waves propagating in the direction of the x 1 axis in a bimaterial consisting of two half spaces x 2 0 and x 2 0 of dissimilar anisotropic elastic materials are considered several invariants independent of x 2 that relate the displacement and the stress are obtained for a general anisotropic elastic bimaterial we then study the case when both materials have the symmetry plane at x 2 0 or x 1 0 the displacement and the traction at the interface x 2 0 describe two separate elliptic paths they are polarized on planes that contain the x 2 axis or x 1 axis moreover the x 2 axis or x 1 axis is one of the principal axes of the ellipse in the rest of the paper we consider special monoclinic bimaterials with the symmetry plane at x 2 0 x 1 0 or x 3 0 the elastic constants for the two monoclinic materials are identical except that those elastic constants that would vanish if the material were orthotropic have the opposite sign for the two materials it is shown that one of the ellipses degenerates into a line along a coordinate axis while the other ellipse is on a coordinate plane normal to this coordinate axis when the symmetry plane is at x 3 0 both ellipses degenerate into lines along the x 1 and x 2 axes explicit expressions of the polarization vectors and the secular equation are presented for all three monoclinic bimaterials

in defense of one vs all classification we consider the problem of multiclass classification our main thesis is that a simple one vs all scheme is as accurate as any other approach assuming that the underlying binary classifiers are well tuned regularized classifiers such as support vector machines this thesis is interesting in that it disagrees with a large body of recent published work on multiclass classification we support our position by means of a critical review of the existing literature a substantial collection of carefully controlled experimental work and theoretical arguments

local distance functions a taxonomy new algorithms and an evaluation we present a taxonomy for local distance functions where most existing algorithms can be regarded as approximations of the geodesic distance defined by a metric tensor we categorize existing algorithms by how where and when they estimate the metric tensor we also extend the taxonomy along each axis how we introduce hybrid algorithms that use a combination of techniques to ameliorate overfitting where we present an exact polynomial time algorithm to integrate the metric tensor along the lines between the test and training points under the assumption that the metric tensor is piecewise constant when we propose an interpolation algorithm where the metric tensor is sampled at a number of references points during the offline phase the reference points are then interpolated during the online classification phase we also present a comprehensive evaluation on tasks in face recognition object recognition and digit recognition

reducing multiclass to binary a unifying approach for margin classifiers we present a unifying framework for studying the solution of multiclass categorization problems by reducing them to multiple binary problems that are then solved using a margin based binary learning algorithm the proposed framework unifies some of the most popular approaches in which each class is compared against all others or in which all pairs of classes are compared to each other or in which output codes with error correcting properties are used we propose a general method for combining the classifiers generated on the binary problems and we prove a general empirical multiclass loss bound given the empirical loss of the individual binary learning algorithms the scheme and the corresponding bounds apply to many popular classification learning algorithms including support vector machines adaboost regression logistic regression and decision tree algorithms we also give a multiclass generalization error analysis for general output codes with adaboost as the binary learner experimental results with svm and adaboost show that our scheme provides a viable alternative to the most commonly used multiclass algorithms

a study of parts based object class detection using complete graphs object detection is one of the key components in modern computer vision systems while the detection of a specific rigid object under changing viewpoints was considered hard just a few years ago current research strives to detect and recognize classes of non rigid articulated objects hampered by the omnipresent confusing information due to clutter and occlusion the focus has shifted from holistic approaches for object detection to representations of individual object parts linked by structural information along with richer contextual descriptions of object configurations along this line of research we present a practicable and expandable probabilistic framework for parts based object class representation enabling the detection of rigid and articulated object classes in arbitrary views we investigate learning of this representation from labelled training images and infer globally optimal solutions to the contextual map detection problem using a search with a novel lower bound as admissible heuristic an assessment of the inference performance of belief propagation and tree reweighted belief propagation is obtained as a by product the generality of our approach is demonstrated on four different datasets utilizing domain dependent information cues

stability and generalization we define notions of stability for learning algorithms and show how to use these notions to derive generalization error bounds based on the empirical error and the leave one out error the methods we use can be applied in the regression framework as well as in the classification one when the classifier is obtained by thresholding a real valued function we study the stability properties of large classes of learning algorithms such as regularization based algorithms in particular we focus on hilbert space regularization and kullback leibler regularization we demonstrate how to apply the results to svm for regression and classification

three dimensional dynamic green s functions for a multilayered transversely isotropic half space abstract with the aid of a method of displacement potentials an efficient and accurate analytical derivation of the three dimensional dynamic green s functions for a transversely isotropic multilayered half space is presented constituted by proper algebraic factorizations a set of generalized transmission reflection matrices and internal source fields that are free of any numerically unstable exponential terms are proposed for effective computations of the potential solution three dimensional point load green s functions for stresses and displacements are given for the first time in the complex plane line integral representations the present formulations and solutions are analytically in exact agreement with the existing solutions given by pak and guzina 2002 for the isotropic case for the numerical computation of the integrals a robust and effective methodology which gives the necessary account of the presence of singularities including branch points and poles on the path of integration is laid out a comparison with the existing numerical solutions for multilayered isotropic half space is made to confirm the accuracy of the numerical solutions

on the algorithmic implementation of multiclass kernel based vector machines in this paper we describe the algorithmic implementation of multiclass kernel based vector machines our starting point is a generalized notion of the margin to multiclass problems using this notion we cast multiclass categorization problems as a constrained optimization problem with a quadratic objective function unlike most of previous approaches which typically decompose a multiclass problem into multiple independent binary classification tasks our notion of margin yields a direct method for training multiclass predictors by using the dual of the optimization problem we are able to incorporate kernels with a compact set of constraints and decompose the dual problem into multiple optimization problems of reduced size we describe an efficient fixed point algorithm for solving the reduced optimization problems and prove its convergence we then discuss technical details that yield significant running time improvements for large datasets finally we describe various experiments with our approach comparing it to previously studied kernel based methods our experiments indicate that for multiclass problems we attain state of the art accuracy

on the existence of interfacial stoneley waves in bonded piezoelectric half spaces a new definition of the piezoelectric limiting speed based on the piezoelectrically stiffened acoustical tensor is introduced the stoneley wave problem is readily formulated for bonded piezoelectric half spaces in both the phi and f representations introduced by lothe barnett the notion of the f piezoelectric interfacial impedance tensor which is an extension of the elastic interfacial impedance tensor is used to show that a maximum of two subsonic stoneley waves can occur a definite existence criterion for a single subsonic piezoelectric stoneley mode is given and a numerical scheme that searches for stoneley modes when they exist for a given geometry and material combination is also presented along with a numerical example for two joined cadmium sulphide half spaces

three dimensional dynamic green s functions in transversely isotropic bi materials by virtue of a complete representation using two displacement potentials an analytical derivation of the elastodynamic green s functions for a linear elastic transversely isotropic bi material full space is presented three dimensional point load green s functions for stresses and displacements are given in complex plane line integral representations the formulation includes a complete set of transformed stress potential and displacement potential relations within the framework of fourier expansions and hankel integral transforms that is useful in a variety of elastodynamic as well as elastostatic problems for numerical computation of the integrals a robust and effective methodology is laid out which gives the necessary account of the presence of singularities including branch points and pole on the path of integration as illustrations the present green s functions are analytically degenerated to the special cases such as half space surface and full space green s functions some typical numerical examples are also given to show the general features of the bi material green s functions

information maximization clustering based on squared loss mutual information information maximization clustering learns a probabilistic classifier in an unsupervised manner so that mutual information between feature vectors and cluster assignments is maximized a notable advantage of this approach is that it involves only continuous optimization of model parameters which is substantially simpler than discrete optimization of cluster assignments however existing methods still involve nonconvex optimization problems and therefore finding a good local optimal solution is not straightforward in practice in this letter we propose an alternative information maximization clustering method based on a squared loss variant of mutual information this novel approach gives a clustering solution analytically in a computationally efficient way via kernel eigenvalue decomposition furthermore we provide a practical model selection procedure that allows us to objectively optimize tuning parameters included in the kernel function through experiments we demonstrate the usefulness of the pr

vertical vibration of a rigid circular disc at the interface of a transversely isotropic bi material a theoretical formulation is presented for the determination of the dynamic interaction of a vertically loaded rigid disc embedded at the interface of a transversely isotropic bi material full space with the aid of hankel integral transforms a relaxed treatment of the mixed boundary value problem is formulated as dual integral equations which can be reduced to a fredholm integral equation of the second kind the dynamic contact pressure under the disc and the impedance function are analytically evaluated in the general dynamic case it is shown that the impedance functions are in exact agreement with the existing solutions for a an elastic half space with isotropic material properties to confirm the accuracy of the numerical evaluation of the integrals involved numerical results are included for cases of different degrees of the material anisotropy and compared with previously published solutions

posterior probability based ensemble strategy using optimizing decision directed acyclic graph for multi class classification ensemble strategy is important to develop a decomposition and ensemble method for multi class classification problems most existing ensemble strategies use predetermined and heuristic decision rules in this work we build up the decision rules by optimizing decision directed acyclic graph oddag with classical and fuzzy decision trees to ensemble the posterior probabilities of binary classifiers from one vs one ovo or one vs all ova decomposition strategies for multi class classification problems four widely used extensible algorithms and ten decomposition and ensemble methods incorporating four binary classifiers bcs have been tested on 25 data sets the empirical results show that the methods based on ensemble strategies using oddag are among the top methods that achieve the best performance in terms of two different measures

a parsimonious mixture of gaussian trees model for oversampling in imbalanced and multimodal time series classification we propose a novel framework of using a parsimonious statistical model known as mixture of gaussian trees for modeling the possibly multimodal minority class to solve the problem of imbalanced time series classification by exploiting the fact that close by time points are highly correlated due to smoothness of the time series our model significantly reduces the number of covariance parameters to be estimated from o d 2 to o ld where l is the number of mixture components and d is the dimensionality thus our model is particularly effective for modeling high dimensional time series with limited number of instances in the minority positive class in addition the computational complexity for learning the model is only of the order o ln d 2 where n is the number of positively labeled samples we conduct extensive classification experiments based on several well known time series data sets both single and multimodal by first randomly generating synthetic instances from our learned mixture model to correct the imbalance we then compare our results with several state of the art oversampling techniques and the results demonstrate that when our proposed model is used in oversampling the same support vector machines classifier achieves much better classification accuracy across the range of data sets in fact the proposed method achieves the best average performance 30 times out of 36 multimodal data sets according to the f value metric our results are also highly competitive compared with nonoversampling based classifiers for dealing with imbalanced time series data sets

classification by pairwise coupling of imprecise probabilities we address combining imprecise classifiers in the framework of imprecise probabilities classifier outputs are probability intervals which are large for irrelevant classifiers the combination consists in computing probabilities consistent with the intervals this approach is particularly interesting when combining few classifiers in this paper we are interested in making decisions by combining classifiers providing uncertain outputs in the form of sets of probability distributions more precisely each classifier provides lower and upper bounds on the conditional probabilities of the associated classes the classifiers are combined by computing the set of unconditional probability distributions compatible with these bounds by solving linear optimization problems when the classifier outputs are inconsistent we propose a correcting step that restores this consistency the experiments show the interest of our approach for solving multi class classification problems particularly when information is scarce i e a limited number of classifiers is available in this case modeling the lack of information associated with classifier outputs gives good results even when they are poorly regularized or overfit the data

operator valued kernels for learning from functional response data in this paper we consider the problems of supervised classification and regression in the case where attributes and labels are functions a data is represented by a set of functions and the label is also a function we focus on the use of reproducing kernel hilbert space theory to learn from such functional data basic concepts and properties of kernel based learning are extended to include the estimation of function valued functions in this setting the representer theorem is restated a set of rigorously defined infinite dimensional operator valued kernels that can be valuably applied when the data are functions is described and a learning algorithm for nonlinear functional data analysis is introduced the methodology is illustrated through speech and audio signal processing experiments

quantum ensemble classification a sampling based learning control approach quantum ensemble classification qec has significant applications in discrimination of atoms or molecules separation of isotopes and quantum information extraction however quantum mechanics forbids deterministic discrimination among nonorthogonal states the classification of inhomogeneous quantum ensembles is very challenging since there exist variations in the parameters characterizing the members within different classes in this paper we recast qec as a supervised quantum learning problem a systematic classification methodology is presented by using a sampling based learning control slc approach for quantum discrimination the classification task is accomplished via simultaneously steering members belonging to different classes to their corresponding target states e g mutually orthogonal states first a new discrimination method is proposed for two similar quantum systems then an slc method is presented for qec numerical results demonstrate the effectiveness of the proposed approach for the binary classification of two level quantum ensembles and the multiclass classification of multilevel quantum ensembles

comments on an exponentially stable adaptive friction compensator this note points out a technique error in the above paper liao and chien ieee trans automat contr vol 45 p 977 80 2000 it is shown that without persistence of excitation pe of the system signals the estimated coulomb friction parameter cannot be guaranteed to converge to its true value an analysis is provided for the pe condition of parameter asymptotic convergence

interfacial viscoelastic sh waves shear horizontal waves in the form of transient perturbations are considered at the interface between two different viscoelastic solids the admissibility of these interfacial waves is studied via the asymptotic expansion of the laplace transform of the viscoelastic kernel the compatibility condition is reduced to a set of algebraic systems which can be solved iteratively to the desired order in the asymptotic expansion two classes of solutions are found which correspond to transient waves decaying away from the interface and attenuated along the propagation direction numerical examples are given to illustrate the results

supervised learning algorithms for multi class classification problems with partial class memberships in several application domains such as biology computer vision social network analysis and information retrieval multi class classification problems arise in which data instances not simply belong to one particular class but exhibit a partial membership to several classes existing machine learning or fuzzy set approaches for representing this type of fuzzy information mainly focus on unsupervised methods in contrast we present in this article supervised learning algorithms for classification problems with partial class memberships where class memberships instead of crisp class labels serve as input for fitting a model to the data using kernel logistic regression klr as a baseline method first a basic one versus all approach is proposed by replacing the binary coded label vectors with 0 1 valued class memberships in the likelihood subsequently we use this klr extension as base classifier to construct one versus one decompositions in which partial class memberships are transformed and estimated in a pairwise manner empirical results on synthetic data and a real world application in bioinformatics confirm that our approach delivers promising results the one versus all method yields the best computational efficiency while the one versus one methods are preferred in terms of predictive performance especially when the observed class memberships are heavily unbalanced

enhancing directed binary trees for multi class classification one approach to multi class classification consists in decomposing the original problem into a collection of binary classification tasks the outputs of these binary classifiers are combined to produce a single prediction winner takes all max wins and tree voting schemes are the most popular methods for this purpose however tree schemes can deliver faster predictions because they need to evaluate less binary models despite previous conclusions reported in the literature this paper shows that their performance depends on the organization of the tree scheme i e the positions where each pairwise classifier is placed on the graph different metrics are studied for this purpose proposing a new one that considers the precision and the complexity of each pairwise model what makes the method to be classifier dependent the study is performed using support vector machines svms as base classifiers but it could be extended to other kind of binary classifiers the proposed method tested on benchmark data sets and on one real world application is able to improve the accuracy of other decomposition multi class classifiers producing even faster predictions

learning the kernel matrix with semidefinite programming kernel based learning algorithms work by embedding the data into a euclidean space and then searching for linear relations among the embedded data points the embedding is performed implicitly by specifying the inner products between each pair of points in the embedding space this information is contained in the so called kernel matrix a symmetric and positive semidefinite matrix that encodes the relative positions of all points specifying this matrix amounts to specifying the geometry of the embedding space and inducing a notion of similarity in the input space classical model selection problems in machine learning in this paper we show how the kernel matrix can be learned from data via semidefinite programming sdp techniques when applied to a kernel matrix associated with both training and test data this gives a powerful transductive algorithm using the labeled part of the data one can learn an embedding also for the unlabeled part the similarity between test points is inferred from training points and their labels importantly these learning problems are convex so we obtain a method for learning both the model class and the function without local minima furthermore this approach leads directly to a convex method for learning the 2 norm soft margin parameter in support vector machines solving an important open problem

vc theory of large margin multi category classifiers in the context of discriminant analysis vapnik s statistical learning theory has mainly been developed in three directions the computation of dichotomies with binary valued functions the computation of dichotomies with real valued functions and the computation of polytomies with functions taking their values in finite sets typically the set of categories itself the case of classes of vector valued functions used to compute polytomies has seldom been considered independently which is unsatisfactory for three main reasons first this case encompasses the other ones second it cannot be treated appropriately through a naive extension of the results devoted to the computation of dichotomies third most of the classification problems met in practice involve multiple categories in this paper a vc theory of large margin multi category classifiers is introduced central in this theory are generalized vc dimensions called the dimensions first a uniform convergence bound on the risk of the classifiers of interest is derived the capacity measure involved in this bound is a covering number this covering number can be upper bounded in terms of the dimensions thanks to generalizations of sauer s lemma as is illustrated in the specific case of the scale sensitive natarajan dimension a bound on this latter dimension is then computed for the class of functions on which multi class svms are based this makes it possible to apply the structural risk minimization inductive principle to those machines

ifrowann imbalanced fuzzy rough ordered weighted average nearest neighbor classification imbalanced classification deals with learning from data with a disproportional number of samples in its classes traditional classifiers exhibit poor behavior when facing this kind of data because they do not take into account the imbalanced class distribution four main kinds of solutions exist to solve this problem modifying the data distribution modifying the learning algorithm for considering the imbalance representation including the use of costs for data samples and ensemble methods in this paper we adopt the second type of solution and introduce a classification algorithm for imbalanced data that uses fuzzy rough set theory and ordered weighted average aggregation the proposal considers different strategies to build a weight vector to take into account data imbalance our methods are validated by an extensive experimental study showing statistically better results than 13 other state of the art methods

sparse multinomial kernel discriminant analysis smkda dimensionality reduction via canonical variate analysis cva is important for pattern recognition and has been extended variously to permit more flexibility e g by kernelizing the formulation this can lead to over fitting usually ameliorated by regularization here a method for sparse multinomial kernel discriminant analysis smkda is proposed using a sparse basis to control complexity it is based on the connection between cva and least squares and uses forward selection via orthogonal least squares to approximate a basis generalizing a similar approach for binomial problems classification can be performed directly via minimum mahalanobis distance in the canonical variates smkda achieves state of the art performance in terms of accuracy and sparseness on 11 benchmark datasets

learning graphical models for hypothesis testing and classification sparse graphical models have proven to be a flexible class of multivariate probability models for approximating high dimensional distributions in this paper we propose techniques to exploit this modeling ability for binary classification by discriminatively learning such models from labeled training data i e using both positive and negative samples to optimize for the structures of the two models we motivate why it is difficult to adapt existing generative methods and propose an alternative method consisting of two parts first we develop a novel method to learn tree structured graphical models which optimizes an approximation of the log likelihood ratio we also formulate a joint objective to learn a nested sequence of optimal forests structured models second we construct a classifier by using ideas from boosting to learn a set of discriminative trees the final classifier can interpreted as a likelihood ratio test between two models with a larger set of pairwise features we use cross validation to determine the optimal number of edges in the final model the algorithm presented in this paper also provides a method to identify a subset of the edges that are most salient for discrimination experiments show that the proposed procedure outperforms generative methods such as tree augmented naive bayes and chow liu as well as their boosted counterparts

a regularization framework for multiclass classification a deterministic annealing approach in this paper we propose a general regularization framework for multiclass classification based on discriminant functions since the objective function in the primal optimization problem of this framework is always not differentiable the optimal solution cannot be obtained directly with the aid of the deterministic annealing approach a differentiable objective function is derived subject to a constraint on the randomness of the solution the problem can be approximated by solving a sequence of differentiable optimization problems and such approximation converges to the original problem asymptotically based on this approach class conditional posterior probabilities can be calculated directly without assuming the underlying probabilistic model we also notice that there is a connection between our approach and some existing statistical models such as fisher discriminant analysis and logistic regression

a new approach for imbalanced data classification based on data gravitation imbalanced classification is an important machine learning research topic that troubles most general classification models because of the imbalanced class distribution a newly developed physical inspired classification method i e the data gravitation based classification dgc model performs well in many general classification problems however like other general classifiers the performance of dgc suffers in imbalanced tasks in this study we develop a specific dgc model namely imbalanced dgc idgc model for imbalanced problems the amplified gravitation coefficient agc is introduced for gravitation computing agc is a type of coefficient that contains class imbalance information which can strengthen and weaken the gravitational field of the minority and majority classes we also design a fitness evaluation function in the weight optimization procedure of the data distribution to ensure that the model parameters adapt to the imbalanced class distributions a total of 44 binary class data sets and 15 multiclass imbalanced data sets are used to test the performance of the proposed method experimental results show that the adapted dgc model is effective for imbalanced problems

active learning favoring points near the border between clusters an active learning svm technique taking advantage of the cluster assumption was proposed in each active learning iteration unlabeled instances in the svm margin were first grouped into two clusters then from each cluster points most similar to the other cluster were selected for labeling such points lying near the border between clusters were expected to become support vectors with higher probability the clustering process was performed in the same kernel space as svm with semi supervised k medoids labeled instances were also used to improve the clustering performance experiments showed that the proposed method was efficient and robust to poor initial samples