a discrimination analysis for unsupervised feature selection via optic diffraction principle this paper proposes an unsupervised discrimination analysis for feature selection based on a property of the fourier transform of the probability density distribution each feature is evaluated on the basis of a simple observation motivated by the concept of optical diffraction which is invariant under feature scaling the time complexity is o mn where m is number of features and n is number of instances when being applied directly to the given data this approach is also extended to deal with data orientation which is the direction of data alignment therefore the discrimination score of any transformed space can be used for evaluating the original features the experimental results on several real world datasets demonstrate the effectiveness of the proposed method

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

feature selection via dependence maximization we introduce a framework for feature selection based on dependence maximization between the selected features and the labels of an estimation problem using the hilbert schmidt independence criterion the key idea is that good features should be highly dependent on the labels our approach leads to a greedy procedure for feature selection we show that a number of existing feature selectors are special cases of this framework experiments on both artificial and real world data show that our feature selector works well in practice

discriminative least squares regression for multiclass classification and feature selection this paper presents a framework of discriminative least squares regression lsr for multiclass classification and feature selection the core idea is to enlarge the distance between different classes under the conceptual framework of lsr first a technique called e dragging is introduced to force the regression targets of different classes moving along opposite directions such that the distances between classes can be enlarged then the e draggings are integrated into the lsr model for multiclass classification our learning framework referred to as discriminative lsr has a compact model form where there is no need to train two class machines that are independent of each other with its compact form this model can be naturally extended for feature selection this goal is achieved in terms of l2 1 norm of matrix generating a sparse learning model for feature selection the model for multiclass classification and its extension for feature selection are finally solved elegantly and efficiently experimental evaluation over a range of benchmark datasets indicates the validity of our method

local adaptive projection framework for feature selection of labeled and unlabeled data most feature selection methods first compute a similarity matrix by assigning a fixed value to pairs of objects in the whole data or to pairs of objects in a class or by computing the similarity between two objects from the original data the similarity matrix is fixed as a constant in the subsequent feature selection process however the similarities computed from the original data may be unreliable because they are affected by noise features moreover the local structure within classes cannot be recovered if the similarities between the pairs of objects in a class are equal in this paper we propose a novel local adaptive projection lap framework instead of computing fixed similarities before performing feature selection lap simultaneously learns an adaptive similarity matrix mathbf s and a projection matrix mathbf w with an iterative method in each iteration mathbf s is computed from the projected distance with the learned mathbf w and w is computed with the learned mathbf s therefore lap can learn better projection matrix mathbf w by weakening the effect of noise features with the adaptive similarity matrix a supervised feature selection with lap slap method and an unsupervised feature selection with lap ulap method are proposed experimental results on eight data sets show the superiority of slap compared with seven supervised feature selection methods and the superiority of ulap compared with five unsupervised feature selection methods

semisupervised feature selection via spline regression for video semantic recognition to improve both the efficiency and accuracy of video semantic recognition we can perform feature selection on the extracted video features to select a subset of features from the high dimensional feature set for a compact and accurate video data representation provided the number of labeled videos is small supervised feature selection could fail to identify the relevant features that are discriminative to target classes in many applications abundant unlabeled videos are easily accessible this motivates us to develop semisupervised feature selection algorithms to better identify the relevant video features which are discriminative to target classes by effectively exploiting the information underlying the huge amount of unlabeled video data in this paper we propose a framework of video semantic recognition by semisupervised feature selection via spline regression rm s 2 rm fs 2 rm r two scatter matrices are combined to capture both the discriminative information and the local geometry structure of labeled and unlabeled training videos a within class scatter matrix encoding discriminative information of labeled training videos and a spline scatter output from a local spline regression encoding data distribution an ell 2 1 norm is imposed as a regularization term on the transformation matrix to ensure it is sparse in rows making it particularly suitable for feature selection to efficiently solve rm s 2 rm fs 2 rm r we develop an iterative algorithm and prove its convergency in the experiments three typical tasks of video semantic recognition such as video concept detection video classification and human action recognition are used to demonstrate that the proposed rm s 2 rm fs 2 rm r achieves better performance compared with the state of the art methods

comparing two integral means for absolutely continuous functions whose absolute value of the derivative are convex and applications abstract some new estimates for the difference between the integral mean of a function and its mean over a subinterval are established and new applications for special means and probability density functions are also given

generalizing discriminant analysis using the generalized singular value decomposition discriminant analysis has been used for decades to extract features that preserve class separability it is commonly defined as an optimization problem involving covariance matrices that represent the scatter within and between clusters the requirement that one of these matrices be nonsingular limits its application to data sets with certain relative dimensions we examine a number of optimization criteria and extend their applicability by using the generalized singular value decomposition to circumvent the nonsingularity requirement the result is a generalization of discriminant analysis that can be applied even when the sample size is smaller than the dimension of the sample data we use classification results from the reduced representation to compare the effectiveness of this approach with some alternatives and conclude with a discussion of their relative merits

locality regularized linear regression discriminant analysis for feature extraction abstract locality regularized linear regression classification llrc is an effective classifier that shows great potential for face recognition however the original feature space cannot guarantee the classification efficiency of llrc to alleviate this problem we propose a novel dimensionality reduction method called locality regularized linear regression discriminant analysis llrda for feature extraction the proposed llrda is developed according to the decision rule of llrc and seeks to generate a subspace that is discriminant for llrc specifically the intra class and inter class local reconstruction scatters are first defined to characterize the compactness and separability of samples respectively then the objective function for llrda is derived by maximizing the inter class local reconstruction scatter and simultaneously minimizing the intra class local reconstruction scatter extensive experimental results on cmu pie orl feret and yale b face databases validate the effectiveness of our proposed method

manifold regularization a geometric framework for learning from labeled and unlabeled examples we propose a family of learning algorithms based on a new form of regularization that allows us to exploit the geometry of the marginal distribution we focus on a semi supervised framework that incorporates labeled and unlabeled data in a general purpose learner some transductive graph learning algorithms and standard methods including support vector machines and regularized least squares can be obtained as special cases we use properties of reproducing kernel hilbert spaces to prove new representer theorems that provide theoretical basis for the algorithms as a result in contrast to purely graph based approaches we obtain a natural out of sample extension to novel examples and so are able to handle both transductive and truly semi supervised settings we present experimental evidence suggesting that our semi supervised algorithms are able to use unlabeled data effectively finally we have a brief discussion of unsupervised and fully supervised learning within our general framework

comments on globally maximizing locally minimizing unsupervised discriminant projection with application to face and palm biometrics this paper develops an unsupervised discriminant projection udp technique for dimensionality reduction of high dimensional data in small sample size cases udp can be seen as a linear approximation of a multimanifolds based learning framework which takes into account both the local and nonlocal quantities udp characterizes the local scatter as well as the nonlocal scatter seeking to find a projection that simultaneously maximizes the nonlocal scatter and minimizes the local scatter this characteristic makes udp more intuitive and more powerful than the most up to date method locality preserving projection lpp which considers only the local scatter for clustering or classification tasks the proposed method is applied to face and palm biometrics and is examined using the yale feret and ar face image databases and the polyu palmprint database the experimental results show that udp consistently outperforms lpp and pca and outperforms lda when the training sample size per class is small this demonstrates that udp is a good choice for real world biometrics applications

binary coding based feature extraction in remote sensing high dimensional data a binary coding based feature extraction bcfe method is proposed in this paper in the bcfe method the spectral signature of each pixel of hyperspectral image is partitioned into some equal segments then the weighted mean of spectral bands in each segment is considered as an extracted feature bcfe uses a new method for calculation of weights in bcfe the binary codes of class means are obtained then the information contained in the binary values and the edges of class means is used for calculation of weight in each band the experimental results on three real hyperspectral images show the better performance of bcfe compared to some popular and state of the art feature extraction methods from the accuracy and computation time point of views in a small sample size situation

feature selection with kernel class separability classification can often benefit from efficient feature selection however the presence of linearly nonseparable data quick response requirement small sample problem and noisy features makes the feature selection quite challenging in this work a class separability criterion is developed in a high dimensional kernel space and feature selection is performed by the maximization of this criterion to make this feature selection approach work the issues of automatic kernel parameter tuning the numerical stability and the regularization for multi parameter optimization are addressed theoretical analysis uncovers the relationship of this criterion to the radius margin bound of the svms the kfda and the kernel alignment criterion providing more insight on using this criterion for feature selection this criterion is applied to a variety of selection modes with different search strategies extensive experimental study demonstrates its efficiency in delivering fast and robust feature selection

comparing two integral means for absolutely continuous mappings whose derivatives are in l a b and applications estimates of the difference of two integral means on a b c d with c d a b in terms of the sup norm of the derivative and applications for pdfs special means jeffreys divergence and continuous streams are given

the contourlet transform an efficient directional multiresolution image representation the limitations of commonly used separable extensions of one dimensional transforms such as the fourier and wavelet transforms in capturing the geometry of image edges are well known in this paper we pursue a true two dimensional transform that can capture the intrinsic geometrical structure that is key in visual information the main challenge in exploring geometry in images comes from the discrete nature of the data thus unlike other approaches such as curvelets that first develop a transform in the continuous domain and then discretize for sampled data our approach starts with a discrete domain construction and then studies its convergence to an expansion in the continuous domain specifically we construct a discrete domain multiresolution and multidirection expansion using nonseparable filter banks in much the same way that wavelets were derived from filter banks this construction results in a flexible multiresolution local and directional image expansion using contour segments and thus it is named the contourlet transform the discrete contourlet transform has a fast iterated filter bank algorithm that requires an order n operations for n pixel images furthermore we establish a precise link between the developed filter bank and the associated continuous domain contourlet expansion via a directional multiresolution analysis framework we show that with parabolic scaling and sufficient directional vanishing moments contourlets achieve the optimal approximation rate for piecewise smooth functions with discontinuities along twice continuously differentiable curves finally we show some numerical experiments demonstrating the potential of contourlets in several image processing applications

feature selection using a piecewise linear network we present an efficient feature selection algorithm for the general regression problem which utilizes a piecewise linear orthonormal least squares ols procedure the algorithm 1 determines an appropriate piecewise linear network pln model for the given data set 2 applies the ols procedure to the pln model and 3 searches for useful feature subsets using a floating search algorithm the floating search prevents the nesting effect the proposed algorithm is computationally very efficient because only one data pass is required several examples are given to demonstrate the effectiveness of the proposed algorithm

calculating the free energy difference by applying the jarzynski equality to a virtual integrable system the jarzynski equality je provides a nonequilibrium method to measure and calculate the free energy difference fed note that if two systems share the same hamiltonian at two equilibrium states respectively they share the same fed between these two equilibrium states as well therefore the calculation of the fed of a system may be facilitated by considering instead another virtual system designed to this end taking advantage of this flexibility and the je we show that by introducing an integrable virtual system the evolution problem involved in the je can be solved as a consequence fed is expressed in the form of an equilibrium equality in contrast with the nonequilibrium je it is based on numerically this result allows fed to be computed by sampling the canonical ensemble directly and the computational cost can be significantly reduced the effectiveness and efficiency of this scheme are illustrated with numerical studies of several representative model systems

discriminative transfer subspace learning via low rank and sparse representation in this paper we address the problem of unsupervised domain transfer learning in which no labels are available in the target domain we use a transformation matrix to transfer both the source and target data to a common subspace where each target sample can be represented by a combination of source samples such that the samples from different domains can be well interlaced in this way the discrepancy of the source and target domains is reduced by imposing joint low rank and sparse constraints on the reconstruction coefficient matrix the global and local structures of data can be preserved to enlarge the margins between different classes as much as possible and provide more freedom to diminish the discrepancy a flexible linear classifier projection is obtained by learning a non negative label relaxation matrix that allows the strict binary label matrix to relax into a slack variable matrix our method can avoid a potentially negative transfer by using a sparse matrix to model the noise and thus is more robust to different types of noise we formulate our problem as a constrained low rankness and sparsity minimization problem and solve it by the inexact augmented lagrange multiplier method extensive experiments on various visual domain adaptation tasks show the superiority of the proposed method over the state of the art methods the matlab code of our method will be publicly available at http www yongxu org lunwen html

feature selection for unsupervised learning in this paper we identify two issues involved in developing an automated feature subset selection algorithm for unlabeled data the need for finding the number of clusters in conjunction with feature selection and the need for normalizing the bias of feature selection criteria with respect to dimension we explore the feature selection problem and these issues through fssem feature subset selection using expectation maximization em clustering and through two different performance criteria for evaluating candidate feature subsets scatter separability and maximum likelihood we present proofs on the dimensionality biases of these feature criteria and present a cross projection normalization scheme that can be applied to any criterion to ameliorate these biases our experiments show the need for feature selection the need for addressing these two issues and the effectiveness of our proposed solutions

feature selection with redundancy constrained class separability scatter matrix based class separability is a simple and efficient feature selection criterion in the literature however the conventional trace based formulation does not take feature redundancy into account and is prone to selecting a set of discriminative but mutually redundant features in this brief we first theoretically prove that in the context of this trace based criterion the existence of sufficiently correlated features can always prevent selecting the optimal feature set then on top of this criterion we propose the redundancy constrained feature selection rcfs to ensure the algorithm s efficiency and scalability we study the characteristic of the constraints with which the resulted constrained 0 1 optimization can be efficiently and globally solved by using the totally unimodular tum concept in integer programming a necessary condition for such constraints is derived this condition reveals an interesting special case in which qualified redundancy constraints can be conveniently generated via a clustering of features we study this special case and develop an efficient feature selection approach based on dinkelbach s algorithm experiments on benchmark data sets demonstrate the superior performance of our approach to those without redundancy constraints

regularized discriminant analysis abstract linear and quadratic discriminant analysis are considered in the small sample high dimensional setting alternatives to the usual maximum likelihood plug in estimates for the covariance matrices are proposed these alternatives are characterized by two parameters the values of which are customized to individual situations by jointly minimizing a sample based estimate of future misclassification risk computationally fast implementations are presented and the efficacy of the approach is examined through simulation studies and application to data these studies indicate that in many circumstances dramatic gains in classification accuracy can be achieved

discriminative elastic net regularized linear regression in this paper we aim at learning compact and discriminative linear regression models linear regression has been widely used in different problems however most of the existing linear regression methods exploit the conventional zero one matrix as the regression targets which greatly narrows the flexibility of the regression model another major limitation of these methods is that the learned projection matrix fails to precisely project the image features to the target space due to their weak discriminative capability to this end we present an elastic net regularized linear regression enlr framework and develop two robust linear regression models which possess the following special characteristics first our methods exploit two particular strategies to enlarge the margins of different classes by relaxing the strict binary targets into a more feasible variable matrix second a robust elastic net regularization of singular values is introduced to enhance the compactness and effectiveness of the learned projection matrix third the resulting optimization problem of enlr has a closed form solution in each iteration which can be solved efficiently finally rather than directly exploiting the projection matrix for recognition our methods employ the transformed features as the new discriminate representations to make final image classification compared with the traditional linear regression model and some of its variants our method is much more accurate in image classification extensive experiments conducted on publicly available data sets well demonstrate that the proposed framework can outperform the state of the art methods the matlab codes of our methods can be available at http www yongxu org lunwen html

discriminative semi supervised feature selection via manifold regularization feature selection has attracted a huge amount of interest in both research and application communities of data mining we consider the problem of semi supervised feature selection where we are given a small amount of labeled examples and a large amount of unlabeled examples since a small number of labeled samples are usually insufficient for identifying the relevant features the critical problem arising from semi supervised feature selection is how to take advantage of the information underneath the unlabeled data to address this problem we propose a novel discriminative semi supervised feature selection method based on the idea of manifold regularization the proposed approach selects features through maximizing the classification margin between different classes and simultaneously exploiting the geometry of the probability distribution that generates both labeled and unlabeled data in comparison with previous semi supervised feature selection algorithms our proposed semi supervised feature selection method is an embedded feature selection method and is able to find more discriminative features we formulate the proposed feature selection method into a convex concave optimization problem where the saddle point corresponds to the optimal solution to find the optimal solution the level method a fairly recent optimization method is employed we also present a theoretic proof of the convergence rate for the application of the level method to our problem empirical evaluation on several benchmark data sets demonstrates the effectiveness of the proposed semi supervised feature selection method

occurrence of discontinuities in the performance of finite time quantum otto cycles we study a quantum otto cycle in which the strokes are performed in finite time the cycle involves energy measurements at the end of each stroke to allow for the respective determination of work we then optimize for the work and efficiency of the cycle by varying the time spent in the different strokes and find that the optimal value of the ratio of time spent on each stroke goes through sudden changes as the parameters of this cycle vary continuously the position of these discontinuities depends on the optimized quantity under consideration such as the net work output or the efficiency

feature selection by optimizing a lower bound of conditional mutual information abstract a unified framework is proposed to select features by optimizing computationally feasible approximations of high dimensional conditional mutual information cmi between features and their associated class label under different assumptions under this unified framework state of the art information theory based feature selection algorithms are re derived and a new algorithm is proposed to select features by optimizing a lower bound of the cmi with a weaker assumption than those adopted by existing methods the new feature selection method integrates a plug in component to distinguish redundant features from irrelevant ones for improving the feature selection robustness furthermore a novel metric is proposed to evaluate feature selection methods based on simulated data the proposed method has been compared with state of the art feature selection methods based on the new evaluation metric and classification performance of classifiers built upon the selected features the experiment results have demonstrated that the proposed method could achieve promising performance in a variety of feature selection problems

differences between means with bounds from a riemann stieltjes integral an identity for the difference between two integral means is obtained in terms of a riemann stieltjes integral this enables bounds to be procured when the integrand is of bounded variation lipschitzian and monotonic if f is absolutely continuous bounds are also obtained for f lp a b 1 p the usual lebesgue norms this supplements earlier results involving f l a b

achieve higher efficiency at maximum power with finite time quantum otto cycle the optimization of heat engines was intensively explored to achieve higher efficiency while maintaining the output power however most investigations were limited to a few finite time cycles e g the carnot like cycle due to the complexity of the finite time thermodynamics in this paper we propose a class of finite time engine with quantum otto cycle and demonstrate a higher achievable efficiency at maximum power the current model can be widely utilized benefitting from the general c 2 scaling of extra work for a finite time adiabatic process with long control time we apply the adiabatic perturbation method to the quantum piston model and calculate the efficiency at maximum power which is validated with an exact solution

global and local structure preservation for feature selection the recent literature indicates that preserving global pairwise sample similarity is of great importance for feature selection and that many existing selection criteria essentially work in this way in this paper we argue that besides global pairwise sample similarity the local geometric structure of data is also critical and that these two factors play different roles in different learning scenarios in order to show this we propose a global and local structure preservation framework for feature selection glspfs which integrates both global pairwise sample similarity and local geometric data structure to conduct feature selection to demonstrate the generality of our framework we employ methods that are well known in the literature to model the local geometric data structure and develop three specific glspfs based feature selection algorithms also we develop an efficient optimization algorithm with proven global convergence to solve the resulting feature selection problem a comprehensive experimental study is then conducted in order to compare our feature selection algorithms with many state of the art ones in supervised unsupervised and semisupervised learning scenarios the result indicates that 1 our framework consistently achieves statistically significant improvement in selection performance when compared with the currently used algorithms 2 in supervised and semisupervised learning scenarios preserving global pairwise similarity is more important than preserving local geometric data structure 3 in the unsupervised scenario preserving local geometric data structure becomes clearly more important and 4 the best feature selection performance is always obtained when the two factors are appropriately integrated in summary this paper not only validates the advantages of the proposed glspfs framework but also gains more insight into the information to be preserved in different feature selection tasks

feature selection using probabilistic prediction of support vector regression this paper presents a new wrapper based feature selection method for support vector regression svr using its probabilistic predictions the method computes the importance of a feature by aggregating the difference over the feature space of the conditional density functions of the svr prediction with and without the feature as the exact computation of this importance measure is expensive two approximations are proposed the effectiveness of the measure using these approximations in comparison to several other existing feature selection methods for svr is evaluated on both artificial and real world problems the result of the experiments show that the proposed method generally performs better than or at least as well as the existing methods with notable advantage when the dataset is sparse

face recognition based on the uncorrelated discriminant transformation abstract the extraction of discriminant features is the most fundamental and important problem in face recognition this paper presents a method to extract optimal discriminant features for face images by using the uncorrelated discriminant transformation and kl expansion experiments on the orl database and the nust603 database have been performed experimental results show that the uncorrelated discriminant transformation is superior to the foley sammon discriminant transformation and the new method to extract uncorrelated discriminant features for face images is very effective an error rate of 2 5 is obtained with the experiments on the orl database an average error rate of 1 2 is obtained with the experiments on the nust603 database experiments show that by extracting uncorrelated discriminant features face recognition could be performed with higher accuracy on lower than 16 16 resolution mosaic images it is suggested that for the uncorrelated discriminant transformation the optimal face image resolution can be regarded as the resolution m n which makes the dimensionality n mn of the original image vector space be larger and closer to the number of known face classes

work and efficiency of quantum otto cycles in power law trapping potentials we study the performance of a quantum otto cycle operating in trapping potentials of different shapes we show that while both the mean work output and the efficiency of two otto cycles in different trapping potentials can be made equal the work probability distribution will still be strongly affected by the difference in structure of the energy levels to exemplify our results we study the family of potentials of the form v t x x 2q this family of potentials possesses a simple scaling property that allows for analytical insights into the efficiency and work output of the cycle we perform a comparison of quantum otto cycles in various physically relevant scenarios and find that in certain instances the efficiency of the cycle is greater when using potentials with larger values of q while in other cases the efficiency is greater with harmonic traps

effective discriminative feature selection with nontrivial solution feature selection and feature transformation the two main ways to reduce dimensionality are often presented separately in this paper a feature selection method is proposed by combining the popular transformation based dimensionality reduction method linear discriminant analysis lda and sparsity regularization we impose row sparsity on the transformation matrix of lda through ell 2 1 norm regularization to achieve feature selection and the resultant formulation optimizes for selecting the most discriminative features and removing the redundant ones simultaneously the formulation is extended to the ell 2 p norm regularized case which is more likely to offer better sparsity when 0 thus the formulation is a better approximation to the feature selection problem an efficient algorithm is developed to solve the ell 2 p norm based optimization problem and it is proved that the algorithm converges when 0 systematical experiments are conducted to understand the work of the proposed method promising experimental results on various types of real world data sets demonstrate the effectiveness of our algorithm

subclass graph embedding and a marginal fisher analysis paradigm subspace learning techniques have been extensively used for dimensionality reduction dr in many pattern classification problem domains recently methods like subclass discriminant analysis sda and clustering based discriminant analysis cda which use subclass information for the discrimination between the data classes have attracted much attention in parallel important work has been accomplished on graph embedding ge which is a general framework unifying several subspace learning techniques in this paper ge has been extended in order to integrate subclass discriminant information resulting to the novel subclass graph embedding sge framework which is the main contribution of our work it is proven that sge encapsulates a diversity of both supervised and unsupervised unimodal methods like locality preserving projections lpp principal component analysis pca and linear discriminant analysis lda the theoretical link of sda and cda methods with sge is also established along these lines it is shown that sge comprises a generalization of the typical ge framework including subclass dr methods moreover it allows for an easy utilization of kernels for confronting non linearly separable data employing sge in this paper a novel dr algorithm which uses subclass discriminant information called subclass marginal fisher analysis smfa has been proposed through a series of experiments on various real world datasets it is shown that smfa outperforms in most of the cases the state of the art demonstrating the efficacy and power of sge as a platform to develop new methods highlightsgraph embedding is extended in order to integrate subclass informationthe novel subclass graph embedding framework is proposed the kernelized version of the new framework is presentedsubclass graph embedding encapsulates various subspace learning methods a novel subclass marginal fisher analysis method is proposed

ensemble manifold regularized sparse low rank approximation for multiview feature embedding in computer vision and pattern recognition researches the studied objects are often characterized by multiple feature representations with high dimensionality thus it is essential to encode that multiview feature into a unified and discriminative embedding that is optimal for a given task to address this challenge this paper proposes an ensemble manifold regularized sparse low rank approximation emr slra algorithm for multiview feature embedding the emr slra algorithm is based on the framework of least squares component analysis in particular the low dimensional feature representation and the projection matrix are obtained by the low rank approximation of the concatenated multiview feature matrix by considering the complementary property among multiple features emr slra simultaneously enforces the ensemble manifold regularization on the output feature embedding in order to further enhance its robustness against the noise the group sparsity is introduced into the objective formulation to impose direct noise reduction on the input multiview feature matrix since there is no closed form solution for emr slra this paper provides an efficient emr slra optimization procedure to obtain the output feature embedding experiments on the pattern recognition applications confirm the effectiveness of the emr slra algorithm compare with some other multiview feature dimensionality reduction approaches highlightsthis paper proposes a novel emr slra algorithm for multiview feature embedding the least squares component analysis is generalized to multiview version the ensemble manifold regularization is enforced to explore the complementarity the group sparsity is introduced to promote the robustness against the noise an efficient iterative procedure is developed to solve emr slra

top k feature selection framework using robust 0 1 integer programming feature selection fs which identifies the relevant features in a data set to facilitate subsequent data analysis is a fundamental problem in machine learning and has been widely studied in recent years most fs methods rank the features in order of their scores based on a specific criterion and then select the k top ranked features where k is the number of desired features however these features are usually not the top k features and may present a suboptimal choice to address this issue we propose a novel fs framework in this article to select the exact top k features in the unsupervised semisupervised and supervised scenarios the new framework utilizes the l0 2 norm as the matrix sparsity constraint rather than its relaxations such as the l1 2 norm since the l0 2 norm constrained problem is difficult to solve we transform the discrete l0 2 norm based constraint into an equivalent 0 1 integer constraint and replace the 0 1 integer constraint with two continuous constraints the obtained top k fs framework with two continuous constraints is theoretically equivalent to the l0 2 norm constrained problem and can be optimized by the alternating direction method of multipliers admm unsupervised and semisupervised fs methods are developed based on the proposed framework and extensive experiments on real world data sets are conducted to demonstrate the effectiveness of the proposed fs framework

an interval type 2 neural fuzzy system for online system identification and feature elimination we propose an integrated mechanism for discarding derogatory features and extraction of fuzzy rules based on an interval type 2 neural fuzzy system nfs in fact it is a more general scheme that can discard bad features irrelevant antecedent clauses and even irrelevant rules high dimensional input variable and a large number of rules not only enhance the computational complexity of nfss but also reduce their interpretability therefore a mechanism for simultaneous extraction of fuzzy rules and reducing the impact of or eliminating the inferior features is necessary the proposed approach namely an interval type 2 neural fuzzy system for online system identification and feature elimination it2nfs sife uses type 2 fuzzy sets to model uncertainties associated with information and data in designing the knowledge base the consequent part of the it2nfs sife is of takagi sugeno kang type with interval weights the it2nfs sife possesses a self evolving property that can automatically generate fuzzy rules the poor features can be discarded through the concept of a membership modulator the antecedent and modulator weights are learned using a gradient descent algorithm the consequent part weights are tuned via the rule ordered kalman filter algorithm to enhance learning effectiveness simulation results show that it2nfs sife not only simplifies the system architecture by eliminating derogatory irrelevant antecedent clauses rules and features but also maintains excellent performance

finding the best not the most classification on structure data such as graphs has drawn wide interest in recent years due to the lack of explicit features to represent graphs for training classification models extensive studies have been focused on extracting the most discriminative subgraphs features from the training graph dataset to transfer graphs into vector data however such filter based methods suffer from two major disadvantages 1 the subgraph feature selection is separated from the model learning process so the selected most discriminative subgraphs may not best fit the subsequent learning model resulting in deteriorated classification results 2 all these methods rely on users to specify the number of subgraph features k and suboptimally specified k values often result in significantly reduced classification accuracy in this paper we propose a new graph classification paradigm which overcomes the above disadvantages by formulating subgraph feature selection as learning a k dimensional feature space from an implicit and large subgraph space with the optimal k value being automatically determined to achieve the goal we propose a regularized loss minimization driven rlmd feature selection method for graph classification rlmd integrates subgraph selection and model learning into a unified framework to find discriminative subgraphs with guaranteed minimum loss w r t the objective function to automatically determine the optimal number of subgraphs k from the exponentially large subgraph space an effective elastic net and a subgradient method are proposed to derive the stopping criterion so that k can be automatically obtained once rlmd converges the proposed rlmd method enjoys gratifying property including proved convergence and applicability to various loss functions experimental results on real life graph datasets demonstrate significant performance gain highlightsour algorithm selects the optimal subgaphs for graph classification we generalize the column generation technique of gboost for graph classification we use the elastic net to produce sparse and robust solutions for subgraph selection we derive an effective pruning rule for search space reduction we demonstrate the effectiveness of our algorithm

a mathematical theorem as the basis for the second law thomson s formulation applied to equilibrium abstract there are several formulations of the second law and they may in principle have different domains of validity here a simple mathematical theorem is proven which serves as the most general basis for the second law namely the thomson formulation cyclic changes cost energy applied to equilibrium this formulation of the second law is a property akin to particle conservation normalization of the wave function it has been strictly proven for a canonical ensemble and made plausible for a micro canonical ensemble as the derivation does not assume time inversion invariance it is applicable to situations where persistent currents occur this clear cut derivation allows to revive the no perpetuum mobile in equilibrium formulation of the second law and to criticize some assumptions which are widespread in literature the result puts recent results devoted to foundations and limitations of the second law in proper perspective and structurizes this relatively new field of research

second order cone programming formulations for feature selection this paper addresses the issue of feature selection for linear classifiers given the moments of the class conditional densities the problem is posed as finding a minimal set of features such that the resulting classifier has a low misclassification error using a bound on the misclassification error involving the mean and covariance of class conditional densities and minimizing an l1 norm as an approximate criterion for feature selection a second order programming formulation is derived to handle errors in estimation of mean and covariances a tractable robust formulation is also discussed in a slightly different setting the fisher discriminant is derived feature selection for fisher discriminant is also discussed experimental results on synthetic data sets and on real life microarray data show that the proposed formulations are competitive with the state of the art linear programming formulation

multiclass feature selection with kernel gram matrix based criteria feature selection has been an important issue in recent decades to determine the most relevant features according to a given classification problem numerous methods have emerged that take into account support vector machines svms in the selection process such approaches are powerful but often complex and costly in this paper we propose new feature selection methods based on two criteria designed for the optimization of svm kernel target alignment and kernel class separability we demonstrate how these two measures when fully expressed can build efficient and simple methods easily applicable to multiclass problems and iteratively computable with minimal memory requirements an extensive experimental study is conducted both on artificial and real world datasets to compare the proposed methods to state of the art feature selection algorithms the results demonstrate the relevance of the proposed methods both in terms of performance and computational cost

dimensionality reduction of multimodal labeled data by local fisher discriminant analysis reducing the dimensionality of data without losing intrinsic information is an important preprocessing step in high dimensional data analysis fisher discriminant analysis fda is a traditional technique for supervised dimensionality reduction but it tends to give undesired results if samples in a class are multimodal an unsupervised dimensionality reduction method called locality preserving projection lpp can work well with multimodal data due to its locality preserving property however since lpp does not take the label information into account it is not necessarily useful in supervised learning scenarios in this paper we propose a new linear supervised dimensionality reduction method called local fisher discriminant analysis lfda which effectively combines the ideas of fda and lpp lfda has an analytic form of the embedding transformation and the solution can be easily computed just by solving a generalized eigenvalue problem we demonstrate the practical usefulness and high scalability of the lfda method in data visualization and classification tasks through extensive simulation studies we also show that lfda can be extended to non linear dimensionality reduction scenarios by applying the kernel trick

face recognition using laplacianfaces we propose an appearance based face recognition method called the laplacianface approach by using locality preserving projections lpp the face images are mapped into a face subspace for analysis different from principal component analysis pca and linear discriminant analysis lda which effectively see only the euclidean structure of face space lpp finds an embedding that preserves local information and obtains a face subspace that best detects the essential face manifold structure the laplacianfaces are the optimal linear approximations to the eigenfunctions of the laplace beltrami operator on the face manifold in this way the unwanted variations resulting from changes in lighting facial expression and pose may be eliminated or reduced theoretical analysis shows that pca lda and lpp can be obtained from different graph models we compare the proposed laplacianface approach with eigenface and fisherface methods on three different face data sets experimental results suggest that the proposed laplacianface approach provides a better representation and achieves lower error rates in face recognition

l0 norm based structural sparse least square regression for feature selection this paper presents a novel approach for feature selection with regard to the problem of structural sparse least square regression sslsr rather than employing the l1 norm regularization to control the sparsity we directly work with sparse solutions via l0 norm regularization in particular we develop an effective greedy algorithm where the forward and backward steps are combined adaptively to resolve the sslsr problem with the intractable l r 0 norm on the one hand features with the strongest correlation to classes are selected in the forward steps on the other hand redundant features which contribute little to the improvement of the objective function are removed in the backward steps furthermore we provide solid theoretical analysis to prove the effectiveness of the proposed method experimental results on synthetic and real world data sets from different domains also demonstrate the superiority of the proposed method over the state of the arts highlightswe impose l0 norm inequality constraint to build the structural sparse lsr problem we develop an adaptive algorithm to ensure the structural sparsity accurately theoretical results on the efficiency and effectiveness of our method are provided experimental results prove the superiority of our method over the state of the arts

a theorem on the uncorrelated optimal discriminant vectors this paper proposes a theorem on the uncorrelated optimal discriminant vectors uodvs it is proved that the classical optimal discriminant vectors are equivalent to uodv which can be used to extract l 1 uncorrelated discriminant features for l class problems without losing any discriminant information in the meaning of fisher discriminant criterion function experiments on concordia university cenparmi handwritten numeral database indicate that uodvs are much more powerful than the foley sammon optimal discriminant vectors it is believed that when the number of training samples is large the conjugate orthogonal set of discriminant vectors can be much more powerful than the orthogonal set of discriminant vectors

efficient linear discriminant analysis with locality preserving for face recognition linear discriminant analysis lda is one of the most popular techniques for extracting features in face recognition lda captures the global geometric structure however local geometric structure has recently been shown to be effective for face recognition in this paper we propose a novel feature extraction algorithm which integrates both global and local geometric structures we first cast lda as a least square problem based on the spectral regression then regularization technique is used to model the global and local geometric structures furthermore we impose penalty on parameters to tackle the singularity problem and design an efficient model selection algorithm to choose the optimal tuning parameter which balances the tradeoff between the global and local structures experimental results on four well known face data sets show that the proposed integration framework is competitive with traditional face recognition algorithms which use either global or local structure only

laplacian embedded regression for scalable manifold regularization semi supervised learning ssl as a powerful tool to learn from a limited number of labeled data and a large number of unlabeled data has been attracting increasing attention in the machine learning community in particular the manifold regularization framework has laid solid theoretical foundations for a large family of ssl algorithms such as laplacian support vector machine lapsvm and laplacian regularized least squares laprls however most of these algorithms are limited to small scale problems due to the high computational cost of the matrix inversion operation involved in the optimization problem in this paper we propose a novel framework called laplacian embedded regression by introducing an intermediate decision variable into the manifold regularization framework by using insensitive loss we obtain the laplacian embedded support vector regression lapesvr algorithm which inherits the sparse solution from svr also we derive laplacian embedded rls laperls corresponding to rls under the proposed framework both lapesvr and laperls posses a simpler form of a transformed kernel which is the summation of the original kernel and a graph kernel that captures the manifold structure the benefits of the transformed kernel are two fold 1 we can deal with the original kernel matrix and the graph laplacian matrix in the graph kernel separately and 2 if the graph laplacian matrix is sparse we only need to perform the inverse operation for a sparse matrix which is much more efficient when compared with that for a dense one inspired by kernel principal component analysis we further propose to project the introduced decision variable into a subspace spanned by a few eigenvectors of the graph laplacian matrix in order to better reflect the data manifold as well as accelerate the calculation of the graph kernel allowing our methods to efficiently and effectively cope with large scale ssl problems extensive experiments on both toy and real world data sets show the effectiveness and scalability of the proposed framework

global plus local linear discriminant analysis lda is one of the most popular supervised feature extraction techniques used in machine learning and pattern classification however lda only captures global geometrical structure information of the data and ignores the geometrical structure information of local data points though many articles have been published to address this issue most of them are incomplete in the sense that only part of the local information is used we show here that there are total three kinds of local information namely local similarity information local intra class pattern variation and local inter class pattern variation we first propose a new method called enhanced within class lda ewlda algorithm to incorporate the local similarity information and then propose a complete framework called complete global local lda cglda algorithm to incorporate all these three kinds of local information experimental results on two image databases demonstrate the effectiveness of our algorithms there are three kinds of local information that exist for linear discriminant analysis for incorporating the local similarity information we propose the ewlda algorithm for incorporating the three kinds of local information we propose the cglda algorithm

retargeted least squares regression algorithm this brief presents a framework of retargeted least squares regression relsr for multicategory classification the core idea is to directly learn the regression targets from data other than using the traditional zero one matrix as regression targets the learned target matrix can guarantee a large margin constraint for the requirement of correct classification for each data point compared with the traditional least squares regression lsr and a recently proposed discriminative lsr models relsr is much more accurate in measuring the classification error of the regression model furthermore relsr is a single and compact model hence there is no need to train two class binary machines that are independent of each other the convex optimization problem of relsr is solved elegantly and efficiently with an alternating procedure including regression and retargeting as substeps the experimental evaluation over a range of databases identifies the validity of our method

characterization of a family of algorithms for generalized discriminant analysis on undersampled problems a generalized discriminant analysis based on a new optimization criterion is presented the criterion extends the optimization criteria of the classical linear discriminant analysis lda when the scatter matrices are singular an efficient algorithm for the new optimization problem is presented the solutions to the proposed criterion form a family of algorithms for generalized lda which can be characterized in a closed form we study two specific algorithms namely uncorrelated lda ulda and orthogonal lda olda ulda was previously proposed for feature extraction and dimension reduction whereas olda is a novel algorithm proposed in this paper the features in the reduced space of ulda are uncorrelated while the discriminant vectors of olda are orthogonal to each other we have conducted a comparative study on a variety of real world data sets to evaluate ulda and olda in terms of classification accuracy

regularization in matrix relevance learning in this paper we present a regularization technique to extend recently proposed matrix learning schemes in learning vector quantization lvq these learning algorithms extend the concept of adaptive distance measures in lvq to the use of relevance matrices in general metric learning can display a tendency towards oversimplification in the course of training an overly pronounced elimination of dimensions in feature space can have negative effects on the performance and may lead to instabilities in the training we focus on matrix learning in generalized lvq glvq extending the cost function by an appropriate regularization term prevents the unfavorable behavior and can help to improve the generalization ability the approach is first tested and illustrated in terms of artificial model data furthermore we apply the scheme to benchmark classification data sets from the uci repository of machine learning we demonstrate the usefulness of regularization also in the case of rank limited relevance matrices i e matrix learning with an implicit low dimensional representation of the data

laplacian eigenmaps for dimensionality reduction and data representation one of the central problems in machine learning and pattern recognition is to develop appropriate representations for complex data we consider the problem of constructing a representation for data lying on a low dimensional manifold embedded in a high dimensional space drawing on the correspondence between the graph laplacian the laplace beltrami operator on the manifold and the connections to the heat equation we propose a geometrically motivated algorithm for representing the high dimensional data the algorithm provides a computationally efficient approach to nonlinear dimensionality reduction that has locality preserving properties and a natural connection to clustering some potential applications and illustrative examples are discussed

an inequality of ostrowski gruss type and its applications to the estimation of error bounds for some special means and for some numerical quadrature rules abstract in this paper we derive a new inequality of ostrowski gruss type and apply it to the estimation of error bounds for some special means and for some numerical quadrature rules

groupwise retargeted least squares regression in this brief we propose a new groupwise retargeted least squares regression grelsr model for multicategory classification the main motivation behind grelsr is to utilize an additional regularization to restrict the translation values of relsr so that they should be similar within same class by analyzing the regression targets of relsr we propose a new formulation of relsr where the translation values are expressed explicitly on the basis of the new formulation discriminative least squares regression can be regarded as a special case of relsr with zero translation values moreover a groupwise constraint is added to relsr to form the new grelsr model extensive experiments on various machine leaning data sets illustrate that our method outperforms the current state of the art approaches

discrete weighted montgomery identityand discrete ostrowski type inequalities discrete analogue of the weighted montgomery identity is presented for the first time aswell as discrete analogue of ostrowski type inequalities and estimations of difference of two arithmetic means

tree structured feature extraction using mutual information one of the most informative measures for feature extraction fe is mutual information mi in terms of mi the optimal fe creates new features that jointly have the largest dependency on the target class however obtaining an accurate estimate of a high dimensional mi as well as optimizing with respect to it is not always easy especially when only small training sets are available in this paper we propose an efficient tree based method for fe in which at each step a new feature is created by selecting and linearly combining two features such that the mi between the new feature and the class is maximized both the selection of the features to be combined and the estimation of the coefficients of the linear transform rely on estimating 2 d mis the estimation of the latter is computationally very efficient and robust the effectiveness of our method is evaluated on several real world data sets the results show that the classification accuracy obtained by the proposed method is higher than that achieved by other fe methods

locality preserved maximum information projection dimensionality reduction is usually involved in the domains of artificial intelligence and machine learning linear projection of features is of particular interest for dimensionality reduction since it is simple to calculate and analytically analyze in this paper we propose an essentially linear projection technique called locality preserved maximum information projection lpmip to identify the underlying manifold structure of a data set lpmip considers both the within locality and the between locality in the processing of manifold learning equivalently the goal of lpmip is to preserve the local structure while maximize the out of locality global information of the samples simultaneously different from principal component analysis pca that aims to preserve the global information and locality preserving projections lpps that is in favor of preserving the local structure of the data set lpmip seeks a tradeoff between the global and local structures which is adjusted by a parameter alpha so as to find a sub space that detects the intrinsic manifold structure for classification tasks computationally by constructing the adjacency matrix lpmip is formulated as an eigenvalue problem lpmip yields orthogonal basis functions and completely avoids the singularity problem as it exists in lpp further we develop an efficient and stable lpmip qr algorithm for implementing lpmip especially on high dimensional data set theoretical analysis shows that conventional linear projection methods such as weighted pca maximum margin criterion mmc linear discriminant analysis lda and lpp could be derived from the lpmip framework by setting different graph models and constraints extensive experiments on face digit and facial expression recognition show the effectiveness of the proposed lpmip method

acquiring linear subspaces for face recognition under variable lighting previous work has demonstrated that the image variation of many objects human faces in particular under variable lighting can be effectively modeled by low dimensional linear spaces even when there are multiple light sources and shadowing basis images spanning this space are usually obtained in one of three ways a large set of images of the object under different lighting conditions is acquired and principal component analysis pca is used to estimate a subspace alternatively synthetic images are rendered from a 3d model perhaps reconstructed from images under point sources and again pca is used to estimate a subspace finally images rendered from a 3d model under diffuse lighting based on spherical harmonics are directly used as basis images in this paper we show how to arrange physical lighting so that the acquired images of each object can be directly used as the basis vectors of a low dimensional linear space and that this subspace is close to those acquired by the other methods more specifically there exist configurations of k point light source directions with k typically ranging from 5 to 9 such that by taking k images of an object under these single sources the resulting subspace is an effective representation for recognition under a wide range of lighting conditions since the subspace is generated directly from real images potentially complex and or brittle intermediate steps such as 3d reconstruction can be completely avoided nor is it necessary to acquire large numbers of training images or to physically construct complex diffuse harmonic light fields we validate the use of subspaces constructed in this fashion within the context of face recognition

msdlsr margin scalable discriminative least squares regression for multicategory classification in this brief we propose a new margin scalable discriminative least squares regression msdlsr model for multicategory classification the main motivation behind the msdlsr is to explicitly control the margin of dlsr model we first prove that the dlsr is a relaxation of the traditional l 2 support vector machine based on this fact we further provide a theorem on the margin of dlsr with this theorem we add an explicit constraint on dlsr to restrict the number of zeros of dragging values so as to control the margin of dlsr the new model is called msdlsr theoretically we analyze the determination of the margin and support vectors of msdlsr extensive experiments illustrate that our method outperforms the current state of the art approaches on various machine leaning and real world data sets

penalized discriminant analysis fisher s linear discriminant analysis lda is a popular data analytic tool for studying the relationship between a set of predictors and a categorical response in this paper we describe a penalized version of lda it is designed for situations in which there are many highly correlated predictors such as those obtained by discretizing a function or the grey scale values of the pixels in a series of images in cases such as these it is natural efficient and sometimes essential to impose a spatial smoothness constraint on the coefficients both for improved prediction performance and interpretability we cast the classification problem into a regression framework via optimal scoring using this our proposal facilitates the use of any penalized regression technique in the classification setting the technique is illustrated with examples in speech recognition and handwritten character recognition

evaluating stability and comparing output of feature selectors that optimize feature subset cardinality stability robustness of feature selection methods is a topic of recent interest yet often neglected importance with direct impact on the reliability of machine learning systems we investigate the problem of evaluating the stability of feature selection processes yielding subsets of varying size we introduce several novel feature selection stability measures and adjust some existing measures in a unifying framework that offers broad insight into the stability problem we study in detail the properties of considered measures and demonstrate on various examples what information about the feature selection process can be gained we also introduce an alternative approach to feature selection evaluation in the form of measures that enable comparing the similarity of two feature selection processes these measures enable comparing e g the output of two feature selection methods or two runs of one method with different parameters the information obtained using the considered stability and similarity measures is shown to be usable for assessing feature selection methods or criteria as such

the ostrowski s integral inequality for lipschitzian mappings and applications abstract a generalization of ostrowski s inequality for lipschitzian mappings and applications in numerical analysis and for euler s beta function are given

local coordinates alignment with global preservation for dimensionality reduction dimensionality reduction is vital in many fields and alignment based methods for nonlinear dimensionality reduction have become popular recently because they can map the high dimensional data into a low dimensional subspace with the property of local isometry however the relationships between patches in original high dimensional space cannot be ensured to be fully preserved during the alignment process in this paper we propose a novel method for nonlinear dimensionality reduction called local coordinates alignment with global preservation we first introduce a reasonable definition of topology preserving landmarks tpls which not only contribute to preserving the global structure of datasets and constructing a collection of overlapping linear patches but they also ensure that the right landmark is allocated to the new test point then an existing method for dimensionality reduction that has good performance in preserving the global structure is used to derive the low dimensional coordinates of tpls local coordinates of each patch are derived using tangent space of the manifold at the corresponding landmark and then these local coordinates are aligned into a global coordinate space with the set of landmarks in low dimensional space as reference points the proposed alignment method called landmarks based alignment can produce a closed form solution without any constraints while most previous alignment based methods impose the unit covariance constraint which will result in the deficiency of global metrics and undesired rescaling of the manifold experiments on both synthetic and real world datasets demonstrate the effectiveness of the proposed algorithm

suppression of work fluctuations by optimal control an approach based on jarzynski s equality understanding and manipulating work fluctuations in microscale and nanoscale systems are of both fundamental and practical interest for example aspects of work fluctuations will be an important factor in designing nanoscale heat engines in this work an optimal control approach directly exploiting jarzynski s equality is proposed to effectively suppress the fluctuations in the work statistics for systems initially at thermal equilibrium subject to a work protocol but isolated from a bath during the protocol the control strategy is to minimize the deviations of individual values of e ensuremath ensuremath beta w from their ensemble average given by e ensuremath ensuremath beta ensuremath delta f where w is the work ensuremath beta is the inverse temperature and ensuremath delta f is the free energy difference between two equilibrium states it is further shown that even when the system hamiltonian is not fully known it is still possible to suppress work fluctuations through a feedback loop by refining the control target function on the fly through jarzynski s equality itself numerical experiments are based on linear and nonlinear parametric oscillators optimal control results for linear parametric oscillators are also benchmarked with early results based on shortcuts to adiabaticity

multitask sparsity via maximum entropy discrimination a multitask learning framework is developed for discriminative classification and regression where multiple large margin linear classifiers are estimated for different prediction problems these classifiers operate in a common input space but are coupled as they recover an unknown shared representation a maximum entropy discrimination med framework is used to derive the multitask algorithm which involves only convex optimization problems that are straightforward to implement three multitask scenarios are described the first multitask method produces multiple support vector machines that learn a shared sparse feature selection over the input space the second multitask method produces multiple support vector machines that learn a shared conic kernel combination the third multitask method produces a pooled classifier as well as adaptively specialized individual classifiers furthermore extensions to regression graphical model structure estimation and other sparse methods are discussed the maximum entropy optimization problems are implemented via a sequential quadratic programming method which leverages recent progress in fast svm solvers fast monotonic convergence bounds are provided by bounding the med sparsifying cost function with a quadratic function and ensuring only a constant factor runtime increase above standard independent svm solvers results are shown on multitask data sets and favor multitask learning over single task or tabula rasa methods

construction and optimization of a quantum analog of the carnot cycle the quantum analog of carnot cycles in few particle systems consists of two quantum adiabatic steps and two isothermal steps this construction is formally justified by use of a minimum work principle it is then shown using minimal assumptions of work or heat in nanoscale systems that the heat to work efficiency of such quantum heat engine cycles can be further optimized via two conditions regarding the expectation value of some generalized force operators evaluated at equilibrium states in general the optimized efficiency is system specific lower than the carnot efficiency and dependent upon both temperatures of the cold and hot reservoirs simple computational examples are used to illustrate our theory the results should be an important guide towards the design of favorable working conditions of a realistic quantum heat engine

minimal work principle proof and counterexamples the minimal work principle states that work done on a thermally isolated equilibrium system is minimal for adiabatically slow reversible realization of a given process this principle one of the formulations of the second law is studied here for finite possibly large quantum systems interacting with macroscopic sources of work it is shown to be valid as long as the adiabatic energy levels do not cross if level crossing does occur counter examples are discussed showing that the minimal work principle can be violated and that optimal processes are neither adiabatically slow nor reversible the results are corroborated by an exactly solvable model