semi supervised metric learning via topology preserving multiple semi supervised assumptions learning an appropriate distance metric is a critical problem in pattern recognition this paper addresses the problem of semi supervised metric learning we propose a new regularized semi supervised metric learning rssml method using local topology and triplet constraints our regularizer is designed and developed based on local topology which is represented by local neighbors from the local smoothness cluster low density and manifold information point of view the regularizer is then combined with the large margin hinge loss on the triplet constraints in other words we keep a large margin between different labeled samples and in the meanwhile we use the unlabeled samples to regularize it then the semi supervised metric learning method is developed we have performed experiments on classification using publicly available databases to evaluate the proposed method to our best knowledge this is the only method satisfying all the three semi supervised assumptions namely smoothness cluster low density and manifold experimental results have shown that the proposed method outperforms state of the art semi supervised distance metric learning algorithms

quadratic transformations on matrices rank preservers abstract letfbe an algebraically closed field of characteristic not 2 and letx xij be then nmatrix whose entriesxijare independent indeterminates overf now letq x qij x be anothern nmatrix each of whose entriesqij x is a quadraticf polynomial in thexij the main result in this paper is forn 5 q x satisfies rank a2 rimplies rank q a r for alla fn nforr 0 1 and 2 if and only if there exist invertible matricesp1 p2infn nsuch that eitherq x p1x2p2orq x p1 x2 tp2

learning a mahalanobis metric from equivalence constraints many learning algorithms use a metric defined over the input space as a principal tool and their performance critically depends on the quality of this metric we address the problem of learning metrics using side information in the form of equivalence constraints unlike labels we demonstrate that this type of side information can sometimes be automatically obtained without the need of human intervention we show how such side information can be used to modify the representation of the data leading to improved clustering and classification specifically we present the relevant component analysis rca algorithm which is a simple and efficient algorithm for learning a mahalanobis metric we show that rca is the solution of an interesting optimization problem founded on an information theoretic basis if dimensionality reduction is allowed within rca we show that it is optimally accomplished by a version of fisher s linear discriminant that uses constraints moreover under certain gaussian assumptions rca can be viewed as a maximum likelihood estimation of the within class covariance matrix we conclude with extensive empirical evaluations of rca showing its advantage over alternative methods

a complete axiomatization of fuzzy functional dependencies using fuzzy function the notion of functional dependency in classical relational databases corresponds to a partial function therefore it appears natural that the notion of fuzzy functional dependency should correspond to the notion of partial fuzzy function following this approach we introduce the notion of fuzzy functional dependency in fuzzy relational databases and show that the classical inference rules for functional dependencies form a sound and complete formal system in our case

generalized iterative relief for supervised distance metric learning the relief algorithm is a popular approach for feature weighting many extensions of the relief algorithm are developed and i relief is one of the famous extensions in this paper i relief is generalized for supervised distance metric learning to yield a mahananobis distance function the proposed approach is justified by showing that the objective function of the generalized i relief is closely related to the expected leave one out nearest neighbor classification rate in addition the relationships among the generalized i relief the neighbourhood components analysis and graph embedding are also pointed out experimental results on various data sets all demonstrate the superiority of the proposed approach

linear rank one preservers between spaces of matrices with zero trace abstract let x y be a pair of vector spaces over a field f associated with a bilinear form such that x y 0 for all y in y implies that x 0 let x y 0 be the subspace of x y spanned by all decomposable elements x y with x y 0 let u v be any two vector spaces over f in this note we study linear mappings from x y 0 to u v that send nonzero decomposable elements to nonzero decomposable elements and some of its consequences

least squares quantization in pcm it has long been realized that in pulse code modulation pcm with a given ensemble of signals to handle the quantum values should be spaced more closely in the voltage regions where the signal amplitude is more likely to fall it has been shown by panter and dite that in the limit as the number of quanta becomes infinite the asymptotic fractional density of quanta per unit voltage should vary as the one third power of the probability density per unit voltage of signal amplitudes in this paper the corresponding result for any finite number of quanta is derived that is necessary conditions are found that the quanta and associated quantization intervals of an optimum finite quantization scheme must satisfy the optimization criterion used is that the average quantization noise power be a minimum it is shown that the result obtained here goes over into the panter and dite result as the number of quanta become large the optimum quautization schemes for 2 b quanta b 1 2 cdots 7 are given numerically for gaussian and for laplacian distribution of signal amplitudes

learning a mahalanobis distance metric for data clustering and classification distance metric is a key issue in many machine learning algorithms this paper considers a general problem of learning from pairwise constraints in the form of must links and cannot links as one kind of side information a must link indicates the pair of the two data points must be in a same class while a cannot link indicates that the two data points must be in two different classes given must link and cannot link information our goal is to learn a mahalanobis distance metric under this metric we hope the distances of point pairs in must links are as small as possible and those of point pairs in cannot links are as large as possible this task is formulated as a constrained optimization problem in which the global optimum can be obtained effectively and efficiently finally some applications in data clustering interactive natural image segmentation and face pose estimation are given in this paper experimental results illustrate the effectiveness of our algorithm

nonlinear component analysis as a kernel eigenvalue problem a new method for performing a nonlinear form of principal component analysis is proposed by the use of integral operator kernel functions one can efficiently compute principal components in high dimensional feature spaces related to input space by some nonlinear map for instance the space of all possible five pixel products in 16 16 images we give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition

large margin subspace learning for feature selection recent research has shown the benefits of large margin framework for feature selection in this paper we propose a novel feature selection algorithm termed as large margin subspace learning lmsl which seeks a projection matrix to maximize the margin of a given sample defined as the distance between the nearest missing the nearest neighbor with the different label and the nearest hit the nearest neighbor with the same label of the given sample instead of calculating the nearest neighbor of the given sample directly we treat each sample with different same labels with the given sample as a potential nearest missing hint with the probability estimated by kernel density estimation by this way the nearest missing hint is calculated as an expectation of all different same class samples in order to perform feature selection an 2 1 norm is imposed on the projection matrix to enforce row sparsity an efficient algorithm is then proposed to solve the resultant optimization problem comprehensive experiments are conducted to compare the performance of the proposed algorithm with the other five state of the art algorithms rfs spfs mrmr tr and llfs it achieves better performance than the former four compared with the algorithm llfs the proposed algorithm has a competitive performance with however a significantly faster computational

non linear metric learning using pairwise similarity and dissimilarity constraints and the geometrical structure of data the problem of clustering with side information has received much recent attention and metric learning has been considered as a powerful approach to this problem until now various metric learning methods have been proposed for semi supervised clustering although some of the existing methods can use both positive must link and negative cannot link constraints they are usually limited to learning a linear transformation i e finding a global mahalanobis metric in this paper we propose a framework for learning linear and non linear transformations efficiently we use both positive and negative constraints and also the intrinsic topological structure of data we formulate our metric learning method as an appropriate optimization problem and find the global optimum of this problem the proposed non linear method can be considered as an efficient kernel learning method that yields an explicit non linear transformation and thus shows out of sample generalization ability experimental results on synthetic and real world data sets show the effectiveness of our metric learning method for semi supervised clustering tasks

gradation of being fuzzy function abstract for crisp sets x and y considering two fuzzy equalities e x e y on x and y respectively the concept of fuzzy function and several types of it were studied in m demirci fuzzy sets and systems 106 1999 239 246 in this paper for a fuzzy relation f on x y considering the fuzzy equalities e x on x and e y on y the degree of f of being fuzzy function being surjective being injective and being bijective is defined then setting the connections between several types of degree of being fuzzy function and various types of fuzzy function the essential properties of these degrees are investigated

cross modal similarity learning with active queries a novel approach coslaq for cross modal similarity learning with active queries is proposed disagreement based active query strategy explores the most valuable supervised information uncertainty of similarity learning model is utilized to avoid querying outliers and noises in real applications data is usually collected from heterogeneous sources and represented with multiple modalities to facilitate the analysis of such complex tasks it is important to learn an effective similarity across different modalities existing similarity learning methods usually requires a large number of labeled training examples leading to high labeling costs in this paper we propose a novel approach coslaq for active cross modal similarity learning which actively queries the most important supervised information based on the disagreement among different intra modal and inter modal similarities furthermore the closeness to decision boundary of similarity is utilized to avoid querying outliers and noises experiments on benchmark datasets demonstrate that the proposed method can reduce the labeling cost effectively

fuzzy functions and their fundamental properties in this paper it is noticed that the definition of fuzzy equality of sasaki 1993 is not concordant with that of es and coker 1995 and sostak 1985 1988 furthermore the construction of fuzzy equality and fuzzy function is far from the way of natural fuzzification for this purpose the fuzzy equality of two points of a crisp set which is more convenient than that in es and coker 1995 and sostak 1985 1988 and the fuzzy function having weaker conditions than that is given in sasaki 1993 are redefined in much more natural way then various types of fuzzy functions are introduced and their fundamental properties are established

positive semidefinite metric learning using boosting like algorithms the success of many machine learning and pattern recognition methods relies heavily upon the identification of an appropriate distance metric on the input data it is often beneficial to learn such a metric from the input training data instead of using a default one such as the euclidean distance in this work we propose a boosting based technique termed boostmetric for learning a quadratic mahalanobis distance metric learning a valid mahalanobis distance metric requires enforcing the constraint that the matrix parameter to the metric remains positive semidefinite semidefinite programming is often used to enforce this constraint but does not scale well and is not easy to implement boostmetric is instead based on the observation that any positive semidefinite matrix can be decomposed into a linear combination of trace one rank one matrices boostmetric thus uses rank one positive semidefinite matrices as weak learners within an efficient and scalable boosting based learning process the resulting methods are easy to implement efficient and can accommodate various types of constraints we extend traditional boosting algorithms in that its weak learner is a positive semidefinite matrix with trace and rank being one rather than a classifier or regressor experiments on various data sets demonstrate that the proposed algorithms compare favorably to those state of the art methods in terms of classification accuracy and running time

distance metric learning with eigenvalue optimization the main theme of this paper is to develop a novel eigenvalue optimization framework for learning a mahalanobis metric within this context we introduce a novel metric learning approach called dml eig which is shown to be equivalent to a well known eigenvalue optimization problem called minimizing the maximal eigenvalue of a symmetric matrix overton 1988 lewis and overton 1996 moreover we formulate lmnn weinberger et al 2005 one of the state of the art metric learning methods as a similar eigenvalue optimization problem this novel framework not only provides new insights into metric learning but also opens new avenues to the design of efficient metric learning algorithms indeed first order algorithms are developed for dml eig and lmnn which only need the computation of the largest eigenvector of a matrix per iteration their convergence characteristics are rigorously established various experiments on benchmark data sets show the competitive performance of our new approaches in addition we report an encouraging result on a difficult and challenging face verification data set called labeled faces in the wild lfw

convex clustering with metric learning abstract the convex clustering formulation of chi and lange 2015 is revisited while this formulation can be precisely and efficiently solved it uses the standard euclidean metric to measure the distance between the data points and their corresponding cluster centers and hence its performance deteriorates significantly in the presence of outlier features to address this issue this paper considers a formulation that combines convex clustering with metric learning it is shown that 1 for any given positive definite mahalanobis distance metric the problem of convex clustering can be precisely and efficiently solved using the alternating direction method of multipliers 2 the problem of learning a positive definite mahalanobis distance metric admits a closed form solution 3 an algorithm that alternates between convex clustering and metric learning can provide a significant performance boost over not only the original convex clustering formulation but also the recently proposed robust convex clustering formulation of wang et al 2017

enhanced fuzzy system models with improved fuzzy clustering algorithm although traditional fuzzy models have proven to have high capacity of approximating the real world systems they have some challenges such as computational complexity optimization problems subjectivity etc in order to solve some of these problems this paper proposes a new fuzzy system modeling approach based on improved fuzzy functions to model systems with continuous output variable the new modeling approach introduces three features i an improved fuzzy clustering ifc algorithm ii a new structure identification algorithm and iii a nonparametric inference engine the ifc algorithm yields simultaneous estimates of parameters of c regression models together with fuzzy c partitioning of the data to calculate improved membership values with a new membership function the structure identification of the new approach utilizes ifc instead of standard fuzzy c means clustering algorithm to fuzzy partition the data and it uses improved membership values as additional input variables along with the original scalar input variables for two different choices of regression methods least squares estimation or support vector regression to determine ldquofuzzy functionsrdquo for each cluster with novel ifc one could learn the system behavior more accurately compared to other fsm models the nonparametric inference engine is a new approach which uses the alike nearest neighbor method for reasoning empirical comparisons indicate that the proposed approach yields comparable or better accuracy than fuzzy or neuro fuzzy models based on fuzzy rules bases as well as other soft computing methods

distance metric learning for large margin nearest neighbor classification the accuracy of k nearest neighbor knn classification depends significantly on the metric used to compute distances between different examples in this paper we show how to learn a mahalanobis distance metric for knn classification from labeled examples the mahalanobis metric can equivalently be viewed as a global linear transformation of the input space that precedes knn classification using euclidean distances in our approach the metric is trained with the goal that the k nearest neighbors always belong to the same class while examples from different classes are separated by a large margin as in support vector machines svms the margin criterion leads to a convex optimization based on the hinge loss unlike learning in svms however our approach requires no modification or extension for problems in multiway as opposed to binary classification in our framework the mahalanobis distance metric is obtained as the solution to a semidefinite program on several data sets of varying size and difficulty we find that metrics trained in this way lead to significant improvements in knn classification sometimes these results can be further improved by clustering the training examples and learning an individual metric within each cluster we show how to learn and combine these local metrics in a globally integrated manner

rapid and brief communication extending the relevant component analysis algorithm for metric learning using both positive and negative equivalence constraints relevant component analysis rca is a recently proposed metric learning method for semi supervised learning applications it is a simple and efficient method that has been applied successfully to give impressive results however rca can make use of supervisory information in the form of positive equivalence constraints only in this paper we propose an extension to rca that allows both positive and negative equivalence constraints to be incorporated experimental results show that the extended rca algorithm is effective

bilinear transformations on matrices rank preservers let a be any element in an algebraically closed field f of characteristic not 2 and let m f x f 4 f be a bilinear map on the algebra f of iz x n matrices over f we prove for n 5 that if ab aba 0 implies m a b 0 and rank ab aba 1 implies rank m a b 1 for all matrices a b e f then there exist invertible matrices p q e f such that either m x y p xy ayx q or m x y p xy ayx q 0 elsevier science inc 1997 throughout f is a field and fnx is the algebra of n x n matrices over f during the past 35 years a substantial effort has been made to solve preserver problems for linear maps on f an entire issue of linear and multilinear algebra 3 consists of recent survey articles on every aspect of the subject the linear preserver problem is to characterize all linear maps on fnxn that preserve some function of a matrix or some subset of matrices one such problem is to characterize all linear maps l such that

logdet divergence based metric learning with triplet constraints and its applications how to select and weigh features has always been a difficult problem in many image processing and pattern recognition applications a data dependent distance measure can address this problem to a certain extent and therefore an accurate and efficient metric learning becomes necessary in this paper we propose a logdet divergence based metric learning with triplet constraints ldmlt approach which can learn mahalanobis distance metric accurately and efficiently first of all we demonstrate the good properties of triplet constraints and apply it in logdet divergence based metric learning model then to deal with high dimensional data we apply a compressed representation method to learn store and evaluate mahalanobis matrix efficiently besides a dynamic triplets building strategy is proposed to build a feedback from the obtained mahalanobis matrix to the triplet constraints which can further improve the ldmlt algorithm furthermore the proposed method is applied to various applications including pattern recognition facial expression recognition and image retrieval the results demonstrate the improved performance of the proposed approach

supervised distance metric learning through maximization of the jeffrey divergence over the past decades distance metric learning has attracted a lot of interest in machine learning and related fields in this work we propose an optimization framework for distance metric learning via linear transformations by maximizing the jeffrey divergence between two multivariate gaussian distributions derived from local pairwise constraints in our method the distance metric is trained on positive and negative difference spaces which are built from the neighborhood of each training instance so that the local discriminative information is preserved we show how to solve this problem with a closed form solution rather than using tedious optimization procedures the solution is easy to implement and tractable for large scale problems experimental results are presented for both a linear and a kernelized version of the proposed method for k nearest neighbors classification we obtain classification accuracies superior to the state of the art distance metric learning methods in several cases while being competitive in others highlightswe propose a novel distance metric learning method dmlmj for classification dmlmj is simple to implement and it can be solved analytically we extend dmlmj into a kernelized version to tackle non linear problems experiments on several data sets show the effectiveness of the proposed method