scaling up cosine interesting pattern discovery a depth first method this paper presents an efficient algorithm called cosminer t for interesting pattern discovery the widely used cosine similarity found to possess the null invariance property and the anti cross support pattern property is adopted as the interestingness measure in cosminer t cosminer t is generally an fp growth like depth first traversal algorithm that rests on an important property of the cosine similarity the conditional anti monotone property camp the combined use of camp and the depth first support ascending traversal strategy enables the pre pruning of uninteresting patterns during the mining process of cosminer t extensive experiments demonstrate the high efficiency of cosminer t in interesting pattern discovery in comparison to the breath first strategy and the post evaluation strategy in particular cosminer t shows its capability in suppressing the generation of cross support patterns and discovering rare but truly interesting patterns finally an interesting case of landmark recognition is presented to illustrate the value of cosine interesting patterns found by cosminer t in real world applications

detection of signals by information theoretic criteria a new approach is presented to the problem of detecting the number of signals in a multichannel time series based on the application of the information theoretic criteria for model selection introduced by akaike aic and by schwartz and rissanen mdl unlike the conventional hypothesis testing based approach the new approach does not requite any subjective threshold settings the number of signals is obtained merely by minimizing the aic or the mdl criteria simulation results that illustrate the performance of the new method for the detection of the number of signals received by a sensor array are presented

stratified feature sampling method for ensemble clustering of high dimensional data high dimensional data with thousands of features present a big challenge to current clustering algorithms sparsity noise and correlation of features are common characteristics of such data another common phenomenon is that clusters in such high dimensional data often exist in different subspaces ensemble clustering is emerging as a prominent technique for improving robustness stability and accuracy of high dimensional data clustering in this paper we propose a stratified sampling method for generating subspace component data sets in ensemble clustering of high dimensional data instead of randomly sampling a subset of features for each component data set in this method we first cluster the features of high dimensional data into a few feature groups called feature strata using stratified sampling we randomly sample some features from each feature stratum and merge the sampled features from different feature strata to generate a component data set in this way the component data sets have better representations of the clustering structure in the original data set comparing with random sampling and random projection methods in synthetic data analysis the component clustering by stratified sampling has demonstrated that the average clustering accuracy was increased without sacrificing clustering diversity we carried out a series of experiments on eight real world data sets from microarray text and image domains to evaluate ensemble clustering methods using three subspace component data generation methods and four consensus functions the experimental results consistently showed that the stratified sampling method produced the best ensemble clustering results in all data sets the ensemble clustering with stratified sampling also outperformed three other ensemble clustering methods which generate component clusters from the entire space of the original data highlightsa new component generation approach is proposed to produce ensemble components stratified sampling is used to generate subspace component data sets the component data can well represent the characteristics of the original data the proposed method can achieve a consistent performance in ensemble clustering the proposed method is easy to implement

a generalized kernel approach to dissimilarity based classification usually objects to be classified are represented by features in this paper we discuss an alternative object representation based on dissimilarity values if such distances separate the classes well the nearest neighbor method offers a good solution however dissimilarities used in practice are usually far from ideal and the performance of the nearest neighbor rule suffers from its sensitivity to noisy examples we show that other more global classification techniques are preferable to the nearest neighbor rule in such cases for classification purposes two different ways of using generalized dissimilarity kernels are considered in the first one distances are isometrically embedded in a pseudo euclidean space and the classification task is performed there in the second approach classifiers are built directly on distance kernels both approaches are described theoretically and then compared using experiments with different dissimilarity measures and datasets including degraded data simulating the problem of missing values

the signal subspace approach for multiple wide band emitter location the rational vector space generalization of the signal subspace approach is presented and applied to the estimation of multiple wide band emitter locations from the signals received at multiple sensors the signal subspace and array manifold concepts first introduced by schmidt are generalized to rational vector space these concepts are used to develop the rational signal subspace theory and prove the signal subspace theorem on which signal subspace algorithms are based the theory is applied in discrete time to derive a class of rational signal subspace algorithms for source location and spectral estimation using unit circle eigendecomposition of multivariate rational models of sensor outputs simulation results are presented for an algorithm in this class including sample statistics from monte carlo trials and comparisons with the cramer rao bound

optimum localization of multiple sources by passive arrays the maximum likelihood ml estimator of the location of multiple sources and the corresponding cramer rao lower bound on the error covariance matrix are derived the derivation is carried out for the general case of correlated sources so that multipath propagation is included as a special case it is shown that the ml processor consists of a bank of beam formers each focused to a different source followed by a variable matrix filter that is controlled by the assumed location of the sources in the special case of uncorrelated sources and very low signal to noise ratio this processor reduces to an aggregate of ml processors for a single source with each processor matched to a different source iterative algorithms for the actual computation of the ml estimator are also presented

multiple emitter location and signal parameter estimation processing the signals received on an array of sensors for the location of the emitter is of great enough interest to have been treated under many special case assumptions the general problem considers sensors with arbitrary locations and arbitrary directional characteristics gain phase polarization in a noise interference environment of arbitrary covariance matrix this report is concerned first with the multiple emitter aspect of this problem and second with the generality of solution a description is given of the multiple signal classification music algorithm which provides asymptotically unbiased estimates of 1 number of incident wavefronts present 2 directions of arrival doa or emitter locations 3 strengths and cross correlations among the incident waveforms 4 noise interference strength examples and comparisons with methods based on maximum likelihood ml and maximum entropy me as well as conventional beamforming are included an example of its use as a multiple frequency estimator operating on time series is included

partially shared latent factor learning with multiview data multiview representations reveal the fundamental attributes of the studied instances from different perspectives some common perspectives are reviewed by multiple views simultaneously while some specific ones are reflected by individual views that is there are two kinds of properties embedded in the multiview data 1 consistency and 2 complementarity different from most multiview learning approaches only focusing on either consistency or complementarity this paper proposes a novel semisupervised multiview learning algorithm called partially shared latent factor pslf learning which jointly exploits both consistent and complementary information among multiple views in pslf a nonnegative matrix factorization nmf based formulation is adopted to learn a compact and comprehensive partially shared latent representation which is composed of common latent factors shared by multiple views and some specific latent factors to each view with the learned representations of multiview data we introduce a robust sparse regression model to predict the cluster labels of labeled data by integrating the nmf based model and the regression model we obtain a unified formulation and propose a multiplicative based alternative algorithm for optimization in addition pslf can learn the weights of different views adaptively according to the reconstruction precisions of data matrices our experimental study indicates different multiview data that contains consistent and complementary information in different degrees in addition the encouraging results of the proposed algorithm are achieved in comparison with the state of the art algorithms on real world data sets

normalized cuts and image segmentation we propose a novel approach for solving the perceptual grouping problem in vision rather than focusing on local features and their consistencies in the image data our approach aims at extracting the global impression of an image we treat image segmentation as a graph partitioning problem and propose a novel global criterion the normalized cut for segmenting the graph the normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups we show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion we applied this approach to segmenting static images as well as motion sequences and found the results to be very encouraging

fractional order embedding canonical correlation analysis and its applications to multi view dimensionality reduction and recognition due to the noise disturbance and limited number of training samples within set and between set sample covariance matrices in canonical correlation analysis cca usually deviate from the true ones in this paper we re estimate within set and between set covariance matrices to reduce the negative effect of this deviation specifically we use the idea of fractional order to respectively correct the eigenvalues and singular values in the corresponding sample covariance matrices and then construct fractional order within set and between set scatter matrices which can obviously alleviate the problem of the deviation on this basis a new approach is proposed to reduce the dimensionality of multi view data for classification tasks called fractional order embedding canonical correlation analysis fecca the proposed method is evaluated on various handwritten numeral face and object recognition problems extensive experimental results on the cenparmi uci at t ar and coil 20 databases show that fecca is very effective and obviously outperforms the existing joint dimensionality reduction or feature extraction methods in terms of classification accuracy moreover its improvements for recognition rates are statistically significant on most cases below the significance level 0 05 a new method namely fecca is presented for multi view dimensionality reduction fractional order within set and between set scatter matrices are constructed by sample spectrum modeling the extracted features by fecca have strong discriminant power for recognition experimental results show fecca has better recognition rates than the existing joint feature extraction methods

a novel clustering method based on hybrid k nearest neighbor graph abstract most of the existing clustering methods have difficulty in processing complex nonlinear data sets to remedy this deficiency in this paper a novel data model termed hybrid k nearest neighbor hknn graph which combines the advantages of mutual k nearest neighbor graph and k nearest neighbor graph is proposed to represent the nonlinear data sets moreover a clustering method based on the hknn graph chknn is proposed the chknn first generates several tight and small subclusters then merges these subclusters by exploiting the connectivity among them in order to select the optimal parameters for chknn we further propose an internal validity index termed k nearest neighbor index knni which can also be used to evaluate the validity of nonlinear clustering results by varying a control parameter experimental results on synthetic and real world data sets as well as that on the video clustering have demonstrated the significant improvement on performance over existing nonlinear clustering methods and internal validity indices

multiple view semi supervised dimensionality reduction multiple view data together with some domain knowledge in the form of pairwise constraints arise in various data mining applications how to learn a hidden consensus pattern in the low dimensional space is a challenging problem in this paper we propose a new method for multiple view semi supervised dimensionality reduction the pairwise constraints are used to derive embedding in each view and simultaneously the linear transformation is introduced to make different embeddings from different pattern spaces comparable hence the consensus pattern can be learned from multiple embeddings of multiple representations we derive an iterating algorithm to solve the above problem some theoretical analyses and out of sample extensions are also provided promising experiments on various data sets together with some important discussions are also presented to demonstrate the effectiveness of the proposed algorithm

computing the local consensus of trees the inference of consensus from a set of evolutionary trees is a fundamental problem in a number of fields such as biology and historical linguistics and many models for inferring this consensus have been proposed in this paper we present a model for deriving what we call a local consensus tree t from a set of trees cal t the model we propose presumes a function f called a total local consensus function which determines for every triple a of species the form that the local consensus tree should take on a we show that all local consensus trees when they exist can be constructed in polynomial time and that many fundamental problems can be solved in linear time we also consider partial local consensus functions and study optimization problems under this model we present linear time algorithms for several variations finally we point out that the local consensus approach ties together many previous approaches to constructing consensus trees

on the robustness of parameter estimation of superimposed signals by dynamic programming we analyze a recently proposed dynamic programming algorithm redp for maximum likelihood ml parameter estimation of superimposed signals in noise we show that it degrades gracefully with deviations from the key assumption of a limited interaction signal model lismo providing exact estimates when the lismo assumption holds exactly in particular we show that the deviations of the redp estimates from the exact ml are continuous in the deviation of the signal model from the lismo assumption these deviations of the redp estimates from the mle are further quantified by a comparison to an ml algorithm with an exhaustive multidimensional search on a lattice in parameter space we derive an explicit expression for the lattice spacing for which the two algorithms have equivalent optimization performance which can be used to assess the robustness of redp to deviations from the lismo assumption the values of this equivalent lattice spacing are found to be small for a classical example of superimposed complex exponentials in noise confirming the robustness of redp for this application

multiset canonical correlations using globality preserving projections with applications to feature extraction and recognition multiset features extracted from the same patterns always represent different characteristics of data thus it is very valuable to perform the extraction on multiple feature sets this paper addresses the issue of multiset correlation feature extraction mcfe in multiple feature representations a novel method is proposed to carry out the mcfe for classification called multiset canonical correlations using globality preserving projections mcc gps which can perform joint dimensionality reduction for high dimensional data mcc gp integrates correlational characteristics of feature pairs and global geometric information of data in the transformed low dimensional space this makes mcc gps have better discriminant ability than a previous method proposed by the authors called multiset integrated canonical correlation analysis micca which only considers correlations for recognition tasks furthermore mcc gp can subsume two popular feature extraction methods into its framework under some constraints this also provides a new insight for these two methods the proposed method is applied to pattern recognition and examined using the coil 100 and eth 80 object databases and ar cmu pie and yale face databases extensive experimental results show that mcc gp outperforms micca and multiset canonical correlation analysis in terms of classification accuracy and efficiency

information theoretic measures for clusterings comparison variants properties normalization and correction for chance information theoretic measures form a fundamental class of measures for comparing clusterings and have recently received increasing interest nevertheless a number of questions concerning their properties and inter relationships remain unresolved in this paper we perform an organized study of information theoretic measures for clustering comparison including several existing popular measures in the literature as well as some newly proposed ones we discuss and prove their important properties such as the metric property and the normalization property we then highlight to the clustering community the importance of correcting information theoretic measures for chance especially when the data size is small compared to the number of clusters present therein of the available information theoretic based measures we advocate the normalized information distance nid as a general measure of choice for it possesses concurrently several important properties such as being both a metric and a normalized measure admitting an exact analytical adjusted for chance form and using the nominal 0 1 range better than other normalized variants

on the structure of f indistinguishability operators abstract some properties concerning the structure of f indistinguishability operators are analyzed it is shown that any of such operators is generated by a family of fuzzy subsets this result since it gives the way to construct f indistinguishabilities facilitates new applications of fuzzy relations the links between f indistinguishability operators and a kind of generalized metrics in the unit interval which are also explored are used to define the canonical generators of an f indistinguishability operator that is the fuzzy partition associated with the operator

on unique localization of multiple sources by passive sensor arrays necessary and sufficient conditions are derived for the unique localization of narrowband sources having the same known center frequency by passive sensor arrays the conditions specify the maximum number of sources that can be uniquely localized by a general array that satisfies some mild geometric constraints the conditions are expressed in terms of the number of sensors and the rank of the correlation matrix of the sources two different conditions are presented the first guarantees uniqueness for every batch of sampled data the second which is weaker guarantees uniqueness for almost every batch of sampled data with the exception of a set of batches of measure zero it is shown that a condition that is slightly weaker than the second one is also necessary

a parameter free community detection method based on centrality and dispersion of nodes in complex networks abstract k means is a simple and efficient clustering algorithm to detect communities in networks however it may suffer from a bad choice of initial seeds also called centers that seriously affect the clustering accuracy and the convergence rate additionally in k means the number of communities should be specified in advance till now it is still an open problem on how to select initial seeds and how to determine the number of communities in this study a new parameter free community detection method named k rank d was proposed first based on the fact that good initial seeds usually have high importance and are dispersedly located in a network we proposed a modified pagerank centrality to evaluate the importance of a node and drew a decision graph to depict the importance and the dispersion of nodes then the initial seeds and the number of communities were selected from the decision graph actively and intuitively as the start parameter of k means experimental results on synthetic and real world networks demonstrate the superior performance of our approach over competing methods for community detection

bayesian detection and estimation of cisoids in colored noise the problem of estimating the number of cisoids in colored noise is addressed it is assumed that the noise can be modeled by an autoregression whose order has also to be estimated a new criterion is proposed for estimating the number of cisoids and the autoregressive model order as well as a new algorithm for estimating the cisoidal frequencies in the derivation a bayesian methodology and subspace decomposition are employed the proposed criterion significantly outperforms the popular mdl and aic as applied in a paper by nagesha and kay in addition an algorithm that reduces the computational complexity of the solution is developed computer simulations that demonstrate the performance of the criterion are included

effective discriminative feature selection with nontrivial solution feature selection and feature transformation the two main ways to reduce dimensionality are often presented separately in this paper a feature selection method is proposed by combining the popular transformation based dimensionality reduction method linear discriminant analysis lda and sparsity regularization we impose row sparsity on the transformation matrix of lda through ell 2 1 norm regularization to achieve feature selection and the resultant formulation optimizes for selecting the most discriminative features and removing the redundant ones simultaneously the formulation is extended to the ell 2 p norm regularized case which is more likely to offer better sparsity when 0 thus the formulation is a better approximation to the feature selection problem an efficient algorithm is developed to solve the ell 2 p norm based optimization problem and it is proved that the algorithm converges when 0 systematical experiments are conducted to understand the work of the proposed method promising experimental results on various types of real world data sets demonstrate the effectiveness of our algorithm

discriminative embedded clustering a framework for grouping high dimensional data in many real applications of machine learning and data mining we are often confronted with high dimensional data how to cluster high dimensional data is still a challenging problem due to the curse of dimensionality in this paper we try to address this problem using joint dimensionality reduction and clustering different from traditional approaches that conduct dimensionality reduction and clustering in sequence we propose a novel framework referred to as discriminative embedded clustering which alternates them iteratively within this framework we are able not only to view several traditional approaches and reveal their intrinsic relationships but also to be stimulated to develop a new method we also propose an effective approach for solving the formulated nonconvex optimization problem comprehensive analyses including convergence behavior parameter determination and computational complexity together with the relationship to other related approaches are also presented plenty of experimental results on benchmark data sets illustrate that the proposed method outperforms related state of the art clustering approaches and existing joint dimensionality reduction and clustering methods

fuzzy submonoids fuzzy preorders and quasi metrics let l be a residuated lattice then given a monoid m we define a galois connection from the lattice of the compatible l preorders in m and the lattice of l submonoids of m given a set s we define a galois connection between the lattice of the l preorders in s and the lattice of l submonoids of the monoid ss i a link with the notion of quasi metric is also established

a dynamic programming algorithm for the maximum likelihood localization of multiple sources a dynamic programming algorithm is presented for simultaneously computing the maximum likelihood ml estimator of the number of multiple sources and their directions of arrival doa in sensor array processing based on the projection matrix decomposition technique the multivariate log likelihood maximization problem is transformed into a multistage one dimensional maximization problem the estimation procedure is obtained with relatively light computational load in addition the estimator gives satisfactory performance in the case of both coherent and incoherent sources the simulation results also show that when the element snr is low and or the number of snapshots is small the dp ml algorithm is much superior to music

the existence of vague objects we consider a category whose objects are defined as triplets s e c where s is a set e is a similarity and c s an object is called crisp if the similarity is crisp vague otherwise we indicate different ways of obtaining a similarity and therefore different ways to obtain an object the paper is mathematical in nature nevertheless i hope that the proposed formalisms are in some ways connected with the very interesting ontological question about the existence of vague objects in real world

maximum likelihood localization of multiple sources by alternating projection an algorithm referred to as apm for computing the maximum likelihood estimator of the locations of simple sources in passive sensor arrays is presented the algorithm is equally applicable to the case of coherent signals and to the case of a single snapshot the algorithm is iterative the maximum of the likelihood function is computed by successive approximations the convergence of the algorithm to the global maximum is demonstrated for a variety of scenarios the key to this global convergence is the initialization scheme

construction of the similarity matrix for the spectral clustering method numerical experiments abstract spectral clustering is a powerful method for finding structure in a dataset through the eigenvectors of a similarity matrix it often outperforms traditional clustering algorithms such as k means when the structure of the individual clusters is highly non convex its accuracy depends on how the similarity between pairs of data points is defined two important items contribute to the construction of the similarity matrix the sparsity of the underlying weighted graph which depends mainly on the distances among data points and the similarity function when a gaussian similarity function is used the choice of the scale parameter can be critical in this paper we examine both items the sparsity and the selection of suitable s based either directly on the graph associated to the dataset or on the minimal spanning tree mst of the graph an extensive numerical experimentation on artificial and real world datasets has been carried out to compare the performances of the methods

near field multiple source localization by passive sensor array the localization of multiple near field sources in a spatially white gaussian noise environment is studied a modified two dimensional 2 d version of the multiple signal classification music algorithm is used to localize the signal sources range and bearing a global optimum maximum likelihood searching approach to localize these sources is discussed it is shown that in the single source situation the covariances of both the 2 d music estimator and the maximum likelihood estimator mle approach the cramer rao lower bound as the number of snapshots increases to infinity in the multiple source situation it is observed that for a high signal to noise ratio snr and a large number of snapshots the root mean square errors rmses of both localization techniques are relatively small however for low snr and or small number of snapshots the performance of the mle is much superior that of the modified 2 d music

graph regularized multiset canonical correlations with applications to joint feature extraction abstract multiset canonical correlation analysis mcca is a powerful technique for analyzing linear correlations among multiple representation data however it usually fails to discover the intrinsic geometrical and discriminating structure of multiple data spaces in real world applications in this paper we thus propose a novel algorithm called graph regularized multiset canonical correlations grmccs which explicitly considers both discriminative and intrinsic geometrical structure in multiple representation data grmcc not only maximizes between set cumulative correlations but also minimizes local intraclass scatter and simultaneously maximizes local interclass separability by using the nearest neighbor graphs on within set data thus it can leverage the power of both mcca and discriminative graph laplacian regularization extensive experimental results on the ar cmu pie yale b at t and eth 80 datasets show that grmcc has more discriminating power and can provide encouraging recognition results in contrast with the state of the art algorithms

cluster ensembles a knowledge reuse framework for combining multiple partitions this paper introduces the problem of combining multiple partitionings of a set of objects into a single consolidated clustering without accessing the features or algorithms that determined these partitionings we first identify several application scenarios for the resultant knowledge reuse framework that we call cluster ensembles the cluster ensemble problem is then formalized as a combinatorial optimization problem in terms of shared mutual information in addition to a direct maximization approach we propose three effective and efficient techniques for obtaining high quality combiners consensus functions the first combiner induces a similarity measure from the partitionings and then reclusters the objects the second combiner is based on hypergraph partitioning the third one collapses groups of clusters into meta clusters which then compete for each object to determine the combined clustering due to the low computational costs of our techniques it is quite feasible to use a supra consensus function that evaluates all three approaches against the objective function and picks the best solution for a given situation we evaluate the effectiveness of cluster ensembles in three qualitatively different application scenarios i where the original clusters were formed based on non identical sets of features ii where the original clustering algorithms worked on non identical sets of objects and iii where a common data set is used and the main purpose of combining multiple clusterings is to improve the quality and robustness of the solution promising results are obtained in all three situations for synthetic as well as real data sets

performance analysis of the approximate dynamic programming algorithm for parameter estimation of superimposed signals we consider the classical problem of fitting a model composed of multiple superimposed signals to noisy data using the criteria of maximum likelihood ml or subspace fitting jointly termed generalized subspace fitting gsf we analyze a previously proposed approximate dynamic programming algorithm adp which provides a computationally efficient solution to the associated multidimensional multimodal optimization problem we quantify the error introduced by the approximations in adp and deviations from the key local interaction signal model lismo modeling assumption in two ways first we upper bound the difference between the exact minimum of the gsf criterion and its value at the adp estimate and compare the adp with gsf estimates obtained by exhaustive multidimensional search on a fine lattice second motivated by the similar accuracy bounds we use perturbation analysis to derive approximate expressions for the mse of the adp estimates these various results provide for the first time an effective tool to predict the performance of the adp algorithm for various signal models at nonasymptotic conditions of interest in practical applications in particular they demonstrate that for the classical problems of sinusoid retrieval and array processing adp performs comparably to exact but expensive maximum likelihood ml over a wide range of signal to noise ratios snrs and is therefore an attractive algorithm