a signal processing approach to generalized 1 d total variation total variation tv is a powerful method that brings great benefit for edge preserving regularization despite being widely employed in image processing it has restricted applicability for 1 d signal processing since piecewise constant signals form a rather limited model for many applications here we generalize conventional tv in 1 d by extending the derivative operator which is within the regularization term to any linear differential operator this provides flexibility for tailoring the approach to the presence of nontrivial linear systems and for different types of driving signals such as spike like piecewise constant and so on conventional tv remains a special case of this general framework we illustrate the feasibility of the method by considering a nontrivial linear system and different types of driving signals

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

border peeling clustering in this paper we present a novel non parametric clustering technique our technique is based on the notion that each latent cluster is comprised of layers that surround its core where the external layers or border points implicitly separate the clusters unlike previous techniques such as dbscan where the cores of the clusters are defined directly by their densities here the latent cores are revealed by a progressive peeling of the border points analyzing the density of the local neighborhoods allows identifying the border points and associating them with points of inner layers we show that the peeling process adapts to the local densities and characteristics to successfully separate adjacent clusters of possibly different densities we extensively tested our technique on large sets of labeled data including high dimensional datasets of deep features that were trained by a convolutional neural network we show that our technique is competitive to other state of the art non parametric methods using a fixed set of parameters throughout the experiments

mathematical properties of the jpeg2000 wavelet filters the legall 5 3 and daubechies 9 7 filters have risen to special prominence because they were selected for inclusion in the jpeg2000 standard we determine their key mathematical features riesz bounds order of approximation and regularity holder and sobolev we give approximation theoretic quantities such as the asymptotic constant for the l sup 2 error and the angle between the analysis and synthesis spaces which characterizes the loss of performance with respect to an orthogonal projection we also derive new asymptotic error formulae that exhibit bound constants that are proportional to the magnitude of the first nonvanishing moment of the wavelet the daubechies 9 7 stands out because it is very close to orthonormal but this turns out to be slightly detrimental to its asymptotic performance when compared to other wavelets with four vanishing moments

least squares recursive projection twin support vector machine for classification in this paper we formulate a least squares version of the recently proposed projection twin support vector machine ptsvm for binary classification this formulation leads to extremely simple and fast algorithm called least squares projection twin support vector machine lsptsvm for generating binary classifiers different from ptsvm we add a regularization term ensuring the optimization problems in our lsptsvm are positive definite and resulting better generalization ability instead of usually solving two dual problems we solve two modified primal problems by solving two systems of linear equations whereas ptsvm need to solve two quadratic programming problems along with two systems of linear equations our experiments on publicly available datasets indicate that our lsptsvm has comparable classification accuracy to that of ptsvm but with remarkably less computational time

reproduction of exponential polynomials by multivariate non stationary subdivision schemes with a general dilation matrix we study scalar multivariate non stationary subdivision schemes with a general integer dilation matrix we characterize the capability of such schemes to reproduce exponential polynomials in terms of simple algebraic conditions on their symbols these algebraic conditions provide a useful theoretical tool for checking the reproduction properties of existing schemes and for constructing new schemes with desired reproduction capabilities and other enhanced properties we illustrate our results with several examples

unsupervised segmentation of markov random field modeled textured images using selectionist relaxation among the existing texture segmentation methods those relying on markov random fields have retained substantial interest and have proved to be very efficient in supervised mode the use of markov random fields in unsupervised mode is however hampered by the parameter estimation problem the recent solutions proposed to overcome this difficulty rely on assumptions about the shapes of the textured regions or about the number of textures in the input image that may not be satisfied in practice in this paper an evolutionary approach selectionist relaxation is proposed as a solution to the problem of segmenting markov random field modeled textures in unsupervised mode in selectionist relaxation the computation is distributed among a population of units that iteratively evolves according to simple and local evolutionary rules a unit is an association between a label and a texture parameter vector the units whose likelihood is high are allowed to spread over the image and to replace the units that receive lower support from the data consequently some labels are growing while others are eliminated starting with an initial random population this evolutionary process eventually results in a stable labelization of the image which is taken as the segmentation in this work the generalized ising model is used to represent textured data because of the awkward nature of the partition function in this model a high temperature approximation is introduced to allow the evaluation of unit likelihoods experimental results on images containing various synthetic and natural textures are reported

a multi prototype clustering algorithm clustering is an important unsupervised learning technique widely used to discover the inherent structure of a given data set some existing clustering algorithms uses single prototype to represent each cluster which may not adequately model the clusters of arbitrary shape and size and hence limit the clustering performance on complex data structure this paper proposes a clustering algorithm to represent one cluster by multiple prototypes the squared error clustering is used to produce a number of prototypes to locate the regions of high density because of its low computational cost and yet good performance a separation measure is proposed to evaluate how well two prototypes are separated multiple prototypes with small separations are grouped into a given number of clusters in the agglomerative method new prototypes are iteratively added to improve the poor cluster separations as a result the proposed algorithm can discover the clusters of complex structure with robustness to initial settings experimental results on both synthetic and real data sets demonstrate the effectiveness of the proposed clustering algorithm

multistage branch and bound merging for planar surface segmentation in disparity space an iterative split and merge framework for the segmentation of planar surfaces in the disparity space is presented disparity of a scene is modeled by approximating various surfaces in the scene to be planar in the split phase the number of planar surfaces along with the underlying plane parameters is assumed to be known from the initialization or from the previous merge phase based on these parameters planar surfaces in the disparity image are labeled to minimize the residuals between the actual disparity and the modeled disparity the labeled planar surfaces are separated into spatially continuous regions which are treated as candidates for the merging that follows the regions are merged together under a maximum variance constraint while maximizing the merged area a multistage branch and bound algorithm is proposed to carry out this optimization efficiently each stage of the branch and bound algorithm separates a planar surface from the set of spatially continuous regions the multistage merging estimates the number of planar surfaces and their labeling the splitting and the multistage merging is repeated till convergence is reached or satisfactory results are achieved experimental results are presented for variety of stereo image data

algebraic conditions on non stationary subdivision symbols for exponential polynomial reproduction we present an accurate investigation of the algebraic conditions that the symbols of a non singular univariate binary non stationary subdivision scheme should fulfill in order to reproduce spaces of exponential polynomials a subdivision scheme is said to possess the property of reproducing exponential polynomials if for any initial data uniformly sampled from some exponential polynomial function the scheme yields the same function in the limit the importance of this property is due to the fact that several curves obtained by combinations of exponential polynomials such as conic sections spirals or special trigonometric and hyperbolic functions are considered of interest in geometric modeling since the space of exponential polynomials trivially includes standard polynomials this work extends the theory on polynomial reproduction to the non stationary context a significant application of the derived algebraic conditions on the subdivision symbols is the construction of new non stationary subdivision schemes with specific reproduction properties

mblda a novel multiple between class linear discriminant analysis abstract linear discriminant analysis lda with its extensions is a group of classical methods in dimensionality reduction for supervised learning however when some classes are far away from the others it may be difficult to find the optimal direction by lda because of the average between class scatter moreover ldas are always time consuming for high dimensional problem since the involved generalized eigenvalue problem is needed to be solved in this paper a multiple between class linear discriminant analysis mblda is proposed for dimensionality reduction mblda finds the transformation directions by approximating the solution to a min max programming problem leading to well separability in the reduced space with a fast learning speed on the high dimensional problem it is proved theoretically that the proposed method can deal with the special generalized eigenvalue problem by solving a underdetermined homogeneous system of linear equations experimental results on the artificial and benchmark datasets show that mblda can not only reduce the dimension to a powerful linear discriminant level but also have a fast learning speed

cardinal exponential splines part i theory and filtering algorithms causal exponentials play a fundamental role in classical system theory starting from those elementary building blocks we propose a complete and self contained signal processing formulation of exponential splines defined on a uniform grid we specify the corresponding b spline basis functions and investigate their reproduction properties green function and exponential polynomials we also characterize their stability riesz bounds we show that the exponential b spline framework allows an exact implementation of continuous time signal processing operators including convolution differential operators and modulation by simple processing in the discrete b spline domain we derive efficient filtering algorithms for multiresolution signal extrapolation and approximation extending earlier results for polynomial splines finally we present a new asymptotic error formula that predicts the magnitude and the nth order decay of the l sub 2 approximation error as a function of the knot spacing t

sparse linear discriminant analysis by thresholding for high dimensional data in many social economical biological and medical studies one objective is to classify a subject into one of several classes based on a set of variables observed from the subject because the probability distribution of the variables is usually unknown the rule of classification is constructed using a training sample the well known linear discriminant analysis lda works well for the situation where the number of variables used for classification is much smaller than the training sample size because of the advance in technologies modern statistical studies often face classification problems with the number of variables much larger than the sample size and the lda may perform poorly we explore when and why the lda has poor performance and propose a sparse lda that is asymptotically optimal under some sparsity conditions on the unknown parameters for illustration of application we discuss an example of classifying human cancer into two classes of leukemia based on a set of 7 129 genes and a training sample of size 72 a simulation is also conducted to check the performance of the proposed method

convergence of nonstationary cascade algorithms a nonstationary multiresolution of l 2 mathbb r s is generated by a sequence of scaling functions phi k in l 2 mathbb r s k in mathbb z we consider phi k that is the solution of the nonstationary refinement equations phi k m sum j h k 1 j phi k 1 m cdot j k in mathbb z where h k is finitely supported for each k and m is a dilation matrix we study various forms of convergence in l 2 mathbb r s of the corresponding nonstationary cascade algorithm phi k n m sum j h k 1 j phi k 1 n 1 m cdot j as k or n tends to infty it is assumed that there is a stationary refinement equation at infty with filter sequence h and that sum k h k j h j infty the results show that the convergence of the nonstationary cascade algorithm is determined by the spectral properties of the transition operator associated with h

multiclass classifiers based on dimension reduction with generalized lda linear discriminant analysis lda has been widely used for dimension reduction of data sets with multiple classes the lda has been recently extended to various generalized lda methods that are applicable regardless of the relative sizes between the data dimension and the number of data items in this paper we propose several multiclass classifiers based on generalized lda glda algorithms taking advantage of the dimension reducing transformation matrix without requiring additional training or parameter optimization a marginal linear discriminant classifier mldc a bayesian linear discriminant classifier bldc and a one dimensional bldc are introduced for multiclass classification our experimental results illustrate that these classifiers produce higher ten fold cross validation accuracy than knn and centroid based classifiers in the reduced dimensional space obtained from glda

multiresolution subdivision snakes we present a new family of snakes that satisfy the property of multiresolution by exploiting subdivision schemes we show in a generic way how to construct such snakes based on an admissible subdivision mask we derive the necessary energy formulations and provide the formulas for their efficient computation depending on the choice of the mask such models have the ability to reproduce trigonometric or polynomial curves they can also be designed to be interpolating a property that is useful in user interactive applications we provide explicit examples of subdivision snakes and illustrate their use for the segmentation of bioimages we show that they are robust in the presence of noise and provide a multiresolution algorithm to enlarge their basin of attraction which decreases their dependence on initialization compared to singleresolution snakes we show the advantages of the proposed model in terms of computation and segmentation of structures with different sizes

stochastic models for sparse and piecewise smooth signals we introduce an extended family of continuous domain stochastic models for sparse piecewise smooth signals these are specified as solutions of stochastic differential equations or equivalently in terms of a suitable innovation model the latter is analogous conceptually to the classical interpretation of a gaussian stationary process as filtered white noise the two specific features of our approach are 1 signal generation is driven by a random stream of dirac impulses poisson noise instead of gaussian white noise and 2 the class of admissible whitening operators is considerably larger than what is allowed in the conventional theory of stationary processes we provide a complete characterization of these finite rate of innovation signals within gelfand s framework of generalized stochastic processes we then focus on the class of scale invariant whitening operators which correspond to unstable systems we show that these can be solved by introducing proper boundary conditions which leads to the specification of random spline type signals that are piecewise smooth these processes are the poisson counterpart of fractional brownian motion they are nonstationary and have the same 1 type spectral signature we prove that the generalized poisson processes have a sparse representation in a wavelet like basis subject to some mild matching condition we also present a limit example of sparse process that yields a map signal estimator that is equivalent to the popular tv denoising algorithm

warped k means an algorithm to cluster sequentially distributed data many devices generate large amounts of data that follow some sort of sequentiality e g motion sensors e pens eye trackers etc and often these data need to be compressed for classification storage and or retrieval tasks traditional clustering algorithms can be used for this purpose but unfortunately they do not cope with the sequential information implicitly embedded in such data thus we revisit the well known k means algorithm and provide a general method to properly cluster sequentially distributed data we present warped k means wkm a multi purpose partitional clustering procedure that minimizes the sum of squared error criterion while imposing a hard sequentiality constraint in the classification step we illustrate the properties of wkm in three applications one being the segmentation and classification of human activity wkm outperformed five state of the art clustering techniques to simplify data trajectories achieving a recognition accuracy of near 97 which is an improvement of around 66 over their peers moreover such an improvement came with a reduction in the computational cost of more than one order of magnitude

why can lda be performed in pca transformed space abstract pca plus lda is a popular framework for linear discriminant analysis lda in high dimensional and singular case in this paper we focus on building a theoretical foundation for this framework moreover we point out the weakness of the previous lda based methods and suggest a complete pca plus lda algorithm experimental results on orl face image database indicate that the proposed method is more effective than the previous ones

pattern identification in dynamical systems via symbolic time series analysis this paper presents symbolic time series analysis stsa of multi dimensional measurement data for pattern identification in dynamical systems the proposed methodology is built upon concepts derived from information theory and automata theory the objective is not merely to classify the time series patterns but also to identify the variations therein to achieve this goal a symbol alphabet is constructed from raw data through partitioning of the data space the maximum entropy method of partitioning is extended to multi dimensional space the resulting symbol sequences generated from time series data are used to model the dynamical information as finite state automata and the patterns are represented by the stationary state probability distributions a novel procedure for determining the structure of the finite state automata based on entropy rate is introduced the diversity among the observed patterns is quantified by a suitable measure the efficacy of the stsa technique for pattern identification is demonstrated via laboratory experimentation on nonlinear systems

on the growth and stability of real analytic functions this paper addresses the issue of whether integrals of real analytic functions remain finite under small deformations an approach based on uniform estimates for certain classes of one dimensional integrals is introduced it is powerful enough to recover the stability properties of real integrals in two dimensions which follow from the work of karpushkin as well as produce new results in higher dimensions in dimension three the new stability results are sharp as shown by the well known example of varchenko

twin support vector machine for clustering the twin support vector machine twsvm is one of the powerful classification methods in this brief a twsvm type clustering method called twin support vector clustering twsvc is proposed our twsvc includes both linear and nonlinear versions it determines k cluster center planes by solving a series of quadratic programming problems to make twsvc more efficient and stable an initialization algorithm based on the nearest neighbor graph is also suggested the experimental results on several benchmark data sets have shown a comparable performance of our twsvc

biorthogonal bases of compactly supported wavelets orthonormal bases of compactly supported wavelet bases correspond to subband coding schemes with exact reconstruction in which the analysis and synthesis filters coincide we show here that under fairly general conditions exact reconstruction schemes with synthesis filters different from the analysis filters give rise to two dual riesz bases of compactly supported wavelets we give necessary and sufficient conditions for biorthogonality of the corresponding scaling functions and we present a sufficient condition for the decay of their fourier transforms we study the regularity of these biorthogonal bases we provide several families of examples all symmetric corresponding to linear phase filters in particular we can construct symmetric biorthogonal wavelet bases with arbitrarily high preassigned regularity we also show how to construct symmetric biorthogonal wavelet bases close to a nonsymmetric orthonormal basis

a family of level dependent biorthogonal wavelet filters for image compression abstract in this work we explore the construction and the applications of a special family of level dependent biorthogonal filters i e filters whose taps depend on the scaling level such a family is generated from a class of functions all related through level dependent or nonstationary refinement equations which contains cardinal polynomial b splines as a particular case the greater flexibility offered by the nonstationarity of these filters allows to achieve better results in some image processing problems such as image compression when compared to classical biorthogonal b spline filters

image coding using wavelet transform a scheme for image compression that takes into account psychovisual features both in the space and frequency domains is proposed this method involves two steps first a wavelet transform used in order to obtain a set of biorthogonal subclasses of images the original image is decomposed at different scales using a pyramidal algorithm architecture the decomposition is along the vertical and horizontal directions and maintains constant the number of pixels required to describe the image second according to shannon s rate distortion theory the wavelet coefficients are vector quantized using a multiresolution codebook to encode the wavelet coefficients a noise shaping bit allocation procedure which assumes that details at high resolution are less visible to the human eye is proposed in order to allow the receiver to recognize a picture as quickly as possible at minimum cost a progressive transmission scheme is presented it is shown that the wavelet transform is particularly well adapted to progressive transmission

wavelet theory demystified we revisit wavelet theory starting from the representation of a scaling function as the convolution of a b spline the regular part of it and a distribution the irregular or residual part this formulation leads to some new insights on wavelets and makes it possible to rederive the main results of the classical theory including some new extensions for fractional orders n a self contained accessible fashion in particular we prove that the b spline component is entirely responsible for five key wavelet properties order of approximation reproduction of polynomials vanishing moments multiscale differentiation property and smoothness regularity of the basis functions we also investigate the interaction of wavelets with differential operators giving explicit time domain formulas for the fractional derivatives of the basis functions this allows us to specify a corresponding dual wavelet basis and helps us understand why the wavelet transform provides a stable characterization of the derivatives of a signal additional results include a new peeling theory of smoothness leading to the extended notion of wavelet differentiability in the l sub p sense and a sharper theorem stating that smoothness implies order

a cellular coevolutionary algorithm for image segmentation clustering is inherently a difficult problem both with respect to the definition of adequate models as well as to the optimization of the models we present a model for the cluster problem that does not need knowledge about the number of clusters a priori this property is among others useful in the image segmentation domain which we especially address further we propose a cellular coevolutionary algorithm for the optimization of the model within this scheme multiple agents are placed in a regular two dimensional 2 d grid representing the image which imposes neighboring relations on them the agents cooperatively consider pixel migration from one agent to the other in order to improve the homogeneity of the ensemble of the image regions they represent if the union of the regions of neighboring agents is homogeneous then the agents form alliances on the other hand if an agent discovers a deviant subject it isolates the subject in the experiments we show the effectiveness of the proposed method and compare it to other segmentation algorithms the efficiency can easily be improved by exploiting the intrinsic parallelism of the proposed method

a maximum variance cluster algorithm we present a partitional cluster algorithm that minimizes the sum of squared error criterion while imposing a hard constraint on the cluster variance conceptually hypothesized clusters act in parallel and cooperate with their neighboring clusters in order to minimize the criterion and to satisfy the variance constraint in order to enable the demarcation of the cluster neighborhood without crucial parameters we introduce the notion of foreign cluster samples finally we demonstrate a new method for cluster tendency assessment based on varying the variance constraint parameter

shape classification using line segment statistics in this study we use straight line segment statistics for shape analysis for each contour point we consider a continuous portion with length equal to a pre defined percentage of the contour size then we compute the length of the straight line segment between its extreme points for the set of straight line segments we compute statistical moments average and stardard deviation contour shape description is an important field in computer vision this is due to the fact that shape is an important low level image feature in light of this many approaches have been proposed in order to analyze it therefore this paper introduces a very simple yet efficient shape descriptor based on straight line segment statistics for each contour point we consider a continuous portion of the contour with length equal to a pre defined percentage of the contour size then we compute the length of the straight line segment between its extreme points for the set of straight line segments we compute statistical moments average and standard deviation lastly we perform this calculation for different lengths of contour portions the proposed shape descriptor is a powerful tool for shape discrimination it is robust it can characterize a huge set of different classes of shapes and is tolerant to variations in the shapes scale and orientation classification results of the proposed method overcome traditional methods found in literature proving that it is an efficient tool for shape analysis

generalized daubechies wavelet families we present a generalization of the orthonormal daubechies wavelets and of their related biorthogonal flavors cohen daubechies feauveau 9 7 our fundamental constraint is that the scaling functions should reproduce a predefined set of exponential polynomials this allows one to tune the corresponding wavelet transform to a specific class of signals thereby ensuring good approximation and sparsity properties the main difference with the classical construction of daubechies is that the multiresolution spaces are derived from scale dependent generating functions however from an algorithmic standpoint mallat s fast wavelet transform algorithm can still be applied the only adaptation consists in using scale dependent filter banks finite support ensures the same computational efficiency as in the classical case we characterize the scaling and wavelet filters construct them and show several examples of the associated functions we prove that these functions are square integrable and that they converge to their classical counterparts of the corresponding order

newton polygons and local integrability of negative powers of smooth functions in the plane let f x y be any smooth real valued function with f 0 0 0 for a sufficiently small neighborhood u of the origin we study the number sup u f x f it is known that sometimes this number can be expressed in a natural way using the newton polygon of f we provide necessary and sufficient conditions for this newton polygon characterization to hold the behavior of the integral at the supremal e is also analyzed

wavelet based feature extraction using probabilistic finite state automata for pattern classification real time data driven pattern classification requires extraction of relevant features from the observed time series as low dimensional and yet information rich representations of the underlying dynamics these low dimensional features facilitate in situ decision making in diverse applications such as computer vision structural health monitoring and robotics wavelet transforms of time series have been widely used for feature extraction owing to their time frequency localization properties in this regard this paper presents a symbolic dynamics based method to model surface images generated by wavelet coefficients in the scale shift space these symbolic dynamics based models e g probabilistic finite state automata pfsa capture the relevant information embedded in the sensor data from the associated perron frobenius operators i e the state transition probability matrices the proposed method of pattern classification has been experimentally validated on laboratory apparatuses for two different applications i early detection of evolving damage in polycrystalline alloy structures and ii classification of mobile robots and their motion profiles

regularized discriminant analysis ridge regression and beyond fisher linear discriminant analysis fda and its kernel extension kernel discriminant analysis kda are well known methods that consider dimensionality reduction and classification jointly while widely deployed in practical problems there are still unresolved issues surrounding their efficient implementation and their relationship with least mean squares procedures in this paper we address these issues within the framework of regularized estimation our approach leads to a flexible and efficient implementation of fda as well as kda we also uncover a general relationship between regularized discriminant analysis and ridge regression this relationship yields variations on conventional fda based on the pseudoinverse and a direct equivalence to an ordinary least squares estimator

a redundancy based measure of dissimilarity among probability distributions for hierarchical clustering criteria we introduce novel dissimilarity into a probabilistic clustering task to properly measure dissimilarity among multiple clusters when each cluster is characterized by a subpopulation in the mixture model this measure of dissimilarity is called redundancy based dissimilarity among probability distributions from aspects of both source coding and a statistical hypothesis test we shed light on several of the theoretical reasons for the redundancy based dissimilarity among probability distributions being a reasonable measure of dissimilarity among clusters we also elucidate a principle in common for the measures of redundancy based dissimilarity and ward s method in terms of hierarchical clustering criteria moreover we show several related theorems that are significant for clustering tasks in the experiments properties of the measure of redundancy based dissimilarity are examined in comparison with several other measures

a multi dimensional resolution of singularities with applications to analysis we formulate a resolution of singularities algorithm for analyzing the zero sets of real analytic functions in dimensions geq 3 rather than using the celebrated result of hironaka the algorithm is modeled on a more explicit and elementary approach used in the contemporary algebraic geometry literature as an application we define a new notion of the height of real analytic functions compute the critical integrability index and obtain sharp growth rate of sublevel sets this also leads to a characterization of the oscillation index of scalar oscillatory integrals with real analytic phases in all dimensions

local structure preserving discriminative projections for rgb d sensor based scene classification rgb d sensor based scene classification has recently received attention because of its potential use in human computer interaction applications however when feature representation schemes are adopted the dimensionality of concatenated scene features extracted from the rgb and depth image pair is very high and faces the curse of dimensionality therefore we present a new subspace selection scheme called local structure preserving discriminative projections lspdp lspdp simultaneously considers two issues 1 achieving a balance between the local patch structure and global within class structure and 2 maximizing between class distances extensive experimentation on the nyu depth v1 dataset with feature sets computed using three popular schemes locality constrained linear coding llc spatial pyramid matching using sparse coding sc spm and efficient match kernels emks demonstrates the robustness and effectiveness of the new method for rgb d sensor based scene classification

a simulated annealing algorithm for the clustering problem abstract in this paper we discuss the solution of the clustering problem usually solved by the k means algorithm the problem is known to have local minimum solutions which are usually what the k means algorithm obtains the simulated annealing approach for solving optimization problems is described and is proposed for solving the clustering problem the parameters of the algorithm are discussed in detail and it is shown that the algorithm converges to a global solution of the clustering problem we also find optimal parameters values for a specific class of data sets and give recommendations on the choice of parameters for general data sets finally advantages and disadvantages of the approach are presented

experiments in projection and clustering by simulated annealing simulated annealing is a stochastic relaxation algorithm which has been used successfully to optimize functions of many variables this paper analyzes the simulated annealing algorithm when applied to the minimization of functions from two common problems encountered in exploratory pattern analysis projection and clustering the projection is a nonlinear mapping of patterns in high dimension to two dimensions the simulated annealing mapping is compared to gradient descent minimization of the same objective function as well as eigenvector projection the simulated annealing clustering is compared to a k means algorithm empirical results show that simulated annealing can produce results as good as those obtained by conventional methods but are impractical for small data sets because of the high computational cost simulated annealing does in the case of the mapping problem yield a better optimization and better retained structure for large data sets containing tight gaussian clusters

convergence of two dimensional weighted integrals a two dimensional weighted integral in r 2 is proposed as a tool for analyzing higher dimensional unweighted integrals and a necessary and sufficient condition for the finiteness of the weighted integral is obtained

genie a new fast and outlier resistant hierarchical clustering algorithm abstract the time needed to apply a hierarchical clustering algorithm is most often dominated by the number of computations of a pairwise dissimilarity measure such a constraint for larger data sets puts at a disadvantage the use of all the classical linkage criteria but the single linkage one however it is known that the single linkage clustering algorithm is very sensitive to outliers produces highly skewed dendrograms and therefore usually does not reflect the true underlying data structure unless the clusters are well separated to overcome its limitations we propose a new hierarchical clustering linkage criterion called genie namely our algorithm links two clusters in such a way that a chosen economic inequity measure e g the gini or bonferroni index of the cluster sizes does not increase drastically above a given threshold the presented benchmarks indicate a high practical usefulness of the introduced method it most often outperforms the ward or average linkage in terms of the clustering quality while retaining the single linkage speed the genie algorithm is easily parallelizable and thus may be run on multiple threads to speed up its execution further on its memory overhead is small there is no need to precompute the complete distance matrix to perform the computations in order to obtain a desired clustering it can be applied on arbitrary spaces equipped with a dissimilarity measure e g on real vectors dna or protein sequences images rankings informetric data etc a reference implementation of the algorithm has been included in the open source genie package for r

an efficient approach for face recognition based on common eigenvalues in this paper a simple technique is proposed for face recognition among many human faces it is based on the polynomial coefficients covariance matrix and algorithm on common eigenvalues the main advantage of the proposed approach is that the identification of similarity between human faces is carried out without computing actual eigenvalues and eigenvectors a symmetric matrix is calculated using the polynomial coefficients based companion matrices of two compared images the nullity of a calculated symmetric matrix is used as similarity measure for face recognition the value of nullity is very small for dissimilar images and distinctly large for similar face images the feasibility of the propose approach is demonstrated on three face databases i e the orl database the yale database b and the feret database experimental results have shown the effectiveness of the proposed approach for feature extraction and classification of the face images having large variation in pose and illumination

analysis of asymptotically equivalent binary subdivision schemes non stationary uniform subdivision schemes are presented including some important examples as the up function and exponential splines the multivariate analysis of subdivision schemes is extended to a class of non stationary schemes which are asymptotically equivalent to converging stationary or non stationary schemes

faster and more robust point symmetry based k means algorithm based on the recently published point symmetry distance psd measure this paper presents a novel psd measure namely symmetry similarity level ssl operator for k means algorithm our proposed modified point symmetry based k means mpsk algorithm is more robust than the previous psk algorithm by su and chou not only the proposed mpsk algorithm is suitable for the symmetrical intra clusters as the psk algorithm does the proposed mpsk algorithm is also suitable for the symmetrical inter clusters in addition two speedup strategies are presented to reduce the time required in the proposed mpsk algorithm experimental results demonstrate the significant execution time improvement and the extension to the symmetrical inter clusters of the proposed mpsk algorithm when compared to the previous psk algorithm

an efficient algorithm to solve the small sample size problem for lda in this paper we present an efficient algorithm to solve the most discriminant vectors of lda for high dimensional data set the experiments on orl face database confirm the effectiveness of the proposed method

transfer latent variable model based on divergence analysis latent variable models are powerful dimensionality reduction approaches in machine learning and pattern recognition however this kind of methods only works well under a necessary and strict assumption that the training samples and testing samples are independent and identically distributed when the samples come from different domains the distribution of the testing dataset will not be identical with the training dataset therefore the performance of latent variable models will be degraded for the reason that the parameters of the training model do not suit for the testing dataset this case limits the generalization and application of the traditional latent variable models to handle this issue a transfer learning framework for latent variable model is proposed which can utilize the distance or divergence of the two datasets to modify the parameters of the obtained latent variable model so we do not need to rebuild the model and only adjust the parameters according to the divergence which will adopt different datasets experimental results on several real datasets demonstrate the advantages of the proposed framework

kpca plus lda a complete kernel fisher discriminant framework for feature extraction and recognition this paper examines the theory of kernel fisher discriminant analysis kfd in a hilbert space and develops a two phase kfd framework i e kernel principal component analysis kpca plus fisher linear discriminant analysis lda this framework provides novel insights into the nature of kfd based on this framework the authors propose a complete kernel fisher discriminant analysis ckfd algorithm ckfd can be used to carry out discriminant analysis in double discriminant subspaces the fact that it can make full use of two kinds of discriminant information regular and irregular makes ckfd a more powerful discriminator the proposed algorithm was tested and evaluated using the feret face database and the cenparmi handwritten numeral database the experimental results show that ckfd outperforms other kfd algorithms

a fast multilevel algorithm for wavelet regularized image restoration we present a multilevel extension of the popular ldquothresholded landweberrdquo algorithm for wavelet regularized image restoration that yields an order of magnitude speed improvement over the standard fixed scale implementation the method is generic and targeted towards large scale linear inverse problems such as 3 d deconvolution microscopy the algorithm is derived within the framework of bound optimization the key idea is to successively update the coefficients in the various wavelet channels using fixed subband adapted iteration parameters step sizes and threshold levels the optimization problem is solved efficiently via a proper chaining of basic iteration modules the higher level description of the algorithm is similar to that of a multigrid solver for pdes but there is one fundamental difference the latter iterates though a sequence of multiresolution versions of the original problem while in our case we cycle through the wavelet subspaces corresponding to the difference between successive approximations this strategy is motivated by the special structure of the problem and the preconditioning properties of the wavelet representation we establish that the solution of the restoration problem corresponds to a fixed point of our multilevel optimizer we also provide experimental evidence that the improvement in convergence rate is essentially determined by the unconstrained linear part of the algorithm irrespective of the type of wavelet finally we illustrate the technique with some image deconvolution examples including some real 3 d fluorescence microscopy data

robust bilateral lp norm two dimensional linear discriminant analysis abstract traditional linear discriminant analysis lda may suffer from a sensitivity to outliers and the small sample size sss problem while the lp norm measure for 0

face recognition using laplacianfaces we propose an appearance based face recognition method called the laplacianface approach by using locality preserving projections lpp the face images are mapped into a face subspace for analysis different from principal component analysis pca and linear discriminant analysis lda which effectively see only the euclidean structure of face space lpp finds an embedding that preserves local information and obtains a face subspace that best detects the essential face manifold structure the laplacianfaces are the optimal linear approximations to the eigenfunctions of the laplace beltrami operator on the face manifold in this way the unwanted variations resulting from changes in lighting facial expression and pose may be eliminated or reduced theoretical analysis shows that pca lda and lpp can be obtained from different graph models we compare the proposed laplacianface approach with eigenface and fisherface methods on three different face data sets experimental results suggest that the proposed laplacianface approach provides a better representation and achieves lower error rates in face recognition

towards parameter independent data clustering and image segmentation while there are a large amount of clustering algorithms proposed in the literature the clustering results of existing algorithms usually depend on user specified parameters heavily and it is usually difficult to determine the optimal parameters with the pairwise data similarity matrix as the input dominant sets clustering has been shown to be an effective data clustering and image segmentation approach partly due to its ability to find out the underlying data structure and determine the number of clusters automatically however we find that the original dominant sets algorithm is sensitive to the similarity measures used in building the similarity matrix this means that parameter tuning is required to generate satisfactory clustering results and dominant sets clustering results are also parameter dependent in order to remove the dependence on the user specified parameter we study how the similarity measures influence the dominant sets clustering results as a result we propose to transform similarity matrices by histogram equalization before clustering while this transformation is shown to remove the sensitiveness to similarity measures effectively it also results in over segmentation therefore in the next step we present a cluster extension method to overcome the over segmentation effect and generate more reasonable clustering results we test the enhanced clustering algorithm in both data clustering and image segmentation experiments and comparisons with the state of the art algorithms validate the effectiveness of our algorithm highlightswe study how similarity measures influence the dominant sets clustering results we use histogram equalization to remove the dependence on similarity parameters we present a density based cluster extension method to overcome over segmentation experiments validate the effectiveness of our algorithm

from differential equations to the construction of new wavelet like bases in this paper an approach is introduced based on differential operators to construct wavelet like basis functions given a differential operator l with rational transfer function elementary building blocks are obtained that are shifted replicates of the green s function of l it is shown that these can be used to specify a sequence of embedded spline spaces that admit a hierarchical exponential b spline representation the corresponding b splines are entirely specified by their poles and zeros they are compactly supported have an explicit analytical form and generate multiresolution riesz bases moreover they satisfy generalized refinement equations with a scale dependent filter and lead to a representation that is dense in l sub 2 this allows us to specify a corresponding family of semi orthogonal exponential spline wavelets which provides a major extension of earlier polynomial spline constructions these wavelets are completely characterized and it is proven that they satisfy the following remarkable properties 1 they are orthogonal across scales and generate riesz bases at each resolution level 2 they yield unconditional bases of l sub 2 either compactly supported b spline type or with exponential decay orthogonal or dual type 3 they have n vanishing exponential moments where n is the order of the differential operator 4 they behave like multiresolution versions of the operator l from which they are derived and 5 their order of approximation is n m where n and m give the number of poles and zeros respectively last but not least the new wavelet like decompositions are as computationally efficient as the classical ones they are computed using an adapted version of mallat s filter bank algorithm where the filters depend on the decomposition level

a new lda based face recognition system which can solve the small sample size problem a new lda based face recognition system is presented in this paper linear discriminant analysis lda is one of the most popular linear projection techniques for feature extraction the major drawback of applying lda is that it may encounter the small sample size problem in this paper we propose a new lda based technique which can solve the small sample size problem we also prove that the most expressive vectors derived in the null space of the within class scatter matrix using principal component analysis pca are equal to the optimal discriminant vectors derived in the original space using lda the experimental results show that the new lda process improves the performance of a face recognition system signi cantly 2000 pattern recognition society published by elsevier science ltd all rights reserved

regularized orthogonal linear discriminant analysis in this paper the regularized orthogonal linear discriminant analysis rolda is studied the major issue of the regularized linear discriminant analysis is to choose an appropriate regularization parameter in existing regularized linear discriminant analysis methods they all select the best regularization parameter from a given parameter candidate set by using cross validation for classification an obvious limitation of such regularized linear discriminant analysis methods is that it is not clear how to choose an appropriate candidate set therefore up to now there is no concrete mathematical theory available in selecting an appropriate regularization parameter in practical applications of the regularized linear discriminant analysis the present work is to fill this gap here we derive the mathematical relationship between orthogonal linear discriminant analysis and the regularized orthogonal linear discriminant analysis first and then by means of this relationship we find a mathematical criterion for selecting the regularization parameter in rolda and consequently we develop a new regularized orthogonal linear discriminant analysis method in which no candidate set of regularization parameter is needed the effectiveness of our proposed regularized orthogonal linear discriminant analysis is illustrated by some real world data sets

a comparison of generalized linear discriminant analysis algorithms linear discriminant analysis lda is a dimension reduction method which finds an optimal linear transformation that maximizes the class separability however in undersampled problems where the number of data samples is smaller than the dimension of data space it is difficult to apply lda due to the singularity of scatter matrices caused by high dimensionality in order to make lda applicable several generalizations of lda have been proposed recently in this paper we present theoretical and algorithmic relationships among several generalized lda algorithms and compare their computational complexities and performances in text classification and face recognition towards a practical dimension reduction method for high dimensional data an efficient algorithm is proposed which reduces the computational complexity greatly while achieving competitive prediction accuracies we also present nonlinear extensions of these lda algorithms based on kernel methods it is shown that a generalized eigenvalue problem can be formulated in the kernel based feature space and generalized lda algorithms are applied to solve the generalized eigenvalue problem resulting in nonlinear discriminant analysis performances of these linear and nonlinear discriminant analysis algorithms are compared extensively

orthonormal bases of compactly supported wavelets we construct orthonormal bases of compactly supported wavelets with arbitrarily high regularity the order of regularity increases linearly with the support width we start by reviewing the concept of multiresolution analysis as well as several algorithms in vision decomposition and reconstruction the construction then follows from a synthesis of these different approaches

a clustering ensemble the aim of clustering ensemble is to combine multiple base partitions into a robust stable and accurate partition one of the key problems of clustering ensemble is how to exploit the cluster structure information in each base partition evidence accumulation is an effective framework which can convert the base partitions into a co association matrix this matrix describes the frequency of a pair of points partitioned into the same cluster but ignores some hidden information in the base partitions in this paper we reveal some of those information by refining the co association matrix from data point and base cluster level from the data point level as pairs of points in the same base cluster may have varied similarities their contributions to the co association matrix can be different from the cluster level since the base clusters may have diversified qualities the contribution of a base cluster as a whole can also be different from those of others after being refined the co association matrix is transformed into a path based similarity matrix so that more global information of the cluster structure is incorporated into the matrix finally spectral clustering is applied to the matrix to generate the final clustering result experimental results on 8 synthetic and 8 real data sets demonstrate that the clustering ensemble based on the refined co association matrix outperforms some state of the art clustering ensemble schemes highlightsa two level refined co association matrix for cluster ensemble is proposed the refined co association matrix is transformed by path based measure a theoretical background of the refinement is given the proposed method outperforms some state of the art ensemble methods

generalized linear discriminant analysis a unified framework and efficient model selection high dimensional data are common in many domains and dimensionality reduction is the key to cope with the curse of dimensionality linear discriminant analysis lda is a well known method for supervised dimensionality reduction when dealing with high dimensional and low sample size data classical lda suffers from the singularity problem over the years many algorithms have been developed to overcome this problem and they have been applied successfully in various applications however there is a lack of a systematic study of the commonalities and differences of these algorithms as well as their intrinsic relationships in this paper a unified framework for generalized lda is proposed which elucidates the properties of various algorithms and their relationships based on the proposed framework we show that the matrix computations involved in lda based algorithms can be simplified so that the cross validation procedure for model selection can be performed efficiently we conduct extensive experiments using a collection of high dimensional data sets including text documents face images gene expression data and gene expression pattern images to evaluate the proposed theories and algorithms

the nearest subclass classifier a compromise between the nearest mean and nearest neighbor classifier we present the nearest subclass classifier nsc which is a classification algorithm that unifies the flexibility of the nearest neighbor classifier with the robustness of the nearest mean classifier the algorithm is based on the maximum variance cluster algorithm and as such it belongs to the class of prototype based classifiers the variance constraint parameter of the cluster algorithm serves to regularize the classifier that is to prevent overfitting with a low variance constraint value the classifier turns into the nearest neighbor classifier and with a high variance parameter it becomes the nearest mean classifier with the respective properties in other words the number of prototypes ranges from the whole training set to only one per class in the experiments we compared the nsc with regard to its performance and data set compression ratio to several other prototype based methods on several data sets the nsc performed similarly to the k nearest neighbor classifier which is a well established classifier in many domains also concerning storage requirements and classification speed the nsc has favorable properties so it gives a good compromise between classification performance and efficiency

regularity of non stationary subdivision a matrix approach in this paper we study scalar multivariate non stationary subdivision schemes with integer dilation matrix m and present a unifying general approach for checking their convergence and for determining their holder regularity latter in the case m mi m ge 2 m mi m 2 the combination of the concepts of asymptotic similarity and approximate sum rules allows us to link stationary and non stationary settings and to employ recent advances in methods for exact computation of the joint spectral radius as an application we prove a recent conjecture by dyn et al on the holder regularity of the generalized daubechies wavelets we illustrate our results with several examples

local linear discriminant analysis framework using sample neighbors the linear discriminant analysis lda is a very popular linear feature extraction approach the algorithms of lda usually perform well under the following two assumptions the first assumption is that the global data structure is consistent with the local data structure the second assumption is that the input data classes are gaussian distributions however in real world applications these assumptions are not always satisfied in this paper we propose an improved lda framework the local lda llda which can perform well without needing to satisfy the above two assumptions our llda framework can effectively capture the local structure of samples according to different types of local data structure our llda framework incorporates several different forms of linear feature extraction approaches such as the classical lda and principal component analysis the proposed framework includes two llda algorithms a vector based llda algorithm and a matrix based llda mllda algorithm mllda is directly applicable to image recognition such as face recognition our algorithms need to train only a small portion of the whole training set before testing a sample they are suitable for learning large scale databases especially when the input data dimensions are very high and can achieve high classification accuracy extensive experiments show that the proposed algorithms can obtain good classification results

fuzzy clustering of fuzzy data based on robust loss functions and ordered weighted averaging abstract in many real cases the data are not expressed in term of single values but are imprecise in all these cases standard clustering methods for single valued data are unable to properly take into account the imprecise nature of the data in this paper by considering the partitioning around medoids pam approach in a fuzzy framework we propose a fuzzy clustering method for imprecise data formalized in a fuzzy manner in particular in order to neutralize the negative effects of possible outlier fuzzy data in the clustering process we proposed a robust fuzzy c medoids clustering method for fuzzy data based on the combination of huber s m estimators and yager s owa ordered weighted averaging operators the proposed method is able to smooth the influence of anomalous data by means of a suitable parameter the so called typicality parameter capable to tune the influence of the outliers the performance of the proposed method has been shown by means of a simulation study composed of experiments on i simple two dimensional dataset ii benchmark datasets and iii the fuzzy art outliers dataset the comparison made with the robust clustering methods known from the literature indicates the competitiveness of the introduced method to others an application of the suggested method to a real dataset is also provided and the results of the method has been compared with other clustering methods suggested in the literature in the application the comparative assessment has shown the informational gain in term of additional information of the proposed method vs the other robust methods

axiomatic generalization of the membership degree weighting function for fuzzy c means clustering for decades practitioners have been using the center based partitional clustering algorithms like fuzzy c means fcm which rely on minimizing an objective function comprising of an appropriately weighted sum of distances of each data point from the cluster representatives numerous generalizations in terms of choice of the distance function have been introduced for fcm since its inception in a stark contrast to this fact to the best of our knowledge there has not been any significant effort to address the issue of convergence of the algorithm under the structured generalization of the weighting function here by generalization we mean replacing the conventional weighting function uijm where uij indicates the membership of data xi to cluster cj and m is the real valued fuzzifier with 1 m with a more general function g uij which can assume a wide variety of forms under some mild assumptions in this article for the first time we present a novel axiomatic development of the general weighting function based on the membership degrees we formulate the fuzzy clustering problem along with the intuitive justification of the technicalities behind the axiomatic approach we develop an alternative optimization ao procedure to solve the main clustering problem by solving the partial optimization problems in an iterative way the novelty of the article lies in the development of an axiomatic generalization of the weighting function of fcm formulation of an ao algorithm for the fuzzy clustering problem with the extension to this general class of weighting functions and a detailed convergence analysis of the proposed algorithm

minimum spanning tree based split and merge a hierarchical clustering method most clustering algorithms become ineffective when provided with unsuitable parameters or applied to datasets which are composed of clusters with diverse shapes sizes and densities to alleviate these deficiencies we propose a novel split and merge hierarchical clustering method in which a minimum spanning tree mst and an mst based graph are employed to guide the splitting and merging process in the splitting process vertices with high degrees in the mst based graph are selected as initial prototypes and k means is used to split the dataset in the merging process subgroup pairs are filtered and only neighboring pairs are considered for merge the proposed method requires no parameter except the number of clusters experimental results demonstrate its effectiveness both on synthetic and real datasets

density peak clustering based on relative density relationship abstract the density peak clustering algorithm treats local density peaks as cluster centers and groups non center data points by assuming that one data point and its nearest higher density neighbor are in the same cluster while this algorithm is shown to be promising in some applications its clustering results are found to be sensitive to density kernels and large density differences across clusters tend to result in wrong cluster centers in this paper we attribute these problems to the inconsistency between the assumption and implementation adopted in this algorithm while the assumption is based totally on relative density relationship this algorithm adopts absolute density as one criterion to identify cluster centers this observation prompts us to present a cluster center identification criterion based only on relative density relationship specifically we define the concept of subordinate to describe the relative density relationship and use the number of subordinates as a criterion to identify cluster centers our approach makes use of only relative density relationship and is less influenced by density kernels and density differences across clusters in addition we discuss the problems of two existing density kernels and present an average distance based kernel in data clustering experiments we validate the new criterion and density kernel respectively and then test the whole algorithm and compare with some other clustering algorithms

a theory for multiresolution signal decomposition the wavelet representation multiresolution representations are effective for analyzing the information content of images the properties of the operator which approximates a signal at a given resolution were studied it is shown that the difference of information between the approximation of a signal at the resolutions 2 sup j 1 and 2 sup j where j is an integer can be extracted by decomposing this signal on a wavelet orthonormal basis of l sup 2 r sup n the vector space of measurable square integrable n dimensional functions in l sup 2 r a wavelet orthonormal basis is a family of functions which is built by dilating and translating a unique function psi x this decomposition defines an orthogonal multiresolution representation called a wavelet representation it is computed with a pyramidal algorithm based on convolutions with quadrature mirror filters wavelet representation lies between the spatial and fourier domains for images the wavelet representation differentiates several spatial orientations the application of this representation to data compression in image coding texture discrimination and fractal analysis is discussed