learning eigenfunctions links spectral embedding and kernel pca in this letter we show a direct relation between spectral embedding methods and kernel principal components analysis and how both are special cases of a more general learning problem learning the principal eigenfunctions of an operator defined from a kernel and the unknown data generating density whereas spectral embedding methods provided only coordinates for the training points the analysis justifies a simple extension to out of sample examples the nystrom formula for multidimensional scaling mds spectral clustering laplacian eigenmaps locally linear embedding lle and isomap the analysis provides for all such spectral embedding methods the definition of a loss function whose empirical average is minimized by the traditional algorithms the asymptotic expected value of that loss defines a generalization performance and clarifies what these algorithms are trying to learn experiments with lle isomap spectral clustering and mds show that this out of sample embedding formula generalizes well with a level of error comparable to the effect of small perturbations of the training set on the embedding

pairwise sparsity preserving embedding for unsupervised subspace learning and classification two novel unsupervised dimensionality reduction techniques termed sparse distance preserving embedding sdpe and sparse proximity preserving embedding sppe are proposed for feature extraction and classification sdpe and sppe perform in the clean data space recovered by sparse representation and enhanced euclidean distances over noise removed data are employed to measure pairwise similarities of points in extracting informative features sdpe and sppe aim at preserving pairwise similarities between data points in addition to preserving the sparse characteristics this paper calculates the sparsest representation of all vectors jointly by a convex optimization the sparsest codes enable certain local information of data to be preserved and can endow sdpe and sppe a natural discriminating power adaptive neighborhood and robust characteristic against noise and errors in delivering low dimensional embeddings we also mathematically show sdpe and sppe can be effectively extended for discriminant learning in a supervised manner the validity of sdpe and sppe is examined by extensive simulations comparison with other related state of the art unsupervised algorithms show that promising results are delivered by our techniques

variational and projection methods for the volume constraint in finite deformation elasto plasticity this paper focuses on the treatment of volume constraints which in the context of elasto plasticity typically arise as a result of assuming volume preserving plastic flow projection methods based on the modification of the discrete gradient operator b often proposed on an ad hoc basis are systematically obtained in the variational context furnished by a three field hu washizu principle the fully nonlinear formulation proposed here is based on a local multiplicative split of the deformation gradient into volume preserving and dilatational parts without relying on rate forms of the weak form of momentum balance this approach fits naturally in a formulation of plasticity based on the multiplicative decomposition of the deformation gradient and enables one to exactly enforce the condition of volume preserving plastic flow within the framework proposed in this paper rate forms and incrementally objective algorithms are entirely bypassed

finite difference time domain implementation of surface impedance boundary conditions surface impedance boundary conditions can be utilized to avoid using small cells made necessary by shorter wavelengths in conducting media throughout the solution volume the standard approach is to approximate the surface impedance over a very small bandwidth by its value at the center frequency and then use that result in the boundary condition in this paper two implementations of the surface impedance boundary condition are presented one implementation is a constant surface impedance boundary condition and the other is a dispersive surface impedance boundary condition that is applicable over a very large frequency bandwidth and over a large range of conductivities frequency domain results are presented in one dimension for two conductivity values and are compared with exact results scattering width results from an infinite square cylinder are presented as a two dimensional demonstration

exact closed form representations for the time domain surface impedances of a homogeneous lossy half space in this paper we derive exact closed form representations for the transverse electric and transverse magnetic time domain surface impedances associated with a plane wave that is obliquely incident on a homogeneous lossy half space the time domain surface impedances are expressed in terms of modified bessel functions and incomplete lipschitz hankel integrals the limitations on the validity of a previously published approximate expression are also discussed

discrete hessian eigenmaps method for dimensionality reduction for a given set of data points lying on a low dimensional manifold embedded in a high dimensional space the dimensionality reduction is to recover a low dimensional parametrization from the data set the recently developed hessian eigenmaps method is a mathematically rigorous method that also sets a theoretical framework for the nonlinear dimensionality reduction problem in this paper we develop a discrete version of the hessian eigenmaps method and present an analysis giving conditions under which the method works as intended as an application a procedure to modify the standard constructions of k nearest neighborhoods is presented to ensure that hessian lle can recover the original coordinates up to an affine transformation

dictionary pair learning on grassmann manifolds for image denoising image denoising is a fundamental problem in computer vision and image processing that holds considerable practical importance for real world applications the traditional patch based and sparse coding driven image denoising methods convert 2d image patches into 1d vectors for further processing thus these methods inevitably break down the inherent 2d geometric structure of natural images to overcome this limitation pertaining to the previous image denoising methods we propose a 2d image denoising model namely the dictionary pair learning dpl model and we design a corresponding algorithm called the dpl on the grassmann manifold dplg algorithm the dplg algorithm first learns an initial dictionary pair i e the left and right dictionaries by employing a subspace partition technique on the grassmann manifold wherein the refined dictionary pair is obtained through a sub dictionary pair merging the dplg obtains a sparse representation by encoding each image patch only with the selected sub dictionary pair the non zero elements of the sparse representation are further smoothed by the graph laplacian operator to remove the noise consequently the dplg algorithm not only preserves the inherent 2d geometric structure of natural images but also performs manifold smoothing in the 2d sparse coding space we demonstrate that the dplg algorithm also improves the structural similarity values of the perceptual visual quality for denoised images using the experimental evaluations on the benchmark images and berkeley segmentation data sets moreover the dplg also produces the competitive peak signal to noise ratio values from popular image denoising algorithms

refined spectral clustering via embedded label propagation spectral clustering is a key research topic in the field of machine learning and data mining most of the existing spectral clustering algorithms are built on gaussian laplacian matrices which is sensitive to parameters we propose a novel parameter free distance consistent locally linear embedding the proposed distance consistent lle can promise that edges between closer data points are heavier we also propose a novel improved spectral clustering via embedded label propagation our algorithm is built on two advancements of the state of the art first is label propagation which propagates a node s labels to neighboring nodes according to their proximity we perform standard spectral clustering on original data and assign each cluster with nearest data points and then we propagate labels through dense unlabeled data regions second is manifold learning which has been widely used for its capacity to leverage the manifold structure of data points extensive experiments on various data sets validate the

fast approximate k nn graph construction for high dimensional data via recursive lanczos bisection nearest neighbor graphs are widely used in data mining and machine learning a brute force method to compute the exact knn graph takes dn2 time for n data points in the d dimensional euclidean space we propose two divide and conquer methods for computing an approximate knn graph in dnt time for high dimensional data large d the exponent t 1 2 is an increasing function of an internal parameter which governs the size of the common region in the divide step experiments show that a high quality graph can usually be obtained with small overlaps that is for small values of t a few of the practical details of the algorithms are as follows first the divide step uses an inexpensive lanczos procedure to perform recursive spectral bisection after each conquer step an additional refinement step is performed to improve the accuracy of the graph finally a hash table is used to avoid repeating distance calculations during the divide and conquer process the combination of these techniques is shown to yield quite effective algorithms for building knn graphs

spectral embedded clustering a framework for in sample and out of sample spectral clustering spectral clustering sc methods have been successfully applied to many real world applications the success of these sc methods is largely based on the manifold assumption namely that two nearby data points in the high density region of a low dimensional data manifold have the same cluster label however such an assumption might not always hold on high dimensional data when the data do not exhibit a clear low dimensional manifold structure e g high dimensional and sparse data the clustering performance of sc will be degraded and become even worse than k means clustering in this paper motivated by the observation that the true cluster assignment matrix for high dimensional data can be always embedded in a linear space spanned by the data we propose the spectral embedded clustering sec framework in which a linearity regularization is explicitly added into the objective function of sc methods more importantly the proposed sec framework can naturally deal with out of sample data we also present a new laplacian matrix constructed from a local regression of each pattern and incorporate it into our sec framework to capture both local and global discriminative information for clustering comprehensive experiments on eight real world high dimensional datasets demonstrate the effectiveness and advantages of our sec framework over existing sc methods and k means based clustering methods our sec framework significantly outperforms sc using the nystrom algorithm on unseen data

learning the dynamical system behind sensory data this letter looks at the physics behind sensory data by identifying the parameters that govern the physical system and estimating them from sensory observations we extend takens s delay embedding theorem to the dynamical systems controlled by parameters an embedding of the product space of the phase and the parameter spaces of the dynamical system can be obtained by the delay embedding map provided that the parameter of the dynamical system changes slowly the reconstruction error is bounded for slowly varying parameters a manifold learning technique is applied to the embedding obtained to extract a low dimensional global coordinate system representing the product space the phase space of the deterministic dynamics can be contracted by using the adjacency relationship in time which enables recovery of only the parameter space as examples the manifolds of synthetic and real world vowels with time varying fundamental frequency f0 are analyzed and the f0 contours are extracted by an unsupervised algorithm experimental results show that the proposed method leads to robust performance under various noise conditions and rapid changes of f0 compared with the current state of the art f0 estimation algorithms

normalized cuts and image segmentation we propose a novel approach for solving the perceptual grouping problem in vision rather than focusing on local features and their consistencies in the image data our approach aims at extracting the global impression of an image we treat image segmentation as a graph partitioning problem and propose a novel global criterion the normalized cut for segmenting the graph the normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups we show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion we applied this approach to segmenting static images as well as motion sequences and found the results to be very encouraging

proximity based frameworks for generating embeddings from multi output data this paper is about supervised and semi supervised dimensionality reduction dr by generating spectral embeddings from multi output data based on the pairwise proximity information two flexible and generic frameworks are proposed to achieve supervised dr sdr for multilabel classification one is able to extend any existing single label sdr to multilabel via sample duplication referred to as mesd the other is a multilabel design framework that tackles the sdr problem by computing weight proximity matrices based on simultaneous feature and label information referred to as mope as a generalization of many current techniques a diverse set of different schemes for label based proximity calculation as well as a mechanism for combining label based and feature based weight information by considering information importance and prioritization are proposed for mope additionally we summarize many current spectral methods for unsupervised dr udr single multilabel sdr and semi supervised dr ssdr and express them under a common template representation as a general guide to researchers in the field we also propose a general framework for achieving ssdr by combining existing sdr and udr models and also a procedure of reducing the computational cost via learning with a target set of relation features the effectiveness of our proposed methodologies is demonstrated with experiments with document collections for multilabel text categorization from the natural language processing domain

noisy dynamic simulations in the presence of symmetry data alignment and model reduction abstract we process snapshots of trajectories of evolution equations with intrinsic symmetries and demonstrate the use of recently developed eigenvector based techniques to successfully quotient out the degrees of freedom associated with the symmetries in the presence of noise our illustrative examples include a one dimensional evolutionary partial differential the kuramoto sivashinsky equation with periodic boundary conditions as well as a stochastic simulation of nematic liquid crystals which can be effectively modeled through a nonlinear smoluchowski equation on the surface of a sphere this is a useful first step towards data mining the symmetry adjusted ensemble of snapshots in search of an accurate low dimensional parametrization and the associated reduction of the original dynamical system we also demonstrate a technique vector diffusion maps that combines in a single formulation the symmetry removal step and the dimensionality reduction step

self regularized fixed rank representation for subspace segmentation abstract consider a set of data points generated from various linear subspaces subspace segmentation tasks which are important in fields such as computer vision and image processing aim to partition the set of data points so as to recover these subspaces the subspace segmentation method fixed rank representation frr was introduced to remedy the problem of insufficient sampling in classical low rank representation lrr in many subspace segmentation applications frr has achieved much better results than those of lrr in this paper a new frr related algorithm called self regularized fixed rank representation srfrr is proposed in srfrr a laplacian regularizer is constructed using the coefficient matrix obtained by srfrr itself further by proving that the laplacian regularizer can be transformed into a structure constraint on the coefficient matrix we show that another existing method sparse frr sfrr is a special case of srfrr to implement the srfrr method we present two optimization algorithms experiments on both synthetic and real databases show that srfrr outperforms some existing frr and lrr related algorithms

mercer kernel based clustering in feature space the article presents a method for both the unsupervised partitioning of a sample of data and the estimation of the possible number of inherent clusters which generate the data this work exploits the notion that performing a nonlinear data transformation into some high dimensional feature space increases the probability of the linear separability of the patterns within the transformed space and therefore simplifies the associated data structure it is shown that the eigenvectors of a kernel matrix which defines the implicit mapping provides a means to estimate the number of clusters inherent within the data and a computationally simple iterative procedure is presented for the subsequent feature space partitioning of the data

large strain deformations of elastic shells constitutive modelling and finite element analysis this paper is concerned with the constitutive modelling of large elastic strains with applications to shells the shell theory itself is characterized by seven degrees of freedom and enjoys the following features 1 it circumvents the use of a rotation tensor preserving the euclidean structure of the configuration space and 2 it is capable to the application of a three dimensional constitutive law in addition an elastic finite strain constitutive law is developed which 1 covers the whole range of deformation experimentally given by treloar and 2 is formulated in terms of invariants avoiding the use of principal stretches the second feature means that the model is generalizable to anisotropic material behaviour and that it allows for straightforward computations of the tangent operator making the whole theory well suited for large strain computations in addition to the four node and nine node displacement based finite elements an enhanced strain finite element is examined its efficiency is compared with that of displacement based element formulations different examples of large strain shell deformations are presented it is shown that the application of the enhanced strain concept in the nonlinear constitutive range limits severely the stability of computation

progressive match expansion via coherent subspace constraint in this paper a quasi dense matching algorithm called progressive match expansion via coherent subspace pecs is proposed our algorithm starts from a set of sparse seeds and iteratively expands potential matches in the region around the seeds until no more matches could be found the main difference from previous methods is that our method computes a coherent non rigid transformation to relate the matching pixels instead of using local planar affine model to approximate the real scene surface within the expanding region the points in the expanding region are treated as a whole and all the matches satisfying the transformation are found at once first dense sift descriptors are extracted and a subspace encoding the feature similarity between two point sets is computed this embedding pulls two points closer if they have similar features and makes the non rigid transformation learning more robust second a coherent non rigid transformation in the subspace is computed to move one point set towards the other and the spatially aligned two points denote a match the proposed model is robust to scenes with non smooth surface and experiment results reveal the good performance in both 2d image matching and 3d reconstruction

improved data visualisation through nonlinear dissimilarity modelling inherent to state of the art dimension reduction algorithms is the assumption that global distances between observations are euclidean despite the potential for altogether non euclidean data manifolds we demonstrate that a non euclidean manifold chart can be approximated by implementing a universal approximator over a dictionary of dissimilarity measures building on recent developments in the field this approach is transferable across domains such that observations can be vectors distributions graphs and time series for instance our novel dissimilarity learning method is illustrated with four standard visualisation datasets showing the benefits over the linear dissimilarity learning approach

a survey of kernel and spectral methods for clustering clustering algorithms are a useful tool to explore data structures and have been employed in many disciplines the focus of this paper is the partitioning clustering problem with a special interest in two recent approaches kernel and spectral methods the aim of this paper is to present a survey of kernel and spectral clustering methods two approaches able to produce nonlinear separating hypersurfaces between clusters the presented kernel clustering methods are the kernel version of many classical clustering algorithms e g k means som and neural gas spectral clustering arise from concepts in spectral graph theory and the clustering problem is configured as a graph cut problem where an appropriate objective function has to be optimized an explicit proof of the fact that these two paradigms have the same objective is reported since it has been proven that these two seemingly different approaches have the same mathematical foundation besides fuzzy kernel clustering methods are presented as extensions of kernel k means clustering algorithm

local linear regression on manifolds and its geometric interpretation high dimensional data analysis has been an active area and the main focus areas have been variable selection and dimension reduction in practice it occurs often that the variables are located on an unknown lower dimensional nonlinear manifold under this manifold assumption one purpose of this article is regression and gradient estimation on the manifold and another is developing a new tool for manifold learning as regards the first aim we suggest directly reducing the dimensionality to the intrinsic dimension d of the manifold and performing the popular local linear regression llr on a tangent plane estimate an immediate consequence is a dramatic reduction in the computational time when the ambient space dimension p d we provide rigorous theoretical justification of the convergence of the proposed regression and gradient estimators by carefully analyzing the curvature boundary and nonuniform sampling effects we propose a bandwidth selector that can handle heteroscedastic errors with re

one shot detection with laplacian object and fast matrix cosine similarity one shot generic object detection involves searching for a single query object in a larger target image relevant approaches have benefited from features that typically model the local similarity patterns in this paper we combine local similarity encoded by local descriptors with a global context i e a graph structure of pairwise affinities among the local descriptors embedding the query descriptors into a low dimensional but discriminatory subspace unlike principal components that preserve global structure of feature space we actually seek a linear approximation to the laplacian eigenmap that permits us a locality preserving embedding of high dimensional region descriptors our second contribution is an accelerated but exact computation of matrix cosine similarity as the decision rule for detection obviating the computationally expensive sliding window search we leverage the power of fourier transform combined with integral image to achieve superior runtime efficiency that allows us to test multiple hypotheses for pose estimation within a reasonably short time our approach to one shot detection is training free and experiments on the standard data sets confirm the efficacy of our model besides low computation cost of the proposed codebook free object detector facilitates rather straightforward query detection in large data sets including movie videos

semantic preserving distance metric learning and applications how do we accurately browse a large set of images or efficiently annotate the images from an image library image clustering methods are invaluable tools for applications such as content based image retrieval and image annotation to perform these tasks it is critical to have proper features to describe the visual and semantic content of images and to define an accurate distance metric to measure the dissimilarity between any two images however existing methods which adopt the features of color histograms edge direction histograms and shape context lack the ability to describe semantic content to solve this problem we propose a new approach that utilizes user provided pairwise constraints to describe the semantic relationship between two images a semantic preserving distance metric learning sp dml algorithm is developed to explore the complementary characteristics of the visual features and pairwise constraints in a unified feature space in this space the learned distance metric can be used to measure the dissimilarity between two images specifically the manifold structure adopted in sp dml is revealed by the image s visual features to integrate semantic contents in distance metric learning sp dml utilizes pairwise constraints to build semantic patches and align these patches to obtain the optimal distance metric for the new feature space experimental results in image clustering demonstrate that the performance of sp dml is appealing

nonlinear component analysis as a kernel eigenvalue problem a new method for performing a nonlinear form of principal component analysis is proposed by the use of integral operator kernel functions one can efficiently compute principal components in high dimensional feature spaces related to input space by some nonlinear map for instance the space of all possible five pixel products in 16 16 images we give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition

flexible manifold embedding a framework for semi supervised and unsupervised dimension reduction we propose a unified manifold learning framework for semi supervised and unsupervised dimension reduction by employing a simple but effective linear regression function to map the new data points for semi supervised dimension reduction we aim to find the optimal prediction labels f for all the training samples x the linear regression function h x and the regression residue f0 f h x simultaneously our new objective function integrates two terms related to label fitness and manifold smoothness as well as a flexible penalty term defined on the residue f0 our semi supervised learning framework referred to as flexible manifold embedding fme can effectively utilize label information from labeled data as well as a manifold structure from both labeled and unlabeled data by modeling the mismatch between h x and f we show that fme relaxes the hard linear constraint f h x in manifold regularization mr making it better cope with the data sampled from a nonlinear manifold in addition we propose a simplified version referred to as fme u for unsupervised dimension reduction we also show that our proposed framework provides a unified view to explain and understand many semi supervised supervised and unsupervised dimension reduction techniques comprehensive experiments on several benchmark databases demonstrate the significant improvement over existing dimension reduction algorithms

on the development of explicit robust schemes for implementation of a class of hyperelastic models in large strain analysis of rubbers the issue of developing effective and robust schemes to implement a class of the ogden type hypereiastic constitutive models for large strain analysis of rubber like materials is addressed to this end explicit forms for the corresponding material tangent stiffness tensors are developed and these are valid for the entire deformation range i e with both distinct as well as repeated principal stretch values throughout the analysis the various implications of the underlying property of separability of the strain energy functions are exploited thus leading to compact final forms of the tensor expressions in particular this facilitated the treatment of the complex cases of uncoupled volumetric deviatoric formulations for incompressible materials which are becoming increasingly popular in recent years the forms derived are also amenable for use with symbolic manipulation packages for systematic code generation

an efficient implementation of surface impedance boundary conditions for the finite difference time domain method an efficient way to implement the surface impedance boundary conditions sibc for the finite difference time domain fdtd method is presented in this paper surface impedance boundary conditions are first formulated for a lossy dielectric half space in the frequency domain the impedance function of a lossy medium is approximated with a series of first order rational functions then the resulting time domain convolution integrals are computed using recursive formulas which are obtained by assuming that the fields are piecewise linear in time thus the recursive formulas derived here are second order accurate unlike a previously published method 7 which requires preprocessing to compute the exponential approximation prior to the fdtd simulation the preprocessing time is eliminated by performing a rational approximation on the normalized frequency domain impedance this approximation is independent of material properties and the results are tabulated for reference the implementation of the sibc for a pec backed lossy dielectric shell is also introduced

riemannian manifold learning recently manifold learning has been widely exploited in pattern recognition data analysis and machine learning this paper presents a novel framework called riemannian manifold learning rml based on the assumption that the input high dimensional data lie on an intrinsically low dimensional riemannian manifold the main idea is to formulate the dimensionality reduction problem as a classical problem in riemannian geometry that is how to construct coordinate charts for a given riemannian manifold we implement the riemannian normal coordinate chart which has been the most widely used in riemannian geometry for a set of unorganized data points first two input parameters the neighborhood size k and the intrinsic dimension d are estimated based on an efficient simplicial reconstruction of the underlying manifold then the normal coordinates are computed to map the input high dimensional data into a low dimensional space experiments on synthetic data as well as real world images demonstrate that our algorithm can learn intrinsic geometric structures of the data preserve radial geodesic distances and yield regular embeddings

clustering documents with labeled and unlabeled documents using fuzzy semi kmeans while focusing on document clustering this work presents a fuzzy semi supervised clustering algorithm called fuzzy semi kmeans the fuzzy semi kmeans is an extension of k means clustering model and it is inspired by an em algorithm and a gaussian mixture model additionally the fuzzy semi kmeans provides the flexibility to employ different fuzzy membership functions to measure the distance between data this work employs gaussian weighting function to conduct experiments but cosine similarity function can be used as well this work conducts experiments on three data sets and compares fuzzy semi kmeans with several methods the experimental results indicate that fuzzy semi kmeans can generally outperform the other methods

toward optimal manifold hashing via discrete locally linear embedding binary code learning also known as hashing has received increasing attention in large scale visual search by transforming high dimensional features to binary codes the original euclidean distance is approximated via hamming distance more recently it is advocated that it is the manifold distance rather than the euclidean distance that should be preserved in the hamming space however it retains as an open problem to directly preserve the manifold structure by hashing in particular it first needs to build the local linear embedding in the original feature space and then quantize such embedding to binary codes such a two step coding is problematic and less optimized besides the off line learning is extremely time and memory consuming which needs to calculate the similarity matrix of the original data in this paper we propose a novel hashing algorithm termed discrete locality linear embedding hashing dllh which well addresses the above challenges the dllh directly reconstructs the manifold structure in the hamming space which learns optimal hash codes to maintain the local linear relationship of data points to learn discrete locally linear embedding codes we further propose a discrete optimization algorithm with an iterative parameters updating scheme moreover an anchor based acceleration scheme termed anchor dllh is further introduced which approximates the large similarity matrix by the product of two low rank matrices experimental results on three widely used benchmark data sets i e cifar10 nus wide and youtube face have shown superior performance of the proposed dllh over the state of the art approaches

surface impedance boundary conditions in time domain for guided structures of arbitrary cross section with lossy dielectric walls this paper presents a procedure to implement the concept of surface impedance boundary condition sibc for lossy dielectrics in time domain the sibc formulation validation and results are obtained with a time domain numerical method namely the transmission line matrix tlm the formulation of surface impedance in frequency domain is converted to the time domain using a rational approximation followed by a bilinear transform with a canonical representation in the state space this surface impedance formulation has been implemented in tlm simulations show that the proposed model is valid for interfaces between air and a lossy dielectric and arbitrary conductivity sigma an application that concerns mode propagation in a rectangular tunnel with lossy dielectric walls is shown a comparison with an approximate method proposed by marcatilli is presented it confirms that the propagation in electrically large guided structures such as tunnels with lossy dielectric walls can be efficiently modeled by this approach

nonlinear embedding preserving multiple local linearities locally linear embedding lle is one of the effective and efficient algorithms for nonlinear dimensionality reduction this paper discusses the stability of lle focusing on the optimal weights for extracting local linearity behind the considered manifold it is proven that there are multiple sets of weights that are approximately optimal and can be used to improve the stability of lle a new algorithm using multiple weights is then proposed together with techniques for constructing multiple weights this algorithm is called as nonlinear embedding preserving multiple local linearities neml neml improves the preservation of local linearity and is more stable than lle a short analysis for neml is also given for isometric manifolds neml is compared with the local tangent space alignment ltsa in methodology since both of them adopt multiple local constraints numerical examples are given to show the improvement and efficiency of neml

clustering and embedding using commute times this paper exploits the properties of the commute time between nodes of a graph for the purposes of clustering and embedding and explores its applications to image segmentation and multibody motion tracking our starting point is the lazy random walk on the graph which is determined by the heat kernel of the graph and can be computed from the spectrum of the graph laplacian we characterize the random walk using the commute time that is the expected time taken for a random walk to travel between two nodes and return and show how this quantity may be computed from the laplacian spectrum using the discrete green s function our motivation is that the commute time can be anticipated to be a more robust measure of the proximity of data than the raw proximity matrix in this paper we explore two applications of the commute time the first is to develop a method for image segmentation using the eigenvector corresponding to the smallest eigenvalue of the commute time matrix we show that our commute time segmentation method has the property of enhancing the intragroup coherence while weakening intergroup coherence and is superior to the normalized cut the second application is to develop a robust multibody motion tracking method using an embedding based on the commute time our embedding procedure preserves commute time and is closely akin to kernel pca the laplacian eigenmap and the diffusion map we illustrate the results on both synthetic image sequences and real world video sequences and compare our results with several alternative methods

image clustering using local discriminant models and global integration in this paper we propose a new image clustering algorithm referred to as clustering using local discriminant models and global integration ldmgi to deal with the data points sampled from a nonlinear manifold for each data point we construct a local clique comprising this data point and its neighboring data points inspired by the fisher criterion we use a local discriminant model for each local clique to evaluate the clustering performance of samples within the local clique to obtain the clustering result we further propose a unified objective function to globally integrate the local models of all the local cliques with the unified objective function spectral relaxation and spectral rotation are used to obtain the binary cluster indicator matrix for all the samples we show that ldmgi shares a similar objective function with the spectral clustering sc algorithms e g normalized cut ncut in contrast to ncut in which the laplacian matrix is directly calculated based upon a gaussian function a new laplacian matrix is learnt in ldmgi by exploiting both manifold structure and local discriminant information we also prove that k means and discriminative k means diskmeans are both special cases of ldmgi extensive experiments on several benchmark image datasets demonstrate the effectiveness of ldmgi we observe in the experiments that ldmgi is more robust to algorithmic parameter when compared with ncut thus ldmgi is more appealing for the real image clustering applications in which the ground truth is generally not available for tuning algorithmic parameters

intrinsic dimension estimation of manifolds by incising balls dimensionality reduction is a very important tool in data mining intrinsic dimension of data sets is a key parameter for dimensionality reduction however finding the correct intrinsic dimension is a challenging task in this paper a new intrinsic dimension estimation method is presented the estimator is derived by finding the exponential relationship between the radius of an incising ball and the number of samples included in the ball the method is compared with the previous dimension estimation methods experiments have been conducted on synthetic and high dimensional image data sets and on data sets of the santa fe time series competition and the results show that the new method is accurate and robust

approximating the surface impedance of a homogeneous lossy half space an example of dialable accuracy we present an approximation by exponentials of the time domain surface impedance of a lossy half space gauss chebyshev quadrature of order n 1 is employed to approximate an integral representation of the modified bessel functions comprising the time domain impedance kernel an explicit error estimate is obtained in terms of the physical parameters the computation time and the number of quadrature points n we show that our approximation is as accurate as other approaches which do not come with such an error estimate the paper investigates the conditions under which the derived error estimate also applies to the approximation of j a roden and s d gedney see trans microwave theory tech vol 47 p 1954 1999

characterizing the functional connectivity diversity of the insula cortex subregions diversity curves and behavior the connectivity of the insula cortex is diverse we present new models to characterize the resting state connectional diversity of the human insula cortex and perform model selection using high quality fmri data from the human connectome project we first attempt to parcellate the insula into distinct subregions using traditional clustering methods but find that the resulting subregions are not homogeneous and that the optimal number of subregions is substantially influenced by data smoothness we then introduce the concept of a diversity curve which we use to continuously parameterize the insula s laplacian eigenmap with respect to streamlines propagated through the eigenmap s gradient field to perform model selection we compare the insula s diversity curve to benchmark diversity curves for i two distinct regions ii a continuum of gradual change and iii an absence of any connectional diversity i e homogenous region of the three benchmarks tested we find that the insula s connectional diversity is most parsimoniously modeled as continuum of gradual change from dorsal posterior to ventral anterior we find that individuals who score high on measures of positive affect self efficacy emotion recognition motor dexterity and gustation show greater diversity within the anterior insula our findings are replicated using data from a second fmri session we conclude that the functional connectivity diversity of the insula can be characterized parsimoniously as a continuum avoiding the vexed task of determining an optimal number of insula subregions and that inter individual variation in this continuum can explain significant variation in behavior

a least squares framework for component analysis over the last century component analysis ca methods such as principal component analysis pca linear discriminant analysis lda canonical correlation analysis cca locality preserving projections lpp and spectral clustering sc have been extensively used as a feature extraction step for modeling classification visualization and clustering ca techniques are appealing because many can be formulated as eigen problems offering great potential for learning linear and nonlinear representations of data in closed form however the eigen formulation often conceals important analytic and computational drawbacks of ca techniques such as solving generalized eigen problems with rank deficient matrices e g small sample size problem lacking intuitive interpretation of normalization factors and understanding commonalities and differences between ca methods this paper proposes a unified least squares framework to formulate many ca methods we show how pca lda cca lpp sc and its kernel and regularized extensions correspond to a particular instance of least squares weighted kernel reduced rank regression ls wkrrr the ls wkrrr formulation of ca methods has several benefits 1 provides a clean connection between many ca techniques and an intuitive framework to understand normalization factors 2 yields efficient numerical schemes to solve ca techniques 3 overcomes the small sample size problem 4 provides a framework to easily extend ca methods we derive weighted generalizations of pca lda sc and cca and several new ca techniques

image completion by diffusion maps and spectral relaxation we present a framework for image inpainting that utilizes the diffusion framework approach to spectral dimensionality reduction we show that on formulating the inpainting problem in the embedding domain the domain to be inpainted is smoother in general particularly for the textured images thus the textured images can be inpainted through simple exemplar based and variational methods we discuss the properties of the induced smoothness and relate it to the underlying assumptions used in contemporary inpainting schemes as the diffusion embedding is nonlinear and noninvertible we propose a novel computational approach to approximate the inverse mapping from the inpainted embedding space to the image domain we formulate the mapping as a discrete optimization problem solved through spectral relaxation the effectiveness of the presented method is exemplified by inpainting real images where it is shown to compare favorably with contemporary state of the art schemes

near field pml optimization for low and high order fdtd algorithms using closed form predictive equations the convolutional perfectly matched layer cpml absorbing boundary condition is fully capable of handling near field wave absorption that usually combines near grazing wave incidence with wave evanescence the appropriate choice of the various cpml parameters to realize this potential for any given simulation problem is a challenging task that is typically achieved through exhaustive and time consuming searches that involve large numbers of full scale simulations the presented work here uses a previously developed predictive system of equations that accurately determines numerical reflections off the pml interface and embeds it into a global optimization routine that reliably computes the required optimum cpml parameters this predictive system of equations has also been extended and validated for the m24 and fv24 integral based high order fdtd algorithms with this approach the task of selecting optimum cpml parameters that would usually take several days of intense computations can now be accomplished within a few minutes on an average personal computer

laplacian regularized low rank representation and its applications low rank representation lrr has recently attracted a great deal of attention due to its pleasing efficacy in exploring low dimensional subspace structures embedded in data for a given set of observed data corrupted with sparse errors lrr aims at learning a lowest rank representation of all data jointly lrr has broad applications in pattern recognition computer vision and signal processing in the real world data often reside on low dimensional manifolds embedded in a high dimensional ambient space however the lrr method does not take into account the non linear geometric structures within data thus the locality and similarity information among data may be missing in the learning process to improve lrr in this regard we propose a general laplacian regularized low rank representation framework for data representation where a hypergraph laplacian regularizer can be readily introduced into i e a non negative sparse hyper laplacian regularized lrr model nshlrr by taking advantage of the graph regularizer our proposed method not only can represent the global low dimensional structures but also capture the intrinsic non linear geometric information in data the extensive experimental results on image clustering semi supervised image classification and dimensionality reduction tasks demonstrate the effectiveness of the proposed method

face recognition using elasticfaces though principle component analysis pca and locality preserving projections lpps are two of the most popular linear methods for face recognition pca can only see the euclidean structure of the training set and lpp preserves the nonlinear submanifold structure hidden in the training set in this paper we propose the elastic preserving projections epps which by incorporating the merits of the local geometry and the global information of the training set epp outputs a sample subspace which simultaneously preserves the local geometrical structure and exploits the global information of the training set different from some other linear dimensionality reduction methods epp can be deemed as learning both the coordinates and the affinities between sample points furthermore the effectiveness of our proposed algorithm is analyzed theoretically and confirmed by some experiments on several well known face databases the obtained results indicate that epp significantly outperforms its other rival algorithms

low dimensional embedding of fmri datasets abstract we propose a novel method to embed a functional magnetic resonance imaging fmri dataset in a low dimensional space the embedding optimally preserves the local functional coupling between fmri time series and provides a low dimensional coordinate system for detecting activated voxels to compute the embedding we build a graph of functionally connected voxels we use the commute time instead of the geodesic distance to measure functional distances on the graph because the commute time can be computed directly from the eigenvectors of a symmetric version the graph probability transition matrix we use these eigenvectors to embed the dataset in low dimensions after clustering the datasets in low dimensions coherent structures emerge that can be easily interpreted we performed an extensive evaluation of our method comparing it to linear and nonlinear techniques using synthetic datasets and in vivo datasets we analyzed datasets from the ebc competition obtained with subjects interacting in an urban virtual reality environment our exploratory approach is able to detect independently visual areas v1 v2 v5 mt auditory areas and language areas our method can be used to analyze fmri collected during natural stimuli

data visualization and dimensionality reduction using kernel maps with a reference point in this paper a new kernel based method for data visualization and dimensionality reduction is proposed a reference point is considered corresponding to additional constraints taken in the problem formulation in contrast with the class of kernel eigenmap methods the solution coordinates in the low dimensional space is characterized by a linear system instead of an eigenvalue problem the kernel maps with a reference point are generated from a least squares support vector machine ls svm core part that is extended with an additional regularization term for preserving local mutual distances together with reference point constraints the kernel maps possess primal and dual model representations and provide out of sample extensions e g for validation based tuning the method is illustrated on toy problems and real life data sets

the use of surface impedance concepts in the finite difference time domain method surface impedance concepts are introduced into the finite difference time domain fdtd method lossy conductors are replaced by surface impedance boundary conditions sibc reducing the solution space and producing significant computational savings specifically a sibc is developed to replace a lossy dielectric half space an efficient implementation of this fdtd sibc based on the recursive properties of convolution with exponentials is presented finally three problems are studied to illustrate the accuracy of the fdtd sibc formulation a plane wave incident on a lossy dielectric half space a line current over a lossy dielectric half space and wave propagation in a parallel plate waveguide with lossy walls

approximate orthogonal sparse embedding for dimensionality reduction locally linear embedding lle is one of the most well known manifold learning methods as the representative linear extension of lle orthogonal neighborhood preserving projection onpp has attracted widespread attention in the field of dimensionality reduction in this paper a unified sparse learning framework is proposed by introducing the sparsity or l 1 norm learning which further extends the lle based methods to sparse cases theoretical connections between the onpp and the proposed sparse linear embedding are discovered the optimal sparse embeddings derived from the proposed framework can be computed by iterating the modified elastic net and singular value decomposition we also show that the proposed model can be viewed as a general model for sparse linear and nonlinear kernel subspace learning based on this general model sparse kernel embedding is also proposed for nonlinear sparse feature extraction extensive experiments on five databases demonstrate that the proposed sparse learning framework performs better than the existing subspace learning algorithm particularly in the cases of small sample sizes

spectral clustering and the high dimensional stochastic blockmodel networks or graphs can easily represent a diverse set of data sources that are characterized by interacting units or actors social ne tworks representing people who communicate with each other are one example communities or clusters of highly connected actors form an essential feature in the structure of several empirical networks spectral clustering is a popular and computationally feasi ble method to discover these communities the stochastic block model holland et al 1983 is a social network model with well defined communities each node is a member of one community for a network generated from the stochastic block model we bound the number of nodes misclus tered by spectral clustering the asymptotic results in th is paper are the first clustering results that allow the number of clusters in the model to grow with the number of nodes hence the name high dimensional in order to study spectral clustering under the stochastic block model we first show that under the more general latent space model the eigenvectors of the normalized graph laplacian asymptotically converge to the eigenvectors of a population normal ized graph laplacian aside from the implication for spectral clustering this provides insight into a graph visualization technique our method of studying the eigenvectors of random matrices is original ams 2000 subject classifications primary 62h30 62h25 secondary 60b20

robust locally linear embedding in the past few years some nonlinear dimensionality reduction nldr or nonlinear manifold learning methods have aroused a great deal of interest in the machine learning community these methods are promising in that they can automatically discover the low dimensional nonlinear manifold in a high dimensional data space and then embed the data points into a low dimensional embedding space using tractable linear algebraic techniques that are easy to implement and are not prone to local minima despite their appealing properties these nldr methods are not robust against outliers in the data yet so far very little has been done to address the robustness problem in this paper we address this problem in the context of an nldr method called locally linear embedding lle based on robust estimation techniques we propose an approach to make lle more robust we refer to this approach as robust locally linear embedding rlle we also present several specific methods for realizing this general rlle approach experimental results on both synthetic and real world data show that rlle is very robust against outliers

pattern vectors from algebraic graph theory graph structures have proven computationally cumbersome for pattern analysis the reason for this is that before graphs can be converted to pattern vectors correspondences must be established between the nodes of structures which are potentially of different size to overcome this problem in this paper we turn to the spectral decomposition of the laplacian matrix we show how the elements of the spectral matrix for the laplacian can be used to construct symmetric polynomials that are permutation invariants the coefficients of these polynomials can be used as graph features which can be encoded in a vectorial manner we extend this representation to graphs in which there are unary attributes on the nodes and binary attributes on the edges by using the spectral decomposition of a hermitian property matrix that can be viewed as a complex analogue of the laplacian to embed the graphs in a pattern space we explore whether the vectors of invariants can be embedded in a low dimensional space using a number of alternative strategies including principal components analysis pca multidimensional scaling mds and locality preserving projection lpp experimentally we demonstrate that the embeddings result in well defined graph clusters our experiments with the spectral representation involve both synthetic and real world data the experiments with synthetic data demonstrate that the distances between spectral feature vectors can be used to discriminate between graphs on the basis of their structure the real world experiments show that the method can be used to locate clusters of graphs

laplacian eigenmaps for dimensionality reduction and data representation one of the central problems in machine learning and pattern recognition is to develop appropriate representations for complex data we consider the problem of constructing a representation for data lying on a low dimensional manifold embedded in a high dimensional space drawing on the correspondence between the graph laplacian the laplace beltrami operator on the manifold and the connections to the heat equation we propose a geometrically motivated algorithm for representing the high dimensional data the algorithm provides a computationally efficient approach to nonlinear dimensionality reduction that has locality preserving properties and a natural connection to clustering some potential applications and illustrative examples are discussed

construction of approximation spaces for reinforcement learning linear reinforcement learning rl algorithms like least squares temporal difference learning lstd require basis functions that span approximation spaces of potential value functions this article investigates methods to construct these bases from samples we hypothesize that an ideal approximation spaces should encode diffusion distances and that slow feature analysis sfa constructs such spaces to validate our hypothesis we provide theoretical statements about the lstd value approximation error and induced metric of approximation spaces constructed by sfa and the state of the art methods krylov bases and proto value functions pvf in particular we prove that sfa minimizes the average over all tasks in the same environment bound on the above approximation error compared to other methods sfa is very sensitive to sampling and can sometimes fail to encode the whole state space we derive a novel importance sampling modification to compensate for this effect finally the lstd and least squares policy iteration lspi performance of approximation spaces constructed by krylov bases pvf sfa and pca is compared in benchmark tasks and a visual robot navigation experiment both in a realistic simulation and with a robot the results support our hypothesis and suggest that i sfa provides subspace invariant features for mdps with self adjoint transition operators which allows strong guarantees on the approximation error ii the modified sfa algorithm is best suited for lspi in both discrete and continuous state spaces and iii approximation spaces encoding diffusion distances facilitate lspi performance

matrix pencil method for estimating parameters of exponentially damped undamped sinusoids in noise a study of a matrix pencil method for estimating frequencies and damping factors of exponentially damped and or undamped sinusoids in noise is presented comparison of this method to a polynomial method svd prony method shows that the matrix pencil method and the polynomial method are two special cases of a matrix prediction approach and that the pencil method is more efficient in computation and less restrictive about signal probes it is found through perturbation analysis and simulation that for signals with unknown damping factors the pencil method is less sensitive to noise than the polynomial method an expression of the cramer rao bound for the exponential signals is presented

linear dimensionality reduction survey insights and generalizations linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data due to their simple geometric interpretations and typically attractive computational properties these methods capture many data features of interest such as covariance dynamical structure correlation between data sets input output relationships and margin between data classes methods have been developed with a variety of names and motivations in many fields and perhaps as a result the connections between all these methods have not been highlighted here we survey methods from this disparate literature as optimization programs over matrix manifolds we discuss principal component analysis factor analysis linear multidimensional scaling fisher s linear discriminant analysis canonical correlations analysis maximum autocorrelation factors slow feature analysis sufficient dimensionality reduction undercomplete independent component analysis linear regression distance metric learning and more this optimization framework gives insight to some rarely discussed shortcomings of well known methods such as the suboptimality of certain eigenvector solutions modern techniques for optimization over matrix manifolds enable a generic linear dimensionality reduction solver which accepts as input data and an objective to be optimized and returns as output an optimal low dimensional projection of the data this simple optimization framework further allows straightforward generalizations and novel variants of classical methods which we demonstrate here by creating an orthogonal projection canonical correlations analysis more broadly this survey and generic solver suggest that linear dimensionality reduction can move toward becoming a blackbox objective agnostic numerical technology

explicit robust schemes for implementation of a class of principal value based constitutive models symbolic and numeric implementation the issue of developing effective and robust schemes to implement a class of the ogden type hyperelastic constitutive models is addressed to this end special purpose functions running under macsyma are developed for the symbolic derivation evaluation and automatic fortran code generation of explicit expressions for the corresponding stress function and material tangent stiffness tensors these explicit forms are valid over the entire deformation range since the singularities resulting from repeated principal stretch values have been theorectically removed the required computational algorithms are outlined and the resulting fortran computer code is presented

principal manifolds and nonlinear dimensionality reduction via tangent space alignment we present a new algorithm for manifold learning and nonlinear dimensionality reduction based on a set of unorganized data points sampled with noise from a parameterized manifold the local geometry of the manifold is learned by constructing an approximation for the tangent space at each data point and those tangent spaces are then aligned to give the global coordinates of the data points with respect to the underlying manifold we also present an error analysis of our algorithm showing that reconstruction errors can be quite small in some cases we illustrate our algorithm using curves and surfaces both in two dimensional three dimensional 2d 3d euclidean spaces and in higher dimensional euclidean spaces we also address several theoretical and algorithmic issues for further research and improvements

cluster ensembles a knowledge reuse framework for combining multiple partitions this paper introduces the problem of combining multiple partitionings of a set of objects into a single consolidated clustering without accessing the features or algorithms that determined these partitionings we first identify several application scenarios for the resultant knowledge reuse framework that we call cluster ensembles the cluster ensemble problem is then formalized as a combinatorial optimization problem in terms of shared mutual information in addition to a direct maximization approach we propose three effective and efficient techniques for obtaining high quality combiners consensus functions the first combiner induces a similarity measure from the partitionings and then reclusters the objects the second combiner is based on hypergraph partitioning the third one collapses groups of clusters into meta clusters which then compete for each object to determine the combined clustering due to the low computational costs of our techniques it is quite feasible to use a supra consensus function that evaluates all three approaches against the objective function and picks the best solution for a given situation we evaluate the effectiveness of cluster ensembles in three qualitatively different application scenarios i where the original clusters were formed based on non identical sets of features ii where the original clustering algorithms worked on non identical sets of objects and iii where a common data set is used and the main purpose of combining multiple clusterings is to improve the quality and robustness of the solution promising results are obtained in all three situations for synthetic as well as real data sets

manifold regularization and semi supervised learning some theoretical analyses manifold regularization belkin et al 2006 is a geometrically motivated framework for machine learning within which several semi supervised algorithms have been constructed here we try to provide some theoretical understanding of this approach our main result is to expose the natural structure of a class of problems on which manifold regularization methods are helpful we show that for such problems no supervised learner can learn effectively on the other hand a manifold based learner that knows the manifold or learns it from unlabeled examples can learn with relatively few labeled examples our analysis follows a minimax style with an emphasis on finite sample results in terms of n the number of labeled examples these results allow us to properly interpret manifold regularization and related spectral and geometric algorithms in terms of their potential use in semi supervised learning

large scale eigenvector approximation via hilbert space embedding nystrom the nystrom method approximates eigenvectors of a given kernel matrix by randomly sampling subset of data previous researches focus on good kernel approximation while the quality of eigenvector approximation is rarely explored in online eigenvector approximation method one can minimize the kernel approximation error to guarantee a good eigenvector approximation however in this work we paradoxically prove that for batch approximation methods like nystrom it is no longer true this unexpected discovery opens a question what criterion should we use in nystrom to generate a decent eigenvector approximation to address this problem we propose a novel criterion named hilbert space embedding hse nystrom criterion which directly minimizes the eigenvector approximation error the proposed hse criterion provides a general framework to approximate eigenvectors within linear time and space complexity we then show that we can rediscover many successful nystrom methods with the proposed criterion including k means nystrom and density nystrom to further demonstrate the power of our criterion we actually design a novel algorithm to approximate eigenvectors of laplacian matrices based on the proposed criterion with better accuracy among existing linear complexity methods we demonstrate the efficiency and efficacy of our proposal in numerical experiments highlightsa new criterion for nystrom method to find accurate eigenvector approximation the conventional criterion does not work well in eigenvector approximation we need to adjust kernel width together with average weight adaptively propose a new criterion to adaptively optimize the parameters efficiently

graph spectral image smoothing using the heat kernel a new method for smoothing both gray scale and color images is presented that relies on the heat diffusion equation on a graph we represent the image pixel lattice using a weighted undirected graph the edge weights of the graph are determined by the gaussian weighted distances between local neighboring windows we then compute the associated laplacian matrix the degree matrix minus the adjacency matrix anisotropic diffusion across this weighted graph structure with time is captured by the heat equation and the solution i e the heat kernel is found by exponentiating the laplacian eigensystem with time image smoothing is accomplished by convolving the heat kernel with the image and its numerical implementation is realized by using the krylov subspace technique the method has the effect of smoothing within regions but does not blur region boundaries we also demonstrate the relationship between our method standard diffusion based pdes fourier domain signal processing and spectral clustering experiments and comparisons on standard images illustrate the effectiveness of the method