the helmholtz machine discovering the structure inherent in a set of patterns is a fundamental aim of statistical inference or learning one fruitful approach is to build a parameterized stochastic generative model independent draws from which are likely to produce the patterns for all but the simplest generative models each pattern can be generated in exponentially many ways it is thus intractable to adjust the parameters to maximize the probability of the observed patterns we describe a way of finessing this combinatorial explosion by maximizing an easily computed lower bound on the probability of the observations our method can be viewed as a form of hierarchical self supervised learning that may relate to the function of bottom up and top down cortical processing pathways

learning gaussian mixture models with entropy based criteria in this paper we address the problem of estimating the parameters of gaussian mixture models although the expectation maximization em algorithm yields the maximum likelihood ml solution its sensitivity to the selection of the starting parameters is well known and it may converge to the boundary of the parameter space furthermore the resulting mixture depends on the number of selected components but the optimal number of kernels may be unknown beforehand we introduce the use of the entropy of the probability density function pdf associated to each kernel to measure the quality of a given mixture model with a fixed number of kernels we propose two methods to approximate the entropy of each kernel and a modification of the classical em algorithm in order to find the optimum number of components of the mixture moreover we use two stopping criteria a novel global mixture entropy based criterion called gaussianity deficiency gd and a minimum description length mdl principle based one our algorithm called entropy based em ebem starts with a unique kernel and performs only splitting by selecting the worst kernel attending to gd we have successfully tested it in probability density estimation pattern classification and color image segmentation experimental results improve the ones of other state of the art model order selection methods

learning nonlinear image manifolds by global alignment of local linear models appearance based methods based on statistical models of the pixel values in an image region rather than geometrical object models are increasingly popular in computer vision in many applications the number of degrees of freedom dof in the image generating process is much lower than the number of pixels in the image if there is a smooth function that maps the dof to the pixel values then the images are confined to a low dimensional manifold embedded in the image space we propose a method based on probabilistic mixtures of factor analyzers to 1 model the density of images sampled from such manifolds and 2 recover global parameterizations of the manifold a globally nonlinear probabilistic two way mapping between coordinates on the manifold and images is obtained by combining several locally valid linear mappings we propose a parameter estimation scheme that improves upon an existing scheme and experimentally compare the presented approach to self organizing maps generative topographic mapping and mixtures of factor analyzers in addition we show that the approach also applies to finding mappings between different embeddings of the same manifold

information theoretic matching of two point sets this paper describes the theoretic roadmap of least relative entropy matching of two point sets the novel feature is to align two point sets without needing to establish explicit point correspondences the recovery of transformational geometry is achieved using a mixture of principal axes registrations whose parameters are estimated by minimizing the relative entropy between the two point distributions and using the expectation maximization algorithm we give evidence of the optimality of the method and we then evaluate the algorithm s performance in both rigid and nonrigid image registration cases

enhanced neural gas network for prototype based clustering in practical cluster analysis tasks an efficient clustering algorithm should be less sensitive to parameter configurations and tolerate the existence of outliers based on the neural gas ng network framework we propose an efficient prototype based clustering pbc algorithm called enhanced neural gas eng network several problems associated with the traditional pbc algorithms and original ng algorithm such as sensitivity to initialization sensitivity to input sequence ordering and the adverse influence from outliers can be effectively tackled in our new scheme in addition our new algorithm can establish the topology relationships among the prototypes and all topology wise badly located prototypes can be relocated to represent more meaningful regions experimental resultson synthetic and uci datasets show that our algorithm possesses superior performance in comparison to several pbc algorithms and their improved variants such as hard c means fuzzy c means ng fuzzy possibilistic c means credibilistic fuzzy c means hard fuzzy robust clustering and alternative hard fuzzy c means in static data clustering tasks with a fixed number of prototypes

a recurrent neural network computing the largest imaginary or real part of eigenvalues of real matrices as the efficient calculation of eigenpairs of a matrix especially a general real matrix is significant in engineering and neural networks run asynchronously and can achieve high performance in calculation this paper introduces a recurrent neural network rnn to extract some eigenpair the rnn whose connection weights are dependent upon the matrix can be transformed into a complex differential system whose variable z t is a complex vector by the analytic expression of z t 2 the convergence properties of the rnn are analyzed in detail with general nonzero initial complex vector the rnn obtains the largest imaginary part of all eigenvalues by a rearrangement of connection matrix the largest real part is obtained a practice of a 7x7 matrix indicates the validity of this method two matrices whose dimensionalities are 50 and 100 respectively are employed to test the efficiency of this approach when dimension number becomes large the results imply that the iteration number at which the network enters into equilibrium state is not sensitive with dimensionality this rnn can be used to estimate the largest modulus of eigenvalues etc compared with other neural networks designed for the similar aims this rnn is applicable to general real matrices

linear fuzzy clustering with selection of variables using graded possibilistic approach linear fuzzy clustering is a useful tool for knowledge discovery in databases kdd and several modifications have been proposed in order to analyze real world data this paper proposes a new approach for estimating local linear models in which linear fuzzy clustering is performed by selecting variables that are useful for extracting correlation structure in each cluster the new clustering model uses two types of memberships one is the conventional membership that represents the degree of membership of each sample in each cluster the other is the additional parameter that represents the relative responsibility of each variable for estimation of local linear models the additional membership takes large values when the variable has close relationship with local principal components and is calculated by using the graded possibilistic approach numerical experiments demonstrate that the proposed method is useful for identifying local linear model taking typicality of each variable into account

compressive sensing on manifolds using a nonparametric mixture of factor analyzers algorithm and performance bounds nonparametric bayesian methods are employed to constitute a mixture of low rank gaussians for data x rn that are of high dimension n but are constrained to reside in a low dimensional subregion of rn the number of mixture components and their rank are inferred automatically from the data the resulting algorithm can be used for learning manifolds and for reconstructing signals from manifolds based on compressive sensing cs projection measurements the statistical cs inversion is performed analytically we derive the required number of cs random measurements needed for successful reconstruction based on easily computed quantities drawing on block sparsity properties the proposed methodology is validated on several synthetic and real datasets

discrete truncated powers and lattice points in rational polytope discrete truncate power is very useful for studying the number of nonnegative integer solutions of linear diophantine equations in this paper some detail information about discrete truncated power is presented to study the number of integer solutions of linear diophantine inequations the generalized truncated power and generalized discrete truncated power are defined and discussed respectively we use generalized discrete truncated powers and multivariate splines to investigate the lattice points in rational polytopes in particular we present the degree and period of multidimensional ehrhart polynomial

class specific feature polynomial classifier for pattern classification and its application to handwritten numeral recognition the polynomial classifier pc that takes the binomial terms of reduced subspace features as inputs has shown superior performance to multilayer neural networks in pattern classification in this paper we propose a class specific feature polynomial classifier cfpc that extracts class specific features from class specific subspaces unlike the ordinary pc that uses a class independent subspace the cfpc can be viewed as a hybrid of ordinary pc and projection distance method the class specific features better separate one class from the others and the incorporation of class specific projection distance further improves the separability the connecting weights of cfpc are efficiently learned class by class to minimize the mean square error on training samples to justify the promise of cfpc we have conducted experiments of handwritten digit recognition and numeral string recognition on the nist special database 19 sd19 the digit recognition task was also benchmarked on two standard databases usps and mnist the results show that the performance of cfpc is superior to that of ordinary pc and is competitive with support vector classifiers svcs

a nonlinear neural network model of mixture of local principal component analysis application to handwritten digits recognition abstract principal component analysis pca is a popular tool in multivariate statistics and pattern recognition recently some mixture models of local principal component analysis have attracted attention due to a number of benefits over global pca in this paper we propose a mixture model by concurrently performing global data partition and local linear pca the partition is optimal or near optimal which is realized by a soft competition algorithm called neural gas the local pca type representation is approximated by a neural learning algorithm in a nonlinear autoencoder network which is set up on the generalization of the least squares reconstruction problem leading to the standard pca such a local pca type representation has a number of numerical advantages for example faster convergence and insensitive to local minima based on this mixture model we describe a modular classification scheme to solve the problem of handwritten digits recognition we use 10 networks modules to capture different features in the 10 classes of handwritten digits with each network being a mixture model of local pca type representations when a test digit is presented to all the modules each module provides a reconstructed pattern by a prescribed principle and the system outputs the class label by comparing the reconstruction errors from the 10 networks compared with some traditional neural network based classifiers our scheme converges faster and recognizes with higher accuracy for a relatively small size of each module the classification accuracy reaches 98 6 on the training set and 97 8 on the testing set

on fuzzy semi preopen sets and fuzzy semi precontinuity the purpose of this paper is to introduce and study the concepts of fuzzy semi preopen sets and fuzzy semi precontinuous mappings in fuzzy topological spaces

on fuzzy strong semicontinuity and fuzzy precontinuity abstract using the notion of fuzzy sets azad in 1981 introduced and studied the concept of semiopen semiclosed sets semi continuous mappings almost continuous mappings and weakly continuous mappings in the fuzzy setting in the same spirit defining fuzzy open closed sets and fuzzy preopen preclosed sets we introduce and make a preliminary study of fuzzy strongly semicontinuous and fuzzy precontinuous mappings in this paper

competitive em algorithm for finite mixture models abstract in this paper we present a novel competitive em cem algorithm for finite mixture models to overcome the two main drawbacks of the em algorithm often getting trapped at local maxima and sometimes converging to the boundary of the parameter space the proposed algorithm is capable of automatically choosing the clustering number and selecting the split or merge operations efficiently based on the new competitive mechanism we propose it is insensitive to the initial configuration of the mixture component number and model parameters experiments on synthetic data show that our algorithm has very promising performance for the parameter estimation of mixture models the algorithm is also applied to the structure analysis of complicated chinese characters the results show that the proposed algorithm performs much better than previous methods with slightly heavier computation burden

unsupervised learning of finite mixture models this paper proposes an unsupervised algorithm for learning a finite mixture model from multivariate data the adjective unsupervised is justified by two properties of the algorithm 1 it is capable of selecting the number of components and 2 unlike the standard expectation maximization em algorithm it does not require careful initialization the proposed method also avoids another drawback of em for mixture fitting the possibility of convergence toward a singular estimate at the boundary of the parameter space the novelty of our approach is that we do not use a model selection criterion to choose one among a set of preestimated candidate models instead we seamlessly integrate estimation and model selection in a single algorithm our technique can be applied to any type of parametric mixture model for which it is possible to write an em algorithm in this paper we illustrate it with experiments involving gaussian mixtures these experiments testify for the good performance of our approach

fuzzy weak totally semicontinuous and fuzzy weak totally precontinuous mappings abstract the aim of this paper is to introduce some new classes of mappings called fuzzy weak totally continuous fuzzy totally precontinuous fuzzy weak totally semicontinuous fuzzy totally m precontinuous fuzzy totally m precontinuous and fuzzy totally irresolute mappings their compositions their relationships with other mappings preservations of some fuzzy spaces under these mappings and some examples are studied we also prove that the property that the mapping is fuzzy totally semicontinuous and fuzzy totally precontinuous is equivalent to the property that the mapping is fuzzy totally continuous

projection pursuit mixture density estimation in this paper we seek a gaussian mixture model gmm of an n variate probability density function usually the parameters of gmms are determined in the original n dimensional space by optimizing a maximum likelihood ml criterion a practical deficiency of this method of fitting gmms is its poor performance when dealing with high dimensional data since a large sample size is needed to match the accuracy that is possible in low dimensions we propose a method for fitting the gmm based on the projection pursuit strategy this gmm is highly constrained and hence its ability to model structure in subspaces is enhanced compared to a direct ml fitting of a gmm in high dimensions our method is closely related to recently developed independent factor analysis ifa mixture models the comparisons with ml fitting of gmm in n dimensions and ifa mixtures show that the proposed method is an attractive choice for fitting gmms using small sizes of training sets

mappings in fuzzy topological spaces abstract in this paper some characterizations of fuzzy strongly semicontinuous and fuzzy precontinuous mappings are given by using the notions of quasi coincident and quasi neighbourhood due to pu and liu j math anal appl 76 1980 571 599 the definitions of open and preopen mappings which are defined by mashhour et al acta math hung 41 1983 213 218 are extended to fuzzy sets

a continuity and ca continuity in fuzzifying topology the concepts of continuity and c continuity are considered and studied in fuzzifying topology and by making use of these concepts some decompositions of fuzzy continuity are introduced it is proved that the family of all sets in fuzzifying topology may not be a fuzzifying topology

discriminant adaptive nearest neighbor classification nearest neighbour classification expects the class conditional probabilities to be locally constant and suffers from bias in high dimensions we propose a locally adaptive form of nearest neighbour classification to try to ameliorate this curse of dimensionality we use a local linear discriminant analysis to estimate an effective metric for computing neighbourhoods we determine the local decision boundaries from centroid information and then shrink neighbourhoods in directions orthogonal to these local decision boundaries and elongate them parallel to the boundaries thereafter any neighbourhood based classifier can be employed using the modified neighbourhoods the posterior probabilities tend to be more homogeneous in the modified neighbourhoods we also propose a method for global dimension reduction that combines local dimension information in a number of examples the methods demonstrate the potential for substantial improvements over nearest neighbour classification

quantifying curvelike structures of measures by using l2 jones quantities we study the curvelike structure of special measures on n in a multiscale fashion more precisely we consider the existence and construction of a sufficiently short curve with a sufficiently large measure our main tool is an l2 variant of jones numbers which measure the scaled deviations of the given measure from a best approximating line at different scales and locations the jones function is formed by adding the squares of the l2 jones numbers at different scales and the same location using a special l2 jones function we construct a sufficiently short curve with a sufficiently large measure the length and measure estimates of the underlying curve are expressed in terms of the size of this jones function 2003 wiley periodicals inc

the number of solutions to linear diophantine equations and multivariate splines etude de la variation du nombre des solutions entieres non negatives de s equations diophantiennes lineaires en fonction des termes non homogenes

mode finding for mixtures of gaussian distributions gradient quadratic and fixed point iteration algorithms and appropriate values for their control parameters are derived for finding all modes of a gaussian mixture a problem with applications in clustering and regression the significance of the modes found is quantified locally by hessian based error bars and globally by the entropy as sparseness measure

a unifying review of linear gaussian models factor analysis principal component analysis mixtures of gaussian clusters vector quantization kalman filter models and hidden markov models can all be unified as variations of unsupervised learning under a single basic generative model this is achieved by collecting together disparate observations and derivations made by many previous authors and introducing a new way of linking discrete and continuous state models using a simple nonlinearity through the use of other nonlinearities we show how independent component analysis is also a variation of the same basic generative model we show that factor analysis and mixtures of gaussians can be implemented in autoencoder neural networks and learned using squared error plus the same regularization term we introduce a new model for static data known as sensible principal component analysis as well as a novel concept of spatially adaptive observation noise we also review some of the literature involving global and local mixtures of the basic models and provide pseudocode for inference and learning for all the basic models

bayesian analysis of mixtures of factor analyzers for bayesian inference on the mixture of factor analyzers natural conjugate priors on the parameters are introduced and then a gibbs sampler that generates parameter samples following the posterior is constructed in addition a deterministic estimation algorithm is derived by taking modes instead of samples from the conditional posteriors used in the gibbs sampler this is regarded as a maximum a posteriori estimation algorithm with hyperparameter search the behaviors of the gibbs sampler and the deterministic algorithm are compared on a simulation experiment

fast transformation invariant component analysis dimensionality reduction techniques such as principal component analysis and factor analysis are used to discover a linear mapping between high dimensional data samples and points in a lower dimensional subspace previously frey and jojic introduced transformation invariant component analysis tca to learn a linear mapping invariant to a set of known form of global transformations however parameter estimation in that model using the previously proposed expectation maximization em algorithm required scalar operations in the order of n2 where n is the dimensionality of each training example this is prohibitive for many applications of interest such as modeling mid to large size images where for instance i n may be as high as 786432 512 512 rgb image in this paper we present an efficient algorithm that reduces the computational requirements to order of nlogn with this speedup we show the effectiveness of transformation invariant component analysis in various applications including tracking learning video textures clustering object recognition and object detection in images software for tca can be downloaded from http www psi toronto edu fasttca htm

mixtures of factor analyzers with common factor loadings applications to the clustering and visualization of high dimensional data mixtures of factor analyzers enable model based density estimation to be undertaken for high dimensional data where the number of observations n is not very large relative to their dimension p in practice there is often the need to further reduce the number of parameters in the specification of the component covariance matrices to this end we propose the use of common component factor loadings which considerably reduces further the number of parameters moreover it allows the data to be displayed in low dimensional plots

a model based approach to simultaneous clustering and dimensional reduction of ordinal data the literature on clustering for continuous data is rich and wide differently that one developed for categorical data is still limited in some cases the clustering problem is made more difficult by the presence of noise variables dimensions that do not contain information about the clustering structure and could mask it the aim of this paper is to propose a model for simultaneous clustering and dimensionality reduction of ordered categorical data able to detect the discriminative dimensions discarding the noise ones following the underlying response variable approach the observed variables are considered as a discretization of underlying first order latent continuous variables distributed as a gaussian mixture to recognize discriminative and noise dimensions these variables are considered to be linear combinations of two independent sets of second order latent variables where only one contains the information about the cluster structure while the other one contains noise dimensions the model specification involves multidimensional integrals that make the maximum likelihood estimation cumbersome and in some cases infeasible to overcome this issue the parameter estimation is carried out through an em like algorithm maximizing a composite log likelihood based on low dimensional margins examples of application of the proposal on real and simulated data are performed to show the effectiveness of the proposal

dimension reduction by local principal component analysis reducing or eliminating statistical redundancy between the components of high dimensional vector data enables a lower dimensional representation without significant loss of information recognizing the limitations of principal component analysis pca researchers in the statistics and neural network communities have developed nonlinear extensions of pca this article develops a local linear approach to dimension reduction that provides accurate representations and is fast to compute we exercise the algorithms on speech and image data and compare performance with pca and with neural network implementations of nonlinear pca we find that both nonlinear techniques can provide more accurate representations than pca and show that the local linear techniques outperform neural network implementations

images frames and connectionist hierarchies the representation of hierarchically structured knowledge in systems using distributed patterns of activity is an abiding concern for the connectionist solution of cognitively rich problems here we use statistical unsupervised learning to consider semantic aspects of structured knowledge representation we meld unsupervised learning notions formulated for multilinear models with tensor product ideas for representing rich information we apply the model to images of faces

translates of multivarlate splines abstract we study spaces generated by translations of a fixed function over lattice points we provide some new algebraic and approximation properties for these spaces which show their applicability for finite element analysis

mixtures of probabilistic principal component analyzers principal component analysis pca is one of the most popular techniques for processing compressing and visualizing data although its effectiveness is limited by its global linearity while nonlinear variants of pca have been proposed an alternative paradigm is to capture data complexity by a combination of local linear pca projections however conventional pca does not correspond to a probability density and so there is no unique way to combine pca models therefore previous attempts to formulate mixture models for pca have been ad hoc to some extent in this article pca is formulated within a maximum likelihood framework based on a specific form of gaussian latent variable model this leads to a well defined mixture model for probabilistic principal component analyzers whose parameters can be determined using an expectationmaximization algorithm we discuss the advantages of this model in the context of clustering density modeling and local dimensionality reduction and we demonstrate its application to image compression and handwritten digit recognition

merging and splitting eigenspace models we present new deterministic methods that given two eigenspace models each representing a set of n dimensional observations will 1 merge the models to yield a representation of the union of the sets and 2 split one model from another to represent the difference between the sets as this is done we accurately keep track of the mean here we give a theoretical derivation of the methods empirical results relating to the efficiency and accuracy of the techniques and three general applications including the construction of gaussian mixture models that are dynamically updateable

incremental nonlinear dimensionality reduction by manifold learning understanding the structure of multidimensional patterns especially in unsupervised cases is of fundamental importance in data mining pattern recognition and machine learning several algorithms have been proposed to analyze the structure of high dimensional data based on the notion of manifold learning these algorithms have been used to extract the intrinsic characteristics of different types of high dimensional data by performing nonlinear dimensionality reduction most of these algorithms operate in a batch mode and cannot be efficiently applied when data are collected sequentially in this paper we describe an incremental version of isomap one of the key manifold learning algorithms our experiments on synthetic data as well as real world images demonstrate that our modified algorithm can maintain an accurate low dimensional representation of the data in an efficient manner

intrinsic dimension estimation via nearest constrained subspace classifier we consider the problems of classification and intrinsic dimension estimation on image data a new subspace based classifier is proposed for supervised classification or intrinsic dimension estimation the distribution of the data in each class is modeled by a union of a finite number of affine subspaces of the feature space the affine subspaces have a common dimension which is assumed to be much less than the dimension of the feature space the subspaces are found using regression based on the 0 norm the proposed method is a generalisation of classical nn nearest neighbor nfl nearest feature line classifiers and has a close relationship to ns nearest subspace classifier the proposed classifier with an accurately estimated dimension parameter generally outperforms its competitors in terms of classification accuracy we also propose a fast version of the classifier using a neighborhood representation to reduce its computational complexity experiments on publicly available datasets corroborate these claims highlightsa new classifier called ncsc and its fast version called ncsc ii are proposed classifiers of nn nfl and ns are the special cases of ncsc ncsc ncsc ii can be used as intrinsic dimension estimator using well tuned intrinsic parameter ncsc ncsc ii outperforms nn nfl and ns

a note on regular semiopen l sets and s closed spaces in this paper we introduce the concept of regular semiopen l set and investigate some of its properties and obtain a characterization of regular semiopen l sets also we characterize that every cover of an l topological space x by semiopen l sets has a finite subcover whose semiclosures cover x

pre continuity and d c p continuity in fuzzifying topology the concepts of fuzzy pre continuity and fuzzy cpre continuity are introduced and studied in fuzzifying topology essentially in order to give decompositions of fuzzy continuity

proper orthogonal decomposition and its applications part ii model reduction for mems dynamical analysis abstract proper orthogonal decomposition pod methods are popular tools for data analysis aimed at obtaining low dimensional approximate descriptions of a high dimensional process in many engineering fields the applications of pod methods to model reduction for microelectromechanical systems mems are reviewed in this paper in view of the fact that existing pod methods in the model reduction for dynamic simulation of mems dealt with only noise free data this paper proposes a neural network based method that combines robust principal component analysis pca neural network model with galerkin procedure for dynamic simulation and analysis of non linear mems with noisy data simulations are given to show the performance of the proposed method in comparison with the existing method compared with the standard pca neural network model the robust pca neural network model has a number of numerical advantages such as the stability and robustness to noise injected data and the faster convergence of iterations in the training stages than the existing neural network technique the macro model generated by using the eigenvectors extracted from the proposed method as basis functions shows its flexibility and efficiency in the representation and simulation of the original non linear partial differential equations

a re weighting strategy for improving margins we present a simple general scheme for improving margins that is inspired on well known margin theory principles the scheme is based on a sample re weighting strategy the very basic idea is in fact to add to the training set new replicas of samples which are not classified with a sufficient margin as a study case we present a new algorithm namely tvq which is an instance of the proposed scheme and involves a tangent distance based 1 nn classifier implementing a sort of quantization of the tangent distance prototypes the tangent distance models created in this way have shown a significant improvement in generalization power with respect to standard tangent models moreover the obtained models were able to outperform other state of the art algorithms such as svm in an ocr task

learning data manifolds with a cutting plane method we consider the problem of classifying data manifolds where each manifold represents invariances that are parameterized by continuous degrees of freedom conventional data augmentation methods rely

em algorithms for ml factor analysis the details of em algorithms for maximum likelihood factor analysis are presented for both the exploratory and confirmatory models the algorithm is essentially the same for both cases and involves only simple least squares regression operations the largest matrix inversion required is for aq q symmetric matrix whereq is the matrix of factors the example that is used demonstrates that the likelihood for the factor analysis model may have multiple modes that are not simply rotations of each other such behavior should concern users of maximum likelihood factor analysis and certainly should cast doubt on the general utility of second derivatives of the log likelihood as measures of precision of estimation

laplacian embedded regression for scalable manifold regularization semi supervised learning ssl as a powerful tool to learn from a limited number of labeled data and a large number of unlabeled data has been attracting increasing attention in the machine learning community in particular the manifold regularization framework has laid solid theoretical foundations for a large family of ssl algorithms such as laplacian support vector machine lapsvm and laplacian regularized least squares laprls however most of these algorithms are limited to small scale problems due to the high computational cost of the matrix inversion operation involved in the optimization problem in this paper we propose a novel framework called laplacian embedded regression by introducing an intermediate decision variable into the manifold regularization framework by using insensitive loss we obtain the laplacian embedded support vector regression lapesvr algorithm which inherits the sparse solution from svr also we derive laplacian embedded rls laperls corresponding to rls under the proposed framework both lapesvr and laperls posses a simpler form of a transformed kernel which is the summation of the original kernel and a graph kernel that captures the manifold structure the benefits of the transformed kernel are two fold 1 we can deal with the original kernel matrix and the graph laplacian matrix in the graph kernel separately and 2 if the graph laplacian matrix is sparse we only need to perform the inverse operation for a sparse matrix which is much more efficient when compared with that for a dense one inspired by kernel principal component analysis we further propose to project the introduced decision variable into a subspace spanned by a few eigenvectors of the graph laplacian matrix in order to better reflect the data manifold as well as accelerate the calculation of the graph kernel allowing our methods to efficiently and effectively cope with large scale ssl problems extensive experiments on both toy and real world data sets show the effectiveness and scalability of the proposed framework

on fuzzy pairwise semi precontinuity abstract in this paper we generalize the concepts of fuzzy semi preopen semi preclosed sets and fuzzy semi precontinuous mappings due to park and lee fuzzy sets and systems 67 1994 359 364 into fuzzy bitopological setting and discuss some of its basic properties

the diabolo classifier we present a new classification architecture based on autoassociative neural networks that are used to learn discriminant models of each class the proposed architecture has several interesting properties with respect to other model based classifiers like nearest neighbors or radial basis functions it has a low computational complexity and uses a compact distributed representation of the models the classifier is also well suited for the incorporation of a priori knowledge by means of a problemspecific distance measure in particular we will show that tangent distance simard le cun denker 1993 can be used to achieve transformation invariance during learning and recognition we demonstrate the application of this classifier to optical character recognition where it has achieved state of the art results on several reference databases relations to other models in particular those based on principal component analysis are also discussed

fuzzy semi preopen sets and fuzzy semi precontinuous mappings abstract the aim of this paper is to introduce and discuss the concepts of fuzzy semi preopen sets and fuzzy semi precontinuous mappings

on fuzzy pairwise a continuity and fuzzy pairwise pre continuity in this paper the concepts of fuzzy open closed sets fuzzy pre open pre closed sets fuzzy continuous pre continuous mappings due to bin shahna fuzzy sets and systems 44 1991 303 308 have been generalized to the fuzzy bitopological setting and some of its basic properties are studied characterizations of these mappings along with their relationship with certain other types of mappings are investigated

symmetric magic squares and multivariate splines abstract we use multivariate splines to investigate linear diophantine equations and related problems in graph theory in particular we solve a conjecture of stanley about symmetric magic squares

neural gas network for vector quantization and its application to time series prediction a neural network algorithm based on a soft max adaptation rule is presented this algorithm exhibits good performance in reaching the optimum minimization of a cost function for vector quantization data compression the soft max rule employed is an extension of the standard k means clustering procedure and takes into account a neighborhood ranking of the reference weight vectors it is shown that the dynamics of the reference weight vectors during the input driven adaptation procedure are determined by the gradient of an energy function whose shape can be modulated through a neighborhood determining parameter and resemble the dynamics of brownian particles moving in a potential determined by the data point density the network is used to represent the attractor of the mackey glass equation and to predict the mackey glass time series with additional local linear mappings for generating output values the results obtained for the time series prediction compare favorably with the results achieved by backpropagation and radial basis function networks

multivariate discrete splines and linear diophantine equations in this paper we investigate the algebraic properties of multivariate discrete splines it turns out that multivariate discrete splines are closely related to linear diophantine equations in particular we use a solvability condition for a system of linear diophantine equations to obtain a necessary and sufficient condition for the integer translates of a discrete box spline to be linearly independent in order to understand the local structure of discrete splines we develop a general theory for certain systems of linear partial difference equations using this theory we prove that the integer translates of a discrete box spline are locally linearly independent if and only if they are linearly independent

on the solution of certain systems of partial difference equations and linear dependence of translates of box splines on resout des systemes d equations aux differences partielles qui apparaissent dans l etude de la dependance lineaire des translatees de splines en boite

discriminant pattern recognition using transformation invariant neurons to overcome the problem of invariant pattern recognition simard lecun and denker 1993 proposed a successful nearest neighbor approach based on tangent distance attaining state of the art accuracy since this approach needs great computational and memory effort hastie simard and sackinger 1995 proposed an algorithm hss based on singular value decomposition svd for the generation of nondiscriminant tangent models in this article we propose a different approach based on a gradientdescent constructive algorithm called td neuron that develops discriminant models we present as well comparative results of our constructive algorithm versus hss and learning vector quantization lvq algorithms specifically we tested the hss algorithm using both the original version based on the two sided tangent distance and a new version based on the one sided tangent distance empirical results over the nist 3 database show that the td neuron is superior to both svd and lvq based algorithms since it reaches a better trade off between error and rejection

fundamental study a concise functional neural network computing the largest modulus eigenvalues and their corresponding eigenvectors of a real skew matrix quick extraction of the largest modulus eigenvalues of a real antisymmetric matrix is important for some engineering applications as neural network runs in concurrent and asynchronous manner in essence using it to complete this calculation can achieve high speed this paper introduces a concise functional neural network fnn which can be equivalently transformed into a complex differential equation to do this work after obtaining the analytic solution of the equation the convergence behaviors of this fnn are discussed simulation result indicates that with general initial complex values the network will converge to the complex eigenvector which corresponds to the eigenvalue whose imaginary part is positive and modulus is the largest of all eigenvalues comparing with other neural networks designed for the like aim this network is applicable to real skew matrices

face recognition using the second order mixture of eigenfaces method the well known eigenface method uses an eigenface set obtained from principal component analysis however the single eigenface set is not enough to represent the complicated face images with large variations of poses and or illuminations to overcome this weakness we propose a second order mixture of eigenfaces method that combines the second order eigenface method iso mpg m5750 noordwijkerhout march 2000 and the mixture of eigenfaces method a k a gaussian mixture model proceedings ijcnn2001 2001 in this method we use a couple of mixtures of multiple eigenface sets one is a mixture of multiple approximate eigenface sets for face images and another is a mixture of multiple residual eigenface sets for residual face images each mixture of multiple eigenface sets has been obtained from expectation maximization learning consecutively based on two mixture of multiple eigenface sets each face image is represented by a couple of feature vectors obtained by projecting the face image onto a selected approximate eigenface set and then by projecting the residual face image onto a selected residual eigenface set recognition is performed by the distance in the feature space between the input image and the template image stored in the face database simulation results show that the proposed second order mixture of eigenfaces method is best for face images with illumination variations and the mixture of eigenfaces method is best for the face images with pose variations in terms of average of the normalized modified retrieval rank and false identification rate

support vector machine with manifold regularization and partially labeling privacy protection a novel support vector machine with manifold regularization and partially labeling privacy protection termed as svm mr plpp is proposed for semi supervised learning ssl scenarios where only few labeled data and the class proportion of unlabeled data due to privacy protection concerns are available it integrates manifold regularization and privacy protection regularization into the laplacian support vector machine lapsvm to improve the classification accuracy privacy protection here refers to use only the class proportion of data in order to circumvent the high computational burden of the matrix inversion operation involved in svm mr plpp its scalable version called ssvm mr plpp is further developed by introducing intermediate decision variables into the original regularization framework so that the computational burden of the corresponding transformed kernel in ssvm mr plpp can be greatly reduced making it highly scalable to large datasets the experimental results on numerous datasets show the effectiveness of the proposed classifiers

fuzzy completely pre irresolute and weakly completely pre irresolute mappings in this paper we introduce fuzzy completely pre irresolute and fuzzy weakly completely pre irresolute mappings between fuzzy topological spaces and study these mappings in relation to some other types of already known mappings

transformation invariant clustering using the em algorithm clustering is a simple effective way to derive useful representations of data such as images and videos clustering explains the input as one of several prototypes plus noise in situations where each input has been randomly transformed e g by translation rotation and shearing in images and videos clustering techniques tend to extract cluster centers that account for variations in the input due to transformations instead of more interesting and potentially useful structure for example if images from a video sequence of a person walking across a cluttered background are clustered it would be more useful for the different clusters to represent different poses and expressions instead of different positions of the person and different configurations of the background clutter we describe a way to add transformation invariance to mixture models by approximating the nonlinear transformation manifold by a discrete set of points we show how the expectation maximization algorithm can be used to jointly learn clusters while at the same time inferring the transformation associated with each input we compare this technique with other methods for filtering noisy images obtained from a scanning electron microscope clustering images from videos of faces into different categories of identification and pose and removing foreground obstructions from video we also demonstrate that the new technique is quite insensitive to initial conditions and works better than standard techniques even when the standard techniques are provided with extra data

modeling the manifolds of images of handwritten digits this paper describes two new methods for modeling the manifolds of digitized images of handwritten digits the models allow a priori information about the structure of the manifolds to be combined with empirical data accurate modeling of the manifolds allows digits to be discriminated using the relative probability densities under the alternative models one of the methods is grounded in principal components analysis the other in factor analysis both methods are based on locally linear low dimensional approximations to the underlying data manifold links with other methods that model the manifold are discussed

hyperdisk based large margin classifier we introduce a large margin linear binary classification framework that approximates each class with a hyperdisk the intersection of the affine support and the bounding hypersphere of its training samples in feature space and then finds the linear classifier that maximizes the margin separating the two hyperdisks we contrast this with support vector machines svms which find the maximum margin separator of the pointwise convex hulls of the training samples arguing that replacing convex hulls with looser convex class models such as hyperdisks provides safer margin estimates that improve the accuracy on some problems both the hyperdisks and their separators are found by solving simple quadratic programs the method is extended to nonlinear feature spaces using the kernel trick and multi class problems are dealt with by combining binary classifiers in the same ways as for svms experiments on a range of data sets show that the method compares favourably with other popular large margin classifiers