dynamical system for computing the eigenvectors associated with the largest eigenvalue of a positive definite matrix a dynamical system for computing the largest eigenpair of a given positive matrix is introduced certain qualitative properties of the proposed system are analyzed in detail the authors results suggest that the weight bounding term in pca algorithms for extracting the first principal component could assume a class of forms

on the growth of nonuniform lattices in pinched negatively curved manifolds we study the relation between the exponential growth rate of volume in a pinched negatively curved manifold and the critical exponent of its lattices these objects have a long and interesting story and are closely related to the geometry and the dynamical properties of the geodesic flow of the manifold

connectedness properties of limit sets we study convergence group actions on continua and give a criterion which ensures that every global cut point is a parabolic fixed point we apply this result to the case of boundaries of relatively hyperbolic groups and consider implications for connectedness properties of such spaces

learning visual spatial pooling by strong pca dimension reduction in visual modeling invariance properties of visual cells are often explained by a pooling mechanism in which outputs of neurons with similar selectivities to some stimulus parameters are integrated so as to gain some extent of invariance to other parameters for example the classical energy model of phase invariant v1 complex cells pools model simple cells preferring similar orientation but different phases prior studies such as independent subspace analysis have shown that phase invariance properties of v1 complex cells can be learned from spatial statistics of natural inputs however those previous approaches assumed a squaring nonlinearity on the neural outputs to capture energy correlation such nonlinearity is arguably unnatural from a neurobiological viewpoint but hard to change due to its tight integration into their formalisms moreover they used somewhat complicated objective functions requiring expensive computations for optimization in this study we show that visual spatial pooling can be learned in a much simpler way using strong dimension reduction based on principal component analysis this approach learns to ignore a large part of detailed spatial structure of the input and thereby estimates a linear pooling matrix using this framework we demonstrate that pooling of model v1 simple cells learned in this way even with nonlinearities other than squaring can reproduce standard tuning properties of v1 complex cells for further understanding we analyze several variants of the pooling model and argue that a reasonable pooling can generally be obtained from any kind of linear transformation that retains several of the first principal components and suppresses the remaining ones in particular we show how the classic wiener filtering theory leads to one such variant

statistically grounded logic operators in fuzzy sets abstract in this study we are concerned with a category of logic operators for fuzzy sets that is inherently associated with underlying statistical properties of membership grades they operate on these constructs are referred to as statistically grounded logic operators namely statistically grounded ors sors for short and statistically grounded ands sands for short operators in essence they arise as a solution to the optimization problem of the following form arg min m 0 1 k 1 n w x k x k m where xk k 1 2 n are the corresponding membership grades to be combined the weight function w 0 1 0 1 captures the nature of the logic and and or aggregation respectively we show that the weight function could be induced by t norms and t conorms the weight functions could be also implied by some statistical characteristics of data membership grades the choice of the t norm t conorm depends upon the predefined form of the logic operators to be developed we demonstrate that sands and sors offer an efficient operational framework for constructing fuzzy rough sets lead to the increased sensitivity of computing possibility and necessity measures bring a new insight into fuzzy relational equations and deliver an interpretation vehicle for fuzzy clustering that is provided in the form of dependency analysis

fast iterative kernel principal component analysis we develop gain adaptation methods that improve convergence of the kernel hebbian algorithm kha for iterative kernel pca kim et al 2005 kha has a scalar gain parameter which is either held constant or decreased according to a predetermined annealing schedule leading to slow convergence we accelerate it by incorporating the reciprocal of the current estimated eigenvalues as part of a gain vector an additional normalization term then allows us to eliminate a tuning parameter in the annealing schedule finally we derive and apply stochastic meta descent smd gain vector adaptation schraudolph 1999 2002 in reproducing kernel hilbert space to further speed up convergence experimental results on kernel pca and spectral clustering of usps digits motion capture and image denoising and image super resolution tasks confirm that our methods converge substantially faster than conventional kha to demonstrate scalability we perform kernel pca on the entire mnist data set

density based fuzzy c means clustering of non convex patterns we propose a new technique to perform unsupervised data classification clustering based on density induced metric and non smooth optimization our goal is to automatically recognize multidimensional clusters of non convex shape we present a modification of the fuzzy c means algorithm which uses the data induced metric defined with the help of delaunay triangulation we detail computation of the distances in such a metric using graph algorithms to find optimal positions of cluster prototypes we employ the discrete gradient method of non smooth optimization the new clustering method is capable to identify non convex overlapped d dimensional clusters

classification of negatively pinched manifolds with amenable fundamental groups we give a diffeomorphism classification of pinched negatively curved manifolds with amenable fundamental groups namely they are precisely the mobius band and the products of r with the total spaces of flat vector bundles over closed infranilmanifolds

energy function for the one unit oja algorithm the one unit oja algorithm plays a very important role in the study of principal component analysis neural networks in this paper we propose an energy function whose steepest descent direction i e negative gradient direction is the same as the average evolution direction of the one unit oja algorithm and the energy function has two global minimal points corresponding to the two converged points of the one unit oja algorithm and it has no other local minimal points

general results on the convergence of stochastic algorithms a deterministic approach is proposed for proving the convergence of stochastic algorithms of the most general form under necessary conditions on the input noise and reasonable conditions on the nonnecessarily continuous mean field emphasis is placed on the case where more than one stationary point exists we also use this approach to prove the convergence of a stochastic algorithm with markovian dynamics

near optimal stochastic approximation for online principal component estimation principal component analysis pca has been a prominent tool for high dimensional data analysis online algorithms that estimate the principal component by processing streaming data are of tremendous practical and theoretical interests despite its rich applications theoretical convergence analysis remains largely open in this paper we cast online pca into a stochastic nonconvex optimization problem and we analyze the online pca algorithm as a stochastic approximation iteration the stochastic approximation iteration processes data points incrementally and maintains a running estimate of the principal component we prove for the first time a nearly optimal finite sample error bound for the online pca algorithm under the subgaussian assumption we show that the finite sample error bound closely matches the minimax information lower bound

a nonnegative pca algorithm for independent component analysis we consider the task of independent component analysis when the independent sources are known to be nonnegative and well grounded so that they have a nonzero probability density function pdf in the region of zero we propose the use of a nonnegative principal component analysis nonnegative pca algorithm which is a special case of the nonlinear pca algorithm but with a rectification nonlinearity and we conjecture that this algorithm will find such nonnegative well grounded independent sources under reasonable initial conditions while the algorithm has proved difficult to analyze in the general case we give some analytical results that are consistent with this conjecture and some numerical simulations that illustrate its operation

recursive estimation for ordered eigenvectors of symmetric matrix with observation noise abstract the principal component analysis is to recursively estimate the eigenvectors and the corresponding eigenvalues of a symmetric matrix a based on its noisy observations a k a n k where a is allowed to have arbitrary eigenvalues with multiplicity possibly bigger than one in the paper the recursive algorithms are proposed and their ordered convergence is established it is shown that the first algorithm a s converges to a unit eigenvector corresponding to the largest eigenvalue the second algorithm a s converges to a unit eigenvector corresponding to either the second largest eigenvalue in the case the largest eigenvalue is of single multiplicity or the largest eigenvalue if the multiplicity of the largest eigenvalue is bigger than one and so on the convergence rate is also derived

applications of axiomatic fuzzy set clustering method on management strategic analysis abstract in this paper first the axiomatic fuzzy set afs clustering method x d liu w wang t y chai the fuzzy clustering analysis based on afs theory ieee transactions on systems man and cybernetics part b 35 5 2005 1013 1027 is investigated further by improving the algorithm to be more applicable then it is used to analyze the evaluation results of 30 companies which have been studied and analyzed by g s liang et al g s liang t y chou t c han cluster analysis based on fuzzy equivalence relation european journal of operational research 166 2005 160 171 compared with liang s algorithm the proposed method is more transparent understandable and interpretable this method can be applied to the management strategic analysis based on the datasets described by mixed features such as real numbers boolean logical values linguistic descriptions the illustrative examples show that the interpretations of the clustering results of the 30 companies are almost consistent with the expert s intuitions

the fuzzy theory based on afs algebras and afs structure abstract in this paper we define a more general algebra class called afs algebras which includes ei algebra and eii algebra published by the author which are completely distributive molecular lattices we establish the fuzzy theory using afs algebras and afs structure published by the author and propose the concept of fuzzy variable and a simple controlling model based on afs theory

a distributed framework for dimensionality reduction and denoising with the recent upsurge in the amount of data transmission and use in networks a lot of emphasis is placed on designing efficient methods to extract useful information by reducing the size of large data sets a major approach in data dimensionality reduction relies on the estimation of the principal data covariance eigenspace in this paper a distributed algorithmic framework is put forth for finding the principal eigenspace of spatially scattered sensor data toward this end the standard principal component analysis framework is reformulated as a separable constrained minimization problem which is solved by utilizing coordinate descent techniques combined with the alternating direction method of multipliers computationally simple local updating recursions are obtained that involve only single hop intersensor communications the proposed distributed algorithm is shown to be robust even in the presence of inter sensor communication noise and converge to the principal eigenspace when inter sensor links are ideal the proposed distributed framework is used in a data denoising setting while extensive numerical tests using both synthetic and real data demonstrate a fast convergence rate better steady state performance of the novel algorithm over existing alternatives and communication noise resilience

a cross associative neural network for svd of non squared data matrix in signal processing this paper proposes a cross associative neural network cann for singular value decomposition svd of a non squared data matrix in signal processing in order to improve the convergence speed and avoid the potential instability of the deterministic networks associated with the cross correlation neural network models we study the global asymptotic stability of the network for tracking all the singular components and show that the selection of its learning rate in the iterative algorithm is independent of the singular value distribution of a non squared matrix the performances of cann are shown via simulations

on relative convergence properties of principal component analysis algorithms we investigate the convergence properties of two different stochastic approximation algorithms for principal component analysis and analytically explain some commonly observed experimental results in our analysis we use the theory of stochastic approximation and in particular the results of fabian 1968 to explore the asymptotic mean square errors amses of the algorithms this study reveals the conditions under which the algorithms produce smaller amses and also the conditions under which one algorithm has a smaller amse than the other experimental study with multidimensional gaussian data corroborate our analytical findings we next explore the convergence rates of the two algorithms our experiments and an analytical explanation reveals the conditions under which the algorithms converge faster to the solution and also the conditions under which one algorithm converges faster than the other

least squares type algorithm for adaptive implementation of pisarenko s harmonic retrieval method pisarenko s harmonic retrieval method involves determining the minimum eigenvalue and the corresponding eigenvector of the covariance matrix of the observed random process recently thompson 9 suggested a constrained gradient search procedure for obtaining an adaptive version of pisarenko s method and his simulations have verified that the frequency estimates provided by his procedure were unbiased however the main cost of this technique was that the initial convergence rate could be very slow for certain poor initial conditions restating the constrained minimization as an unconstrained nonlinear problem we derived an alternative gauss newton type recursive algorithm which also used the second derivative matrix or hessian this algorithm may also be viewed as an approximate least squares algorithm simulations have been performed to compare this algorithm to a slight variant of thompson s original algorithm the most important conclusions are that the least squares type algorithm has faster convergence in the beginning while its convergence rate close to the true parameters depends on the signal to noise ratio of the input signal the approximate least squares algorithm resolves the sinusoids much faster than the gradient version

new families of skewed higher order kernel estimators to solve the bss ica problem for multimodal sources mixtures this letter suggests two new types of asymmetrical higher order kernels hok that are generated using the orthogonal polynomials laguerre positive or right skew and bessel negative or left skew these skewed hok are implemented in the blind source separation independent component analysis bss ica algorithm the tests for these proposed hok are accomplished using three scenarios to simulate a real environment using actual sound sources an environment of mixtures of multimodal fast changing probability density function pdf sources that represent a challenge to the symmetrical hok and an environment of an adverse case near gaussian the separation is performed by minimizing the mutual information mi among the mixed sources the performance of the skewed kernels is compared to the performance of the standard kernels such as epanechnikov bisquare trisquare and gaussian and the performance of the symmetrical hok generated using the polynomials chebyshev1 chebyshev2 gegenbauer jacobi and legendre to the tenth order the gaussian hok are generated using the hermite polynomial and the wand and schucany procedure the comparison among the 96 kernels is based on the average intersymbol interference ratio aisir and the time needed to complete the separation in terms of aisir the skewed kernels performance is better than that of the standard kernels and rivals most of the symmetrical kernels performance the importance of these new skewed hok is manifested in the environment of the multimodal pdf mixtures in such an environment the skewed hok come in first place compared with the symmetrical hok these new families can substitute for symmetrical hoks in such applications

convergence of learning algorithms with constant learning rates the behavior of neural network learning algorithms with a small constant learning rate epsilon in stationary random input environments is investigated it is rigorously established that the sequence of weight estimates can be approximated by a certain ordinary differential equation in the sense of weak convergence of random processes as epsilon tends to zero as applications backpropagation in feedforward architectures and some feature extraction algorithms are studied in more detail

oja s algorithm for graph clustering markov spectral decomposition and risk sensitive control given a positive definite matrix m and an integer n m 1 oja s subspace algorithm will provide convergent estimates of the first n m eigenvalues of m along with the corresponding eigenvectors it is a common approach to principal component analysis this paper introduces a normalized stochastic approximation implementation of oja s subspace algorithm as well as new applications to the spectral decomposition of a reversible markov chain recall that this means that the stationary distribution satisfies the detailed balance equations meyn tweedie 2009 equivalently the statistics of the process in steady state do not change when time is reversed stability and convergence of oja s algorithm are established under conditions far milder than that assumed in previous work applications to graph clustering markov spectral decomposition and multiplicative ergodic theory are surveyed along with numerical results

cluster analysis based on fuzzy equivalence relation in this paper a cluster analysis method based on fuzzy equivalence relation is proposed at first the distance formula between two trapezoidal fuzzy numbers is used to aggregate subjects linguistic assessments about attributes ratings to obtain the compatibility relation then a fuzzy equivalence relation based on the fuzzy compatibility relation can be constructed finally using a cluster validity index to determine the best number of clusters and taking suitable cut value the clustering analysis can be effectively implemented by utilizing this clustering analysis the subjects fuzzy assessments with various rating attitudes can be taken into account in the aggregation process to assure more convincing and accurate cluster analysis

algorithms for nonnegative independent component analysis we consider the task of solving the independent component analysis ica problem x as given observations x with a constraint of nonnegativity of the source random vector s we refer to this as nonnegative independent component analysis and we consider methods for solving this task for independent sources with nonzero probability density function pdf p s down to s 0 it is sufficient to find the orthonormal rotation y wz of prewhitened sources z vx which minimizes the mean squared error of the reconstruction of z from the rectified version y sup of y we suggest some algorithms which perform this both based on a nonlinear principal component analysis pca approach and on a geodesic search method driven by differential geometry considerations we demonstrate the operation of these algorithms on an image separation problem which shows in particular the fast convergence of the rotation and geodesic methods and apply the approach to a musical audio analysis task

asymptotic performance analysis of subspace adaptive algorithms introduced in the neural network literature in the neural network literature many algorithms have been proposed for estimating the eigenstructure of covariance matrices we first show that many of these algorithms when presented in a common framework show great similitudes with the gradient like stochastic algorithms usually encountered in the signal processing literature we derive the asymptotic distribution of these different recursive subspace estimators a closed form expression of the covariances in distribution of eigenvectors and associated projection matrix estimators are given and analyzed in particular closed form expressions of the mean square error of these estimators are given it is found that these covariance matrices have a structure very similar to those describing batch estimation techniques the accuracy of our asymptotic analysis is checked by numerical simulations and it is found to be valid not only for a small step size but in a very large domain finally convergence speed and deviation from orthonormality of the different algorithms are compared and several tradeoffs are analyzed

global convergence of oja s subspace algorithm for principal component extraction oja s principal subspace algorithm is a well known and powerful technique for learning and tracking principal information in time series a thorough investigation of the convergence property of oja s algorithm is undertaken in this paper the asymptotic convergence rates of the algorithm is discovered the dependence of the algorithm on its initial weight matrix and the singularity of the data covariance matrix is comprehensively addressed

theory of topological molecular lattices abstract the aim of this paper is to bring the theories of point set topology fuzzy topology and l fuzzy topology into a unified theory the theory of topological molecular lattices for this purpose we first introduce the concepts of molecule for completely distributive complete lattice molecular lattice for short the cotopology on it and the concept of remote neighborhoods of molecules therefrom we develop the theory of pointwise topology on molecular lattices

pinching pontrjagin classes and negatively curved vector bundles we prove several finiteness results for the class a b n of n manifolds that have fundamental groups isomorphic to and that can be given complete riemannian metrics of sectional curvatures within a b where a b 0 in particular if m is a closed negatively curved manifold of dimension at least three then only finitely many manifolds in the class a b 1 m n are total spaces of vector bundles over m furthermore given a word hyperbolic group and an integer n there exists a positive e e n such that the tangent bundle of any manifold in the class 1 e 1 n has zero rational pontrjagin classes

stability of oja s pca subspace rule this paper deals with stability of oja s symmetric algorithm for estimating the principal component subspace of the input data exact conditions are derived for the gain parameter on which the discrete algorithm remains bounded the result is extended for a nonlinear version of oja s algorithm

some pathological traps for stochastic approximation we consider different kinds of pathological traps for stochastic algorithms thus extending a previous study on regular traps an illustration is given by the complete proof of the convergence of a principal component analysis pca algorithm when the eigenvalues are multiple

on afs algebra part ii this paper addresses the properties and structures of ei algebra and ein algebra in the framework of liu fuzzy sets and systems 95 1998 179 the main contribution of the paper is that the expressions of both irreducible elements or irreducible elements in molecular lattice em and ex1 xnm are developed and that the standard minimal family of elements in molecular lattice em and ex1 xnm is formulated it is proved that neither molecular lattice em nor molecular lattice ex1 xnm is a fuzzy lattice

performance analysis of adaptive eigenanalysis algorithms we present a rigorous analysis of several popular forms of short memory adaptive eigenanalysis algorithms using a stochastic averaging method a first order analysis shows that the algorithms do not have any equilibrium points despite published claims to the contrary through averaging analysis we show that they hover around an appropriate eigenvector a second order analysis is also given without the gaussian noise assumption and our results greatly outperform an earlier approximation in the literature the second order analysis has been of much interest in the offline study but in the dynamic adaptive case is uncommon

a note on linear function approximation using random projections linear function approximations based on random projections are proposed and justified for a class of fixed point and minimization problems

global analysis of oja s flow for neural networks a detailed study of oja s learning equation in neural networks is undertaken in this paper not only are such fundamental issues as existence uniqueness and representation of solutions completely resolved but also the convergence issue is resolved it is shown that the solution of oja s equation is exponentially convergent to an equilibrium from any initial value moreover the necessary and sufficient conditions are given on the initial value for the solution to converge to a dominant eigenspace of the associated autocorrelation matrix as a by product this result confirms one of oja s conjectures that the solution converges to the principal eigenspace from almost all initial values some other characteristics of the limiting solution are also revealed these facilitate the determination of the limiting solution in advance using only the initial information two examples are analyzed demonstrating the explicit dependence of the limiting solution on the initial value in another respect it is found that oja s equation is the gradient flow of generalized rayleigh quotients on a stiefel manifold

on self organizing algorithms and networks for class separability features we describe self organizing learning algorithms and associated neural networks to extract features that are effective for preserving class separability as a first step an adaptive algorithm for the computation of q sup 1 2 where q is the correlation or covariance matrix of a random vector sequence is described convergence of this algorithm with probability one is proven by using stochastic approximation theory and a single layer linear network architecture for this algorithm is described which we call the q sup 1 2 network using this network we describe feature extraction architectures for 1 unimodal and multicluster gaussian data in the multiclass case 2 multivariate linear discriminant analysis lda in the multiclass case and 3 bhattacharyya distance measure for the two class case the lda and bhattacharyya distance features are extracted by concatenating the q sup 1 2 network with a principal component analysis network and the two layer network is proven to converge with probability one every network discussed in the study considers a flow or sequence of inputs for training numerical studies on the performance of the networks for multiclass random data are presented

convergence analysis of a deterministic discrete time system of oja s pca learning algorithm the convergence of oja s principal component analysis pca learning algorithms is a difficult topic for direct study and analysis traditionally the convergence of these algorithms is indirectly analyzed via certain deterministic continuous time dct systems such a method will require the learning rate to converge to zero which is not a reasonable requirement to impose in many practical applications recently deterministic discrete time ddt systems have been proposed instead to indirectly interpret the dynamics of the learning algorithms unlike dct systems ddt systems allow learning rates to be constant which can be a nonzero this paper will provide some important results relating to the convergence of a ddt system of oja s pca learning algorithm it has the following contributions 1 a number of invariant sets are obtained based on which we can show that any trajectory starting from a point in the invariant set will remain in the set forever thus the nondivergence of the trajectories is guaranteed 2 the convergence of the ddt system is analyzed rigorously it is proven in the paper that almost all trajectories of the system starting from points in an invariant set will converge exponentially to the unit eigenvector associated with the largest eigenvalue of the correlation matrix in addition exponential convergence rate are obtained providing useful guidelines for the selection of fast convergence learning rate 3 since the trajectories may diverge the careful choice of initial vectors is an important issue this paper suggests to use the domain of unit hyper sphere as initial vectors to guarantee convergence 4 simulation results will be furnished to illustrate the theoretical results achieved

a class of learning algorithms for principal component analysis and minor component analysis principal component analysis pca and minor component analysis mca are a powerful methodology for a wide variety of applications such as pattern recognition and signal processing in this paper we first propose a differential equation for the generalized eigenvalue problem we prove that the stable points of this differential equation are the eigenvectors corresponding to the largest eigenvalue based on this generalized differential equation a class of pca and mca learning algorithms can be obtained we demonstrate that many existing pca and mca learning algorithms are special cases of this class and this class includes some new and simpler mca learning algorithms our results show that all the learning algorithms of this class have the same order of convergence speed and they are robust to implementation error

the dynamics of on line principal component analysis the learning dynamics of an on line algorithm for principal component analysis is described exactly in the thermodynamic limit by means of coupled ordinary differential equations for a set of order parameters it is demonstrated that learning is delayed significantly because existing symmetries among student vectors have to be broken a closely related effect is the perfect or partial loss of initial knowledge in the course of learning the analysis shows that different learning rates for the student vectors improve the performance of the algorithm drastically

an overlapping cluster algorithm to provide non exhaustive clustering abstract the partitioning clustering is a technique to classify n objects into k disjoint clusters and has been developed for years and widely used in many applications in this paper a new overlapping cluster algorithm is defined it differs from traditional clustering algorithms in three respects first the new clustering is overlapping because clusters are allowed to overlap with one another second the clustering is non exhaustive because an object is permitted to belong to no cluster third the goals considered in this research are the maximization of the average number of objects contained in a cluster and the maximization of the distances among cluster centers while the goals in previous research are the maximization of the similarities of objects in the same clusters and the minimization of the similarities of objects in different clusters furthermore the new clustering is also different from the traditional fuzzy clustering because the object cluster relationship in the new clustering is represented by a crisp value rather than that represented by using a fuzzy membership degree accordingly a new overlapping partitioning cluster opc algorithm is proposed to provide overlapping and non exhaustive clustering of objects finally several simulation and real world data sets are used to evaluate the effectiveness and the efficiency of the opc algorithm and the outcomes indicate that the algorithm can generate satisfactory clustering results

adaptive multiple minor directions extraction in parallel using a pca neural network a principal component analysis pca neural network is developed for online extraction of the multiple minor directions of an input signal the neural network can extract the multiple minor directions in parallel by computing the principal directions of the transformed input signal so that the stability speed problem of directly computing the minor directions can be avoided to a certain extent on the other hand the learning algorithms for updating the net weights use constant learning rates this overcomes the shortcoming of the learning rates approaching zero in addition the proposed algorithms are globally convergent so that it is very simple to choose the initial values of the learning parameters this paper presents the convergence analysis of the proposed algorithms by studying the corresponding deterministic discrete time ddt equations rigorous mathematical proof is given to prove the global convergence the theoretical results are further confirmed via simulations

distributed estimation and control of algebraic connectivity over random graphs in this paper we propose a distributed algorithm for the estimation and control of the connectivity of ad hoc networks in the presence of a random topology first given a generic random graph we introduce a novel stochastic power iteration method that allows each node to estimate and track the algebraic connectivity of the underlying expected graph using results from stochastic approximation theory we prove that the proposed method converges almost surely a s to the desired value of connectivity even in the presence of imperfect communication scenarios the estimation strategy is then used as a basic tool to adapt the power transmitted by each node of a wireless network in order to maximize the network connectivity in the presence of realistic medium access control mac protocols or simply to drive the connectivity toward a desired target value numerical results corroborate our theoretical findings thus illustrating the main features of the algorithm and its robustness to fluctuations of the network graph due to the presence of random link failures

approaches to the representations and logic operations of fuzzy concepts in the framework of axiomatic fuzzy set theory ii in this paper the representations of fuzzy concepts based on raw data have been investigated within the framework of afs axiomatic fuzzy set theory first a brief review of afs theory is presented and a completely distributive lattice the e i algebra is proposed secondly two kinds of e i algebra representations of fuzzy concepts are derived in detail in order to represent the membership functions of fuzzy concepts in the interval 0 1 the norm of afs algebra is defined and studied finally the relationships of various representations with their advantages and drawbacks are analyzed

the topology of afs structure and afs algebras abstract in this paper we will study the topological structure of the afs structures published by the author discuss the topological molecular lattice structures on ei ei eii eii algebras published by the author and give the relations of these topological structures as applications we study the topological produced by a family of fuzzy concepts in the afs algebras and apply these to analyze relations among fuzzy concepts using these we believe that we can study the law of human thinking the most important fact is that all these can be operated by computers we are also sure afs theory will play an important role in information processing

on stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix abstract in applications of signal processing and pattern recognition eigenvectors and eigenvalues of the statistical mean of a random matrix sequence are needed iterative methods are suggested and analyzed in which no sample moments are used convergence is shown by stochastic approximation theory

self stabilized gradient algorithms for blind source separation with orthogonality constraints developments in self stabilized algorithms for gradient adaptation of orthonormal matrices have resulted in simple but powerful principal and minor subspace analysis methods we extend these ideas to develop algorithms for instantaneous prewhitened blind separation of homogeneous signal mixtures our algorithms are proven to be self stabilizing to the stiefel manifold of orthonormal matrices such that the rows of the adaptive demixing matrix do not need to be periodically reorthonormalized several algorithm forms are developed including those that are equivariant with respect to the prewhitened mixing matrix simulations verify the excellent numerical properties of the proposed methods for the blind source separation task

online kernel principal component analysis a reduced order model kernel principal component analysis kernel pca is an elegant nonlinear extension of one of the most used data analysis and dimensionality reduction techniques the principal component analysis in this paper we propose an online algorithm for kernel pca to this end we examine a kernel based version of oja s rule initially put forward to extract a linear principal axe as with most kernel based machines the model order equals the number of available observations to provide an online scheme we propose to control the model order we discuss theoretical results such as an upper bound on the error of approximating the principal functions with the reduced order model we derive a recursive algorithm to discover the first principal axis and extend it to multiple axes experimental results demonstrate the effectiveness of the proposed approach both on synthetic data set and on images of handwritten digits with comparison to classical kernel pca and iterative kernel pca

the mca exin neuron for the minor component analysis the minor component analysis mca deals with the recovery of the eigenvector associated to the smallest eigenvalue of the autocorrelation matrix of the input data and is a very important tool for signal processing and data analysis it is almost exclusively solved by linear neurons this paper presents a linear neuron endowed with a novel learning law called mca exinn and analyzes its features the neural literature about mca is very poor in the sense that both a little theoretical basis is given almost always focusing on the ode asymptotic approximation and only experiments on toy problems at most four dimensional problems are presented without any numerical analysis this work addresses these problems and lays sound theoretical foundations for the neural mca theory in particular it classifies the mca neurons according to the riemannian metric and justifies from the analysis of the degeneracy of the error cost the different behavior in approaching convergence the cost landscape is studied and used as a basis for the analysis of the asymptotic behavior all the phases of the dynamics of the mca algorithms are investigated in detail and together with the numerical analysis lead to the identification of three possible kinds of divergence here called sudden dynamic and numerical the importance of the choice of low initial conditions is also explained a lot of importance is given to the experimental part where simulations on high dimensional problems are presented and analyzed the orthogonal regression or total least squares tls technique is also presented together with a real world application on the identification of the parameters of an electrical machine it can be concluded that mca exin is the best mca neuron in terms of stability no finite time divergence speed and accuracy

concept analysis via rough set and afs algebra formal concept analysis fca was originally proposed by wille 1982 which is an important theory for data analysis and knowledge discovery afs axiomatic fuzzy set algebra was proposed by liu x liu the fuzzy theory based on afs algebras and afs structure journal of mathematical analysis and applications 217 1998 459 478 x liu the topology on afs algebra and afs structure journal of mathematical analysis and applications 217 1998 479 489 which is a semantic methodology relating to the fuzzy theory combining above two theories we propose afs formal concept which can be viewed as the generalization and development of monotone concept proposed by deogun and saquer 2003 moreover we show that the set of all afs formal concepts forms a complete lattice afs formal concept can be applied to represent the logic operations of queries in information retrieval furthermore we give an approach to find the afs formal concepts whose intents extents approximate any element of afs algebra by virtue of rough set theory

algorithms for accelerated convergence of adaptive pca we derive and discuss adaptive algorithms for principal component analysis pca that are shown to converge faster than the traditional pca algorithms due to oja and karhunen 1985 sanger 1989 and xu 1993 it is well known that traditional pca algorithms that are derived by using gradient descent on an objective function are slow to converge furthermore the convergence of these algorithms depends on appropriate choices of the gain sequences since online applications demand faster convergence and an automatic selection of gains we present new adaptive algorithms to solve these problems we first present an unconstrained objective function which can be minimized to obtain the principal components we derive adaptive algorithms from this objective function by using 1 gradient descent 2 steepest descent 3 conjugate direction and 4 newton raphson methods although gradient descent produces xu s lmser algorithm the steepest descent conjugate direction and newton raphson methods produce new adaptive algorithms for pca we also provide a discussion on the landscape of the objective function and present a global convergence proof of the adaptive gradient descent pca algorithm using stochastic approximation theory extensive experiments with stationary and nonstationary multidimensional gaussian sequences show faster convergence of the new algorithms over the traditional gradient descent methods we also compare the steepest descent adaptive algorithm with state of the art methods on stationary and nonstationary sequences

robust recursive least squares learning algorithm for principal component analysis a learning algorithm for the principal component analysis pca is developed based on the least square minimization the dual learning rate parameters are adjusted adaptively to make the proposed algorithm capable of fast convergence and high accuracy for extracting all principal components the proposed algorithm is robust to the error accumulation existing in the sequential pca algorithm we show that all information needed for pca can he completely represented by the unnormalized weight vector which is updated based only on the corresponding neuron input output product the updating of the normalized weight vector can be referred to as a leaky hebb s rule the convergence of the proposed algorithm is briefly analyzed we also establish the relation between oja s rule and the least squares learning rule finally the simulation results are given to illustrate the effectiveness of this algorithm for pca and tracking time varying directions of arrival