a vectorization optimization method based type 2 fuzzy neural network for noisy data classification this paper proposes a vectorization optimization method vom based type 2 fuzzy neural network vom2fnn for noisy data classification in handling problems with uncertainties such as noisy data type 2 fuzzy systems usually outperform their type 1 counterparts hence type 2 fuzzy sets are adopted in the antecedent parts to model the uncertainty to consider the classification problems the discriminative capability is crucial to determine the performance therefore a vom is proposed in the consequent parts to increase the discriminability and reduce the parameters compared with other existing fuzzy neural networks the novelty of the proposed vom2fnn is its consideration of both uncertainty and discriminability the effectiveness of the proposed vom2fnn is demonstrated by three classification problems experimental results and theoretical analysis indicate that the proposed vom2fnn performs better than the other fuzzy neural networks

dynamical system for computing the eigenvectors associated with the largest eigenvalue of a positive definite matrix a dynamical system for computing the largest eigenpair of a given positive matrix is introduced certain qualitative properties of the proposed system are analyzed in detail the authors results suggest that the weight bounding term in pca algorithms for extracting the first principal component could assume a class of forms

optimal linear compression under unreliable representation and robust pca neural models in a typical linear data compression system the representation variables resulting from the coding operation are assumed totally reliable and therefore the solution in the mean squared error sense is an orthogonal projector to the so called principal component subspace when the representation variables are contaminated by additive noise which is uncorrelated with the signal the problem is called noisy principal component analysis npca and the optimal mse solution is not a trivial extension of pca we show that the problem is not well defined unless we impose explicit or implicit constraints on either the coding or the decoding operator orthogonality is not a property of the optimal solution under most constraints and the signal components may or may not be reconstructed depending on the noise level as the noise power increases we observe rank reduction in the optimal solution under most reasonable constraints in these cases it appears that it is preferable to omit the smaller signal components rather than attempting to reconstruct them finally we show that standard hebbian type pca learning algorithms are not optimally robust to noise and propose a new hebbian type learning algorithm which is optimally robust in the npca sense

on the unification of line processes outlier rejection and robust statistics with applications in early vision the modeling of spatial discontinuities for problems such as surface recovery segmentation image reconstruction and optical flow has been intensely studied in computer vision while line process models of discontinuities have received a great deal of attention there has been recent interest in the use of robust statistical techniques to account for discontinuities this paper unifies the two approaches to achieve this we generalize the notion of a line process to that of an analog outlier process and show how a problem formulated in terms of outlier processes can be viewed in terms of robust statistics we also characterize a class of robust statistical problems for which an equivalent outlier process formulation exists and give a straightforward method for converting a robust estimation problem into an outlier process formulation we show how prior assumptions about the spatial structure of outliers can be expressed as constraints on the recovered analog outlier processes and how traditional continuation methods can be extended to the explicit outlier process formulation these results indicate that the outlier process approach provides a general framework which subsumes the traditional line process approaches as well as a wide class of robust estimation problems examples in surface reconstruction image segmentation and optical flow are presented to illustrate the use of outlier processes and to show how the relationship between outlier processes and robust statistics can be exploited an appendix provides a catalog of common robust error norms and their equivalent outlier process formulations

global convergence of oja s pca learning algorithm with a non zero approaching adaptive learning rate a non zero approaching adaptive learning rate is proposed to guarantee the global convergence of oja s principal component analysis pca learning algorithm most of the existing adaptive learning rates for oja s pca learning algorithm are required to approach zero as the learning step increases however this is not practical in many applications due to the computational round off limitations and tracking requirements the proposed adaptive learning rate overcomes this shortcoming the learning rate converges to a positive constant thus it increases the evolution rate as the learning step increases this is different from learning rates which approach zero which slow the convergence considerably and increasingly with time rigorous mathematical proofs for global convergence of oja s algorithm with the proposed learning rate are given in detail via studying the convergence of an equivalent deterministic discrete time ddt system extensive simulations are carried out to illustrate and verify the theory derived simulation results show that this adaptive learning rate is more suitable for oja s pca algorithm to be used in an online learning situation

analytic study of the effect of persistence on a one dimensional biased random walk abstract an analytic study of a one dimensional biased random walk with correlations between nearest neighbour steps is presented both in a lattice model and in its continuous version first the treatment of the unbiased problem is recalled and the effect of correlations on the diffusion coefficient is discussed then the study is extended to the biased case the problem is then completely determined by two independent parameters the degree of correlations in the motion on the one hand and the value of the bias on the other both the velocity of the particle and its diffusion coefficient are computed as a result the velocity as well as the diffusion coefficient are enhanced when there are positive correlations qualified as persistence in the motion and reduced in the opposite case

the influence function of principal component analysis by self organizing rule this article is concerned with a neural network approach to principal component analysis pca an algorithm for pca by the self organizing rule has been proposed and its robustness observed through the simulation study by xu and yuille 1995 in this article the robustness of the algorithm against outliers is investigated by using the theory of influence function the influence function of the principal component vector is given in an explicit form through this expression the method is shown to be robust against any directions orthogonal to the principal component vector in addition a statistic generated by the self organizing rule is proposed to assess the influence of data in pca

boundary problems for diffusion in a fluctuating potential abstract using a newly developed formalism adapted for diffusion in the presence of partially absorbing boundaries we treat the overdamped motion of a particle in a fluctuating potential and in a restricted space the potential is assumed to be linear with a slope switching between two possible values the switching process is taken as the standard markov alternating process we first construct the free dynamics propagator no boundaries which controls the evolution of the joint probability for both the particle s position and the state of the potential secondly at a given spatial point we affix a two channel boundary a given channel absorbs the trajectories arriving at the boundary with the potential in a given state we demonstrate the asymmetry of the filling process for the individual channels and we discuss their asymptotic occupation depending on the mean switching frequency finally we study the motion in a domain restricted by two boundaries of the above two channel type and the corresponding first passage time the simple problem setting allows for an explicit analysis of the correlation between the particle s motion and the potential switching process

energy function for the one unit oja algorithm the one unit oja algorithm plays a very important role in the study of principal component analysis neural networks in this paper we propose an energy function whose steepest descent direction i e negative gradient direction is the same as the average evolution direction of the one unit oja algorithm and the energy function has two global minimal points corresponding to the two converged points of the one unit oja algorithm and it has no other local minimal points

homogenization of elliptic boundary value problems in lipschitz domains in this paper we study the lp boundary value problems for mathcal l u 0 in mathbb r d 1 where mathcal l rm div a nabla is a second order elliptic operator with real and symmetric coefficients assume that a is periodic in xd 1 and satisfies some minimal smoothness condition in the xd 1 variable we show that the lp neumann and regularity problems are uniquely solvable for 1 p 2 we also present a new proof of dahlberg s theorem on the lp dirichlet problem for 2 p dahlberg s original unpublished proof is given in the appendix as the periodic and smoothness conditions are imposed only on the xd 1 variable these results extend directly from mathbb r d 1 to regions above lipschitz graphs consequently by localization techniques we obtain uniform lp estimates for the dirichlet neumann and regularity problems on bounded lipschitz domains for a family of second order elliptic operators arising in the theory of homogenization the results on the neumann and regularity problems are new even for smooth domains

unraveling the structure of treelike networks from first passage times of lazy random walkers we study the problem of random search in finite networks with a tree topology where it is expected that the distribution of the first passage time f t decays exponentially we show that the slope of the exponential tail is independent of the initial conditions of entering the tree in general and scales exponentially or as a power law with the extent of the tree l depending on the tendency p to jump toward the target node it is unfeasible to uniquely determine l and p from measuring the tail slope or the mean first passage time mfpt of an ordinary diffusion along the tree to unravel the structure we consider lazy random walkers that take steps with probability m when jumping on the nodes and return with probability q from the leaves by deriving an exact analytical expression for the mfpt of the intermittent random walk we verify that the structural information of the tree can be uniquely extracted by measuring the mfpt for two randomly chosen types of tracer particles with distinct experimental parameters m and q we also address the applicability of our approach in the presence of disorder in the structure of the tree or statistical uncertainty in the experimental parameters

principal component analysis of fuzzy data using autoassociative neural networks this paper describes an extension of principal component analysis pca allowing the extraction of a limited number of relevant features from high dimensional fuzzy data our approach exploits the ability of linear autoassociative neural networks to perform information compression in just the same way as pca without explicit matrix diagonalization fuzzy input values are propagated through the network using fuzzy arithmetics and the weights are adjusted to minimize a suitable error criterion the inputs being taken as target outputs the concept of correlation coefficient is extended to fuzzy numbers allowing the interpretation of the new features in terms of the original variables experiments with artificial and real sensory evaluation data demonstrate the ability of our method to provide concise representations of complex fuzzy data

memory based persistence in a counting random walk process this paper considers a memory based persistent counting random walk based on a markov memory of the last event this persistent model is a different than the weiss persistent random walk model however leading thereby to different results we point out to some preliminary result in particular we provide an explicit expression for the mean and the variance both nonlinear in time of the underlying memory based persistent process and discuss the usefulness to some problems in insurance finance and risk analysis the motivation for the paper arose from the counting of events whether rare or not in insurance that presume that events are time independent and therefore based on the poisson distribution for counting these events

on relative convergence properties of principal component analysis algorithms we investigate the convergence properties of two different stochastic approximation algorithms for principal component analysis and analytically explain some commonly observed experimental results in our analysis we use the theory of stochastic approximation and in particular the results of fabian 1968 to explore the asymptotic mean square errors amses of the algorithms this study reveals the conditions under which the algorithms produce smaller amses and also the conditions under which one algorithm has a smaller amse than the other experimental study with multidimensional gaussian data corroborate our analytical findings we next explore the convergence rates of the two algorithms our experiments and an analytical explanation reveals the conditions under which the algorithms converge faster to the solution and also the conditions under which one algorithm converges faster than the other

correlated biased random walk with latency in one and two dimensions asserting patterned and unpredictable movement the correlated biased random walk with latency in one and two dimensions is discussed with regard to the portion of irreducible random movement and structured movement it is shown how a quantitative analysis can be carried out by using computational mechanics the stochastic matrix for both dynamics are reported latency introduces new states in the finite state machine description of the system in both dimensions allowing for a full nearest neighbor coordination in the two dimensional case complexity analysis is used to characterize the movement independently of the set of control parameters making it suitable for the discussion of other random walk models the complexity map of the system dynamics is reported for the two dimensional case

principal component extraction using recursive least squares learning a new neural network based approach is introduced for recursive computation of the principal components of a stationary vector stochastic process the neurons of a single layer network are sequentially trained using a recursive least squares squares rls type algorithm to extract the principal components of the input process the optimality criterion is based on retaining the maximum information contained in the input sequence so as to be able to reconstruct the network inputs from the corresponding outputs with minimum mean squared error the proof of the convergence of the weight vectors to the principal eigenvectors is also established a simulation example is given to show the accuracy and speed advantages of this algorithm in comparison with the existing methods finally the application of this learning algorithm to image data reduction and filtering of images degraded by additive and or multiplicative noise is considered

coupled principal component analysis a framework for a class of coupled principal component learning rules is presented in coupled rules eigenvectors and eigenvalues of a covariance matrix are simultaneously estimated in coupled equations coupled rules can mitigate the stability speed problem affecting noncoupled learning rules since the convergence speed in all eigendirections of the jacobian becomes widely independent of the eigenvalues of the covariance matrix a number of coupled learning rule systems for principal component analysis two of them new is derived by applying newton s method to an information criterion the relations to other systems of this class the adaptive learning algorithm ala the robust recursive least squares algorithm rrlsa and a rule with explicit renormalization of the weight vector length are established

layer potential methods for elliptic homogenization problems in this paper we use the method of layer potentials to study l 2 boundary value problems in a bounded lipschitz domain for a family of second order elliptic systems with rapidly oscillating periodic coefficients defining l e div a e 1 x under the assumption that a x is elliptic symmetric periodic and holder continuous we establish the solvability of the l 2 dirichlet regularity and neumann problems for l e u e 0 in with optimal estimates uniform in e 0

on lp estimates in homogenization of elliptic equations of maxwell s type abstract for a family of second order elliptic systems of maxwell s type with rapidly oscillating periodic coefficients in a c 1 domain we establish uniform estimates of solutions u e and u e in l p for 1 p the proof relies on the uniform w 1 p and lipschitz estimates for solutions of scalar elliptic equations with periodic coefficients

global convergence of oja s subspace algorithm for principal component extraction oja s principal subspace algorithm is a well known and powerful technique for learning and tracking principal information in time series a thorough investigation of the convergence property of oja s algorithm is undertaken in this paper the asymptotic convergence rates of the algorithm is discovered the dependence of the algorithm on its initial weight matrix and the singularity of the data covariance matrix is comprehensively addressed

uniform regularity estimates in homogenization theory of elliptic systems with lower order terms on the neumann boundary problem abstract in this paper we mainly employed the idea of the previous paper 34 to study the sharp uniform w 1 p estimates with 1 p for more general elliptic systems with the neumann boundary condition on a bounded c 1 domain arising in homogenization theory based on the skills developed by z shen in 27 and by t suslina in 31 32 we also established the l 2 convergence rates on a bounded c 1 1 domain and a lipschitz domain respectively here we found a rough version of the first order correctors see 1 12 which can unify the proof in 27 and 32 it allows us to skip the corresponding convergence results on r d that are the preconditions in 31 32 our results can be regarded as an extension of 23 developed by c kenig f lin z shen as well as of 32 investigated by t suslina

global analysis of oja s flow for neural networks a detailed study of oja s learning equation in neural networks is undertaken in this paper not only are such fundamental issues as existence uniqueness and representation of solutions completely resolved but also the convergence issue is resolved it is shown that the solution of oja s equation is exponentially convergent to an equilibrium from any initial value moreover the necessary and sufficient conditions are given on the initial value for the solution to converge to a dominant eigenspace of the associated autocorrelation matrix as a by product this result confirms one of oja s conjectures that the solution converges to the principal eigenspace from almost all initial values some other characteristics of the limiting solution are also revealed these facilitate the determination of the limiting solution in advance using only the initial information two examples are analyzed demonstrating the explicit dependence of the limiting solution on the initial value in another respect it is found that oja s equation is the gradient flow of generalized rayleigh quotients on a stiefel manifold

robust principal component analysis by self organizing rules based on statistical physics approach this paper applies statistical physics to the problem of robust principal component analysis pca the commonly used pca learning rules are first related to energy functions these functions are generalized by adding a binary decision field with a given prior distribution so that outliers in the data are dealt with explicitly in order to make pca robust each of the generalized energy functions is then used to define a gibbs distribution from which a marginal distribution is obtained by summing over the binary decision field the marginal distribution defines an effective energy function from which self organizing rules have been developed for robust pca under the presence of outliers both the standard pca methods and the existing self organizing pca rules studied in the literature of neural networks perform quite poorly by contrast the robust rules proposed here resist outliers well and perform excellently for fulfilling various pca like tasks such as obtaining the first principal component vector the first k principal component vectors and directly finding the subspace spanned by the first k vector principal component vectors without solving for each vector individually comparative experiments have been made and the results show that the authors robust rules improve the performances of the existing pca algorithms significantly when outliers are present

convergence analysis of a deterministic discrete time system of oja s pca learning algorithm the convergence of oja s principal component analysis pca learning algorithms is a difficult topic for direct study and analysis traditionally the convergence of these algorithms is indirectly analyzed via certain deterministic continuous time dct systems such a method will require the learning rate to converge to zero which is not a reasonable requirement to impose in many practical applications recently deterministic discrete time ddt systems have been proposed instead to indirectly interpret the dynamics of the learning algorithms unlike dct systems ddt systems allow learning rates to be constant which can be a nonzero this paper will provide some important results relating to the convergence of a ddt system of oja s pca learning algorithm it has the following contributions 1 a number of invariant sets are obtained based on which we can show that any trajectory starting from a point in the invariant set will remain in the set forever thus the nondivergence of the trajectories is guaranteed 2 the convergence of the ddt system is analyzed rigorously it is proven in the paper that almost all trajectories of the system starting from points in an invariant set will converge exponentially to the unit eigenvector associated with the largest eigenvalue of the correlation matrix in addition exponential convergence rate are obtained providing useful guidelines for the selection of fast convergence learning rate 3 since the trajectories may diverge the careful choice of initial vectors is an important issue this paper suggests to use the domain of unit hyper sphere as initial vectors to guarantee convergence 4 simulation results will be furnished to illustrate the theoretical results achieved

a class of learning algorithms for principal component analysis and minor component analysis principal component analysis pca and minor component analysis mca are a powerful methodology for a wide variety of applications such as pattern recognition and signal processing in this paper we first propose a differential equation for the generalized eigenvalue problem we prove that the stable points of this differential equation are the eigenvectors corresponding to the largest eigenvalue based on this generalized differential equation a class of pca and mca learning algorithms can be obtained we demonstrate that many existing pca and mca learning algorithms are special cases of this class and this class includes some new and simpler mca learning algorithms our results show that all the learning algorithms of this class have the same order of convergence speed and they are robust to implementation error

robust recursive least squares learning algorithm for principal component analysis a learning algorithm for the principal component analysis pca is developed based on the least square minimization the dual learning rate parameters are adjusted adaptively to make the proposed algorithm capable of fast convergence and high accuracy for extracting all principal components the proposed algorithm is robust to the error accumulation existing in the sequential pca algorithm we show that all information needed for pca can he completely represented by the unnormalized weight vector which is updated based only on the corresponding neuron input output product the updating of the normalized weight vector can be referred to as a leaky hebb s rule the convergence of the proposed algorithm is briefly analyzed we also establish the relation between oja s rule and the least squares learning rule finally the simulation results are given to illustrate the effectiveness of this algorithm for pca and tracking time varying directions of arrival

cattaneo type subdiffusion reaction equation subdiffusion in a system in which mobile particles a can chemically react with static particles b according to the rule a b b is considered within a persistent random walk model this model which assumes a correlation between successive steps of particles provides hyperbolic cattaneo normal diffusion or fractional subdiffusion equations starting with the difference equation which describes a persistent random walk in a system with chemical reactions using the generating function method and the continuous time random walk formalism we will derive the cattaneo type subdiffusion differential equation with fractional time derivatives in which the chemical reactions mentioned above are taken into account we will also find its solution over a long time limit based on the obtained results we will find the cattaneo type subdiffusion reaction equation in the case in which mobile particles of species a and b can chemically react according to a more complicated rule

on stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix abstract in applications of signal processing and pattern recognition eigenvectors and eigenvalues of the statistical mean of a random matrix sequence are needed iterative methods are suggested and analyzed in which no sample moments are used convergence is shown by stochastic approximation theory

a family of fuzzy learning algorithms for robust principal component analysis neural networks in this paper we analyze xu and yuille s robust principal component analysis rpca learning algorithms by means of the distance measurement in space based on the analysis a family of fuzzy rpca learning algorithms is proposed which is robust against outliers these algorithms can explicitly be understood from the viewpoint of fuzzy set theory though xu and yuille s algorithms were proposed based on a statistical physics approach in the proposed algorithms an adaptive learning procedure overcomes the difficulty of selection of learning parameters in xu and yuille s algorithms furthermore the robustness of proposed algorithms is investigated by using the theory of influence functions simulations are carried out to illustrate the robustness of these algorithms

against the convergence of the minor component analysis neurons we investigate the dynamical behavior of some linear neurons devised for the minor component analysis contrary to their convergence theorems our analysis reveals the existence of a divergence in a finite time

the mca exin neuron for the minor component analysis the minor component analysis mca deals with the recovery of the eigenvector associated to the smallest eigenvalue of the autocorrelation matrix of the input data and is a very important tool for signal processing and data analysis it is almost exclusively solved by linear neurons this paper presents a linear neuron endowed with a novel learning law called mca exinn and analyzes its features the neural literature about mca is very poor in the sense that both a little theoretical basis is given almost always focusing on the ode asymptotic approximation and only experiments on toy problems at most four dimensional problems are presented without any numerical analysis this work addresses these problems and lays sound theoretical foundations for the neural mca theory in particular it classifies the mca neurons according to the riemannian metric and justifies from the analysis of the degeneracy of the error cost the different behavior in approaching convergence the cost landscape is studied and used as a basis for the analysis of the asymptotic behavior all the phases of the dynamics of the mca algorithms are investigated in detail and together with the numerical analysis lead to the identification of three possible kinds of divergence here called sudden dynamic and numerical the importance of the choice of low initial conditions is also explained a lot of importance is given to the experimental part where simulations on high dimensional problems are presented and analyzed the orthogonal regression or total least squares tls technique is also presented together with a real world application on the identification of the parameters of an electrical machine it can be concluded that mca exin is the best mca neuron in terms of stability no finite time divergence speed and accuracy

algorithms for accelerated convergence of adaptive pca we derive and discuss adaptive algorithms for principal component analysis pca that are shown to converge faster than the traditional pca algorithms due to oja and karhunen 1985 sanger 1989 and xu 1993 it is well known that traditional pca algorithms that are derived by using gradient descent on an objective function are slow to converge furthermore the convergence of these algorithms depends on appropriate choices of the gain sequences since online applications demand faster convergence and an automatic selection of gains we present new adaptive algorithms to solve these problems we first present an unconstrained objective function which can be minimized to obtain the principal components we derive adaptive algorithms from this objective function by using 1 gradient descent 2 steepest descent 3 conjugate direction and 4 newton raphson methods although gradient descent produces xu s lmser algorithm the steepest descent conjugate direction and newton raphson methods produce new adaptive algorithms for pca we also provide a discussion on the landscape of the objective function and present a global convergence proof of the adaptive gradient descent pca algorithm using stochastic approximation theory extensive experiments with stationary and nonstationary multidimensional gaussian sequences show faster convergence of the new algorithms over the traditional gradient descent methods we also compare the steepest descent adaptive algorithm with state of the art methods on stationary and nonstationary sequences

influence in principal components analysis summary in linear regression the theoretical influence function and the various sample versions of it have an established place as diagnostic tools these same functions are developed here to provide methods for the detection of influential observations in principal components analysis the perturbation theory of real symmetric matrices unifies this development some interesting points of contrast with the regression case are noted and explained theoretically

boundary layers on sobolev besov spaces and poisson s equation for the laplacian in lipschitz domains we study inhomogeneous boundary value problems for the laplacian in arbitrary lipschitz domains with data in sobolev besov spaces as such this is a natural continuation of work in jerison and kenig j funct anal 1995 16 219 where the inhomogeneous dirichlet problem is treated via harmonic measure techniques the novelty of our approach resides in the systematic use of boundary integral methods in this regard the key results are establishing the invertibility of the classical layer potential operators on scales of sobolev besov spaces on lipschitz boundaries for optimal ranges of indices applications tolp based helmholtz type decompositions of vector fields in lipschitz domains are also presented

generalized master equations and the telegrapher s equation abstract the possibility of using generalized telegrapher s equations gtes to describe the time evolution of probability densities of non markovian stochastic processes is consedered the solutions of hyperbolic equations of the type of gte being generally not positive definite we derive conditions for which this global property if fulfilled

regularized linear fuzzy clustering and probabilistic pca mixture models fuzzy c means fcm type fuzzy clustering approaches are closely related to gaussian mixture models gmms and em like algorithms have been used in fcm clustering with regularized objective functions especially fcm with regularization by kullback leibler information klfcm is a fuzzy counterpart of gmms in this paper we propose to apply probabilistic principal component analysis pca mixture models to linear clustering following a discussion on the relationship between local pca and linear fuzzy clustering although the proposed method is a kind of the constrained model of klfcm the algorithm includes the fuzzy c varieties fcv algorithm as a special case and the algorithm can be regarded as a modified fcv algorithm with regularization by k l information numerical experiments demonstrate that the proposed clustering algorithm is more flexible than the maximum likelihood approaches and is useful for capturing local substructures properly