dynamical system for computing the eigenvectors associated with the largest eigenvalue of a positive definite matrix a dynamical system for computing the largest eigenpair of a given positive matrix is introduced certain qualitative properties of the proposed system are analyzed in detail the authors results suggest that the weight bounding term in pca algorithms for extracting the first principal component could assume a class of forms

batalin vilkovisky formalism in the functional approach to classical field theory we develop the batalin vilkovisky formalism for classical field theory on generic globally hyperbolic spacetimes a crucial aspect of our treatment is the incorporation of the principle of local covariance which amounts to formulate the theory without reference to a distinguished spacetime in particular this allows a homological construction of the poisson algebra of observables in classical gravity our methods heavily rely on the differential geometry of configuration spaces of classical fields

batalin vilkovisky formalism in perturbative algebraic quantum field theory on the basis of a thorough discussion of the batalin vilkovisky formalism for classical field theory presented in our previous publication we construct in this paper the batalin vilkovisky complex in perturbatively renormalized quantum field theory the crucial technical ingredient is an extended notion of the renormalized time ordered product as a binary product equivalent to the pointwise product of classical field theory originally in causal perturbation theory the time ordered product is understood merely as a sequence of multilinear maps on the space of local functionals our extended notion of the renormalized time ordered product denoted by cdot mathcal t rm r is consistent with the old one and we found a subspace of the quantum algebra which is closed with respect to cdot mathcal t rm r on this space the renormalized batalin vilkovisky algebra is then the classical algebra but written in terms of the time ordered product together with an operator which replaces the ill defined graded laplacian of the unrenormalized theory we identify it with the anomaly term of the anomalous master ward identity of brennecke and dutsch contrary to other approaches we do not refer to the path integral formalism and do not need to use regularizations in intermediate steps

fast iterative kernel principal component analysis we develop gain adaptation methods that improve convergence of the kernel hebbian algorithm kha for iterative kernel pca kim et al 2005 kha has a scalar gain parameter which is either held constant or decreased according to a predetermined annealing schedule leading to slow convergence we accelerate it by incorporating the reciprocal of the current estimated eigenvalues as part of a gain vector an additional normalization term then allows us to eliminate a tuning parameter in the annealing schedule finally we derive and apply stochastic meta descent smd gain vector adaptation schraudolph 1999 2002 in reproducing kernel hilbert space to further speed up convergence experimental results on kernel pca and spectral clustering of usps digits motion capture and image denoising and image super resolution tasks confirm that our methods converge substantially faster than conventional kha to demonstrate scalability we perform kernel pca on the entire mnist data set

general results on the convergence of stochastic algorithms a deterministic approach is proposed for proving the convergence of stochastic algorithms of the most general form under necessary conditions on the input noise and reasonable conditions on the nonnecessarily continuous mean field emphasis is placed on the case where more than one stationary point exists we also use this approach to prove the convergence of a stochastic algorithm with markovian dynamics

near optimal stochastic approximation for online principal component estimation principal component analysis pca has been a prominent tool for high dimensional data analysis online algorithms that estimate the principal component by processing streaming data are of tremendous practical and theoretical interests despite its rich applications theoretical convergence analysis remains largely open in this paper we cast online pca into a stochastic nonconvex optimization problem and we analyze the online pca algorithm as a stochastic approximation iteration the stochastic approximation iteration processes data points incrementally and maintains a running estimate of the principal component we prove for the first time a nearly optimal finite sample error bound for the online pca algorithm under the subgaussian assumption we show that the finite sample error bound closely matches the minimax information lower bound

local wick polynomials and time ordered products of quantum fields in curved spacetime in order to have well defined rules for the perturbative calculation of quantities of interest in an interacting quantum field theory in curved spacetime it is necessary to construct wick polynomials and their time ordered products for the noninteracting theory a construction of these quantities has recently been given by brunetti fredenhagen and kohler and by brunetti and fredenhagen but they did not impose any locality or covariance condition in their constructions as a consequence their construction of time ordered products contained ambiguities involving arbitrary functions of spacetime point rather than arbitrary parameters in this paper we construct an extended wick polynomial algebra large enough to contain the wick polynomials and their time ordered products by generalizing a construction of dutsch and fredenhagen to curved spacetime we then define the notion of a local covariant quantum field and seek a definition of local wick polynomials and their time ordered products as local covariant quantum fields we introduce a new notion of the scaling behavior of a local covariant quantum field and impose scaling requirements on our local wick polynomials and their time ordered products as well as certain additional requirements such as commutation relations with the free field and appropriate continuity properties under variations of the spacetime metric for a given polynomial order in powers of the field we prove that these conditions uniquely determine the local wick polynomials and their time ordered products up to a finite number of parameters these parameters correspond to the usual renormalization ambiguities occurring in minkowski spacetime together with additional parameters corresponding to the coupling of the field to curvature we also prove existence of local wick polynomials however the issue of existence of local time ordered products is deferred to a future investigation

asymptotic stability and energy decay rates for solutions of the wave equation with memory in a star shaped domain we study the asymptotic stability and give the energy decay rates for solutions of the wave equation with boundary dissipation of the memory type

a distributed framework for dimensionality reduction and denoising with the recent upsurge in the amount of data transmission and use in networks a lot of emphasis is placed on designing efficient methods to extract useful information by reducing the size of large data sets a major approach in data dimensionality reduction relies on the estimation of the principal data covariance eigenspace in this paper a distributed algorithmic framework is put forth for finding the principal eigenspace of spatially scattered sensor data toward this end the standard principal component analysis framework is reformulated as a separable constrained minimization problem which is solved by utilizing coordinate descent techniques combined with the alternating direction method of multipliers computationally simple local updating recursions are obtained that involve only single hop intersensor communications the proposed distributed algorithm is shown to be robust even in the presence of inter sensor communication noise and converge to the principal eigenspace when inter sensor links are ideal the proposed distributed framework is used in a data denoising setting while extensive numerical tests using both synthetic and real data demonstrate a fast convergence rate better steady state performance of the novel algorithm over existing alternatives and communication noise resilience

a cross associative neural network for svd of non squared data matrix in signal processing this paper proposes a cross associative neural network cann for singular value decomposition svd of a non squared data matrix in signal processing in order to improve the convergence speed and avoid the potential instability of the deterministic networks associated with the cross correlation neural network models we study the global asymptotic stability of the network for tracking all the singular components and show that the selection of its learning rate in the iterative algorithm is independent of the singular value distribution of a non squared matrix the performances of cann are shown via simulations

algebraic structure of classical field theory i kinematics and linearized dynamics for real scalar fields we describe the elements of a novel structural approach to classical field theory inspired by recent developments in perturbative algebraic quantum field theory this approach is local and focuses mainly on the observables over field configurations given by certain spaces of functionals which are studied here in depth the analysis of such functionals is characterized by a combination of geometric analytic and algebraic elements which 1 make our approach closer to quantum field theory 2 allow for a rigorous analytic refinement of many computational formulae from the functional formulation of classical field theory and 3 provide a new pathway towards understanding dynamics particular attention will be paid to aspects related to nonlinear hyperbolic partial differential equations and their linearizations

least squares type algorithm for adaptive implementation of pisarenko s harmonic retrieval method pisarenko s harmonic retrieval method involves determining the minimum eigenvalue and the corresponding eigenvector of the covariance matrix of the observed random process recently thompson 9 suggested a constrained gradient search procedure for obtaining an adaptive version of pisarenko s method and his simulations have verified that the frequency estimates provided by his procedure were unbiased however the main cost of this technique was that the initial convergence rate could be very slow for certain poor initial conditions restating the constrained minimization as an unconstrained nonlinear problem we derived an alternative gauss newton type recursive algorithm which also used the second derivative matrix or hessian this algorithm may also be viewed as an approximate least squares algorithm simulations have been performed to compare this algorithm to a slight variant of thompson s original algorithm the most important conclusions are that the least squares type algorithm has faster convergence in the beginning while its convergence rate close to the true parameters depends on the signal to noise ratio of the input signal the approximate least squares algorithm resolves the sinusoids much faster than the gradient version

algebraic quantum field theory perturbation theory and the loop expansion the perturbative treatment of quantum field theory is formulated within the framework of algebraic quantum field theory we show that the algebra of interacting fields is additive i e fully determined by its subalgebras associated to arbitrary small subregions of minkowski space we also give an algebraic formulation of the loop expansion by introducing a projective system n of observables up to n loops where 0 is the poisson algebra of the classical field theory finally we give a local algebraic formulation for two cases of the quantum action principle and compare it with the usual formulation in terms of green s functions

convergence of learning algorithms with constant learning rates the behavior of neural network learning algorithms with a small constant learning rate epsilon in stationary random input environments is investigated it is rigorously established that the sequence of weight estimates can be approximated by a certain ordinary differential equation in the sense of weak convergence of random processes as epsilon tends to zero as applications backpropagation in feedforward architectures and some feature extraction algorithms are studied in more detail

oja s algorithm for graph clustering markov spectral decomposition and risk sensitive control given a positive definite matrix m and an integer n m 1 oja s subspace algorithm will provide convergent estimates of the first n m eigenvalues of m along with the corresponding eigenvectors it is a common approach to principal component analysis this paper introduces a normalized stochastic approximation implementation of oja s subspace algorithm as well as new applications to the spectral decomposition of a reversible markov chain recall that this means that the stationary distribution satisfies the detailed balance equations meyn tweedie 2009 equivalently the statistics of the process in steady state do not change when time is reversed stability and convergence of oja s algorithm are established under conditions far milder than that assumed in previous work applications to graph clustering markov spectral decomposition and multiplicative ergodic theory are surveyed along with numerical results

anomalies in time ordered products and applications to the bv brst formulation of quantum gauge theories we show that every graded derivation on the algebra of free quantum fields and their wick powers in curved spacetimes gives rise to a set of anomalous ward identities for time ordered products with an explicit formula for their classical limit we study these identities for the koszul tate and the full brst differential in the bv brst formulation of perturbatively interacting quantum gauge theories and clarify the relation to previous results in particular we show that the quantum brst differential the quantum antibracket and the higher order anomalies form an l infty algebra the defining relations of this algebra ensure that the gauge structure is well defined on cohomology classes of the quantum brst operator i e observables furthermore we show that one can determine contact terms such that also the interacting time ordered products of multiple interacting fields are well defined on cohomology classes an important technical improvement over previous treatments is the fact that all our relations hold off shell and are independent of the concrete form of the lagrangian including the case of open gauge algebras

algorithms for nonnegative independent component analysis we consider the task of solving the independent component analysis ica problem x as given observations x with a constraint of nonnegativity of the source random vector s we refer to this as nonnegative independent component analysis and we consider methods for solving this task for independent sources with nonzero probability density function pdf p s down to s 0 it is sufficient to find the orthonormal rotation y wz of prewhitened sources z vx which minimizes the mean squared error of the reconstruction of z from the rectified version y sup of y we suggest some algorithms which perform this both based on a nonlinear principal component analysis pca approach and on a geodesic search method driven by differential geometry considerations we demonstrate the operation of these algorithms on an image separation problem which shows in particular the fast convergence of the rotation and geodesic methods and apply the approach to a musical audio analysis task

asymptotic performance analysis of subspace adaptive algorithms introduced in the neural network literature in the neural network literature many algorithms have been proposed for estimating the eigenstructure of covariance matrices we first show that many of these algorithms when presented in a common framework show great similitudes with the gradient like stochastic algorithms usually encountered in the signal processing literature we derive the asymptotic distribution of these different recursive subspace estimators a closed form expression of the covariances in distribution of eigenvectors and associated projection matrix estimators are given and analyzed in particular closed form expressions of the mean square error of these estimators are given it is found that these covariance matrices have a structure very similar to those describing batch estimation techniques the accuracy of our asymptotic analysis is checked by numerical simulations and it is found to be valid not only for a small step size but in a very large domain finally convergence speed and deviation from orthonormality of the different algorithms are compared and several tradeoffs are analyzed

global convergence of oja s subspace algorithm for principal component extraction oja s principal subspace algorithm is a well known and powerful technique for learning and tracking principal information in time series a thorough investigation of the convergence property of oja s algorithm is undertaken in this paper the asymptotic convergence rates of the algorithm is discovered the dependence of the algorithm on its initial weight matrix and the singularity of the data covariance matrix is comprehensively addressed

some pathological traps for stochastic approximation we consider different kinds of pathological traps for stochastic algorithms thus extending a previous study on regular traps an illustration is given by the complete proof of the convergence of a principal component analysis pca algorithm when the eigenvalues are multiple

performance analysis of adaptive eigenanalysis algorithms we present a rigorous analysis of several popular forms of short memory adaptive eigenanalysis algorithms using a stochastic averaging method a first order analysis shows that the algorithms do not have any equilibrium points despite published claims to the contrary through averaging analysis we show that they hover around an appropriate eigenvector a second order analysis is also given without the gaussian noise assumption and our results greatly outperform an earlier approximation in the literature the second order analysis has been of much interest in the offline study but in the dynamic adaptive case is uncommon

the master ward identity and generalized schwinger dyson equation in classical field theory in the framework of perturbative quantum field theory a new universal renormalization condition called master ward identity was recently proposed by one of us m d in a joint paper with f m boas the main aim of the present paper is to get a better understanding of the master ward identity by analyzing its meaning in classical field theory it turns out that it is the most general identity for classical local fields which follows from the field equations it is equivalent to a generalization of the schwinger dyson equation and is closely related to the quantum action principle of lowenstein and lam as a byproduct we give a self contained treatment of peierls manifestly covariant definition of the poisson bracket

a note on linear function approximation using random projections linear function approximations based on random projections are proposed and justified for a class of fixed point and minimization problems

on self organizing algorithms and networks for class separability features we describe self organizing learning algorithms and associated neural networks to extract features that are effective for preserving class separability as a first step an adaptive algorithm for the computation of q sup 1 2 where q is the correlation or covariance matrix of a random vector sequence is described convergence of this algorithm with probability one is proven by using stochastic approximation theory and a single layer linear network architecture for this algorithm is described which we call the q sup 1 2 network using this network we describe feature extraction architectures for 1 unimodal and multicluster gaussian data in the multiclass case 2 multivariate linear discriminant analysis lda in the multiclass case and 3 bhattacharyya distance measure for the two class case the lda and bhattacharyya distance features are extracted by concatenating the q sup 1 2 network with a principal component analysis network and the two layer network is proven to converge with probability one every network discussed in the study considers a flow or sequence of inputs for training numerical studies on the performance of the networks for multiclass random data are presented

a class of learning algorithms for principal component analysis and minor component analysis principal component analysis pca and minor component analysis mca are a powerful methodology for a wide variety of applications such as pattern recognition and signal processing in this paper we first propose a differential equation for the generalized eigenvalue problem we prove that the stable points of this differential equation are the eigenvectors corresponding to the largest eigenvalue based on this generalized differential equation a class of pca and mca learning algorithms can be obtained we demonstrate that many existing pca and mca learning algorithms are special cases of this class and this class includes some new and simpler mca learning algorithms our results show that all the learning algorithms of this class have the same order of convergence speed and they are robust to implementation error

the dynamics of on line principal component analysis the learning dynamics of an on line algorithm for principal component analysis is described exactly in the thermodynamic limit by means of coupled ordinary differential equations for a set of order parameters it is demonstrated that learning is delayed significantly because existing symmetries among student vectors have to be broken a closely related effect is the perfect or partial loss of initial knowledge in the course of learning the analysis shows that different learning rates for the student vectors improve the performance of the algorithm drastically

distributed estimation and control of algebraic connectivity over random graphs in this paper we propose a distributed algorithm for the estimation and control of the connectivity of ad hoc networks in the presence of a random topology first given a generic random graph we introduce a novel stochastic power iteration method that allows each node to estimate and track the algebraic connectivity of the underlying expected graph using results from stochastic approximation theory we prove that the proposed method converges almost surely a s to the desired value of connectivity even in the presence of imperfect communication scenarios the estimation strategy is then used as a basic tool to adapt the power transmitted by each node of a wireless network in order to maximize the network connectivity in the presence of realistic medium access control mac protocols or simply to drive the connectivity toward a desired target value numerical results corroborate our theoretical findings thus illustrating the main features of the algorithm and its robustness to fluctuations of the network graph due to the presence of random link failures

on stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix abstract in applications of signal processing and pattern recognition eigenvectors and eigenvalues of the statistical mean of a random matrix sequence are needed iterative methods are suggested and analyzed in which no sample moments are used convergence is shown by stochastic approximation theory

online kernel principal component analysis a reduced order model kernel principal component analysis kernel pca is an elegant nonlinear extension of one of the most used data analysis and dimensionality reduction techniques the principal component analysis in this paper we propose an online algorithm for kernel pca to this end we examine a kernel based version of oja s rule initially put forward to extract a linear principal axe as with most kernel based machines the model order equals the number of available observations to provide an online scheme we propose to control the model order we discuss theoretical results such as an upper bound on the error of approximating the principal functions with the reduced order model we derive a recursive algorithm to discover the first principal axis and extend it to multiple axes experimental results demonstrate the effectiveness of the proposed approach both on synthetic data set and on images of handwritten digits with comparison to classical kernel pca and iterative kernel pca

microlocal analysis and interacting quantum field theories renormalization on physical backgrounds we present a perturbative construction of interacting quantum field theories on smooth globally hyperbolic curved space times we develop a purely local version of the stuckelberg bogoliubov epstein glaser method of renormalization by using techniques from microlocal analysis relying on recent results of radzikowski kohler and the authors about a formulation of a local spectrum condition in terms of wave front sets of correlation functions of quantum fields on curved space times we construct time ordered operator valued products of wick polynomials of free fields they serve as building blocks for a local perturbative definition of interacting fields renormalization in this framework amounts to extensions of expectation values of time ordered products to all points of space time the extensions are classified according to a microlocal generalization of steinmann scaling degree corresponding to the degree of divergence in other renormalization schemes as a result we prove that the usual perturbative classification of interacting quantum field theories holds also on curved space times finite renormalizations are deferred to a subsequent paper as byproducts we describe a perturbative construction of local algebras of observables present a new definition of wick polynomials as operator valued distributions on a natural domain and we find a general method for the extension of distributions which were defined on the complement of some surface

relating nets and factorization algebras of observables free field theories in this paper we relate two mathematical frameworks that make perturbative quantum field theory rigorous perturbative algebraic quantum field theory paqft and the factorization algebras framework developed by costello and gwilliam to make the comparison as explicit as possible we use the free scalar field as our running example while giving proofs that apply to any field theory whose equations of motion are green hyperbolic which includes for instance free fermions the main claim is that for such free theories there is a natural transformation intertwining the two constructions in fact both approaches encode equivalent information if one assumes the time slice axiom the key technical ingredient is to use time ordered products as an intermediate step between a net of associative algebras and a factorization algebra

robust recursive least squares learning algorithm for principal component analysis a learning algorithm for the principal component analysis pca is developed based on the least square minimization the dual learning rate parameters are adjusted adaptively to make the proposed algorithm capable of fast convergence and high accuracy for extracting all principal components the proposed algorithm is robust to the error accumulation existing in the sequential pca algorithm we show that all information needed for pca can he completely represented by the unnormalized weight vector which is updated based only on the corresponding neuron input output product the updating of the normalized weight vector can be referred to as a leaky hebb s rule the convergence of the proposed algorithm is briefly analyzed we also establish the relation between oja s rule and the least squares learning rule finally the simulation results are given to illustrate the effectiveness of this algorithm for pca and tracking time varying directions of arrival

all order bounds for correlation functions of gauge invariant operators in yang mills theory we give a complete self contained and mathematically rigorous proof that euclidean yang mills theories are perturbatively renormalisable in the sense that all correlation functions of arbitrary composite local operators fulfill suitable ward identities our proof treats rigorously both all ultraviolet and infrared problems of the theory and provides in the end detailed analytical bounds on the correlation functions of an arbitrary number of composite local operators these bounds are formulated in terms of certain weighted spanning trees extending between the insertion points of these operators our proofs are obtained within the framework of the wilson wegner polchinski wetterich renormalisation group flow equations combined with estimation techniques based on tree structures compared with previous mathematical treatments of massless theories without local gauge invariance r guida and ch kopper e print arxiv 1103 5692 and j holland s hollands and ch kopper commun math phys 342 385 2