matrix momentum for practical natural gradient learning an on line learning rule based on the introduction of a matrix momentum term is presented aimed at alleviating the computational costs of standard natural gradient learning the new rule natural gradient matrix momentum is analysed in the case of two layer feed forward neural network learning via methods of statistical physics it appears to provide a practical algorithm that performs as well as standard natural gradient descent in both the transient and asymptotic regimes but with a hugely reduced complexity

gradient systems in view of information geometry abstract dualistic properties of a gradient flow on a manifold m associated with a dualistic structure g is studied from an information geometrical viewpoint some useful applications are also investigated

formulation and integration of learning differential equations on the stiefel manifold this letter aims at illustrating the relevance of numerical integration of learning differential equations on differential manifolds in particular the task of learning with orthonormality constraints is dealt with which is naturally formulated as an optimization task with the compact stiefel manifold as neural parameter space intrinsic properties of the derived learning algorithms such as stability and constraints preservation are illustrated through experiments on minor and independent component analysis ica

generalized polar coordinates on lie groups and numerical integrators motivated by developments in numerical lie group integrators we introduce a family of local coordinates on lie groups denoted generalized polar coordinates fast algorithms are derived for the computation of the coordinate maps their tangent maps and the inverse tangent maps in particular we discuss algorithms for all the classical matrix lie groups and optimal complexity integrators for n spheres

approximating the exponential from a lie algebra to a lie group consider a differential equation y a t y y y 0 y0 with y 0 g and a r g g where g is a lie algebra of the matricial lie group g every b g can be mapped to g by the matrix exponential map exp tb with t r most numerical methods for solving ordinary differential equations odes on lie groups are based on the idea of representing the approximation y n of the exact solution y t n t n r by means of exact exponentials of suitable elements of the lie algebra applied to the initial value y 0 this ensures that y n g when the exponential is difficult to compute exactly as is the case when the dimension is large an approximation of exp tb plays an important role in the numerical solution of odes on lie groups in some cases rational or polynomial approximants are unsuitable and we consider alternative techniques whereby exp tb is approximated by a product of simpler exponentials in this paper we present some ideas based on the use of the strang splitting for the approximation of matrix exponentials several cases of g and g are considered in tandem with general theory order conditions are discussed and a number of numerical experiments conclude the paper

spectral and structural analysis of high precision finite difference matrices for elliptic operators abstract in this paper we study the structural properties of matrices coming from high precision finite difference fd formulae when discretizing elliptic or semielliptic differential operators l a u of the form k d k d x k a x d k d x k u x strong relationships with toeplitz structures and linear positive operators lpo are highlighted these results allow one to give a detailed analysis of the eigenvalues localisation distribution of the arising matrices the obtained spectral analysis is then used to define optimal toeplitz preconditioners in a very compact and natural way and in addition to prove szego like and widom like ergodic theorems for the spectra of the related preconditioned matrices a wide numerical experimentation confirming the theoretical results is also reported

spectral behavior of matrix sequences and discretized boundary value problems abstract in this paper we provide theoretical tools for dealing with the spectral properties of general sequences of matrices of increasing dimension more specifically we give a unified treatment of notions such as distribution equal distribution localization equal localization clustering and sub clustering as a case study we consider the matrix sequences arising from the finite difference fd discretization of elliptic and semielliptic boundary value problems bvps the spectral analysis is then extended to toeplitz based preconditioned matrix sequences with special attention to the case where the coefficients of the differential operator are not regular belong to l 1 and to the case of multidimensional problems the related clustering properties allow the establishment of some ergodic formulas for the eigenvalues of the preconditioned matrices

blind separation of a mixture of uniformly distributed source signals a novel approach a new efficient algorithm for blind separation of uniformly distributed sources is proposed the mixing matrix is assumed to be orthogonal by prewhitening the observed signals the learning rule adaptively estimates the mixing matrix by conceptually rotating a unit hypercube so that all output signal components are contained within or on the hypercube under some ideal constraints it has been theoretically shown that the algorithm is very similar to an ideal convergent algorithm which is much faster than the existing convergent algorithms the algorithm has been generalized to take care of the noisy signals by adaptively dilating the hypercube in conjunction with its rotation

neural learning by geometric integration of reduced rigid body equations in previous contributions the second author of this paper presented a new class of algorithms for orthonormal learning of linear neural networks with p inputs and m outputs based on the equations describing the dynamics of a massive rigid frame on the stiefel manifold these algorithms exhibit good numerical stability strongly binding to the sub manifold of constraints and good controllability of the learning dynamics but are not completely satisfactory from a computational complexity point of view in the recent literature efficient methods of integration on the stiefel manifold have been proposed by various authors see for example phys d 156 2001 219 numer algorithms 32 2003 163 j numer anal 21 2001 463 numer math 83 1999 599 inspired by these approaches in this paper we propose a new and efficient representation of the mentioned learning equations and a new way to integrate them the numerical experiments show how the new formulation leads to significant computational savings especially when p m the effectiveness of the algorithms is substantiated by numerical experiments concerning principal subspace analysis and independent component analysis these experiments were carried out with both synthetic and real world data

computation of orthonormal factors for fundamental solution matrices in this work we introduce and analyze two new techniques for obtaining the q factor in the qr factorization of some or all columns of a fundamental solution matrix y of a linear differential system these techniques are based on elementary householder and givens transformations we implement and compare these new techniques with existing approaches on some examples

multigrid methods for multilevel circulant matrices we introduce a multigrid technique for the solution of multilevel circulant linear systems whose coefficient matrix has eigenvalues of the form f x j n where f is continuous and independent of n n 1 ldots n d and x j n equiv 2 pi j n 2 pi j 1 n 1 ldots 2 pi j d n d 0 le j r le n r 1 the interest of the proposed technique pertains to the multilevel banded case where the total cost is optimal i e o n arithmetic operations ops n prod r 1 d n r instead of o n log n ops arising from the use of ffts in fact multilevel banded circulants are used as preconditioners for elliptic and parabolic pdes with dirichlet or periodic boundary conditions and for some two dimensional image restoration problems where the point spread function psf is numerically banded so that the overall cost is reduced from o k varepsilon n n log n to o k varepsilon n n where k varepsilon n is the number of pcg iterations to reach the solution within an accuracy of varepsilon several numerical experiments concerning one rank regularized circulant discretization of elliptic 2q differential operators over one dimensional and two dimensional square domains with mixed boundary conditions are performed and discussed

distribution results on the algebra generated by toeplitz sequences a finite dimensional approach abstract let a be an l 1 symbol defined on q d q with d 1 and let us consider the multi indexed sequence of toeplitz matrices t n a with n n d it is well known that t n a is spectrally distributed as a in the sense of the singular values in this paper we prove that a sequence as t n a is spectrally distributed in the sense of the singular values as the measurable function a for a l 1 and and ranging in any finite set of values

exponentiated gradient versus gradient descent for linear predictors we consider two algorithm for on line prediction based on a linear model the algorithms are the well known gradient descent gd algorithm and a new algorithm which we call eg they both maintain a weight vector using simple updates for the gd algorithm the update is based on subtracting the gradient of the squared error made on a prediction the eg algorithm uses the components of the gradient in the exponents of factors that are used in updating the weight vector multiplicatively we present worst case loss bounds for eg and compare them to previously known bounds for the gd algorithm the bounds suggest that the losses of the algorithms are in general incomparable but eg has a much smaller loss if only a few components of the input are relevant for the predictions we have performed experiments which show that our worst case upper bounds are quite tight already on simple artificial data

on line versus off line learning from random examples general results i propose a general model of on line learning from random examples which when applied to a smooth realizable stochastic rule yields the same asymptotic generalization error rate as optimal batch algorithms the approach is based on an iterative gaussian approximation to the posterior gibbs distribution of rule parameters

principal component analysis in linear systems controllability observability and model reduction kalman s minimal realization theory involves geometric objects controllable unobservable subspaces which are subject to structural instability specifically arbitrarily small perturbations in a model may cause a change in the dimensions of the associated subspaces this situation is manifested in computational difficulties which arise in attempts to apply textbook algorithms for computing a minimal realization structural instability associated with geometric theories is not unique to control it arises in the theory of linear equations as well in this setting the computational problems have been studied for decades and excellent tools have been developed for coping with the situation one of the main goals of this paper is to call attention to principal component analysis hotelling 1933 and an algorithm golub and reinsch 1970 for computing the singular value decompositon of a matrix together they form a powerful tool for coping with structural instability in dynamic systems as developed in this paper principal component analysis is a technique for analyzing signals singular value decomposition provides the computational machinery for this reason kalman s minimal realization theory is recast in terms of responses to injected signals application of the signal analysis to controllability and observability leads to a coordinate system in which the internally balanced model has special properties for asymptotically stable systems this yields working approximations of x c x bar o the controllable and unobservable subspaces it is proposed that a natural first step in model reduction is to apply the mechanics of minimal realization using these working subspaces

an introduction to lie group integrators basics new developments and applications we give a short and elementary introduction to lie group methods a selection of applications of lie group integrators are discussed finally a family of symplectic integrators on cotangent bundles of lie groups is presented and the notion of discrete gradient methods is generalised to lie groups

riemannian gradient based learning on the complex matrix hypersphere this brief tackles the problem of learning over the complex valued matrix hypersphere sn p c the developed learning theory is formulated in terms of riemannian gradient based optimization of a regular criterion function and is implemented by a geodesic stepping method the stepping method is equipped with a geodesic search sub algorithm to compute the optimal learning stepsize at any step numerical results show the effectiveness of the developed learning method and of its implementation

a unifying information theoretic framework for independent component analysis abstract we show that different theories recently proposed for independent component analysis ica lead to the same iterative learning algorithm for blind separation of mixed independent sources we review those theories and suggest that information theory can be used to unify several lines of research pearlmutter and parra 1 and cardoso 2 showed that the infomax approach of bell and sejnowski 3 and the maximum likelihood estimation approach are equivalent we show that negentropy maximization also has equivalent properties and therefore all three approaches yield the same learning rule for a fixed nonlinearity girolami and fyfe 4 have shown that the nonlinear principal component analysis pca algorithm of karhunen and joutsensalo 5 and oja 6 can also be viewed from information theoretic principles since it minimizes the sum of squares of the fourth order marginal cumulants and therefore approximately minimizes the mutual information 7 lambert 8 has proposed different bussgang cost functions for multichannel blind deconvolution we show how the bussgang property relates to the infomax principle finally we discuss convergence and stability as well as future research issues in blind source separation

a theory for learning by weight flow on stiefel grassman manifold recently we introduced the concept of neural network learning on stiefel grassman manifold for multilayer perceptron like networks contributions of other authors have also appeared in the scientific literature about this topic this article presents a general theory for it and illustrates how existing theories may be explained within the general framework proposed here

dynamical systems that sort lists diagonalize matrices and solve linear programming problems abstract we establish a number of properties associated with the dynamical system h h h n where h and n are symmetric n by n matrices and a b ab ba the most important of these come from the fact that this equation is equivalent to a certain gradient flow on the space of orthogonal matrices we are especially interested in the role of this equation as an analog computer for example we show how to map the data associated with a linear programming problem into h 0 and n in such a way as to have h h h n evolve to a solution of the linear programming problem this result can be applied to find systems which solve a variety of genetic combinatorial optimization problems and it even provides an algorithm for diagonalizing symmetric matrices

the conditioning of fd matrix sequences coming from semi elliptic differential equations abstract in this paper we are concerned with the study of spectral properties of the sequence of matrices a n a coming from the discretization using centered finite differences of minimal order of elliptic or semielliptic differential operators l a u of the form 1 d d x a x d d x u x f x on 0 1 dirichlet b c on where the nonnegative bounded coefficient function a x of the differential operator may have some isolated zeros in more precisely we state and prove the explicit form of the inverse of a n a and some formulas concerning the relations between the orders of zeros of a x and the asymptotic behavior of the minimal eigenvalue condition number of the related matrices as a conclusion and in connection with our theoretical findings first we extend the analysis to higher order semi elliptic differential operators and then we present various numerical experiments showing that similar results must hold true in 2d as well

learning dynamics on different timescales the special character of certain degrees of freedom in two layered neural networks is investigated for on line learning of realizable rules our analysis shows that the dynamics of these degrees of freedom can be put on a faster timescale than those remaining with the profit of speeding up the overall adaptation process this is shown for two groups of degrees of freedom second layer weights and bias weights for the former case our analysis provides a theoretical explanation of phenomenological findings the resulting learning algorithm is compared with natural gradient descent in order to check whether the proposed scaling can be naturally derived from that type of learning rule

adaptive online learning algorithms for blind separation maximum entropy and minimum mutual information there are two major approaches for blind separation maximum entropy me and minimum mutual information mmi both can be implemented by the stochastic gradient descent method for obtaining the demixing matrix the mi is the contrast function for blind separation the entropy is not to justify the me the relation between me and mmi is first elucidated by calculating the first derivative of the entropy and proving that the mean subtraction is necessary in applying the me and at the solution points determined by the mi the me will not update the demixing matrix in the directions of increasing the cross talking second the natural gradient instead of the ordinary gradient is introduced to obtain efficient algorithms because the parameter space is a riemannian space consisting of matrices the mutual information is calculated by applying the gram charlier expansion to approximate probability density functions of the outputs finally we propose an efficient learning algorithm that incorporates with an adaptive method of estimating the unknown cumulants it is shown by computer simulation that the convergence of the stochastic descent algorithms is improved by using the natural gradient and the adaptively estimated cumulants

a general equidistribution theorem for the roots of orthogonal polynomials abstract it is well known that the roots of any two orthogonal polynomials are distributed equally if the weights satisfy the szeg condition in this paper we propose a general equidistribution theorem that does not use the szeg condition and admits an elementary proof

analysis of preconditioning strategies for collocation linear systems in the past few years several authors have studied the preconditioning of collocation matrices by finite differences fds matrices arising from the associated collocation points here we discuss how to solve in an efficient way nonuniform grid fd linear systems including those related to a generic fd collocation preconditioner the main idea is based on a further step of preconditioning defined in terms of diagonal and toeplitz matrices first we identify the limit spectral distributions of the involved fd collocation matrix sequences and then we prove that the proposed toeplitz based preconditioners assure a clustering at the unity with respect to the eigenvalues in the 1d case in the 2d case the situation is different so that more appropriate strategies are discussed a wide numerical experimentation emphasizing the correctness of the theoretical results is also reported

real time integration methods for mechanical system simulation this study focuses on the mechanical system dynamics applications of differential algebraic equations dae selected papers develop the foundations of dae as they appear in mechanical system dynamics applications and explore the foundations of contemporary methods for the numerical solution of dae in mechanical system dynamics many theoretical methods and methods of generalized co ordinate partitioning for the numerical integration of dae are examined the text has been designed to encompass a broad range of applications in engineering and applied physics

spectral analysis of nonsymmetric quasi toeplitz matrices with applications to preconditioned multistep formulas the eigenvalue spectrum of a class of nonsymmetric preconditioned matrices arising in time dependent partial differential equations is analyzed and discussed the matrices generated by the underlying numerical integrators are small rank perturbations of block toeplitz matrices circulant like preconditioners based on the former are considered the eigenvalue distribution of the preconditioned matrix influences often crucially the convergence of krylov iterative accelerators due to several reasons lack of symmetry band structure and coefficients depending on the size the classical approach based on smooth generating functions gives very little insight here therefore to characterize the eigenvalues a difference equation approach exploiting the band toeplitz and circulant patterns generalizing the well known results of trench is proposed

analysis of natural gradient descent for multilayer neural networks natural gradient descent ngd is an on line algorithm for redefining the steepest descent direction an analysis of ngd for training a multilayer neural network is presented using statistical mechanics the performance can be significantly improved using ngd algorithm and can be used for both the transient and asymptotic stages of learning

natural gradient works efficiently in learning when a parameter space has a certain underlying structure the ordinary gradient of a function does not represent its steepest direction but the natural gradient does information geometry is used for calculating the natural gradients in the parameter space of perceptrons the space of matrices for blind source separation and the space of linear dynamical systems for blind source deconvolution the dynamical behavior of natural gradient online learning is analyzed and is proved to be fisher efficient implying that it has asymptotically the same performance as the optimal batch estimation of parameters this suggests that the plateau phenomenon which appears in the backpropagation learning algorithm of multilayer perceptrons might disappear or might not be so serious when the natural gradient is used an adaptive method of updating the learning rate is proposed and analyzed

higher order boltzmann machines and entropy bounds we examine some aspects of the interface area between mathematical statistics and statistical physics relevant to the study of boltzmann machines the boltzmann machine learning algorithm is based on a variational principle gibbs lemma for relative entropy this fact suggests the possibility of a scheme of successive approximations here we consider successive approximations parametrized by the order of many body interactions among individual units we prove bounds on the gain in relative entropy in the crucial step of adding and estimating by hebb s rule a new parameter we address the problem of providing on the basis of local observations upper and lower bounds on the entropy while upper bounds are easily obtained by subadditivity lower bounds involve localization of hirschman bounds on a dual quantum system

on the asymptotic spectrum of finite element matrix sequences we derive a new formula for the asymptotic eigenvalue distribution of stiffness matrices obtained by applying p 1 finite elements with standard mesh refinement to the semielliptic pde of second order in divergence form nabla k nabla t u f on o u g on partial omega here omega subset r 2 and k is supposed to be piecewise continuous and pointwise symmetric semipositive definite the symbol describing this asymptotic eigenvalue distribution depends on the pde but also both on the numerical scheme for approaching the underlying bilinear form and on the geometry of triangulation of the domain our work is motivated by recent results on the superlinear convergence behavior of the conjugate gradient method which requires the knowledge of such asymptotic eigenvalue distributions for sequences of matrices depending on a discretization parameter h when h to 0 we compare our findings with similar results for the finite difference method which were published in recent years in particular we observe that our sequence of stiffness matrices is part of the class of generalized locally toeplitz sequences for which many theoretical tools are available this enables us to derive some results on the conditioning and preconditioning of such stiffness matrices

spectral analysis of coupled pdes and of their schur complements via generalized locally toeplitz sequences in 2d spectral analysis of coupled pdes and of their schur complements via generalized locally toeplitz sequences in 2d

direct determination of periodic solutions of the dynamical equations of flexible mechanisms and manipulators a method based on finite elements characterized by generalized strains as functions of the nodal co ordinates is the basis for the computer program spacar for the kinematic and dynamic analysis of flexible multibody systems this program allows also the evaluation of the linearized dynamical equations which makes possible the implementation of a method for the direct determination of stationary and periodic solutions and of a continuation method for these solutions if some parameter is varied as an illustration these methods are applied to a duffing equation a double pendulum and a slider crank mechanism with an elastic connecting rod

solution of a tridiagonal operator equation let h be a separable hilbert space with an orthonormal basis en n n t be a bounded tridiagonal operator on h and tn be its truncation on span e1 e2 en we study the operator equation tx y through its finite dimensional truncations tnxn yn it is shown that if tn 1en and tn 1en are bounded then t is invertible and the solution of tx y can be obtained as a limit in the norm topology of the solutions of its finite dimensional truncations this leads to uniform boundedness of the sequence tn 1 we also give sufficient conditions for the boundedness of tn 1en and tn 1en in terms of the entries of the matrix of t

extended hamiltonian learning on riemannian manifolds theoretical aspects this paper introduces a general theory of extended hamiltonian second order learning on riemannian manifolds as an instance of learning by constrained criterion optimization the dynamical learning equations are derived within the general framework of extended hamiltonian stationary action principle and are expressed in a coordinate free fashion a theoretical analysis is carried out in order to compare the features of the dynamical learning theory with the features exhibited by the gradient based ones in particular gradient based learning is shown to be an instance of dynamical learning and the classical gradient based learning modified by a momentum term is shown to resemble discrete time dynamical learning moreover the convergence features of gradient based and dynamical learning are compared on a theoretical basis this paper discusses cases of learning by dynamical systems on manifolds of interest in the scientific literature namely the stiefel manifold the special orthogonal group the grassmann manifold the group of symmetric positive definite matrices the generalized flag manifold and the real symplectic group of matrices

generalized locally toeplitz sequences spectral analysis and applications to discretized partial differential equations abstract starting from the finite difference discretization of an elliptic second order pde as i j 1 d x i a i j x x j u x b x over a bounded domain we introduce the notion of generalized locally toeplitz sequence of matrices the singular value distribution the eigenvalue distribution in the hermitian case is studied and characterized for generalized locally toeplitz sequences in terms of weighted multidimensional szeg formulas this extends preceding results attributed to tilli which concern the unilevel case the application of this theoretic analysis to the numerical solution of pdes is finally discussed

the glt class as a generalized fourier analysis and applications recently the class of generalized locally toeplitz glt sequences has been introduced as a generalization both of classical toeplitz sequences and of variable coefficient differential operators and for every sequence of the class it has been demonstrated that it is possible to give a rigorous description of the asymptotic spectrum in terms of a function the symbol that can be easily identified this generalizes the notion of a symbol for differential operators discrete and continuous or for toeplitz sequences for which it is identified through the fourier coefficients and is related to the classical fourier analysis the glt class has nice algebraic properties and indeed it has been proven that it is stable under linear combinations and products in this paper we prove that the considered class is closed under inversion as well when the sequence which is inverted shows a sparsely vanishing symbol sparsely vanishing symbol a symbol which vanishes at most in a set of zero lebesgue measure furthermore we show that the glt class virtually includes any finite difference or finite element discretization of pdes and based on this we demonstrate that our results on glt sequences can be used in a pde setting in various directions 1 as a generalized fourier analysis for the study of iterative and semi iterative methods when dealing with variable coefficients non rectangular domains non uniform gridding or triangulations 2 in order to provide a tool for the stability analysis of pde numerical schemes e g a necessary von neumann criterium for variable coefficient systems of pdes is obtained uniformly with respect to the boundary conditions 3 for a multigrid analysis of convergence and for providing spectral information on large preconditioned systems in the variable coefficient case etc the final part of the paper deals indeed with problems 1 3 and other possible directions in which the glt analysis can be conveniently employed

a class of intrinsic schemes for orthogonal integration numerical integration of odes on the orthogonal stiefel manifold is considered points on this manifold are represented as n k matrices with orthonormal columns of particular interest is the case when n gg k mainly two requirements are imposed on the integration schemes first they should have arithmetic complexity of order nk2 second they should be intrinsic in the sense that they require only the ode vector field to be defined on the stiefel manifold as opposed to for instance projection methods the design of the methods makes use of retractions maps two algorithms are proposed one where the retraction map is based on the qr decomposition of a matrix and one where it is based on the polar decomposition numerical experiments show that the new methods are superior to standard lie group methods with respect to arithmetic complexity and may be more reliable than projection methods owing to their intrinsic nature

computing lyapunov exponents on a stiefel manifold the problem of numerical computation of a few lyapunov exponents les of finite dimensional dynamical systems is considered from the viewpoint of the differential geometry of stiefel manifolds whether one computes one many or all les of a continuous dynamical system by time integration discrete or continuous orthonormalization is essential for stable numerical integration a differential geometric view of continuous orthogonalization suggests that one restricts the linearized vectorfield to a stiefel manifold however the stiefel manifold is not in general an attracting submanifold of the ambient euclidean space it is a constraint manifold with a weak numerical invariant new numerical algorithms for this problem are then designed which use the fiber bundle characterization of these manifolds this framework leads to a new class of systems for continuous orthogonalization which have strong numerical invariance properties and the strong skew symmetry property numerical integration of these new systems with geometric integrators leads to a new class of numerical methods for computing a few les which preserve orthonormality to machine accuracy this idea is also taken a step further by making the stiefel manifold an attracting invariant manifold in which case standard explicit runge kutta algorithms can be used this leads to an algorithm which requires only marginally more computation than a standard explicit integration without orthogonalization these class of methods should be particularly effective for computing a few les for large dimension dynamical systems the new schemes are straightforward to implement a test case is presented for illustration and an example from dynamical systems is presented where a few les are computed for an array of coupled oscillators

equivariant adaptive source separation source separation consists of recovering a set of independent signals when only mixtures with unknown coefficients are observed this paper introduces a class of adaptive algorithms for source separation that implements an adaptive version of equivariant estimation and is henceforth called equivariant adaptive separation via independence easi the easi algorithms are based on the idea of serial updating this specific form of matrix updates systematically yields algorithms with a simple structure for both real and complex mixtures most importantly the performance of an easi algorithm does not depend on the mixing matrix in particular convergence rates stability conditions and interference rejection levels depend only on the normalized distributions of the source signals closed form expressions of these quantities are given via an asymptotic performance analysis the theme of equivariance is stressed throughout the paper the source separation problem has an underlying multiplicative structure the parameter space forms a matrix multiplicative group we explore the favorable consequences of this fact on implementation performance and optimization of easi algorithms

some theorems on linear positive operators and functionals and their applications abstract in this paper we prove two theorems concerning linear positive operators and functionals in a very general sense we show how these results can be used to derive new results in a wide range of applications from approximation theory to the numerical treatment of differential equations and to the numerical analysis connected with signals and images

a multiharmonic method for non linear vibration analysis a multiharmonic method for analysis of non linear dynamic systems submitted to periodic loading conditions is presented the approach is based on a systematic use of the fast fourier transform the exact linearization of the resulting equations in the frequency domain allows to obtain full quadratic convergence rate

convergence analysis of two grid methods for elliptic toeplitz and pdes matrix sequences summary the solution of large toeplitz systems with nonnegative generating functions by multigrid methods was proposed in previous papers 13 14 22 the technique was modified in 6 36 and a rigorous proof of convergence of the tgm two grid method was given in the special case where the generating function has only a zero at x 0 0 of order at most two here by extending the latter approach we perform a complete analysis of convergence of the tgm under the sole assumption that f is nonnegative and with a zero at x 0 0 of finite order an extension of the same analysis in the multilevel case and in the case of finite difference matrix sequences discretizing elliptic pdes with nonconstant coefficients and of any order is then discussed

a korovkin based approximation of multilevel toeplitz matrices with rectangular unstructured blocks via multilevel trigonometric matrix spaces in this paper we are concerned with the approximation of multilevel toeplitz matrices generated by multivariate rectangular matrix valued continuous also l2 functions f i p rightarrow cal c s times t with i pi pi we first define a class cal a t of multilevel trigonometric matrix spaces with unstructured s times t blocks second we introduce a multi sequence of linear approximation operators cal p n x n n 1 n 2 ldots n p acting on complex matrices of dimensions n 1 cdots n p s times n 1 cdots n p t n j in cal n and giving the minimizer of the distance of x from a fixed representative of cal a t with regard to the frobenius norm then by stating and proving new versions of the korovkin theorem for f ranging in the banach space bf c i p cal c s times t cdot infty and by linking these theorems to the operators cal p n we prove the goodness of the proposed approximation in terms of the strong and weak equidistribution of the eigenvalues singular values of a n f and cal p n a n f the discussed results provide a very uniform tool for treating the preconditioning problem with applications to a wide variety of structures as analyzed in the twin paper s serra capizzano calcolo in press

the relation between information theory and the differential geometry approach to statistics abstract cencov has shown that the riemannian metric on the probability simplex xi 1 defined by ds 2 dx i 2 x i has an invariance property under certain probabilistically natural mappings no other riemannian metric has the same property the geometry associated with this metric is shown to lead almost automatically to measures of divergence between probability distributions which are associated with kullback bhattacharyya and matusita certain vector fields are associated in a natural way with random variables the integral curves of these vector fields yield the maximum entropy or minimum divergence estimates of probabilities some other consequences of this geometric view are also explored

on the spectrum of stiffness matrices arising from isogeometric analysis we study the spectral properties of stiffness matrices that arise in the context of isogeometric analysis for the numerical solution of classical second order elliptic problems motivated by the applicative interest in the fast solution of the related linear systems we are looking for a spectral characterization of the involved matrices in particular we investigate non singularity conditioning extremal behavior spectral distribution in the weyl sense as well as clustering of the eigenvalues to a certain compact subset of mathbb c c all the analysis is related to the notion of symbol in the toeplitz setting and is carried out both for the cases of 1d and 2d problems

discriminant pattern recognition using transformation invariant neurons to overcome the problem of invariant pattern recognition simard lecun and denker 1993 proposed a successful nearest neighbor approach based on tangent distance attaining state of the art accuracy since this approach needs great computational and memory effort hastie simard and sackinger 1995 proposed an algorithm hss based on singular value decomposition svd for the generation of nondiscriminant tangent models in this article we propose a different approach based on a gradientdescent constructive algorithm called td neuron that develops discriminant models we present as well comparative results of our constructive algorithm versus hss and learning vector quantization lvq algorithms specifically we tested the hss algorithm using both the original version based on the two sided tangent distance and a new version based on the one sided tangent distance empirical results over the nist 3 database show that the td neuron is superior to both svd and lvq based algorithms since it reaches a better trade off between error and rejection

orthogonal trajectory accession to the nonlinear equilibrium curve abstract an alternative to the riks wempner crisfield iterative correction scheme is presented that does not require an explicit displacement load accession path to the nonlinear equilibrium curve nor a known equilibrium point its symmetry with respect to the displacement and load assures success in rounding limit points as well as turning points

an information maximization approach to blind separation and blind deconvolution we derive a new self organizing learning algorithm that maximizes the information transferred in a network of nonlinear units the algorithm does not assume any knowledge of the input distributions and is defined here for the zero noise limit under these conditions information maximization has extra properties not found in the linear case linsker 1989 the nonlinearities in the transfer function are able to pick up higher order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation this enables the network to separate statistically independent components in the inputs a higher order generalization of principal components analysis we apply the network to the source separation or cocktail party problem successfully separating unknown mixtures of up to 10 speakers we also show that a variant on the network architecture is able to perform blind deconvolution cancellation of unknown echoes and reverberation in a speech signal finally we derive dependencies of information transfer on time delays we suggest that information maximization provides a unifying framework for problems in blind signal processing

projection approximation subspace tracking subspace estimation plays an important role in a variety of modern signal processing applications we present a new approach for tracking the signal subspace recursively it is based on a novel interpretation of the signal subspace as the solution of a projection like unconstrained minimization problem we show that recursive least squares techniques can be applied to solve this problem by making an appropriate projection approximation the resulting algorithms have a computational complexity of o nr where n is the input vector dimension and r is the number of desired eigencomponents simulation results demonstrate that the tracking capability of these algorithms is similar to and in some cases more robust than the computationally expensive batch eigenvalue decomposition relations of the new algorithms to other subspace tracking methods and numerical issues are also discussed

statistical theory of learning curves under entropic loss criterion the present paper elucidates a universal property of learning curves which shows how the generalization error training error and the complexity of the underlying stochastic machine are related and how the behavior of a stochastic machine is improved as the number of training examples increases the error is measured by the entropic loss it is proved that the generalization error converges to h0 the entropy of the conditional distribution of the true machine as h0 m 2t while the training error converges as h0 m 2t where t is the number of examples and m shows the complexity of the network when the model is faithful implying that the true machine is in the model m is reduced to m the number of modifiable parameters this is a universal law because it holds for any regular machine irrespective of its structure under the maximum likelihood estimator similar relations are obtained for the bayes and gibbs learning algorithms these learning curves show the relation among the accuracy of learning the complexity of a model and the number of training examples

linearization of dynamic equations of flexible mechanisms a finite element approach a finite element based method is presented for evaluation of linearized dynamic equations of flexible mechanisms about a nominal trajectory the coefficient matrices of the linearized equations of motion are evaluated as explicit analytical expressions involving mixed sets of generalized co ordinates of the mechanism with rigid links and deformation mode co ordinates that characterize deformation of flexible link elements this task is accomplished by employing the general framework of the geometric transfer function formalism the proposed method is general in nature and can be applied to spatial mechanisms and manipulators having revolute and prismatic joints the method also permits investigation of the dynamics of flexible rotors and spinning shafts application of the theory is illustrated through a detailed model development of a four bar mechanism and the analysis of bending vibrations of two single link mechanisms in which the link is considered as a rotating flexible arm or as an unsymmetrical rotating shaft respectively the algorithm for the calculation of the matrix coefficients is directly emenable to numerical computation and has been incorporated into the linearization module of the computer program spacar