local linear regression smoothers and their minimax efficiencies in this paper we introduce a smooth version of local linear regression estimators and address their advantages the mse and mise of the estimators are computed explicitly it turns out that the local linear regression smoothers have nice sampling properties and high minimax efficiency they are not only efficient in rates but also nearly efficient in constant factors in the nonparametric regression context the asymptotic minimax lower bound is developed via the heuristic of the hardest onedimensional subproblem of donoho and liu connections of the minimax risk with the modulus of continuity are made the lower bound is also applicable for estimating conditional mean regression and conditional quantiles for both fixed and random design regression problems

preserving poisson structure and orthogonality in numerical integration of differential equations we consider the numerical integration of two types of systems of differential equations we first consider hamiltonian systems of differential equations with a poisson structure we show that symplectic runge kutta methods preserve this structure when the poisson tensor is constant using nonlinear changes of coordinates this structure can also be preserved for nonconstant poisson tensors as exemplified on the euler equations for the free rigid body we also consider orthogonal flows and the closely related class of isospectral flows to numerically preserve the orthogonality property we take the approach of formulating an equivalent system of differential algebraic equations daes and of integrating the system with a special combination of a particular class of runge kutta methods this approach requires only matrix matrix products and can preserve geometric properties of the flow such as reversibility

estimators of diffusions with randomly spaced discrete observations a general theory we provide a general method to analyze the asymptotic properties of a variety of estimators of continuous time diffusion processes when the data are not only discretely sampled in time but the time separating successive observations may possibly be random we introduce a new operator the generalized infinitesimal generator to obtain taylor expansions of the asymptotic moments of the estimators as a special case our results apply to the situation where the data are discretely sampled at a fixed nonrandom time interval we include as specific examples estimators based on maximum likelihood and discrete approximations such as the euler scheme

neural learning by geometric integration of reduced rigid body equations in previous contributions the second author of this paper presented a new class of algorithms for orthonormal learning of linear neural networks with p inputs and m outputs based on the equations describing the dynamics of a massive rigid frame on the stiefel manifold these algorithms exhibit good numerical stability strongly binding to the sub manifold of constraints and good controllability of the learning dynamics but are not completely satisfactory from a computational complexity point of view in the recent literature efficient methods of integration on the stiefel manifold have been proposed by various authors see for example phys d 156 2001 219 numer algorithms 32 2003 163 j numer anal 21 2001 463 numer math 83 1999 599 inspired by these approaches in this paper we propose a new and efficient representation of the mentioned learning equations and a new way to integrate them the numerical experiments show how the new formulation leads to significant computational savings especially when p m the effectiveness of the algorithms is substantiated by numerical experiments concerning principal subspace analysis and independent component analysis these experiments were carried out with both synthetic and real world data

computation of orthonormal factors for fundamental solution matrices in this work we introduce and analyze two new techniques for obtaining the q factor in the qr factorization of some or all columns of a fundamental solution matrix y of a linear differential system these techniques are based on elementary householder and givens transformations we implement and compare these new techniques with existing approaches on some examples

solvability of a three point nonlinear boundary value problem for a second order ordinary differential equation in case f x u u ps x p x u pjx u with pk 0 l 4 r locally integrable the boundary value problem l lt 1 2 was studied by kiguradze and lomtatidze i the purpose of this paper is to obtain existence and uniqueness theorems for the boundary value problem 1 1 t 1 2 under natural conditions on f using degree theoretic arguments we note that if u is a solution of 1 1 1 2 then there exists at least one 5 e q l such that u c 0 accordingly the boundary value problem

twin positive solutions of boundary value problems for finite difference equations with p laplacian operator abstract in this paper a new twin fixed point theorem is applied to obtain the existence of at least two positive solutions for the boundary value problem p u k a k f u k 0 k 0 n u n 1 0 u 0 b 0 u 0 0 in addition two corollaries and an example are given to illustrate the main results

support recovery for the drift coefficient of high dimensional diffusions consider the problem of learning the drift coefficient of a p dimensional stochastic differential equation from a sample path of length t we assume that the drift is parametrized by a high dimensional vector and study the support recovery problem when both p and t can tend to infinity in particular we prove a general lower bound on the sample complexity t by using a characterization of mutual information as a time integral of conditional variance due to kadota zakai and ziv for linear stochastic differential equations the drift coefficient is parametrized by a p times p matrix that describes which degrees of freedom interact under the dynamics in this case we analyze a ell 1 regularized least squares estimator and prove an upper bound on t that nearly matches the lower bound on specific classes of sparse matrices

numerical methods for ordinary differential equations on matrix manifolds in recent years differential systems whose solutions evolve on manifolds of matrices have acquired a certain relevance in numerical analysis a classical example of such a differential system is the well known toda flow this paper is a partial survey of numerical methods recently proposed for approximating the solutions of ordinary differential systems evolving on matrix manifolds in particular some results recently obtained by the author jointly with his co workers will be presented we will discuss numerical techniques for isospectral and isodynamical flows where the eigenvalues of the solutions are preserved during the evolution and numerical methods for odes on the orthogonal group or evolving on a more general quadratic group like the symplectic or lorentz group we mention some results for systems evolving on the stiefel manifold and also review results for the numerical solution of odes evolving on the general linear group of matrices

multiple positive solutions of quasi linear boundary value problems for finite difference equations abstract in this paper we consider a discrete second order quasi linear boundary value problem which nonlinear term f is involved with the first order difference by using a generalization of the leggett williams fixed point theorem due to avery and peterson we provide sufficient conditions for the existence of at least three positive solutions

adaptive drift estimation for nonparametric diffusion model we consider a nonparametric diffusion process whose drift and diffusion coefficients are nonparametric functions of the state variable the goal is to estimate the unknown drift coefficient we apply a locally linear smoother with a data driven bandwidth choice the procedure is fully adaptive and nearly optimal up to a log log factor the results about the quality of estimation are nonasymptotic and do not require any ergodic or mixing properties of the observed process

numerical solution of isospectral flows in this paper we are concerned with the problem of solving numerically isospectral flows these flows are characterized by the differential equation l b l l l 0 l 0 where l 0 is a d x d symmetric matrix b l is a skew symmetric matrix function of l and b l is the lie bracket operator we show that standard runge kutta schemes fail in recovering the main qualitative feature of these flows that is isospectrality since they cannot recover arbitrary cubic conservation laws this failure motivates us to introduce an alternative approach and establish a framework for generation of isospectral methods of arbitrarily high order

global error analysis and inertial manifold reduction four types of global error for initial value problems are considered in a common framework they include classical forward error analysis and shadowing error analysis together with extensions of both to include rescaling of time to determine the amplification of the local error that bounds the global error we present a linear analysis similar in spirit to condition number estimation for linear systems of equations we combine these ideas with techniques for dimension reduction of differential equations via a boundary value formulation of numerical inertial manifold reduction these global error concepts are exercised to illustrate their utility on the lorenz equations and inertial manifold reductions of the kuramoto sivashinsky equation

a note on the computation of an orthonormal basis for the null space of a matrix a highly regarded method to obtain an orthonormal basis z for the null space of a matrix a t is the qr decomposition of a where q is the product of householder matrices in several optimization contexts a x varies continuously with x and it is desirable the z x vary continuously also in this note we demonstrate that the standard implementation of the qr decomposition does not yield an orthonormal basis z x whose elements vary continuously with x we suggest three possible remedies

multiplicity of positive solutions for second order three point boundary value problems we study the multiplicity of positive solutions for the second order three point boundary value problem u h t f u 0 t 0 1 u 0 0 u u 1 where 0 1 0 1 the methods employed are fixed point index theorems and leray schauder degree and upper and lower solutions

exact adaptive pointwise drift estimation for multidimensional ergodic diffusions the problem of pointwise adaptive estimation of the drift coefficient of a multivariate diffusion process is investigated we propose an estimator which is sharp adaptive on scales of sobolev smoothness classes the analysis of the exact risk asymptotics allows to identify the impact of the dimension and other influencing values such as the geometry of the diffusion coefficient of the prototypical drift estimation problem for a large class of multidimensional diffusion processes we further sketch generalizations of our results to arbitrary diffusions satisfying suitable bernstein type inequalities

on the estimation of quadratic functionals we discuss the difficulties of estimating quadratic functionals based on observations y t from the white noise model y t int t 0 f u du sigma w t quad t in lbrack 0 1 rbrack where w t is a standard wiener process on lbrack 0 1 rbrack the optimal rates of convergence as sigma rightarrow 0 for estimating quadratic functionals under certain geometric constraints are found specifically the optimal rates of estimating int 1 0 lbrack f k x rbrack 2 dx under hyperrectangular constraints sum f x j f leq cj alpha and weighted l p body constraints sum p f sum infty 1 j r x j f p leq c are computed explicitly where x j f is the j th fourier bessel coefficient of the unknown function f we develop lower bounds based on testing two highly composite hypercubes and address their advantages the attainable lower bounds are found by applying the hardest one dimensional approach as well as the hypercube method we demonstrate that for estimating regular quadratic functionals i e the functionals which can be estimated at rate o sigma 2 the difficulties of the estimation are captured by the hardest one dimensional subproblems and for estimating nonregular quadratic functionals i e no o sigma 2 consistent estimator exists the difficulties are captured at certain finite dimensional the dimension goes to infinity as sigma rightarrow 0 hypercube subproblems

oracle inequalities for inverse problems we consider a sequence space model of statistical linear inverse problems where we need to estimate a function f from indirect noisy observations let a finite set a of linear estimators be given our aim is to mimic the estimator in a that has the smallest risk on the true f under general conditions we show that this can be achieved by simple minimization of an unbiased risk estimator provided the singular values of the operator of the inverse problem decrease as a power law the main result is a nonasymptotic oracle inequality that is shown to be asymptotically exact this inequality can also be used to obtain sharp minimax adaptive results in particular we apply it to show that minimax adaptation on ellipsoids in the multivariate anisotropic case is realized by minimization of unbiased risk estimator without any loss of efficiency with respect to optimal nonadaptive procedures

arch models as diffusion approximations abstract this paper investigates the convergence of stochastic difference equations e g arch to stochastic differential equations as the length of the discrete time intervals between observations goes to zero these results are applied to the garch 1 1 model of bollerslev 1986 and to the ar 1 exponential arch model of nelson 1989 in their continuous time limits the conditional variance processes in these models have stationary distributions that are inverted gamma and lognormal respectively in addition a class of diffusion approximations based on the exponential arch model is developed

computing lyapunov exponents on a stiefel manifold the problem of numerical computation of a few lyapunov exponents les of finite dimensional dynamical systems is considered from the viewpoint of the differential geometry of stiefel manifolds whether one computes one many or all les of a continuous dynamical system by time integration discrete or continuous orthonormalization is essential for stable numerical integration a differential geometric view of continuous orthogonalization suggests that one restricts the linearized vectorfield to a stiefel manifold however the stiefel manifold is not in general an attracting submanifold of the ambient euclidean space it is a constraint manifold with a weak numerical invariant new numerical algorithms for this problem are then designed which use the fiber bundle characterization of these manifolds this framework leads to a new class of systems for continuous orthogonalization which have strong numerical invariance properties and the strong skew symmetry property numerical integration of these new systems with geometric integrators leads to a new class of numerical methods for computing a few les which preserve orthonormality to machine accuracy this idea is also taken a step further by making the stiefel manifold an attracting invariant manifold in which case standard explicit runge kutta algorithms can be used this leads to an algorithm which requires only marginally more computation than a standard explicit integration without orthogonalization these class of methods should be particularly effective for computing a few les for large dimension dynamical systems the new schemes are straightforward to implement a test case is presented for illustration and an example from dynamical systems is presented where a few les are computed for an array of coupled oscillators

three positive fixed points of nonlinear operators on ordered banach spaces we generalize the fixed point theorem of leggett williams which is a theorem giving conditions that imply the existence of three fixed points of an operator defined on a cone in a banach space we then show how to apply our theorem to prove the existence of three positive solutions to a second order discrete boundary value problem 2001 elsevier science ltd all rights reserved

unitary integrators and applications to continuous orthonormalization techniques in this paper the issue of integrating matrix differential systems whose solutions are unitary matrices is addressed such systems have skew hermitian coefficient matrices in the linear case and a

stability and lyapunov stability of dynamical systems a differential approach and a numerical method abstract a set of differential equations for the eigenvalues and eigenvectors of the stability matrix of a dynamical system as well as for the lyapunov exponents and the corresponding eigenvectors is derived the rate of convergence of the lyapunov eigenvectors is shown to be exponential the eigenvectors of the stability matrix can be grouped into sets each spanning a subspace which converges at an exponential rate it is demonstrated that generically the real parts of the eigenvalues of the stability matrix equal the corresponding lyapunov exponents this statement has been tested numerically the values of the lyapunov exponents i are shown to be related to the corresponding finite time values of the lyapunov exponents e g those deduced from a finite time numerical simulation i t by i t i b i i t t the b i s are constants and i t are noise terms of zero mean this observation leads to a method of extrapolation which has been used to predict lyapunov exponents from a finite amount of data it is shown that the use of the standard numerical methods to compute lyapunov exponents introduces an error a i t in the value of i t where the a i s are constants thus the standard method has a rate of convergence which is the same as that of the exact i t s finally we have shown how one can compute the eigenvectors associated with each of the eigenvalues of the stability matrix as well as the lyapunov eigenvectors

on the compuation of lyapunov exponents for continuous dynamical systems in this paper we consider discrete and continuous qr algorithms for computing all of the lyapunov exponents of a regular dynamical system we begin by reviewing theoretical results for regular systems and present general perturbation results for lyapunov exponents we then present the algorithms give an error analysis of them and describe their implementation finally we give several numerical examples and some conclusions

on the error in qr integration an important change of variables for a linear time varying system dot x a t x t ge 0 is that induced by the qr factorization of the underlying fundamental matrix solution x qr with q orthogonal and r upper triangular with positive diagonal to find this change of variable one needs to solve a nonlinear matrix differential equation for q practically this means finding a numerical approximation to q by using some appropriate discretization scheme whereby one attempts to control the local error during the integration our contribution in this work is to obtain global error bounds for the numerically computed q these bounds depend on the local error tolerance used to integrate for q and on structural properties of the problem itself but not on the length of the interval over which we integrate this is particularly important since in principle q may need to be found on the half line t ge 0

lyapunov spectral intervals theory and computation different definitions of spectra have been proposed over the years to characterize the asymptotic behavior of nonautonomous linear systems here we consider the spectrum based on exponential dichotomy of sacker and sell j differential equations 7 1978 pp 320 358 and the spectrum defined in terms of upper and lower lyapunov exponents a main goal of ours is to understand to what extent these spectra are computable by using an orthogonal change of variables transforming the system to upper triangular form and the assumption of integral separation for the diagonal of the new triangular system we justify how popular numerical methods the so called continuous qr and svd approaches can be used to approximate these spectra we further discuss how to verify the property of integral separation and hence how to a posteriori infer stability of the attained spectral information finally we discuss the algorithms we have used to approximate the lyapunov and sacker sell spectra and present some numerical results

numerical computation of an analytic singular value decomposition of a matrix valued function this paper extends the singular value decomposition to a path of matricese t an analytic singular value decomposition of a path of matricese t is an analytic path of factorizationse t x t s t y t t wherex t andy t are orthogonal ands t is diagonal to maintain differentiability the diagonal entries ofs t are allowed to be either positive or negative and to appear in any order this paper investigates existence and uniqueness of analytic svd s and develops an algorithm for computing them we show that a real analytic pathe t always admits a real analytic svd a full rank smooth pathe t with distinct singular values admits a smooth svd we derive a differential equation for the left factor develop euler like and extrapolated euler like numerical methods for approximating an analytic svd and prove that the euler like method converges

numerical shadowing using componentwise bounds and a sharper fixed point result shadowing provides a means of obtaining global error bounds for approximate solutions of nonlinear differential equations with interesting dynamics in particular for initial value problems with positive lyapunov exponents shadowing breaks down in the presence of zero lyapunov exponents although some results such as shadowing with rescaling of time have been obtained using a reformulation of the original differential equations and an improved fixed point result we take advantage of componentwise local error bounds to use relatively smaller error tolerances in nonhyperbolic and contractive directions i e in directions corresponding to zero and negative lyapunov exponents the result is a decrease in the shadowing global error

sharp adaptive estimation of the drift function for ergodic diffusions the global estimation problem of the drift function is considered for a large class of ergodic diffusion processes the unknown drift s is supposed to belong to a nonparametric class of smooth functions of order k 1 but the value of k is not known to the statistician a fully data driven procedure of estimating the drift function is proposed using the estimated risk minimization method the sharp adaptivity of this procedure is proven up to an optimal constant when the quality of the estimation is measured by the integrated squared error weighted by the square of the invariant density

a class of intrinsic schemes for orthogonal integration numerical integration of odes on the orthogonal stiefel manifold is considered points on this manifold are represented as n k matrices with orthonormal columns of particular interest is the case when n gg k mainly two requirements are imposed on the integration schemes first they should have arithmetic complexity of order nk2 second they should be intrinsic in the sense that they require only the ode vector field to be defined on the stiefel manifold as opposed to for instance projection methods the design of the methods makes use of retractions maps two algorithms are proposed one where the retraction map is based on the qr decomposition of a matrix and one where it is based on the polar decomposition numerical experiments show that the new methods are superior to standard lie group methods with respect to arithmetic complexity and may be more reliable than projection methods owing to their intrinsic nature