information theoretic determination of minimax rates of convergence we present some general results determining minimax bounds on statistical risk for density estimation based on certain information theoretic considerations these bounds depend only on metric entropy conditions and are used to identify the minimax rates of convergence

erratum swarming for faster convergence in stochastic optimization in this erratum we correct the statements and or proofs of lemma 4 6 theorem 4 7 and theorem 4 16 in s pu and a garcia siam j control optim 56 2018 pp 2997 3020 the main conclusion

stochastic first and zeroth order methods for nonconvex stochastic programming in this paper we introduce a new stochastic approximation type algorithm namely the randomized stochastic gradient rsg method for solving an important class of nonlinear possibly nonconvex stochastic programming problems we establish the complexity of this method for computing an approximate stationary point of a nonlinear programming problem we also show that this method possesses a nearly optimal rate of convergence if the problem is convex we discuss a variant of the algorithm which consists of applying a postoptimization phase to evaluate a short list of solutions generated by several independent runs of the rsg method and we show that such modification allows us to improve significantly the large deviation properties of the algorithm these methods are then specialized for solving a class of simulation based optimization problems in which only stochastic zeroth order information is available

on a class of second order anti periodic boundary value problems of concern is the existence and uniqueness of anti periodic solutions to a class of abstract nonlinear second order differential equations the investigation relies on the theory of m accretive operators in banach or hilbert spaces examples of boundary value problems for ordinary and partial differential equations are also discussed

ground states for diffusion system with periodic and asymptotically periodic nonlinearity in this paper we study the following diffusion system t u x u b t x x u v x u g t x v t v x v b t x x v v x v f t x u where b c 1 r r n r n and v x c r n r by using non nehari manifold method developed in tang 2014 we investigate the existence of the ground state solutions of nehari type for the above problem with periodic and asymptotically periodic nonlinearity

robust stochastic approximation approach to stochastic programming in this paper we consider optimization problems where the objective function is given in a form of the expectation a basic difficulty of solving such stochastic optimization problems is that the involved multidimensional integrals expectations cannot be computed with high accuracy the aim of this paper is to compare two computational approaches based on monte carlo sampling techniques namely the stochastic approximation sa and the sample average approximation saa methods both approaches the sa and saa methods have a long history current opinion is that the saa method can efficiently use a specific say linear structure of the considered problem while the sa approach is a crude subgradient method which often performs poorly in practice we intend to demonstrate that a properly modified sa approach can be competitive and even significantly outperform the saa method for a certain class of convex stochastic problems we extend the analysis to the case of convex concave stochastic saddle point problems and present in our opinion highly encouraging results of numerical experiments

eigenvalues of second order difference equations with periodic and antiperiodic boundary conditions this paper is concerned with periodic and antiperiodic boundary value problems for self adjoint second order difference equations existence of eigenvalues of these two different boundary value problems is proved numbers of their eigenvalues are calculated and their relationships are obtained in addition a representation of solutions of a nonhomogeneous linear equation with initial conditions is given

stochastic block mirror descent methods for nonsmooth and stochastic optimization in this paper we present a new stochastic algorithm namely the stochastic block mirror descent sbmd method for solving large scale nonsmooth and stochastic optimization problems the basic idea of this algorithm is to incorporate block coordinate decomposition and an incremental block averaging scheme into the classic stochastic mirror descent method in order to significantly reduce the cost per iteration of the latter algorithm we establish the rate of convergence of the sbmd method along with its associated large deviation results for solving general nonsmooth and stochastic optimization problems we also introduce variants of this method and establish their rate of convergence for solving strongly convex smooth and composite optimization problems as well as certain nonconvex optimization problems to the best of our knowledge all these developments related to the sbmd methods are new in the stochastic optimization literature moreover some of our results seem to be new for block coordinate descent methods for deterministic optimization

homoclinic solutions of an infinite dimensional hamiltonian system we consider the system begin equation left begin array l partial t u delta x u v x u h v t x u v 3mm partial t v delta x v v x v h u t x u v end array qquad text for t x in mathbb r times mathbb r n right end equation which is an unbounded hamiltonian system in l 2 mathbb r n mathbb r 2m we assume that the constant function u 0 v 0 equiv 0 0 in mathbb r 2m is a stationary solution and that h and v are periodic in the t and x variables we present a variational formulation in order to obtain homoclinic solutions z u v satisfying z t x to 0 as t x to infty it is allowed that v changes sign and that delta v has essential spectrum below and above 0 we also treat the case of a bounded domain omega instead of mathbb r n with dirichlet boundary conditions

accelerated gradient methods for nonconvex nonlinear and stochastic programming in this paper we generalize the well known nesterov s accelerated gradient ag method originally designed for convex smooth optimization to solve nonconvex and possibly stochastic optimization problems we demonstrate that by properly specifying the stepsize policy the ag method exhibits the best known rate of convergence for solving general nonconvex smooth optimization problems by using first order information similarly to the gradient descent method we then consider an important class of composite optimization problems and show that the ag method can solve them uniformly i e by using the same aggressive stepsize policy as in the convex case even if the problem turns out to be nonconvex we demonstrate that the ag method exhibits an optimal rate of convergence if the composite problem is convex and improves the best known rate of convergence if the problem is nonconvex based on the ag method we also present new nonconvex stochastic approximation methods and show that they can improve a few existing rates of convergence for nonconvex stochastic optimization to the best of our knowledge this is the first time that the convergence of the ag method has been established for solving nonconvex nonlinear programming in the literature

the existence of periodic solution for infinite dimensional hamiltonian systems abstract in this paper we will consider a kind of infinite dimensional hamiltonian system hs by the method of saddle point reduction topology degree and the index defined in wang and liu 2015 we will get the existence of periodic solution for hs

exponentiated gradient versus gradient descent for linear predictors we consider two algorithm for on line prediction based on a linear model the algorithms are the well known gradient descent gd algorithm and a new algorithm which we call eg they both maintain a weight vector using simple updates for the gd algorithm the update is based on subtracting the gradient of the squared error made on a prediction the eg algorithm uses the components of the gradient in the exponents of factors that are used in updating the weight vector multiplicatively we present worst case loss bounds for eg and compare them to previously known bounds for the gd algorithm the bounds suggest that the losses of the algorithms are in general incomparable but eg has a much smaller loss if only a few components of the input are relevant for the predictions we have performed experiments which show that our worst case upper bounds are quite tight already on simple artificial data

misspecified nonconvex statistical optimization for sparse phase retrieval existing nonconvex statistical optimization theory and methods crucially rely on the correct specification of the underlying true statistical models to address this issue we take a first step towards taming model misspecification by studying the high dimensional sparse phase retrieval problem with misspecified link functions in particular we propose a simple variant of the thresholded wirtinger flow algorithm that given a proper initialization linearly converges to an estimator with optimal statistical accuracy for a broad family of unknown link functions we further provide extensive numerical experiments to support our theoretical findings

an anti periodic lasalle oscillation theorem for a class of functional differential equations in this paper we establish a result on the existence of anti periodic solution for a class of functional differential equations which substantially extends and improves some important results in the literature

dynamic stochastic approximation for multi stage stochastic optimization in this paper we consider multi stage stochastic optimization problems with convex objectives and conic constraints at each stage we present a new stochastic first order method namely the dynamic stochastic approximation dsa algorithm for solving these types of stochastic optimization problems we show that dsa can achieve an optimal mathcal o 1 epsilon 4 rate of convergence in terms of the total number of required scenarios when applied to a three stage stochastic optimization problem we further show that this rate of convergence can be improved to mathcal o 1 epsilon 2 when the objective function is strongly convex we also discuss variants of dsa for solving more general multi stage stochastic optimization problems with the number of stages t 3 the developed dsa algorithms only need to go through the scenario tree once in order to compute an epsilon solution of the multi stage stochastic optimization problem as a result the memory required by dsa only grows linearly with respect to the number of stages to the best of our knowledge this is the first time that stochastic approximation type methods are generalized for multi stage stochastic optimization with t ge 3

conditional gradient type methods for composite nonlinear and stochastic optimization in this paper we present a conditional gradient type cgt method for solving a class of composite optimization problems where the objective function consists of a weakly smooth term and a strongly convex regularization term while including a strongly convex term in the subproblems of the classical conditional gradient method improves its rate of convergence it does not cost per iteration as much as general proximal type algorithms more specifically we present a unified analysis for the cgt method in the sense that it achieves the best known rate of convergence when the weakly smooth term is nonconvex and possesses nearly optimal complexity if it turns out to be convex while implementation of the cgt method requires explicitly estimating problem parameters like the level of smoothness of the first term in the objective function we also present a few variants of this method which relax such estimation unlike general proximal type parameter free methods these variants of the cgt method do not require any additional effort for computing sub gradients of the objective function and or solving extra subproblems at each iteration we then generalize these methods under stochastic setting and present a few new complexity results to the best of our knowledge this is the first time that such complexity results are presented for solving stochastic weakly smooth nonconvex and strongly convex optimization problems

boundary flows in general coset theories in this paper we study the boundary effects for off critical integrable field theories which have close analogues with integrable lattice models our models are the coset conformal field theories perturbed by an integrable boundary and bulk operators the boundary interactions are encoded into the boundary reflection matrix using the thermodynamic bethe ansatz tba method we verify the flows of the conformal bcs by computing the boundary entropies these flows of the bcs have direct interpretations for the fusion restricted solid on solid rsos lattice models for super conformal field theories cfts k 2 we show that these flows are possible only for the neveu schwarz sector and are consistent with the lattice results the models we consider cover a wide class of integrable models in particular we show how the impurity spin is screened by electrons for the k channel kondo model by taking the limit we also study the problem using an independent method based on the boundary roaming tba our numerical results are consistent with the boundary cfts and rsos tba analysis

on antiperiodic solutions for cohen grossberg shunting inhibitory neural networks with time varying delays and impulses in this letter a class of cohen grossberg shunting inhibitory neural networks with time varying delays and impulses is investigated sufficient conditions for the existence and exponential stability of antiperiodic solutions of such a class of neural networks are established our results are new and complementary to previously known results an example is given to illustrate the feasibility and effectiveness of our main results

functional determinants from wronski green functions a general technique is developed for calculating functional determinants of second order differential operators with dirichlet periodic and antiperiodic boundary conditions without the knowledge of spectral properties as an example we give explicit formulas for a harmonic oscillator with an arbitrary time dependent frequency where our result is a generalization of the gel fand yaglom famous formula for dirichlet boundary conditions our technique is based on the wronski s construction of green functions which does not require spectral knowledge our final formula expresses the ratios of functional determinants in terms of an ordinary 2 2 determinant of a constant matrix constructed from two linearly independent solutions of the homogeneous differential equations associated with second order differential operators for ratios of determinants encountered in semiclassical fluctuations around a classical solution the result can further be expressed in terms of the classical solution special properties of operators with a zero mode are exhibited

on sampling rates in simulation based recursions we consider the context of simulation based recursions that is recursions that involve quantities needing to be estimated using a stochastic simulation examples include stochastic adaptations of fixed point and gradient descent recursions obtained by replacing function and derivative values appearing within the recursion by their monte carlo counterparts the primary motivating settings are simulation optimization and stochastic root finding problems where the low point and the zero of a function are sought respectively with only monte carlo estimates of the functions appearing within the problem we ask how much monte carlo sampling needs to be performed within simulation based recursions in order that the resulting iterates remain consistent and more importantly efficient where efficient implies convergence at the fastest possible rate answering these questions involves trading off two types of error inherent in the iterates the deterministic error due to recursion and the stochastic er

periodic solutions for a class of non autonomous hamiltonian systems abstract we consider the existence of nontrivial periodic solutions for a superlinear hamiltonian system h j u a t u h t u 0 u r 2 n t r we prove an abstract result on the existence of a critical point for a real valued functional on a hilbert space via a new deformation theorem different from the works in the literature the new deformation theorem is constructed under the cerami type condition instead of palais smale type condition in addition the main assumption here is weaker than the usual ambrosetti rabinowitz type condition 0 h t u u h t u 2 u r 0 this result extends theorems given by li and willem j math anal appl 189 1995 6 32 and li and szulkin j differential equations 112 1994 226 238

an alternative to em for gaussian mixture models batch and stochastic riemannian optimization we consider maximum likelihood estimation for gaussian mixture models gmm s this task is almost invariably solved in theory and practice via the expectation maximization em algorithm em owes its success to various factors of which is its ability to fulfill positive definiteness constraints in closed form is of key importance we propose an alternative to em grounded in the riemannian geometry of positive definite matrices using which we cast gmm parameter estimation as a riemannian optimization problem surprisingly such an out of the box riemannian formulation completely fails and proves much inferior to em this motivates us to take a closer look at the problem geometry and derive a better formulation that is much more amenable to riemannian optimization we then develop riemannian batch and stochastic gradient algorithms that outperform em often substantially we provide a non asymptotic convergence analysis for our stochastic method which is also the first to our knowledge such global analysis for riemannian stochastic gradient numerous empirical results are included to demonstrate the effectiveness of our methods

impulsive anti periodic boundary value problem of first order integro differential equations this paper is concerned with the anti periodic boundary value problem of first order nonlinear impulsive integro differential equations we first establish a new comparison principle and then obtain the existence of extremal solutions by upper lower solution and monotone iterative techniques some examples are presented to illustrate the main results

the relative morse index theory for infinite dimensional hamiltonian systems with applications abstract we established an index theory for a kind of infinite dimensional linear hamiltonian system as applications we considered the existence and multiplicity of periodic solution for some nonlinear infinite dimensional hamiltonian systems

stochastic optimization using a trust region method and random models in this paper we propose and analyze a trust region model based algorithm for solving unconstrained stochastic optimization problems our framework utilizes random models of an objective function f x obtained from stochastic observations of the function or its gradient our method also utilizes estimates of function values to gauge progress that is being made the convergence analysis relies on requirements that these models and these estimates are sufficiently accurate with high enough but fixed probability beyond these conditions no assumptions are made on how these models and estimates are generated under these general conditions we show an almost sure global convergence of the method to a first order stationary point in the second part of the paper we present examples of generating sufficiently accurate random models under biased or unbiased noise assumptions lastly we present some computational results showing the benefits of the proposed method compared to existing approaches that are based on sample averaging or stochastic gradients

ground state solutions for a diffusion system in this paper we study the following diffusion system t u x u v x u g t x v t v x v v x v f t x u where z u v r r n r 2 v x c r n r is a general periodic function under weaker conditions on nonlinearity we establish the existence of ground state solutions via variational methods

global convergence rate analysis of unconstrained optimization methods based on probabilistic models we present global convergence rates for a line search method which is based on random first order models and directions whose quality is ensured only with certain probability we show that in terms of the order of the accuracy the evaluation complexity of such a method is the same as its counterparts that use deterministic accurate models the use of probabilistic models only increases the complexity by a constant which depends on the probability of the models being good we particularize and improve these results in the convex and strongly convex case we also analyse a probabilistic cubic regularization variant that allows approximate probabilistic second order models and show improved complexity bounds compared to probabilistic first order methods again as a function of the accuracy the probabilistic cubic regularization bounds are of the same optimal order as for the deterministic case

optimal rates for zero order convex optimization the power of two function evaluations we consider derivative free algorithms for stochastic and nonstochastic convex optimization problems that use only function values rather than gradients focusing on nonasymptotic bounds on convergence rates we show that if pairs of function values are available algorithms for d dimensional optimization that use gradient estimates based on random perturbations suffer a factor of at most sqrt d in convergence rate over traditional stochastic gradient methods we establish such results for both smooth and nonsmooth cases sharpening previous analyses that suggested a worse dimension dependence and extend our results to the case of multiple m ge 2 evaluations we complement our algorithmic development with information theoretic lower bounds on the minimax convergence rate of such problems establishing the sharpness of our achievable results up to constant sometimes logarithmic factors

primal dual optimization algorithms over riemannian manifolds an iteration complexity analysis in this paper we study nonconvex and nonsmooth multi block optimization over euclidean embedded smooth riemannian submanifolds with coupled linear constraints such optimization problems naturally arise from machine learning statistical learning compressive sensing image processing and tensor pca among others by utilizing the embedding structure we develop an admm like primal dual approach based on decoupled solvable subroutines such as linearized proximal mappings where the duality is with respect to the embedded euclidean spaces first we introduce the optimality conditions for the afore mentioned optimization models then the notion of epsilon stationary solutions is introduced as a result the main part of the paper is to show that the proposed algorithms possess an iteration complexity of o 1 epsilon 2 to reach an epsilon stationary solution for prohibitively large size tensor or machine learning models we present a sampling based stochastic algorithm with the same iteration complexity bound in expectation in case the subproblems are not analytically solvable a feasible curvilinear line search variant of the algorithm based on retraction operators is proposed finally we show specifically how the algorithms can be implemented to solve a variety of practical problems such as the np hard maximum bisection problem the ell q regularized sparse tensor principal component analysis and the community detection problem our preliminary numerical results show great potentials of the proposed methods

stochastic quasi newton methods for nonconvex stochastic optimization in this paper we study stochastic quasi newton methods for nonconvex stochastic optimization where we assume that noisy information about the gradients of the objective function is available via a

periodic solutions of an infinite dimensional hamiltonian system abstract we consider the existence of periodic solutions of the unbounded hamiltonian system h t u x u h v t x u v t v x v h u t x u v for t x r unlike previous work in our case the energy functional does not satisfy the palais smale conditions

the ordered subsets mirror descent optimization method with applications to tomography we describe an optimization problem arising in reconstructing three dimensional medical images from positron emission tomography pet a mathematical model of the problem based on the maximum likelihood principle is posed as a problem of minimizing a convex function of several million variables over the standard simplex to solve a problem of these characteristics we develop and implement a new algorithm ordered subsets mirror descent and demonstrate theoretically and computationally that it is well suited for solving the pet reconstruction problem

periodic solutions of non autonomous second order hamiltonian systems the purpose of this paper is to study the existence of periodic solutions for a class of non autonomous second order hamiltonian system some new existence theorems are obtained by the least action principle