a kalman tracking filter approach to nonlinear programming the problem considered herein is that of finding the minimum of a nonlinear function f when the gradient and hessian matrix are unknown or cannot be easily computed the function f may also be observed in the presence of stochastic noise typical nonlinear programming algorithms solve this problem deterministically by employing function approximations which do not account for approximation errors in the presence of noise the algorithm may either terminate prematurely or may not converge at all in this paper we show that this type of nonlinear programming problem can be associated with a tracking problem commonly arising in aerospace applications in this new domain finding the minimum of a function is equivalent to tracking a point the minimum in the parameter space based on noisy measurements of its position function evaluations f a new nonlinear programming algorithm based on a two level kalman filter is presented which accounts for both modeling and approximation errors the lower level kalman filter performs function approximations while the upper level filter accounts for the tracking of the parameter simulations show that the algorithm performs similarly to newton s method

translation representations for automorphic solutions of the wave equation in non euclidean spaces i this paper deals with the spectral theory of the laplace beltrami operator acting on automorphic functions in n dimensional hyperbolic space hn we study discrete subgroups which have a fundamental polyhedron f with a finite number of sides and infinite volume concerning these we have shown previously that the spectrum of contains at most a finite number of point eigenvalues in 1 2 n 1 2 0 and none less than 1 2 n 1 2 here we prove that the spectrum of is absolutely continuous and of infinite multiplicity in 1 2 n 1 2 our approach uses the non euclidean wave equation introduced by faddeev and pavlov energy ef is defined as ut ut u lu where the bracket is the l2 scalar product over a fundamental polyhedron with respect to the invariant volume of the hyperbolic metric energy is conserved under the group of operator u t relating initial data to data at time t we construct two isometric representations of the space of automorphic data by l2 r n which transmute the action of u t into translation these representations are given explicitly in terms of integrals of the data over horospheres in part ii we shall show the completeness of these representations utt lu 0 l 1 2 n 1 2

relational valued fuzzy sets abstract a generalization of the notion of a fuzzy set is given an r fuzzy set a is a mapping from a set a into the relational system s where is a binary relation on s necessary and sufficient conditions for a unique decomposition and synthesis of an r fuzzy set into the family of ordinary subsets p cuts are given as a consequence it is possible to obtain a fuzzy set as a synthesis of any collection of characteristics functions on a nonvoid set which was impossible in classical fuzzy set theory as a contribution to coding theory a possibility to express any binary block code syntheticly by a single fuzzy set is obtained note that up to now only some classes of binary codes have been characterized by fuzzy sets suitable examples bcd and a linear code are given at the end of the paper together with necessary and sufficient conditions under which an r valued fuzzy set corresponds to a linear code

res regularized stochastic bfgs algorithm res a regularized stochastic version of the broyden fletcher goldfarb shanno bfgs quasi newton method is proposed to solve strongly convex optimization problems with stochastic objectives the use of stochastic gradient descent algorithms is widespread but the number of iterations required to approximate optimal arguments can be prohibitive in high dimensional problems application of second order methods on the other hand is impracticable because the computation of objective function hessian inverses incurs excessive computational cost bfgs modifies gradient descent by introducing a hessian approximation matrix computed from finite gradient differences res utilizes stochastic gradients in lieu of deterministic gradients for both the determination of descent directions and the approximation of the objective function s curvature since stochastic gradients can be computed at manageable computational cost res is realizable and retains the convergence rate advantages of its deterministic counterparts convergence results show that lower and upper bounds on the hessian eigenvalues of the sample functions are sufficient to guarantee almost sure convergence of a subsequence generated by res and convergence of the sequence in expectation to optimal arguments numerical experiments showcase reductions in convergence time relative to stochastic gradient descent algorithms and non regularized stochastic versions of bfgs an application of res to the implementation of support vector machines is developed

analysis of a general recursive prediction error identification algorithm a general class of algorithms for recursive identification of stochastic dynamical systems is studied in this class the discrepancy between the measured output and the output predicted from previous data according to a candidate model the prediction error is minimized over the model set using a stochastic approximation approach it is proved that this class of methods has the same convergence properties as its off line counterparts under mild and general assumptions these assumptions do not for example include stationary conditions or conditions that the true system can be exactly represented within the model set the considered class of methods contains as special cases several well known algorithms other common recursive identification methods like the extended least squares method and the extended kalman filter can be interpreted as approximate prediction error methods with simplified gradient calculations therefore the approach taken here may also serve as a basis for unified description of many recursive identification methods

meromorphic extension of the resolvent on complete spaces with asymptotically constant negative curvature abstract we give a detailed description of the schwartz kernel of the resolvent of the laplacian on a certain class of complete riemannian manifolds with negative sectional curvature near infinity these manifolds have compactifications as c manifolds with boundary on which the metric is conformai to a c metric up to the boundary with conformal factor 2 where is a defining function for the boundary the modified resolvent r s 2 n 1 defined and analytic in r n is shown to extend to be meromorphic in the whole complex plane when s 2 is equal to the sectional curvature at infinity for hyperbolic spaces obtained as quotients hn 1 where is a discrete group of fractional linear transformations having a geometrically finite fundamental domain without cusps at infinity s is constant we then deduce from the existence of this meromorphic extension of the resolvent the existence of a similar extension for the eisenstein series the basic method used to produce a parametrix for the laplacian is part of a more general construction by g mendoza and the second author but since this is not yet available the present paper has been made essentially self contained

stochastic approximation method with gradient averaging for unconstrained problems a stochastic approximation algorithm is studied in which random gradient estimates are averaged by an auxiliary filter convergence of the algorithm is proved under a rather general noise condition a practical version of the method is described and numerical results are given

the asymptotic distribution of lattice points in euclidean and non euclidean spaces the counting numbers for discrete subgroups of motions in euclidean and non euclidean spaces are obtained using the wave equation as the principal tool in dimensions 2 and 3 the error estimates are close to the best known

scattering theory for the wave equation on a hyperbolic manifold this talk is a survey of work done jointly mainly with peter lax 3 4 5 but also with bettina wilkott and alex woo 10 over the past ten years it deals with the spectrum of the perturbed and unperturbed laplace beltrami operator acting on automorphic functions on an n dimensional hyperbolic space ihn the associated discrete subgroup of motions is assumed to have the finite geometric property but is otherwise unrestricted this means that the fundamental domain when derived by the polygonal method has a finite number of sides its volume may be finite or infinite and it can have cusps of arbitrary rank with the obvious identifications the fundamental domain can be treated as a manifold m

on convergence of the stochastic subgradient method with on line stepsize rules abstract a stochastic subgradient method for solving convex stochastic programming problems is considered new on line rules for determining stepsizes are derived from the concept of local regularized improvement functions convergence with probability 1 of the whole method is established

smoothed conditional perturbation analysis of discrete event dynamical systems a new sample path analysis approach based on the smoothing property of conditional expectation for estimating the performance sensitivity of discrete event dynamical systems is proposed several examples are presented to show how this approach overcomes a difficulty of the ordinary infinitesimal perturbation analysis the basic message is that one can get more knowledge about the system performance by observing and analyzing the sample path than by using the conventional simulation approach it is also pointed out that the classical queueing theory approach for getting the performance sensitivity and the sample path based infinitesimal perturbation analysis approach can be unified in the framework of the new approach the smoothed conditional perturbation analysis

on convergence of the recursive identification algorithms the paper is concerned with identification of a linear dynamic system through recursive algorithms of the approximate maximum likelihood aml method in all papers on the convergence of the recursive aml procedure no matter what method of analysis is used a certain rather restrictive positive real condition is imposed on the model of disturbance in this work it is shown that recursive aml algorithms converge with probability one under a fairly broad also positive real condition which significantly expands the aml application area the well known principle of averaging from physics is used as a tool of analysis a sketch of the proof of the principle of averaging for stochastic difference equations is given

spectral analysis of second order elliptic operators on noncompact manifolds on demontre l estimation de mourre et les bornes qui y sont reliees pour une classe d operateurs elliptiques qui inclut des laplaciens pour des quotients cofinis et coinfinis d espaces hyperboliques

a distributed algorithm for convex constrained optimization under noise we present a novel distributed algorithm for convex constrained optimization problems that are subject to noise corruption and uncertainties the proposed scheme can be classified as a distributed stochastic approximation method where a unique feature here is that we allow for multiple noise terms to appear in both the computation and communication stages of the distributed iterative process specifically we consider problems that involve multiple agents optimizing a separable convex objective function subject to convex local constraints and linear coupling constraints this is a richer class of problems compared to those that can be handled by existing distributed stochastic approximation methods which consider only consensus constraints and fewer sources of noise the proposed algorithm utilizes the augmented lagrangian al framework which has been widely used recently to solve deterministic optimization problems in a distributed way we show that the proposed method generates sequences of primal and dual variables that converge to their respective optimal sets almost surely

on stochastic gradient and subgradient methods with adaptive steplength sequences traditionally stochastic approximation sa schemes have been popular choices for solving stochastic optimization problems however the performance of standard sa implementations can vary significantly based on the choice of the steplength sequence and in general little guidance is provided about good choices motivated by this gap we present two adaptive steplength schemes for strongly convex differentiable stochastic optimization problems equipped with convergence theory that aim to overcome some of the reliance on user specific parameters the first scheme referred to as a recursive steplength stochastic approximation rsa scheme optimizes the error bounds to derive a rule that expresses the steplength at a given iteration as a simple function of the steplength at the previous iteration and certain problem parameters the second scheme termed as a cascading steplength stochastic approximation csa scheme maintains the steplength sequence as a piecewise constant decreasing function with the reduction in the steplength occurring when a suitable error threshold is met then we allow for nondifferentiable objectives but with bounded subgradients over a certain domain in such a regime we propose a local smoothing technique based on random local perturbations of the objective function that leads to a differentiable approximation of the function assuming a uniform distribution on the local randomness we establish a lipschitzian property for the gradient of the approximation and prove that the obtained lipschitz bound grows at a modest rate with problem size this facilitates the development of an adaptive steplength stochastic approximation framework which now requires sampling in the product space of the original measure and the artificially introduced distribution

scattering for schrodinger operators in a class of domains with non compact boundaries abstract schrodinger operators in a class of domains with asymptotic cones are considered a generalized fourier transform representing the absolutely continuous part of the schrodinger operator as multiplication by 2 in the asymptotic cone is constructed wave operators relating the free laplacian to schrodinger operators are computed using the generalized fourier transform the wave operators relating schrodinger operators acting in domains with the same asymptotic cone are computed and shown to be complete

assessing fuzzy sets and the potential of possibility theory the alpha cut is generalized to depend on relations other than greater than or equal to various measures of fuzziness and specificity are also generalized as assessment functions a consistent notation and terminology are developed possibility theory is introduced and the assessment functions are recast as potential functions the nonspecificity function is seen to have special relevance

a mourre estimate and related bounds for hyperbolic manifolds with cusps of non maximal rank we consider the laplace operator on quotients of hyperbolic n dimensional space by a geometrically finite discrete group of hyperbolic isometries with parabolic subgroups of non maximal rank using methods developed by the first two authors we prove a mourre estimate and commutator estimates on the laplacian which imply absolute continuity of the spectrum and quantitative resolvent estimates these estimates will be used elsewhere to study the scattering matrix and eisenstein series and their meromorphic continuations and should be useful in studying trace formulas for these discrete groups

incoming and outgoing waves on etudie des espaces de solutions de l equation d onde u tt lu 0 ou lf f xx qf qui apparait dans la theorie de la diffusion de lax phillips

the laplace operator on a hyperbolic manifold i spectral and scattering theory using techniques of stationary scattering theory for the schrodinger equation we show absence of singular spectrum and obtain incoming and outgoing spectral representations for the laplace beltrami operator on manifolds mn arising as the quotient of hyperbolic n dimensional space by a geometrically finite discrete group of hyperbolic isometries we consider manifolds mn of infinite volume in subsequent papers we will use the techniques developed here to analytically continue eisenstein series for a large class of discrete groups including some groups with parabolic elements

convergence of parameter sensitivity estimates in a stochastic experiment to reduce the error in estimating the gradient parameter sensitivity of an unknown function is of great importance in stochastic optimization problems three kinds of parameter sensitivity estimates using the monte carlo method are discussed in this paper the estimates depend on the number of replications n and the change in parameter delta d the convergence properties as n rightarrow infty and delta d rightarrow 0 for these estimates are obtained the result explains many theoretical and practical issues in the study of discrete event dynamic systems as well as continuous dynamic systems by the monte carlo method it is proved that an estimate based on averaging the gradients calculated along each sample path by a perturbation of the path is much better than the other estimates if the output functions are uniformly differentiable with probability one w p 1 it is also concluded that in computer simulations one should always choose the same seed for both d and d delta d in estimating the parameter sensitivity combining the results in this paper with existing stochastic approximation algorithms may yield algorithms with faster convergence

unitary groups and spectral sets abstract we study spectral theory for bounded borel subsets of r and in particular finite unions of intervals for hilbert space we take l 2 of the union of the intervals this yields a boundary value problem arising from the minimal operator d 1 2 i d d x with domain consisting of c functions vanishing at the endpoints we offer a detailed interplay between geometric configurations of unions of intervals and a spectral theory for the corresponding self adjoint extensions of d and for the associated unitary groups of local translations while motivated by scattering theory and quantum graphs our present focus is on the fuglede spectral pair problem stated more generally this problem asks for a determination of those bounded borel sets in r k such that l 2 has an orthogonal basis of fourier frequencies spectrum i e a total set of orthogonal complex exponentials restricted to in the general case we characterize borel sets having this spectral property in terms of a unitary representation of r acting by local translations the case of k 1 is of special interest hence the interval configurations we give a characterization of those geometric interval configurations which allow fourier spectra directly in terms of the self adjoint extensions of the minimal operator d this allows for a direct and explicit interplay between geometry and spectra

multivariate stochastic approximation using a simultaneous perturbation gradient approximation the problem of finding a root of the multivariate gradient equation that arises in function minimization is considered when only noisy measurements of the function are available a stochastic approximation sa algorithm for the general kiefer wolfowitz type is appropriate for estimating the root the paper presents an sa algorithm that is based on a simultaneous perturbation gradient approximation instead of the standard finite difference approximation of keifer wolfowitz type procedures theory and numerical experience indicate that the algorithm can be significantly more efficient than the standard algorithms in large dimensional problems

acceleration of stochastic approximation by averaging a new recursive algorithm of stochastic approximation type with the averaging of trajectories is investigated convergence with probability one is proved for a variety of classical optimization and identification problems it is also demonstrated for these problems that the proposed algorithm achieves the highest possible rate of convergence

global convergence of online limited memory bfgs global convergence of an online stochastic limited memory version of the broyden fletcher goldfarb shanno bfgs quasi newton method for solving optimization problems with stochastic objectives that arise in large scale machine learning is established lower and upper bounds on the hessian eigenvalues of the sample functions are shown to suffice to guarantee that the curvature approximation matrices have bounded determinants and traces which in turn permits establishing convergence to optimal arguments with probability 1 experimental evaluation on a search engine advertising problem showcase reductions in convergence time relative to stochastic gradient descent algorithms

generalized representation and extension a very general abstraction of a fuzzy set is presented appropriate conditions for representation and extension are presented some insight into the fundamental nature of fuzzy sets is provided