a deterministic analysis of stochastic approximation with randomized directions we study the convergence of two stochastic approximation algorithms with randomized directions the simultaneous perturbation stochastic approximation algorithm and the random direction kiefer wolfowitz algorithm we establish deterministic necessary and sufficient conditions on the random directions and noise sequences for both algorithms and these conditions demonstrate the effect of the random directions on the sample path behavior of the algorithms studied we discuss ideas for further research in the analysis and design of these algorithms

the em algorithm an old folk song sung to a fast new tune celebrating the 20th anniversary of the presentation of the paper by dempster laird and rubin which popularized the em algorithm we investigate after a brief historical account strategies that aim to make the em algorithm converge faster while maintaining its simplicity and stability e g automatic monotone convergence in likelihood first we introduce the idea of a working parameter to facilitate the search for efficient data augmentation schemes and thus fast em implementations second summarizing various recent extensions of the em algorithm we formulate a general alternating expectation conditional maximization algorithm aecm that couples flexible data augmentation schemes with model reduction schemes to achieve efficient computations we illustrate these methods using multivariate t models with known or unknown degrees of freedom and poisson models for image reconstruction we show through both empirical and theoretical evidence the potential for a dramatic reduction in computational time with little increase in human effort we also discuss the intrinsic connection between em type algorithms and the gibbs sampler and the possibility of using the techniques presented here to speed up the latter the main conclusion of the paper is that with the help of statistical considerations it is possible to construct algorithms that are simple stable and fast

multidimensional gains for stochastic approximation this paper deals with iterative jacobian based recursion technique for the root finding problem of the vector valued function whose evaluations are contaminated by noise instead of a scalar step size we use an iterate dependent matrix gain to effectively weigh the different elements associated with the noisy observations the analytical development of the matrix gain is built on an iterative dependent linear function interfered by additive zero mean white noise where the dimension of the function is m ge 1 and the dimension of the unknown variable is n ge 1 necessary and sufficient conditions for m ge n algorithms are presented pertaining to algorithm stability and convergence of the estimate error covariance matrix two algorithms are proposed one for the case where m ge n and the second one for the antithesis the two algorithms assume full knowledge of the jacobian the recursive algorithms are proposed for generating the optimal iterative dependent matrix gain the proposed algorithms here aim for per iteration minimization of the mean square estimate error we show that the proposed algorithm satisfies the presented conditions for stability and convergence of the covariance in addition the convergence rate of the estimation error covariance is shown to be inversely proportional to the number of iterations for the antithesis m contraction of the error covariance is guaranteed this underdetermined system of equations can be helpful in training neural networks numerical examples are presented to illustrate the performance capabilities of the proposed multidimensional gain while considering nonlinear functions

validation analysis of mirror descent stochastic approximation method the main goal of this paper is to develop accuracy estimates for stochastic programming problems by employing stochastic approximation sa type algorithms to this end we show that while running a mirror descent stochastic approximation procedure one can compute with a small additional effort lower and upper statistical bounds for the optimal objective value we demonstrate that for a certain class of convex stochastic programs these bounds are comparable in quality with similar bounds computed by the sample average approximation method while their computational cost is considerably smaller

on a multistage discrete stochastic optimization problem with stochastic constraints and nested sampling we consider a multistage stochastic discrete program in which constraints on any stage might involve expectations that cannot be computed easily and are approximated by simulation we study a sample average approximation saa approach that uses nested sampling in which at each stage a number of scenarios are examined and a number of simulation replications are performed for each scenario to estimate the next stage constraints this approach provides an approximate solution to the multistage problem to establish the consistency of the saa approach we first consider a two stage problem and show that in the second stage problem given a scenario the optimal values and solutions of the saa converge to those of the true problem with probability one when the sample sizes go to infinity these convergence results do not hold uniformly over all possible scenarios for the second stage problem we are nevertheless able to prove that the optimal values and solutions of the saa converge to the true ones with probability one when the sample sizes at both stages increase to infinity we also prove exponential convergence of the probability of a large deviation for the optimal value of the saa the true value of an optimal solution of the saa and the probability that any optimal solution to the saa is an optimal solution of the true problem all of these results can be extended to a multistage setting and we explain how to do it our framework and saa results cover a large variety of resource allocation problems for which at each stage after the first one new information becomes available and the allocation can be readjusted under constraints that involve expectations estimated by monte carlo as an illustration we apply this saa method to a staffing problem in a call center in which the goal is to optimize the numbers of agents of each type under some constraints on the quality of service qos the staffing allocation has to be decided under an uncertain arrival rate with a prior distribution in the first stage and can be adjusted at some additional cost when better information on the arrival rate becomes available in later stages

time to absorption for a heterogeneous neutral competition model neutral models aspire to explain biodiversity patterns in ecosystems where species difference can be neglected and perfect symmetry is assumed between species voter like models capture the essential ingredients of the neutral hypothesis and represent a paradigm for other disciplines like social studies and chemical reactions in a system where each individual can interact with all the other members of the community the typical time to reach an absorbing state with a single species scales linearly with the community size here we show by using a rigorous approach based on a large deviation principle and confirming previous approximate and numerical results that in a mean field heterogeneous voter model the typical time to reach an absorbing state scales exponentially with the system size

variance based extragradient methods with line search for stochastic variational inequalities in this paper we propose dynamic sampled stochastic approximated ds sa extragradient methods for stochastic variational inequalities svis that are robust with respect to an unknown lipschitz c

lms algorithms for tracking slow markov chains with applications to hidden markov estimation and adaptive multiuser detection this paper analyzes the tracking properties of the least mean squares lms algorithm when the underlying parameter evolves according to a finite state markov chain with infrequent jumps first using perturbed liapunov function methods mean square error estimates are obtained for the tracking error then using recent results on two time scale markov chains mean ordinary differential equation and diffusion approximation results are obtained it is shown that a sequence of the centered tracking errors converges to an ordinary differential equation moreover a suitably scaled sequence of the tracking errors converges weakly to a diffusion process it is also shown that iterate averaging of the tracking algorithm results in optimal asymptotic convergence rate in an appropriate sense two application examples analysis of the performance of an adaptive multiuser detection algorithm in a direct sequence code division multiple access ds cdma system and tracking analysis of the state of a hidden markov model hmm with infrequent jumps are presented

on the choice of random directions for stochastic approximation algorithms stochastic approximation provides a simple and effective approach for finding roots and minima of functions whose evaluations are contaminated with noise we investigate variants of the random direction stochastic approximation rdsa algorithm for optimizing noisy loss functions in high dimensional spaces the most popular variant is random selection from a bernoulli distribution which also goes by the name simultaneous perturbation stochastic approximation spsa viable alternatives include an axis aligned distribution normal distribution and uniform distribution on a spherical shell although there are special cases where the bernoulli distribution is optimal we identify other cases where it performs worse than the alternatives we show that performance depends on the orientation of the loss function with respect to its coordinate axes and consider averages over all orientations we find that the average asymptotic performance depends only on the radial fourth moment of the distribution of random directions and is identical for the bernoulli axis aligned and spherical shell distributions of these variants the spherical shell is optimal in the sense of minimum variance over random orientations of the loss function with respect to the coordinate axes we also show that for unaligned loss functions the performance of the kiefer wolfowitz blum finite difference stochastic approximation fdsa is asymptotically equivalent to the rdsa algorithms and we observe numerically that the pre asymptotic performance of fdsa is often superior

robust stochastic approximation approach to stochastic programming in this paper we consider optimization problems where the objective function is given in a form of the expectation a basic difficulty of solving such stochastic optimization problems is that the involved multidimensional integrals expectations cannot be computed with high accuracy the aim of this paper is to compare two computational approaches based on monte carlo sampling techniques namely the stochastic approximation sa and the sample average approximation saa methods both approaches the sa and saa methods have a long history current opinion is that the saa method can efficiently use a specific say linear structure of the considered problem while the sa approach is a crude subgradient method which often performs poorly in practice we intend to demonstrate that a properly modified sa approach can be competitive and even significantly outperform the saa method for a certain class of convex stochastic problems we extend the analysis to the case of convex concave stochastic saddle point problems and present in our opinion highly encouraging results of numerical experiments

an optimal method for stochastic composite optimization this paper considers an important class of convex programming cp problems namely the stochastic composite optimization sco whose objective function is given by the summation of general nonsmooth and smooth stochastic components since sco covers non smooth smooth and stochastic cp as certain special cases a valid lower bound on the rate of convergence for solving these problems is known from the classic complexity theory of convex programming note however that the optimization algorithms that can achieve this lower bound had never been developed in this paper we show that the simple mirror descent stochastic approximation method exhibits the best known rate of convergence for solving these problems our major contribution is to introduce the accelerated stochastic approximation ac sa algorithm based on nesterov s optimal method for smooth cp nesterov in doklady an sssr 269 543 547 1983 nesterov in math program 103 127 152 2005 and show that the ac sa algorithm can achieve the aforementioned lower bound on the rate of convergence for sco to the best of our knowledge it is also the first universally optimal algorithm in the literature for solving non smooth smooth and stochastic cp problems we illustrate the significant advantages of the ac sa algorithm over existing methods in the context of solving a special but broad class of stochastic programming problems

analytical solution of a neutral model of biodiversity the unified neutral model of biodiversity proposed by s hubbell is solved analytically the distributions of species abundance in the metacommunity and in a local community are calculated exactly as a function of speciation and migration rates and of the size of the community in the limit of large population sizes the densities of species of given relative abundance are found to be given by universal functions depending only on two parameters

a central limit theorem and hypotheses testing for risk averse stochastic programs we study statistical properties of the optimal value and optimal solutions of the sample average approximation of risk averse stochastic problems central limit theorem type results are derived for the optimal value when the stochastic program is expressed in terms of a law invariant coherent risk measure having a discrete kusuoka representation the obtained results are applied to hypotheses testing problems aiming at comparing the optimal values of several risk averse convex stochastic programs on the basis of samples of the underlying random vectors we also consider nonasymptotic tests based on confidence intervals on the optimal values of the stochastic programs obtained using the stochastic mirror descent algorithm numerical simulations show how to use our developments to choose among different distributions and on the considered class of risk averse stochastic programs the asymptotic tests show better results

a one measurement form of simultaneous perturbation stochastic approximation the simultaneous perturbation stochastic approximation spsa algorithm has proven very effective for difficult multivariate optimization problems where it is not possible to obtain direct gradient information as discussed to date spsa is based on a highly efficient gradient approximation requiring only two measurements of the loss function independent of the number of parameters being estimated this note presents a form of spsa that requires only one function measurement for any dimension theory is presented that identifies the class of problems for which this one measurement form will be asymptotically superior to the standard two measurement form

online estimation of the geometric median in hilbert spaces nonasymptotic confidence balls estimation procedures based on recursive algorithms are interesting and powerful techniques that are able to deal rapidly with very large samples of high dimensional data the collected data may be contaminated by noise so that robust location indicators such as the geometric median may be preferred to the mean in this context an estimator of the geometric median based on a fast and efficient averaged nonlinear stochastic gradient algorithm has been developed by bernoulli19 2013 18 43 this work aims at studying more precisely the nonasymptotic behavior of this nonlinear algorithm by giving nonasymptotic confidence balls in general separable hilbert spaces this new result is based on the derivation of improved l 2 rates of convergence as well as an exponential inequality for the nearly martingale terms of the recursive nonlinear robbins monro algorithm

accelerated gradient methods for nonconvex nonlinear and stochastic programming in this paper we generalize the well known nesterov s accelerated gradient ag method originally designed for convex smooth optimization to solve nonconvex and possibly stochastic optimization problems we demonstrate that by properly specifying the stepsize policy the ag method exhibits the best known rate of convergence for solving general nonconvex smooth optimization problems by using first order information similarly to the gradient descent method we then consider an important class of composite optimization problems and show that the ag method can solve them uniformly i e by using the same aggressive stepsize policy as in the convex case even if the problem turns out to be nonconvex we demonstrate that the ag method exhibits an optimal rate of convergence if the composite problem is convex and improves the best known rate of convergence if the problem is nonconvex based on the ag method we also present new nonconvex stochastic approximation methods and show that they can improve a few existing rates of convergence for nonconvex stochastic optimization to the best of our knowledge this is the first time that the convergence of the ag method has been established for solving nonconvex nonlinear programming in the literature

on the design of a stable adaptive filter for state estimation in high dimensional systems design of specific parametrized gain structures provides a new tool for ensuring stability of reduced order filter for high dimensional systems which is optimized adaptively in the minimum prediction error framework

asymptotic and finite sample properties of estimators based on stochastic gradients stochastic gradient descent procedures have gained popularity for parameter estimation from large data sets however their statistical properties are not well understood in theory and in practice avoiding numerical instability requires careful tuning of key parameters here we introduce implicit stochastic gradient descent procedures which involve parameter updates that are implicitly defined intuitively implicit updates shrink standard stochastic gradient descent updates the amount of shrinkage depends on the observed fisher information matrix which does not need to be explicitly computed thus implicit procedures increase stability without increasing the computational burden our theoretical analysis provides the first full characterization of the asymptotic behavior of both standard and implicit stochastic gradient descent based estimators including finite sample error bounds importantly analytical expressions for the variances of these stochastic gradient based estimators reveal their exact loss of efficiency we also develop new algorithms to compute implicit stochastic gradient descent based estimators for generalized linear models cox proportional hazards m estimators in practice and perform extensive experiments our results suggest that implicit stochastic gradient descent procedures are poised to become a workhorse for approximate inference from large data sets

optimal bayesian experimental design for priors of compact support with application to shock tube experiments for combustion kinetics summary the analysis of reactive systems in combustion science and technology relies on detailed models comprising many chemical reactions that describe the conversion of fuel and oxidizer into products and the formation of pollutants shock tube experiments are a convenient setting for measuring the rate parameters of individual reactions the temperature pressure and concentration of reactants are chosen to maximize the sensitivity of the measured quantities to the rate parameter of the target reaction in this study we optimize the experimental setup computationally by optimal experimental design in a bayesian framework we approximate the posterior probability density functions pdf using truncated gaussian distributions in order to account for the bounded domain of the uniform prior pdf of the parameters the underlying gaussian distribution is obtained in the spirit of the laplace method more precisely the mode is chosen as the maximum a posteriori map estimate and the covariance is chosen as the negative inverse of the hessian of the misfit function at the map estimate the model related entities are obtained from a polynomial surrogate the optimality quantified by the information gain measures can be estimated efficiently by a rejection sampling algorithm against the underlying gaussian probability distribution rather than against the true posterior this approach offers a significant error reduction when the magnitude of the invariants of the posterior covariance are comparable with the size of the bounded domain of the prior we demonstrate the accuracy and superior computational efficiency of our method for shock tube experiments aiming to measure the model parameters of a key reaction which is part of the complex kinetic network describing the hydrocarbon oxidation in the experiments the initial temperature and fuel concentration are optimized with respect to the expected information gain in the estimation of the parameters of the target reaction rate we show that the expected information gain surface can change its shape dramatically according to the level of noise introduced into the synthetic data the information that can be extracted from the data saturates as a logarithmic function of the number of experiments and few experiments are needed when they are conducted at the optimal experimental design conditions furthermore inversion of the legacy data indicates the validity and robustness of our designs copyright 2016 john wiley sons ltd

stochastic approximation method with gradient averaging for unconstrained problems a stochastic approximation algorithm is studied in which random gradient estimates are averaged by an auxiliary filter convergence of the algorithm is proved under a rather general noise condition a practical version of the method is described and numerical results are given

simulation based optimization with stochastic approximation using common random numbers the method of common random numbers is a technique used to reduce the variance of difference estimates in simulation optimization problems these differences are commonly used to estimate gradients of objective functions as part of the process of determining optimal values for parameters of a simulated system asymptotic results exist which show that using the common random numbers method in the iterative finite difference stochastic approximation optimization algorithm fdsa can increase the optimal rate of convergence of the algorithm from the typical rate of k 1 3 to the faster k 1 2 where k is the algorithm s iteration number simultaneous perturbation stochastic approximation spsa is a newer and often much more efficient optimization algorithm and we will show that this algorithm too converges faster when the common random numbers method is used we will also provide multivariate asymptotic covariance matrices for both the spsa and fdsa errors

analysis and optimization of diagonally layered lattice schemes for mimo fading channels embodiments of the diagonal bell laboratories layered space time d blast architecture for multiple input multiple output mimo communication are developed wherein information symbol vectors are encoded using codewords from a lattice code called a diagonally layered lattice dll code which are formatted onto the diagonals of a space time frame decoding is done using a sphere decoder for each diagonal based on soft statistics obtained after zero forcing zf or minimum mean square error mmse filtering and decision feedback these operations give rise to an effective parallel channel model with channel gains with nonidentical statistics and additive noise which is gaussian in the zf filtering case and non gaussian in the mmse filtering case the so called full modulation diversity fmd property is nevertheless shown to yield the maximum achievable diversity orders over the mimo channel for both the zf and the mmse filtering based decoders respectively for any arbitrary fading distribution in the case of the independent identically distributed i i d rayleigh fading mimo channel with k transmit and n receive antennas with ngesk these diversity orders are nk k k 1 2 and nk for zf and mmse filtering based decoding respectively the error probability analysis also yields a design criterion for optimizing transmit power allocations several lattice design methods are proposed for the effective parallel channel models two methods are proposed to achieve high coding gain in the rayleigh fading mimo channel a third method is proposed that minimizes the exact symbol error probability sep and can be tailored for any given fading distribution a novel soft decision feedback decoder is also proposed based on the list sphere decoder to mitigate error propagation due to hard decision feedback the salient feature of the proposed dll schemes is that they have nearly full rate and full or high diversity order and yet a much lower decoding complexity than other existing full rate full diversity space time block codes stbcs the frame error probability fep performance of the optimized dll schemes for moderate to high spectral efficiencies and a wide range of signal to noise ratios snrs can be quite close to the performance of the best performing but more complex to decode stbcs moreover the proposed dll schemes significantly outperform other existing mimo systems of comparable decoding complexity

multistep stochastic mirror descent for risk averse convex stochastic programs based on extended polyhedral risk measures we consider risk averse convex stochastic programs expressed in terms of extended polyhedral risk measures we derive computable confidence intervals on the optimal value of such stochastic programs using the robust stochastic approximation and the stochastic mirror descent smd algorithms when the objective functions are uniformly convex we also propose a multistep extension of the stochastic mirror descent algorithm and obtain confidence intervals on both the optimal values and optimal solutions numerical simulations show that our confidence intervals are much less conservative and are quicker to compute than previously obtained confidence intervals for smd and that the multistep stochastic mirror descent algorithm can obtain a good approximate solution much quicker than its nonmultistep counterpart

convergence and efficiency of adaptive importance sampling techniques with partial biasing we consider a generalization of the discrete time self healing umbrella sampling method which is an adaptive importance technique useful to sample multimodal target distributions the importance function is based on the weights of disjoint sets which form a partition of the space in the context of computational statistical physics the logarithm of these weights is up to a multiplicative constant the free energy and the discrete valued function defining the partition is called the reaction coordinate the algorithm is a generalization of the original self healing umbrella sampling method in two ways i the updating strategy leads to a larger penalization strength of already visited sets and ii the target distribution is biased using only a fraction of the free energy in order to increase the effective sample size and reduce the variance of importance sampling estimators the algorithm can also be seen as a generalization of well tempered metadynamics we prove the convergence of the algorithm and analyze numerically its efficiency on a toy example

stochastic approximation with averaging of the iterates optimal asymptotic rate of convergence for general processes consider the stochastic approximation algorithm x n 1 x n a n g x n xi n in an important paper polyak and juditsky siam j control optim 30 1992 pp 838 855 showed that loosely speaking if the coefficients a n go to zero slower than o 1 n then the averaged sequence sum nolimits i 1 n x i n converged to its limit at an optimum rate for any coefficient sequence the conditions were rather special and direct constructions were used here a rather simple proof is given that results of this type are generic to stochastic approximation and essentially hold any time that the classical asymptotic normality of the normalized and centered iterates holds considerable intuitive insight is provided into the procedure simulations have well borne out the importance of the method

ascent based monte carlo expectation maximization summary the expectation maximization em algorithm is a popular tool for maximizing likelihood functions in the presence of missing data unfortunately em often requires the evaluation of analytically intractable and high dimensional integrals the monte carlo em mcem algorithm is the natural extension of em that employs monte carlo methods to estimate the relevant integrals typically a very large monte carlo sample size is required to estimate these integrals within an acceptable tolerance when the algorithm is near convergence even if this sample size were known at the onset of implementation of mcem its use throughout all iterations is wasteful especially when accurate starting values are not available we propose a data driven strategy for controlling monte carlo resources in mcem the algorithm proposed improves on similar existing methods by recovering em s ascent i e likelihood increasing property with high probability being more robust to the effect of user defined inputs and handling classical monte carlo and markov chain monte carlo methods within a common framework because of the first of these properties we refer to the algorithm as ascent based mcem we apply ascent based mcem to a variety of examples including one where it is used to accelerate the convergence of deterministic em dramatically

least mean square algorithms with markov regime switching limit we analyze the tracking performance of the least mean square lms algorithm for adaptively estimating a time varying parameter that evolves according to a finite state markov chain we assume the markov chain jumps infrequently between the finite states at the same rate of change as the lms algorithm we derive mean square estimation error bounds for the tracking error of the lms algorithm using perturbed lyapunov function methods then combining results in two time scale markov chains with weak convergence methods for stochastic approximation we derive the limit dynamics satisfied by continuous time interpolation of the estimates unlike most previous analyzes of stochastic approximation algorithms the limit we obtain is a system of ordinary differential equations with regime switching controlled by a continuous time markov chain next to analyze the rate of convergence we take a continuous time interpolation of a scaled sequence of the error sequence and derive its diffusion limit somewhat remarkably for correlated regression vectors we obtain a jump markov diffusion finally two novel examples of the analysis are given for state estimation of hidden markov models hmms and adaptive interference suppression in wireless code division multiple access cdma networks

a direct search algorithm for optimization with noisy function evaluations we consider the unconstrained optimization of a function when each function evaluation is subject to a random noise we assume that there is some control over the variance of the noise term in the sense that additional computational effort will reduce the amount of noise this situation may occur when function evaluations involve simulation or the approximate solution of a numerical problem it also occurs in an experimental setting when averaging repeated observations at the same point can lead to a better estimate of the underlying function value we describe a new direct search algorithm for this type of problem we prove convergence of the new algorithm when the noise is controlled so that the standard deviation of the noise approaches zero faster than the step size we also report some numerical results on the performance of the new algorithm

fisher waves an individual based stochastic model the propagation of a beneficial mutation in a spatially extended population is usually studied using the phenomenological stochastic fisher kolmogorov petrovsky piscounov sfkpp equation we derive here an individual based stochastic model founded on the spatial moran process where fluctuations are treated exactly the mean field approximation of this model leads to an equation that is different from the phenomenological fkpp equation at small selection pressure the front behavior can be mapped into a brownian motion with drift the properties of which can be derived from the microscopic parameters of the moran model finally we generalize the model to take into account dispersal kernels beyond migration to nearest neighbors we show how the effective population size which controls the noise amplitude and the diffusion coefficient can both be computed from the dispersal kernel

how does a stochastic optimization approximation algorithm adapt to a randomly evolving optimum root with jump markov sample paths stochastic optimization approximation algorithms are widely used to recursively estimate the optimum of a suitable function or its root under noisy observations when this optimum or root is a constant or evolves randomly according to slowly time varying continuous sample paths in comparison this paper analyzes the asymptotic properties of stochastic optimization approximation algorithms for recursively estimating the optimum or root when it evolves rapidly with nonsmooth jump changing sample paths the resulting problem falls into the category of regime switching stochastic approximation algorithms with two time scales motivated by emerging applications in wireless communications and system identification we analyze asymptotic behavior of such algorithms our analysis assumes that the noisy observations contain a nonsmooth jump process modeled by a discrete time markov chain whose transition frequency varies much faster than the adaptation rate of the stochastic optimization algorithm using stochastic averaging we prove convergence of the algorithm rate of convergence of the algorithm is obtained via bounds on the estimation errors and diffusion approximations remarks on improving the convergence rates through iterate averaging and limit mean dynamics represented by differential inclusions are also presented

on line estimation of dynamic shock error models based on the kullback leibler information measure develops two sequential or on line estimation schemes in the time domain for dynamic shock error models which are special cases of errors in variables models the author s approach utilizes a state space representation of the model kalman filtering techniques and on line algorithms the first on line algorithm is based on the expectation maximization algorithm and uses a recursive gauss newton scheme to maximize the kullback leibler information measure the second on line algorithm the author proposes is a gradient based scheme and uses stochastic approximations to maximize the log likelihood in comparison to the off line maximum likelihood estimation scheme used in ghosh 1989 the author s on line algorithms have significantly reduced computational costs and negligible memory requirements simulations illustrate the satisfactory performance of the algorithms in estimating errors in variables systems with parameters that vary slowly with time or undergo infrequent jump changes

on sampling rates in simulation based recursions we consider the context of simulation based recursions that is recursions that involve quantities needing to be estimated using a stochastic simulation examples include stochastic adaptations of fixed point and gradient descent recursions obtained by replacing function and derivative values appearing within the recursion by their monte carlo counterparts the primary motivating settings are simulation optimization and stochastic root finding problems where the low point and the zero of a function are sought respectively with only monte carlo estimates of the functions appearing within the problem we ask how much monte carlo sampling needs to be performed within simulation based recursions in order that the resulting iterates remain consistent and more importantly efficient where efficient implies convergence at the fastest possible rate answering these questions involves trading off two types of error inherent in the iterates the deterministic error due to recursion and the stochastic er

a stochastic gradient type algorithm for closed loop problems we focus on the numerical solution of closed loop stochastic problems and propose a perturbed gradient algorithm to achieve this goal the main hurdle in such problems is the fact that the control variables are infinite dimensional due to e g the information constraints alternatively said control variables are feedbacks i e functions such controls have hence to be represented in a finite way in order to solve the problem numerically in the same way the gradient of the criterion is itself an infinite dimensional object our algorithm replaces this exact and unknown gradient by a perturbed one which consists of the product of the true gradient evaluated at a random point and a kernel function which extends this gradient to the neighbourhood of the random point proceeding this way we explore the whole space iteration after iteration through random points since each kernel function is perfectly known by a small number of parameters say n the control at iteration k is perfectly known as an infinite dimensional object by at most n k parameters the main strength of this method is that it avoids any discretization of the underlying space provided that we can sample as many points as needed in this space moreover our algorithm can take into account the possible measurability constraints of the problem in a new way finally the randomized strategy implemented by the algorithm causes the most probable parts of the space to be the most explored ones which is a priori an interesting feature in this paper we first prove two convergence results of this algorithm in the strongly convex and convex cases and then give some numerical examples showing the interest of this method for practical stochastic optimization problems

analysis of stochastic gradient algorithms for linear regression problems parameter estimation problems that can be formulated as linear regressions are quite common in many applications recursive on line sequential estimation of such parameters can be performed using the recursive least squares rls algorithm or a stochastic gradient version of this algorithm in this paper the convergence properties of the gradient algorithm are analyzed under the assumption that the gain tends to zero the technique is the same as the so called ordinary differential equation approach but the treatment here is self contained and includes a proof of the boundedness of the estimates a main result is that the convergence conditions for the gradient algorithm are the same as those for the recursive least squares algorithm

accelerated randomized stochastic optimization we propose a general class of randomized gradient estimates to be employed in a recursive search for the minimum of an unknown multivariate regression function here only two observations per iteration step are used special cases include random direction stochastic approximation kushner and clark simultaneous perturbation stochastic approximation spall and a special kernel based stochastic approximation method polyak and tsybakov if the unknown regression is p smooth p 2 at the point of minimum these methods achieve the optimal rate of convergence o n p 1 2p for both the classical stochastic approximation scheme kiefer and wolfowitz and the averaging scheme ruppert and polyak the related asymptotic distributions are computed

optimal primal dual methods for a class of saddle point problems we present a novel accelerated primal dual apd method for solving a class of deterministic and stochastic saddle point problems spps the basic idea of this algorithm is to incorporate a multistep acceleration scheme into the primal dual method without smoothing the objective function for deterministic spp the apd method achieves the same optimal rate of convergence as nesterov s smoothing technique our stochastic apd method exhibits an optimal rate of convergence for stochastic spp not only in terms of its dependence on the number of the iteration but also on a variety of problem parameters to the best of our knowledge this is the first time that such an optimal algorithm has been developed for stochastic spp in the literature furthermore for both deterministic and stochastic spp the developed apd algorithms can deal with the situation when the feasible region is unbounded as long as a saddle point exists in the unbounded case we incorporate the modified termination criterion introduced b

continuous probabilistic approach to species dynamics in hubbell s zero sum local community in this paper a continuous probabilistic approach formulated using kolmogorov fokker planck forward and backward models is applied to hubbell s zero sum neutral theory for species dynamics in local community using this technique the probability density of species abundance distribution of the first passage time to extinction or fixation and probability of extinction are defined the resulting values for the distribution of the first passage time to extinction are verified by the simulation study of hubbell s zero sum neutral model for the local community based on the sensitivity analysis for the continuous probabilistic models the realistic classification of local communities subject to their diversity and species dynamics is proposed with respect to the immigration probability the species metacommunity relative abundance and the size of local community doi 10 1103 physreve 74 021902 pacs numbers 87 23 cc 02 50 r the main goals of biodiversity study are to explain and quantify the distribution abundance and dynamics of living organisms in ecological communities these questions are also of central theoretical and practical importance in conservation biology and ecosystem management 1 2 since understanding the mechanisms of species abundance dynamics is of high concern in viability analysis species abundance relationships have long been studied by ecologists who defined them as species commonness and rarity in ecological communities 1 3 4 these relationships are usually described by species abundance distributions which show the number of species as the functions of their observed abundances early studies on species abundance relationships were focused on finding distributions that could fit well empirical data among the proposed distributions were the log series 5 and the log normal 6 later the preference was given to modeling species abundance relationships using one or another ecological theory of community organization using this approach the broken stick model was proposed by macarthur 7 8 the log normal patterns of species abundances were explained using the niche preemption model 9 11 and using the dynamic population model with gompertz density dependence 12 the gamma type of abundance distribution including fisher s log series the extended gamma distribution and macarthur s broken stick model was elucidated using the dynamic approach with logistic density dependence 13

stochastic approximation with averaging and feedback rapidly convergent on line algorithms consider the stochastic approximation x sub n 1 x sub n a sub n g x sub n spl xi sub n where 0 0 the averaging method is essentially off line in the sense that the actual sa iterate x sub n is not influenced by the averaging in many applications x sub n itself is of greatest interest since that is the operating parameter this paper deals with the problem of stochastic approximation with averaging and with appropriate feedback of the averages into the original algorithm it is shown both mathematically and via simulation that it works very well and has numerous advantages it is a clear improvement over the system x sub n by itself it is fairly robust and quite often it is much preferable to the use of the above averages without feedback the authors deal in particular with linear algorithms of the type appearing in parameter estimators adaptive noise cancellers channel equalizers adaptive control and similar applications the main development is for the constant parameter case because of its importance in applications but analogous results hold for the case where a sub n spl rarr 0

recursive k distribution parameter estimation recursive estimation of the parameter of the k distribution is studied and tested the probability density function pdf of the k distribution is seen as a mixture pdf allowing the application of titterington s recursive expectation maximization em technique under mild conditions the technique produces estimates that are strongly consistent and asymptotically normal for the k distribution the complete data information matrix required by the recursive em has an explicit form making the algorithm easy to implement the algorithm is tested using k distributed data with both constant and time varying parameter for the constant parameter case recursive em estimates are compared to numerical maximization of the likelihood for the time varying parameter case recursive em estimates are compared to estimates obtained using a fast routine from the literature and implemented by sliding a rectangular window over the observations

stochastic optimization using a trust region method and random models in this paper we propose and analyze a trust region model based algorithm for solving unconstrained stochastic optimization problems our framework utilizes random models of an objective function f x obtained from stochastic observations of the function or its gradient our method also utilizes estimates of function values to gauge progress that is being made the convergence analysis relies on requirements that these models and these estimates are sufficiently accurate with high enough but fixed probability beyond these conditions no assumptions are made on how these models and estimates are generated under these general conditions we show an almost sure global convergence of the method to a first order stationary point in the second part of the paper we present examples of generating sufficiently accurate random models under biased or unbiased noise assumptions lastly we present some computational results showing the benefits of the proposed method compared to existing approaches that are based on sample averaging or stochastic gradients

recursive algorithms for parameter estimation with adaptive quantizer this paper studies a parameter estimation problem of networked linear systems with fixed rate quantization under the minimum mean square error criterion we propose a recursive estimator of stochastic approximation type and derive a necessary and sufficient condition for its asymptotic unbiasedness this motivates to design an adaptive quantizer for the estimator whose strong consistency asymptotic unbiasedness and asymptotic normality are rigorously proved using the newton based and averaging techniques we obtain two accelerated recursive estimators with the fastest convergence speed of o 1 k and exactly evaluate the quantization effect on the estimation accuracy if the observation noise is gaussian an optimal quantizer and the accelerated estimators are co designed to asymptotically approach the minimum cramer rao lower bound all the estimators share almost the same computational complexity as the gradient algorithms with un quantized observations and can be easily implemented finally the theoretical results are validated by simulations

convergence rate of moments in stochastic approximation with simultaneous perturbation gradient approximation and resetting the sequence of recursive estimators for function minimization generated by spall s 1998 simultaneous perturbation stochastic approximation spsa method combined with a suitable restarting mechanism is considered it is proved that this sequence converges under certain conditions with rate o n sup spl beta 2 for some spl beta 0 the best value being spl beta 2 3 where the rate is measured by the l sub q norm of the estimation error for any 1 spl les q spl infin the authors also present higher order spsa methods it is shown that the error exponent spl beta 2 can be arbitrarily close to 1 2 if the hessian matrix of the cost function at the minimizing point has all its eigenvalues to the right of 1 2 the cost function is sufficiently smooth and a sufficiently high order approximation of the derivative is used

a least angle regression control chart for multidimensional data in multidimensional applications it is very rare that all variables shift at the same time a statistical process control procedure would have superior efficiency when limited to the subset of variables likely responsible for the out of control conditions the key idea of this article consists of combining a variable selection method with a multivariate control chart to detect changes in both the mean and variability of a multidimensional process with gaussian errors in particular we develop a control chart for phase ii monitoring which integrates the least angle regression algorithm with a multivariate exponentially weighted moving average comparisons with related multivariate control schemes demonstrate the efficiency of the proposed control chart in a wide range of practical applications including profile and multistage process monitoring further the proposed scheme may also provide valuable diagnostic information for fault isolation supplemental materials including an r package are available

statistical inference for the population landscape via moment adjusted stochastic gradients modern statistical inference tasks often require iterative optimization methods to compute the solution convergence analysis from an optimization viewpoint informs us only how well the solution is approximated numerically but overlooks the sampling nature of the data in contrast recognizing the randomness in the data statisticians are keen to provide uncertainty quantification or confidence for the solution obtained by using iterative optimization methods the paper makes progress along this direction by introducing moment adjusted stochastic gradient descent a new stochastic optimization method for statistical inference we establish non asymptotic theory that characterizes the statistical distribution for certain iterative methods with optimization guarantees on the statistical front the theory allows for model misspecification with very mild conditions on the data for optimization the theory is flexible for both convex and non convex cases remarkably the moment adjusting idea motivated from error standardization in statistics achieves a similar effect to acceleration in first order optimization methods that are used to fit generalized linear models we also demonstrate this acceleration effect in the non convex setting through numerical experiments

weak convergence properties of constrained emphatic temporal difference learning with constant and slowly diminishing stepsize we consider the emphatic temporal difference td algorithm etd for learning the value functions of stationary policies in a discounted finite state and action markov decision process the etd algorithm was recently proposed by sutton mahmood and white 2016 to solve a long standing divergence problem of the standard td algorithm when it is applied to off policy training where data from an exploratory policy are used to evaluate other policies of interest the almost sure convergence of etd has been proved in our recent work under general off policy training conditions but for a narrow range of diminishing stepsize in this paper we present convergence results for constrained versions of etd with constant stepsize and with diminishing stepsize from a broad range our results characterize the asymptotic behavior of the trajectory of iterates produced by those algorithms and are derived by combining key properties of etd with powerful convergence theorems from the weak convergence methods in stochastic approximation theory for the case of constant stepsize in addition to analyzing the behavior of the algorithms in the limit as the stepsize parameter approaches zero we also analyze their behavior for a fixed stepsize and bound the deviations of their averaged iterates from the desired solution these results are obtained by exploiting the weak feller property of the markov chains associated with the algorithms and by using ergodic theorems for weak feller markov chains in conjunction with the convergence results we get from the weak convergence methods besides etd our analysis also applies to the off policy td algorithm when the divergence issue is avoided by setting sufficiently large it yields for that case new results on the asymptotic convergence properties of constrained off policy td with constant or slowly diminishing stepsize

adaptive learning vector quantization for online parametric estimation this paper addresses the problem of parameter estimation in a quantized and online setting a sensing unit collects random vector valued samples from the environment these samples are quantized and transmitted to a central processor which generates an online estimate of the unknown parameter this paper provides a closed form expression of the excess mean square error mse caused by quantization in the high rate regime i e when the number of quantization levels is supposed to be large next we determine the quantizers which mitigate the excess mse the optimal quantization rule unfortunately depends on the unknown parameter to circumvent this issue we introduce a novel adaptive learning vector quantization scheme which allows to simultaneously estimate the parameter of interest and select an efficient quantizer

a parallel decomposition method for nonconvex stochastic multi agent optimization problems this paper considers the problem of minimizing the expected value of a possibly nonconvex cost function parameterized by a random vector variable when the expectation cannot be computed accurately e g because the statistics of the random variables are unknown and or the computational complexity is prohibitive classical stochastic gradient methods for solving this problem may suffer from slow convergence in this paper we propose a stochastic parallel successive convex approximation based best response algorithm for general nonconvex stochastic sum utility optimization problems which arise naturally in the design of multi agent networks the proposed novel decomposition approach enables all users to update their optimization variables in parallel by solving a sequence of strongly convex subproblems one for each user almost sure convergence to stationary points is proved we then customize the algorithmic framework to solve the stochastic sum rate maximization problem over single input single output siso frequency selective interference channels multiple input multiple output mimo interference channels and mimo multiple access channels numerical results corroborate that the proposed algorithms can converge faster than state of the art stochastic gradient schemes while achieving the same or better sum rates

convergence of an adaptive linear estimation algorithm in this work we prove the almost sure convergence of an adaptive linear estimator governed by a stochastic gradient algorithm with decreasing step size in the presence of correlated observations two complementary contributions are added to the famous 1977 ljung theorem first we drop the condition of nondivergence of the algorithm assumed by ljung while that condition can be ensured by adding a barrier the convergence of the suitably bounded algorithm itself is not established even on the basis of ljung theorem here the barrier problem is overcome by proving that it is not necessary for the convergence our second contribution is to generalize the model describing the correlated observations no state space model is used and no linear relationship between the observations and the signal to be estimated needs to be assumed instead we use a decreasing covariance model that agrees with a very wide class of practical applications

adaptive stochastic approximation by the simultaneous perturbation method stochastic approximation sa has long been applied for problems of minimizing loss functions or root finding with noisy input information as with all stochastic search algorithms there are adjustable algorithm coefficients that must be specified and that can have a profound effect on algorithm performance it is known that choosing these coefficients according to an sa analog of the deterministic newton raphson algorithm provides an optimal or near optimal form of the algorithm however directly determining the required hessian matrix or jacobian matrix for root finding to achieve this algorithm form has often been difficult or impossible in practice the paper presents a general adaptive sa algorithm that is based on a simple method for estimating the hessian matrix while concurrently estimating the primary parameters of interest the approach applies in both the gradient free optimization kiefer wolfowitz and root finding stochastic gradient based robbins monro settings and is based on the simultaneous perturbation sp idea introduced previously the algorithm requires only a small number of loss function or gradient measurements per iteration independent of the problem dimension to adaptively estimate the hessian and parameters of primary interest aside from introducing the adaptive sp approach the paper presents practical implementation guidance asymptotic theory and a nontrivial numerical evaluation also included is a discussion and numerical analysis comparing the adaptive sp approach with the iterate averaging approach to accelerated sa

stochastic approximation methods for systems over an infinitehorizon the paper develops efficient and general stochastic approximation sa methods for improving the operation of parametrized systems of either the continuous or discrete event dynamical systems types and which are of interest over a long time period for example one might wish to optimize or improve the stationary or average cost per unit time performance by adjusting the systems parameters the number of applications and the associated literature are increasing at a rapid rate this is partly due to the increasing activity in computing pathwise derivatives and adapting them to the average cost problem although the original motivation and the examples come from an interest in the infinite horizon problem the techniques and results are of general applicability in sa we present an updating and review of powerful ordinary differential equation type methods in a fairly general context and based on weak convergence ideas the results and proof techniques are applicable to a wide variety of applications exploiting the full potential of these ideas can greatly simplify and extend much current work their breadth as well as the relative ease of using the basic ideas are illustrated in detail via typical examples drawn from discrete event dynamical systems piecewise deterministic dynamical systems and a stochastic differential equations model in these particular illustrations we use either infinitesimal perturbation analysis type estimators mean square derivative type estimators or finite difference type estimators markov and non markov models are discussed the algorithms for distributed asynchronous updating as well as the fully synchronous schemes are developed

stochastic compositional gradient descent algorithms for minimizing compositions of expected value functions classical stochastic gradient methods are well suited for minimizing expected value objective functions however they do not apply to the minimization of a nonlinear function involving expected values or a composition of two expected value functions i e the problem min x mathbf e v left f v big mathbf e w g w x big right minxevfv ew gw x in order to solve this stochastic composition problem we propose a class of stochastic compositional gradient descent scgd algorithms that can be viewed as stochastic versions of quasi gradient method scgd update the solutions based on noisy sample gradients of f v g w fv gw and use an auxiliary variable to track the unknown quantity mathbf e w left g w x right ewgw x we prove that the scgd converge almost surely to an optimal solution for convex optimization problems as long as such a solution exists the convergence involves the interplay of two iterations with different time scales for nonsmooth convex problems the scgd achieves a convergence rate of mathcal o k 1 4 o k 1 4 in the general case and mathcal o k 2 3 o k 2 3 in the strongly convex case after taking k samples for smooth convex problems the scgd can be accelerated to converge at a rate of mathcal o k 2 7 o k 2 7 in the general case and mathcal o k 4 5 o k 4 5 in the strongly convex case for nonconvex problems we prove that any limit point generated by scgd is a stationary point for which we also provide the convergence rate analysis indeed the stochastic setting where one wants to optimize compositions of expected value functions is very common in practice the proposed scgd methods find wide applications in learning estimation dynamic programming etc

on stochastic gradient and subgradient methods with adaptive steplength sequences traditionally stochastic approximation sa schemes have been popular choices for solving stochastic optimization problems however the performance of standard sa implementations can vary significantly based on the choice of the steplength sequence and in general little guidance is provided about good choices motivated by this gap we present two adaptive steplength schemes for strongly convex differentiable stochastic optimization problems equipped with convergence theory that aim to overcome some of the reliance on user specific parameters the first scheme referred to as a recursive steplength stochastic approximation rsa scheme optimizes the error bounds to derive a rule that expresses the steplength at a given iteration as a simple function of the steplength at the previous iteration and certain problem parameters the second scheme termed as a cascading steplength stochastic approximation csa scheme maintains the steplength sequence as a piecewise constant decreasing function with the reduction in the steplength occurring when a suitable error threshold is met then we allow for nondifferentiable objectives but with bounded subgradients over a certain domain in such a regime we propose a local smoothing technique based on random local perturbations of the objective function that leads to a differentiable approximation of the function assuming a uniform distribution on the local randomness we establish a lipschitzian property for the gradient of the approximation and prove that the obtained lipschitz bound grows at a modest rate with problem size this facilitates the development of an adaptive steplength stochastic approximation framework which now requires sampling in the product space of the original measure and the artificially introduced distribution

averaged stochastic gradient algorithms for adaptive blind multiuser detection in ds cdma systems in this paper we present a blind adaptive gradient bag algorithm for code aided suppression of multiple access interference mai and narrow band interference nbi in direct sequence code division multiple access ds cdma systems this bag algorithm is based on the concept of accelerating the convergence of a stochastic gradient algorithm by averaging this ingenious concept of averaging was invented by polyak and juditsky 1992 this paper examines its application to blind multiuser detection and nbi suppression in ds cdma systems we prove that bag has identical convergence and tracking properties to recursive least squares lms but has a computational cost similar to the least mean squares lms algorithm i e an order of magnitude lower computational cost than rls simulations are used to compare our averaged gradient algorithm with the blind lms and lms schemes

on projected stochastic gradient descent algorithm with weighted averaging for least squares regression the problem of least squares regression of a d dimensional unknown parameter is considered a stochastic gradient descent based algorithm with weighted iterate averaging that uses a single pass over the data is studied and its convergence rate is analyzed we first consider a bounded constraint set of the unknown parameter under some standard regularity assumptions we provide an explicit o 1 k upper bound on the convergence rate depending on the variance due to the additive noise in the measurements and the size of the constraint set we show that the variance term dominates the error and decreases with rate 1 k while the term that is related to the size of the constraint set decreases with rate log k k 2 we then compare the asymptotic ratio rho between the convergence rate of the proposed scheme and the empirical risk minimizer erm as the number of iterations approaches infinity we show that rho leq 4 for all d geq 1 when the random entries of the sensing vector are uncorrelated and identically distributed we further improve the upper bound by showing that rho leq 4 3 for the case of d 1 and unbounded parameter set when the random sensing entries are equal across time simulation results demonstrate strong performance of the algorithm as compared to existing methods and coincide with rho leq 4 3 even for large d in practice

adaptive estimation of hmm transition probabilities this paper presents new schemes for recursive estimation of the state transition probabilities for hidden markov models hmms via extended least squares els and recursive state prediction error rspe methods local convergence analysis for the proposed rspe algorithm is shown using the ordinary differential equation ode approach developed for the more familiar recursive output prediction error rpe methods the presented scheme converges and is relatively well conditioned compared with the previously proposed rpe scheme for estimating the transition probabilities that perform poorly in low noise the els algorithm presented is computationally of order n sup 2 which is less than the computational effort of order n sup 4 required to implement the rspe and previous rpe scheme where n is the number of markov states building on earlier work an algorithm for simultaneous estimation of the state output mappings and the state transition probabilities that requires less computational effort than earlier schemes is also presented and discussed implementation aspects of the proposed algorithms are discussed and simulation studies are presented to illustrate the convergence and convergence rates

recursive em and sage inspired algorithms with application to doa estimation this paper is concerned with recursive estimation using augmented data we study two recursive procedures closely linked with the well known expectation and maximization em and space alternating generalized em sage algorithms unlike iterative methods the recursive em and sage inspired algorithms give a quick update on estimates given new data under mild conditions estimates generated by these procedures are strongly consistent and asymptotically normally distributed these mathematical properties are valid for a broad class of problems when applied to direction of arrival doa estimation the recursive em and sage inspired algorithms lead to a very simple and fast implementation of the maximum likelihood ml method the most complicated computation in each recursion is inversion of the augmented information matrix through data augmentation this matrix is diagonal and easy to invert more importantly there is no search in such recursive procedures consequently the computational time is much less than that associated with existing numerical methods for finding ml estimates this feature greatly increases the potential of the ml approach in real time processing numerical experiments show that both algorithms provide good results with low computational cost

on line expectation maximization algorithm for latent data models in this contribution we propose a generic online also sometimes called adaptive or recursive version of the expectation maximisation em algorithm applicable to latent variable models of independent observations compared to the algorithm of titterington 1984 this approach is more directly connected to the usual em algorithm and does not rely on integration with respect to the complete data distribution the resulting algorithm is usually simpler and is shown to achieve convergence to the stationary points of the kullback leibler divergence between the marginal distribution of the observation and the model distribution at the optimal rate i e that of the maximum likelihood estimator in addition the proposed approach is also suitable for conditional or regression models as illustrated in the case of the mixture of linear regressions model

analysis of adaptive step size sa algorithms for parameter tracking we present proofs and data for adaptive step size algorithms for tracking time varying parameters when recursive stochastic approximation type algorithms are used a classical problem in adaptive control and communication theory concerns the tracking of the best fit of a given form when the statistics or the parameters change slowly a major and yet unresolved problem has been the choice of the step sizes in the tracking algorithm an algorithm for adapting the step size using the same system measurements which are used for the tracking was suggested by benveniste and various examples worked out by brossier the numerical results were very encouraging but proofs were lacking these proofs are supplied here together with supporting numerical data the proofs are based on recent results in stochastic approximation the adaptive step size technique works very well indeed much supporting analysis is presented particularly concerning the interpretation of certain stationary processes as stationary pathwise derivatives finite difference forms are also treated these are mathematically simpler and can be applied to a wide variety of systems even when the system is not well modeled the data shows that they work well

multivariate stochastic approximation using a simultaneous perturbation gradient approximation the problem of finding a root of the multivariate gradient equation that arises in function minimization is considered when only noisy measurements of the function are available a stochastic approximation sa algorithm for the general kiefer wolfowitz type is appropriate for estimating the root the paper presents an sa algorithm that is based on a simultaneous perturbation gradient approximation instead of the standard finite difference approximation of keifer wolfowitz type procedures theory and numerical experience indicate that the algorithm can be significantly more efficient than the standard algorithms in large dimensional problems

acceleration of stochastic approximation by averaging a new recursive algorithm of stochastic approximation type with the averaging of trajectories is investigated convergence with probability one is proved for a variety of classical optimization and identification problems it is also demonstrated for these problems that the proposed algorithm achieves the highest possible rate of convergence

stochastic root finding and efficient estimation of convex risk measures reliable risk measurement is a key problem for financial institutions and regulatory authorities the current industry standard value at risk has several deficiencies improved risk measures have been suggested and analyzed in the recent literature but their computational implementation has largely been neglected so far we propose and investigate stochastic approximation algorithms for the convex risk measure utility based shortfall risk our approach combines stochastic root finding schemes with importance sampling we prove that the resulting shortfall risk estimators are consistent and asymptotically normal and provide formulas for confidence intervals the performance of the proposed algorithms is tested numerically we finally apply our techniques to the normal copula model which is also known as the industry model creditmetrics this provides guidance for future implementations in practice

on complexity issues of online learning algorithms in this paper some new probabilistic upper bounds are presented for the online learning algorithm proposed in and more generally for linear stochastic approximations in hilbert spaces with these upper bounds not only does one recover almost sure convergence but also relaxes the square summable condition on the step size appeared in the early work furthermore two probabilistic upper bounds are given for an averaging process both of which achieve the same rate with respect to sample size as in batch learning algorithms and one of which is tight in both sample size and regularization parameter

two stage linear decision rules for multi stage stochastic programming multi stage stochastic linear programs mslps are notoriously hard to solve in general linear decision rules ldrs yield an approximation of an mslp by restricting the decisions at each stage to be an affine function of the observed uncertain parameters finding an optimal ldr is a static optimization problem that provides an upper bound on the optimal value of the mslp and under certain assumptions can be formulated as an explicit linear program similarly as proposed by kuhn et al math program 130 1 177 209 2011 a lower bound for an mslp can be obtained by restricting decisions in the dual of the mslp to follow an ldr we propose a new approximation approach for mslps two stage ldrs the idea is to require only the state variables in an mslp to follow an ldr which is sufficient to obtain an approximation of an mslp that is a two stage stochastic linear program 2slp we similarly propose to apply ldr only to a subset of the variables in the dual of the mslp which yields a 2slp approximation of the dual that provides a lower bound on the optimal value of the mslp although solving the corresponding 2slp approximations exactly is intractable in general we investigate how approximate solution approaches that have been developed for solving 2slp can be applied to solve these approximation problems and derive statistical upper and lower bounds on the optimal value of the mslp in addition to potentially yielding better policies and bounds this approach requires many fewer assumptions than are required to obtain an explicit reformulation when using the standard static ldr approach a computational study on two example problems demonstrates that using a two stage ldr can yield significantly better primal policies and modestly better dual policies than using policies based on a static ldr

manifold identification in dual averaging for regularized stochastic online learning iterative methods that calculate their steps from approximate subgradient directions have proved to be useful for stochastic learning problems over large and streaming data sets when the objective consists of a loss function plus a nonsmooth regularization term the solution often lies on a low dimensional manifold of parameter space along which the regularizer is smooth when an l1 regularizer is used to induce sparsity in the solution for example this manifold is defined by the set of nonzero components of the parameter vector this paper shows that a regularized dual averaging algorithm can identify this manifold with high probability before reaching the solution this observation motivates an algorithmic strategy in which once an iterate is suspected of lying on an optimal or near optimal manifold we switch to a local phase that searches in this manifold thus converging rapidly to a near optimal point computational results are presented to verify the identification property and to illustrate the effectiveness of this approach

combining monte carlo and mean field like methods for inference in hidden markov random fields issues involving missing data are typical settings where exact inference is not tractable as soon as nontrivial interactions occur between the missing variables approximations are required and most of them are based either on simulation methods or on deterministic variational methods while variational methods provide fast and reasonable approximate estimates in many scenarios simulation methods offer more consideration of important theoretical issues such as accuracy of the approximation and convergence of the algorithms but at a much higher computational cost in this work we propose a new class of algorithms that combine the main features and advantages of both simulation and deterministic methods and consider applications to inference in hidden markov random fields hmrfs these algorithms can be viewed as stochastic perturbations of variational expectation maximization vem algorithms which are not tractable for hmrf we focus more specifically on one of these perturbations and we prove their almost sure convergence to the same limit set as the limit set of vem in addition experiments on synthetic and real world images show that the algorithm performance is very close and sometimes better than that of other existing simulation based and variational em like algorithms

alternative to the diffusion equation in population genetics since its inception by kimura in 1955 m kimura proc natl acad sci u s a 41 144 1955 the diffusion equation has become a standard technique of population genetics the diffusion equation is however only an approximation valid in the limit of large populations and small selection moreover useful quantities such as the fixation probabilities are not easily extracted from it and need the concomitant use of a forward and backward equation we show here that the partial differential equation governing the probability generating function can be used as an alternative to the diffusion equation with none of its drawbacks it does not involve any approximation it has well defined initial and boundary conditions and its solutions are finite polynomials we apply this technique to derive analytical results for the moran process with selection which encompasses the kimura diffusion equation

on closed loop adaptive noise cancellation given the mean limit ordinary differential equation for the stochastic approximation defining the adaptive algorithm for a closed loop adaptive noise cancellation we characterize the limit points under appropriate conditions it is shown that as the dimension of the weight vector increases the sequence of corresponding limit points converges in the sense of l sub 2 to the infinite dimensional optimal weight vector also the limit point of the algorithm is nearly optimal if the dimension of the weight vector is large enough the gradient of the mean square error with respect to the weight vector evaluated at the limit goes to zero in l sub 1 and l sub 2 as the dimension increases as does the gradient with respect to the coefficients in the transfer function connecting the reference noise signal with the error output thus the algorithm is nearly a gradient descent algorithm and is error reducing for large enough dimension under broad conditions iterative averaging can be used to get a nearly optimal rate of convergence

rates of convergence of adaptive step size of stochastic approximation algorithms we propose a new adaptive algorithm with decreasing step size for stochastic approximations the use of adaptive algorithms in various applications is widely spread across fields such as system identification and adaptive control we analyze the rate of convergence of the proposed algorithms an averaging algorithm on its optimality of the rate of convergence is used to control the step sizes our proofs are based on recent results in stochastic approximations and gaussian approximation theorem