the em algorithm an old folk song sung to a fast new tune celebrating the 20th anniversary of the presentation of the paper by dempster laird and rubin which popularized the em algorithm we investigate after a brief historical account strategies that aim to make the em algorithm converge faster while maintaining its simplicity and stability e g automatic monotone convergence in likelihood first we introduce the idea of a working parameter to facilitate the search for efficient data augmentation schemes and thus fast em implementations second summarizing various recent extensions of the em algorithm we formulate a general alternating expectation conditional maximization algorithm aecm that couples flexible data augmentation schemes with model reduction schemes to achieve efficient computations we illustrate these methods using multivariate t models with known or unknown degrees of freedom and poisson models for image reconstruction we show through both empirical and theoretical evidence the potential for a dramatic reduction in computational time with little increase in human effort we also discuss the intrinsic connection between em type algorithms and the gibbs sampler and the possibility of using the techniques presented here to speed up the latter the main conclusion of the paper is that with the help of statistical considerations it is possible to construct algorithms that are simple stable and fast

validation analysis of mirror descent stochastic approximation method the main goal of this paper is to develop accuracy estimates for stochastic programming problems by employing stochastic approximation sa type algorithms to this end we show that while running a mirror descent stochastic approximation procedure one can compute with a small additional effort lower and upper statistical bounds for the optimal objective value we demonstrate that for a certain class of convex stochastic programs these bounds are comparable in quality with similar bounds computed by the sample average approximation method while their computational cost is considerably smaller

variance based extragradient methods with line search for stochastic variational inequalities in this paper we propose dynamic sampled stochastic approximated ds sa extragradient methods for stochastic variational inequalities svis that are robust with respect to an unknown lipschitz c

lms algorithms for tracking slow markov chains with applications to hidden markov estimation and adaptive multiuser detection this paper analyzes the tracking properties of the least mean squares lms algorithm when the underlying parameter evolves according to a finite state markov chain with infrequent jumps first using perturbed liapunov function methods mean square error estimates are obtained for the tracking error then using recent results on two time scale markov chains mean ordinary differential equation and diffusion approximation results are obtained it is shown that a sequence of the centered tracking errors converges to an ordinary differential equation moreover a suitably scaled sequence of the tracking errors converges weakly to a diffusion process it is also shown that iterate averaging of the tracking algorithm results in optimal asymptotic convergence rate in an appropriate sense two application examples analysis of the performance of an adaptive multiuser detection algorithm in a direct sequence code division multiple access ds cdma system and tracking analysis of the state of a hidden markov model hmm with infrequent jumps are presented

a central limit theorem and hypotheses testing for risk averse stochastic programs we study statistical properties of the optimal value and optimal solutions of the sample average approximation of risk averse stochastic problems central limit theorem type results are derived for the optimal value when the stochastic program is expressed in terms of a law invariant coherent risk measure having a discrete kusuoka representation the obtained results are applied to hypotheses testing problems aiming at comparing the optimal values of several risk averse convex stochastic programs on the basis of samples of the underlying random vectors we also consider nonasymptotic tests based on confidence intervals on the optimal values of the stochastic programs obtained using the stochastic mirror descent algorithm numerical simulations show how to use our developments to choose among different distributions and on the considered class of risk averse stochastic programs the asymptotic tests show better results

a one measurement form of simultaneous perturbation stochastic approximation the simultaneous perturbation stochastic approximation spsa algorithm has proven very effective for difficult multivariate optimization problems where it is not possible to obtain direct gradient information as discussed to date spsa is based on a highly efficient gradient approximation requiring only two measurements of the loss function independent of the number of parameters being estimated this note presents a form of spsa that requires only one function measurement for any dimension theory is presented that identifies the class of problems for which this one measurement form will be asymptotically superior to the standard two measurement form

online estimation of the geometric median in hilbert spaces nonasymptotic confidence balls estimation procedures based on recursive algorithms are interesting and powerful techniques that are able to deal rapidly with very large samples of high dimensional data the collected data may be contaminated by noise so that robust location indicators such as the geometric median may be preferred to the mean in this context an estimator of the geometric median based on a fast and efficient averaged nonlinear stochastic gradient algorithm has been developed by bernoulli19 2013 18 43 this work aims at studying more precisely the nonasymptotic behavior of this nonlinear algorithm by giving nonasymptotic confidence balls in general separable hilbert spaces this new result is based on the derivation of improved l 2 rates of convergence as well as an exponential inequality for the nearly martingale terms of the recursive nonlinear robbins monro algorithm

on the design of a stable adaptive filter for state estimation in high dimensional systems design of specific parametrized gain structures provides a new tool for ensuring stability of reduced order filter for high dimensional systems which is optimized adaptively in the minimum prediction error framework

optimal bayesian experimental design for priors of compact support with application to shock tube experiments for combustion kinetics summary the analysis of reactive systems in combustion science and technology relies on detailed models comprising many chemical reactions that describe the conversion of fuel and oxidizer into products and the formation of pollutants shock tube experiments are a convenient setting for measuring the rate parameters of individual reactions the temperature pressure and concentration of reactants are chosen to maximize the sensitivity of the measured quantities to the rate parameter of the target reaction in this study we optimize the experimental setup computationally by optimal experimental design in a bayesian framework we approximate the posterior probability density functions pdf using truncated gaussian distributions in order to account for the bounded domain of the uniform prior pdf of the parameters the underlying gaussian distribution is obtained in the spirit of the laplace method more precisely the mode is chosen as the maximum a posteriori map estimate and the covariance is chosen as the negative inverse of the hessian of the misfit function at the map estimate the model related entities are obtained from a polynomial surrogate the optimality quantified by the information gain measures can be estimated efficiently by a rejection sampling algorithm against the underlying gaussian probability distribution rather than against the true posterior this approach offers a significant error reduction when the magnitude of the invariants of the posterior covariance are comparable with the size of the bounded domain of the prior we demonstrate the accuracy and superior computational efficiency of our method for shock tube experiments aiming to measure the model parameters of a key reaction which is part of the complex kinetic network describing the hydrocarbon oxidation in the experiments the initial temperature and fuel concentration are optimized with respect to the expected information gain in the estimation of the parameters of the target reaction rate we show that the expected information gain surface can change its shape dramatically according to the level of noise introduced into the synthetic data the information that can be extracted from the data saturates as a logarithmic function of the number of experiments and few experiments are needed when they are conducted at the optimal experimental design conditions furthermore inversion of the legacy data indicates the validity and robustness of our designs copyright 2016 john wiley sons ltd

stochastic approximation method with gradient averaging for unconstrained problems a stochastic approximation algorithm is studied in which random gradient estimates are averaged by an auxiliary filter convergence of the algorithm is proved under a rather general noise condition a practical version of the method is described and numerical results are given

analysis and optimization of diagonally layered lattice schemes for mimo fading channels embodiments of the diagonal bell laboratories layered space time d blast architecture for multiple input multiple output mimo communication are developed wherein information symbol vectors are encoded using codewords from a lattice code called a diagonally layered lattice dll code which are formatted onto the diagonals of a space time frame decoding is done using a sphere decoder for each diagonal based on soft statistics obtained after zero forcing zf or minimum mean square error mmse filtering and decision feedback these operations give rise to an effective parallel channel model with channel gains with nonidentical statistics and additive noise which is gaussian in the zf filtering case and non gaussian in the mmse filtering case the so called full modulation diversity fmd property is nevertheless shown to yield the maximum achievable diversity orders over the mimo channel for both the zf and the mmse filtering based decoders respectively for any arbitrary fading distribution in the case of the independent identically distributed i i d rayleigh fading mimo channel with k transmit and n receive antennas with ngesk these diversity orders are nk k k 1 2 and nk for zf and mmse filtering based decoding respectively the error probability analysis also yields a design criterion for optimizing transmit power allocations several lattice design methods are proposed for the effective parallel channel models two methods are proposed to achieve high coding gain in the rayleigh fading mimo channel a third method is proposed that minimizes the exact symbol error probability sep and can be tailored for any given fading distribution a novel soft decision feedback decoder is also proposed based on the list sphere decoder to mitigate error propagation due to hard decision feedback the salient feature of the proposed dll schemes is that they have nearly full rate and full or high diversity order and yet a much lower decoding complexity than other existing full rate full diversity space time block codes stbcs the frame error probability fep performance of the optimized dll schemes for moderate to high spectral efficiencies and a wide range of signal to noise ratios snrs can be quite close to the performance of the best performing but more complex to decode stbcs moreover the proposed dll schemes significantly outperform other existing mimo systems of comparable decoding complexity

multistep stochastic mirror descent for risk averse convex stochastic programs based on extended polyhedral risk measures we consider risk averse convex stochastic programs expressed in terms of extended polyhedral risk measures we derive computable confidence intervals on the optimal value of such stochastic programs using the robust stochastic approximation and the stochastic mirror descent smd algorithms when the objective functions are uniformly convex we also propose a multistep extension of the stochastic mirror descent algorithm and obtain confidence intervals on both the optimal values and optimal solutions numerical simulations show that our confidence intervals are much less conservative and are quicker to compute than previously obtained confidence intervals for smd and that the multistep stochastic mirror descent algorithm can obtain a good approximate solution much quicker than its nonmultistep counterpart

ascent based monte carlo expectation maximization summary the expectation maximization em algorithm is a popular tool for maximizing likelihood functions in the presence of missing data unfortunately em often requires the evaluation of analytically intractable and high dimensional integrals the monte carlo em mcem algorithm is the natural extension of em that employs monte carlo methods to estimate the relevant integrals typically a very large monte carlo sample size is required to estimate these integrals within an acceptable tolerance when the algorithm is near convergence even if this sample size were known at the onset of implementation of mcem its use throughout all iterations is wasteful especially when accurate starting values are not available we propose a data driven strategy for controlling monte carlo resources in mcem the algorithm proposed improves on similar existing methods by recovering em s ascent i e likelihood increasing property with high probability being more robust to the effect of user defined inputs and handling classical monte carlo and markov chain monte carlo methods within a common framework because of the first of these properties we refer to the algorithm as ascent based mcem we apply ascent based mcem to a variety of examples including one where it is used to accelerate the convergence of deterministic em dramatically

least mean square algorithms with markov regime switching limit we analyze the tracking performance of the least mean square lms algorithm for adaptively estimating a time varying parameter that evolves according to a finite state markov chain we assume the markov chain jumps infrequently between the finite states at the same rate of change as the lms algorithm we derive mean square estimation error bounds for the tracking error of the lms algorithm using perturbed lyapunov function methods then combining results in two time scale markov chains with weak convergence methods for stochastic approximation we derive the limit dynamics satisfied by continuous time interpolation of the estimates unlike most previous analyzes of stochastic approximation algorithms the limit we obtain is a system of ordinary differential equations with regime switching controlled by a continuous time markov chain next to analyze the rate of convergence we take a continuous time interpolation of a scaled sequence of the error sequence and derive its diffusion limit somewhat remarkably for correlated regression vectors we obtain a jump markov diffusion finally two novel examples of the analysis are given for state estimation of hidden markov models hmms and adaptive interference suppression in wireless code division multiple access cdma networks

on line estimation of dynamic shock error models based on the kullback leibler information measure develops two sequential or on line estimation schemes in the time domain for dynamic shock error models which are special cases of errors in variables models the author s approach utilizes a state space representation of the model kalman filtering techniques and on line algorithms the first on line algorithm is based on the expectation maximization algorithm and uses a recursive gauss newton scheme to maximize the kullback leibler information measure the second on line algorithm the author proposes is a gradient based scheme and uses stochastic approximations to maximize the log likelihood in comparison to the off line maximum likelihood estimation scheme used in ghosh 1989 the author s on line algorithms have significantly reduced computational costs and negligible memory requirements simulations illustrate the satisfactory performance of the algorithms in estimating errors in variables systems with parameters that vary slowly with time or undergo infrequent jump changes

on sampling rates in simulation based recursions we consider the context of simulation based recursions that is recursions that involve quantities needing to be estimated using a stochastic simulation examples include stochastic adaptations of fixed point and gradient descent recursions obtained by replacing function and derivative values appearing within the recursion by their monte carlo counterparts the primary motivating settings are simulation optimization and stochastic root finding problems where the low point and the zero of a function are sought respectively with only monte carlo estimates of the functions appearing within the problem we ask how much monte carlo sampling needs to be performed within simulation based recursions in order that the resulting iterates remain consistent and more importantly efficient where efficient implies convergence at the fastest possible rate answering these questions involves trading off two types of error inherent in the iterates the deterministic error due to recursion and the stochastic er

a stochastic gradient type algorithm for closed loop problems we focus on the numerical solution of closed loop stochastic problems and propose a perturbed gradient algorithm to achieve this goal the main hurdle in such problems is the fact that the control variables are infinite dimensional due to e g the information constraints alternatively said control variables are feedbacks i e functions such controls have hence to be represented in a finite way in order to solve the problem numerically in the same way the gradient of the criterion is itself an infinite dimensional object our algorithm replaces this exact and unknown gradient by a perturbed one which consists of the product of the true gradient evaluated at a random point and a kernel function which extends this gradient to the neighbourhood of the random point proceeding this way we explore the whole space iteration after iteration through random points since each kernel function is perfectly known by a small number of parameters say n the control at iteration k is perfectly known as an infinite dimensional object by at most n k parameters the main strength of this method is that it avoids any discretization of the underlying space provided that we can sample as many points as needed in this space moreover our algorithm can take into account the possible measurability constraints of the problem in a new way finally the randomized strategy implemented by the algorithm causes the most probable parts of the space to be the most explored ones which is a priori an interesting feature in this paper we first prove two convergence results of this algorithm in the strongly convex and convex cases and then give some numerical examples showing the interest of this method for practical stochastic optimization problems

stochastic approximation with averaging and feedback rapidly convergent on line algorithms consider the stochastic approximation x sub n 1 x sub n a sub n g x sub n spl xi sub n where 0 0 the averaging method is essentially off line in the sense that the actual sa iterate x sub n is not influenced by the averaging in many applications x sub n itself is of greatest interest since that is the operating parameter this paper deals with the problem of stochastic approximation with averaging and with appropriate feedback of the averages into the original algorithm it is shown both mathematically and via simulation that it works very well and has numerous advantages it is a clear improvement over the system x sub n by itself it is fairly robust and quite often it is much preferable to the use of the above averages without feedback the authors deal in particular with linear algorithms of the type appearing in parameter estimators adaptive noise cancellers channel equalizers adaptive control and similar applications the main development is for the constant parameter case because of its importance in applications but analogous results hold for the case where a sub n spl rarr 0

the analysis of very small samples of repeated measurements ii a modified box correction there is a need for appropriate methods for the analysis of very small samples of continuous repeated measurements a key feature of such analyses is the role played by the covariance matrix of the repeated observations when subjects are few it can be difficult to assess the fit of parsimonious structures for this matrix while the use of an unstructured form may lead to a serious lack of power the kenward roger adjustment is now widely adopted as a means of providing an appropriate inferences in small samples but does not perform adequately in very small samples adjusted tests based on the empirical sandwich estimator can be constructed that have good nominal properties but are seriously underpowered further when such data are incomplete or unbalanced or non saturated mean models are used exact distributional results do not exist that justify analyses with any sample size in this paper a modification of box s correction applied to a linear model based f statistic is developed for such small sample settings and is shown to have both the required nominal properties and acceptable power across a range of settings for repeated measurements

recursive k distribution parameter estimation recursive estimation of the parameter of the k distribution is studied and tested the probability density function pdf of the k distribution is seen as a mixture pdf allowing the application of titterington s recursive expectation maximization em technique under mild conditions the technique produces estimates that are strongly consistent and asymptotically normal for the k distribution the complete data information matrix required by the recursive em has an explicit form making the algorithm easy to implement the algorithm is tested using k distributed data with both constant and time varying parameter for the constant parameter case recursive em estimates are compared to numerical maximization of the likelihood for the time varying parameter case recursive em estimates are compared to estimates obtained using a fast routine from the literature and implemented by sliding a rectangular window over the observations

statistical inference for the population landscape via moment adjusted stochastic gradients modern statistical inference tasks often require iterative optimization methods to compute the solution convergence analysis from an optimization viewpoint informs us only how well the solution is approximated numerically but overlooks the sampling nature of the data in contrast recognizing the randomness in the data statisticians are keen to provide uncertainty quantification or confidence for the solution obtained by using iterative optimization methods the paper makes progress along this direction by introducing moment adjusted stochastic gradient descent a new stochastic optimization method for statistical inference we establish non asymptotic theory that characterizes the statistical distribution for certain iterative methods with optimization guarantees on the statistical front the theory allows for model misspecification with very mild conditions on the data for optimization the theory is flexible for both convex and non convex cases remarkably the moment adjusting idea motivated from error standardization in statistics achieves a similar effect to acceleration in first order optimization methods that are used to fit generalized linear models we also demonstrate this acceleration effect in the non convex setting through numerical experiments

weak convergence properties of constrained emphatic temporal difference learning with constant and slowly diminishing stepsize we consider the emphatic temporal difference td algorithm etd for learning the value functions of stationary policies in a discounted finite state and action markov decision process the etd algorithm was recently proposed by sutton mahmood and white 2016 to solve a long standing divergence problem of the standard td algorithm when it is applied to off policy training where data from an exploratory policy are used to evaluate other policies of interest the almost sure convergence of etd has been proved in our recent work under general off policy training conditions but for a narrow range of diminishing stepsize in this paper we present convergence results for constrained versions of etd with constant stepsize and with diminishing stepsize from a broad range our results characterize the asymptotic behavior of the trajectory of iterates produced by those algorithms and are derived by combining key properties of etd with powerful convergence theorems from the weak convergence methods in stochastic approximation theory for the case of constant stepsize in addition to analyzing the behavior of the algorithms in the limit as the stepsize parameter approaches zero we also analyze their behavior for a fixed stepsize and bound the deviations of their averaged iterates from the desired solution these results are obtained by exploiting the weak feller property of the markov chains associated with the algorithms and by using ergodic theorems for weak feller markov chains in conjunction with the convergence results we get from the weak convergence methods besides etd our analysis also applies to the off policy td algorithm when the divergence issue is avoided by setting sufficiently large it yields for that case new results on the asymptotic convergence properties of constrained off policy td with constant or slowly diminishing stepsize

a parallel decomposition method for nonconvex stochastic multi agent optimization problems this paper considers the problem of minimizing the expected value of a possibly nonconvex cost function parameterized by a random vector variable when the expectation cannot be computed accurately e g because the statistics of the random variables are unknown and or the computational complexity is prohibitive classical stochastic gradient methods for solving this problem may suffer from slow convergence in this paper we propose a stochastic parallel successive convex approximation based best response algorithm for general nonconvex stochastic sum utility optimization problems which arise naturally in the design of multi agent networks the proposed novel decomposition approach enables all users to update their optimization variables in parallel by solving a sequence of strongly convex subproblems one for each user almost sure convergence to stationary points is proved we then customize the algorithmic framework to solve the stochastic sum rate maximization problem over single input single output siso frequency selective interference channels multiple input multiple output mimo interference channels and mimo multiple access channels numerical results corroborate that the proposed algorithms can converge faster than state of the art stochastic gradient schemes while achieving the same or better sum rates

adaptive stochastic approximation by the simultaneous perturbation method stochastic approximation sa has long been applied for problems of minimizing loss functions or root finding with noisy input information as with all stochastic search algorithms there are adjustable algorithm coefficients that must be specified and that can have a profound effect on algorithm performance it is known that choosing these coefficients according to an sa analog of the deterministic newton raphson algorithm provides an optimal or near optimal form of the algorithm however directly determining the required hessian matrix or jacobian matrix for root finding to achieve this algorithm form has often been difficult or impossible in practice the paper presents a general adaptive sa algorithm that is based on a simple method for estimating the hessian matrix while concurrently estimating the primary parameters of interest the approach applies in both the gradient free optimization kiefer wolfowitz and root finding stochastic gradient based robbins monro settings and is based on the simultaneous perturbation sp idea introduced previously the algorithm requires only a small number of loss function or gradient measurements per iteration independent of the problem dimension to adaptively estimate the hessian and parameters of primary interest aside from introducing the adaptive sp approach the paper presents practical implementation guidance asymptotic theory and a nontrivial numerical evaluation also included is a discussion and numerical analysis comparing the adaptive sp approach with the iterate averaging approach to accelerated sa

on stochastic gradient and subgradient methods with adaptive steplength sequences traditionally stochastic approximation sa schemes have been popular choices for solving stochastic optimization problems however the performance of standard sa implementations can vary significantly based on the choice of the steplength sequence and in general little guidance is provided about good choices motivated by this gap we present two adaptive steplength schemes for strongly convex differentiable stochastic optimization problems equipped with convergence theory that aim to overcome some of the reliance on user specific parameters the first scheme referred to as a recursive steplength stochastic approximation rsa scheme optimizes the error bounds to derive a rule that expresses the steplength at a given iteration as a simple function of the steplength at the previous iteration and certain problem parameters the second scheme termed as a cascading steplength stochastic approximation csa scheme maintains the steplength sequence as a piecewise constant decreasing function with the reduction in the steplength occurring when a suitable error threshold is met then we allow for nondifferentiable objectives but with bounded subgradients over a certain domain in such a regime we propose a local smoothing technique based on random local perturbations of the objective function that leads to a differentiable approximation of the function assuming a uniform distribution on the local randomness we establish a lipschitzian property for the gradient of the approximation and prove that the obtained lipschitz bound grows at a modest rate with problem size this facilitates the development of an adaptive steplength stochastic approximation framework which now requires sampling in the product space of the original measure and the artificially introduced distribution

on projected stochastic gradient descent algorithm with weighted averaging for least squares regression the problem of least squares regression of a d dimensional unknown parameter is considered a stochastic gradient descent based algorithm with weighted iterate averaging that uses a single pass over the data is studied and its convergence rate is analyzed we first consider a bounded constraint set of the unknown parameter under some standard regularity assumptions we provide an explicit o 1 k upper bound on the convergence rate depending on the variance due to the additive noise in the measurements and the size of the constraint set we show that the variance term dominates the error and decreases with rate 1 k while the term that is related to the size of the constraint set decreases with rate log k k 2 we then compare the asymptotic ratio rho between the convergence rate of the proposed scheme and the empirical risk minimizer erm as the number of iterations approaches infinity we show that rho leq 4 for all d geq 1 when the random entries of the sensing vector are uncorrelated and identically distributed we further improve the upper bound by showing that rho leq 4 3 for the case of d 1 and unbounded parameter set when the random sensing entries are equal across time simulation results demonstrate strong performance of the algorithm as compared to existing methods and coincide with rho leq 4 3 even for large d in practice

adaptive estimation of hmm transition probabilities this paper presents new schemes for recursive estimation of the state transition probabilities for hidden markov models hmms via extended least squares els and recursive state prediction error rspe methods local convergence analysis for the proposed rspe algorithm is shown using the ordinary differential equation ode approach developed for the more familiar recursive output prediction error rpe methods the presented scheme converges and is relatively well conditioned compared with the previously proposed rpe scheme for estimating the transition probabilities that perform poorly in low noise the els algorithm presented is computationally of order n sup 2 which is less than the computational effort of order n sup 4 required to implement the rspe and previous rpe scheme where n is the number of markov states building on earlier work an algorithm for simultaneous estimation of the state output mappings and the state transition probabilities that requires less computational effort than earlier schemes is also presented and discussed implementation aspects of the proposed algorithms are discussed and simulation studies are presented to illustrate the convergence and convergence rates

analysis of adaptive step size sa algorithms for parameter tracking we present proofs and data for adaptive step size algorithms for tracking time varying parameters when recursive stochastic approximation type algorithms are used a classical problem in adaptive control and communication theory concerns the tracking of the best fit of a given form when the statistics or the parameters change slowly a major and yet unresolved problem has been the choice of the step sizes in the tracking algorithm an algorithm for adapting the step size using the same system measurements which are used for the tracking was suggested by benveniste and various examples worked out by brossier the numerical results were very encouraging but proofs were lacking these proofs are supplied here together with supporting numerical data the proofs are based on recent results in stochastic approximation the adaptive step size technique works very well indeed much supporting analysis is presented particularly concerning the interpretation of certain stationary processes as stationary pathwise derivatives finite difference forms are also treated these are mathematically simpler and can be applied to a wide variety of systems even when the system is not well modeled the data shows that they work well

diversity of poissonian populations populations represented by collections of points scattered randomly on the real line are ubiquitous in science and engineering the statistical modeling of such populations leads naturally to poissonian populations poisson processes on the real line with a distinguished maximal point poissonian populations are infinite objects underlying key issues in statistical physics probability theory and random fractals due to their infiniteness measuring the diversity of poissonian populations depends on the lower bound cut off applied this research characterizes the classes of poissonian populations whose diversities are invariant with respect to the cut off level applied and establishes an elemental connection between these classes and extreme value theory the measures of diversity considered are variance and dispersion simpson s index and inverse participation ratio shannon s entropy and renyi s entropy and gini s index

acceleration of stochastic approximation by averaging a new recursive algorithm of stochastic approximation type with the averaging of trajectories is investigated convergence with probability one is proved for a variety of classical optimization and identification problems it is also demonstrated for these problems that the proposed algorithm achieves the highest possible rate of convergence

stochastic root finding and efficient estimation of convex risk measures reliable risk measurement is a key problem for financial institutions and regulatory authorities the current industry standard value at risk has several deficiencies improved risk measures have been suggested and analyzed in the recent literature but their computational implementation has largely been neglected so far we propose and investigate stochastic approximation algorithms for the convex risk measure utility based shortfall risk our approach combines stochastic root finding schemes with importance sampling we prove that the resulting shortfall risk estimators are consistent and asymptotically normal and provide formulas for confidence intervals the performance of the proposed algorithms is tested numerically we finally apply our techniques to the normal copula model which is also known as the industry model creditmetrics this provides guidance for future implementations in practice

on complexity issues of online learning algorithms in this paper some new probabilistic upper bounds are presented for the online learning algorithm proposed in and more generally for linear stochastic approximations in hilbert spaces with these upper bounds not only does one recover almost sure convergence but also relaxes the square summable condition on the step size appeared in the early work furthermore two probabilistic upper bounds are given for an averaging process both of which achieve the same rate with respect to sample size as in batch learning algorithms and one of which is tight in both sample size and regularization parameter

combining monte carlo and mean field like methods for inference in hidden markov random fields issues involving missing data are typical settings where exact inference is not tractable as soon as nontrivial interactions occur between the missing variables approximations are required and most of them are based either on simulation methods or on deterministic variational methods while variational methods provide fast and reasonable approximate estimates in many scenarios simulation methods offer more consideration of important theoretical issues such as accuracy of the approximation and convergence of the algorithms but at a much higher computational cost in this work we propose a new class of algorithms that combine the main features and advantages of both simulation and deterministic methods and consider applications to inference in hidden markov random fields hmrfs these algorithms can be viewed as stochastic perturbations of variational expectation maximization vem algorithms which are not tractable for hmrf we focus more specifically on one of these perturbations and we prove their almost sure convergence to the same limit set as the limit set of vem in addition experiments on synthetic and real world images show that the algorithm performance is very close and sometimes better than that of other existing simulation based and variational em like algorithms

manifold identification in dual averaging for regularized stochastic online learning iterative methods that calculate their steps from approximate subgradient directions have proved to be useful for stochastic learning problems over large and streaming data sets when the objective consists of a loss function plus a nonsmooth regularization term the solution often lies on a low dimensional manifold of parameter space along which the regularizer is smooth when an l1 regularizer is used to induce sparsity in the solution for example this manifold is defined by the set of nonzero components of the parameter vector this paper shows that a regularized dual averaging algorithm can identify this manifold with high probability before reaching the solution this observation motivates an algorithmic strategy in which once an iterate is suspected of lying on an optimal or near optimal manifold we switch to a local phase that searches in this manifold thus converging rapidly to a near optimal point computational results are presented to verify the identification property and to illustrate the effectiveness of this approach

on closed loop adaptive noise cancellation given the mean limit ordinary differential equation for the stochastic approximation defining the adaptive algorithm for a closed loop adaptive noise cancellation we characterize the limit points under appropriate conditions it is shown that as the dimension of the weight vector increases the sequence of corresponding limit points converges in the sense of l sub 2 to the infinite dimensional optimal weight vector also the limit point of the algorithm is nearly optimal if the dimension of the weight vector is large enough the gradient of the mean square error with respect to the weight vector evaluated at the limit goes to zero in l sub 1 and l sub 2 as the dimension increases as does the gradient with respect to the coefficients in the transfer function connecting the reference noise signal with the error output thus the algorithm is nearly a gradient descent algorithm and is error reducing for large enough dimension under broad conditions iterative averaging can be used to get a nearly optimal rate of convergence