a deterministic analysis of stochastic approximation with randomized directions we study the convergence of two stochastic approximation algorithms with randomized directions the simultaneous perturbation stochastic approximation algorithm and the random direction kiefer wolfowitz algorithm we establish deterministic necessary and sufficient conditions on the random directions and noise sequences for both algorithms and these conditions demonstrate the effect of the random directions on the sample path behavior of the algorithms studied we discuss ideas for further research in the analysis and design of these algorithms

the em algorithm an old folk song sung to a fast new tune celebrating the 20th anniversary of the presentation of the paper by dempster laird and rubin which popularized the em algorithm we investigate after a brief historical account strategies that aim to make the em algorithm converge faster while maintaining its simplicity and stability e g automatic monotone convergence in likelihood first we introduce the idea of a working parameter to facilitate the search for efficient data augmentation schemes and thus fast em implementations second summarizing various recent extensions of the em algorithm we formulate a general alternating expectation conditional maximization algorithm aecm that couples flexible data augmentation schemes with model reduction schemes to achieve efficient computations we illustrate these methods using multivariate t models with known or unknown degrees of freedom and poisson models for image reconstruction we show through both empirical and theoretical evidence the potential for a dramatic reduction in computational time with little increase in human effort we also discuss the intrinsic connection between em type algorithms and the gibbs sampler and the possibility of using the techniques presented here to speed up the latter the main conclusion of the paper is that with the help of statistical considerations it is possible to construct algorithms that are simple stable and fast

validation analysis of mirror descent stochastic approximation method the main goal of this paper is to develop accuracy estimates for stochastic programming problems by employing stochastic approximation sa type algorithms to this end we show that while running a mirror descent stochastic approximation procedure one can compute with a small additional effort lower and upper statistical bounds for the optimal objective value we demonstrate that for a certain class of convex stochastic programs these bounds are comparable in quality with similar bounds computed by the sample average approximation method while their computational cost is considerably smaller

variance based extragradient methods with line search for stochastic variational inequalities in this paper we propose dynamic sampled stochastic approximated ds sa extragradient methods for stochastic variational inequalities svis that are robust with respect to an unknown lipschitz c

lms algorithms for tracking slow markov chains with applications to hidden markov estimation and adaptive multiuser detection this paper analyzes the tracking properties of the least mean squares lms algorithm when the underlying parameter evolves according to a finite state markov chain with infrequent jumps first using perturbed liapunov function methods mean square error estimates are obtained for the tracking error then using recent results on two time scale markov chains mean ordinary differential equation and diffusion approximation results are obtained it is shown that a sequence of the centered tracking errors converges to an ordinary differential equation moreover a suitably scaled sequence of the tracking errors converges weakly to a diffusion process it is also shown that iterate averaging of the tracking algorithm results in optimal asymptotic convergence rate in an appropriate sense two application examples analysis of the performance of an adaptive multiuser detection algorithm in a direct sequence code division multiple access ds cdma system and tracking analysis of the state of a hidden markov model hmm with infrequent jumps are presented

on the choice of random directions for stochastic approximation algorithms stochastic approximation provides a simple and effective approach for finding roots and minima of functions whose evaluations are contaminated with noise we investigate variants of the random direction stochastic approximation rdsa algorithm for optimizing noisy loss functions in high dimensional spaces the most popular variant is random selection from a bernoulli distribution which also goes by the name simultaneous perturbation stochastic approximation spsa viable alternatives include an axis aligned distribution normal distribution and uniform distribution on a spherical shell although there are special cases where the bernoulli distribution is optimal we identify other cases where it performs worse than the alternatives we show that performance depends on the orientation of the loss function with respect to its coordinate axes and consider averages over all orientations we find that the average asymptotic performance depends only on the radial fourth moment of the distribution of random directions and is identical for the bernoulli axis aligned and spherical shell distributions of these variants the spherical shell is optimal in the sense of minimum variance over random orientations of the loss function with respect to the coordinate axes we also show that for unaligned loss functions the performance of the kiefer wolfowitz blum finite difference stochastic approximation fdsa is asymptotically equivalent to the rdsa algorithms and we observe numerically that the pre asymptotic performance of fdsa is often superior

a central limit theorem and hypotheses testing for risk averse stochastic programs we study statistical properties of the optimal value and optimal solutions of the sample average approximation of risk averse stochastic problems central limit theorem type results are derived for the optimal value when the stochastic program is expressed in terms of a law invariant coherent risk measure having a discrete kusuoka representation the obtained results are applied to hypotheses testing problems aiming at comparing the optimal values of several risk averse convex stochastic programs on the basis of samples of the underlying random vectors we also consider nonasymptotic tests based on confidence intervals on the optimal values of the stochastic programs obtained using the stochastic mirror descent algorithm numerical simulations show how to use our developments to choose among different distributions and on the considered class of risk averse stochastic programs the asymptotic tests show better results

a one measurement form of simultaneous perturbation stochastic approximation the simultaneous perturbation stochastic approximation spsa algorithm has proven very effective for difficult multivariate optimization problems where it is not possible to obtain direct gradient information as discussed to date spsa is based on a highly efficient gradient approximation requiring only two measurements of the loss function independent of the number of parameters being estimated this note presents a form of spsa that requires only one function measurement for any dimension theory is presented that identifies the class of problems for which this one measurement form will be asymptotically superior to the standard two measurement form

online estimation of the geometric median in hilbert spaces nonasymptotic confidence balls estimation procedures based on recursive algorithms are interesting and powerful techniques that are able to deal rapidly with very large samples of high dimensional data the collected data may be contaminated by noise so that robust location indicators such as the geometric median may be preferred to the mean in this context an estimator of the geometric median based on a fast and efficient averaged nonlinear stochastic gradient algorithm has been developed by bernoulli19 2013 18 43 this work aims at studying more precisely the nonasymptotic behavior of this nonlinear algorithm by giving nonasymptotic confidence balls in general separable hilbert spaces this new result is based on the derivation of improved l 2 rates of convergence as well as an exponential inequality for the nearly martingale terms of the recursive nonlinear robbins monro algorithm

on the design of a stable adaptive filter for state estimation in high dimensional systems design of specific parametrized gain structures provides a new tool for ensuring stability of reduced order filter for high dimensional systems which is optimized adaptively in the minimum prediction error framework

lp decodable permutation codes based on linearly constrained permutation matrices a set of linearly constrained permutation matrices are proposed for constructing a class of permutation codes the main feature of this class of permutation codes called linear programming lp decodable permutation codes is this lp decodability it is demonstrated that the lp decoding performance of the proposed class of permutation codes is characterized by the vertices of the code polytope of the code two types of linear constraints are discussed one is structured constraints and the other is random constraints the structured constraints allow an efficient encoding algorithm on the other hand the random constraints enable us to use probabilistic methods for analyzing several code properties such as the average cardinality and the average weight distribution

optimal bayesian experimental design for priors of compact support with application to shock tube experiments for combustion kinetics summary the analysis of reactive systems in combustion science and technology relies on detailed models comprising many chemical reactions that describe the conversion of fuel and oxidizer into products and the formation of pollutants shock tube experiments are a convenient setting for measuring the rate parameters of individual reactions the temperature pressure and concentration of reactants are chosen to maximize the sensitivity of the measured quantities to the rate parameter of the target reaction in this study we optimize the experimental setup computationally by optimal experimental design in a bayesian framework we approximate the posterior probability density functions pdf using truncated gaussian distributions in order to account for the bounded domain of the uniform prior pdf of the parameters the underlying gaussian distribution is obtained in the spirit of the laplace method more precisely the mode is chosen as the maximum a posteriori map estimate and the covariance is chosen as the negative inverse of the hessian of the misfit function at the map estimate the model related entities are obtained from a polynomial surrogate the optimality quantified by the information gain measures can be estimated efficiently by a rejection sampling algorithm against the underlying gaussian probability distribution rather than against the true posterior this approach offers a significant error reduction when the magnitude of the invariants of the posterior covariance are comparable with the size of the bounded domain of the prior we demonstrate the accuracy and superior computational efficiency of our method for shock tube experiments aiming to measure the model parameters of a key reaction which is part of the complex kinetic network describing the hydrocarbon oxidation in the experiments the initial temperature and fuel concentration are optimized with respect to the expected information gain in the estimation of the parameters of the target reaction rate we show that the expected information gain surface can change its shape dramatically according to the level of noise introduced into the synthetic data the information that can be extracted from the data saturates as a logarithmic function of the number of experiments and few experiments are needed when they are conducted at the optimal experimental design conditions furthermore inversion of the legacy data indicates the validity and robustness of our designs copyright 2016 john wiley sons ltd

simulation based optimization with stochastic approximation using common random numbers the method of common random numbers is a technique used to reduce the variance of difference estimates in simulation optimization problems these differences are commonly used to estimate gradients of objective functions as part of the process of determining optimal values for parameters of a simulated system asymptotic results exist which show that using the common random numbers method in the iterative finite difference stochastic approximation optimization algorithm fdsa can increase the optimal rate of convergence of the algorithm from the typical rate of k 1 3 to the faster k 1 2 where k is the algorithm s iteration number simultaneous perturbation stochastic approximation spsa is a newer and often much more efficient optimization algorithm and we will show that this algorithm too converges faster when the common random numbers method is used we will also provide multivariate asymptotic covariance matrices for both the spsa and fdsa errors

stochastic approximation method with gradient averaging for unconstrained problems a stochastic approximation algorithm is studied in which random gradient estimates are averaged by an auxiliary filter convergence of the algorithm is proved under a rather general noise condition a practical version of the method is described and numerical results are given

analysis and optimization of diagonally layered lattice schemes for mimo fading channels embodiments of the diagonal bell laboratories layered space time d blast architecture for multiple input multiple output mimo communication are developed wherein information symbol vectors are encoded using codewords from a lattice code called a diagonally layered lattice dll code which are formatted onto the diagonals of a space time frame decoding is done using a sphere decoder for each diagonal based on soft statistics obtained after zero forcing zf or minimum mean square error mmse filtering and decision feedback these operations give rise to an effective parallel channel model with channel gains with nonidentical statistics and additive noise which is gaussian in the zf filtering case and non gaussian in the mmse filtering case the so called full modulation diversity fmd property is nevertheless shown to yield the maximum achievable diversity orders over the mimo channel for both the zf and the mmse filtering based decoders respectively for any arbitrary fading distribution in the case of the independent identically distributed i i d rayleigh fading mimo channel with k transmit and n receive antennas with ngesk these diversity orders are nk k k 1 2 and nk for zf and mmse filtering based decoding respectively the error probability analysis also yields a design criterion for optimizing transmit power allocations several lattice design methods are proposed for the effective parallel channel models two methods are proposed to achieve high coding gain in the rayleigh fading mimo channel a third method is proposed that minimizes the exact symbol error probability sep and can be tailored for any given fading distribution a novel soft decision feedback decoder is also proposed based on the list sphere decoder to mitigate error propagation due to hard decision feedback the salient feature of the proposed dll schemes is that they have nearly full rate and full or high diversity order and yet a much lower decoding complexity than other existing full rate full diversity space time block codes stbcs the frame error probability fep performance of the optimized dll schemes for moderate to high spectral efficiencies and a wide range of signal to noise ratios snrs can be quite close to the performance of the best performing but more complex to decode stbcs moreover the proposed dll schemes significantly outperform other existing mimo systems of comparable decoding complexity

multistep stochastic mirror descent for risk averse convex stochastic programs based on extended polyhedral risk measures we consider risk averse convex stochastic programs expressed in terms of extended polyhedral risk measures we derive computable confidence intervals on the optimal value of such stochastic programs using the robust stochastic approximation and the stochastic mirror descent smd algorithms when the objective functions are uniformly convex we also propose a multistep extension of the stochastic mirror descent algorithm and obtain confidence intervals on both the optimal values and optimal solutions numerical simulations show that our confidence intervals are much less conservative and are quicker to compute than previously obtained confidence intervals for smd and that the multistep stochastic mirror descent algorithm can obtain a good approximate solution much quicker than its nonmultistep counterpart

ascent based monte carlo expectation maximization summary the expectation maximization em algorithm is a popular tool for maximizing likelihood functions in the presence of missing data unfortunately em often requires the evaluation of analytically intractable and high dimensional integrals the monte carlo em mcem algorithm is the natural extension of em that employs monte carlo methods to estimate the relevant integrals typically a very large monte carlo sample size is required to estimate these integrals within an acceptable tolerance when the algorithm is near convergence even if this sample size were known at the onset of implementation of mcem its use throughout all iterations is wasteful especially when accurate starting values are not available we propose a data driven strategy for controlling monte carlo resources in mcem the algorithm proposed improves on similar existing methods by recovering em s ascent i e likelihood increasing property with high probability being more robust to the effect of user defined inputs and handling classical monte carlo and markov chain monte carlo methods within a common framework because of the first of these properties we refer to the algorithm as ascent based mcem we apply ascent based mcem to a variety of examples including one where it is used to accelerate the convergence of deterministic em dramatically

least mean square algorithms with markov regime switching limit we analyze the tracking performance of the least mean square lms algorithm for adaptively estimating a time varying parameter that evolves according to a finite state markov chain we assume the markov chain jumps infrequently between the finite states at the same rate of change as the lms algorithm we derive mean square estimation error bounds for the tracking error of the lms algorithm using perturbed lyapunov function methods then combining results in two time scale markov chains with weak convergence methods for stochastic approximation we derive the limit dynamics satisfied by continuous time interpolation of the estimates unlike most previous analyzes of stochastic approximation algorithms the limit we obtain is a system of ordinary differential equations with regime switching controlled by a continuous time markov chain next to analyze the rate of convergence we take a continuous time interpolation of a scaled sequence of the error sequence and derive its diffusion limit somewhat remarkably for correlated regression vectors we obtain a jump markov diffusion finally two novel examples of the analysis are given for state estimation of hidden markov models hmms and adaptive interference suppression in wireless code division multiple access cdma networks

a direct search algorithm for optimization with noisy function evaluations we consider the unconstrained optimization of a function when each function evaluation is subject to a random noise we assume that there is some control over the variance of the noise term in the sense that additional computational effort will reduce the amount of noise this situation may occur when function evaluations involve simulation or the approximate solution of a numerical problem it also occurs in an experimental setting when averaging repeated observations at the same point can lead to a better estimate of the underlying function value we describe a new direct search algorithm for this type of problem we prove convergence of the new algorithm when the noise is controlled so that the standard deviation of the noise approaches zero faster than the step size we also report some numerical results on the performance of the new algorithm

codes in permutations and error correction for rank modulation codes for rank modulation have been recently proposed as a means of protecting flash memory devices from errors we study basic coding theoretic problems for such codes representing them as subsets of the set of permutations of n elements equipped with the kendall tau distance we derive several lower and upper bounds on the size of codes these bounds enable us to establish the exact scaling of the size of optimal codes for large values of n we also show the existence of codes whose size is within a constant factor of the sphere packing bound for any fixed number of errors

modeling delayed processes in biological systems delayed processes are ubiquitous in biological systems and are often characterized by delay differential equations ddes and their extension to include stochastic effects ddes do not explicitly incorporate intermediate states associated with a delayed process but instead use an estimated average delay time in an effort to examine the validity of this approach we study systems with significant delays by explicitly incorporating intermediate steps we show that such explicit models often yield significantly different equilibrium distributions and transition times as compared to ddes with deterministic delay values additionally different explicit models with qualitatively different dynamics can give rise to the same ddes revealing important ambiguities we also show that dde based predictions of oscillatory behavior may fail for the corresponding explicit model

on line estimation of dynamic shock error models based on the kullback leibler information measure develops two sequential or on line estimation schemes in the time domain for dynamic shock error models which are special cases of errors in variables models the author s approach utilizes a state space representation of the model kalman filtering techniques and on line algorithms the first on line algorithm is based on the expectation maximization algorithm and uses a recursive gauss newton scheme to maximize the kullback leibler information measure the second on line algorithm the author proposes is a gradient based scheme and uses stochastic approximations to maximize the log likelihood in comparison to the off line maximum likelihood estimation scheme used in ghosh 1989 the author s on line algorithms have significantly reduced computational costs and negligible memory requirements simulations illustrate the satisfactory performance of the algorithms in estimating errors in variables systems with parameters that vary slowly with time or undergo infrequent jump changes

reflection group codes and their decoding this paper builds on mittelholzer and lahtonen s study of group codes for the gaussian channel based on reflection groups a careful analysis of the action of a reflection group on its roots leads to the development of improved methods for encoding and decoding the new algorithm is proved to achieve maximum likelihood decoding the complexity of decoding is analyzed and it is shown that a proper choice of the sequence of subgroups used in the algorithm can yield significant gains in the efficiency of decoding

on sampling rates in simulation based recursions we consider the context of simulation based recursions that is recursions that involve quantities needing to be estimated using a stochastic simulation examples include stochastic adaptations of fixed point and gradient descent recursions obtained by replacing function and derivative values appearing within the recursion by their monte carlo counterparts the primary motivating settings are simulation optimization and stochastic root finding problems where the low point and the zero of a function are sought respectively with only monte carlo estimates of the functions appearing within the problem we ask how much monte carlo sampling needs to be performed within simulation based recursions in order that the resulting iterates remain consistent and more importantly efficient where efficient implies convergence at the fastest possible rate answering these questions involves trading off two types of error inherent in the iterates the deterministic error due to recursion and the stochastic er

a stochastic gradient type algorithm for closed loop problems we focus on the numerical solution of closed loop stochastic problems and propose a perturbed gradient algorithm to achieve this goal the main hurdle in such problems is the fact that the control variables are infinite dimensional due to e g the information constraints alternatively said control variables are feedbacks i e functions such controls have hence to be represented in a finite way in order to solve the problem numerically in the same way the gradient of the criterion is itself an infinite dimensional object our algorithm replaces this exact and unknown gradient by a perturbed one which consists of the product of the true gradient evaluated at a random point and a kernel function which extends this gradient to the neighbourhood of the random point proceeding this way we explore the whole space iteration after iteration through random points since each kernel function is perfectly known by a small number of parameters say n the control at iteration k is perfectly known as an infinite dimensional object by at most n k parameters the main strength of this method is that it avoids any discretization of the underlying space provided that we can sample as many points as needed in this space moreover our algorithm can take into account the possible measurability constraints of the problem in a new way finally the randomized strategy implemented by the algorithm causes the most probable parts of the space to be the most explored ones which is a priori an interesting feature in this paper we first prove two convergence results of this algorithm in the strongly convex and convex cases and then give some numerical examples showing the interest of this method for practical stochastic optimization problems

stochastic approximation with averaging and feedback rapidly convergent on line algorithms consider the stochastic approximation x sub n 1 x sub n a sub n g x sub n spl xi sub n where 0 0 the averaging method is essentially off line in the sense that the actual sa iterate x sub n is not influenced by the averaging in many applications x sub n itself is of greatest interest since that is the operating parameter this paper deals with the problem of stochastic approximation with averaging and with appropriate feedback of the averages into the original algorithm it is shown both mathematically and via simulation that it works very well and has numerous advantages it is a clear improvement over the system x sub n by itself it is fairly robust and quite often it is much preferable to the use of the above averages without feedback the authors deal in particular with linear algorithms of the type appearing in parameter estimators adaptive noise cancellers channel equalizers adaptive control and similar applications the main development is for the constant parameter case because of its importance in applications but analogous results hold for the case where a sub n spl rarr 0

convergence rate of moments in stochastic approximation with simultaneous perturbation gradient approximation and resetting the sequence of recursive estimators for function minimization generated by spall s 1998 simultaneous perturbation stochastic approximation spsa method combined with a suitable restarting mechanism is considered it is proved that this sequence converges under certain conditions with rate o n sup spl beta 2 for some spl beta 0 the best value being spl beta 2 3 where the rate is measured by the l sub q norm of the estimation error for any 1 spl les q spl infin the authors also present higher order spsa methods it is shown that the error exponent spl beta 2 can be arbitrarily close to 1 2 if the hessian matrix of the cost function at the minimizing point has all its eigenvalues to the right of 1 2 the cost function is sufficiently smooth and a sufficiently high order approximation of the derivative is used

statistical inference for the population landscape via moment adjusted stochastic gradients modern statistical inference tasks often require iterative optimization methods to compute the solution convergence analysis from an optimization viewpoint informs us only how well the solution is approximated numerically but overlooks the sampling nature of the data in contrast recognizing the randomness in the data statisticians are keen to provide uncertainty quantification or confidence for the solution obtained by using iterative optimization methods the paper makes progress along this direction by introducing moment adjusted stochastic gradient descent a new stochastic optimization method for statistical inference we establish non asymptotic theory that characterizes the statistical distribution for certain iterative methods with optimization guarantees on the statistical front the theory allows for model misspecification with very mild conditions on the data for optimization the theory is flexible for both convex and non convex cases remarkably the moment adjusting idea motivated from error standardization in statistics achieves a similar effect to acceleration in first order optimization methods that are used to fit generalized linear models we also demonstrate this acceleration effect in the non convex setting through numerical experiments

a parallel decomposition method for nonconvex stochastic multi agent optimization problems this paper considers the problem of minimizing the expected value of a possibly nonconvex cost function parameterized by a random vector variable when the expectation cannot be computed accurately e g because the statistics of the random variables are unknown and or the computational complexity is prohibitive classical stochastic gradient methods for solving this problem may suffer from slow convergence in this paper we propose a stochastic parallel successive convex approximation based best response algorithm for general nonconvex stochastic sum utility optimization problems which arise naturally in the design of multi agent networks the proposed novel decomposition approach enables all users to update their optimization variables in parallel by solving a sequence of strongly convex subproblems one for each user almost sure convergence to stationary points is proved we then customize the algorithmic framework to solve the stochastic sum rate maximization problem over single input single output siso frequency selective interference channels multiple input multiple output mimo interference channels and mimo multiple access channels numerical results corroborate that the proposed algorithms can converge faster than state of the art stochastic gradient schemes while achieving the same or better sum rates

adaptive stochastic approximation by the simultaneous perturbation method stochastic approximation sa has long been applied for problems of minimizing loss functions or root finding with noisy input information as with all stochastic search algorithms there are adjustable algorithm coefficients that must be specified and that can have a profound effect on algorithm performance it is known that choosing these coefficients according to an sa analog of the deterministic newton raphson algorithm provides an optimal or near optimal form of the algorithm however directly determining the required hessian matrix or jacobian matrix for root finding to achieve this algorithm form has often been difficult or impossible in practice the paper presents a general adaptive sa algorithm that is based on a simple method for estimating the hessian matrix while concurrently estimating the primary parameters of interest the approach applies in both the gradient free optimization kiefer wolfowitz and root finding stochastic gradient based robbins monro settings and is based on the simultaneous perturbation sp idea introduced previously the algorithm requires only a small number of loss function or gradient measurements per iteration independent of the problem dimension to adaptively estimate the hessian and parameters of primary interest aside from introducing the adaptive sp approach the paper presents practical implementation guidance asymptotic theory and a nontrivial numerical evaluation also included is a discussion and numerical analysis comparing the adaptive sp approach with the iterate averaging approach to accelerated sa

analysis and generic properties of gene regulatory networks with graded response functions the mass of new genomic data has lead to a growing interest in gene regulatory models describing the regulatory aspects of gene activity in these models the rates of change of gene product concentrations are expressed as a sum of regulatory switches in terms of sums of products of sigmoid functions mimicking complex logical on off functions turning gene activity on and off the customary way to analyse such equations is to replace the sigmoid functions by step functions and disregard models with autoregulation this leads to discontinuous equations of motion but in models without autoregulation continuous solutions can easily be obtained with effective autoregulation this simple approach breaks down as real regulators have finite gain and autoregulation is ubiquitous in biological systems we propose a generalised gene regulatory model framework admitting autoregulation and graded sigmoid functions with different steepnesses using singular perturbation analysis methods we derive a set of reformulated continuous equations for the limit solution when the sigmoids approach step functions and show that the solution for steep sigmoids approaches this limit solution uniformly in a finite time interval a conspicuous feature of the limit solution is so called sliding motionduring which the solution slides along a threshold hyperplane or the intersection of such hyperplanes a particular mapping leads to a dual picture of the flow where the fast parts are magnified at the expense of the slow parts combining the two dual pictures a simple and powerful method to analyse the flow is obtained we show that for steep sigmoid functions the flow may be highly sensitive to the relative steepnesses of the sigmoids this has important consequences when stochastic effects are taken into account also it implies that the observation by glass and

on stochastic gradient and subgradient methods with adaptive steplength sequences traditionally stochastic approximation sa schemes have been popular choices for solving stochastic optimization problems however the performance of standard sa implementations can vary significantly based on the choice of the steplength sequence and in general little guidance is provided about good choices motivated by this gap we present two adaptive steplength schemes for strongly convex differentiable stochastic optimization problems equipped with convergence theory that aim to overcome some of the reliance on user specific parameters the first scheme referred to as a recursive steplength stochastic approximation rsa scheme optimizes the error bounds to derive a rule that expresses the steplength at a given iteration as a simple function of the steplength at the previous iteration and certain problem parameters the second scheme termed as a cascading steplength stochastic approximation csa scheme maintains the steplength sequence as a piecewise constant decreasing function with the reduction in the steplength occurring when a suitable error threshold is met then we allow for nondifferentiable objectives but with bounded subgradients over a certain domain in such a regime we propose a local smoothing technique based on random local perturbations of the objective function that leads to a differentiable approximation of the function assuming a uniform distribution on the local randomness we establish a lipschitzian property for the gradient of the approximation and prove that the obtained lipschitz bound grows at a modest rate with problem size this facilitates the development of an adaptive steplength stochastic approximation framework which now requires sampling in the product space of the original measure and the artificially introduced distribution

effect of time delay on pattern formation competition between homogenisation and patterning abstract we study a mathematical model for juxtacrine signalling in a discrete lattice of cells as the signalling is assumed to be under transcriptional control and transcription is a time consuming process we incorporate time delays in the equations and study the effect of this on the pattern forming potential of the model previous models without time delays have shown that the mechanism is able to generate spatial patterns the analysis of the delay model reveals a transient competition between patterning and homogeneous oscillations a fine grained pattern eventually appears over the whole lattice but the duration of the oscillatory behaviour increases as the time delay increases the results illustrate the importance of including the known delays in a model and of studying transients as these may not be favourable to the system in addition the results may suggest that there are other mechanisms regulating the signalling than transcription for example protein protein interactions which would render the patterning process much faster

geometrically uniform codes a signal space code c is defined as geometrically uniform if for any two code sequences in c there exists an isometry that maps one sequence into the other while leaving the code c invariant geometrical uniformity a strong kind of symmetry implies such properties as a the distance profiles from code sequences in c to all other code sequences are all the same and b all voronoi regions of code sequences in c have the same shape it is stronger than ungerboeck zehavi wolf symmetry or calderbank sloane regularity nonetheless most known good classes of signal space codes are shown to be generalized coset codes and therefore geometrically uniform including a lattice type trellis codes based on lattice partitions lambda lambda such that z sup n lambda lambda 4z sup n is a lattice partition chain and b phase shift keying psk type trellis codes based on up to four way partitions of a 2 sup n psk signal set

adaptive estimation of hmm transition probabilities this paper presents new schemes for recursive estimation of the state transition probabilities for hidden markov models hmms via extended least squares els and recursive state prediction error rspe methods local convergence analysis for the proposed rspe algorithm is shown using the ordinary differential equation ode approach developed for the more familiar recursive output prediction error rpe methods the presented scheme converges and is relatively well conditioned compared with the previously proposed rpe scheme for estimating the transition probabilities that perform poorly in low noise the els algorithm presented is computationally of order n sup 2 which is less than the computational effort of order n sup 4 required to implement the rspe and previous rpe scheme where n is the number of markov states building on earlier work an algorithm for simultaneous estimation of the state output mappings and the state transition probabilities that requires less computational effort than earlier schemes is also presented and discussed implementation aspects of the proposed algorithms are discussed and simulation studies are presented to illustrate the convergence and convergence rates

on projected stochastic gradient descent algorithm with weighted averaging for least squares regression the problem of least squares regression of a d dimensional unknown parameter is considered a stochastic gradient descent based algorithm with weighted iterate averaging that uses a single pass over the data is studied and its convergence rate is analyzed we first consider a bounded constraint set of the unknown parameter under some standard regularity assumptions we provide an explicit o 1 k upper bound on the convergence rate depending on the variance due to the additive noise in the measurements and the size of the constraint set we show that the variance term dominates the error and decreases with rate 1 k while the term that is related to the size of the constraint set decreases with rate log k k 2 we then compare the asymptotic ratio rho between the convergence rate of the proposed scheme and the empirical risk minimizer erm as the number of iterations approaches infinity we show that rho leq 4 for all d geq 1 when the random entries of the sensing vector are uncorrelated and identically distributed we further improve the upper bound by showing that rho leq 4 3 for the case of d 1 and unbounded parameter set when the random sensing entries are equal across time simulation results demonstrate strong performance of the algorithm as compared to existing methods and coincide with rho leq 4 3 even for large d in practice

analysis of adaptive step size sa algorithms for parameter tracking we present proofs and data for adaptive step size algorithms for tracking time varying parameters when recursive stochastic approximation type algorithms are used a classical problem in adaptive control and communication theory concerns the tracking of the best fit of a given form when the statistics or the parameters change slowly a major and yet unresolved problem has been the choice of the step sizes in the tracking algorithm an algorithm for adapting the step size using the same system measurements which are used for the tracking was suggested by benveniste and various examples worked out by brossier the numerical results were very encouraging but proofs were lacking these proofs are supplied here together with supporting numerical data the proofs are based on recent results in stochastic approximation the adaptive step size technique works very well indeed much supporting analysis is presented particularly concerning the interpretation of certain stationary processes as stationary pathwise derivatives finite difference forms are also treated these are mathematically simpler and can be applied to a wide variety of systems even when the system is not well modeled the data shows that they work well

multivariate stochastic approximation using a simultaneous perturbation gradient approximation the problem of finding a root of the multivariate gradient equation that arises in function minimization is considered when only noisy measurements of the function are available a stochastic approximation sa algorithm for the general kiefer wolfowitz type is appropriate for estimating the root the paper presents an sa algorithm that is based on a simultaneous perturbation gradient approximation instead of the standard finite difference approximation of keifer wolfowitz type procedures theory and numerical experience indicate that the algorithm can be significantly more efficient than the standard algorithms in large dimensional problems

acceleration of stochastic approximation by averaging a new recursive algorithm of stochastic approximation type with the averaging of trajectories is investigated convergence with probability one is proved for a variety of classical optimization and identification problems it is also demonstrated for these problems that the proposed algorithm achieves the highest possible rate of convergence

stochastic root finding and efficient estimation of convex risk measures reliable risk measurement is a key problem for financial institutions and regulatory authorities the current industry standard value at risk has several deficiencies improved risk measures have been suggested and analyzed in the recent literature but their computational implementation has largely been neglected so far we propose and investigate stochastic approximation algorithms for the convex risk measure utility based shortfall risk our approach combines stochastic root finding schemes with importance sampling we prove that the resulting shortfall risk estimators are consistent and asymptotically normal and provide formulas for confidence intervals the performance of the proposed algorithms is tested numerically we finally apply our techniques to the normal copula model which is also known as the industry model creditmetrics this provides guidance for future implementations in practice

on complexity issues of online learning algorithms in this paper some new probabilistic upper bounds are presented for the online learning algorithm proposed in and more generally for linear stochastic approximations in hilbert spaces with these upper bounds not only does one recover almost sure convergence but also relaxes the square summable condition on the step size appeared in the early work furthermore two probabilistic upper bounds are given for an averaging process both of which achieve the same rate with respect to sample size as in batch learning algorithms and one of which is tight in both sample size and regularization parameter

combining monte carlo and mean field like methods for inference in hidden markov random fields issues involving missing data are typical settings where exact inference is not tractable as soon as nontrivial interactions occur between the missing variables approximations are required and most of them are based either on simulation methods or on deterministic variational methods while variational methods provide fast and reasonable approximate estimates in many scenarios simulation methods offer more consideration of important theoretical issues such as accuracy of the approximation and convergence of the algorithms but at a much higher computational cost in this work we propose a new class of algorithms that combine the main features and advantages of both simulation and deterministic methods and consider applications to inference in hidden markov random fields hmrfs these algorithms can be viewed as stochastic perturbations of variational expectation maximization vem algorithms which are not tractable for hmrf we focus more specifically on one of these perturbations and we prove their almost sure convergence to the same limit set as the limit set of vem in addition experiments on synthetic and real world images show that the algorithm performance is very close and sometimes better than that of other existing simulation based and variational em like algorithms

manifold identification in dual averaging for regularized stochastic online learning iterative methods that calculate their steps from approximate subgradient directions have proved to be useful for stochastic learning problems over large and streaming data sets when the objective consists of a loss function plus a nonsmooth regularization term the solution often lies on a low dimensional manifold of parameter space along which the regularizer is smooth when an l1 regularizer is used to induce sparsity in the solution for example this manifold is defined by the set of nonzero components of the parameter vector this paper shows that a regularized dual averaging algorithm can identify this manifold with high probability before reaching the solution this observation motivates an algorithmic strategy in which once an iterate is suspected of lying on an optimal or near optimal manifold we switch to a local phase that searches in this manifold thus converging rapidly to a near optimal point computational results are presented to verify the identification property and to illustrate the effectiveness of this approach

group codes generated by finite reflection groups slepian type group codes generated by finite coxeter groups are considered the resulting class of group codes is a generalization of the well known permutation modulation codes of slepian 1965 it is shown that a restricted initial point problem for these codes has a canonical solution that can easily be computed this allows one to enumerate all optimal group codes in this restricted sense and essentially solves the initial point problem for all finite reflection groups formulas for the cardinality and the minimum distance of such codes are given the new optimal group codes from exceptional reflection groups that are obtained achieve high rates and have excellent distance properties the decoding regions for maximum likelihood ml decoding are explicitly characterized and an efficient ml decoding algorithm is presented this algorithm relies on an extension of slepian s decoding of permutation modulation and has similar low complexity

on closed loop adaptive noise cancellation given the mean limit ordinary differential equation for the stochastic approximation defining the adaptive algorithm for a closed loop adaptive noise cancellation we characterize the limit points under appropriate conditions it is shown that as the dimension of the weight vector increases the sequence of corresponding limit points converges in the sense of l sub 2 to the infinite dimensional optimal weight vector also the limit point of the algorithm is nearly optimal if the dimension of the weight vector is large enough the gradient of the mean square error with respect to the weight vector evaluated at the limit goes to zero in l sub 1 and l sub 2 as the dimension increases as does the gradient with respect to the coefficients in the transfer function connecting the reference noise signal with the error output thus the algorithm is nearly a gradient descent algorithm and is error reducing for large enough dimension under broad conditions iterative averaging can be used to get a nearly optimal rate of convergence