an alternating variable method with varying replications for simulation response optimization simulation response optimization is an important problem often encountered in behaviorinvestigation of systems that are so complicated that the performance can only be evaluated by using simulation this paper modifies the alternating variable method used in deterministic optimization to suit the stochastic environment in simulation response optimization the main idea underlying the proposed method is to conduct several replications at each trial point to obtain reli able estimate of the theoretical response in particular the number of replications is not fixed but is set to a variable automatically adjusted on the basis of the distance between the two successive trial points to avoid misjudging the real different between two points due to the stochastic nature a t test instead of a simple comparison of the mean responses is performed empirical results from a stochastic watson function with nine variables a queueing problem and an inventory problem indicate that this method is able to find the optimal solutions in a statistical sense and the varying replications has demonstrated to be able to alleviate the computational burden in the whole optimization procedure moreover this method is robust with respect to the parameter used in determining the varying replications conducted at each trial point

convexity of sample path performance and strong consistency of infinitesimal perturbation analysis estimates a strong consistency of infinitesimal perturbation analysis ipa estimates for a class of stochastic processes satisfying certain convex properties is established the approach is based on convex analysis of sample paths applications of the results to tandem and cyclic queueing systems are discussed possible extensions to more general systems are also illustrated

a new stochastic derivative estimator for discontinuous payoff functions with application to financial derivatives motivated by infinitesimal perturbation analysis ipa and the likelihood ratio lr method we derive a new unbiased stochastic derivative estimator for a class of discontinuous payoff functions that arise in many options pricing settings from finance our method includes ipa and the lr method as special cases and can be applied to functions of more general forms containing indicator functions this new estimator can be computed from a single sample path or simulation whereas existing estimators generally require additional simulations for the class of discontinuous payoff functions considered here we apply this method to sensitivity analysis for european call options and american style call options and numerical experiments indicate that the estimator is computationally more efficient than other estimators

on the interchange of derivative and expectation for likelihood ratio derivative estimators sufficient conditions for the validity of interchange between derivative and expectation in the context of likelihood ratio gradient estimation were given in l ecuyer 1990 the aim of this paper is to shed additional light on these conditions and introduce specific variants of them which are often easier to check sufficient conditions for the derivative estimator to have finite moments up to a given order are also given and illustrated by examples in particular we give an example of an unbiased derivative estimator which satisfies the interchange conditions but which has infinite variance

a unified view of the ipa sf and lr gradient estimation techniques we study the links between the likelihood ratio lr gradient estimation technique sometimes called the score function sf method and infinitesimal perturbation analysis ipa we show how ipa can be viewed as a degenerate special case of the lr and sf techniques by selecting an appropriate representation of the underlying sample space for a given simulation experiment we also show how different definitions of the sample space yield different variants of the lr method some of them mixing ipa with more straightforward lr we illustrate this by many examples we also give sufficient conditions under which the gradient estimators are unbiased

a simulation based algorithm for ergodic control of markov chains conditioned on rare events we study the problem of long run average cost control of markov chains conditioned on a rare event in a related recent work a simulation based algorithm for estimating performance measures associated with a markov chain conditioned on a rare event has been developed we extend ideas from this work and develop an adaptive algorithm for obtaining online optimal control policies conditioned on a rare event our algorithm uses three timescales or step size schedules on the slowest timescale a gradient search algorithm for policy updates that is based on one simulation simultaneous perturbation stochastic approximation spsa type estimates is used deterministic perturbation sequences obtained from appropriate normalized hadamard matrices are used here the fast timescale recursions compute the conditional transition probabilities of an associated chain by obtaining solutions to the multiplicative poisson equation for a given policy estimate further the risk parameter associated with the value function for a given policy estimate is updated on a timescale that lies in between the two scales above we briefly sketch the convergence analysis of our algorithm and present a numerical application in the setting of routing multiple flows in communication networks

machine learning and nonparametric bandit theory in its most basic form bandit theory is concerned with the design problem of sequentially choosing members from a given collection of random variables so that the regret i e r sub n spl sigma sub j spl mu spl mu sub j et sub n j grows as slowly as possible with increasing n here spl mu sub j is the expected value of the bandit arm i e random variable indexed by j t sub n j is the number of times arm j has been selected in the first n decision stages and spl mu sup sup sub j spl mu sub j the present paper contributes to the theory by considering the situation in which observations are dependent to begin with the dependency is presumed to depend only on past observations of the same arm but later we allow that it may be with respect to the entire past and that the set of arms is infinite this brings queues and more generally controlled markov processes into our purview thus our black box methodology is suitable for the case when the only observables are cost values and in particular the probability structure and loss function are unknown to the designer the conclusion of the analysis is that under lenient conditions using algorithms prescribed herein risk growth is commensurate with that in the simplest i i d cases our methods represent an alternative to stochastic approximation perturbation analysis ideas for tuning queues

on line sensitivity analysis of markov chains discrete event systems modeled as continuous time markov processes and characterized by some integer valued parameter are considered the problem addressed is that of estimating performance sensitivities with respect to this parameter by directly observing a single sample path of the system the approach is based on transforming the nominal markov chain into a reduced augmented chain the stationary state probabilities which can be easily combined to obtain stationary state probability sensitivities with respect to the given parameter under certain conditions the reduced augmented chain state transitions are observable with respect to the state transitions of the system itself and no knowledge of the nominal markov chain state of the transition rates is required applications for some queueing systems are included the approach incorporates estimation of unknown transition rates when needed and is extended to real valued parameters

dynamically identifying regenerative cycles in simulation based optimization algorithms for markov chains simulation based algorithms for maximizing the average reward of a parameterized markov chain often rely upon the existence of a state which is recurrent for all choices of parameter values the question of which recurrent state should serve to mark the end of a regenerative cycle is a very important practical consideration in applications even when all of the states of the process are recurrent some states tend to be visited more often than others and lengthy renewal cycles tend to result in high variance estimates of the gradient to address this difficulty we analyze a mechanism for adjusting this special state dynamically i sup adaptation as applied to the batch simulation based optimization algorithm of a previous paper we show that the desirable convergence properties of the original batch algorithm are retained with i sup adaptation namely the almost sure convergence of the parameter vector to a critical point

approximating an optimal production policy in a continuous flow line recurrence and asymptotic properties this work is concerned with manufacturing systems with two failure prone tandem machines the production is regulated by a continuous version of buffer control our goal is to obtain an optimal buffer control policy to minimize a long run average cost function concentrating on threshold type of control policies our effort is devoted to parameter optimization problems for the continuous material produce to stock models we estimate the gradients of the cost function with respect to the parameter using perturbation analysis techniques and approximate the optimal value of the parameter via a constant step size stochastic approximation algorithm an analysis for error accumulation in perturbation propagation is undertaken and a sufficient condition for breaking the propagation chain is derived in addition we show that the event of breaking the perturbation propagation chain is recurrent if the system has sufficient capacity derive the consistency of the gradient estimators and establish the convergence of the iterative algorithm we also treat non markovian models with the machine repair time following an erlang distribution and provide numerical examples to illustrate the proposed algorithm

on sampling controlled stochastic approximation the authors examine a novel class of stochastic approximation procedures which are based on carefully controlling the number of observations or measurements taken before each computational iteration this method referred to as sampling controlled stochastic approximation has advantages over standard stochastic approximation such as requiring less computation and the ability to handle bias in estimation the authors address the growth rate required of the number of samples and prove a general convergence theorem for the proposed stochastic approximation method in addition they present applications to optimize and also derive a sampling controlled version of the classic robbins munro algorithm

asymptotics of likelihood ratio derivative estimators in simulations of highly reliable markovian systems we discuss the estimation of derivatives of a performance measure using the likelihood ratio method in simulations of highly reliable markovian systems we compare the difficulties of estimating the performance measure and of estimating its partial derivatives with respect to component failure rates as the component failure rates tend to 0 and the component repair rates remain fixed we first consider the case when the quantities are estimated using naive simulation i e when no variance reduction technique is used in particular we prove that in the limit some of the partial derivatives can be estimated as accurately as the performance measure itself this result is of particular interest in light of the somewhat pessimistic empirical results others have obtained when applying the likelihood ratio method to other types of systems however the result only holds for certain partial derivatives of the performance measure when using naive simulation more specifically we can estimate a certain partial

lacunary ideal convergence in intuitionistic fuzzy normed linear spaces the aim of this article is to introduce and study the notion of lacunary ideal convergence in intuitionistic fuzzy normed linear spaces as a variant of the notion of ideal convergence also a new concept called lacunary convergence is introduced using these two notions lacunary i limit points and lacunary i cluster points have been defined and the relation between them has been established characterization for lacunary ideal convergence preserving linear operators has been given furthermore the notions of lacunary cauchy and lacunary i cauchy sequences are introduced and studied

stochastic optimization by simulation convergence proofs for the gi g 1 queue in steady state approaches like finite differences with common random numbers infinitesimal perturbation analysis and the likelihood ratio method have drawn a great deal of attention recently as ways of estimating the gradient of a performance measure with respect to continuous parameters in a dynamic stochastic system in this paper we study the use of such estimators in stochastic approximation algorithms to perform so called single run optimizations of steady state systems under mild conditions for an objective function that involves the mean system time in a gi g 1 queue we prove that many variant of these algorithms converge to the minimizer in most cases however the simulation length must be increased from iteration to iteration otherwise the algorithm may converge to the wrong value one exception is a particular implementation of infinitesimal perturbation analysis for which the single run optimization converges to the optimum even with a fixed and small number of ends of service per iteration as a by product of our convergence proofs we obtain some properties of the derivative estimators that could be of independent interest our analysis exploits the regenerative structure of the system but our derivative estimation and optimization algorithms do not always take advantage of that regenerative structure in a companion paper we report numerical experiments with an m m 1 queue which illustrate the basis convergence properties and possible pitfalls of the various techniques

global stochastic optimization with low dispersion point sets this study concerns a generic model free stochastic optimization problem requiring the minimization of a risk function defined on a given bounded domain in a euclidean space smoothness assumptions regarding the risk function are hypothesized and members of the underlying space of probabilities are presumed subject to a large deviation principle however the risk function may well be nonconvex and multimodal a general approach to finding the risk minimizer on the basis of decision observation pairs is proposed it consists of repeatedly observing pairs over a collection of design points principles are derived for choosing the number of these design points on the basis of an observation budget and for allocating the observations between these points in both prescheduled and adaptive settings on the basis of these principles large deviation type bounds of the minimizer in terms of sample size are established

on the ideal convergence of sequences of fuzzy numbers in this paper we introduce and study the concepts of i convergence i convergence and i cauchy sequence for sequences of fuzzy numbers where i denotes the ideal of subsets of n the set of positive integers

on perturbation analysis of queueing networks with finitely supported service time distributions infinitesimal perturbation analysis ipa has emerged as an efficient tool for estimating the gradient of a function defined on the steady state of a queuing network differentiability of such functions is often assumed due to difficulties in proving it it is pointed out that such functions may be nondifferentiable at an infinite set of points dense in a given interval for a large class of realistic system models this phenomenon is largely due to correlation of traffic patterns on the links of a network and to the presence of atoms in the distributions of their service times the issue of nondifferentiability goes beyond ipa to any method for estimating the gradients of functions in steady state

convergence rates of perturbation analysis robbins monro single run algorithms for single server queues in this paper the perturbation analysis robbins monro single run algorithm is applied to estimating the optimal parameter of a performance measure for the gi g 1 queueing systems where the algorithm is updated after every fixed length observation period our aim is to analyze the limiting behavior of the algorithm the almost sure convergence rate of the algorithm is established it is shown that the convergence rate depends on the second derivative of the performance measure at the optimal point

a stochastic quasi newton method for simulation response optimization abstract simulation response optimization has wide applications for management of systems that are so complicated that the performance can only be evaluated by using simulation this paper modifies the quasi newton method used in deterministic optimization to suit the stochastic environment in simulation response optimization the basic idea is to use the estimated subgradient calculated from different replications and a metric matrix updated from the broyden fletcher goldfarb shanno bfgs formula to yield a quasi newton search direction to avoid misjudging the minimal point in both the line search and the quasi newton iterations due to the stochastic nature a t test instead of a simple comparison of the mean responses is performed it is proved that the resulting stochastic quasi newton algorithm is able to generate a sequence that converges to the optimal point under certain conditions empirical results from a four station queueing problem and an s s inventory problem indicate that this method is able to find the optimal solutions in a statistical sense moreover this method is robust with respect to the number of replications conducted at each trial point

large deviations for a random walk model with state dependent noise in this paper we prove the large deviation principle for a class of random walks with state dependent noise this type of model has important applications in queueing and communication theory and in the area of stochastic approximation

event based optimization of markov systems recent research indicates that markov decision processes mdps and perturbation analysis pa based optimization can be derived easily from two fundamental performance sensitivity formulas with this sensitivity point of view an event based optimization approach including event based sensitivity analysis and event based policy iteration was proposed via an example by x r cao discrete event dyn syst theory appl vol 15 pp 169 197 2005 this approach utilizes the special feature of a system and illustrates how the potentials can be aggregated using the special feature the approach applies to many practical problems that do not fit well the standard mdp formulation this note provides a mathematical formulation and proves the main results for this approach

poisson arrivals see time averages in many stochastic models particularly in queueing theory poisson arrivals both observe see a stochastic process and interact with it in particular cases and or under restrictive assumptions it has been shown that the fraction of arrivals that see the process in some state is equal to the fraction of time the process is in that state in this paper we present a proof of this result under one basic assumption the process being observed cannot anticipate the future jumps of the poisson process

gradient estimation for sensitivity analysis and adaptive multiuser interference rejection in code division multiple access systems in this paper we consider a direct sequence code division multiple access ds cdma system in the framework of a discrete event dynamic system deds in order to optimize the system performance based on this formulation we develop infinitesimal perturbation analysis ipa for estimating the sensitivity of the average probability of bit error to factors ranging from near far effects to imperfections in power control the above estimates are shown to be unbiased and this technique is then further incorporated into a stochastic gradient algorithm for achieving adaptive multiuser interference rejection for such systems which is also subject to frequency nonselective slow fading we use an ipa based stochastic training algorithm for developing an adaptive linear detector with the average probability of error being the minimization criterion we also develop a practical implementation of such an adaptive detector where we use a joint estimation detection algorithm for minimizing the average probability of bit error a sequential implementation that does not require a stochastic training sequence or a preamble is also developed

applications of a kushner and clark lemma to general classes of stochastic algorithms two general classes of stochastic algorithms are considered including algorithms considered by ljung as well as algorithms of the form theta n 1 theta n gamma n 1 v n 1 theta n z where z is a stationary ergodic process it is shown how one can apply a lemma of kushner and clark to obtain properties of these algorithms this is done by using in particular martingale arguments in the generalized ljung case in these various situations the convergence is obtained by the method of the associated ordinary differential equation under the classical boundedness assumptions in the case of linear algorithms the boundedness assumptions are dropped

on closed loop adaptive noise cancellation given the mean limit ordinary differential equation for the stochastic approximation defining the adaptive algorithm for a closed loop adaptive noise cancellation we characterize the limit points under appropriate conditions it is shown that as the dimension of the weight vector increases the sequence of corresponding limit points converges in the sense of l sub 2 to the infinite dimensional optimal weight vector also the limit point of the algorithm is nearly optimal if the dimension of the weight vector is large enough the gradient of the mean square error with respect to the weight vector evaluated at the limit goes to zero in l sub 1 and l sub 2 as the dimension increases as does the gradient with respect to the coefficients in the transfer function connecting the reference noise signal with the error output thus the algorithm is nearly a gradient descent algorithm and is error reducing for large enough dimension under broad conditions iterative averaging can be used to get a nearly optimal rate of convergence

sample path optimization of convex stochastic performance functions in this paper we propose a method for optimizing convex performance functions in stochastic systems these functions can include expected performance in static systems and steady state performance in discrete event dynamic systems they may be nonsmooth the method is closely related to retrospective simulation optimization it appears to overcome some limitations of stochastic approximation which is often applied to such problems we explain the method and give computational results for two classes of problems tandem production lines with up to 50 machines and stochastic pert program evaluation and review technique problems with up to 70 nodes and 110 arcs

optimization of the transient and steady state behavior of discrete event systems we present a general framework for applying simulation to optimize the behavior of discrete event systems our approach involves modeling the discrete event system under study as a general state space markov chain whose distribution depends on the decision parameters we then show how simulation and the likelihood ratio method can be used to evaluate the performance measure of interest and its gradient and we present conditions that guarantee that the robbins monro stochastic approximation algorithm will converge almost surely to the optimal values of the decision parameters both transient and steady state performance measures are considered for steady state performance measures we consider both the case when the markov chain of interest is regenerative in the standard sense as well as the case when this markov chain is harris recurrent and thereby regenerative in a wider sense

simulation based optimization of markov reward processes this paper proposes a simulation based algorithm for optimizing the average reward in a finite state markov reward process that depends on a set of parameters as a special case the method applies to markov decision processes where optimization takes place within a parametrized set of policies the algorithm relies on the regenerative structure of finite state markov processes involves the simulation of a single sample path and can be implemented online a convergence result with probability 1 is provided

consistency of infinitesimal perturbation analysis for the gi g m queue abstract we extend the infinitesimal perturbation analysis algorithm developed for the gi g 1 queue to the gi g m queue the primary difference is the need for additional accumulators for each server for the m m 2 queue a consistency proof is given which is extendable to the m m m queue for the more general gi g m queue a heuristic argument is given as to why the algorithm works in this argument it is seen that the need for the servers to be identical is crucial for the correctness of the algorithm numerical results are provided

infinitesimal perturbation analysis estimates for moments of the system time of an m m 1 queue we consider infinitesimal perturbation analysis estimates for the derivative of moments of the system time in an m m 1 queue in steady state the unbiasedness of these estimates is established for derivatives with respect to the mean service time and the arrival rate a similar result is obtained for the system time of the ith customer in a gi g 1 queue with fixed initial conditions

on line optimization of simulated markovian processes let zn be a markovian process the transition of which depends on a control parameter x let x be its invariant law it is shown that the solution of the optimization problem fx hz xd xz min x s can be found with a recursive estimation procedure of the stochastic approximation type the method consists in finding a stochastic quasigradient of fx and in adapting the parameter x in the direction of descent an a s convergence is proved and a practical example is given

using stochastic approximation methods to compute optimal base stock levels in inventory control problems in this paper we consider numerous inventory control problems for which the base stock policies are known to be optimal and we propose stochastic approximation methods to compute the optimal base stock levels the existing stochastic approximation methods in the literature guarantee that their iterates converge but not necessarily to the optimal base stock levels in contrast we prove that the iterates of our methods converge to the optimal base stock levels moreover our methods continue to enjoy the well known advantages of the existing stochastic approximation methods in particular they only require the ability to obtain samples of the demand random variables rather than to compute expectations explicitly and they are applicable even when the demand information is censored by the amount of available inventory

optimal design via curve fitting of monte carlo experiments abstract this article explores numerical methods for stochastic optimization with special attention to bayesian design problems a common and challenging situation occurs when the objective function in bayesian applications the expected utility is very expensive to evaluate perhaps because it requires integration over a space of very large dimensionality our goal is to explore a class of optimization algorithms designed to gain efficiency in such situations by exploiting smoothness of the expected utility surface and borrowing information from neighboring design points the central idea is that of implementing stochastic optimization by curve fitting of monte carlo samples this is done by simulating draws from the joint parameter sample space and evaluating the observed utilities fitting a smooth surface through these simulated points serves as estimate for the expected utility surface the optimal design can then be found deterministically in this article we introduce a general algorithm for cu

perturbation analysis of discrete event systems concepts algorithms and applications abstract perturbation analysis of discrete event systems is a technique that provides estimates of performance sensitivities with respect to system parameters by analyzing a single sample path of a discrete event system this technique saves a great amount of computation in simulation and can be applied to on line performance optimization of real world systems where changing the values of parameters is infeasible this paper gives an overview of the main concepts general methodologies applications and further directions of the area

potential based online policy iteration algorithms for markov decision processes performance potentials play a crucial role in performance sensitivity analysis and policy iteration of markov decision processes the potentials can be estimated on a single sample path of a markov process in this paper we propose two potential based online policy iteration algorithms for performance optimization of markov systems the algorithms are based on online estimation of potentials and stochastic approximation we prove that with these two algorithms the optimal policy can be attained after a finite number of iterations a simulation example is given to illustrate the main ideas and the convergence rates of the algorithms

algorithms for edge coloring bipartite graphs and multigraphs a minimum edge coloring of a bipartite graph is a partition of the edges into delta matchings where delta is the maximum degree in the graph coloring algorithms that run in time o min m log n 2 n 2 log n are presented the algorithms rely on an efficient procedure for the special case of delta an exact power of two the coloring algorithms can be used to find maximum cardinality matchings on regular bipartite graphs in the above time bound an algorithm for coloring multigraphs with large multiplicities is also presented

bipartite edge coloring in o delta m time we show that a minimum edge coloring of a bipartite graph can be found in o delta m time where delta and m denote the maximum degree and the number of edges of g respectively it is equivalent to finding a perfect matching in a k regular bipartite graph in o km time by sharpening the methods a minimum edge coloring of a bipartite graph can be found in o p max delta log delta m time where p max delta is the largest prime factor of delta moreover a perfect matching in a k regular bipartite graph can be found in o pmax k m time

stochastic optimization of regenerative systems using infinitesimal perturbation analysis we consider the problem of optimizing the steady state mean of a controlled regenerative process using a stochastic optimization algorithm driven by infinitesimal perturbation analysis ipa derivative estimates we derive ipa derivative estimates for our problem and prove almost sure convergence of the algorithm the generality of our formulation should encompass a wide variety of practical systems we illustrate our framework and results via several examples

optimization of queues using an infinitesimal perturbation analysis based stochastic algorithm with general update times convergence with probability one of a stochastic optimization algorithm for a single server queue is proved the parameter to be optimized is updated using an infinitesimal perturbation analysis estimate of the gradient of the performance measure and the updates are performed at general times first an algorithm in which the parameter is updated before each customer begins service is considered then it is shown how this analysis can be extended to an algorithm that updates at certain stopping times the analysis suggests that the sample path behavior of the algorithm is similar to that of an algorithm that updates at the start of every busy period

continuous optimization via simulation using golden region search simulation optimization so is a class of mathematical optimization techniques in which the objective function can only be numerically evaluated through simulation in this paper a new so approach called golden region gr search is developed for continuous problems gr divides the feasible region into a number of sub regions and selects one region in each iteration for further search based on the quality and distribution of simulated points in the feasible region and the result of scanning the response surface through a metamodel monte carlo experiments show that the gr method is efficient compared to three well established approaches in the literature we also prove the asymptotic convergence in probability to a global optimum for a large class of random search methods in general and gr in particular

a stochastic approximation algorithm with varying bounds many optimization problems that are intractable with conventional approaches will yield to stochastic approximation algorithms this is because these algorithms can be used to optimize functions that cannot be evaluated analytically but have to be estimated for instance through simulation or measured thus stochastic approximation algorithms can be used for optimization in simulation unfortunately the classical stochastic approximation algorithm sometimes diverges because of unboundedness problems we study the convergence of a variant of stochastic approximation defined over a growing sequence of compact sets we show that this variant converges under more general conditions on the objective function than the classical algorithm while maintaining the same asymptotic convergence rate we also present empirical evidence that shows that this algorithm sometimes converges much faster than the classical algorithm

weak differentiability of product measures in this paper we study cost functions over a finite collection of random variables for these types of models a calculus of differentiation is developed that allows us to obtain a closed form expression for derivatives where differentiation has to be understood in the weak sense the technique for proving the results is new and establishes an interesting link between functional analysis and gradient estimation the key contribution of this paper is a product rule of weak differentiation in addition a product rule of weak analyticity is presented that allows for taylor series approximations of finite products measures in particular from characteristics of the individual probability measures a lower bound i e domain of convergence can be established for the set of parameter values for which the taylor series converges to the true value applications of our theory to the ruin problem from insurance mathematics and to stochastic activity networks arising in project evaluation review techniques are provided

an event domain formalism for sample path perturbation analysis of discrete event dynamic systems we present a state space representation of a general discrete event dynamic system deds model on the basis of a natural event domain which replaces time perturbation analysis of a sample path of a deds is then formalized using some fundamental concepts introduced to establish conditions for its validity a set of simple perturbation equations is then derived under these conditions so that performance gradients with respect to parameters are estimated from observation of real or simulated data on a single sample path

structural conditions for perturbation analysis derivative estimation finite time performance indices in recent years there has been a surge of research into methods for estimating derivatives of performance measures from sample paths of stochastic systems in the case of queueing systems typical performance measures are mean queue lengths throughputs etc and the derivatives estimated are with respect to system parameters such as parameters of service and interarrival time distributions derivative estimates potentially offer a general means of optimizing performance and are useful in sensitivity analysis this paper concerns one approach to derivative estimation known as infinitesimal perturbation analysis we first develop a general framework for these types of estimates then give simple sufficient conditions for them to be unbiased the key to our results is identifying conditions under which certain finite horizon performance measures are almost surely continuous functions of the parameter of differentiation throughout an interval the sufficient conditions we introduce are formulated in the setting of generalized semi markov processes but translate into readily verifiable conditions for queueing systems these results substantially extend the domain of problems in which infinitesimal perturbation analysis is provably applicable

asymptotic efficiency of perturbation analysis based stochastic approximation with averaging central limit theorems are obtained for the perturbation analysis robbins monro single run parmsr algorithm updated either after every regenerative cycle or after every fixed length observation period and with averaging of the iterates for one dependent regenerative processes when the convergence to the optimizer is expressed in terms of the total observation time of the system or the total computing budget in the case of a simulation the convergence rate and the limit covariance matrix turn out to be the same for all updating schemes and are optimal within the class of stochastic approximation algorithms under certain assumptions a bound on the strong convergence rate of the usual parmsr algorithm updated after every fixed length observation period is established using a limit theorem on double array martingales this is the key step for obtaining central limit theorems for the algorithms with averaging and has interest in its own right

on the learning mechanism of adaptive filters this paper highlights both analytically and by simulations some interesting phenomena regarding the behavior of ensemble average learning curves of adaptive filters that may have gone unnoticed among other results the paper shows that even ensemble average learning curves of single tap lms filters actually exhibit two distinct rates of convergence one for the initial time instants and another faster one for later time instants in addition such curves tend to converge faster than predicted by mean square theory and can converge even when a mean square stability analysis predicts divergence these effects tend to be magnified by increasing the step size two of the conclusions that follow from this work are 1 the mean square stability alone may not be the most appropriate performance measure especially for larger step sizes a combination of mean square stability and almost sure a s stability seems to be more appropriate 2 care is needed while interpreting ensemble average curves for larger step sizes the curves can lead to erroneous conclusions unless a large number of experiments are averaged at times of the order of tens of thousands or higher

convergence properties of infinitesimal perturbation analysis infinitesimal perturbation analysis ipa is a method for computing a sample path derivative with respect to an input parameter in a discrete event simulation the ipa algorithm is based on the fact that for certain parameters and any realization of a simulation the change in parameter can be made small enough so that only the times of events get shifted but their order does not change this paper considers the convergence properties of the ipa sample path derivatives in particular the question of when an ipa estimate converges to the derivative of a steady state performance measure is studied necessary and sufficient conditions for this convergence are derived for a class of regenerative processes although these conditions are not guaranteed to be satisfied in general they are satisfied for the mean stationary response time in the m g 1 queue a necessary condition for multiple ipa estimates to simultaneously converge to the derivatives of steady state throughputs in a queueing network is determined the implications of this necessary condition are that except in special cases the original ipa algorithm cannot be used to consistently estimate steady state throughput derivatives in queueing networks with multiple types of customers state dependent routing or blocking numerical studies on ipa convergence properties are also presented

estimation of derivatives of nonsmooth performance measures in regenerative systems we investigate the problem of estimating derivatives of expected steady state performance measures in parametric systems unlike most of the existing work in the area we allow those functions to benonsmooth and study the estimation ofdirectional derivatives for the class of regenerative markovian systems we provide conditions under which we can obtainconsistent estimators of those directional derivatives an example illustrates that the conditions imposed must be different from those in the differentiable case the result also allows us to derive necessary and sufficient conditions for differentiability of the expected steady state function we then analyze the process formed by thesubdifferentials of the original process and show that the subdifferential set of the expected steady state function can be expressed as an average of integrals of multifunctions which is the approach commonly found in the literature for integrals of sets the latter result can also be viewed as a limit theorem for more general compact convex multivalued processes

edge coloring of bipartite graphs with constraints a classical result from graph theory states that the edges of an l regular bipartite graph can be colored using exactly l colors so that edges that share an endpoint are assigned different colors in this paper we study two constrained versions of the bipartite edge coloring problem some of the edges adjacent to a specific pair of opposite vertices of an l regular bipartite graph are already colored with s colors that appear only on one edge single colors and d colors that appear on two edges double colors we show that the rest of the edges can be colored using at most max min l d 3l 2 l s d 2 total colors we also show that this bound is tight by constructing instances in which max min l d 3l 2 l s d 2 colors are indeed necessary some of the edges of an l regular bipartite graph are already colored with s colors that appear only on one edge we show that the rest of the edges can be colored using at most max l s 2 s total colors we also show that this bound is tight by constructing instances in which max l s 2 s total colors are necessary

adaptive control for manufacturing systems using infinitesimal perturbation analysis combining infinitesimal perturbation analysis ipa with stochastic approximation gives identification algorithms to estimate the optimal threshold value for failure prone manufacturing systems consisting of one machine producing one part type two adaptive control schemes are proposed the adaptive control schemes do not require the knowledge of the distribution functions of the up and down times under some appropriate conditions the strong consistency as well as the convergence rates of the identification algorithms and the cost function is established for the adaptive control schemes in particular it is shown that central limit theorems hold for the identification algorithms

some guidelines and guarantees for common random numbers common random numbers crn is a widely used technique for reducing variance in comparing stochastic systems through simulation its popularity derives from its intuitive appeal and ease of implementation however though crn has been observed to work well with a broad range of models the class of systems for which it is provably advantageous has remained rather limited this paper has two purposes we first discuss the effectiveness and optimality of crn in a general setting stressing the roles played by monotonicity and continuity properties we then present specific new classes of systems and comparisons for which crn is beneficial and even optimal our conclusions for these systems are largely consistent with simulation practice and lend further theoretical support to folklore our results differ from those of previous analyses primarily because we put conditions on the timing of events rather than the sequence of states in a discrete event simulation we formulate our results in three settings corresponding to three applications of crn distributional comparisons structural comparisons and sensitivity analysis in each case we make use of conditions that simultaneously ensure monotonicity and continuity in the timing of events these properties are established through explicit recursions for event epochs in terms of increasing continuous functions

stochastic approximation methods for systems over an infinitehorizon the paper develops efficient and general stochastic approximation sa methods for improving the operation of parametrized systems of either the continuous or discrete event dynamical systems types and which are of interest over a long time period for example one might wish to optimize or improve the stationary or average cost per unit time performance by adjusting the systems parameters the number of applications and the associated literature are increasing at a rapid rate this is partly due to the increasing activity in computing pathwise derivatives and adapting them to the average cost problem although the original motivation and the examples come from an interest in the infinite horizon problem the techniques and results are of general applicability in sa we present an updating and review of powerful ordinary differential equation type methods in a fairly general context and based on weak convergence ideas the results and proof techniques are applicable to a wide variety of applications exploiting the full potential of these ideas can greatly simplify and extend much current work their breadth as well as the relative ease of using the basic ideas are illustrated in detail via typical examples drawn from discrete event dynamical systems piecewise deterministic dynamical systems and a stochastic differential equations model in these particular illustrations we use either infinitesimal perturbation analysis type estimators mean square derivative type estimators or finite difference type estimators markov and non markov models are discussed the algorithms for distributed asynchronous updating as well as the fully synchronous schemes are developed

constrained optimization via stochastic approximation with a simultaneous perturbation gradient approximation this paper deals with a projection algorithm for stochastic approximation using simultaneous perturbation gradient approximation for optimization under inequality constraints where no direct gradient of the loss function is available and the inequality constraints are given as explicit functions of the optimization parameters it is shown that under application of the projection algorithm the parameter iterate converges almost surely to a kuhn tucker point the procedure is illustrated by a numerical example

addendum to extensions and generalizations of smoothed perturbation analysis in a generalized semi markov process framework under a very general framework both in terms of finite time performance measures and system structure the authors derive smoothed perturbation analysis spa estimators and prove their unbiasedness the commuting condition which has been key in previous work is not required a priori and thus the framework includes such systems as the gi g 1 k queue and multiclass queueing networks which do not satisfy the commuting condition the generality achieved is traded off against the fact that the estimator is not always easily implementable on a single sample path the use of the commuting condition in a local sense is proposed to help simplify the estimators derived queueing and multiclass queueing networks are used as illustrative examples for a simple multiclass closed queueing network some simulation results are provided when the commuting condition is satisfied globally the framework allows the recovery of previous results on ipa and spa estimators as corollaries of the main theorems

perturbation analysis gives strongly consistent sensitivity estimates for the m g 1 queue the technique of perturbation analysis has recently been introduced as an efficient way to compute parameter sensitivities for discrete event systems thus far the statistical properties of perturbation analysis have been validated mainly through experiments this paper considers for an m g 1 queueing system the sensitivity of mean system time of a customer to a parameter of the arrival or service distribution it shows analytically that i the steady state value of the perturbation analysis estimate of this sensitivity is unbiased and ii a perturbation analysis algorithm implemented on a single sample path of the system gives asymptotically unbiased and strongly consistent estimates of this sensitivity no previous knowledge of perturbation analysis is assumed so the paper also serves to introduce this technique to the unfamiliar reader numerical extensions to gi g 1 queues and applications to optimization problems are also illustrated

technical note on estimating quantile sensitivities via infinitesimal perturbation analysis hong 2009 hong lj 2009 estimating quantile sensitivities oper res 57 1 118 130 introduced a general framework based on probability sensitivities and a conditional expectation relationship for estimating quantile sensitivities by infinitesimal perturbation analysis ipa we present an alternative more direct derivation of the ipa estimators that leads to simplified proofs for strong consistency and convergence rate of the unbatched estimator and strong consistency and a central limit theorem for the batched estimator

perturbation realization potentials and sensitivity analysis of markov processes two fundamental concepts and quantities realization factors and performance potentials are introduced for markov processes the relations among these two quantities and the group inverse of the infinitesimal generator are studied it is shown that the sensitivity of the steady state performance with respect to the change of the infinitesimal generator can be easily calculated by using either of these three quantities and that these quantities can be estimated by analyzing a single sample path of a markov process based on these results algorithms for estimating performance sensitivities on a single sample path of a markov process can be proposed the potentials in this paper are defined through realization factors and are shown to be the same as those defined by poisson equations the results provide a uniform framework of perturbation realization for infinitesimal perturbation analysis ipa and non ipa approaches to the sensitivity analysis of steady state performance they also provide a theoretical background for the pa algorithms developed in recent years

convergence of parameter sensitivity estimates in a stochastic experiment to reduce the error in estimating the gradient parameter sensitivity of an unknown function is of great importance in stochastic optimization problems three kinds of parameter sensitivity estimates using the monte carlo method are discussed in this paper the estimates depend on the number of replications n and the change in parameter delta d the convergence properties as n rightarrow infty and delta d rightarrow 0 for these estimates are obtained the result explains many theoretical and practical issues in the study of discrete event dynamic systems as well as continuous dynamic systems by the monte carlo method it is proved that an estimate based on averaging the gradients calculated along each sample path by a perturbation of the path is much better than the other estimates if the output functions are uniformly differentiable with probability one w p 1 it is also concluded that in computer simulations one should always choose the same seed for both d and d delta d in estimating the parameter sensitivity combining the results in this paper with existing stochastic approximation algorithms may yield algorithms with faster convergence

observable augmented systems for sensitivity analysis of markov and semi markov processes previous work by the authors 1989 on sensitivity analysis of discrete event systems dess where an augmented chain approach for estimating performance sensitivities from a single nominal sample path was developed is extended in contrast to existing methods this approach can be applied to cases involving discrete e g integer valued parameters such as queue capacities or routing thresholds however the basic method is limited by some rather restrictive observability conditions these conditions are relaxed through an observability transformation which generalizes the method s applicability direct extensions to more general non markovian des are also presented via a simplified generalized semi markov process reformulation of the approach experimental verification of the techniques is included

on the convergence rates of ipa and fdc derivative estimators we show that under the sufficient conditions usually given for infinitesimal perturbation analysis ipa to apply for derivative estimation a finite difference scheme with common random numbers fdc has the same order of convergence namely o n 1 2 provided that the size of the finite difference interval converges to zero fast enough this holds for both one and two sided fdc this also holds for different variants of ipa such as some versions of smoothed perturbation analysis spa which is based on conditional expectation finally this also holds for the estimation of steady state performance measures by truncated horizon estimators under some ergodicity assumptions our developments do not involve monotonicity but are based on continuity and smoothness we give examples and numerical illustrations which show that the actual difference in mean square error mse between ipa and fdc is typically negligible we also obtain the order of convergence of that difference which is faster than the c

strong points of weak convergence a study using rpa gradient estimation for automatic learning this paper analyzes the behavior of adaptive control schemes for automatic learning estimates of the sensitivities are used in a gradient based stochastic approximation procedure in order to drive the process along the steepest descent trajectory in search for the optimum the learning rates are kept constant for adaptability for such procedures convergence can be established in a weak sense a model problem of a flexible machine is presented for which the control parameter is a probability vector we propose a new sensitivity estimator generalizing the phantom rare perturbation analysis rpa estimator to multi valued decisions from the basic properties of the estimators we build several updating rules based on the weak convergence theory to ensure asymptotic optimality we illustrate the predicted theoretical behavior with computer simulations finally we present the comparison between the behavior of our proposed scheme with a regenerative one for which we can establish strong convergence our results show that weak convergence yields a dramatic improvement in the rate of convergence in addition to the capability of adaptation or tracking

acceleration of stochastic approximation by averaging a new recursive algorithm of stochastic approximation type with the averaging of trajectories is investigated convergence with probability one is proved for a variety of classical optimization and identification problems it is also demonstrated for these problems that the proposed algorithm achieves the highest possible rate of convergence

multivariate stochastic approximation using a simultaneous perturbation gradient approximation the problem of finding a root of the multivariate gradient equation that arises in function minimization is considered when only noisy measurements of the function are available a stochastic approximation sa algorithm for the general kiefer wolfowitz type is appropriate for estimating the root the paper presents an sa algorithm that is based on a simultaneous perturbation gradient approximation instead of the standard finite difference approximation of keifer wolfowitz type procedures theory and numerical experience indicate that the algorithm can be significantly more efficient than the standard algorithms in large dimensional problems

weak convergence and asymptotic properties of adaptive filters with constant gains the basic adaptive filtering algorithm x n 1 epsilon x n epsilon epsilon y n y n x n epsilon psi n is analyzed using the theory of weak convergence apart from some very special cases the analysis is hard when done for each fixed epsilon 0 but the weak convergence techniques are set up to provide much information for small epsilon the relevant facts from the theory are given define x epsilon cdot by x epsilon t x n epsilon on n epsilon n epsilon epsilon then weak distributional convergence of x epsilon cdot and of x epsilon cdot t epsilon is proved under very weak assumptions where t epsilon rightarrow infty as epsilon rightarrow 0 the normalized errors x n epsilon theta sqrt epsilon are analyzed where theta is a stable point for the mean algorithm the asymptotic properties of a projection algorithm are developed where the x n epsilon are truncated at each iteration if they fall outside of a given set

an invariant measure approach to the convergence of stochastic approximations with state dependent noise a new method is presented for quickly getting the ode ordinary differential equation associated with the asymptotic properties of the stochastic approximation x n 1 x n a n f x n zeta n or the projected algorithm for the constrained problem either a n to 0 or a n can be constant in which case the analysis is on the sequence obtained when a to 0 the method requires that x n zeta n 1 be markov with a feller transition function but little else the simplest result requires that if x n equiv x the corresponding noise process zeta n x n geqq 0 have a unique invariant measure but the nonunique case can also be treated no mixing condition is required nor the construction of averaged test functions and f cdot cdot need not be continuous a detailed analysis of the way that zeta n varies with x n is not required for the class of sequences treated the conditions seem easier to verify than for other methods there

budget dependent convergence rate of stochastic approximation convergence rate results are derived for a stochastic optimization problem where a performance measure is minimized with respect to a vector parameter t assuming that a gradient estimator is available and that both the bias and the variance of the estimator are known functions of the budget devoted to its computation the gradient estimator is employed in conjunction with a stochastic approximation sa algorithm our interest is to figure out how to allocate the total available computational budget to the successive sa iterations the effort is devoted to solving the asymptotic version of this problem by finding the convergence rate of sa toward the optimizer first as a function of the number of iterations and then as a function of the total computational effort as a result the optimal rate of increase of the computational budget per iteration can be found explicit expressions for the case where the computational budget devoted to an iteration is a polynomial in the iteration number and where the bias and variance of the gradient estimator are polynomials of the computational budget are derived applications include the optimization of steady state simulation models with likelihood ratio perturbation analysis or finite difference gradient estimators optimization of infinite horizon models with discounting optimization of functions of several expectations and so on several examples are discussed our results readily generalize to general root finding problems

stochastic optimization by simulation numerical experiments with the m m 1 queue in steady state this paper gives numerical illustrations of the behavior of stochastic approximation combined with different derivative estimation techniques to optimize a steady state system it is a companion paper to l ecuyer and glynn 1993 which gives convergence proofs for most of the variants experimented here the numerical experiments are made with a simple m m 1 queue which while simple serves to illustrate the basic convergence properties and possible pitfalls of the various techniques

characterization of the probabilistic traveling salesman problem we show that stochastic annealing can be successfully applied to gain new results on the probabilistic traveling salesman problem the probabilistic traveling salesman must decide on an a priori order in which to visit n cities randomly distributed over a unit square before learning that some cities can be omitted we find the optimized average length of the pruned tour follows e l over bar pruned rootnp 0 872 0 105p f np where p is the probability of a city needing to be visited and f np 1 as np infinity the average length of the a priori tour before omitting any cities is found to follow e l a priori rootn pbeta p where beta p 1 1 25 0 82 ln p is measured for 0 05less than or equal topless than or equal to0 6 scaling arguments and indirect measurements suggest that beta p tends towards a constant for p 0 03 our stochastic annealing algorithm is based on limited sampling of the pruned tour lengths exploiting the sampling error to provide the analog of thermal fluctuations in simulated thermal annealing the method has general application to the optimization of functions whose cost to evaluate rises with the precision required

convergences of fuzzy sets based on decomposition theory and fuzzy polynomial function abstract on r let f s be the family of all fuzzy numbers and level 1 fuzzy intervals by using the decomposition principle we can work on the problems of convergences for the fuzzy polynomial functions in f s let the fuzzy sets on r 0 be f s which is a subfamily of f s we then define fuzzy polynomial functions on f s and deal with their differentiation integration and some other properties

paper a new approach to the analysis of discrete event dynamic systems we present a new time domain approach to the study of discrete event dynamical systems deds typified by queueing networks and production systems a general state space representation is developed and perturbation analysis is carried out observation of a single sample realization of such a system can be used to predict behavior over other sample realizations when some parameter is perturbed without having to make additional observations conditions under which this is always possible are investigated and explicit results for some special cases are included

sensitivity analysis for simulations via likelihood ratios we present a simple method of estimating the sensitivity of quantities obtained from simulation with respect to a class of parameters here sensitivity means the derivative of an expectation with respect to a parameter the class of parameters includes for example poisson rates discrete probabilities and the mean and variance of a normal distribution the method is extremely well suited to regenerative simulation and can be implemented on extant simulations with little effort increase in running time or memory requirements it is based on some change of measure formulas derived from likelihood ratios in addition to the theorems that underly the technique we present some numerical examples