the special functions of fractional calculus as generalized fractional calculus operators of some basic functions abstract we propose a unified approach to the so called special functions of fractional calculus sfs of fc recently enjoying increasing interest from both theoretical mathematicians and applied scientists this is due to their role as solutions of fractional order differential and integral equations as the better mathematical models of phenomena of various physical engineering automatization chemical biological earth science economics etc nature our approach is based on the use of generalized fractional calculus gfc operators namely we show that all the wright generalized hypergeometric functions w ghf s q p z can be represented as generalized fractional integrals derivatives or differ integrals of three basic simpler functions as cos q p z exp z and 0 1 z reducible in particular to the elementary function z 1 z the beta distribution depending on whether p q p q or p q 1 and on the values of their indices and parameters in this way the sfs of fc can be separated into three classes with similar behaviour and also new integral and differential formulas can be derived useful for computational procedures

mean variance optimization of discrete time discounted markov decision processes abstract in this paper we study a mean variance optimization problem in an infinite horizon discrete time discounted markov decision process mdp the objective is to minimize the variance of system rewards with the constraint of mean performance different from most of works in the literature which require the mean performance already achieve optimum we can let the discounted performance equal any constant the difficulty of this problem is caused by the quadratic form of the variance function which makes the variance minimization problem not a standard mdp by proving the decomposable structure of the feasible policy space we transform this constrained variance minimization problem to an equivalent unconstrained mdp under a new discounted criterion and a new reward function the difference of the variances of markov chains under any two feasible policies is quantified by a difference formula based on the variance difference formula a policy iteration algorithm is developed to find the optimal policy we also prove the optimality of deterministic policy over the randomized policy generated in the mean constrained policy space numerical experiments demonstrate the effectiveness of our approach

optimization of markov decision processes under the variance criterion in this paper we study a variance minimization problem in an infinite stage discrete time markov decision process mdp regardless of the mean performance for the markov chain under the variance criterion since the value of the cost function at the current stage will be affected by future actions this problem is not a standard mdp and the traditional mdp theory is not applicable in this paper we convert the variance minimization problem into a standard mdp by introducing a concept called pseudo variance then we derive a variance difference formula that quantifies the difference of variances of markov systems under any two policies with the difference formula the correlation of the variance cost function at different stages can be decoupled through a nonnegative term a necessary condition of the optimal policy is obtained it is also proved that the optimal policy with the minimal variance can be found in the deterministic policy space furthermore we propose an efficient iterative algorithm to reduce the variance of markov systems we prove that this algorithm can converge to a local optimum finally a numerical experiment is conducted to demonstrate the efficiency of our algorithm compared with the gradient based method widely adopted in the literature

on line sensitivity analysis of markov chains discrete event systems modeled as continuous time markov processes and characterized by some integer valued parameter are considered the problem addressed is that of estimating performance sensitivities with respect to this parameter by directly observing a single sample path of the system the approach is based on transforming the nominal markov chain into a reduced augmented chain the stationary state probabilities which can be easily combined to obtain stationary state probability sensitivities with respect to the given parameter under certain conditions the reduced augmented chain state transitions are observable with respect to the state transitions of the system itself and no knowledge of the nominal markov chain state of the transition rates is required applications for some queueing systems are included the approach incorporates estimation of unknown transition rates when needed and is extended to real valued parameters

limiting average criteria for nonstationary markov decision processes this paper deals with the so called limiting average criteria for nonstationary markov decision processes with possibly unbounded rewards and borel state space a new set of conditions is provided under which the existence of both a solution to the optimality equations and the limiting average epsilon geq 0 optimal markov policies is derived also a rolling horizon algorithm for computing limiting average epsilon 0 optimal markov policies is developed furthermore the results in this paper are illustrated by several examples such as the water regulation problem

technical communique a unified approach to markov decision problems and performance sensitivity analysis we propose a simple approach that provides a unified formulation for the performance sensitivity analysis of markov chains and markov decision problems with both infinite horizon average cost and discounted performance criteria

a single sample path based performance sensitivity formula for markov chains using a sample path approach we derive a new formula for performance sensitivities of discrete time markov chains a distinguished feature of this formula is that the quantities involved can be estimated by analyzing a single sample path of a markov chain thus the formula provides a new direction for sensitivity analysis and can be viewed as an extension of the perturbation realization theory to problems where infinitesimal perturbation analysis does not work well

certain image formulas of generalized hypergeometric functions we aim at establishing certain new image formulas of generalized hypergeometric functions by applying the operators of fractional derivative involving appell s function f3 due to saigo maeda furthermore by employing some integral transforms on the resulting formulas we presented some more image formulas all the results derived here are of general character and can yield a number of known and new results in the theory of special functions

the control of a two level markov decision process by time aggregation the solution of markov decision processes mdps often relies on special properties of the processes for two level mdps the difference in the rates of state changes of the upper and lower levels has led to limiting or approximate solutions of such problems in this paper we solve a two level mdp without making any assumption on the rates of state changes of the two levels we first show that such a two level mdp is a non standard one where the optimal actions of different states can be related to each other then we give assumptions conditions under which such a specially constrained mdp can be solved by policy iteration we further show that the computational effort can be reduced by decomposing the mdp a two level mdp with m upper level states can be decomposed into one mdp for the upper level and m to m m 1 mdps for the lower level depending on the structure of the two level mdp the upper level mdp is solved by time aggregation a technique introduced in a recent paper cao x r ren z y bhatnagar s fu m marcus s 2002 a time aggregation approach to markov decision processes automatica 38 6 929 943 and the lower level mdps are solved by embedded markov chains

optimization of average rewards of time nonhomogeneous markov chains we study the optimization of average rewards of discrete time nonhomogeneous markov chains in which the state spaces transition probabilities and reward functions depend on time the analysis encounters a few major difficulties 1 notions crucial to homogeneous markov chains such as ergodicity stationarity periodicity and connectivity no longer apply 2 the average reward criterion is under selective i e it does not depend on the decisions in any finite period and thus dynamic programming is not amenable and 3 because of the under selectivity an optimal average reward policy may not be the best in any finite period

markov decision processes with a minimum variance criterion abstract we consider the optimization of the variance of the sum of costs as well as that of an average expected cost in markov decision processes with unbounded cost in case of general state and action space we find the stationary policy which makes the average variance as small as possible in the class of policies which are e optimal in an average expected cost

on the period function of lienard systems abstract we study the period functiontof a centeroof a lienard system a sufficient condition for the monotonicity oft or for the isochronicity ofo is given such a condition is also necessary whenfandgare analytic andgis odd in this case a characterization of isochronous centers of lienard systems is given strict monotonicity and global monotonicity oftare also investigated

discrete time controlled markov processes with average cost criterion a survey this work is a survey of the average cost control problem for discrete time markov processes the authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades the exposition ranges from finite to borel state and action spaces and includes a variety of methodologies to find and characterize optimal policies the authors have included a brief historical perspective of the research efforts in this area and have compiled a substantial yet not exhaustive bibliography the authors have also identified several important questions that are still open to investigation

linearizability of three families of cubic systems we obtain the necessary and sufficient conditions for linearizability isochronicity of three families of eight parametric cubic systems this completes the classification of the linearizable systems studied by romanovski and robnik 2001 j phys a math gen 34 10267 92

nonhomogeneous markov decision processes with borel state space the average criterion with nonuniformly bounded rewards this paper deals with nonhomogeneous markov decision processes with borel state space and nonuniformly bounded rewards under the average criterion first under the minorant type assumption we prove the existence of an appropriate solution to the optimality equations second from the optimality equations we also establish the existence of e 0 optimal markov policies under the additional conditions third some sufficient conditions for the validity of the assumptions in this paper and several examples such as inventory production systems are provided finally as an application of the optimality equations a rolling horizon algorithm is given

fractional calculus with an integral operator containing a generalized mittag leffler function in the kernel in this paper we introduce and investigate a fractional calculus with an integral operator which contains the following family of generalized mittag leffler functions e a b c k z email protected n 0 c k email protected email protected z nn z b email protected c r a max 0 r k 1 r k 0 in its kernel l n being the familiar pochhammer symbol a number of corollaries and consequences of the main results presented here are also considered

optimal control of ergodic continuous time markov chains with average sample path rewards in this paper we study continuous time markov decision processes with the average sample path reward aspr criterion and possibly unbounded transition and reward rates we propose conditions on the system s primitive data for the existence of epsilon aspr optimal deterministic stationary policies in a class of randomized markov policies satisfying some additional continuity assumptions the proof of this fact is based on the time discretization technique the martingale stability theory and the concept of potential we also provide both policy and value iteration algorithms for computing or at least approximating the epsilon aspr optimal stationary policies we illustrate with examples our main results as well as the difference between the aspr and the average expected reward criteria

isochronous centers of a linear center perturbed by fifth degree homogeneous polynomials in this work we study isochronous centers of two dimensional autonomous system in the plane with linear part of center type and nonlinear part given by homogeneous polynomials of fifth degree a complete classification of the necessary conditions for the time reversible systems of this class is given in order to have an isochronous center at the origin an open problem is stated for the sufficient conditions moreover we find two nonreversible isochronous families from the center cases known all the computations in order to obtain necessary conditions for such isochronous centers are given in polar coordinates and we give a proof of the isochronicity of these systems by using different methods

algorithmic derivation of centre conditions the conditions for a critical point of a polynomial differential system to be a centre are of particular significance because of the frequency with which they are required in applications we demonstrate how computer algebra can be effectively employed in the search for necessary and sufficient conditions for critical points of such systems to be centres we survey recent developments and illustrate our approach by means of examples

a note on the convergence rate of the value iteration scheme in controlled markov chains this work is concerned with controlled markov chains with bounded costs assuming that the transition probabilities satisfy a simultaneous doeblin condition it is shown that schweitzer s transformation on the transition law yields a strong ergodicity condition that implies that the solution to the average cost optimality equation can be approximated at a geometric rate via the value iteration scheme

isochronicity into a family of time reversible cubic vector fields in this work we study necessary and sufficient conditions for the existence of isochronous centers into a family of cubic time reversible systems this class of reversible systems is characterized by the existence of an inverse integrating factor which is a certain power of an invariant straight line

are limits of discounted optimal policies blackwell optimal a counter example abstract in markov decision chains mdc s with a finite state space and finite action sets it is a well known result that the limits of discounted optimal policies for tending to 1 are blackwell optimal it was conjectured in a recent paper by cavazos cadena and lasserre that this property of limiting policies also holds for unichain mdc s we disprove this conjecture by constructing a non blackwell limiting policy in a unichain mdc with finitely many states and compact action sets

finding optimal observation based policies for constrained pomdps under the expected average reward criterion in this technical note constrained partially observable markov decision processes with discrete state and action spaces under the average reward criterion are studied from a sensitivity point of view by analyzing the derivatives of performance criteria we develop a simulation based optimization algorithm to find the optimal observation based policy on the basis of a single sample path this algorithm does not need any overly strict assumption and can be applied to the general ergodic markov systems with the imperfect state information the performance is proved to converge to the optimum with probability 1 one numerical example is provided to illustrate the applicability of the algorithm

zero sum stochastic games in borel spaces average payoff criteria this paper is concerned with two person zero sum dynamic stochastic games in borel spaces with possibly unbounded payoff function and several average or ergodic payoff criteria we give conditions under which the long run expected average payoff criterion the sample path average criterion the existence of solutions to the average payoff shapley equations and a certain martingale condition are all equivalent

another set of conditions for markov decision processes with average sample path costs abstract this paper deals with discrete time markov decision processes with average sample path costs aspc in borel spaces the costs may have neither upper nor lower bounds we propose new conditions for the existence of e aspc optimal deterministic stationary policies in the class of all randomized history dependent policies our conditions are weaker than those in the previous literature moreover some sufficient conditions for the existence of aspc optimal stationary policies are imposed on the primitive data of the model in particular the stochastic monotonicity condition in this paper has first been used to study the aspc criterion also the approach provided here is slightly different from the optimality equation approach widely used in the previous literature on the other hand under mild assumptions we show that average expected cost optimality and aspc optimality are equivalent finally we use a controlled queueing system to illustrate our results

optimization of queues using an infinitesimal perturbation analysis based stochastic algorithm with general update times convergence with probability one of a stochastic optimization algorithm for a single server queue is proved the parameter to be optimized is updated using an infinitesimal perturbation analysis estimate of the gradient of the performance measure and the updates are performed at general times first an algorithm in which the parameter is updated before each customer begins service is considered then it is shown how this analysis can be extended to an algorithm that updates at certain stopping times the analysis suggests that the sample path behavior of the algorithm is similar to that of an algorithm that updates at the start of every busy period

on isochronous cases of the cherkas system and jacobi s last multiplier we consider a large class of polynomial planar differential equations proposed by cherkas 1976 differensial nye uravneniya 12 201 6 and show that these systems admit a lagrangian description via the jacobi last multiplier jlm it is shown how the potential term can be mapped either to a linear harmonic oscillator potential or into an isotonic potential for specific values of the coefficients of the polynomials this enables the identification of the specific cases of isochronous motion without making use of the computational procedure suggested by hill et al 2007 nonlinear anal theor methods appl 67 52 69 based on the pleshkan algorithm finally we obtain a lagrangian description and perform a similar analysis for a cubic system to illustrate the applicability of this procedure based on jacobi s last multiplier

some new results on sample path optimality in ergodic control of diffusions we present some new results on sample path optimality for the ergodic control problem of a class of nondegenerate diffusions controlled through the drift the hypothesis most often used in the literature to ensure the existence of an almost sure sample path optimal stationary markov control requires finite second moments of the first hitting times tau of bounded domains over all admissible controls we show that this can be considerably weakened mathbb e tau 2 may be replaced with mathbb e tau ln tau thus reducing the required rate of convergence of averages from polynomial to logarithmic a foster lyapunov condition that guarantees this is also exhibited moreover we study a large class of models that are neither uniformly stable nor have a near monotone running cost and we exhibit sufficient conditions for the existence of a sample path optimal stationary markov control

necessary and sufficient conditions for a bounded solution to the optimality equation in average reward markov decision chains abstract we consider average reward markov decision processes with discrete time parameter and denumerable state space we are concerned with the following problem find necessary and sufficient conditions so that for arbitrary bounded reward function the corresponding average reward optimality equation has a bounded solution this problem is solved for a class of systems including the case in which under the action of any stationary policy the state space is an irreducible positive recurrent class

smoothed conditional perturbation analysis of discrete event dynamical systems a new sample path analysis approach based on the smoothing property of conditional expectation for estimating the performance sensitivity of discrete event dynamical systems is proposed several examples are presented to show how this approach overcomes a difficulty of the ordinary infinitesimal perturbation analysis the basic message is that one can get more knowledge about the system performance by observing and analyzing the sample path than by using the conventional simulation approach it is also pointed out that the classical queueing theory approach for getting the performance sensitivity and the sample path based infinitesimal perturbation analysis approach can be unified in the framework of the new approach the smoothed conditional perturbation analysis

bias optimality and strong n n 1 0 discount optimality for markov decision processes abstract in this paper we study both bias optimality and strong n n 1 0 discount optimality for denumerable discrete time markov decision processes the rewards may have neither upper nor lower bounds we give sufficient conditions on the system s primitive data and under which we prove 1 the existence of the bias optimality equation and bias optimal policies 2 a condition equivalent to bias optimal policies 3 average expected reward optimality and strong 1 discount optimality are equivalent 4 bias optimality and strong 0 discount optimality are equivalent 5 the existence of strong n n 1 0 discount optimal stationary policies our conditions are weaker than those in the previous literature moreover our results are illustrated by a controlled random walk

recurrence conditions for average and blackwell optimality in denumerable state markov decision chains in a previous paper dekker and hordijk 1988 presented an operator theoretical approach for multichain markov decision processes with a countable state space compact action sets and unbounded rewards conditions were presented guaranteeing the existence of a laurent series expansion for the discounted rewards the existence of average and blackwell optimal policies and the existence of solutions for the average and blackwell optimality equations while these assumptions were operator oriented and formulated as conditions for the deviation matrix we will show in this paper that the same approach can also be carried out under recurrence conditions these new conditions seem easier to check in general and are especially suited for applications in queueing models

markov decision processes with a borel measurable cost function the average case we consider a markov decision process with a borel measurable cost function we introduce a p step contraction property for the average cost case by use of this method the validity of the optimality equation and the existence of e optimal stationary policies are proved as some applications the sequential replacement model and the inventory model are considered

the potential structure of sample paths and performance sensitivities of markov systems we study the structure of sample paths of markov systems by using performance potentials as the fundamental units with a sample path based approach we show that performance sensitivity formulas performance gradients and performance differences of markov systems can be constructed intuitively by first principles with performance potentials or equivalently perturbation realization factors as building blocks in particular we derive sensitivity formulas for two markov chains with possibly different state spaces the proposed approach can be used to obtain flexibly the sensitivity formulas for a wide range of problems including those with partial information these formulas are the basis for performance optimization of discrete event dynamic systems including perturbation analysis markov decision processes and reinforcement learning the approach thus provides insight on on line learning and performance optimization and opens up new research directions sample path based algorithms can be developed

zero sum semi markov games this paper deals with borel state and action spaces zero sum semi markov games under the expected long run average payoff criterion the transition probabilities are assumed to satisfy some generalized geometric ergodicity conditions the main result states that the optimality equation has a solution which is approximated by the solutions of some varepsilon perturbed semi markov games as a corollary the existence of value and average optimal strategies for the players is established

convergence properties of infinitesimal perturbation analysis infinitesimal perturbation analysis ipa is a method for computing a sample path derivative with respect to an input parameter in a discrete event simulation the ipa algorithm is based on the fact that for certain parameters and any realization of a simulation the change in parameter can be made small enough so that only the times of events get shifted but their order does not change this paper considers the convergence properties of the ipa sample path derivatives in particular the question of when an ipa estimate converges to the derivative of a steady state performance measure is studied necessary and sufficient conditions for this convergence are derived for a class of regenerative processes although these conditions are not guaranteed to be satisfied in general they are satisfied for the mean stationary response time in the m g 1 queue a necessary condition for multiple ipa estimates to simultaneously converge to the derivatives of steady state throughputs in a queueing network is determined the implications of this necessary condition are that except in special cases the original ipa algorithm cannot be used to consistently estimate steady state throughput derivatives in queueing networks with multiple types of customers state dependent routing or blocking numerical studies on ipa convergence properties are also presented

sample path optimality and variance minimization of average cost markov control processes this paper studies several average cost criteria for markov control processes on borel spaces with possibly unbounded costs under suitable hypotheses we show i the existence of a sample path average cost spac optimal stationary policy ii a stationary policy is spac optimal if and only if it is expected average cost eac optimal and iii within the class of stationary spac optimal equivalently eac optimal policies there exists one with a minimal limiting average variance

a mean variance optimization problem for discounted markov decision processes in this paper we consider a mean variance optimization problem for markov decision processes mdps over the set of deterministic stationary policies different from the usual formulation in mdps we aim to obtain the mean variance optimal policy that minimizes the variance over a set of all policies with a given expected reward for continuous time mdps with the discounted criterion and finite state and action spaces we prove that the mean variance optimization problem can be transformed to an equivalent discounted optimization problem using the conditional expectation and markov properties then we show that a mean variance optimal policy and the efficient frontier can be obtained by policy iteration methods with a finite number of iterations we also address related issues such as a mutual fund theorem and illustrate our results with an example

average sensitive and blackwell optimal policies in denumerable markov decisionchains with unbounded rewards in this paper we consider a discrete time markov decision chain with a denumerable state space and compact action sets and we assume that for all states the rewards and transition probabilities depend continuously on the actions the first objective of this paper is to develop an analysis for average optimality without assuming a special markov chain structure in doing so we present a set of conditions guaranteeing average optimality which are automatically fulfilled in the finite state and action model the second objective is to study simultaneously average and discount optimality as veinott veinott a f jr 1969 on discrete dynamic programming with sensitive discount optimality criteria ann math statist 40 1635 1660 did for the finite state and action model we investigate the concepts of n discount and blackwell optimality in the denumerable state space using a laurent series expansion for the discounted rewards under the same condition as for average optimality we establish solutions to the n discount optimality equations for every n

extension of gamma beta and hypergeometric functions the main object of this paper is to present generalizations of gamma beta and hypergeometric functions some recurrence relations transformation formulas operation formulas and integral representations are obtained for these new generalizations

markov decision processes with sample path constraints the communicating case we consider time average markov decision processes mdps which accumulate a reward and cost at each decision epoch a policy meets the sample path constraint if the time average cost is below a specified value with probability one the optimization problem is to maximize the expected average reward over all policies that meet the sample path constraint the sample path constraint is compared with the more commonly studied constraint of requiring the average expected cost to be less than a specified value although the two criteria are equivalent for certain classes of mdps their feasible and optimal policies differ for many nontrivial problems in general there does not exist optimal or nearly optimal stationary policies when the expected average cost constraint is employed assuming that a policy exists that meets the sample path constraint we establish that there exist nearly optimal stationary policies for communicating mdps a parametric linear programming algorithm is given to construct nearly optimal stationary policies the discussion relies on well known results from the theory of stochastic processes and linear programming the techniques lead to simple proofs of the existence of optimal and nearly optimal stationary policies for unichain and deterministic mdps respectively

perturbation analysis for the design of flexible manufacturing system flow controllers dynamic allocation of stochastic capacity among competing activities in a just in time manufacturing environment is addressed by optimal flow control optimal policies are characterized by generally intractable bellman equations a near optimal controller design technique is proposed it provides an approximate numerical solution to the bellman equation a tight lower bound for the optimality gap of tractable near optimal controller designs and a building block for improved near optimal controller designs that rely on the decomposition of a multiple part type problem to smaller two or three part type problems computational experience is reported for two and three part type problems

on urabe s criteria of isochronicity we give a short proof of urabe s criteria for the isochronicity of periodical solutions of the equation we show that apart from the harmonic oscillator there exists a large family of isochronous potentials which must all be non polynomial and not symmetric an even function of the coordinate x

a liapounov bound for solutions of the poisson equation in this paper we consider psi irreducible markov processes evolving in discrete or continuous time on a general state space we develop a liapounov function criterion that permits one to obtain explicit bounds on the solution to the poisson equation and in particular obtain conditions under which the solution is square integrable these results are applied to obtain sufficient conditions that guarantee the validity of a functional central limit theorem for the markov process as a second consequence of the bounds obtained a perturbation theory for markov processes is developed which gives conditions under which both the solution to the poisson equation and the invariant probability for the process are continuous functions of its transition kernel the techniques are illustrated with applications to queueing theory and autoregressive processes

average optimality for continuous time markov decision processes with a policy iteration approach this paper deals with the average expected reward criterion for continuous time markov decision processes in general state and action spaces the transition rates of underlying continuous time jump markov processes are allowed to be unbounded a nd the reward rates may have neither upper nor lower bounds we give conditions on the system s primitive data and under which we prove the existence of the average reward optimality equation and an average optimal stationary policy also under our conditions we ensure the existence of average optimal stationary policies moreover we study some properties of average optimal stationary policies we not only establish another average optimality equation on an average optimal stationary policy but also present an interesting martingale characterization of such a policy the approach provided in this paper is based on the policy iteration algorithm it should be noted that our way is rather different from both the usually vanishing discounting factor approach and the optimality inequality approach widely used in the previous literature

markov decision problems and state action frequencies consider a controlled markov chain with countable state and action spaces basic quantities that determine the values of average cost functionals are identified under some regularity conditions these turn out to be a collection of numbers one for each state action pair describing for each state the relative number of uses of each action these conditional frequencies which are defined pathwise are shown to determine the state action frequencies that in the finite case are known to determine the costs this is extended to the countable case allowing for unbounded costs the space of frequencies is shown to be compact and convex and the extreme points are identified with stationary deterministic policies conditions under which the search for optimality in several optimization problems may be restricted to stationary policies are given these problems include the standard markov decision process as well as constrained optimization both in terms of average cost functionals and variability sensitive optimization an application to a queueing problem is given where these results imply the existence and explicit computation of optimal policies in constrained optimization problems the pathwise definition of the conditional frequencies implies that their values can be controlled directly moreover they depend only on the limiting behavior of the control this has immediate application to adaptive control of markov chains including adaptive control under constraints

limit cycles of a predator prey model with intratrophic predation abstract we present some properties of a differential system that can be used to model intratrophic predation in simple predator prey models in particular for the model we determine the maximum number of limit cycles that can exist around the only fine focus in the first quadrant and show that this critical point cannot be a centre

solutions of the average cost optimality equation for markov decision processes with weakly continuous kernel the fixed point approach revisited abstract this paper shows the existence of lower semicontinuous solutions of the average cost optimality equation for markov decision processes with borel spaces possible unbounded cost function and weakly continuous transition kernel this is done imposing a growth condition on the cost function a lyapunov stability condition on the transition kernel and a set of standard compactness continuity conditions the solution of the average cost optimality equation is obtained by means of the banach fixed point theorem

addendum to extensions and generalizations of smoothed perturbation analysis in a generalized semi markov process framework under a very general framework both in terms of finite time performance measures and system structure the authors derive smoothed perturbation analysis spa estimators and prove their unbiasedness the commuting condition which has been key in previous work is not required a priori and thus the framework includes such systems as the gi g 1 k queue and multiclass queueing networks which do not satisfy the commuting condition the generality achieved is traded off against the fact that the estimator is not always easily implementable on a single sample path the use of the commuting condition in a local sense is proposed to help simplify the estimators derived queueing and multiclass queueing networks are used as illustrative examples for a simple multiclass closed queueing network some simulation results are provided when the commuting condition is satisfied globally the framework allows the recovery of previous results on ipa and spa estimators as corollaries of the main theorems

perturbation analysis gives strongly consistent sensitivity estimates for the m g 1 queue the technique of perturbation analysis has recently been introduced as an efficient way to compute parameter sensitivities for discrete event systems thus far the statistical properties of perturbation analysis have been validated mainly through experiments this paper considers for an m g 1 queueing system the sensitivity of mean system time of a customer to a parameter of the arrival or service distribution it shows analytically that i the steady state value of the perturbation analysis estimate of this sensitivity is unbiased and ii a perturbation analysis algorithm implemented on a single sample path of the system gives asymptotically unbiased and strongly consistent estimates of this sensitivity no previous knowledge of perturbation analysis is assumed so the paper also serves to introduce this technique to the unfamiliar reader numerical extensions to gi g 1 queues and applications to optimization problems are also illustrated

structural infinitesimal perturbation analysis sipa for derivative estimation of discrete event dynamic systems in this paper we present an algorithm for derivative estimation of discrete event dynamic systems ded s this algorithm is applicable to derivative estimation with respect to most structural parameters in ded s it extends the applicability of existing perturbation analysis techniques including infinitesimal perturbation analysis on structural parameters furthermore it offers an alternative approach to derivative estimation in ded s which sheds additional light on discrete event simulation finally unifying relationships with existing techniques are discussed

linearization of isochronous centers abstract in this paper we study isochronous centers of polynomial systems we first discuss isochronous centers of quadratic systems cubic symmetric systems and reduced kukles system all these systems have rational first integrals we give a unified proof of the isochronicity of these systems by constructing algebraic linearizing changes of coordinates we then study two other classes of systems with isochronous centers namely the class of complex systems z ip z and the class of cubic systems symmetric with respect to a line and satisfying 1 both classes consist of darboux integrable systems we discuss their geometric properties and construct the linearizing changes of coordinates we show that the class of polynomial isochronous systems carries a very rich geometry finally we discuss the geometry of the linearizing changes of coordinates in the complex plane

operators of fractional integration and their applications the main purpose of this paper is to present a systematic and historical account of the investigations carried out by various authors in the field of fractional calculus and its applications several interesting results relevant to the present investigation are also considered

the exact solution of certain differential equations of fractional order by using operational calculus abstract in the present paper the authors first develop an operational calculus for the familiar riemann liouville fractional differential operator this operational calculus is then used here to solve a cauchy boundary value problem for a certain linear equation involving the riemann liouville fractional derivatives relevant connections are also indicated with the special cases of the equation which were solved earlier by using other methods

extensions of infinitesimal perturbation analysis an extension of the concept of perturbation analysis pa is introduced that can be applied to cases where sample path discontinuity with respect to system parameters has prevented the application of simple pa rules both markov and generalized semi markov processes are considered the robustness of the extended pa is examined and it is found to be sufficiently robust

convergence of parameter sensitivity estimates in a stochastic experiment to reduce the error in estimating the gradient parameter sensitivity of an unknown function is of great importance in stochastic optimization problems three kinds of parameter sensitivity estimates using the monte carlo method are discussed in this paper the estimates depend on the number of replications n and the change in parameter delta d the convergence properties as n rightarrow infty and delta d rightarrow 0 for these estimates are obtained the result explains many theoretical and practical issues in the study of discrete event dynamic systems as well as continuous dynamic systems by the monte carlo method it is proved that an estimate based on averaging the gradients calculated along each sample path by a perturbation of the path is much better than the other estimates if the output functions are uniformly differentiable with probability one w p 1 it is also concluded that in computer simulations one should always choose the same seed for both d and d delta d in estimating the parameter sensitivity combining the results in this paper with existing stochastic approximation algorithms may yield algorithms with faster convergence

perturbation realization potentials and sensitivity analysis of markov processes two fundamental concepts and quantities realization factors and performance potentials are introduced for markov processes the relations among these two quantities and the group inverse of the infinitesimal generator are studied it is shown that the sensitivity of the steady state performance with respect to the change of the infinitesimal generator can be easily calculated by using either of these three quantities and that these quantities can be estimated by analyzing a single sample path of a markov process based on these results algorithms for estimating performance sensitivities on a single sample path of a markov process can be proposed the potentials in this paper are defined through realization factors and are shown to be the same as those defined by poisson equations the results provide a uniform framework of perturbation realization for infinitesimal perturbation analysis ipa and non ipa approaches to the sensitivity analysis of steady state performance they also provide a theoretical background for the pa algorithms developed in recent years

zero sum average semi markov games fixed point solutions of the shapley equation this paper deals with zero sum average semi markov games with borel state and action spaces unbounded payoffs and mean holding times a solution to the shapley equation is obtained via the banach fixed point theorem assuming that the model satisfies a lyapunov like condition a growth hypothesis on the payoff function and the mean holding times besides standard continuity and compactness requirements

kukles revisited advances in computing techniques we revisit the kukles system to show how advances in computer hardware and software have improved symbolic calculations we confirm directly that there are no non persistent centres for the kukles system we also prove an earlier conjecture regarding the isochronous centres for the extended kukles system we introduce a technique whereby modular resultants can be used to investigate the properties of resultants that cannot be readily calculated using the existing software

algorithmic derivation of isochronicity conditions abstract this paper is concerned with obtaining the conditions under which the origin is an isochronous centre for certain planar polynomial systems the necessity of these conditions is proved using a computer implementation of an algorithm first proposed by i i pleshkan their sufficiency is then proved using various methods

on the classification of lienard systems with amplitude independent periods abstract we consider conditions under which the second order differential equation x f x x g x 0 has a family of periodic orbits with constant period this condition is equivalent to seeking conditions under which the two dimensional autonomous system x y y g x f x y has a center with constant period i e an isochronous center in turn this is equivalent to the latter system being locally linearizable we give a simple necessary and sufficient condition for this when f and g are analytic functions in the case where when f and g are polynomials we show that this reduces to a finitely determinable system of equations a complete classification is given of all such systems of degree 34 or less

a centre theorem for two dimensional complex holomorphic systems and its generalization we consider a two dimensional complex holomorphic system in particular we use the centre manifold theory together with the singular point theory of c h briot j c bouquet j ec imp polyt 21 133 1856 to establish a centre theorem concerning the behaviour of the phase paths of the system in the neighbourhood of an equilibrium point having a single purely imaginary eigenvalue an extended centre theorem is established for the corresponding n dimensional complex holomorphic system n 3

strong 1 optimal stationary policies in denumerable markov decision processes abstract usual conditions for existence of stationary average optimal policies in denumerable mdps with general bounded rewards are shown to be also sufficient for strong 1 optimality moreover we prove that all limit points of discounted optimal stationary policies when the discount factor goes to 1 are strong 1 optimal

an explicit expression of the first liapunov and period constants with applications in this paper we study systems in the plane having a critical point with pure imaginary eigenvalues and we search for effective conditions to discern whether this critical point is a focus or a center in the case of it being a center we look for additional conditions in order to be isochronous we stress that the essential differences between the techniques used in this work and the more usual ones are basically two the elimination of the integration constants when we consider primitives of functions see also remark 3 2 and the fact that we maintain the complex notation in the whole study thanks to these aspects we reach with relative ease an expression of the first three liapunov constants v3 v5 andv7 and of the first two period ones p2andp4 for a general system as far as we know this is the first time that a general and compact expression ofv7has been given moreover the use of a computer algebra system is only needed in the computation ofv7andp4 these results are applied to give a classification of centers and isochronous centers for certain families of differential equations

sample path average optimality for markov control processes the authors consider a markov control process with borel state and actions spaces unbounded costs and under the long run sample path average cost criterion they prove that under very weak assumptions on the transition law and a moment assumption for the one step cost there exists a stationary policy with invariant probability distribution v that is sample path average cost optimal for v almost all initial states in addition every expected average cost optimal stationary policy is in fact liminf sample path average cost optimal and strongly expected average cost optimal

semi markov decision problems and performance sensitivity analysis recent research indicates that markov decision processes mdps can be viewed from a sensitivity point of view and the perturbation analysis pa mdps and reinforcement learning rl are three closely related areas in optimization of discrete event dynamic systems that can be modeled as markov processes the goal of this paper is two fold first we develop the pa theory for semi markov processes smps and then we extend the aforementioned results about the relation among pa mdp and rl to smps in particular we show that performance sensitivity formulas and policy iteration algorithms of semi markov decision processes can be derived based on the performance potential and realization matrix both the long run average and discounted cost problems are considered this approach provides a unified framework for both problems and the long run average problem corresponds to the discounted factor being zero the results indicate that performance sensitivities and optimization depend only on first order statistics single sample path based implementations are discussed

brief paper infinitesimal and finite perturbation analysis for queueing networks the sample path perturbation analysis technique is extended to include finite and possibly large perturbations typically introduced by changes in queue sizes or other parameters it is shown that there is a natural hierarchy of perturbation analysis which takes care of increasingly large perturbations experiments with zero infinitesimal and first order finite perturbation analysis show that significant accuracy improvement can be obtained with small increase in computational effort