introduction to matrix analytic methods in stochastic modeling preface part i quasi birth and death processes 1 examples part ii the method of phases 2 ph distributions 3 markovian point processes part iii the matrix geometric distribution 4 birth and death processes 5 processes under a taboo 6 homogeneous qbds 7 stability condition part iv algorithms 8 algorithms for the rate matrix 9 spectral analysis 10 finite qbds 11 first passage times part v beyond simple qbds 12 nonhomogeneous qbds 13 processes skip free in one direction 14 tree processes 15 product form networks 16 nondenumerable states bibliography index

on the time to extinction from quasi stationarity a unified approach this note provides a unified approach to the distribution of the time to extinction from quasi stationarity for general markov chains evolving both in discrete and in continuous time our results generalize a number of similar derivations which were established ad hoc for a variety of stochastic epidemic models on the other hand the obtained results unify the infinite irreducible case and the finite reducible or irreducible case which are typically presented under separate formulations in the literature for markov chains

on a discrete risk model with two sided jumps in this paper we consider a discrete renewal risk model with phase type interarrival times and two sided jumps in this model downward jumps represent claim loss while upward jumps are also allowed to represent random gains assume that the downward jumps have an arbitrary probability function and the upward jumps have a rational probability generating function we study the gerber shiu discounted penalty function the generating function the recursive formula as well as an explicit expression for the discounted penalty function are obtained

nonlinear matrix equations and structured linear algebra different kinds of nonlinear matrix equations are analyzed and their interplay with certain structured matrices is pointed out by using the point of view and the tools of structured linear algebra both theoretical results and numerical methods are derived for solving different classes of nonlinear matrix equations

asymptotic properties of powers of nonnegative matrices with applications abstract let a be a finite square reducible nonnegative matrix the main theorem of the paper gives first order approximations of f n a for certain sequences f n of analytic functions in particular the theorem holds when f n a a n the theorem is applied to study the local behavior of large powers of a nonnegative matrix to study the limiting output vectors of a nonnegative multiplicative process and to characterize the nonnegative eigenvectors of a nonnegative matrix finally an application to absorbing markov chains is given

recursive markov chains stochastic grammars and monotone systems of nonlinear equations we define recursive markov chains rmcs a class of finitely presented denumerable markov chains and we study algorithms for their analysis informally an rmc consists of a collection of finite state markov chains with the ability to invoke each other in a potentially recursive manner rmcs offer a natural abstract model for probabilistic programs with procedures they generalize in a precise sense a number of well studied stochastic models including stochastic context free grammars scfg and multi type branching processes mt bp we focus on algorithms for reachability and termination analysis for rmcs what is the probability that an rmc started from a given state reaches another target state or that it terminatesq these probabilities are in general irrational and they arise as least fixed point solutions to certain monotone systems of nonlinear equations associated with rmcs we address both the qualitative problem of determining whether the probabilities are 0 1 or in between and the quantitative problems of comparing the probabilities with a given bound or approximating them to desired precision we show that all these problems can be solved in pspace using a decision procedure for the existential theory of reals we provide a more practical algorithm based on a decomposed version of multi variate newton s method and prove that it always converges monotonically to the desired probabilities we show this method applies more generally to any monotone polynomial system we obtain polynomial time algorithms for various special subclasses of rmcs among these for scfgs and mt bps equivalently for 1 exit rmcs the qualitative problem can be solved in p time for linearly recursive rmcs the probabilities are rational and can be computed exactly in p time we show that our pspace upper bounds cannot be substantially improved without a breakthrough on long standing open problems the square root sum problem and an arithmetic circuit decision problem that captures p time on the unit cost rational arithmetic ram model we show that these problems reduce to the qualitative problem and to the approximation problem to within any nontrivial error for termination probabilities of general rmcs and to the quantitative decision problem for termination extinction of scfgs mt bps

a discrete time inventory system with postponed demands in this article we consider a discrete time inventory model in which demands arrive according to a discrete markovian arrival process the inventory is replenished according to an s s policy and the lead time is assumed to follow a discrete phase type distribution the demands that occur during stock out periods either enter a pool which has a finite capacity n or leave the system with a predefined probability any demand that arrives when the pool is full and the inventory level is zero is assumed to be lost the demands in the pool are selected one by one if the on hand inventory level is above s 1 and the interval time between any two successive selections is assumed to have discrete phase type distribution the joint probability distribution of the number of customers in the pool and the inventory level is obtained in the steady state case the measures of system performance in the steady state are derived and the total expected cost rate is also calculated the results are illustrated numerically

a ph ph n c c state dependent queuing model for metro station corridor width design abstract metro station corridor width design considering demand fluctuation as well as the randomness and state dependence of service time is an urgent concern and a complicated random planning issue this paper confirms the accuracy of phase type distribution ph fitting for passenger arrival intervals and service times with randomness and state dependence in metro station corridors a ph ph n c c state dependent queuing model is thus established by a finite level dependent quasi birth death qbd process the existing m g n c c m g 1 c and d d 1 c models are proved to be special cases of the ph ph n c c model through theoretical derivation and the precision of the proposed model is analyzed through simulation tests the quantitative relationship between the level of service los and the corridor width is established based on the proposed model a total of 81 experiments are designed to compare the calculations between the proposed model and the m g n c c m g 1 c and d d 1 c models comparison results demonstrate that 1 the value of effective width of the ph ph n c c queuing model is higher than those of the m g n c c m g 1 c and d d 1 c models 2 the real area occupied per person in the corridor of the ph ph n c c queuing model is mostly proximate to the designed los whereas those of the m g n c c m g 1 c and d d 1 c models fail to meet the designed los and 3 the performance measures of the ph ph n c c queuing model enjoy high performance width elasticity and are significantly improved compared with those of the m g n c c m g 1 c and d d 1 c models

a fluid limit for an overloaded x model via a stochastic averaging principle we prove a many server heavy traffic fluid limit for an overloaded markovian queueing system having two customer classes and two service pools known in the call center literature as the x model the system uses the fixed queue ratio with thresholds fqr t control which we proposed in a recent paper as a way for one service system to help another in face of an unexpected overload under fqr t customers are served by their own service pool until a threshold is exceeded then one way sharing is activated with customers from one class allowed to be served in both pools after the control is activated it aims to keep the two queues at a prespecified fixed ratio for large systems that fixed ratio is achieved approximately for the fluid limit or fwlln functional weak law of large numbers we consider a sequence of properly scaled x models in overload operating under fqr t our proof of the fwlln follows the compactness approach i e we show that the sequence of scaled processes is tight and then show that all converging subsequences have the specified limit the characterization step is complicated because the queue difference processes which determine the customer server assignments need to be considered without spatial scaling asymptotically these queue difference processes operate on a faster time scale than the fluid scaled processes in the limit because of a separation of time scales the driving processes converge to a time dependent steady state or local average of a time varying fast time scale process ftsp this averaging principle allows us to replace the driving processes with the long run average behavior of the ftsp

modified hss iteration methods for a class of non hermitian positive definite linear systems abstract we consider the numerical solution of a class of non hermitian positive definite linear systems by the modified hermitian and skew hermitian splitting mhss iteration method we show that the mhss iteration method converges unconditionally even when the real and the imaginary parts of the coefficient matrix are nonsymmetric and positive semidefinite and at least one of them is positive definite at each step the mhss iteration method requires to solve two linear sub systems with real nonsymmetric positive definite coefficient matrices we propose to use inner iteration methods to compute approximate solutions of these linear sub systems we illustrate the performance of the mhss method and its inexact variant by two numerical examples

quasi birth and death processes level geometric distributions an aggregation disaggregation approach a special class of homogeneous continuous time quasi birth and death qbd markov chains mcs which possess level geometric lg stationary distribution are considered a functional analytic approach is applied which provides not only a clear analytic interpretation of the concepts introduced elsewhere but also represents a suitable basis for computations an iterative aggregation disaggregation method is proposed as tool for numerical computations

random matrix ensembles with column row constraints ii we analyze statistical properties of a complex system subjected to conditions which manifests through specific constraints on the column row sum of the matrix elements of its hermitian operators the presence of additional constraints besides real symmetric nature leads to new correlations among their eigenfunctions hinders a complete delocalization of dynamics and affects the eigenvalues too the statistical analysis of the latter indicates the presence of a new universality class analogous to that of a special type of brownian ensemble appearing between poisson and gaussian orthogonal ensemble

a class of iteration methods based on the hss for toeplitz systems of weakly nonlinear equations for toeplitz systems of weakly nonlinear equations combining the separability and strong dominance between the linear and the nonlinear terms with the hermitian and skew hermitian splitting hss iteration technique we establish two nonlinear composite iteration schemes called picard csss and nonlinear csss like iteration methods which are based on a special case of the hss where the symmetric part h 1 2 a a t is a centrosymmetric matrix and the skew symmetric part h 1 2 a a t is a skew centrosymmetric matrix the advantages of these methods are that they can transfer the linear sub systems involved in inner iteration to two linear systems of half an order besides fast methods are available for computing the two half steps involved in the inner iteration numerical results are provided to further show that both picard csss and nonlinear csss like iteration methods are feasible and effective

block triangular and skew hermitian splitting methods for positive definite linear systems by further generalizing the concept of hermitian or normal and skew hermitian splitting for a non hermitian and positive definite matrix we introduce a new splitting called positive definite and skew hermitian splitting pss and then establish a class of pss methods similar to the hermitian or normal and skew hermitian splitting hss or nss method for iteratively solving the positive definite systems of linear equations theoretical analysis shows that the pss method converges unconditionally to the exact solution of the linear system with the upper bound of its convergence factor dependent only on the spectrum of the positive definite splitting matrix and independent of the spectrum of the skew hermitian splitting matrix as well as the eigenvectors of all matrices involved when we specialize the pss to block triangular or triangular and skew hermitian splitting btss or tss the pss method naturally leads to a btss or tss iteration method which may be more practical and efficient than the hss and nss iteration methods applications of the btss method to the linear systems of block two by two structures are discussed in detail numerical experiments further show the effectiveness of our new methods

preconditioned hermitian and skew hermitian splitting methods for non hermitian positive semidefinite linear systems for the positive semidefinite system of linear equations of a block two by two structure by making use of the hermitian skew hermitian splitting iteration technique we establish a class of preconditioned hermitian skew hermitian splitting iteration methods theoretical analysis shows that the new method converges unconditionally to the unique solution of the linear system moreover the optimal choice of the involved iteration parameter and the corresponding asymptotic convergence rate are computed exactly numerical examples further confirm the correctness of the theory and the effectiveness of the method

circulant and skew circulant splitting methods for toeplitz systems we study efficient iterative methods for toeplitz systems based on the circulant and skew circulant splitting cscs of the toeplitz matrix theoretical analysis show that if the circulant and the skew circulant splitting matrices are positive definite then the cscs method converges to the unique solution of the system of linear equations moreover we derive an upper bound of the contraction factor of the cscs iteration which is dependent solely on the spectra of the circulant and the skew circulant matrices involved numerical examples are presented to demonstrate the method

trigonometric transform splitting methods for real symmetric toeplitz systems abstract in this paper we study efficient iterative methods for real symmetric toeplitz systems based on the trigonometric transformation splitting tts of the real symmetric toeplitz matrix a theoretical analyses show that if the generating function f of the n n toeplitz matrix a is a real positive even function then the tts iterative methods converge to the unique solution of the linear system of equations for sufficient large n moreover we derive an upper bound of the contraction factor of the tts iteration which is dependent solely on the spectra of the two tts matrices involved different from the cscs iterative method in ng 2003 in which all operations counts concern complex operations when the dfts are employed even if the toeplitz matrix a is real and symmetric our method only involves real arithmetics when the dcts and dsts are used the numerical experiments show that our method works better than cscs iterative method and much better than the positive definite and skew symmetric splitting pss iterative method in bai et al 2005 and the symmetric gauss seidel sgs iterative method

stochastic darboux transformations for quasi birth and death processes and urn models abstract we consider stochastic ul and lu block factorizations of the one step transition probability matrix for a discrete time quasi birth and death process namely a stochastic block tridiagonal matrix the simpler case of random walks with only nearest neighbors transitions gives a unique lu factorization and a one parameter family of factorizations in the ul case the block structure considered here yields many more possible factorizations resulting in a much enlarged class of potential applications by reversing the order of the factors also known as a darboux transformation we get new families of quasi birth and death processes where it is possible to identify the matrix valued spectral measures in terms of a geronimus ul or a christoffel lu transformation of the original one we apply our results to one example going with matrix valued jacobi polynomials arising in group representation theory we also give urn models for some particular cases

green s functions for elliptic and parabolic equations with random coefficients ii this paper is concerned with linear parabolic partial differential equations in divergence form and their discrete analogues it is assumed that the coefficients of the equation are stationary random variables random in both space and time the green s functions for the equations are then random variables regularity properties for expectation values of green s functions are obtained in particular it is shown that the expectation value is a continuously differentiable function in the space variable whose derivatives are bounded by the corresponding derivatives of the green s function for the heat equation similar results are obtained for the related finite difference equations this paper generalises results of a previous paper which considered the case when the coefficients are constant in time but random in space

consensus over random graph processes network borel cantelli lemmas for almost sure convergence distributed consensus computation over random graph processes is considered the random graph process is defined as a sequence of random variables which take values from the set of all possible dig

the maximum number of infected individuals in sis epidemic models computational techniques and quasi stationary distributions we study the maximum number of infected individuals observed during an epidemic for a susceptible infected susceptible sis model which corresponds to a birth death process with an absorbing state we develop computational schemes for the corresponding distributions in a transient regime and till absorption moreover we study the distribution of the current number of infected individuals given that the maximum number during the epidemic has not exceeded a given threshold in this sense some quasi stationary distributions of a related process are also discussed

convergence properties of preconditioned hermitian and skew hermitian splitting methods for non hermitian positive semidefinite matrices for the non hermitian and positive semidefinite systems of linear equations we derive necessary and sufficient conditions for guaranteeing the unconditional convergence of the preconditioned hermitian and skew hermitian splitting iteration methods we then apply these results to block tridiagonal linear systems in order to obtain convergence conditions for the corresponding block variants of the preconditioned hermitian and skew hermitian splitting iteration methods

moments of random variables a systems theoretic interpretation moments of continuous random variables admitting a probability density function are studied we show that under certain assumptions the moments of a random variable can be characterized in terms of a sylvester equation and of the steady state output response of a specific interconnected system this allows to interpret well known notions and results of probability theory and statistics in the language of systems theory including the sum of independent random variables the notion of mixture distribution and results from renewal theory the theory developed is based on tools from center manifold theory the theory of the steady state response of nonlinear systems and the theory of output regulation our formalism is illustrated by means of several examples and can be easily adapted to the case of discrete and multivariate random variables

quasi stationary distributions for discrete state models this paper contains a survey of results related to quasi stationary distributions which arise in the setting of stochastic dynamical systems that eventually evanesce and which may be useful in describing the long term behaviour of such systems before evanescence we are concerned mainly with continuous time markov chains over a finite or countably infinite state space since these processes most often arise in applications but will make reference to results for other processes where appropriate next to giving an historical account of the subject we review the most important results on the existence and identification of quasi stationary distributions for general markov chains and give special attention to birth death processes and related models the question of under what circumstances a quasi stationary distribution given its existence is indeed a good descriptor of the long term behaviour of a system before evanescence is addressed as well we conclude with a discussion of computational aspects with more details given in a web appendix accompanying this paper

the intercept term of the asymptotic variance curve for some queueing output processes we consider the output processes of some elementary queueing models such as the m m 1 k queue and the m g 1 queue an important performance measure for these counting processes is their variance curve v t which gives the variance of the number of customers in the time interval 0 t recent work has revealed some non trivial properties dealing with the asymptotic rate at which the variance curve grows in this paper we add to these results by finding explicit expressions for the intercept term of the linear asymptote for m m 1 k queues our results are based on the deviation matrix of the generator it turns out that by viewing output processes as markovian point processes and considering the deviation matrix one can obtain explicit expressions for the intercept term together with some further insight regarding the bravo balancing reduces asymptotic variance of outputs effect for m g 1 queues our results are based on a classic transform of d j daley in this case we represent the intercept term of the variance curve in terms of the first three moments of the service time distribution in addition we shed light on a conjecture of daley dealing with characterization of stationary m m 1 queues within the class of stationary m g 1 queues based on the variance curve

gerbershiu analysis with two sided acceptable levels in this paper insurers surplus process moved within upper and lower levels is analyzed to this end a truncated type of gerbershiu function is proposed by further incorporating the minimum and the maximum surplus before ruin into the existing ones e g gerber and shiu 1998 cheung et al 2010a a key component in our analysis of this proposed gerbershiu function is the so called transition kernel explicit expressions of the transition function under two different risk models are obtained these two models are both generalizations of the classical poisson risk model i the first model provides flexibility in the net premium rate which is dependent on the surplus such as linear or step function and ii the second model assumes that claims arrive according to a markovian arrival process map finally we discuss some applications of the truncated gerbershiu function with numerical examples under various scenarios

equilibrium fluctuations for nabla_ varphi interface model we study the large scale space time fluctuations of an interface which is modeled by a massless scalar field with reversible langevin dynamics for a strictly convex interaction potential we prove that on a large space time scale these fluctuations are governed by an infinite dimensional ornstein uhlenbeck process its effective diffusion type covariance matrix is characterized through a variational formula

on choices of iteration parameter in hss method the hss iteration method is effective to solve non hermitian positive definite linear systems but the choice of its optimal parameter is a difficult and challenging problem in theoretical analysis and practical computations in this paper we obtain an accurate estimate to the optimal parameter of the hss iteration method by adopting a reasonable and simple optimization principle numerical experiments show that this principle is feasible to produce an accurate estimate to the optimal parameter of the hss iteration method

optimal parameter in hermitian and skew hermitian splitting method for certain two by two block matrices the optimal parameter of the hermitian skew hermitian splitting hss iteration method for a real two by two linear system is obtained the result is used to determine the optimal parameters for linear systems associated with certain two by two block matrices and to estimate the optimal parameters of the hss iteration method for linear systems with n by n real coefficient matrices numerical examples are given to illustrate the results

convection diffusion equation with space time ergodic random flow we prove the homogenization of convection diffusion in a time dependent ergodic incompressible random flow which has a bounded stream matrix and a constant mean drift we also prove two variational formulas for the effective diffusivity as a consequence we obtain both upper and lower bounds on the effective diffusivity

the influence of the marked reduced graph of a nonnegative matrix on the jordan form and on related properties a survey abstract for a nonnegative matrix p we discuss the relation of its marked reduced graph to that part of the jordan form that is associated with the perron frobenius root to the nonnegativity of the eigenvectors and generalized eigenvectors to the nonnegativity of solutions of linear equations and to the asymptotic growth of powers of the matrix results are often stated in terms of m matrices and standard results on irreducible matrices are assumed we give examples to illustrate the theorems surveyed

on the hss iteration methods for positive definite toeplitz linear systems we study the hss iteration method for large sparse non hermitian positive definite toeplitz linear systems which first appears in bai golub and ng s paper published in 2003 z z bai g h golub m k ng hermitian and skew hermitian splitting methods for non hermitian positive definite linear systems siam j matrix anal appl 24 2003 603 626 and hss stands for the hermitian and skew hermitian splitting of the coefficient matrix a in this note we use the hss iteration method based on a special case of the hss splitting where the symmetric part h 12 a a t is a centrosymmetric matrix and the skew symmetric part s 12 a a t is a skew centrosymmetric matrix for a given toeplitz matrix hence fast methods are available for computing the two half steps involved in the hss and ihss iteration methods some numerical results illustrate their effectiveness

real eigenvalues of certain tridiagonal matrix polynomials with queueing applications many queueing problems lead to tridiagonal lambda matrices containing polynomials that have except for the diagonal non negative coefficients this paper deals with the question addressed in the literature only for special cases whether the eigenvalues corresponding to such lambda matrices are real in most cases they are as the theorems of this paper show but sometimes complex eigenvalues occur our results are derived by using sturm sequences in addition to simplifying the proofs of our theorems sturm sequences are also valuable to verify whether or not a given interval contains eigenvalues

on estimating the derivatives of symmetric diffusions in stationary random environment with applications to ϕ interface model we consider diffusions on d or random walks on d in a random environment which is stationary in space and in time and with symmetric and uniformly elliptic coefficients we show existence and holder continuity of second space derivatives and time derivatives for the annealed kernels of such diffusions and give estimates for these derivatives in the case of random walks these estimates are applied to the ginzburg landau interface model

on the averaged green s function of an elliptic equation with random coefficients we consider a divergence form elliptic difference operator on the lattice mathbb z d with a coefficient matrix that is an i i d perturbation of the identity matrix recently bourgain introduced novel techniques from harmonic analysis to prove the convergence of the feshbach schur perturbation series related to the averaged green s function of this model our main contribution is a refinement of bourgain s approach which improves the key decay rate from 2d epsilon to 3d epsilon the optimal decay rate is conjectured to be 3d as an application we derive estimates on higher derivatives of the averaged green s function which go beyond the second derivatives considered by delmotte deuschel and related works

survival in a quasi death process we consider a markov chain in continuous time with an absorbing coffin state and a finite set s of transient states when s is irreducible the limiting distribution of the chain as t to infty conditional on survival up to time t is known to equal the unique quasi stationary distribution of the chain we address the problem of generalizing this result to a setting in which s may be reducible and obtain a complete solution if the eigenvalue with maximal real part of the generator of the sub markov chain on s has multiplicity one the result is applied to pure death processes and more generally to quasi death processes

on hss and ahss iteration methods for nonsymmetric positive definite toeplitz systems two iteration methods are proposed to solve real nonsymmetric positive definite toeplitz systems of linear equations these methods are based on hermitian and skew hermitian splitting hss and accelerated hermitian and skew hermitian splitting ahss by constructing an orthogonal matrix and using a similarity transformation the real toeplitz linear system is transformed into a generalized saddle point problem then the structured hss and the structured ahss iteration methods are established by applying the hss and the ahss iteration methods to the generalized saddle point problem we discuss efficient implementations and demonstrate that the structured hss and the structured ahss iteration methods have better behavior than the hss iteration method in terms of both computational complexity and convergence speed moreover the structured ahss iteration method outperforms the hss and the structured hss iteration methods the structured ahss iteration method also converges unconditionally to the unique solution of the toeplitz linear system in addition an upper bound for the contraction factor of the structured ahss iteration method is derived numerical experiments are used to illustrate the effectiveness of the structured ahss iteration method

rate tilting for fast simulation of level phase processes abstract this paper study the efficient simulation methods for estimating some metrics that depend heavily on a certain rarely occurring event the process considered is the so called level phase process a markov process in which the level and the phase are two state variables furthermore changes of level and phase are induced by events which have rates that are independent of the level except at a boundary if a system typically stays at lower levels then reaching a high level n is a rare event thus direct simulation for the related metrics is very inefficient we change the events rates in a level phase process to accelerate simulation and find from simulation the so called hitting probability the probability of entering a rare event set this method is called rate tilting and in our approach a proper construct of rate tilting relates to a generalized eigenvalue problem involving the infinitesimal generator matrix of the process being considered we can show that the relative estimation error of the hitting probability resulting from the proposed simulation remains bounded as the level increases provided that the boundary set of the state space satisfies certain conditions if these conditions are met rate tilting will be advantageous