# The Limit Theorems for Function of Markov Chains in the Environment of Single Infinite Markovian Systems

Authors: Zhanfeng Li; Min Huang; Xiaohua Meng; Xiangyu Ge
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

This paper is intended to study the limit theorem of Markov chain function in the environment of single infinite Markovian systems. Moreover, the problem of the strong law of large numbers in the infinite environment is presented by means of constructing martingale differential sequence for the measurement under some different sufficient conditions. If the sequence of even functionsgnx,n≥0 satisfies different conditions when the value ranges of x are different, we have obtained SLLN for function of Markov chain in the environment of single infinite Markovian systems. In addition, the paper studies the strong convergence of the weighted sums of function for finite state Markov Chains in single infinitely Markovian environments. Although the similar conclusions have been carried out, the difference results performed by previous scholars are that we give weaker different sufficient conditions of the strong convergence of weighted sums compared with the previous conclusions.


---

## Body

## 1. Introduction

The definition and properties of limit theorems have been studied for some time, especially for functions of Markov chain, which becomes one of the most popular research areas in the field of stochastic processes. In a random process model, the theory of Markov chain describes the change from one system state to another system state, and also the advent of quantitative analysis is explored according to the real system situations. In effect, the theory of Markov chain is not only widely used in scientific research but also used in the economic field. Recently, emphasis is placed on the application to explain many systemic problems of economic phenomena, and the interpretation of most economic phenomena can be realized under the framework of Markov chain.Over the course of the past 40 years, a comprehensive study of the Markov chains has been undertaken. In the early 1980s, Cogburn [1] introduced the definition of Markov chain in the environment of random systems and discussed the state classification of Markov chain in the environment of Markovian systems. A relevant paper was published by Nawrotzki [2, 3], which discussed the state of classification about the Markov chain with feedback, based on other systems, namely, the single infinitely stable Markovian systems, and established the general theory of the topic. Subsequently, a further study on Markov chains under the condition of random environment has been reported by Cogburn [4] who have made a great contribution in this area. In random environments, Cogburn proposed a generic theory about the function of Markov chains and developed a lot of profound results with the theory of Hopf-Markov chain and some further studies of limit theories for function of Markov chain have been conducted. For instance, Cogburn [5, 6] discussed the convergence of the transfer probability, periodicity, and conditions for the establishment of the central limit theorem under the special circumstances, which are in the environment of the bi-infinite stable, as well as listing the connection with these theories. Orey [7] has studied the Markov chain in stochastic environment in depth based on Cogburn’s study and put forward a series of problems, which attracted the attention of many probability scholars. Liu and Liu [8, 9] have investigated a series of limit properties of the random variables sequences with Lebesgue’s theorem and then gave limit properties with the similar method for nonhomogeneous Markov chain.The general theory about the function of Markov chain in the environment of random systems has become a popular research direction. As known, the limit theorem has been a hot topic in the study of classical Markov chain theory. Subsequently, a lot of scholars have conducted in-depth research in this field and achieved a series of profound and rich results. Various research theories about the function of Markov chain in Markovian environments have been proposed, called MCME for short (see [10–15]), and the same as theories about the function of Markov chain in random environments, which are called MCRE for short (see [16–18]). Exactly, the random environments can be catalogued into different situations, such as in space-time random environments (see [19]), in bi-infinite random environments (see [20]), and in single infinite random environments (see [21]).Currently, a lot of research literatures on the strong limit theory for function of Markov chains in the environment of random systems or in the environment of Markovian systems have been found. The strong law of large numbers about the function of Markov chains in the environment of Markovian systems with discrete parameter was proposed by Wan [22] who obtained the sufficient conditions. Besides, Guo [16] also put forward the sufficient conditions, which are different from Wan [22], and the difference is mainly to prove this theorem in the case of the random environments. On the basis of existing research, Li [23] indicated the sufficient conditions established in the case of countable states for this theorem. Furthermore, for complete and imprecise knowledge of Markov chains, Li et al. [14] have developed a strong limit theorem of the Markov chain quaternion function in the environment of Markovian systems and extended the Shannon theorem in this environment. It becomes apparent that Markovian environments can be classified into different catalogues, such as in bi-infinite environments (see [24, 25]) and in single infinite environments (see [26]).To the best of our knowledge, along with the increasing development of Markov chains in decision-making state of financial market, they have been widely used in financial insurance theory. Recently, statistical estimation of ruin related functions has become a popular topic in risk theory. However, some of the scholars proposed different estimators for the ruin probability in the classical risk. For example, a study by Yang and Yuen [27] offered a comprehensive analysis of two-dimensional delayed renewal risk model with a constant interest. They derived some asymptotic formulas for the finite-time and infinite-time ruin probabilities in the presence of heavy-tailed claim sizes. In addition, Yang et al. [28] constructed by the two-dimensional Fourier cosine series expansion to estimate the discounted density of the deficit at ruin. Similarly, with method of the Fourier cosine series expansion, one study by Yu et al. [29] valued the guaranteed minimum death benefit products. On valuation of the products with guaranteed minimum death benefit, Zhang et al. [30] applied a projection method combined with Fast Fourier Transform. Recently, a qualitative study by Yang et al. [31] described a discrete-time insurance risk model with insurance and financial risks. Then, a key study is that of Yu et al. [32], which proposed a new risk model called compound Poisson risk model by introducing a periodic capital injection strategy and a barrier dividend strategy into the classical risk model. Furthermore, the risk model can be further extended and applied to a wider range of practical problems. An example can be made by the optimal control problem (see [33]). Above all, most of existing studies failed to deal with the practical problems; it is necessary for us to study the limit theorems for function of Markov chains deeply.Therefore, this paper set out to advance the research on the limit theory of a class of Markov chain functions, aiming to provide clarity understanding of the limit theorem for function of Markov chain in environment of single infinite Markovian systems. Based on the results of Li [17] and Wan [18], we have found two lemmas and derive the results of almost sure convergence with the finding of the lemmas. The results of the analysis of the strong law of large numbers in single infinite environment are presented to be an extension of the conclusion in the inference [34] on different sufficient conditions. Also, in single infinite Markovian environments, we come to the conclusions about the strong convergence and present the weighted sums for function of Markov chains. Driven by LLN (law of large numbers), the strong convergence of the weighted sum is also discussed when considering the compatibility of the least squares estimates of linear models. Although we prove similar conclusions, the difference from results obtained by previous scholars can be made based on weaker different sufficient conditions of the strong convergence of weighted sums provided [26].The subsequent structure of our paper is as follows. A description of some basic notations, fundamental definitions, and lemmas is shown briefly in Section 2. Details under the condition of the environment of single infinite Markovian systems are discussed in Section 3 and Section 4. Section3 is about SLLN (strong law of large numbers) for function of Markov chain in the environment of the setting of this paper. Section 4 derives the strong convergence of weighted sum for function of Markov chain in the environment of the setting of this paper. A series of sufficient conditions for the limit theorem are obtained in Sections 3 and 4, and specific proof process of theorem and corollary is considered, respectively. This paper concludes with a discussion in Section 5.


## 2. Fundamental Preliminaries

At first, we begin to introduce some basic notations which shall be used in the following sections. LetN represent an integer set and Ω,ℱ,P represent a probability space; both X,A and Θ,ℬare arbitrary measurable spaces. Respectively, let ξ⟶0∞=ξn:n≥0 and X⟶=Xn:n≥0 be two random sequences defined on Ω,ℱ,P with value on set of Θ and X. Assume a family of transition function Pθ:θ∈Θ defined on arbitrary measurable spaces of X,A. For any A∈A, we suppose P⋅;⋅,A is measurable regarding ℬ×A. Given a family of one-step transition probability function Kn⋅,⋅ defined on arbitrary measurable spaces of Θ,ℬ, where we assume that Kn⋅,B is measurable about ℬ for any B∈ℬ. For arbitrary sequence of η⟶=ηn:n≥0, we denote η⟶kr=ηn:0≤n≤r≤∞.If, for any ofA∈A and n≥0, at the same time we have(1)PX0∈Aξ⟶0∞=PX0∈Aξ0,PXn+1∈AX⟶0n,ξ⟶0∞=Pξn;Xn,A,then random variable sequence of X⟶ is called the Markov chain in the random system of ξ⟶0∞; here ξ⟶0∞ is a sequence in single infinite random environments. In other words, if ξ⟶0∞ is a Markov sequence, X⟶ is called the Markov chain in the environment of single infinite Markovian systems.Given a random variable sequenceXn,n≥0 on Ω,ℱ,P, the following statement is satisfied. If, for arbitrary x>0 and n≥0, there exists PXn>x≤CPV>x, where V represents a nonnegative random variable, C appearing here represents a constant and is greater than zero. Thus we call Xn,n≥0 the tail probability uniformly bounded by V and denote it as Xn<V. This paper always sets X⟶ as the Markov chain in single infinite Markovian environments. We assume that C appearing in this paper represents a positive constant that represents different values in different positions. The indicative function of the set of A is denoted as IA.The purpose of the study is to propose limit theorems for function of Markov chain in analysis of the environment of single infinite Markovian systems and to conclude some different sufficient conditions for the almost sure convergence by means of constructing martingale differential sequence. In addition, we derive the similar conclusion about strong convergence of the weighted sums in the given environment in terms of some weaker sufficient conditions.Two specific contributions have been included as follows, which are both built on the environment of single infinite Markovian systems. Firstly, this study will offer a fresh insight into the following Theorem1 and Theorem 2 and Corollaries 1 and 2 to show Markov chain’s SLLN (strong law of large numbers) in the given environment. Secondly, this study will provide an important opportunity to advance the understanding of Markov chain strong convergence of weighted sum in the given environment. The results are given by Theorem 3 and Corollary 3.To prove the main theorems, the following deformation lemmas are needed.Lemma 1 (see, e.g., Conclusion 1 in [17]).
Given a Markov chainX⟶ in the environment of single infinite Markovian systems ξ⟶0∞, the random variable sequence of Xn,ξn:n≥0 is the Markovian chain in double. In particular, if the one-step transition function in the single infinite Markovian environments ξ⟶0∞ is Knθ,B, the one-step transition probability of Xn,ξn:n≥0 is(2)Qnx,θ;A×B=Knθ,BPθ;x,A.Ifξ⟶0∞ is time-homogeneous, then Xn,ξn:n≥0 is time-homogeneous too.Lemma 2 (see Lemma2 in [18]).
Assume thatX is a random variable; for any x>0, there exists PX>x≤CPV>x, where V is a nonnegative random variable and C>0 is a constant, such that, for any x>0 and q>0, we have(3)EXqIX≤x≤CxqPV>x+CEVqIV≤x.


## 3. SLLN in the Environment of Single Infinite Markovian Systems

To begin with, we use Lemmas1 and 2 to investigate SLLN for function of Markov chain in the environment of single infinite Markovian systems. A series of sufficient conditions for SLLN are given for function of Markov Chains in the environment of single infinite Markovian systems. The relevant results are in accordance with the following major Theorems 1 and 2, as well as Corollaries 1 and 2.Theorem 1.
Given the probability space ofΩ,ℱ,P with values on set of X×Θ, we assume that Xn,ξn:n≥0 is a Markov chain, FnXn,ξn:n≥0 is a sequence of measurable functions defined on X×Θ,A×ℬ, and gnx,n≥0 is a sequence of even functions defined on set of R which is taking a positive value on the interval of 0,∞. For any n≥0, there always exists a value of λ>0, if one of the following conditions holds:(i)
gnx is monotonically nondecreasing on the interval of 0,∞. When 0<x≤1, there is gnx≥λxθ0<θ≤1, and EFnXn,ξn=0,n≥0;(ii)
gnx≥λxα0<α≤2,0<x≤1,λxββ≥1,x>1.At the same time, for positive constant sequences ofan,n≥0 where an↑∞ is satisfied, if there exist(4)∑m=0∞EgmFmXm,ξmam<∞,then, for anyk≥1, we get the following series, which are convergent almost surely:(5)∑m=0∞FmXm,ξm−EFmXm,ξmXm−k,ξm−kama.s.convergence,and consequently we have the following formula, which is true almost surely:(6)limn⟶∞1an∑m=0nFmXm,ξm−EFmXm,ξmXm−k,ξm−k=0a.s..Here, we agreeX−k≡0 and ξ−k≡0 for any k≥1.According to the derivation of Theorem1, we can further generalize Corollary 1 as follows.Corollary 1.
Assume a Markov chainXn,ξn:n≥0 which is defined on probability space of Ω,ℱ,P with values on set of X×Θ. FnXn,ξn:n≥0 is a sequence of measurable functions defined on X×Θ,A×ℬ. For the positive constant sequences of an,n≥0, satisfying an↑∞, if one of the following two conditions holds:(iii)
∑m=0∞EFmXm,ξmr/amr+FmXm,ξmr<∞, where 0<r<1 and EFnXn,ξn=0,n≥0;(iv)
∑m=0∞EFmXm,ξmr/amr+amFmXm,ξmr−1<∞, where 1≤r≤2, then (5) and (6) hold.Proof of Theorem 1.
Let us now prove the above Theorem1. In the process of proof, we discuss two situations. One situation is k=1, and the other situation is k>1.
Firstly, we consider the situation ofk=1.
Under condition (i), there satisfiesFnXn,ξn>an, that is, FnXn,ξn/an>1. We know from the condition that gnx is monotonically nondecreasing on the interval of 0,∞, and we can get gn1≥λ when condition (i) is satisfied. Thus, we can get the inequality gnFnXn,ξn/an≥Cgn1 which will be used in the following derivation. Next, let us give the detailed process of the derivation of inequalities (7) and (8) by using Lemma 2. The derivation of inequality is as follows:(7)∑m=0∞PFmXm,ξm>am≤C∑m=1∞Egm1IFmXm,ξm>am≤C∑m=1∞EgmFmXm,ξmamIFmXm,ξm>am≤C∑m=1∞EgmFmXm,ξmam<∞,(8)E∑m=0∞EFmXm,ξmamIFmXm,ξm>amXm−1,ξm−1=E∑m=0∞EFmXm,ξmamIFmXm,ξm≤amXm−1,ξm−1≤∑m=0∞EFmXm,ξmamIFmXm,ξm≤am≤∑m=0∞EFmXm,ξmθamθIFmXm,ξm≤am≤C∑m=0∞EgmFmXm,ξmamIFmXm,ξm≤am≤C∑m=0∞EgmFmXm,ξmam<∞.
Under condition (ii), whenFnXn,ξn>an is satisfied at the same time, we use the second case of condition (ii) gnx≥λxββ≥1,x>1 to get the inequality gnFnXn,ξn/an≥CFnXn,ξnβ/anβ which will be used in the following derivation. Next, let us give the detailed process of the derivation of inequalities (9) and (10). The derivation of inequality is as follows:(9)∑m=0∞PFmXm,ξm>am≤C∑m=0∞EFmXm,ξmβamβIFmXm,ξm>am≤C∑m=0∞EgmFmXm,ξmam<∞,(10)E∑m=0∞EFmXm,ξmamIFmXm,ξm>amXm−1,ξm−1≤∑m=0∞EFmXm,ξmamIFmXm,ξm>am≤∑m=0∞EFmXm,ξmβamβIFmXm,ξm>am≤C∑m=0∞EgmFmXm,ξmam<∞.
Obviously, by using formulas (7) and (9), it is obvious that the result below is established almost surely:(11)∑m=0∞IFmXm,ξm>am<∞a.s..
BecausePFmXm,ξm>am:i.o.=0, consequently we can get that the following series are convergent almost surely:(12)∑m=0∞FmXm,ξmamIFmXm,ξm>ama.s.convergence.
At the same time, by using formulas (8) and (10), we can get that the following series are convergent almost surely:(13)∑m=0∞EFmXm,ξmamIFmXm,ξm>amXm−1,ξm−1a.s.convergence.
Note that(14)Zn=FnXn,ξnIFnXn,ξn≤anan−EFnXn,ξnIFnXn,ξn≤anXn−1,ξn−1an,ℬn=σX⟶0n,ξ⟶0n.
Z
n
,
ℬ
n
,
n
≥
0 is known as a martingale difference sequence by the nature of function of Markov chain of Xn,ξn,n≥0. Under condition (i), the following results can be derived by the orthogonality of martingale difference sequence:(15)E∑m=0nZm2=∑m=0nEZm2≤C∑m=0nEFmXm,ξm2am2IFmXm,ξm≤am≤C∑m=0nEFmXm,ξm|θamθIFmXm,ξm≤am≤C∑m=0∞EgmFmXm,ξmamIFmXm,ξm≤am≤C∑m=0∞EgmFmXm,ξmam.
Also, under condition (ii), the inequality is derived as follows:(16)E∑m=0nZm2=∑m=0nEZm2≤C∑m=0nEFmXm,ξm2am2IFmXm,ξm≤am≤C∑m=0nEFmXm,ξmαamαIFmXm,ξm≤am≤C∑m=0∞EgmFmXm,ξmam.
By formula (4), we know that supn≥0E∑m=0nZm2<∞, which means ∑m=0nZm,ℬn,n≥0 is bounded martingale on L2. Thus, we can get that the series ∑m=0∞Zm are convergent almost surely. Then, by combining formulas (12) and (13), we can see that formula (5) holds, and it is also easy to know that formula (6) holds by Kronecker’s Lemma.
Secondly, we consider the other situation ofk>1.
By the nature of function of the Markov chainXn,ξn:n≥0, it is known that Xmk+n,Ymk+n:m≥0 is a Markov chain for any n=1,2,3,…,k−1. It is easy to derive the following result regarding convergence by using formula (4):(17)∑m=0∞Egmk+nFmk+nXmk+n,ξmk+namk+n<∞.
Therefore, for anyn=1,2,3,…,k−1, we have the series of almost sure convergence as follows:(18)∑m=0∞Fmk+nXmk+n,ξmk+n−EFmk+nXmk+n,ξmk+nXmk+n−k,ξmk+n−kamk+na.s.convergence.
Thus, the following results of convergence almost surely can be derived:(19)∑m=0∞FmXm,ξm−EFmXm,ξmXm−k,Ym−kam=∑m=0∞∑n=0k−1Fmk+nXmk+n,ξmk+n−EFmk+nXmk+n,ξmk+nXmk+n−k,ξmk+n−kamk+n=∑n=0k−1∑m=0∞Fmk+nXmk+n,ξmk+n−EFmk+nXmk+n,ξmk+nXmk+n−k,ξmk+n−kamk+n,where, for the situation of k>1, formula (6) is true. Obviously, by Kronecker’s Lemma, formula (7) also holds for the situation of k>1.
Now, the conclusions on convergence have been demonstrated and proven completely. The method of using constructing martingale differential sequence to implement different sufficient conditions of the strong limit theorems is different from the evidence from previous observations. Our paper advances the research on the limit theory of a class of Markov chain functions by using different sufficient conditions.Remark 1.
Both Theorem1 and Lemma 4 in literature [24] give the sufficient conditions for SLLN of Markov chain, but the preconditions in the two theorems are different.
From Theorem1, we can see that the sequence of even functions gnx,n≥0 satisfies different conditions when the value ranges of x are different. In condition (i), when 0<x≤1 and 0<θ≤1, gnx needs to be monotonically nondecreasing and EFnXn,ξn=0 with n≥0. However, in condition (ii), these conditions are not required to approach limit but only to obtain the segment gnx and also a range of values are satisfied. From these two different sufficient conditions, we have obtained SLLN for function of Markov chain in the environment of single infinite Markovian systems. On the basis of Theorem 1, Corollary 1 shows that SLLN can be obtained by assigning different functional forms to gnx, which generalize the previous conclusions.Proof of Corollary 1.
At first, when condition (iii) is established, there existsgnx=xr/1+|x|r, where the range of value of r is 0<r<1. When condition (iv) is established, there is gn′x=xr/1+xr−1, where the range of value of r is 1≤r≤2. Then, for any n≥0, gnx and gn′x are both even nondecreasing functions, taking a positive value on interval of 0,∞. At the same time, respectively, there are the following formulas:(20)gnx≥12xr,0<x≤1,0<r<1,gn′x≥12xr,0<x≤1,1≤r≤2,12x,x>1.
If condition (iii) is satisfied, we have(21)∑m=0∞EgnFmXm,ξmam=∑m=0∞EFmXm,ξmramr+FmXm,ξmr<∞.
If condition (iv) is satisfied, we have(22)∑m=0∞Egm′FmXm,ξmam=∑m=0∞EFmXm,ξmramr+amFmXm,ξmr−1<∞.
From the above results, it can be seen that both condition (i) and condition (ii) under Theorem1 are satisfied, so we can obtain that Corollary 1 holds by Theorem 1.
The sample frequency as a significant part of information theory plays a very important role in statistical hypothesis testing and coding theory. Based on the above results, a class of SLLN on the frequency of occurrence of the state are obtained for function of Markov chain in the environment of single infinite Markovian systems. Meanwhile, the relationship between the frequency of occurrence of statex,θ and the product of corresponding one-step transition probability function can be seen from the following Theorem 2. In particular, we can use the initial state to describe the frequency of occurrence of the state when the time-homogeneity is satisfied. Under the certain environment of the single infinite Markovian systems, we have concluded Corollary 2, which can be extended to a class of SLLN on the frequency of occurrence of the state for function of Markov chain.Theorem 2.
Assume that a Markov chainX is in the environment of the single infinite Markovian systems ξ0∞⟶. Snx,θrepresents the frequency where x,θ appears in the sequence of X1,ξ1,X2,ξ2,…,Xn,ξn; if ∑m=0∞am−r<∞, when 1<r≤2 , we then have(23)limn⟶∞Snx,θan−1an∑m=0nKmξm−1,θPξm−1;Xm−1,x=0a.s..In particular, whenξ0∞⟶ is time-homogeneous and limn⟶∞n/an=C, then we have(24)limn⟶∞Snx,θan=CKξ0,θPξ0;X0,xa.s..Corollary 2.
Assume that a Markov chainX is in the environment of the single infinite Markovian systems ξ0∞⟶. Snx represents the frequency where x appears in the sequence of X1,X2,…,Xn; if ∑m=0∞am−r<∞, when 1<r≤2, then we have(25)limn⟶∞Snxan−1n∑m=0nPξm−1;Xm−1,x=0a.s..In particular, whenξ0∞⟶ is time-homogeneous and limn⟶∞n/an=C, we have(26)limn⟶∞Snxan=CPξ0;X0,xa.s..Proof of Theorem 2.
GivenFnXn,ξn=δxXnδθξn, we have Snx,θ=∑m=1nFmXm,ξm. Then the following inequality can be derived through equation (10) above:(27)∑m=0∞EFmXm,ξmramFmXm,ξmr−1+amr≤∑m=0∞am−r<∞.
So we can get from the above formula that the series is convergent.
Obviously, it is known that the following limit holds almost surely by Corollary1:(28)limn⟶∞1an∑m=0nFmXm,ξm−EFmXm,ξmXm−1,ξm−1=0a.s..
Then, by the combination of the formula∑m=0nFmXm,ξm=Snx,θ+δxX0δθξ0 and Lemma 1, we can get the results below.(29)EFnXn,ξnXn−1,ξn−1=EδxXnδθξnXn−1,ξn−1=QnXn−1,ξn−1;x,θ=Knξn−1,θPξn−1;Xn−1,x.
Substituting the above formula into the formula limitation equation (28), we can get formula (23).
Ifξ0∞⟶ is time-homogeneous, it is known that Xn,ξn:n≥0 is also time-homogeneous by Lemma 1, and hence QnXn−1,ξn−1;x,θ=QX0,ξ0;x,θ=Kξ0,θPξ0;X0,x. So we can get formula (24) by using formula (23). Therefore, Theorem 2 has been proven.Proof of Corollary 2.
Given the formulaFnXn,ξn=δxXn, we have the formula Snx+δxX0=∑m=0nFmXm,ξm=∑m=0nδxXm. By combining these conditions and Lemma 1, we obtain the following results:(30)EFnXn,ξnXn−1,ξn−1=EδxXmXn−1,ξn−1=Pξn−1;Xn−1,x,and, therefore, Corollary 2 can be proven by using the method which is similar to that used in the Proof of Theorem 2.


## 4. The Strong Convergence of Weighted Sums in the Environment of Single Infinite Markovian Systems

In this section, we use Lemmas1 and 2 to study the strong convergence of weighted sum for the function of Markov chain in the environment of the single infinite Markovian systems. Some sufficient conditions for the strong convergence of weighted sum for the function of Markov Chains are obtained. The relevant results will be obtained from the analysis of Theorem 3 and Corollary 3.Theorem 3.
Assume that a Markov chainXn,ξn:n≥0 is defined on Ω,ℱ,P with values on set of X×Θ. FnXn,ξn:n≥0 is a sequence of measurable functions defined on X×Θ,A×ℬ. an,n≥0 and bn,n≥0 are arbitrary sequences of positive real numbers, respectively, and we denote cn=bn/an,bn↑∞, if the following conditions both hold:(a)
∑m=1∞cm−1EFmXm,ξmIFmXm,ξm>cm<∞;(b)
∑m=1∞cm−pEFmXm,ξmpIFmXm,ξm≤cm<∞,where there is 1≤p≤2,such that, for anyk≥1, we can get that the following series are convergent almost surely:(31)∑m=0∞FmXm,ξm−EFmXm,ξmXm−k,ξm−kcna.s.convergence,and we have(32)limn⟶∞1bn∑m=0namFmXm,ξm−EFmXm,ξmXm−k,ξm−k=0a.s..Consequently, we get Corollary3 as follows.Corollary 3.
Assume that a Markov chainXn,ξn:n≥0 is defined on Ω,ℱ,P with values on set of X×Θ. FnXn,ξn:n≥0 is a sequence of measurable functions defined on X×Θ,A×ℬ, and it satisfies FnXn,ξn<V. an,n≥0 and bn,n≥0 are arbitrary sequences of positive real numbers, respectively, and we denote cn=bn/an, bn↑∞. For arbitrary x>0, we denote Nx=Cardn:cn≤x, if V satisfies the following two conditions:(c)
ENV<∞;(d)
∫1∞ENV/tdt<∞.In addition, one of the following conditions is satisfied:(e)
∫0∞tp−1PV>t∫t∞Ny/yp+1dydt<∞, where there is 1≤p≤2;(f)
max0≤j≤ncjp∑m=n∞cm−p=On, where there is 1≤p≤2;(g)
∫01ENV/t1/pdt<∞,where there is 1≤p≤2, and, for any k≥1, both (31) and (32) hold.Proof of Theorem 3.
In a similar way to Theorem1, we also discuss two situations in the process of proof. One situation is k=1; the other situation is k>1.
We consider the first situation ofk=1. For any m≥0, we denote that(33)Zm=FmXm,ξmIFmXm,ξm≤cmcm−EFmXm,ξmIFmXm,ξm≤cmXm−1,ξm−1cm;Zm′=FmXm,ξmIFmXm,ξm>cmcm.
Under condition (a), it is known that the following series are absolute convergent almost surely:(34)∑m=0∞Zm′<∞a.s..
Hence, the following series are convergent almost surely:(35)∑m=0∞Zm′a.s..
Meanwhile, the following series of conditional expectation are convergent almost surely:(36)∑m=0∞EZm′Xm−1,ξm−1a.s..
Then, due to the orthogonal properties of the martingale difference sequence, it is known that, for any1≤p≤2, we can get(37)E∑m=0nZm2=∑m=0nEZm2≤C∑m=0n1cmpEFmXm,ξmpIFmXm,ξm≤cm.
Through the combination of condition (b) and formula (37), we have the formula supn≥0E∑m=0nZm2<∞. In other words, this formula shows that the result of ∑m=0nZm,σn,n≥0 is a bounded martingale on L2. Thus, we have that the series ∑m=0∞Zm are convergent almost surely, which is the weighted sum mentioned in the previous section.
The joint two formulas (35) and (36) on almost sure convergence show that formula (31) holds for the situation of k=1, and formula (32) for the situation of k=1 is also true by Kronecker’s Lemma.
Finally, the proof of the other situation ofk>1 is similar to Theorem 1, so it is omitted.
Here Theorem3 has been proven regarding the strong convergence of weighted sums. We proved the similar conclusions, but it was from a different viewpoint compared to previous research.Remark 2.
Both Theorem3 and Theorem 2.1 in literature [25] give the sufficient conditions for strong convergence of weighted sums of Markov chains, which is unlike the preconditions in these two theorems. Furthermore, the results of the strong convergence of weighted sums can be demonstrated under weaker sufficient conditions compared with Theorem 1 in literature [26].Proof of Corollary 3.
Firstly, ifV satisfies conditions (c), (d), and (e), for any k≥1, then both (31) and (32) hold. Now let us show this proof process as follows.
It is only necessary to verify that condition (a) and condition (b) of Theorem3 are immediately available. As we have the inequality(38)∑m=1∞PFmXm,ξm>cm≤C∑m=1∞PV>cm≤CENV,we can derive the following inequality:(39)∑m=1∞cm−1EFmXm,ξmIFmXm,ξm>cm≤C∑m=1∞1cm∫cm∞PFmXm,ξm>tdt≤C∑m=1∞1cmcmPFmXm,ξm>cm+∫cm∞PFmXm,ξm>tdt≤C∑m=1∞PV>cm+∑m=1∞∫1∞PV>tcmdt≤CENV+∫1∞ENVtdt,and, from conditions (c) and (d) of this corollary, we can see that condition (a) of Theorem 3 holds.
By Lemma2, we have the inequality(40)∑m=1∞cm−pEFmXm,ξmpIFmXm,ξm≤cm≤C∑m=1∞PV>cm+C∑m=1∞cm−pEVpIV≤cm≤CENV+C∑m=1∞cm−pEVpIV≤cm,as well as the inequality(41)∑m=1∞cm−pEVpIV≤cm≤C∑m=1∞cm−p∫0cmtp−1PV>tdt=C∫0∞tp−1PV>t∑m:cm>tcm−pdt≤C∫0∞tp−1PV>t∫t∞Nyyp+1dydt.
The last inequality above is established based on the facts that(42)∑m:cm>tcm−p=lims⟶∞∑m:t<cm<scm−p=lims⟶∞∫tsy−pdNy=lims⟶∞s−pNs−t−pNt+p∫t<y≤sNyyp+1dy;s−pNs≤p∫s∞Nyyp+1dy⟶0,s⟶∞.
Then, by the combination of formula (40), formula (41), condition (c), and condition (e), we know that condition (b) of Theorem 3 holds. Therefore, the proof is completed when conditions (c), (d), and (e) are satisfied.
Secondly, ifV satisfies conditions (c), (d), and (f), for any k≥1, we can get the same conclusions. The proof process is as follows.
Following the proof of Corollary3 by using formula (40), we only need to prove(43)∑m=1∞cm−pEVpIV≤cm<∞.
In fact, given some particular valuesd0=0, dn=max1≤m≤ncm, combining the two conditions of (c) and (f), we have(44)∑m=1∞cm−pEVpIV≤cm≤∑m=1∞cm−pEVpIV≤dm=∑m=1∞∑j=1mcm−pEVpIdj−1<V≤dj=∑j=1∞EVpIdj−1<V≤dj∑m=j∞cm−p≤∑j=1∞Pdj−1<V≤djdjp∑m=j∞cm−p≤C∑j=1∞jPdj−1<V≤dj=C∑j=1∞∑m=1jPdj−1<V≤dj≤C∑m=1∞∑j=m∞Pdj−1<V≤dj=C∑m=1∞PV>dm−1≤C∑m=1∞PV>cm+1<∞.
Therefore, the proof is completed when conditions (c), (d), and (f) are satisfied.
Thirdly, ifV satisfies conditions (c), (d), and (g), for any k≥1, we can also have formulas (31) and (32). The proof process is as follows.
Givens=mpt, we use the similar proof method of the first case by condition (g); then we have(45)∑m=1∞cm−pEVpIV≤cm≤C∑m=0∞1cmp∫0cmpPVp>sds≤∫01ENVt1/pdt<∞.
Therefore, the proof is completed when conditions (c), (d), and (g) are satisfied.Remark 3.
The sufficient conditions of Corollary3 are similar to those of Theorem 2.1 and Theorem 2.2 in literature [25], which shows that the conditions are applicable to the single infinite environment or double infinite environment.


## 5. Conclusions

From the above discussion, this study confirms that a series of limit theorems for the function of Markov chains in the environment of single infinite Markovian systems are proven. Furthermore, consistent with the literature, the study has found the sufficient conditions for convergence. Moreover, the results of strong law of large numbers and strong convergence of the weighted sums in the infinite environment are given by constructing martingale differential sequence. The finding of preconditions used in this research is broadly different from the previous observations. Although there are similarities between conclusions, they can be expressed by more sufficient conditions of the limit theorems, which are simpler than the analytical techniques proposed earlier. For the further research work, it may be extended to the limit theory related to the function of Markov chain in the environment of double infinite Markovian systems on finite state space.



---

*Source: 8175723-2020-05-05.xml*