# Uniqueness of Maximum Likelihood Estimators for a Backup System in a Condition-Based Maintenance

Authors: Qihong Duan; Ying Wei; Xiang Chen
Journal: International Journal of Quality, Statistics, and Reliability
 (2012)

---

## Abstract

A parameter estimation problem for a backup system in a condition-based maintenance is considered. We model a backup system by a hidden, three-state continuous time Markov process. Data are obtained through condition monitoring at discrete time points. Maximum likelihood estimates of the model parameters are obtained using the EM algorithm. We establish conditions under which there is no more than one limitation in the parameter space for any sequence derived by the EM algorithm.


---

## Body

## 1. Introduction

Suppose a backup system is represented by a continuous time homogeneous Markov chainX={Xt:t≥0} with a state space S={0,1,2}. States 0, 1, and 2 are the healthy state, unhealthy state, and the failure state, respectively. Assume that the system is in a healthy state at time 0, and the transition rate matrix is given by
(1)Λ=(-1/θ1,1/θ1,00,-1/θ2,1/θ20,0,0),
where θi∈(α,+∞) for i=1,2 are unknown. Here α>0 is a known extreme edge of the parameters. Suppose the system is observed at time points 0,Δ,2Δ,…, where [kΔ,(k+1)Δ] is a benchmark interval. While the system is failed at an inspection, a new system replaces it. Let two processes Y={Y(k):k=1,…,n}, and let R={R(k):k=1,…,n} be a record of the system. Here R(k)=1 if the system is failed during (k-1,k), and R(k)=0 otherwise. And Y(k)=XkΔ. As a path of X is a stepped right-continuous function, Y(k)=0 when a replacement occurs at time kΔ. Moreover, we set Y(0)=0 for convenience. The process R represents the replacement of the system, and the process Y represents observable information of the system collected through condition monitoring.The maximum likelihood estimates (MLE) of the model parameters for such models have been studied by [1, 2]. As the stochastic processes, Y,R are not Markov processes and the sample path of X is not observable, the likelihood function of incomplete data (Y,R) is complex. Hence, it is difficult to obtain directly the MLE θ^=argmaxθ∈ΘlnL(θ∣Y,R), where θ=(θ1,θ2) and Θ={(x,y):x,y>β}. Here β>0 is a prearranged constant. Both [1, 2] suggest the EM algorithm (see e.g., [3, 4]). Let θ(0) be initial values of the unknown parameters. The EM algorithm works as follows.TheE step. For n>0, compute the following pseudo-likelihood function:
(2)Q(θ′∣θ(n))=Eθ(n)(lnL(θ′∣D)∣Y,R).
Here D is the complete data set of the process X. The forms of the complete data set may be different for different purpose. For example, the forms are different in [1, 2].TheM step. Choose θ(n+1)=M~(θ(n)). Here
(3)M~(θ)=argsupθ′∈ΘQ(θ′∣θ).
The E and M steps are repeated. According to the theory of EM algorithms (Theorem  1 in [3]), L(θ(n+1)∣Y,R)≥L(θ(n)∣Y,R) for any given initial value θ(0) and n=0,1,…. It is clear that if an MLE θ^ in Θ is one of these fixed-points when it exists.In this paper, we consider the uniqueness of the MLEθ^. As the likelihood function lnL(θ∣Y,R) of incomplete data is complex, we do not follow the classical method by which the uniqueness of a MLE is demonstrated by establishing the global concavity of the log-likelihood function (see e.g., [5–7].) Alternatively, we investigate conditions under which the operator M~ is a contraction. The conditions ensure that the MLE is unique if it exists. Moreover, the conditions implies that there is not more than one limitations in Θ for different sequences derived by the EM algorithm. For the complete data set we present in the next section, we have the following main theorem of this paper.Theorem 1.
There is not more than one fixed-point of the operatorM~ in Θ provided that ∆<(2/2)α and the record {Y,R} has at least one replacement.


## 2. Complete Data Set

To establish the expression of operatorsQ, we present our construction of the path of X and the complete data set D of the process X. Suppose that random variables Tki, i=1,2, k=0,1,… are independent, Tki has an exponential distribution, and ETki=θi/Δ. As every state of X has an exponential duration distribution, we may construct a path of X through the approach introduced by Theorem  5.4 in [8]. The path of X restricted on t∈[0,Δ) has the following form:
(4)Xt10≤t<Δ=jforΔ∑i=1jT1i≤t<Δ(∑i=1j+1T1i∧1),j=0,1,
and Xt10≤t≤Δ=2 for Δ(T11+T12)≤t<Δ. If m=limt↑kΔXt≠2 for k≥1, we can construct the following path of X restricted on t∈[kΔ,(k+1). Consider
(5)Xt1k≤t<k+1={j,Δ∑i=mjTki≤t-kΔ<Δ(∑i=mj+1Tki∧1),j≠2,2,Δ∑i=m2Tki≤t-kΔ<Δ.
If limt↑kΔXt=2, that is, the system is failed during (kΔ,(k+1)Δ), a new system replaces it at time (k+1)Δ and a path of X restricted on t∈[kΔ,(k+1)Δ) is presented by setting m=1 in (5). In this paper, the complete data set D={Tki:i=1,2,k=0,1,…,n-1}.Our choice of the complete data setD ensures a simple form of the operator Q. The log-likelihood function has the following form:
(6)lnL(θ′∣D~)=2nlnΔ-n(lnθ1′+lnθ2′)-Δ∑k=0n-1(Tk1θ1′+Tk2θ2′).
And the Markovian property of the process X implies that
(7)Eθ(Tki∣Y)=Eθ(Tki∣Y(k-1),Y(k),R(k)),
where Y(0)=0.In Table1, we denote by ni the number of different values of the triple (Y(k-1),Y(k),R(k)). For example, n1 is the number of triple (Y(k-1),Y(k),R(k))=(0,0,0). It follows from (6) that
(8)Q(θ′∣θ)=2nlnΔ-n(lnθ1′+lnθ2′)-Δ∑i=15ni(Mi1θ1′+Mi2θ2′).Table 1
The number for values of(Y(k-1),Y(k),R(k)).
Values
(
0,0
,
0
)
(
0,1
,
0
)
(
1,1
,
0
)
(
0,0
,
1
)
(
1,0
,
1
)
Number
n
1
n
2
n
3
n
4
n
5Here the following forms ofMim for m=1,2 follows from (7) and Table 1. Consider
(9)M1m=Eθ(T1m∣T11≥1),(10)M2m=Eθ(T1m∣T12≥1),(11)M3m=Eθ(T1m∣T12<1),(12)M4m=Eθ(T1m∣T11<1,T11+T12≥1),(13)M5m=Eθ(T1m∣T11+T12<1).For any givenθ, it is obvious that the function Q~(λ1,λ2)=Q(1/λ1,1/λ2∣θ) is a concave function for 0<λ1,λ2<∞. Hence there is a unique vector θ'=M(θ) satisfying
(14)∂Q(θ′∣θ)∂θi′=0,i=1,2.
Moreover, it follows from the definition (3) that if M~(θ)∈Θ, M(θ)=M~(θ). Therefore, every fixed-point of M~ in Θ is a fixed-point of M and vice versa. In this paper, we will prove Theorem 1 through studying the number of fixed-points of M in Θ. Here we present the from of M(θ)=(M1(θ),M2(θ)) derived by (14). Consider
(15)Mm(θ)=Δn∑i=15niMim,m=1,2.


## 3. Two Lemmas

Lemma 2.
Consider the following:(16)|∂M4m∂θi|≤Δθi2,i=1,2,m=1,2.Proof.
Writingσ={(x1,x2)∣x1,x2≥0,x1+x2≤1} and
(17)V=∬σΔ2xmθ1θ2e(-Δ/θ1)x1-(Δ/θ2)x2dx1dx2,D=∬σΔ2θ1θ2e(-Δ2/θ1)x1-(Δ/θ2)x2dx1dx2,
we have M4m=V/D. Hence,
(18)∂M4m∂θi=M4m(V-1∂V∂θi-D-1∂D∂θi).
As0≤xm≤1 on σ, we have that 0≤V≤D and 0≤M4m≤1. Therefore,
(19)|∂M4m∂θi|≤|(V-1∂V∂θi-D-1∂D∂θi)|.
Moreover, we have
(20)∂V∂θi=-Vθi+Δ3θi2∬σxmxiθ1θ2e-(Δ/θ1)x1-(Δ/θ2)x2dx1dx2.
As 0≤xi≤1 on the region σ, it follows from the definition to V that there is 0<η1<1 such that
(21)∂V∂θi=-Vθi+η1ΔVθi2.
Similarly, there is η2∈(0,1) such that
(22)∂D∂θi=-Dθi+η2ΔDθi2.
The result follows from (19), (20), and (22).Lemma 3.
Let a functionA:R→R be A(0)=1/2 and A(x)=1/(1-ex)+1/x for x≠0. We have 0≤A'(x)≤-1/12 for any x∈R.Proof.
By a routine analysis, we may obtain thatA'(0)=-1/12 and for x≠0,
(23)A′(x)=(e-x/2-ex/2)-2-x-2.
Moreover, we may obtain that for x>0,
(24)A′′(x)=A2(x)x-3(ex-1)-3,
where A2(x)=(ex-1)3-x3ex(1+ex). We have
(25)A2′(x)=-exA3(x),
where A3(x)=3x2(1+ex)+x3(1+2ex)-6(ex-1)2. A routine analysis may confirm that A3(4)(x)=exA4(x), where A4(x)=96+96x+27x2+2x3-96ex. Then it follows from ex≤1+x+x2/2+x3/6 that A4(x)≤0. Moreover, as A3(0)=A3′(0)=A3′′(0)=A3(3)(0)=0, we have that for any x>0, there is ξ∈[0,x] such that
(26)A3(x)=124A3(4)(ξ)x4=124A4(ξ)eξx4≤0.
Then it follows from A2(0)=0 and (25) that A2(x)≥0. Hence, it follows from (24) that A′′(x)≥0 for any x>0. Therefore, for any x>0(27)-112=A′(0)≤A′(x)≤limx→+∞A′(x)=0.
The result follows from the above formula, and the fact that A'(x) is an even function.


## 4. Proof of the Main Theorem

We may derive thatM(θ) is a contraction by investigating the Jacobian matrix ∂(M1,M2)/∂(θ1,θ2).By a routine analysis, it follows from (12) that
(28)M21=1+A(Δθ1-Δθ2),M22=1+θ2Δ+A(Δθ1-Δθ2).
Hence, it follows from Lemma 3 that
(29)|∂M21∂θ2|=Δθ22|A'(Δθ1-Δθ2)|≤Δ12θ22.
Similarly, we have
(30)|∂M21∂θ1|≤Δ12θ12.
As Δ<α<θ2, it follows from (28) and Lemma 3 that
(31)|∂M22∂θ1|+|∂M22∂θ2|=1Δ.It follows from (11) that M52=A(θ2/Δ), and M51=θ1/Δ. Therefore,
(32)∂M52∂θ1=0,∂M51∂θ1=0,∂M51∂θ2=1Δ.
From the definition (11), we have that M52=A(Δ/θ2). Then, it follows from Lemma 3 that
(33)|∂M52∂θ2|≤Δ12θ22.It follows from (9) and (10) that M11=1+θ1/Δ, M12=θ2/Δ, and hence for m=1,2,
(34)|∂M1m∂θ1|+|∂M1m∂θ2|≤1Δ,|∂M3m∂θ1|+|∂M3m∂θ2|≤1Δ.WriteS=max{S1,S2}<1, where for i=1,2,
(35)Si=supθ∈Θ(|∂Mi∂θ1|+|∂Mi∂θ2|).
As ∂Mi/∂θj are continuous on the convex set Θ, it follows from Theorem  5.19 in [9] that for x=(x1,x2),y=(y1,y2)∈Θ, there exist z∈Θ such that
(36)M(x)-M(y)=∂(M1,M2)∂(θ1,θ2)|z(x-y).
Therefore, we have that ∥M(x)-M(y)∥≤S∥x-y∥, where ∥x∥=max{|x1|,|x2|}. Hence, there is not more than one fixed-point of M in Θ when S<1. As every fixed-point of M~ in Θ is a fixed-point of M, we will prove the theorem by indicating S<1.It follows from (29) to (35), (15), and Lemma 2 that
(37)S1≤n1n+n3n+n5n+n4n2Δ2α2+n2nΔ26α2,(38)S2≤n1n+n3n+n2n+n4n2Δ2α2+n5nΔ212α2.
Then it follows from Δ<2α/2 that S1≤1 and S2≤1.The record has at least one replacement. That is, there isk≥1 such that R(k)=1. As a new system replaces the old failed system at time kΔ, we have that Y(k)=0. Now the theorem will be accomplished in two cases.Case 1.
The caseY(k-1)=0. We have that n4≠0. It follows from (37) and (38) that
(39)S≤1-n4n(1-2Δ2α2)<1.Case 2.
The caseY(k-1)≠0. According to the condition-based maintenance policy, a failed system is replaced at an inspection. Hence Y(k-1)=1 follows from Y(k-1)≠0. As Y(k-1)=1, Y(k)=0, and R(k)=1, we have that n5≠0. Assume that the last replacement occurs at sΔ if any replacement occurs before (k-1)Δ. And we write s=0 when there is no replacement before (k-1)Δ. Now we study the sequence Y(s+1),…,Y(k-1), which consists of digits 0 and 1, corresponding healthy and unhealthy states of a system without replacement. It is obvious that there is s+1≤t≤k-1 such that Y(t)=1 and Y(t-1)=0. Then, n2≠0 follows from Y(t-1)=0, Y(t)=1 and R(t)=0. It follows from (37) and n2≠0 that
(40)S1≤1-n2n(1-Δ26α2)<1.
Moreover, it follows from (38) and n5≠0 that
(41)S2≤1-n5n(1-Δ212α2)<1.


## 5. Example and Discussion

We will apply the EM algorithm to a simulation dataset. Based on this example, we will show the efficiency and accuracy of the EM algorithm. Moreover, by this example, we will show some limitations and shortcomings of Theorem1. In this example, we make ensembles of 103 consecutive inspection of a simulating backup system defined by (1). The true parameters are θ1=10 and θ2=10/3, which is adopted from [2].We describe first the process of iterations described by the EM algorithm (2) and (3). For a given couple of initial value θ(0) in a given parameter space Θ, we may derive M(θ(0)) from (15). If M(θ(0))∈Θ, it is the unique solution to (3), and we have that θ(1)=M(θ(0)). If we are fortunate and can repeat the operation again and again, then we obtain a sequence θ(n)∈Θ,n=1,2,….It follows from the expression (8) of Q(θ′∣θ) that Q is continuous with respect to both θ' and θ. Similar to the discussion of the Theorem  1 in [4], we can prove that if the limitation of the sequence θ(n)∈Θ,n=1,2,… exists and is also in Θ, then the limitation is a fixed-point of the operator M~. Theorem 1 shows that the limitation is unique for all such sequences.In the first experiment, we run the EM algorithm for different initial values. In this experiment, we setβ=1 and the parameter space Θ={(x,y):x,y>β}. We run the algorithm for 64 couples of initial values which are chosen randomly from 2 to 12. For each couple of initial value, we run the algorithm for 200 iterations. Figure 1 draws the final estimations of the parameters for initial values. We can see that the algorithm converges to the same result for a great range of initial values. As Theorem 1 points out that the number of fixed points is not more than 1. So we can conclude that there is a unique fixed points, and hence it is the MLE of the model parameters on the parameter space Θ={(x,y):x,y>1}.Figure 1
Estimated parameters for different initial values.Sometimes, the above procedure ofθ(n),n=1,2,… must stop without the output of the estimated parameters. In general, for a θ(n)∈Θ, if M(θ(n)) derived from (15) is not an element of Θ, then M~(θ(n))≠M(θ(n)). For this case, the solution M~(θ(n)) to (3) is on the boundary of Θ. As we do not obtain the explicit expression M~(θ(n)) for this case, the procedure is aborted.In the following second experiment, for the same dataset of the first experiment, we run the EM algorithm for another parameter spaceΘ={(x,y):x,y>β'} with β'=12. We run the algorithm for 32 couples of initial values which are chosen randomly from 20 to 80. As we predict, the procedure is aborted for every couple of initial values. For these initial values, Figure 2 draws the maximal iteration numbers before the procedure is aborted.Figure 2
Maximal iteration number for different initial values.As we know, MLEθ^ in Θ, when it exists, is one of fixed-points of the operator M~. However, there may be other fixed-points of M~, such as stationary points of Q. Theorem 1 provides a sufficient condition under which such fixed-point does not exist. Our first experiment and Figure 1 illustrate this fact. However, in some cases, there is not a fixed point of M~ on the parameter space Θ. Such a phenomenon occurs when we set a wrong parameter space as in the second experiment.


## 6. Conclusion

A parameter estimation problem for a backup system has been considered. We established an EM algorithm, which can be used to iteratively determine the maximum likelihood estimators given observations of the system at discrete time points. It has been found that for any initial values, the sequence derived by the EM algorithm converges to a unique point when the limitation belongs to the specified parameter space.



---

*Source: 675830-2012-11-28.xml*