# An Automatic Partition Time-Varying Markov Model for Reliability Evaluation
## 2 Materials and Methods
### The Automatic Partition Time-Varying Markov Model for Reliability Evaluation
#### 2.2.2 The Time-Varying Markov Model  
The automatic segmentation method provides a scientific basis for the states number selection of the Markov process. Moreover, we argue that the transition probability between states is not only related to the state that the device stays, but also related to the sojourn time that the equipment stays in the present state. In practical situations, the longer a device stays in a state, the greater the probability of its transition from that state to another states. That is to say, the state sojourn time affects the state transition probability to a certain extent. Therefore, the state transition matrix (Sojourn time-based state transition probabilities) related to the state sojourn time is introduced to establish a time-varying Markov process for real-time reliability evaluation. The proposed Markov process is noted as \(M=(\pi,A,ST)\), where, \(\pi\) is the initial state probability distribution \(\pi=\{\pi_{i}\},1\leq i\leq N\) and \(\pi_{i}=P[s_{1}=i],1\leq i\leq N\); \(N\) is the state number, which is determined by the automatic partition method mentioned above; \(A\) is the time-varying state transition matrix, \(A=\big{\{}a_{ij}(st_{t}^{i})\big{\}},1\leq i,j\leq N,1\leq st_{t}^{i}\leq ST^{i}\), and \(a_{ij}(st_{t}^{i})\) represents the state transition probability of the device transitioning from state \(i\) to state \(j\) when the device stays in state \(i\) at time \(t\), \(\sum\limits_{j=1}^{N}a_{ij}(st_{t}^{i})=1,1\leq st_{t}^{i}\leq ST^{i},1\leq i,j\leq N\). \(ST^{i}\) is the maximum sojourn time in state \(i\), and \(ST\) is the state sojourn time distribution, which is assumed to obey a Gaussian distribution. Firstly, according to the sample set, the sojourn time distribution of each state is determined, and then the model parameters are estimated by the improved Baum-Welch algorithm. (1) When the model parameters \(M=(\pi,A,ST)\) are known, the probability that the device stays in state \(i\) for a sojourn time, \(T^{*}\), at time \(t\) is \(\alpha_{t}(i,T_{t}^{i*})=P(q_{t}=i,st_{t}^{i}=T_{t}^{i*}|M)\). When \(t=1\), the initial forward probability is  
\[\alpha_{1}(1,st_{1}^{i})=1 \tag{22}\]  
\[\alpha_{1}(j,1)=\alpha_{1}(1,st_{1}^{i})a_{1j}(st_{1}^{i}) \tag{23}\]  
\[\alpha_{1}(N,st_{1}^{N})=\sum\limits_{i=2}^{N}\alpha_{1}(i,st_{1}^{i}=1)P(st_{ 1}^{i}=1)a_{iN}(st_{1}^{i}=1) \tag{24}\]  
When \(t=2,3,\ldots,T\), the forward probability recursion formula is  
\[\alpha_{t}(j,1)=\sum\limits_{ \begin{array}{c}i=2\\ i\
eq j\end{array} }^{N-1}\sum\limits_{st_{t}^{i}=1}^{ST^{i}}\alpha_{t-1}(i,st_{ t}^{i})P(st_{t}^{i})a_{ij}(st_{t}^{i}) \tag{25}\]  
\[\alpha_{t}(j,d)=\alpha_{t-1}(j,st_{t}^{i}-1) \tag{26}\]  
\[\alpha_{t}(N,st_{t}^{N})=\sum\limits_{i=2}^{N-1}\sum\limits_{st_{t}^{i}=1}^{ ST^{i}}\alpha_{t}(i,st_{t}^{i})P(st_{t}^{i})a_{iN}(st_{t}^{i}) \tag{27}\]  
(2) At time \(t\), when the equipment stays in state \(i\) for a sojourn time \(st_{t}^{i}\), the backward probability of producing the state sequence in time \(T-t\) is \(\beta_{t}(i,st_{t}^{i})=P(s_{t}=i,st_{t}^{i}|M)\). When \(t=T\), the initial backward probability formula is  
\[\beta_{T}(N,st_{T}^{N})=1 \tag{28}\]  
\[\beta_{T}(i,st_{t}^{i})=P(st_{t}^{i})a_{iN}(st_{t}^{i}) \tag{29}\]  
\[\beta_{T}(1,st_{1}^{i})=\sum\limits_{j=2}^{N-1}\alpha_{1j}(st_{1}^{i})\beta_{ T}(j,st_{1}^{i}) \tag{30}\]  
When \(0<t<T\), the backward probability recursion formula is  
\[\beta_{t}(i,st_{t}^{i})=P(st_{t}^{i})\sum\limits_{ \begin{array}{c}j=2\\ j\
eq i\end{array}