# Online Health Management for Complex Nonlinear Systems Based on Hidden Semi-Markov Model Using Sequential Monte Carlo Methods

Authors: Qinming Liu; Ming Dong
Journal: Mathematical Problems in Engineering
 (2012)

---

## Abstract

Health management for a complex nonlinear system is becoming more important for condition-based maintenance and minimizing the related risks and costs over its entire life. However, a complex nonlinear system often operates under dynamically operational and environmental conditions, and it subjects to high levels of uncertainty and unpredictability so that effective methods for online health management are still few now. This paper combines hidden semi-Markov model (HSMM) with sequential Monte Carlo (SMC) methods. HSMM is used to obtain the transition probabilities among health states and health state durations of a complex nonlinear system, while the SMC method is adopted to decrease the computational and space complexity, and describe the probability relationships between multiple health states and monitored observations of a complex nonlinear system. This paper proposes a novel method of multisteps ahead health recognition based on joint probability distribution for health management of a complex nonlinear system. Moreover, a new online health prognostic method is developed. A real case study is used to demonstrate the implementation and potential applications of the proposed methods for online health management of complex nonlinear systems.


---

## Body

## 1. Introduction

With the rapid development of modern economy, the manufacturing industry increasingly needs to operate equipment at high reliability, low environmental risks, and human safety. The technological development has resulted in increasing complexity in both industrial machinery and production systems. It is difficult or almost impossible to identify and predict failure conditions in a timely manner. In a complex nonlinear system, high maintenance cost and uncertain downtime are often caused by system’s failures. Health management for a complex nonlinear system has a significant impact on the profitability of a business. So system’s health diagnosis and prognosis implementing CBM (condition-based maintenance) become a basic and desirable requirement in many application domains, where safety, reliability, and availability of systems are considered critically [1]. Health management involves evaluating the current system condition, observing the future system condition and predicting the system residual useful life (RUL) before the failures.In the system health management framework, health prognosis is a complex process and it is particularly difficult when the system under study is operating in the real time environment. Traditional linear methods such as ARIMA (autoregressive integrated moving average) cannot precisely predict the system health. For this reason, the study on online health management has attracted much attention. Some prognostic models have been introduced and developed with various degrees of success. Usually, three categories of nonlinear prognostic methods can be classified to obtain system health prognosis, including physical models, data-driven models, and model-driven models.With intelligent monitoring systems, physical models are useful to account for different operating conditions. They integrate physical features of a system for monitoring. Therefore, the functional mappings between the drifting parameters and the selected prognostic features can be established. Kacprzynski et al. [2] proposed a helicopter gear prognosis method that fused physics of failure modeling and relevant diagnostic information. Oppenheimer and Loparo [3] applied a physical model to estimate residual useful lifetime based on the crack growth law. These methods applied physical modeling and advanced parametric identification techniques, along with failure detection and prognosis algorithm for estimating the time to failure of complex nonlinear systems. The limitations of these models are their higher costs and lower accuracy. Furthermore, it is very difficult to build a good physical model.Data-driven methods use the monitored data to predict the health of complex nonlinear systems. Li [4] gave a tutorial review about fractal time series that was substantially different from the conventional time series in its statistic properties such as heavy-tailed probability distribution functions and slowly decayed autocorrelation functions. M. Li and J. Y. Li [5] addressed the particularity of the predictability of long-range dependence series and presented a generalized mean square error in the domain of generalized functions to prove the existence of long-range dependence series prediction. Li et al. [6] introduced the concept of one-dimensional random functions with long-range dependence based on a specific class of processes called the Cauchy-class process and presented the power spectrum density function of the Cauchy-class process in the closed form. Li and Zhao [7] addressed the power laws related to some physical systems and discussed that power-law-type data may be governed by stochastically differential equations of fractional order. Chen et al. [8] proposed a novel kernel machine-based rank-lifting regularized discriminant analysis method for nonlinear problems. However, the slow convergence and local minimum value are main drawbacks of these models and the computational explosion problems will occur when the number of observations increases dramatically. All disadvantages limit these models in the applications of online equipment health management.Model-based prognostic methods rely on the use of a mathematical model to represent the degradation behavior of the system. A main nonlinear time series method of model-driven methods called hidden Markov model has been quite effective for health prognosis. Zhou et al. [9] applied a new method for real time failure prognosis, by combining the capabilities of the HMM and the belief rule base. However, only ordinary HMM techniques are adopted and the inherent limitations within HMM still exist in their models. By allowing one state to generate a sequence of observations and have an explicit distribution for the duration of states, the HMM is generalized to HSMM. The HSMM has been an active research topic since the late 1980s. It has attracted considerable research attention in some fields [10–13]. Dong and He [14] presented an integrated framework based on HSMM for multisensors equipment diagnosis and prognosis. And in [15], they proposed a segmental hidden semi-Markov model for prognosis and diagnosis of the equipment. The HSMM has very rich mathematical structures and can form the solid theoretical foundation for use. Thus, HSMM has been effective for the health management of systems. However, the major drawback of HSMM is that recognizing and training processes of HSMM are often time consuming which is disadvantageous when real time prognosis is required. So HSMM is only applied to the offline equipment health prediction. The improvement for traditional HSMM is needed for online health management of complex nonlinear systems.The SMC method could decrease the computational and space complexity. And if the mathematical model or statistical model of a system is known, the filter based methods including SMC method can predict the health by estimating states or parameters of systems. The SMC method can also recognize the hidden failures of complex nonlinear systems. Unfortunately, when the mathematical models or the statistical models of systems are difficult to obtain, the SMC method is not applied to predict the health of systems. However, HSMM has very rich mathematical structures. So we combine the SMC method with HSMM in this paper. HSMM is used to obtain the transition probabilities among health states and health state durations of a complex nonlinear system, while the SMC method is adopted to decrease the computational and space complexity and describe the probability relationships between multiple health states and monitored observations of a complex nonlinear system. This paper proposes a novel multistep ahead recognition algorithm based on the joint probability distribution for health management of complex nonlinear systems. The proposed method has a better mathematical structure and better performance for real time prognosis. It can also decrease the computational complexity for health prognosis. Moreover, it overcomes the offline only constraint of the HSMM Viterbi algorithm and can recognize the health states in real time. Thus, it is effective for online health management of complex nonlinear systems.This paper aims to develop a new method for online health management of complex nonlinear systems. The paper is organized as follows. In Sections2 and 3, the theories of HSMM and SMC methods are introduced. Section 4 proposes a joint multistep ahead health recognition algorithm based on HSMM using SMC methods and develops the corresponding online health prognostic algorithm for complex nonlinear systems. In Section 5, a case study for online health management of a complex nonlinear system is provided. Finally, conclusions are drawn in Section 6.


## 2. Hidden Semi-Markov Model

A HSMM is constructed by adding the state duration into the well-defined HMM structures. Unlike a state in a standard HMM, a state in HSMM generates a segment of observations, as opposed to a single observation in the HMM. The states in a segmental hidden semi-Markov model are called macrostates (i.e., segments). Each macrostate consists of several single states, which are called microstates [15]. Suppose that a macrostate sequence has n segments and let qi be the time index of the endpoint of the ith segment (0≤i≤n-1). The segments are illustrated in Figure 1.Figure 1
Mapping of microstates and macrostates for a general HSMM.For theith macrostate, the observations are oqi-1+1,…,oqi, respectively, and they have the same macrostate labels as follows:(2.1)sqi-1+1=sqi-2+1=⋯=sqi=hi.In this paper, the monitored observations are continuous signals. In order to deal with the continuous observations, instead of the discrete-state situation, the continuous-state situation is adopted. The HSMM can establish a rational relationship between the states and observations. For some nonlinear problems, the state space model of HSMM can be described as follows(2.2)xn~g(xn∣xn-1,Λ),yn~h(yn∣xn,Λ).Based on the state space model, a standard HSMM can be established as follows:(2.3)xn~p(xn∣xn-1,A),yn~h(yn∣xn,B),di~N(di∣ui,σi),0≤d≤D.For convenience, the above equation can be rewritten as follows:(2.4)xn~p(xn∣xn-1),yn~q(yn∣xn),di~N(di∣ui,σi),0≤d≤D,
where Λ is the parameter set of HSMM in the state space model. di is the duration of the state i and D is the maximum state duration. ui and σi are the duration mean and variance of state i, respectively. xn∈S is a hidden state at time n  (S={s1,s2,…,sN} is the hidden state set with N elements). xn~gn(xn∣xn-1,A) is the system model and depends on the state transition probability A={aij}, (aij=P(xt=sj∣xt-1=si)). yn~hn(yn∣xn-1,B) is the observation model (yn∈O) and O={o1,o2,…,oT} is the observation sequence with T elements. B={bi(o),1≤i≤N} is the conditional probability distribution of observation (bi(o)=p(yn∣xn=si)).Besides, there is an initial state distributionπ(πi=p(x1=si),1≤i≤N). It can be seen that an HSMM can be described by λ(λ={A,B,D,π}). In real applications, there are three basic problems associated with an HSMM.(1)
Evaluation (Also Called Classification). Given the observation sequence O=o1o2⋯oT, and an HSMM λ, what is the probability of the observation sequence given the model (i.e., P(O∣λ))? The solution of this problem is obtained by using the forward-backward algorithm.(2)
Decoding (Also Called Recognition). Given the observation sequence O=o1o2⋯oT, and an HSMM λ, what sequence of hidden states S=(s1,s2,…,sN) most probably generates the given sequence of observations? This problem is solved by using the Viterbi algorithm. Here, a novel algorithm is used to solve this problem.(3)
Learning (Also Called Training). How to adjust the model parameters λ={A,B,D,π} to maximize P(O∣λ)? This problem is solved by using the Baum-Welch algorithm.Different algorithms have been developed for the above three problems. The details of three problems and the corresponding algorithms can be found in Rabiner [16] and Dong and He [15].


## 3. Sequential Monte Carlo Methods

Sequential Monte Carlo method has been introduced in the 1960s, and it is a significant and powerful methodology for coping with difficult nonlinear problems. The key idea of the SMC method is to represent the posterior density function by a set of random samples with associated weights and compute recognition probability based on these samples.Let{xn-1i,wn-1i}i=1Ns denote the sample from importance function, and let wn-1i denote the weights of particles (xn-1i(i=1,2,…,Ns)), and Ns is the number of particles for the computation. Then, p(xn-1∣y0:n-1) can be approximated as follows:
(3.1)p(xn-1∣y0:n-1)≈∑i=1Nwn-1iδ(xn-1-xn-1i),
where wn-1i∝p(x0:n-1i∣y0:n-1)/q(x0:n-1i∣y0:n-1),∑i=1Nwn-1i=1, q(x0:n-1i∣y0:n-1) is the importance probability density, and the δ(·) is Dirac delta function [17].With the increase of new observation data, in order to obtain the new sample(x0:ni~q(x0:n∣y0:n)), the new data (xni~q(xn∣x0:n-1i,y0:n)) can be added into the old sample (x0:n-1i~q(x0:n-1∣y0:n-1)). And, the weights of particles xni can be obtained as follows:
(3.2)w~ni=p(xni∣xn-1i)p(yn∣xni)q(xni∣x0:n-1i,y0:n)wn-1i.So the posterior probability density ofxn can be written as follows:
(3.3)p(xn∣y0:n)=∑i=1Nwniδ(xn-xni).However, the choice of the importance density function is critical for the performance of the SMC method. In the paper, the a priori probability density function is chosen as the importance density function and it is written as follows:(3.4)q(xni∣x0:n-1i,y0:n)=p(xki∣xk-1i).From (3.2) and (3.4), we can obtain(3.5)w~ni=p(yn∣xn(i))wn-1(i),wn(i)=w~ni∑i=1Nw~ni.In the sequential importance sampling implementation of SMC methods, since most particles have negligible weights after a few iterations, the main difficulty is the degeneracy problem [18]. Several researchers have developed methods to overcome the degeneracy in the particle population N^eff, which is an estimation of the effective sample size Neff  [18, 19]. Neff and N^eff can be approximated as follows:
(3.6)Neff=N*(1+var(wn(i))-1),N^eff=(∑i=1Ns(wn(i))2)-1.In (3.6), if the valid sample Neff becomes smaller, the degeneracy of SMC methods becomes more serious. But, whenever N^eff is smaller than Nthres, a resampling algorithm will be applied to eliminate the particles with small weights, and it also reserves the computational impacts on the optimal results in those having large weights [18].After the resampling procedure, the new particle population({x~0:n(i)}i=1,…,N) is an independent and identically distributed sample of the empirical distribution; thus, the weights are reset to w~ni=N-1 and we have
(3.7)p(xn∣y0:n)=∑i=1Nwniδ(xn-xni)=1N∑i=1Nδ(xn-xni).


## 4. Online Prognosis Model for Complex Nonlinear Systems

The health management for complex nonlinear systems uses both state transition relations and duration information to predict the evolving trend or estimate the remaining useful life. In the previous sections, the HSMM given by (2.4) is considered and its parameters are obtained through the training algorithm in which state duration probabilities are estimated on the trellis of observations and states. In this section, the framework for online health management of a system is described.Firstly, a novel algorithm for online health recognition of a system based on HSMM is proposed by using SMC method and its computational framework is given. Then, the collected sensor data is used as the input of the proposed model. With the state information obtained from the proposed method, an online health prognostic model for a system is proposed. The intention is to apply SMC method to make joint multisteps ahead health recognition with all available information. The principle of SMC-based HSMM involves two main phases, as shown in Figure2. For a complex nonlinear system, the learning phase will generate an appropriate model and an exploitation phase will recognize its current health condition and predict its RUL. In the first phase, which is achieved offline, the raw data provided by the sensors is processed for HSMM training. The second phase, which is an online one, exploits the trained models for recognizing the current system state (by using the following proposed method) and computing the corresponding RUL.Figure 2
Fault prognosis scheme of HSMM-based SMC method.In the following, the data processing algorithm is described first. Then, a novel HSMM-based algorithm for online health recognition of complex nonlinear systems using SMC methods is proposed, and the computational framework is given. Finally, with the state information obtained from the proposed method, the online health prognostic model for complex nonlinear systems is proposed.


### 4.1. Data Processing Algorithm

The data processing algorithm includes feature extraction using wavelets and vector quantization. The feature extraction is described first. Since the structures of each part are different, when a system fails, it often generates a large number of nonstationary signals. The performance of output signals in each frequency band is different. The wavelet analytical method can automatically decompose different frequency signals into different frequency bands, which can increase the time-frequency resolution and realize failure feature extraction. The failure feature extraction algorithm using wavelet can be described as follows.Step 1.
The vibration signals can be decomposed intoV-layers wavelet, and coefficients (Xj(j=1,2,…,2V)) of 2V frequency bands from low frequency to high frequency of the V-layers are extracted, respectively.Step 2.
Reconstruct the wavelet package coefficients and extract each frequency band signalSj(j=1,2,…,2V).Step 3.
The total energy of each frequency band signal is computed by the following equation:(4.1)Ej=∫|Sj(t)|2dt=∑k=1n|xjk|2,
where xjk(j=1,2,…,2V,k=1,2,…,n) is the amplitude of discrete points for reconstructing signals.Step 4.
Construct and normalize feature vector as follows:(4.2)E=(∑j=12V|Ej|2)1/2.
So the feature vectorF equals F′ (i.e., F′=(E1,E2,…,E2V)/E).The feature vector must be quantized before the wavelet fault feature vector is used for HSMM training. The self-organizing feature maps (SOMs) are often used for vector quantization. The SOMs simulates self-organizing feature maps function of the brain’s nervous system, it is a competitive learning network, and the learning process is unsupervised and self-organized [20].


### 4.2. Joint Multisteps Ahead Health Recognition for Systems Using SMC in HSMM

One purpose of the paper is to offer the appropriate methods for real time recognition of health states and RUL prognosis as a continuous function of time. A new HSMM-based SMC method is applied to address the recognition problems of HSMM. Currently, the primary concerns in HSMM are two aspects: one-step ahead health recognition of states by estimating the probability distributionp(xn∣y1:n-1) and multistep ahead health recognition of states by estimating the probability distribution p(xn+h∣y1:n-1) [18]. However, the recognizing trajectories of p(xn∣y1:n-1) and p(xn+h∣y1:n-1) may be incredible due to neglecting the consistency principle of HSMM. The consistency is very important and required for many cases. Therefore, a novel method of multistep ahead state health recognition based on joint probability distribution p(xn:n+h∣y1:n-1), which provides the most possible recognition for the recognizing trajectory, is proposed. In order to illustrate the superiority of the proposed method, the proposed joint multisteps ahead algorithm will be compared with the general multistep ahead algorithm and one-step ahead algorithm on the aspect of prognostic accuracy.In the following, the SMC method is used for joint multistep ahead health recognition given all available information up to timen-1. The joint multistep ahead health recognition method is capable of capturing the relationship between hidden states and observations of a system. By combining the capabilities of HSMM and SMC method, a new health recognition algorithm is proposed here to recognize the hidden failure online. And HSMM can provide rich mathematical structures for health management of complex nonlinear systems.This health recognition algorithm is related to the one-step ahead health recognition. The state probability density function of the one-step ahead health recognition can be obtained by a recursive manner based on Algorithm4.1. Suppose that the state pdf p(xn-1∣y1:n-1) can be approximated by {x0:n-1(i),wn-1i}i=1Ns, as it is shown as follows:
(4.3)p(xn-1∣y1:n-1)=∑i=1Nswn-1iδ(xn-1-xn-1i).The state probability densityp(x̂n∣y1:n-1) can be approximated with new samples evolved from {x0:n-1i,wn-1i}i=1Ns (the symbol ̂ represents the recognized value). The procedure of the one-step ahead health recognition is described as follows.Algorithm 4.1.
Online one-step ahead health recognition is as follows
Step 1.
Letx̂ni~p(x̂n∣xn-1i), then
(4.4)p(x̂n∣y1:n-1)≈∑i=1Nswn-1iδ(x̂n-x̂ni).Step 2.
Update the weightsŵni=p(yn∣x̂ni)wn-1i, then
(4.5)wni=ŵni∑i=1Nsŵni.Step 3.
Compute the state pdf as follows:(4.6)p(x̂n∣y1:n)≈∑i=1Nswniδ(x̂n-x̂ni).
Then, we extend to the joint multistep ahead health recognition. Based on (2.4) and the current state recognition Algorithm 4.1, p(xn-1∣y1:n-1) can be obtained. According to the conditional probability and chain rule of probability, we can obtain
(4.7)p(x̂n:n+h-1∣y1:n-1)=∏j=0h-1p(x̂n+j∣y1:n-1,x̂n:n+j-1).
In the above equation,p(x̂n+j∣y1:n-1,x̂n:n+j-1), j=0,1,…,h-1 is just one-step ahead health recognition probability and can be approximated by Algorithm 4.1 with {x̂n+hix0:n-1i,wn+h-1i}i=1Ns such that
(4.8)p(xn+j∣y1:n,xn:n+j-1)=∑i=1Nswn+hiδ(x̂n+h-x̂n+hi).
The jointh-step ahead health recognition can be approximated by the particles with corresponding weights:
(4.9){(x̂ni0,…,x̂n+h-1ih-1),(ŵn-1i0,…,ŵn+h-2ih-1)}i0,…,ih-1=1Ns.
And we have(4.10)p(x̂n:n+h-1∣y1:n-1)=∑i0=1Ns⋯∑ih-1=1Ns(ŵn-1i0,…,ŵn+h-2ih-1)δ(x̂n-x̂ni0,…,x̂n+h-1-x̂n+h-1ih-1).
So the overall online joint multistep ahead health recognition algorithm can be described as follows.Algorithm 4.2.
Online joint multistep (h-steps) ahead health recognition is as follows.Step 1.
Based on Step1 in Algorithm 4.1, we can obtain
(4.11)p(x̂n∣y1:n-1)≈∑i=1Nswn-1iδ(x̂n-x̂ni).Step 2.
Ifh=1, then go to Step 3, otherwise set k=1.Substep 1.
Use the prognosis value{x̂n+k-1i}i=1Ns to update weights {wn+k-2i}i=1Ns, according to wn+k-1i=p(ŷn+k-1i∣x̂n+k-1i)wn+k-2i, then we have
(4.12)p(x̂n+k-1∣y1:n-1)≈∑i=1Nswn+k-1iδ(x̂n+k-1-x̂n+k-1i).Substep 2.
Letx̂n+ki~p(x̂n+k∣x̂n+k-1i), then
(4.13)p(x̂n+k∣y1:n-1)≈∑i=1Nswn+k-1iδ(x̂n+k-x̂n+ki).Substep 3.
Letk=k+1. If k=h, then the joint h-step ahead prognosis can be obtained based on (4.10), and go to Step 3, otherwise return to Substep 1.Step 3.
When the online monitoring measurementyn is available, update the weights ŵni=p(yn∣x̂ni)wn-1i, and
(4.14)p(x̂n∣y1:n)≈∑i=1Nswniδ(x̂n-x̂ni).
Based on Algorithm4.2, for a given new sequence of continuously observed data, the state recognition probability of the current time can be obtained. Based on the state recognition probability, the RUL values of a complex nonlinear system at the current time can be predicted based on the prognostic algorithm.Obviously, the relationship between joint multistep ahead health recognition and simple multistep ahead recognition is given as follows:(4.15)p(x̂n+h-1∣y1:n-1)=∑x̂n⋯∑x̂n+h-1p(x̂n:n+h-1∣y1:n-1).In the above equation,p(x̂n+h-1∣y1:n-1) is the marginal distribution of p(x̂n:n+h-1∣y1:n-1). The simple multistep ahead recognition is suitable for the situation where the health management of a system needs to be obtained at given time with lower accuracy. However, the joint multistep ahead health recognition algorithm is much better for online health management of a system due to the ability of keeping the Markov consistency in HSMM.After obtaining the state recognition probability of a system, the health state change point needs to be determined. The health state change point is defined as the point at which the system changes from health statexl to health state xl+1. Through the health state change point detection, the time from the system’s current condition to the health state change point can be estimated, as the health state change point corresponds to the switching from health state xl to health state xl+1 in the model and the determination of health state change point can be obtained in the following way: apply the joint multistep ahead health recognition algorithm online when new observations are obtained. If the value of p(Sl=xl+1∣y1:l-1) is increasing up to time l and becomes the maximum at time  l, then l will be the health state change point.For new observationsy1y2⋯yT and two system health states, the recognized state sequences of a system for computing RUL are shown in Figure 3. From Figure 3, it can be seen that each state sequence will provide the location recognition of the health state change point. Moreover, the probability of a system staying in the state based on Algorithm 4.2 can be recognized and the RUL values of a system can be computed.Figure 3
The recognized state sequences for a system having two health states.


### 4.3. Online Health Prognosis Model for Complex Nonlinear Systems

For the health prognosis of a system, its purpose is to predict the progression of a failure condition in order to estimate the RUL of a system. Assuming that the system will go through health statessi(i=1,2,…,n-1) before entering the failure state F(sn). And let D(si) denote the expected duration of a system at health state si. Once the system enters health state si, its RUL will equal the summation of the residual useful duration of a system staying at health state si and the residual useful duration of system staying in the future health states before failure. This type of prognostic structure is shown in Figure 4. Let D̂(si(l)) denote the residual duration of a system staying in health state si at the lth observation time point. Then, we can obtain
(4.16)D(si)=μ(si)+ρσ2(si),ρ=(lifetime-∑i=1Nμ(si))∑i=1Nσ2(si),D̂(si(l))=p(Sl=si∣y1:l)D(si),
where μ(si) is the mean of duration probability function pi(d) of state si and σ2(si) is the variance of duration probability function pi(d) of state si. l represents the step length. p(Sl=si∣y1:l) denotes the state probability of si at the lth observation time point based on observation y1:l.Figure 4
Prognostic structure based on HSMM using SMC method. Life time of a component=D(s1)+D(s2)+D(s3)+⋯+D(sN). D(si): duration of a component staying at state si. s1: health state 1 (baseline), s2: health state 2 (contamination 1), …, sN: health state N (failure), aij: transition probability.Based on (4.16), the RUL at the lth observation time point since a system stays at the health state si can be computed as follows:
(4.17)RUL(l)=D̂(si(l))+∑j=i+1n-1D(sj).By integrating (4.16) with (4.17), the RUL at the lth observation time point since a system enters health state si can be obtained.


### 4.4. Prognosis Algorithm

The proposed methods are used during the online phase in order to recognize health states and estimate the RUL values. In terms of the HSMM-based SMC method, the online health management procedure of a system can be described as follows.Step 1.
This step consists of learning the appropriate HSMM that best fits and represents the online observed sequence. Indeed, the observations are continuously fed into the set of learned (completely defined) models and a likelihood value is calculated in order to learn the appropriate model that will be used for the computation of the expected state duration and the initial state transition probability.Step 2.
This step is related to the recognition of the current state. Based on Algorithm4.2, the pdf of state si at the lth observation time since a system enters state si can be computed with the online monitored data using joint multistep ahead health recognition algorithm. And the health state change point of a system can also be obtained.Step 3.
Based on Step2, the residual duration of a system keeping the health state si at the lth observation time point can be computed based on (4.16).Step 4.
Based on (4.17), the RUL at the lth observation time point since a system enters health state si can be obtained.


## 4.1. Data Processing Algorithm

The data processing algorithm includes feature extraction using wavelets and vector quantization. The feature extraction is described first. Since the structures of each part are different, when a system fails, it often generates a large number of nonstationary signals. The performance of output signals in each frequency band is different. The wavelet analytical method can automatically decompose different frequency signals into different frequency bands, which can increase the time-frequency resolution and realize failure feature extraction. The failure feature extraction algorithm using wavelet can be described as follows.Step 1.
The vibration signals can be decomposed intoV-layers wavelet, and coefficients (Xj(j=1,2,…,2V)) of 2V frequency bands from low frequency to high frequency of the V-layers are extracted, respectively.Step 2.
Reconstruct the wavelet package coefficients and extract each frequency band signalSj(j=1,2,…,2V).Step 3.
The total energy of each frequency band signal is computed by the following equation:(4.1)Ej=∫|Sj(t)|2dt=∑k=1n|xjk|2,
where xjk(j=1,2,…,2V,k=1,2,…,n) is the amplitude of discrete points for reconstructing signals.Step 4.
Construct and normalize feature vector as follows:(4.2)E=(∑j=12V|Ej|2)1/2.
So the feature vectorF equals F′ (i.e., F′=(E1,E2,…,E2V)/E).The feature vector must be quantized before the wavelet fault feature vector is used for HSMM training. The self-organizing feature maps (SOMs) are often used for vector quantization. The SOMs simulates self-organizing feature maps function of the brain’s nervous system, it is a competitive learning network, and the learning process is unsupervised and self-organized [20].


## 4.2. Joint Multisteps Ahead Health Recognition for Systems Using SMC in HSMM

One purpose of the paper is to offer the appropriate methods for real time recognition of health states and RUL prognosis as a continuous function of time. A new HSMM-based SMC method is applied to address the recognition problems of HSMM. Currently, the primary concerns in HSMM are two aspects: one-step ahead health recognition of states by estimating the probability distributionp(xn∣y1:n-1) and multistep ahead health recognition of states by estimating the probability distribution p(xn+h∣y1:n-1) [18]. However, the recognizing trajectories of p(xn∣y1:n-1) and p(xn+h∣y1:n-1) may be incredible due to neglecting the consistency principle of HSMM. The consistency is very important and required for many cases. Therefore, a novel method of multistep ahead state health recognition based on joint probability distribution p(xn:n+h∣y1:n-1), which provides the most possible recognition for the recognizing trajectory, is proposed. In order to illustrate the superiority of the proposed method, the proposed joint multisteps ahead algorithm will be compared with the general multistep ahead algorithm and one-step ahead algorithm on the aspect of prognostic accuracy.In the following, the SMC method is used for joint multistep ahead health recognition given all available information up to timen-1. The joint multistep ahead health recognition method is capable of capturing the relationship between hidden states and observations of a system. By combining the capabilities of HSMM and SMC method, a new health recognition algorithm is proposed here to recognize the hidden failure online. And HSMM can provide rich mathematical structures for health management of complex nonlinear systems.This health recognition algorithm is related to the one-step ahead health recognition. The state probability density function of the one-step ahead health recognition can be obtained by a recursive manner based on Algorithm4.1. Suppose that the state pdf p(xn-1∣y1:n-1) can be approximated by {x0:n-1(i),wn-1i}i=1Ns, as it is shown as follows:
(4.3)p(xn-1∣y1:n-1)=∑i=1Nswn-1iδ(xn-1-xn-1i).The state probability densityp(x̂n∣y1:n-1) can be approximated with new samples evolved from {x0:n-1i,wn-1i}i=1Ns (the symbol ̂ represents the recognized value). The procedure of the one-step ahead health recognition is described as follows.Algorithm 4.1.
Online one-step ahead health recognition is as follows
Step 1.
Letx̂ni~p(x̂n∣xn-1i), then
(4.4)p(x̂n∣y1:n-1)≈∑i=1Nswn-1iδ(x̂n-x̂ni).Step 2.
Update the weightsŵni=p(yn∣x̂ni)wn-1i, then
(4.5)wni=ŵni∑i=1Nsŵni.Step 3.
Compute the state pdf as follows:(4.6)p(x̂n∣y1:n)≈∑i=1Nswniδ(x̂n-x̂ni).
Then, we extend to the joint multistep ahead health recognition. Based on (2.4) and the current state recognition Algorithm 4.1, p(xn-1∣y1:n-1) can be obtained. According to the conditional probability and chain rule of probability, we can obtain
(4.7)p(x̂n:n+h-1∣y1:n-1)=∏j=0h-1p(x̂n+j∣y1:n-1,x̂n:n+j-1).
In the above equation,p(x̂n+j∣y1:n-1,x̂n:n+j-1), j=0,1,…,h-1 is just one-step ahead health recognition probability and can be approximated by Algorithm 4.1 with {x̂n+hix0:n-1i,wn+h-1i}i=1Ns such that
(4.8)p(xn+j∣y1:n,xn:n+j-1)=∑i=1Nswn+hiδ(x̂n+h-x̂n+hi).
The jointh-step ahead health recognition can be approximated by the particles with corresponding weights:
(4.9){(x̂ni0,…,x̂n+h-1ih-1),(ŵn-1i0,…,ŵn+h-2ih-1)}i0,…,ih-1=1Ns.
And we have(4.10)p(x̂n:n+h-1∣y1:n-1)=∑i0=1Ns⋯∑ih-1=1Ns(ŵn-1i0,…,ŵn+h-2ih-1)δ(x̂n-x̂ni0,…,x̂n+h-1-x̂n+h-1ih-1).
So the overall online joint multistep ahead health recognition algorithm can be described as follows.Algorithm 4.2.
Online joint multistep (h-steps) ahead health recognition is as follows.Step 1.
Based on Step1 in Algorithm 4.1, we can obtain
(4.11)p(x̂n∣y1:n-1)≈∑i=1Nswn-1iδ(x̂n-x̂ni).Step 2.
Ifh=1, then go to Step 3, otherwise set k=1.Substep 1.
Use the prognosis value{x̂n+k-1i}i=1Ns to update weights {wn+k-2i}i=1Ns, according to wn+k-1i=p(ŷn+k-1i∣x̂n+k-1i)wn+k-2i, then we have
(4.12)p(x̂n+k-1∣y1:n-1)≈∑i=1Nswn+k-1iδ(x̂n+k-1-x̂n+k-1i).Substep 2.
Letx̂n+ki~p(x̂n+k∣x̂n+k-1i), then
(4.13)p(x̂n+k∣y1:n-1)≈∑i=1Nswn+k-1iδ(x̂n+k-x̂n+ki).Substep 3.
Letk=k+1. If k=h, then the joint h-step ahead prognosis can be obtained based on (4.10), and go to Step 3, otherwise return to Substep 1.Step 3.
When the online monitoring measurementyn is available, update the weights ŵni=p(yn∣x̂ni)wn-1i, and
(4.14)p(x̂n∣y1:n)≈∑i=1Nswniδ(x̂n-x̂ni).
Based on Algorithm4.2, for a given new sequence of continuously observed data, the state recognition probability of the current time can be obtained. Based on the state recognition probability, the RUL values of a complex nonlinear system at the current time can be predicted based on the prognostic algorithm.Obviously, the relationship between joint multistep ahead health recognition and simple multistep ahead recognition is given as follows:(4.15)p(x̂n+h-1∣y1:n-1)=∑x̂n⋯∑x̂n+h-1p(x̂n:n+h-1∣y1:n-1).In the above equation,p(x̂n+h-1∣y1:n-1) is the marginal distribution of p(x̂n:n+h-1∣y1:n-1). The simple multistep ahead recognition is suitable for the situation where the health management of a system needs to be obtained at given time with lower accuracy. However, the joint multistep ahead health recognition algorithm is much better for online health management of a system due to the ability of keeping the Markov consistency in HSMM.After obtaining the state recognition probability of a system, the health state change point needs to be determined. The health state change point is defined as the point at which the system changes from health statexl to health state xl+1. Through the health state change point detection, the time from the system’s current condition to the health state change point can be estimated, as the health state change point corresponds to the switching from health state xl to health state xl+1 in the model and the determination of health state change point can be obtained in the following way: apply the joint multistep ahead health recognition algorithm online when new observations are obtained. If the value of p(Sl=xl+1∣y1:l-1) is increasing up to time l and becomes the maximum at time  l, then l will be the health state change point.For new observationsy1y2⋯yT and two system health states, the recognized state sequences of a system for computing RUL are shown in Figure 3. From Figure 3, it can be seen that each state sequence will provide the location recognition of the health state change point. Moreover, the probability of a system staying in the state based on Algorithm 4.2 can be recognized and the RUL values of a system can be computed.Figure 3
The recognized state sequences for a system having two health states.


## 4.3. Online Health Prognosis Model for Complex Nonlinear Systems

For the health prognosis of a system, its purpose is to predict the progression of a failure condition in order to estimate the RUL of a system. Assuming that the system will go through health statessi(i=1,2,…,n-1) before entering the failure state F(sn). And let D(si) denote the expected duration of a system at health state si. Once the system enters health state si, its RUL will equal the summation of the residual useful duration of a system staying at health state si and the residual useful duration of system staying in the future health states before failure. This type of prognostic structure is shown in Figure 4. Let D̂(si(l)) denote the residual duration of a system staying in health state si at the lth observation time point. Then, we can obtain
(4.16)D(si)=μ(si)+ρσ2(si),ρ=(lifetime-∑i=1Nμ(si))∑i=1Nσ2(si),D̂(si(l))=p(Sl=si∣y1:l)D(si),
where μ(si) is the mean of duration probability function pi(d) of state si and σ2(si) is the variance of duration probability function pi(d) of state si. l represents the step length. p(Sl=si∣y1:l) denotes the state probability of si at the lth observation time point based on observation y1:l.Figure 4
Prognostic structure based on HSMM using SMC method. Life time of a component=D(s1)+D(s2)+D(s3)+⋯+D(sN). D(si): duration of a component staying at state si. s1: health state 1 (baseline), s2: health state 2 (contamination 1), …, sN: health state N (failure), aij: transition probability.Based on (4.16), the RUL at the lth observation time point since a system stays at the health state si can be computed as follows:
(4.17)RUL(l)=D̂(si(l))+∑j=i+1n-1D(sj).By integrating (4.16) with (4.17), the RUL at the lth observation time point since a system enters health state si can be obtained.


## 4.4. Prognosis Algorithm

The proposed methods are used during the online phase in order to recognize health states and estimate the RUL values. In terms of the HSMM-based SMC method, the online health management procedure of a system can be described as follows.Step 1.
This step consists of learning the appropriate HSMM that best fits and represents the online observed sequence. Indeed, the observations are continuously fed into the set of learned (completely defined) models and a likelihood value is calculated in order to learn the appropriate model that will be used for the computation of the expected state duration and the initial state transition probability.Step 2.
This step is related to the recognition of the current state. Based on Algorithm4.2, the pdf of state si at the lth observation time since a system enters state si can be computed with the online monitored data using joint multistep ahead health recognition algorithm. And the health state change point of a system can also be obtained.Step 3.
Based on Step2, the residual duration of a system keeping the health state si at the lth observation time point can be computed based on (4.16).Step 4.
Based on (4.17), the RUL at the lth observation time point since a system enters health state si can be obtained.


## 5. Case Study

To evaluate the performance of the HSMM-based SMC method for online health management of complex nonlinear systems, a real hydraulic pump health monitored application is used.In the case study, the long-term wear test experiments were conducted at a research laboratory facility. In the test experiments, three pumps (pump 6, pump 24, and pump 82) were worn by running them using oil-containing dust. In other words, it is a seeding fault experiment and the deterioration of the three pumps does not follow a nature process. Each pump experienced four states: baseline (normal state), contamination 1 (5 mg of 20-um dust injected into the oil reservoir), contamination 2 (10 mg of 20-um dust injected into the oil reservoir), and failure (15 mg of 20-um dust injected into the oil reservoir). The contamination stages in this hydraulic pump wear test case study correspond to different stages of flow loss in the pumps. As the flow rate of a pump clearly indicates pump’s health state, the contamination stages corresponding to different degrees of flow loss in a pump were defined as the health states of the pump in the test [15, 21].Vibration signals were collected from a pump accelerometer that was positioned parallel to the axis of swash plate swivel axis and data was continuously sampled. Figure5 shows the schematic diagram of the experimental setup. The pump used for testing in the experiments was a Back Hoe Loader: a 74 cc/rev variable displacement pump. The data was collected at a sample rate of 60 kHz with antialiasing filters from accelerometers designed to have a usable range of 10 kHz. In many cases, the most distinguished information is hidden in the frequency content of signals. So the time-frequency representation of signals is needed. In this case, the signals were processed using wavelet packet with Daubechies wavelet 10 (db10) and five decomposition levels as the db10 wavelet provide the most effective way to capture the fault information in the pump vibration data. The coefficients obtained by the wavelet packet decomposition were used as the inputs.Figure 5
Schematic diagram of the experimental setup.


### 5.1. Hidden States and Probability Density of Parameters

For prognosis, the health changing mechanisms of a system usually involve several degraded states. From normal to failure, a system goes through a series of different degraded states and the certain transition relation among states is existed. There are four hidden states in this case study which are defined as baseline, contamination 1, contamination 2, and failure, respectively. The state transition process for the experiment is described by Figure6.Figure 6
A HSMM scheme describing failure mechanism of a system.There should be three probability distributions which will be modeled by the following sections, including the probability of state transition, the probability of state duration, and the probability of observation.


#### 5.1.1. Probability of State Transitions

Assuming thatAij is the jth element of the ith row of the transition matrix A, then the transition probability can be computed as follows:
(5.1)Aij=((1-λ)mij+λA̅ij)∑i=1N((1-λ)mij+λA̅ij),i,j=1,2,3,4,
where the values of A̅ij come from the expert knowledge, mij denotes the number of transitions from state xj to xi (mi0 is the count of occurrences of the state xi). λ is used to control the weights between the expert knowledge and observation evidence.


#### 5.1.2. Probability of State Duration

The duration probability distributionpi(d) can be modeled using the half-normal distribution and expressed as follows:
(5.2)pi(d)=1σi2πexp⁡(-12(d-μiσi)),i=1,2,3,4,
where σ2(si) is the variance for the ith state and μ(si) is the mean for the ith state.


#### 5.1.3. Probability of Observations

For the observation sequences, two Gaussian mixtures for each state are chosen. Thus, the following probability density can be obtained:(5.3)bi(ot)=∑k=12ωikN(ot,uik,Uik),i=1,2,3,4,k=1,2,
where N(ot,uik,Uik2) is the kth mixture Gaussian distribution at state si. uik, Uik2, and ωik are the mean, variance, and mixture weight of the kth Gaussian distribution at state si, respectively. However, the weights must satisfy the following constraints:(5.4)∑k=12wik=1wik≥0,i=1,2,3,4,k=1,2.Based on Sections2, 5.1.1 and 5.1.2, the parameters of HSMM can be described as follows:
(5.5){A,B,D,π}={aij,uik,Uik2,wik,μi,σi,πi,1≤i,j≤4,1≤k≤2}.


### 5.2. Results and Discussion

For the health prognosis, the lifetime training data from three hydraulic pumps is used. An HSMM with four health states (baseline, contamination 1, contamination 2, and failure) can be trained. The initial transition probabilities among four health states are transition probabilities at the beginning of pump’s running. And the expected values of the duration time in each state are also available through the training process. The results are given in Tables1 and 2.Table 1
Transition probabilities among four health states.
StateBaselineContamination 1Contamination 2FailureBaseline0.90560.08790.00630.0002Contamination 100.84910.15060.0003Contamination 2000.91290.0871Failure0001Table 2
Expected values of duration time for each health state.
StateBaselineContamination 1Contamination 2FailureExp. of duration10.45499.792311.3375—In this case study, we denote baseline state ass1, contamination 1 state as s2, contamination 2 state as s3, failure state as s4, and the states set as S(Sl∈S, and Sl denotes the health state at observation time point l).  Historical health data for pumps, at each observation time point in one of the failure events, is shown in Figure 7. y-axis indicates the health state of a pump, where 1 means  s1, 2 means s2, 3 means s3, and 4 means s4. It can be seen from Figure 7 that the health states for pumps are rigidly degrading over time.Figure 7
One sequence of the historical health data for a failure event.For testing the proposed methods, after the pumps enter baseline state, firstly, the 5 observation time points are used to compute the RUL and test the effectiveness. Based on Section4.2, the state recognition probability of the pumps staying at health state “baseline” (i.e., p(Sl=s1∣y1:l-1),l=1,…,5) can be computed and the results are given in Table 3.Table 3
State density of 5 observation time points.
Observation time point (l)State probabilityHealth state10.8821s120.855630.790840.774450.6379Then, based on (4.16) and (4.17), the RUL values of 5 observation time points for pumps can be obtained. The comparison of the prognostic and actual RULs and their relative errors are provided in Table 4. It can be seen that the proposed methods are effective for online failure prognosis of pumps. The relative error is computed as follows: (5.6)Error=100%×|ActualRUL-PrognosticRUL|ActualRUL.Table 4
Prognostic and actual RUL (5 observation time points).
Actual RULComputed RULRelative error (%)32.000030.35215.149831.000030.07462.985130.000029.39802.006729.000029.22620.780128.000027.79890.7181In order to determine whether the proposed methods have a better performance for online failure prognosis of pumps, the sample size is enlarged from 5 observation time points to 29 observation time points.First, based on Algorithm4.2, the observation step is set as 4 (i.e., h=4, h is the observation steps) and the state is recognized after 4 observation time points.Then, the state recognition probability is computed. The health state change point is obtained and the changing trends of the state probability with the observation time points are shown in Figure8. Figures 8(a), 8(b), and 8(c) are the probability changing trends of states s1, s2, and s3, respectively. Each health state of pumps is assumed to begin from the observation time point when the corresponding health state probability has the largest value. From Figure 8, it can be seen that the health state of pumps stays at state s1 from the 6th observation time point to the 12th observation time point (see Figure 8(a)), the health state of pumps stays at state s2 from the 13th observation time point to the 22nd observation time point (see Figure 8(b)), and the health state of pumps stays at state s3 from the 23rd observation time point to the 29th observation time point (see Figure 8(c)). Each health state probability for the corresponding observation time points can be found in Table 5.Table 5
State prognosis probability.
Observation time pointState probabilityHealth state30.0711s140.070350.089560.518670.390780.341390.2379100.2092110.1327120.1203130.8792s2140.7982150.7328160.6307170.5961180.4841190.3384200.2306210.1864220.1664230.8935s3240.8632250.8014260.6902270.6187280.5341290.3062Changing trends of the state probability.
(a)
p(Sl+4=s1∣y0:l).(b)
p(Sl+4=s2∣y0:l)(c)
p(Sl+4=s3∣y0:l)From Table5 and Figure 8(a), it can be seen when the observation time point is 5, the failure prognosis probability p(Sl+4=s1∣y0:l) suddenly begins to increase its values and when the observation time point is 6, the failure prognosis probability obtains the maximum value. And it progressively becomes stable from the 6th observation time point, so it indicates that the pumps have more failure possibility at about 6th observation time point. In fact, at the 6th observation time point, the health state of the pumps begins to change from the baseline. From Figure 8(b), it can be seen when the observation time point is 6, the failure prognosis probability p(Sl+4=s2∣y0:l) obviously begins to increase its values. When the observation time point is 7, the failure prognosis probability exceeds 0.5. It is shown that the failure mode of the pumps begins to shift into the second failure mode (contamination 1). In fact, the failure mode of the pumps lies in the contamination 1 at the 13th observation time point. From Figure 8(c), it can be seen when the observation time point is 13, the failure prognosis probability p(Sl+4=s3∣y0:l) obviously begins to increase its values. When the observation time point is 18, the failure prognosis probability exceeds 0.5. It is shown that the failure mode of the pumps begins to shift into the third fault mode (contamination 2). In fact, the failure mode of the pumps lies in the contamination 2 at the 23rd observation time point. It can also be seen that the simulation results are consistent with the actual states and the proposed methods of HSMM-based SMC method can be used to predict the evolution of the system.Based on (4.16), each D̂(si(l)) (l represents the observation time point and i=1,2,3) can be computed. And the expected residual duration of the pumps staying at each health state can be obtained. According to (4.17) and the values of each D̂(si(l)), the RUL can be obtained and the results of HSMM, multistep ahead and one-step ahead health recognition methods can be compared. The comparison between the prognostic and actual RUL is given in Table 6, and the comparison scheme is illustrated in Figure 9. The mean absolute percentage error (MAPE) is set as follows:
(5.7)MAPE=(1n)∑i=1nεi,
where i is the observation time point and n is the total observation time points.Table 6
RUL prognostic comparison with 4 methods.
Actual RULHSMM (joint multisteps)Relative errorHSMM (multisteps)Relative errorHSMM (one-step)Relative errorHSMMRelative error30.0029.3982.006729.39802.006729.39802.006730.25580.852726.0025.21423.022428.58549.943827.85367.129129.964315.247322.0022.51722.350726.061618.461825.329715.135229.795435.433617.0017.51343.019915.38979.472118.32747.808519.498114.694715.0016.07767.183713.70828.611917.625117.500919.208128.05412.0013.16259.687813.489212.409715.447628.730218.866657.221711.0012.966717.87913.330325.265314.309630.086910.24716.84459.009.78668.740113.235219.415613.529050.322410.029111.43445.006.055821.11668.029235.84597.022440.44909.767595.35MAPE8.3341 (%)15.7147 (%)22.1298 (%)29.4592 (%)Figure 9
The results’ comparison scheme for 4 methods.Based on Table6, the relative error of the proposed methods between the prognostic RUL and actual RUL is quite small. So it indicates that the proposed method has a better performance for online failure prognosis of the pumps. And the MAPE of the proposed methods is smaller than that of HSMM [14], multistep ahead, and one-step ahead health recognition methods, respectively. So the prognostic accuracy of the proposed method is higher than that of the HSMM, multisteps ahead, and one-step ahead health recognition methods, respectively.Through the overall case study, the sample of the pumps is modeled based on HSMM using SMC methods and the failure prognosis is implemented based on the available observations. The results show that the proposed methods have a better performance for online health management of a system, including prognostic effectiveness and accuracy. Furthermore, the computational complexity of the proposed methods is decreased compared with HSMM-based method [15] the prognostic accuracy is better than both multisteps ahead and one-step ahead health recognition methods. The proposed methods also reduce the computing storage space. This indicates that the proposed methods are more effective for online health management of complex nonlinear systems and could be used in the real time applications with large data sets.


## 5.1. Hidden States and Probability Density of Parameters

For prognosis, the health changing mechanisms of a system usually involve several degraded states. From normal to failure, a system goes through a series of different degraded states and the certain transition relation among states is existed. There are four hidden states in this case study which are defined as baseline, contamination 1, contamination 2, and failure, respectively. The state transition process for the experiment is described by Figure6.Figure 6
A HSMM scheme describing failure mechanism of a system.There should be three probability distributions which will be modeled by the following sections, including the probability of state transition, the probability of state duration, and the probability of observation.


### 5.1.1. Probability of State Transitions

Assuming thatAij is the jth element of the ith row of the transition matrix A, then the transition probability can be computed as follows:
(5.1)Aij=((1-λ)mij+λA̅ij)∑i=1N((1-λ)mij+λA̅ij),i,j=1,2,3,4,
where the values of A̅ij come from the expert knowledge, mij denotes the number of transitions from state xj to xi (mi0 is the count of occurrences of the state xi). λ is used to control the weights between the expert knowledge and observation evidence.


### 5.1.2. Probability of State Duration

The duration probability distributionpi(d) can be modeled using the half-normal distribution and expressed as follows:
(5.2)pi(d)=1σi2πexp⁡(-12(d-μiσi)),i=1,2,3,4,
where σ2(si) is the variance for the ith state and μ(si) is the mean for the ith state.


### 5.1.3. Probability of Observations

For the observation sequences, two Gaussian mixtures for each state are chosen. Thus, the following probability density can be obtained:(5.3)bi(ot)=∑k=12ωikN(ot,uik,Uik),i=1,2,3,4,k=1,2,
where N(ot,uik,Uik2) is the kth mixture Gaussian distribution at state si. uik, Uik2, and ωik are the mean, variance, and mixture weight of the kth Gaussian distribution at state si, respectively. However, the weights must satisfy the following constraints:(5.4)∑k=12wik=1wik≥0,i=1,2,3,4,k=1,2.Based on Sections2, 5.1.1 and 5.1.2, the parameters of HSMM can be described as follows:
(5.5){A,B,D,π}={aij,uik,Uik2,wik,μi,σi,πi,1≤i,j≤4,1≤k≤2}.


## 5.1.1. Probability of State Transitions

Assuming thatAij is the jth element of the ith row of the transition matrix A, then the transition probability can be computed as follows:
(5.1)Aij=((1-λ)mij+λA̅ij)∑i=1N((1-λ)mij+λA̅ij),i,j=1,2,3,4,
where the values of A̅ij come from the expert knowledge, mij denotes the number of transitions from state xj to xi (mi0 is the count of occurrences of the state xi). λ is used to control the weights between the expert knowledge and observation evidence.


## 5.1.2. Probability of State Duration

The duration probability distributionpi(d) can be modeled using the half-normal distribution and expressed as follows:
(5.2)pi(d)=1σi2πexp⁡(-12(d-μiσi)),i=1,2,3,4,
where σ2(si) is the variance for the ith state and μ(si) is the mean for the ith state.


## 5.1.3. Probability of Observations

For the observation sequences, two Gaussian mixtures for each state are chosen. Thus, the following probability density can be obtained:(5.3)bi(ot)=∑k=12ωikN(ot,uik,Uik),i=1,2,3,4,k=1,2,
where N(ot,uik,Uik2) is the kth mixture Gaussian distribution at state si. uik, Uik2, and ωik are the mean, variance, and mixture weight of the kth Gaussian distribution at state si, respectively. However, the weights must satisfy the following constraints:(5.4)∑k=12wik=1wik≥0,i=1,2,3,4,k=1,2.Based on Sections2, 5.1.1 and 5.1.2, the parameters of HSMM can be described as follows:
(5.5){A,B,D,π}={aij,uik,Uik2,wik,μi,σi,πi,1≤i,j≤4,1≤k≤2}.


## 5.2. Results and Discussion

For the health prognosis, the lifetime training data from three hydraulic pumps is used. An HSMM with four health states (baseline, contamination 1, contamination 2, and failure) can be trained. The initial transition probabilities among four health states are transition probabilities at the beginning of pump’s running. And the expected values of the duration time in each state are also available through the training process. The results are given in Tables1 and 2.Table 1
Transition probabilities among four health states.
StateBaselineContamination 1Contamination 2FailureBaseline0.90560.08790.00630.0002Contamination 100.84910.15060.0003Contamination 2000.91290.0871Failure0001Table 2
Expected values of duration time for each health state.
StateBaselineContamination 1Contamination 2FailureExp. of duration10.45499.792311.3375—In this case study, we denote baseline state ass1, contamination 1 state as s2, contamination 2 state as s3, failure state as s4, and the states set as S(Sl∈S, and Sl denotes the health state at observation time point l).  Historical health data for pumps, at each observation time point in one of the failure events, is shown in Figure 7. y-axis indicates the health state of a pump, where 1 means  s1, 2 means s2, 3 means s3, and 4 means s4. It can be seen from Figure 7 that the health states for pumps are rigidly degrading over time.Figure 7
One sequence of the historical health data for a failure event.For testing the proposed methods, after the pumps enter baseline state, firstly, the 5 observation time points are used to compute the RUL and test the effectiveness. Based on Section4.2, the state recognition probability of the pumps staying at health state “baseline” (i.e., p(Sl=s1∣y1:l-1),l=1,…,5) can be computed and the results are given in Table 3.Table 3
State density of 5 observation time points.
Observation time point (l)State probabilityHealth state10.8821s120.855630.790840.774450.6379Then, based on (4.16) and (4.17), the RUL values of 5 observation time points for pumps can be obtained. The comparison of the prognostic and actual RULs and their relative errors are provided in Table 4. It can be seen that the proposed methods are effective for online failure prognosis of pumps. The relative error is computed as follows: (5.6)Error=100%×|ActualRUL-PrognosticRUL|ActualRUL.Table 4
Prognostic and actual RUL (5 observation time points).
Actual RULComputed RULRelative error (%)32.000030.35215.149831.000030.07462.985130.000029.39802.006729.000029.22620.780128.000027.79890.7181In order to determine whether the proposed methods have a better performance for online failure prognosis of pumps, the sample size is enlarged from 5 observation time points to 29 observation time points.First, based on Algorithm4.2, the observation step is set as 4 (i.e., h=4, h is the observation steps) and the state is recognized after 4 observation time points.Then, the state recognition probability is computed. The health state change point is obtained and the changing trends of the state probability with the observation time points are shown in Figure8. Figures 8(a), 8(b), and 8(c) are the probability changing trends of states s1, s2, and s3, respectively. Each health state of pumps is assumed to begin from the observation time point when the corresponding health state probability has the largest value. From Figure 8, it can be seen that the health state of pumps stays at state s1 from the 6th observation time point to the 12th observation time point (see Figure 8(a)), the health state of pumps stays at state s2 from the 13th observation time point to the 22nd observation time point (see Figure 8(b)), and the health state of pumps stays at state s3 from the 23rd observation time point to the 29th observation time point (see Figure 8(c)). Each health state probability for the corresponding observation time points can be found in Table 5.Table 5
State prognosis probability.
Observation time pointState probabilityHealth state30.0711s140.070350.089560.518670.390780.341390.2379100.2092110.1327120.1203130.8792s2140.7982150.7328160.6307170.5961180.4841190.3384200.2306210.1864220.1664230.8935s3240.8632250.8014260.6902270.6187280.5341290.3062Changing trends of the state probability.
(a)
p(Sl+4=s1∣y0:l).(b)
p(Sl+4=s2∣y0:l)(c)
p(Sl+4=s3∣y0:l)From Table5 and Figure 8(a), it can be seen when the observation time point is 5, the failure prognosis probability p(Sl+4=s1∣y0:l) suddenly begins to increase its values and when the observation time point is 6, the failure prognosis probability obtains the maximum value. And it progressively becomes stable from the 6th observation time point, so it indicates that the pumps have more failure possibility at about 6th observation time point. In fact, at the 6th observation time point, the health state of the pumps begins to change from the baseline. From Figure 8(b), it can be seen when the observation time point is 6, the failure prognosis probability p(Sl+4=s2∣y0:l) obviously begins to increase its values. When the observation time point is 7, the failure prognosis probability exceeds 0.5. It is shown that the failure mode of the pumps begins to shift into the second failure mode (contamination 1). In fact, the failure mode of the pumps lies in the contamination 1 at the 13th observation time point. From Figure 8(c), it can be seen when the observation time point is 13, the failure prognosis probability p(Sl+4=s3∣y0:l) obviously begins to increase its values. When the observation time point is 18, the failure prognosis probability exceeds 0.5. It is shown that the failure mode of the pumps begins to shift into the third fault mode (contamination 2). In fact, the failure mode of the pumps lies in the contamination 2 at the 23rd observation time point. It can also be seen that the simulation results are consistent with the actual states and the proposed methods of HSMM-based SMC method can be used to predict the evolution of the system.Based on (4.16), each D̂(si(l)) (l represents the observation time point and i=1,2,3) can be computed. And the expected residual duration of the pumps staying at each health state can be obtained. According to (4.17) and the values of each D̂(si(l)), the RUL can be obtained and the results of HSMM, multistep ahead and one-step ahead health recognition methods can be compared. The comparison between the prognostic and actual RUL is given in Table 6, and the comparison scheme is illustrated in Figure 9. The mean absolute percentage error (MAPE) is set as follows:
(5.7)MAPE=(1n)∑i=1nεi,
where i is the observation time point and n is the total observation time points.Table 6
RUL prognostic comparison with 4 methods.
Actual RULHSMM (joint multisteps)Relative errorHSMM (multisteps)Relative errorHSMM (one-step)Relative errorHSMMRelative error30.0029.3982.006729.39802.006729.39802.006730.25580.852726.0025.21423.022428.58549.943827.85367.129129.964315.247322.0022.51722.350726.061618.461825.329715.135229.795435.433617.0017.51343.019915.38979.472118.32747.808519.498114.694715.0016.07767.183713.70828.611917.625117.500919.208128.05412.0013.16259.687813.489212.409715.447628.730218.866657.221711.0012.966717.87913.330325.265314.309630.086910.24716.84459.009.78668.740113.235219.415613.529050.322410.029111.43445.006.055821.11668.029235.84597.022440.44909.767595.35MAPE8.3341 (%)15.7147 (%)22.1298 (%)29.4592 (%)Figure 9
The results’ comparison scheme for 4 methods.Based on Table6, the relative error of the proposed methods between the prognostic RUL and actual RUL is quite small. So it indicates that the proposed method has a better performance for online failure prognosis of the pumps. And the MAPE of the proposed methods is smaller than that of HSMM [14], multistep ahead, and one-step ahead health recognition methods, respectively. So the prognostic accuracy of the proposed method is higher than that of the HSMM, multisteps ahead, and one-step ahead health recognition methods, respectively.Through the overall case study, the sample of the pumps is modeled based on HSMM using SMC methods and the failure prognosis is implemented based on the available observations. The results show that the proposed methods have a better performance for online health management of a system, including prognostic effectiveness and accuracy. Furthermore, the computational complexity of the proposed methods is decreased compared with HSMM-based method [15] the prognostic accuracy is better than both multisteps ahead and one-step ahead health recognition methods. The proposed methods also reduce the computing storage space. This indicates that the proposed methods are more effective for online health management of complex nonlinear systems and could be used in the real time applications with large data sets.


## 6. Conclusions

Currently, prognosis is still in its infancy and the literature is yet to present a working model for effective prognosis, but a new trend is that more combination models are designed to deal with data extraction, data processing, and modeling for prognosis. In this paper, in order to obtain the health management of a system, the detailed models and algorithm of HSMM-based SMC method are proposed. Because it is difficult for a single method to obtain satisfied results online health management of a system, the novel online prognostic methods for a system are developed based on both the mathematical structure of HSMM and online features of SMC method, including online health recognition algorithm and online prognostic model. The methods can eliminate the disadvantages of each method, increase the prognostic accuracy, and utilize the advantages of each method. A case study is used to illustrate the superiority of the proposed methods. Through the comparisons among the proposed methods, HSMM, multisteps ahead, and one-step ahead health recognition algorithm, the results show that the proposed methods have a better performance and are more effective for online health management of complex nonlinear systems.



---

*Source: 951584-2012-05-17.xml*