# A Markov Process-Based Approach for Reliability Evaluation of the Propulsion System in Multi-rotor Drones

Koorosh Aslansefat, Francisco Marques, Ricardo Mendonça, José Barata

► To cite this version:

Koorosh Aslansefat, Francisco Marques, Ricardo Mendonça, José Barata. A Markov Process-Based Approach for Reliability Evaluation of the Propulsion System in Multi-rotor Drones. 10th Doctoral Conference on Computing, Electrical and Industrial Systems (DoCEIS), May 2019, Costa de Caparica, Portugal. pp.91-98, 10.1007/978-3-030-17771-3_8. hal-02295227

HAL Id: hal-02295227

https://inria.hal.science/hal-02295227

Submitted on 24 Sep 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A Markov Process-based Approach for Reliability Evaluation of the Propulsion System in Multi-Rotor Drones

Koorosh Aslansefat¹, Francisco Marques¹, Ricardo Mendonça¹, and José Barata¹

¹ CTS-UNINOVA, Universidade Nova de Lisboa (UNL)

Caparica, Portugal

{koorosh.aslansefat, fam, rmm, jab} @uninova.pt

**Abstract.** Autonomous multirotor drones as a popular type of Unmanned Aerial Vehicles (UAVs) have a tremendous potential to facilitate activities such as logistics, emergency response, recording video, capturing special events, and traffic management. Despite the potential benefits the possibility of harming people during operation should be considered. This paper focuses on modeling the multirotor drones' propulsion system with Markov chains. Using the proposed model, both reliability and Mean Time To Failure (MTTF) of the propulsion system are evaluated. This study proposes a fault detection and recovery system based on a Markov Model for mission control of multirotor drones. Concretely, the proposed system aims to reduce potential injuries by increasing safety.

**Keywords:** Markov models, Mission planning, Decision Making, Reliability evaluation, Flight control, UAV.

## 1 Introduction

Reliability can be defined as the probability of multirotor functioning correctly during a given timespan. Safety, on the other hand, is the probability that a multirotor either functions correctly or stops its operation in a safe manner without causing injuries. To design a system that is safe and reliable, multirotor faults and failures need to be studied. Furthermore, performance modeling of the multirotor in the presence of faults and failures needs also to be done. By modeling multirotors for performance and functionality evaluation, bottlenecks and weak points of the system can be determined and strengthened. Having the Markov model of the multirotor is not only beneficial for reliability evaluation. But also, for proposing a mission control and diagnostics system to reduce possible injuries caused by faults and failures in multirotors' systems and subsystems. As failures in the propulsion system can potentially cause a complete loss of control of the UAV it was chosen as the focus of this paper. In the following paragraphs, a brief literature survey on the research of faults and failures in UAVs is provided. In (Murtha, 2009) an FTA based procedure is proposed for reliability improvement in a cost-effective way. Dempster-Shafer Theory is also used to reduce
the uncertainty in failure data. However, while the aforementioned references have used
FTA that is limited to modeling static characteristics of the systems we propose to cover
the dynamic behavior of the multirotors by leveraging on a Markov model.
Furthermore, while in (Olson & Atkins, 2013), only the qualitative failure analysis for
the Michigan UAV has been addressed we will focus on the quantitative failure
analysis. In this paper, the remaining useful life (RUL) of the UAV is estimated through
fault tree analysis and has been used for health-based task allocation (Shi, Yang, &
Quan, 2016), based on the controllability degree, the reliability of multirotor with
different configurations have been studied. In (Belcastro, Klyde, Logan, Newman, &
Foster, 2017) experimental flight test techniques with different possible sets of hazard-
based test scenarios are proposed in order to evaluate the safety of sUAS operations. In
spite of intensive research conducted so far, there is no published work that we are
aware of on Markov modeling for fault detection and recovery for multirotor robots.
This paper introduces the Markov model of a hexacopter and evaluates its reliability
and MTTF. The organization of this paper is as follows. In section 2, the relationship
to industrial and service systems will be addressed briefly. Section 3 introduces Markov
modeling of the propulsion system in multirotors and section 4 illustrates numerical
results obtained from introduced Markov models. In section 5, a brief discussion on the
idea of Markov-based fault detection and recovery is briefly explained. The paper ends
with some concluding remarks and possible future avenues.

## 2 Relationship to Industrial and Service Systems

Nowadays, multirotors and small unmanned aircraft systems (sUAS) are being
developed for consumer and commercial applications in urban areas that encompass
logistics, emergency response, filming, traffic monitoring, agriculture monitoring,
search and rescue, railway surveillance, and corrosion inspection in bridges (B. Klyde,
et al., 2017). Many applications entail operating in populated areas where a failure of
the multirotor can endanger humans. Therefore, reliability and safe landing of
multirotors are important and challenging because of their dynamic behavior and
complex models (Sadeghzadeh, Mehta, & Zhang, 2011).

## 3 Markov Modelling of the UAV's Propulsion Systems

In this section, the proposed Markov model and its solution will be provided for
reliability and MTTF calculation. In addition, the simplification of Markov model for
multirotors will be introduced.

### 3.1 Simple Markov Model and Its Continuous-Time Solution

Figure 1 shows the Markov reliability model of a simple system with one operational
and one failure states. States and transitions between states are first recognized, and the
rates of transitions are allocated. With exponential failure rate distribution function, the
reliability of a system after period of time is then expressed by (1), and its unreliability (failure probability) is given by (2).

$$e^{-n\lambda t} = 1 - n\lambda\Delta t + n^2\lambda^2\Delta t^2 - \dots \quad (1)$$

$$1 - e^{-n\lambda t} = 1 - (1 - n\lambda\Delta t + n^2\lambda^2\Delta t^2 - \dots) \rightarrow n\lambda\Delta t \quad (2)$$

Thus, the probability of the system at the first state after elapsing $\Delta t$ time is given by: $1 - \lambda\Delta t$. In this figure, Lambda stands for the rate of failure.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Reliability Markov model of a simple system.</figcaption></figure>

### 3.1.1 Reliability Evaluation

Based on the Markov model illustrated in Figure 1 and according to the Markov theorem, (3) can be written as follows (Dubrova, 2013).

$$P(t + \Delta t) = M \cdot P(t) \quad (3)$$

where P is the "probability states vector" Error! Reference source not found. and M is the discrete state transition matrix Error! Reference source not found..

$$P(t) = [P_{OP}(t), P_{F}(t)]^{T} \quad (4)$$

$$M = \begin{bmatrix} 1 - \lambda \Delta t & \lambda \Delta t \\ 0 & 1 \end{bmatrix} \quad (5)$$

Equation (3) can be recursively solved if as an initial probability vector is known. The result for different times is given by (6).

$$P(n\Delta t) = P^n \cdot P(0) \quad (6)$$

Equation (7) is the continuous form of (3).

$$\dot{P}(t) = A \cdot P(0) \quad (7)$$

where A is the continuous Markov transition matrix in the form of (8).

$$A = \frac{\partial M^T}{\partial \Delta t} = \begin{bmatrix} -\lambda & 0 \\ \lambda & 0 \end{bmatrix} \quad (8)$$
Solving (7) gives the probability of system states at any time t.

$$P_{OP}(t) = e^{-\lambda t} \qquad (9)$$

$$P_{F}(t) = 1 - e^{-\lambda t} \qquad (10)$$

The reliability of a system can be obtained from the probability of states in Markov model those which represent the operational condition of the system. Hence, the reliability of a simple system is now calculated as equations and (11). In this figure, each Lambda stands for the failure rate of a motor. For example, Lambda a stands for the failure rate of the motor a.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Reliability Markov model of a Quadcopter</figcaption></figure>

Having the Markov model for a quadcopter, the reliability expression of the system can be obtained easily by (12).

$$R_{Quadcopter}(t) = P_{OP}(t) = e^{-(\lambda_a t + \lambda_b t + \lambda_c t + \lambda_d t)} \qquad (11)$$

### 3.1.2 MTTF Calculation

Same Markov model for Mean Time To Failure (MTTF) calculation can be used as in Figure 1. This Markov model is an arbitrary absorbing Markov chain which is described by a transition matrix P. If there are r absorbing states and t transient states, the transition matrix will have the following canonical form. The MTTF Markov model has only one absorbing state and so in this case (I = 1).

$$P = \begin{bmatrix} Q & R \\ 0 & I \end{bmatrix} \qquad (12)$$

Consider $N = (I - Q)^{-1}$ as the fundamental matrix of P and Let $t_i$ be the expected number of steps before the chain is absorbed (goes to state A), given that the chain starts in state $s_i$ and let TTF be the column vector whose $i^{th}$ entry is $TTF_i$. The column vector of TTF can be written as follows:

$$TTF = [t(NA_1) \ t(NA_2) \ ... \ t(NA_n)]^T = NC \qquad (13)$$

where C is a column vector all of whose entries are one. Once the system starts from state Op (Operational), the expected number of steps to be in the failure state (F) can be achieved by (14).
$$MTTF = TTF(Op) \qquad (14)$$

## 3.2 Simple Markov Model and Its Continuous-Time Solution

Fig. 3 illustrates a simple hexacopter configuration with rotors labeled from a to f. In this configuration, motors are PNPNPN. P stands for positive clockwise direction and N stands for negative anti-clockwise direction (Shi, Yang, & Quan, 2016).

<figure><img src="image_4.png" /><figcaption>Fig. 3. A simple hexacopter with rotors labeled from a to f (PNPNPN configuration).</figcaption></figure>

Based on the hexacopter illustrated in Fig. 3, the Markov model of Fig. 4 can be obtained. In this model, 19 states are considered. The first state in this model is "a b c d e f" where all rotors are fully operational. If during the mission one of the rotors fails (for example the one labeled "a"), then with the failure rate of $\lambda_a \Delta t$ the system goes to the "b c d e f" state. In the state: b c d e f", if the rotor "f" or "b" fails, then the systems goes to the "Failure" state. With all these possible failures in the hexacopter, the following model will be constructed. (For more information about the operational and failure modes based on the controllability theory please refer to (Shi, Yang, & Quan, 2016)).
<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Reliability Markov model of a hexacopter with PNPNPN configuration.</figcaption></figure>

**3.3 Model Simplification**

Assuming that “the failure rates of all rotors are the same” the model presented in Fig. 4 can be simplified into the one depicted in Fig. 5 the first state in this model labeled with "6" and "M" means that in this state all six rotors are operational. In case of having only four rotors operational, two states can be considered. The star state demonstrates the system with operational rotors in a quadcopter configuration while the second one covers the other possible configurations. With the assumption of having same failure rates of all rotors, the 19 states Markov model is simplified to an understandable and less complicated 6 states Markov model.

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Simplified reliability Markov model of Hexacopter with PNPNPN configuration.</figcaption></figure>
# 4 Numerical Results

For understanding the system's behavior in terms of reliability and MTTF, numerical results are provided in this section. Fig. 6 shows the evaluation of the reliability of a quadcopter and hexacopter with two kinds of rotors configuration versus time. In this figure, the failure rate of each rotor is assumed as 0.04 failures per hour. The reliability of hexacopter with PNPNPN configuration is higher than the reliability of hexacopter with PPNNPN configuration and both are higher than the reliability of quadcopter. In (Shi, Yang, & Quan, 2016), the reliability of hexacopter with both configurations and without yaw control has been considered with the same expression. However, the result achieved from Markov model shows that they have different values of reliability vs. time in PPNNPN configuration.

<figure><img src="image_7.png" /><figcaption>Fig. 6. Reliability evaluation of Multirotors (Quadcopter and Hex-copter) vs. time with failure rate of 0.04 failure/hour</figcaption></figure>

# 5 Discussion

Markov modeling of multirotors is the first step in designing the fault detection and recovery system. A fault diagnosis module will be needed to specify the current state of the multirotor in its Markov model. Hence, it is assumed that there is a fault detection and diagnosis module that can detect the failure of each rotor. Based on the proposed Markov model and detection of a failed rotor, the mean time to failure and the probability of failure from a related state can be estimated. If both probabilities of failure and MTTF are less than desired value, the robot will continue the mission. However, if the probability of failure or MTTF is more than the desired value the robot will abort the mission or perform a safe landing.
# 6 Conclusions

The higher maneuverability of multi-rotors compared with other types of UAVs makes them very popular. However, safety when operating multi-rotors in populated areas remain one of the most important challenges. In this sense, a Markov model was herein proposed as a fault detection and recovery module for the propulsion system of multi-rotors and was validated through Monte Carlo simulations. Reliability and MTTF of the robot were also evaluated via the developed models. In particular, it was shown that the reliability of a hexacopter configuration is higher than a quadcopter. Moreover, between the two hexacopter configurations studied, the PNPNPN configuration has higher reliability and MTTF when compared to the PPNNPN.

# References

1. Belcastro, C. M., Klyde, D. H., Logan, M. J., Newman, R. L., & Foster, J. V. (2017). Experimental Flight Testing for Assessing the Safety of Unmanned Aircraft System Safety-Critical Operations. *17th AIAA Aviation Technology, Integration, and Operations Conference*. Denver, Colorado.

2. Belcastro, C. M., Newman, R. L., Evans, J. K., Klyde, D. H., Barr, L. C., & Ancel, E. (2017). Hazards Identification and Analysis for Unmanned Aircraft System Operations. *17th AIAA Aviation Technology, Integration, and Operations Conference*. Denver, Colorado.

3. De Medeiros, I. P., Rodrigues, L. R., Santos, R., Shiguemori, E. H., & Júnior, C. L. (2014). PHM-Based Multi-UAV Task Assignment. *8th Annual IEEE Systems Conference (SysCon)*. Ottawa, ON, Canada.

4. Dubrova, E. (2013). *Fault-Tolerant Design*. Berlin: Springer.

5. Juliana de Oliveira Martins Franco, B., & Carlos Sandoval Góes, L. (2007). Failure Analysis Methods in Unmanned Arial Vehicle (UAV) Applications. *19th International Congress of Mechanical Engineering*. Brazil.

6. Murtha, J. F. (2009). *Evidence Theory and Fault-tree Analysis to Cost-effectively Improve Reliability in Small UAV Design*. Virginia, USA: Virginia Polytechnic Institute & State University.

7. Olson, I., & Atkins, E. M. (2013). Qualitative Failure Analysis for a Small Quadrotor Unmanned Aircraft System. *AIAA Guidance, Navigation, and Control (GNC) Conference*. Boston, MA.

8. Sadeghzadeh, I., Mehta, A., & Zhang, Y. (2011). Fault/Damage Tolerant Control of a Quadrotor Helicopter UAV Using Model Reference Adaptive Control and Gain-Scheduled PID. *AIAA Guidance, Navigation, and Control Conference*. Portland, Oregon.

9. Shi, D., Yang, B., & Quan, Q. (2016). Reliability Analysis of Multicopter Configurations Based on Controllability Theory. *IEEE 35th Chinese Control Conference*. Chengdu, China.