# Estimation and Reliability for a Special Type of Semi-Markov Process

Vlad Stefan Barbu, Alex Karagrigoriou, Andreas Makrides

► To cite this version:

Vlad Stefan Barbu, Alex Karagrigoriou, Andreas Makrides. Estimation and Reliability for a Special Type of Semi-Markov Process. Journal of Mathematics and Statistics, 2019, 15 (1), pp.259-272. 10.3844/jmssp.2019.259.272 . hal-02437539

HAL Id: hal-02437539

https://hal.science/hal-02437539

Submitted on 23 May 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Estimation and Reliability for a Special Type of Semi-Markov Process

Vlad Stefan Barbu, Alex Karagrigoriou, Andreas Makrides

► To cite this version:

Vlad Stefan Barbu, Alex Karagrigoriou, Andreas Makrides. Estimation and Reliability for a Special Type of Semi-Markov Process. Journal of Mathematics and Statistics, 2019, 15 (1), pp.259-272. 10.3844/jmssp.2019.259.272 . hal-02437539

HAL Id: hal-02437539

https://hal.science/hal-02437539v1

Submitted on 23 May 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Estimation and Reliability for a Special Type of Semi-Markov Process

¹Vlad Stefan Barbu, ²Alex Karagrigoriou and ³Andreas Makrides

¹Université de Rouen, Laboratoire de Mathématiques Raphaël Salem, UMR 6085,
Avenue de l'Université, BP.12, F76801, Saint-Étienne-du-Rouvray, France

²Laboratory of Statistics and Data Analysis, University of the Aegean, Greece

³University o Rouen, France, University of Central Lancashire, Cyprus

## Article history

Received: 30-07-2019

Revised: 30-08-2019

Accepted: 18-10-2019

Corresponding author:
Andreas Makrides
University o Rouen, France,
University of Central
Lancashire, Cyprus
Email: amakrides@uclan.ac.uk

**Abstract:** This work deals with multi-state systems modelled by means of semi-Markov processes. The sojourn times are seen to be independent not identically distributed random variables and assumed to belong to one of two different general classes of distributions so that the first class is closed under maxima and the second one is closed under minima. We obtain maximum likelihood estimators of the parameters of interest under several estimation schemes and investigate their asymptotic properties. Plug-in type estimators are furnished for various quantities related to the system under study.

**Keywords:** Multi-State System, Reliability Theory, Survival Analysis, Reliability Indicators, Semi-Markov Processes, Parameter Estimation

## Introduction

Let us consider a multi-state system with state space $E = \{1, 2, ..., N\}$, defined on a probability space $(\Omega, \mathcal{F}, P)$ (cf. Natvig, 1982; El-Neveihi and Proschan, 1984). As it will be introduced in the next section the evolution of the system is assumed to follow a continuous time semi-Markov process.

The MSS could be investigated and analyzed with the use of Markov processes (cf. Limnios and Ouhbi, 2006; Limnios et al., 2005; Lisnianski et al., 2010; Lisnianski and Levitin, 2003). Markov processes are widely used for reliability analysis because the number of failures in arbitrary time intervals in many practical cases can be described as a Poisson process and time up to the failure and repair (or maintenance) is often Exponentially distributed. Such systems play a key role in reliability theory, engineering, economics and finance, geosciences and biomedicine with special interest lying in extreme or rare events like among others, natural disasters, total power supply failures, global economic crises, etc.

The main characteristic of this paper is that the sojourn times in a given state are assumed to belong to one of two different general classes of distributions, cf. Relations (3) and (30) respectively. The first class of distributions is closed under maxima and contains several distributions, like the Bernoulli distribution, the power function distribution and the extreme value Type I distribution.

The second class of distributions is closed under minima and includes the exponential, the Weibull, the Pareto, the Rayleigh and the Erlang truncated Exponential distribution (cf. Balasubramanian et al., 1991).

Barbu et al. (2017) the class of distributions closed under minima has been considered. In this paper we deal with: (a) The class of distributions closed under maxima, where we consider several cases of no censoring and censoring at the beginning and/or at the end and appropriate estimators have been proposed; (b) the class of distributions closed under minima, where we consider censoring sojourn times at the beginning and/or at the end.

Our main objective of this paper is the proposal of parsimonious modeling for multistate systems, considering also a semi-Markov framework. Thus, we can have a useful and powerful tool with a reduced number of parameters, which can be of great importance from a practical point of view.

The outline of the paper is the following: In Section 2, a brief discussion about semi-Markov processes and multi-state systems regarding a family of distributions closed under maxima is introduced. Section 3 provides the likelihood function in addition with the maximum likelihood estimators of the parameters under investigation obtained under several statistical settings. In Section 4 a special case for the class of distributions closed under minima is established while the maximum likelihood estimators of the parameters of interest in this
# Estimation and Reliability for a Special Type of Semi-Markov Process

¹Vlad Stefan Barbu, ²Alex Karagrigoriou and ³Andreas Makrides

### Article history

Received: 30-07-2019

Revised: 30-08-2019

Accepted: 18-10-2019

Corresponding author:
Andreas Makrides
University of Rouen, France,
University of Central
Lancashire, Cyprus
Email: amakrides@uclan.ac.uk

**Abstract:** This work deals with multi-state systems modelled by means of semi-Markov processes. The sojourn times are seen to be independent not identically distributed random variables and assumed to belong to one of two different general classes of distributions so that the first class is closed under maxima and the second one is closed under minima. We obtain maximum likelihood estimators of the parameters of interest under several estimation schemes and investigate their asymptotic properties. Plug-in type estimators are furnished for various quantities related to the system under study.

**Keywords:** Multi-State System, Reliability Theory, Survival Analysis, Reliability Indicators, Semi-Markov Processes, Parameter Estimation

## Introduction

Let us consider a multi-state system with state space $E = \{1, 2, ..., N\}$, defined on a probability space $(\Omega, \mathcal{F}, P)$ (cf. Natvig, 1982; El-Neveihi and Proschan, 1984). As it will be introduced in the next section the evolution of the system is assumed to follow a continuous time semi-Markov process.

The MSS could be investigated and analyzed with the use of Markov processes (cf. Limnios and Ouhbi, 2006; Limnios et al., 2005; Lisnianski et al., 2010; Lisnianski and Levitin, 2003). Markov processes are widely used for reliability analysis because the number of failures in arbitrary time intervals in many practical cases can be described as a Poisson process and time up to the failure and repair (or maintenance) is often Exponentially distributed. Such systems play a key role in reliability theory, engineering, economics and finance, geosciences and biomedicine with special interest lying in extreme or rare events like among others, natural disasters, total power supply failures, global economic crises, etc.

The main characteristic of this paper is that the sojourn times in a given state are assumed to belong to one of two different general classes of distributions, cf. Relations (3) and (30) respectively. The first class of distributions is closed under maxima and contains several distributions, like the Bernoulli distribution, the power function distribution and the extreme value Type I distribution.

The second class of distributions is closed under minima and includes the exponential, the Weibull, the Pareto, the Rayleigh and the Erlang truncated Exponential distribution (cf. Balasubramanian et al., 1991).

Barbu et al. (2017) the class of distributions closed under minima has been considered. In this paper we deal with: (a) The class of distributions closed under maxima, where we consider several cases of no censoring and censoring at the beginning and/or at the end and appropriate estimators have been proposed; (b) the class of distributions closed under minima, where we consider censoring sojourn times at the beginning and/or at the end.

Our main objective of this paper is the proposal of parsimonious modeling for multistate systems, considering also a semi-Markov framework. Thus, we can have a useful and powerful tool with a reduced number of parameters, which can be of great importance from a practical point of view.

The outline of the paper is the following: In Section 2, a brief discussion about semi-Markov processes and multi-state systems regarding a family of distributions closed under maxima is introduced. Section 3 provides the likelihood function in addition with the maximum likelihood estimators of the parameters under investigation obtained under several statistical settings. In Section 4 a special case for the class of distributions closed under minima is established while the maximum likelihood estimators of the parameters of interest in this

<figure><img src="image_2.png" /></figure>

<footnote n="1">¹Université de Rouen, Laboratoire de Mathématiques Raphaël Salem, UMR 6085,
Avenue de l'Université, BP.12, F76801, Saint-Étienne-du-Rouvray, France</footnote>

<footnote n="2">²Laboratory of Statistics and Data Analysis, University of the Aegean, Greece</footnote>

<footnote n="3">³University o Rouen, France, University of Central Lancashire, Cyprus</footnote>
case are furnished as well. Section 5 is devoted to study the Markov renewal function $\Psi_{ij}(t)$ and the semi-Markov renewal matrix $P_{ij}(t)$ in order to establish their estimators. In Section 6, the corresponding estimators of reliability indicators are proposed.

## Semi-Markov Processes and Multi State Systems

Semi-Markov (SM) processes are typical tools for the modeling of technical systems. Such class of stochastic processes generalizes typical Markov jump processes by allowing general distributions for sojourn times (Limnios and Oprisan, 2001).

Assume that the time evolution of the system is governed by a stochastic process $Z = (Z_t)_{t \in \mathbb{R}_+}$. Let $S = (S_n)_{n \in \mathbb{N}}$ be the jump times of the process $(Z_t)_{t \in \mathbb{R}_+}$, and $J = (J_n)_{n \in \mathbb{N}}$ the visited states at these jump times. Furthermore, let $X = (X_n)_{n \in \mathbb{N}}$ be the sojourn times in the above states. Therefore, $X_n = S_n - S_{n-1}$, $n \in \mathbb{N}^+$ and without loss of generality let $X_0 = S_0 = 0$.

We assume that $(J, S)$ is a Markov renewal process (cf. Limnios and Oprisian, 2001) and $Z = (Z_t)_{t \in \mathbb{R}_+}$ is a semi-Markov (SM) process associated to $(J, S)$; where $Z_t := J_{N(t)}$, with $N(t) := \sup\{n \in \mathbb{N} | S_n \le t\}$, $t \in \mathbb{R}_+$. In order investigate an SM model the initial law $\alpha = (\alpha_1, \dots, \alpha_N)$, $\alpha_j := \mathbb{P}(J_0 = j)$, $j \in E$ and the semi-Markov kernel $Q_{ij}(t) := \mathbb{P}(J_n = j, X_n \le t | J_{n-1} = i, J_n = j)$, are furnished. Note that we consider regular Markov processes. In addition, the transition probabilities of the embedded Markov chain $(J_n)_{n \in \mathbb{N}}$:

$$p_{ij} := \mathbb{P}(J_n = j | J_{n-1} = i) = \lim_{t \to \infty} Q_{ij}(t),$$

and the conditional sojourn time distribution functions:

$$ \begin{aligned} W_{ij}(t) &:= \mathbb{P}(S_n - S_{n-1} \le t | J_{n-1} = i, J_n = j) \\ &= \mathbb{P}(X_n \le t | J_{n-1} = i, J_n = j), \end{aligned} \quad (1) $$

are also defined. Observe that:

$$Q_{ij}(t) = p_{ij}W_{ij}(t). \quad (2)$$

Let $T_{ij}$ be a potential time in state $i$ before transit to the next state $j$. Let also $F_{ij}(t, \theta_{ij})$ be the cumulative distribution function (cdf) and $\theta_{ij}$ its $m$-dimensional parameter. The above distribution is assumed to be absolutely continuous with respect to the Lebesgue measure with density $f_{ij}(t, \theta_{ij})$.

The evolution of the model is governed by the following characteristic: The maximum value of $T_{il}$ identifies the next visited state, assuming that the process was in state $i$. Hence, under this setting the semi-Markov kernel takes the form:

$$ \begin{aligned} Q_{ij}(t) &= \mathbb{P}\left(\max_{l} T_{il} \le t \ \& \ \max_{l} \text{occurs for } j | J_{n-1} = i\right) \\ &= \mathbb{P}\left(\max_{l} T_{il} \le t, T_{ij} \ge T_{ij}, \forall l | J_{n-1} = i\right) \\ &= \mathbb{P}\left(\max_{l} T_{il} \le t | J_{n-1} = i, J_n = j\right) \times \mathbb{P}\left(T_{ij} \ge T_{il}, \forall l | J_{n-1} = i\right) \\ &= p_{ij}W_{i\bullet}(t), \end{aligned} $$

where:

$$p_{ij} = \mathbb{P}(J_n = j | J_{n-1} = i) = \mathbb{P}(T_{ij} \ge T_{il}, \forall l | J_{n-1} = i)$$

and:

$$ \begin{aligned} W_{ij}(t) &= \mathbb{P}(S_n - S_{n-1} \le t | J_{n-1} = i, J_n = j) \\ &= \mathbb{P}\left(\max_{l} T_{il} \le t | J_{n-1} = i, J_n = j\right) \\ &= \mathbb{P}\left(\max_{l} T_{il} \le t | J_{n-1} = i\right) :=: W_{i\bullet}(t), \text{ independent of } j, \end{aligned} $$

which represents the cdf of the sojourn time in state $i$ (unconditional to the next state to be visited).

Note that $\sum_j Q_{ij}(t) = W_{i\bullet}(t)$, where $W_{i\bullet}(t)$ is absolutely continuous w.r.t. the Lebesgue measure and has a density denoted by $f_{i\bullet}(t)$.

## Class of Distributions

As mentioned at the beginning, we consider two classes of distributions; the first is discussed below while the second one in Section 4. The first one is closed under maxima and the second one is closed under minima. More specifically, although the distribution functions $F_{ij}(\cdot; \theta_{ij})$, have different parameters for different values of $I, j = 1, \dots, N$, they have the same form-pattern. Equivalently, the random variables involved in the above distributions are independent but not identically distributed (inid). Any member of the specific class of distribution functions with parameter $a$ can be written in the following form (Balasubramanian et al., 1991):

$$F(t; a) := (F(t; 1))^a, \quad (3)$$

which is assumed to be absolutely continuous w.r.t. the Lebesgue measure with density $f(t; a)$.

The above class of distributions includes the Bernoulli distribution, the power function distribution and the extreme value Type I distribution. A representative
case are furnished as well. Section 5 is devoted to study the Markov renewal function $\Psi_{ij}(t)$ and the semi-Markov renewal matrix $P_{ij}(t)$ in order to establish their estimators. In Section 6, the corresponding estimators of reliability indicators are proposed.

## Semi-Markov Processes and Multi State Systems

Semi-Markov (SM) processes are typical tools for the modeling of technical systems. Such class of stochastic processes generalizes typical Markov jump processes by allowing general distributions for sojourn times (Limnios and Oprisan, 2001).

Assume that the time evolution of the system is governed by a stochastic process $Z = (Z_t)_{t \in \mathbb{R}_+}$. Let $S = (S_n)_{n \in \mathbb{N}}$ be the jump times of the process $(Z_t)_{t \in \mathbb{R}_+}$ and $J = (J_n)_{n \in \mathbb{N}}$ the visited states at these jump times. Furthermore, let $X = (X_n)_{n \in \mathbb{N}}$ be the sojourn times in the above states. Therefore, $X_n = S_n - S_{n-1}$, $n \in \mathbb{N}^+$ and without loss of generality let $X_0 = S_0 = 0$.

We assume that $(J, S)$ is a Markov renewal process (cf. Limnios and Oprisian, 2001) and $Z = (Z_t)_{t \in \mathbb{R}_+}$ is a semi-Markov (SM) process associated to $(J, S)$; where $Z_t := J_{N(t)}$, with $N(t) := \sup\{n \in \mathbb{N} | S_n \le t\}$, $t \in \mathbb{R}_+$. In order investigate an SM model the initial law $\alpha = (\alpha_1, \dots, \alpha_N)$, $\alpha_j := \mathbb{P}(J_0 = j)$, $j \in E$ and the semi-Markov kernel $Q_{ij}(t) := \mathbb{P}(J_n = j, X_n \le t | J_{n-1} = i, J_n = j)$, are furnished. Note that we consider regular Markov processes. In addition, the transition probabilities of the embedded Markov chain $(J_n)_{n \in \mathbb{N}}$:

$$p_{ij} := \mathbb{P}(J_n = j | J_{n-1} = i) = \lim_{t \to \infty} Q_{ij}(t),$$

and the conditional sojourn time distribution functions:

$$ 
\begin{aligned}
W_{ij}(t) &:= \mathbb{P}(S_n - S_{n-1} \le t | J_{n-1} = i, J_n = j) \\
&= \mathbb{P}(X_n \le t | J_{n-1} = i, J_n = j),
\end{aligned} 
\quad (1) $$

are also defined. Observe that:

$$Q_{ij}(t) = p_{ij}W_{ij}(t). \quad (2)$$

Let $T_{ij}$ be a potential time in state $i$ before transit to the next state $j$. Let also $F_{ij}(t, \theta_{ij})$ be the cumulative distribution function (cdf) and $\theta_{ij}$ its $m$-dimensional parameter. The above distribution is assumed to be absolutely continuous with respect to the Lebesgue measure with density $f_{ij}(t, \theta_{ij})$.

The evolution of the model is governed by the following characteristic: The maximum value of $T_{il}$ identifies the next visited state, assuming that the process was in state $i$. Hence, under this setting the semi-Markov kernel takes the form:

$$ 
\begin{aligned}
Q_{ij}(t) &= \mathbb{P}\left(\max_{l} T_{il} \le t \ \& \ \max_{l} \text{occurs for } j | J_{n-1} = i\right) \\
&= \mathbb{P}\left(\max_{l} T_{il} \le t, T_{ij} \ge T_{ij}, \forall l | J_{n-1} = i\right) \\
&= \mathbb{P}\left(\max_{l} T_{il} \le t | J_{n-1} = i, J_n = j\right) \times \mathbb{P}\left(T_{ij} \ge T_{il}, \forall l | J_{n-1} = i\right) \\
&= p_{ij}W_{i\bullet}(t),
\end{aligned}
$$

where:

$$p_{ij} = \mathbb{P}(J_n = j | J_{n-1} = i) = \mathbb{P}(T_{ij} \ge T_{il}, \forall l | J_{n-1} = i)$$

and:

$$ 
\begin{aligned}
W_{ij}(t) &= \mathbb{P}(S_n - S_{n-1} \le t | J_{n-1} = i, J_n = j) \\
&= \mathbb{P}\left(\max_{l} T_{il} \le t | J_{n-1} = i, J_n = j\right) \\
&= \mathbb{P}\left(\max_{l} T_{il} \le t | J_{n-1} = i\right) := W_{i\bullet}(t), \text{independent of } j,
\end{aligned}
$$

which represents the cdf of the sojourn time in state $i$ (unconditional to the next state to be visited).

Note that $\sum_j Q_{ij}(t) = W_{i\bullet}(t)$, where $W_{i\bullet}(t)$ is absolutely continuous w.r.t. the Lebesgue measure and has a density denoted by $f_{i\bullet}(t)$.

## Class of Distributions

As mentioned at the beginning, we consider two classes of distributions; the first is discussed below while the second one in Section 4. The first one is closed under maxima and the second one is closed under minima. More specifically, although the distribution functions $F_{ij}(\cdot; \theta_{ij})$, have different parameters for different values of $I, j = 1, \dots, N$, they have the same form-pattern. Equivalently, the random variables involved in the above distributions are independent but not identically distributed (inid). Any member of the specific class of distribution functions with parameter $a$ can be written in the following form (Balasubramanian et al., 1991):

$$F(t; a) := (F(t; 1))^a, \quad (3)$$

which is assumed to be absolutely continuous w.r.t. the Lebesgue measure with density $f(t; a)$.

The above class of distributions includes the Bernoulli distribution, the power function distribution and the extreme value Type I distribution. A representative
example comes from structural engineering where engineers are interested in stress and strain diagrams that graphically display the basic material characteristics when designing various types of constructions like bridges, highways or building. Strain is the elongation or contraction of a material per unit length of the material while stress is the applied load divided by the material area it is acting on. For measuring the extend of stress and strain, engineers often use (a) the tensile strength which is the amount of tensile stress that a material can resist before failing, (b) the compressive strength which is the amount of compressive stress that a material can resist before failing and (c) the ductile which is the ability of a material to be subjected to large strains before it ruptures or fails. In such cases the investigator is looking for the maximum time that represents the moment the material ruptures, fails or cracks (cf. Carreira and Chu, 1985; 1986; Courtney, 2005).

The following Lemma, shows that the distribution function of the maximum order statistic falls also to class 3, in other words the above class of distributions is closed under maxima. This is stated in the next result.

### Lemma 1

Let $X_1, \dots, X_N$ be iid random variables such that $X_i \sim F(x; a_i)$ which belongs to class (3). Then the distribution function $F^{(N)}$ of the maximum order statistic $X_{(N)}$ belongs also to (3) (Balasubramanian et al. 1991).

For notational convenience and for $T_{ij}$ the sojourn time as defined in the previous section, we set $F_{ij}(t) := F_{ij}(t; 1)$, $f_{ij}(t) := f_{ij}(t; 1)$ and $Q_{ij}(t; a_{im}; m = 1, \dots, N) := Q_{ij}(t)$.

Note that the dependence of semi-Markov kernel to $a_{im}$ is due to the fact that the parameter of the distribution function $F(t)$ that appears in the semi-Markov kernel is $\sum_{m \in E} a_{im}$ (Equation (4)).

### Proposition 1

According to the setting of the present section, the following quantities can be derived:

$$Q_{ij}(t) = \frac{a_{ij}}{\sum_{m \in E} a_{im}} \cdot [F(t)]_{\sum_{m \in E} a_{im}}, \quad (4)$$

$$p_{ij} = \frac{a_{ij}}{\sum_{m \in E} a_{im}}, \quad (5)$$

$$W_{i, \bullet}(t) = [F(t)]_{\sum_{m=1}^{N} a_{im}} \quad (6)$$

and:

$$f_{i, \bullet}(t) = \sum_{m=1}^{N} a_{im} \cdot [F(t)]_{\sum_{m=1}^{N} a_{im}} \frac{f(t)}{F(t)}. \quad (7)$$

### Proof

Note that:

$$\begin{align} Q_{ij}(t) &= \int_0^1 [F(u)]^{a_{i,1}} \dots [F(u)]^{a_{i,j-1}} [F(u)]^{a_{j,1}} \dots \tag{8} \\ & \qquad \cdot d\left[1-(F(u))^{a_{ij}}\right] \nonumber \\ &= \int_0^1 a_{ij} [F(u)]_{\sum_{m \in E} a_{im}-1} dF(u) = \frac{a_{ij}}{\sum_{m \in E} a_{im}} \cdot [F(t)]_{\sum_{m \in E} a_{im}}. \nonumber \end{align}$$

By taking the limit:

$$\lim_{t \to \infty} Q_{ij}(t) = p_{ij} = \frac{a_{ij}}{\sum_{m \in E} a_{im}}. \quad (9)$$

We also need the distribution of the maximum which is given by:

$$\begin{align} W_{i, \bullet}(t) &= \mathbb{P}[T_{\max} \le t | J_n = i] = \mathbb{P}[T_{\max} \le t, \forall m] \nonumber \\ &= \prod_{m=1}^{N} [F_{im}(t)] = \prod_{m=1}^{N} [F(t)]^{a_{im}} = [F(t)]^{\sum_{m=1}^{N} a_{im}}. \tag{10} \end{align}$$

Thus, by taking the derivative of the above equation, the pdf in (7) is easily derived.

### Maximum Likelihood Estimation

We consider two statistical settings. In the first one we have only one sample path of the system, while in the second one we have several sample paths. On each situation we take into account several cases: One where all sojourn times are observed and one where the sojourn time in the last and/or in the first visited state can be right censored. In addition, the cases where right censoring appears only at the beginning or at the end are also considered. Right censoring at the end is associated with "the lost to follow up" while right censoring at the beginning is associated with the lack of information in the moment the action had began.

#### Maximum Likelihood Estimation for one Trajectory

Let denote the total observation time with $M$. For the scope of this section we introduce the processes $N_i(t)$ which counts the number of visits to state $i$ of the process $(J_n)_{n \in \mathbb{N}}$ up to time $t$ and $N_{ij}(t)$ which counts the number of jumps of the process $(J_n)_{n \in \mathbb{N}}$ from state $i$ to state $j$ up to time $t$, where $I, j \in E$ and $t \in \mathbb{R}_+$, i.e.:

$$N_i(t) := \sum_{n=0}^{N(t)-1} 1_{\{J_n=i\}} = \sum_{n=0}^{\infty} 1_{\{J_n=i, S_{n-1} \le t\}},$$

$$N_{ij}(t) := \sum_{n=1}^{N(t)} 1_{\{J_{n-1}=i, J_n=i\}} = \sum_{n=0}^{\infty} 1_{\{J_{n-1}=i, J_n=i, S_n \le t\}}.$$
example comes from structural engineering where engineers are interested in stress and strain diagrams that graphically display the basic material characteristics when designing various types of constructions like bridges, highways or building. Strain is the elongation or contraction of a material per unit length of the material while stress is the applied load divided by the material area it is acting on. For measuring the extend of stress and strain, engineers often use (a) the tensile strength which is the amount of tensile stress that a material can resist before failing, (b) the compressive strength which is the amount of compressive stress that a material can resist before failing and (c) the ductile which is the ability of a material to be subjected to large strains before it ruptures or fails. In such cases the investigator is looking for the maximum time that represents the moment the material ruptures, fails or cracks (cf. Carreira and Chu, 1985; 1986; Courtney, 2005).

The following Lemma, shows that the distribution function of the maximum order statistic falls also to class 3, in other words the above class of distributions is closed under maxima. This is stated in the next result.

### Lemma 1

Let $X_1, \dots, X_N$ be iid random variables such that $X_i \sim F(x; a_i)$ which belongs to class (3). Then the distribution function $F^{(N)}$ of the maximum order statistic $X_{(N)}$ belongs also to (3) (Balasubramanian et al. 1991).

For notational convenience and for $T_{ij}$ the sojourn time as defined in the previous section, we set $F_{ij}(t) := F_{ij}(t; 1)$, $f_{ij}(t) := f_{ij}(t; 1)$ and $Q_{ij}(t; a_{im}; m = 1, \dots, N) := Q_{ij}(t)$.

Note that the dependence of semi-Markov kernel to $a_{im}$ is due to the fact that the parameter of the distribution function $F(t)$ that appears in the semi-Markov kernel is $\sum_{m \in E} a_{im}$ (Equation (4)).

### Proposition 1

According to the setting of the present section, the following quantities can be derived:

$$Q_{ij}(t) = \frac{a_{ij}}{\sum_{m \in E} a_{im}} \cdot [F(t)]_{\sum_{m \in E} a_{im}}, \quad (4)$$

$$p_{ij} = \frac{a_{ij}}{\sum_{m \in E} a_{im}}, \quad (5)$$

$$W_{i, \bullet}(t) = [F(t)]_{\sum_{m=1}^{N} a_{im}} \quad (6)$$

and:

$$f_{i, \bullet}(t) = \sum_{m=1}^{N} a_{im} \cdot [F(t)]_{\sum_{m=1}^{N} a_{im}} \frac{f(t)}{F(t)}. \quad (7)$$

### Proof

Note that:

$$\begin{align} Q_{ij}(t) &= \int_0^1 [F(u)]^{a_{i,1}} \dots [F(u)]^{a_{i,j-1}} [F(u)]^{a_{j,1}} \dots \tag{8} \\ & \qquad \times d\left[1-(F(u))^{a_{ij}}\right] \nonumber \\ &= \int_0^1 a_{ij} [F(u)]_{\sum_{m \in E} a_{im}-1} dF(u) = \frac{a_{ij}}{\sum_{m \in E} a_{im}} \cdot [F(t)]_{\sum_{m \in E} a_{im}}. \nonumber \end{align}$$

By taking the limit:

$$\lim_{t \to \infty} Q_{ij}(t) = p_{ij} = \frac{a_{ij}}{\sum_{m \in E} a_{im}}. \quad (9)$$

We also need the distribution of the maximum which is given by:

$$\begin{align} W_{i, \bullet}(t) &= \mathbb{P}[T_{\max} \le t | J_n = i] = \mathbb{P}[T_{\max} \le t, \forall m] \nonumber \\ &= \prod_{m=1}^{N} [F_{im}(t)] = \prod_{m=1}^{N} [F(t)]^{a_{im}} = [F(t)]_{\sum_{m=1}^{N} a_{im}}. \tag{10} \end{align}$$

Thus, by taking the derivative of the above equation, the pdf in (7) is easily derived.

## Maximum Likelihood Estimation

We consider two statistical settings. In the first one we have only one sample path of the system, while in the second one we have several sample paths. On each situation we take into account several cases: One where all sojourn times are observed and one where the sojourn time in the last and/or in the first visited state can be right censored. In addition, the cases where right censoring appears only at the beginning or at the end are also considered. Right censoring at the end is associated with "the lost to follow up" while right censoring at the beginning is associated with the lack of information in the moment the action had began.

### Maximum Likelihood Estimation for one Trajectory

Let denote the total observation time with $M$. For the scope of this section we introduce the processes $N_i(t)$ which counts the number of visits to state $i$ of the process $(J_n)_{n \in \mathbb{N}}$ up to time $t$ and $N_{ij}(t)$ which counts the number of jumps of the process $(J_n)_{n \in \mathbb{N}}$ from state $i$ to state $j$ up to time $t$, where $I, j \in E$ and $t \in \mathbb{R}_+$, i.e.:

$$N_i(t) := \sum_{n=0}^{N(t)-1} 1_{\{J_n=i\}} = \sum_{n=0}^{\infty} 1_{\{J_n=i, S_{n-1} \le t\}},$$

$$N_{ij}(t) := \sum_{n=1}^{N(t)} 1_{\{J_{n-1}=i, J_n=i\}} = \sum_{n=0}^{\infty} 1_{\{J_{n-1}=i, J_n=i, S_n \le t\}}.$$
## The Case of no Censoring for One Trajectory

Firstly, we take into account one single sample path of a semi-Markov process {$j_0, x_1, j_1, x_2, \dots, j_{N(M)}$} where the evolution of the system for 1 trajectory is given by the following likelihood:

$$ 
\begin{aligned}
\mathcal{L} &= \alpha_{j_0} p_{j_0 j_1} f_{j_0 \bullet} (x_1) \dots p_{j_{N(M)-1} j_{N(M)}} f_{j_{N(M)-1} \bullet} (x_{N(M)}) \\
&= \alpha_{j_0} \left( \prod_{i,j \in E} P_{ij}^{N(M)} \right) \left( \prod_{i \in E} \prod_{k=1}^{N(M)} f_{\bullet} (x_i^{(1,k)}) \right),
\end{aligned} 
\quad (11) $$

where, $x_i^{(1,k)}$ is the sojourn time in state $i$ during the $k^{th}$ visit, $k = 1, \dots, N(M)$. The superscript "1" indicates the use of a single trajectory; later we will have several trajectories.

Under the class of distributions (3), the likelihood above can be written as:

$$ \mathcal{L} = \alpha_{j_0} \left( \prod_{i,j \in E} P_{ij}^{N(M)} \right) \prod_{i,k} \left\{ \frac{F(x_i^{(1,k)})}{F(x_j^{(1,k)})} \right\} $$

The estimator of $a_{ij}$ are obtained through derivation of the preceding log-likelihood w.r.t. $a_{ij}$, $I, j \in E$, that is:

$$ \hat{a}_{ij}(M) = - \frac{N_{ij}(M)}{\sum_{k=1}^{N_{ij}(M)} \log(F(x_i^{(1,k)}))} \quad (12) $$

As for the estimators of $Q$, $p$, $W$ and $f$ can be derived using expressions (4) - (7).

## Censoring at the End for One Trajectory

Let {$j_0, x_1, j_1, x_2, \dots, j_{N(M)}, u_M$} a censored sample path, where $u_M := M-S_{N(M)}$ is the last censored sojourn time in the last visited state. We follow the methodology proposed in Barbu et al. (2018) for parametric estimation of discrete-time semi-Markov processes. The contribution to the likelihood of a right censored time $k$, in state $i$ is defined as:

$$ \bar{W}_{i \bullet}(k) = \mathbb{P}(S_{n+1} - S_n > k | J_n = i) = 1 - \sum_{t=1}^{k} f_{i \bullet}(t). $$

Then, the likelihood is given by:

$$ 
\begin{aligned}
\mathcal{L} &= \alpha_{j_0} p_{j_0 j_1} f_{j_0 \bullet} (x_1) \dots f_{j_{N(M)-1} \bullet} (x_{N(M)}) \times \mathbb{P}(X_{j_{N(M)}} > u_M) \\
&= \alpha_{j_0} \left( \prod_{i,j \in E} P_{ij}^{N(M)} \right) \left( \prod_{i \in E} \prod_{k=1}^{N(M)} f_{\bullet} (x_i^{(1,k)}) \right) \\
&\quad \times \left( \bar{W}_{j_{N(M)} \bullet} (u_M) \right).
\end{aligned} 
\quad (13) $$

Under the class of distributions in (3), the estimator $\hat{a}_{ij}(M)$, for the case of censoring at the end, is obtained by solving the following equation:

$$ 
\begin{aligned}
\frac{\partial \log \mathcal{L}}{\partial a_{ij}} &= \frac{N_{ij}(M)}{a_{ij}} + \sum_{k=1}^{N_{ij}(M)} \log(F(x_i^{(1,k)})) \\
&\quad -1 \left\{ \frac{F(u_M) \sum_{m \in E} a_{im} \log F(u_M)}{1 - (F(u_M)) \sum_{m \in E} a_{im}} \right\} = 0.
\end{aligned} 
\quad (14) $$

## Censoring at the Beginning for One Trajectory

Let now {$x_0, j_0, x_1, j_1, x_2, \dots, j_{N(M)}$}, where $x_1$ is the first censored sojourn time in the first visited state. Note that the contribution to the likelihood is the same as before. Hence, the likelihood can be written as:

$$ 
\begin{aligned}
\mathcal{L} &= \alpha_{j_0} p_{j_0 j_1} \bar{W}_{j_0 \bullet} (x_0) p_{j_1 j_2} f_{j_1 \bullet} (x_1) \dots p_{j_{N(M)-1} j_{N(M)}} f_{j_{N(M)-1} \bullet} (x_{N(M)}) \\
&= \alpha_{j_0} \bar{W}_{j_0 \bullet} (x_0) \left( \prod_{i,j \in E} P_{ij}^{N(M)} \right) \left( \prod_{i \in E} \prod_{k=1}^{N(M)} f_{\bullet} (x_i^{(1,k)}) \right).
\end{aligned} 
\quad (15) $$

Under the class of distributions in (3), by taking the derivative of the log-likelihood w.r.t. $a_{ij}$, $i, j \in E$, we get that the estimator $\hat{a}_{ij}(M)$ is given by solving the equation:

$$ 
\begin{aligned}
\frac{\partial \log \mathcal{L}}{\partial a_{ij}} &= \frac{N_{ij}(M)}{a_{ij}} + \sum_{k=1}^{N_{ij}(M)} \log(F(x_i^{(1,k)})) \\
&\quad -1 \left\{ \frac{F(x_0) \sum_{m \in E} a_{im} \log F(x_0)}{1 - (F(x_0)) \sum_{m \in E} a_{im}} \right\} = 0.
\end{aligned} 
\quad (16) $$

## Maximum Likelihood Estimation for Several Trajectories

The following counting processes will be useful later:

* $\bar{N}_{i \bullet}^b(L) = \sum_{i} \sum_{l=1}^{L} 1\{j_0^{(l)} = i, S_1^{(l)} - S_0^{(l)} > t, S_1^{(l)} < M\}$ is the number of trajectories starting in state $i$ with censored first sojourn time in state $i$

* $\bar{N}_{i \bullet}^c(L) = \sum_{i} \sum_{l=1}^{L} 1\{J_0^{(l)} = i, x_1^{(l)} > t, x_1^{(l)} < M\}$ is the number of trajectories ending in state $i$ with censored last sojourn time in state $i$

Note that, the inequality $S_1^{(l)} < M$ is involved in the definition of $\bar{N}_{i \bullet}^b(L)$ in order not to count the case when we have censoring at the beginning without any transition until the observed time $M$ (so, having also
## The Case of no Censoring for One Trajectory

Firstly, we take into account one single sample path of a semi-Markov process {$j_0, x_1, j_1, x_2, \dots, j_{N(M)}$} where the evolution of the system for 1 trajectory is given by the following likelihood:

$$ 
\begin{aligned}
\mathcal{L} &= \alpha_{j_0} p_{j_0 j_1} f_{j_0 \bullet} (x_1) \dots p_{j_{N(M)-1} j_{N(M)}} f_{j_{N(M)-1} \bullet} (x_{N(M)}) \\
&= \alpha_{j_0} \left( \prod_{i,j \in E} P_{ij}^{N(M)} \right) \left( \prod_{i \in E} \prod_{k=1}^{N(M)} f_{\bullet} (x_i^{(1,k)}) \right),
\end{aligned} 
\quad (11) $$

where, $x_i^{(1,k)}$ is the sojourn time in state $i$ during the $k^{th}$ visit, $k = 1, \dots, N(M)$. The superscript "1" indicates the use of a single trajectory; later we will have several trajectories.

Under the class of distributions (3), the likelihood above can be written as:

$$ \mathcal{L} = \alpha_{j_0} \left( \prod_{i,j \in E} P_{ij}^{N(M)} \right) \prod_{i,k} \left\{ \frac{F(x_i^{(1,k)})}{F(x_j^{(1,k)})} \right\} $$

The estimator of $a_{ij}$ are obtained through derivation of the preceding log-likelihood w.r.t. $a_{ij}$, $I, j \in E$, that is:

$$ \hat{a}_{ij}(M) = - \frac{N_{ij}(M)}{\sum_{k=1}^{N_{ij}(M)} \log(F(x_i^{(1,k)}))} \quad (12) $$

As for the estimators of $Q$, $p$, $W$ and $f$ can be derived using expressions (4) - (7).

## Censoring at the End for One Trajectory

Let {$j_0, x_1, j_1, x_2, \dots, j_{N(M)}, u_M$} a censored sample path, where $u_M := M-S_{N(M)}$ is the last censored sojourn time in the last visited state. We follow the methodology proposed in Barbu et al. (2018) for parametric estimation of discrete-time semi-Markov processes. The contribution to the likelihood of a right censored time $k$, in state $i$ is defined as:

$$ \bar{W}_{i \bullet}(k) = \mathbb{P}(S_{n+1} - S_n > k | J_n = i) = 1 - \sum_{t=1}^{k} f_{i \bullet}(t). $$

Then, the likelihood is given by:

$$ 
\begin{aligned}
\mathcal{L} &= \alpha_{j_0} p_{j_0 j_1} f_{j_0 \bullet} (x_1) \dots f_{j_{N(M)-1} \bullet} (x_{N(M)}) \times \mathbb{P}(X_{j_{N(M)}} > u_M) \\
&= \alpha_{j_0} \left( \prod_{i,j \in E} P_{ij}^{N(M)} \right) \left( \prod_{i \in E} \prod_{k=1}^{N(M)} f_{\bullet} (x_i^{(1,k)}) \right) \\
&\quad \times \left( \bar{W}_{j_{N(M)} \bullet} (u_M) \right).
\end{aligned} 
\quad (13) $$

Under the class of distributions in (3), the estimator $\hat{a}_{ij}(M)$, for the case of censoring at the end, is obtained by solving the following equation:

$$ 
\begin{aligned}
\frac{\partial \log \mathcal{L}}{\partial a_{ij}} &= \frac{N_{ij}(M)}{a_{ij}} + \sum_{k=1}^{N_{ij}(M)} \log(F(x_i^{(1,k)})) \\
&\quad -1 \left\{ \frac{F(u_M) \sum_{m \in E} a_m \log F(u_M)}{1 - (F(u_M)) \sum_{m \in E} a_m} \right\} = 0.
\end{aligned} 
\quad (14) $$

## Censoring at the Beginning for One Trajectory

Let now {$x_0, j_0, x_1, j_1, x_2, \dots, j_{N(M)}$}, where $x_1$ is the first censored sojourn time in the first visited state. Note that the contribution to the likelihood is the same as before. Hence, the likelihood can be written as:

$$ 
\begin{aligned}
\mathcal{L} &= \alpha_{j_0} p_{j_0 j_1} \bar{W}_{j_0 \bullet} (x_0) p_{j_1 j_2} f_{j_1 \bullet} (x_1) \dots p_{j_{N(M)-1} j_{N(M)}} f_{j_{N(M)-1} \bullet} (x_{N(M)}) \\
&= \alpha_{j_0} \bar{W}_{j_0 \bullet} (x_0) \left( \prod_{i,j \in E} P_{ij}^{N(M)} \right) \left( \prod_{i \in E} \prod_{k=1}^{N(M)} f_{\bullet} (x_i^{(1,k)}) \right).
\end{aligned} 
\quad (15) $$

Under the class of distributions in (3), by taking the derivative of the log-likelihood w.r.t. $a_{ij}$, $i, j \in E$, we get that the estimator $\hat{a}_{ij}(M)$ is given by solving the equation:

$$ 
\begin{aligned}
\frac{\partial \log \mathcal{L}}{\partial a_{ij}} &= \frac{N_{ij}(M)}{a_{ij}} + \sum_{k=1}^{N_{ij}(M)} \log(F(x_i^{(1,k)})) \\
&\quad -1 \left\{ \frac{F(x_0) \sum_{m \in E} a_m \log F(x_0)}{1 - (F(x_0)) \sum_{m \in E} a_m} \right\} = 0.
\end{aligned} 
\quad (16) $$

## Maximum Likelihood Estimation for Several Trajectories

The following counting processes will be useful later:

* $\bar{N}_{i \bullet}^b(L) = \sum_{i} \sum_{l=1}^{L} 1\{j_0^{(l)} = i, S_1^{(l)} - S_0^{(l)} < M\}$ is the number of trajectories starting in state $i$ with censored first sojourn time in state $i$

* $\bar{N}_{i \bullet}^c(L) = \sum_{i} \sum_{l=1}^{L} 1\{J_0^{(l)} = i, x_1^{(l)} > t\}$ is the number of trajectories ending in state $i$ with censored last sojourn time in state $i$

Note that, the inequality $S_1^{(l)} < M$ is involved in the definition of $\bar{N}_{i \bullet}^b(L)$ in order not to count the case when we have censoring at the beginning without any transition until the observed time $M$ (so, having also
censoring at the end). Note that, this case (only 1 visited state with censoring at the beginning and at the end) is counted only in $\bar{N}_{i,\bullet}^e(L)$.

### The case of no Censoring for Several Trajectories

Let us consider L sample paths of a semi-Markov process, $\{j_0^{(l)}, x_1^{(l)}, j_1^{(l)}, x_2^{(l)}, \dots, j_{N'(M)}^{(l)}\}$, $l = 1, \dots, L$, without censoring. In this case the likelihood becomes:

$$ \mathcal{L} = \prod_{l=1}^{L} \alpha_{j_0}^{(l)} p_{j_0^{(l)}, j_1^{(l)}} f_{j_0^{(l)}}(x_1^{(l)}) \dots f_{j_{N'(M)}^{(l)}}(x_{N'(M)}^{(l)}) \times \left( \prod_{i \in E} \alpha_i^{N_{i,0}(L)} \right) \left( \prod_{i,j \in E} p_{j_1^{(l)}, 1}^{\sum_{k=1}^{L} N_{k,j}^{(l)}(M)} \right) \times \left( \prod_{l=1}^{L} \prod_{i \in E} \prod_{k=1}^{N_{i,M}^{(l)}} f_{i,\bullet}(x_i^{(1,k)}) \right), $$

where:

* $N_{i,0}(L) := \sum_{l=1}^{L} 1_{\{j_0^{(l)}=1\}}$: The number of trajectories starting in state i

* $N_i^{(l)}(M)$: The number of visits to state i up to time M of the lth trajectory, $l = 1, \dots, L$

* $N_{ij}^{(l)}(M)$: The number of transitions from state i to state j up to time M during the lth trajectory, $l = 1, \dots, L$

* $N_{ij}(L, M) := \sum_{l=1}^{L} N_{ij}^{(l)}(M)$

* $x_i^{(l,k)}$: The sojourn time in state i during the kth visit, $k = 1, \dots, N_i^{(l)}(M)$ of the lth trajectory, $l = 1, \dots, L$

Note that for $L = 1$ the likelihood (17) reduces to the likelihood of the one trajectory case given in (11).

Under the class of distributions given in (3) the likelihood for L trajectories without censoring becomes:

$$ \mathcal{L} = \prod_{i \in E} \alpha_i^{N_{i,0}(L)} \left( \prod_{l=1}^{L} \prod_{i,j \in E} \alpha_{ij}^{N_{ij}^{(l)}(M)} \right) \prod_{l,i,k} \left\{ \frac{\left( F(x_i^{(l,k)}) \right)^{\sum_{m=\bar{N}_{i,k}} a_m} \frac{f(x_i^{(l,k)})}{F(x_i^{(l,k)})} \right\} $$

and we obtain the estimator of $a_{ij}$, namely $\hat{a}_{ij}(L, M)$, which is given by:

$$ \hat{a}_{ij}(L, M) = - \frac{N_{ij}(L, M)}{\sum_{l=1}^{L} \sum_{k=1}^{N_{i,M}^{(l)}(M)} \log(F(X_i^{(l,k)}))} \quad (18) $$

The MLE of the initial law is given by:

$$ \hat{\alpha}_i(L, M) = \frac{N_{i,0}(L)}{L}. \quad (19) $$

### Censoring at the end for Several Trajectories

Let $\{j_0^{(l)}, x_1^{(l)}, j_1^{(l)}, x_2^{(l)}, \dots, j_{N'(M)}^{(l)}, u_{(M)}^{(l)}\}$, $l = 1, \dots, L$, be the L censored sample paths of a semi-Markov process with censoring at time M, then the associated likelihood is:

$$ \mathcal{L} = \left( \prod_{i \in E} \alpha_i^{N_{i,0}(L)} \right) \left( \prod_{i, j \in E} p_{ij}^{\sum_{k=1}^{L} N_{ij}^{(l)}(M)} \right) \times \left( \prod_{l=1}^{L} \prod_{i \in E} \prod_{k=1}^{N_{i,M}^{(l)}(M)} f_{i, \bullet}(x_i^{(l,k)}) \right) \prod_{j \in E} \prod_{k=1}^{\bar{N}_{i,\bullet}^e(L)} W_{i, \bullet}(x_{i,M}^{(k)}), \quad (20) $$

Where:

* $u_M^{(l)} := M - S_{N'(M)}$ is the censored sojourn time in the last visited state of the lth trajectory

* $x_{i,M}^{(k)}$ is the last censored sojourn time in state i during the kth visit, $k = 1, \dots, \bar{N}_{i,\bullet}^e(L)$.

Note that, if the censoring time M in a certain trajectory l is a jump time, then for the corresponding observed censored time we have $u_M^{(l)} = 0$. Consequently, the contribution to the likelihood of the associated term will be equal to 1. For this reason, if no censoring is involved, the uncensored likelihood given in (17) is just a particular case of (20). Note also that for $L = 1$ the likelihood given in (20) becomes the likelihood of one trajectory with censoring, given in (13).

The estimator $\hat{a}_{ij}(L, M)$, under the class of distributions in (3), is established by solving the following equation:

$$ \frac{\partial \log \mathcal{L}}{\partial a_{ij}} = \frac{N_{ij}(L, M)}{a_{ij}} + \sum_{l=1}^{L} \sum_{k=1}^{N_{i,M}^{(l)}(M)} \log(F(x_i^{(l,k)})) - \sum_{k=1}^{\bar{N}_{i,\bullet}^e(L)} \frac{F(x_{i,M}^{(k)}) \sum_{m=\bar{N}_{i,k}} a_m \log F(x_{i,M}^{(k)})}{1 - (F(x_{i,M}^{(k)}))^{\sum_{m=\bar{N}_{i,k}} a_m}} = 0. \quad (21) $$

Regarding the estimator of the initial distribution, given by (19).

### Censoring at the Beginning for Several Trajectories

Let now $\{x_0^{(l)}, j_0^{(l)}, x_1^{(l)}, j_1^{(l)}, x_2^{(l)}, \dots, j_{N'(M)}^{(l)}\}$, $l = 1, \dots, L$, be the L censored sample paths of a semi-Markov process with censoring at the beginning. In this case the likelihood takes the form:
censoring at the end). Note that, this case (only 1 visited state with censoring at the beginning and at the end) is counted only in $\bar{N}_{i,\bullet}^e(L)$.

### The case of no Censoring for Several Trajectories

Let us consider L sample paths of a semi-Markov process, $\{j_0^{(l)}, x_1^{(l)}, j_1^{(l)}, x_2^{(l)}, \dots, j_{N'(M)}^{(l)}\}$, $l = 1, \dots, L$, without censoring. In this case the likelihood becomes:

$$ \mathcal{L} = \prod_{l=1}^{L} \alpha_{j_0}^{(l)} p_{j_0^{(l)}, j_1^{(l)}} f_{j_0^{(l)}}(x_1^{(l)}) \dots f_{j_{N'(M)}^{(l)}}(x_{N'(M)}^{(l)}) \\ = \left(\prod_{i \in E} \alpha_i^{N_{i,0}(L)}\right) \left(\prod_{i,j \in E} p_{j_1^{(l)}}^{\sum_{k=1}^{L} N_{k,j}^{(l)}(M)}\right) \times \left(\prod_{l=1}^{L} \prod_{i \in E} \prod_{k=1}^{N_{i,M}^{(l)}} f_{i,\bullet}(x_i^{(l,k)})\right), $$

where:

* $N_{i,0}(L) := \sum_{l=1}^{L} 1_{\{j_0^{(l)}=i\}}$: The number of trajectories starting in state i

* $N_i^{(l)}(M)$: The number of visits to state i up to time M of the lth trajectory, $l = 1, \dots, L$

* $N_{ij}^{(l)}(M)$: The number of transitions from state i to state j up to time M during the lth trajectory, $l = 1, \dots, L$

* $N_{ij}(L, M) := \sum_{l=1}^{L} N_{ij}^{(l)}(M)$

* $x_i^{(l,k)}$: The sojourn time in state i during the kth visit, $k = 1, \dots, N_i^{(l)}(M)$ of the lth trajectory, $l = 1, \dots, L$

Note that for $L = 1$ the likelihood (17) reduces to the likelihood of the one trajectory case given in (11).

Under the class of distributions given in (3) the likelihood for L trajectories without censoring becomes:

$$ \mathcal{L} = \prod_{i \in E} \alpha_i^{N_{i,0}(L)} \left( \prod_{l=1}^{L} \prod_{i,j \in E} \alpha_{ij}^{N_{ij}^{(l)}(M)} \right) \prod_{l,i,k} \left\{ \left( F(x_i^{(l,k)}) \right)^{\sum_{m=\bar{N}_{i,k}(L)} a_m} \frac{f(x_i^{(l,k)})}{F(x_i^{(l,k)})} \right\} $$

and we obtain the estimator of $a_{ij}$, namely $\hat{a}_{ij}(L, M)$, which is given by:

$$ \hat{a}_{ij}(L, M) = - \frac{N_{ij}(L, M)}{\sum_{l=1}^{L} \sum_{k=1}^{N_{i,M}^{(l)}(M)} \log(F(X_i^{(l,k)}))} \quad (18) $$

The MLE of the initial law is given by:

$$ \hat{\alpha}_i(L, M) = \frac{N_{i,0}(L)}{L}. \quad (19) $$

### Censoring at the end for Several Trajectories

Let $\{j_0^{(l)}, x_1^{(l)}, j_1^{(l)}, x_2^{(l)}, \dots, j_{N'(M)}^{(l)}, u_{(M)}^{(l)}\}$, $l = 1, \dots, L$, be the L censored sample paths of a semi-Markov process with censoring at time M, then the associated likelihood is:

$$ \mathcal{L} = \left( \prod_{i \in E} \alpha_i^{N_{i,0}(L)} \right) \left( \prod_{i, j \in E} p_{ij}^{\sum_{k=1}^{L} N_{ij}^{(l)}(M)} \right) \times \left( \prod_{l=1}^{L} \prod_{i \in E} \prod_{k=1}^{N_{i,M}^{(l)}(M)} f_{i, \bullet}(x_i^{(l,k)}) \right) \prod_{j \in E} \prod_{k=1}^{\bar{N}_{i,\bullet}^e(L)} W_{i, \bullet}(x_{i,M}^{(k)}) \quad (20) $$

Where:

* $u_M^{(l)} := M - S_{N'(M)}$ is the censored sojourn time in the last visited state of the lth trajectory

* $x_{i,M}^{(k)}$ is the last censored sojourn time in state i during the kth visit, $k = 1, \dots, \bar{N}_{i,\bullet}^e(L)$.

Note that, if the censoring time M in a certain trajectory l is a jump time, then for the corresponding observed censored time we have $u_M^{(l)} = 0$. Consequently, the contribution to the likelihood of the associated term will be equal to 1. For this reason, if no censoring is involved, the uncensored likelihood given in (17) is just a particular case of (20). Note also that for $L = 1$ the likelihood given in (20) becomes the likelihood of one trajectory with censoring, given in (13).

The estimator $\hat{a}_{ij}(L, M)$, under the class of distributions in (3), is established by solving the following equation:

$$ \frac{\partial \log \mathcal{L}}{\partial a_{ij}} = \frac{N_{ij}(L, M)}{a_{ij}} + \sum_{l=1}^{L} \sum_{k=1}^{N_{i,M}^{(l)}(M)} \log(F(x_i^{(l,k)})) - \sum_{k=1}^{\bar{N}_{i,\bullet}^e(L)} \frac{F(x_{i,M}^{(k)}) \sum_{m=\bar{N}_{i,k}(L)} a_m \log F(x_{i,M}^{(k)})}{1 - (F(x_{i,M}^{(k)}))^{\sum_{m=\bar{N}_{i,k}(L)} a_m}} = 0. \quad (21) $$

Regarding the estimator of the initial distribution, given by (19).

### Censoring at the Beginning for Several Trajectories

Let now $\{x_0^{(l)}, j_0^{(l)}, x_1^{(l)}, j_1^{(l)}, x_2^{(l)}, \dots, j_{N'(M)}^{(l)}\}$, $l = 1, \dots, L$, be the L censored sample paths of a semi-Markov process with censoring at the beginning. In this case the likelihood takes the form:
$$ \mathcal{L} = \left( \prod_{i \in E} \alpha_{i}^{N_{i,0}(L)} \right) \prod_{i \in E} \prod_{k=1}^{\bar{N}_{\bullet}^{b}(L)} W_{i, \bullet} \left( x_{i,0}^{(k)} \right) \left( \prod_{i,j \in E} P_{ij}^{N_{ij}(L,M)} \right) \\ \times \left( \prod_{l=1}^{L} \prod_{i \in E} \prod_{k=1}^{N_{i}^{(l)}(M)} f_{i, \bullet} \left( x_{i}^{(l,k)} \right) \right), \qquad (22) $$

where, $x_{i,0}^{(k)}$ is the first censored sojourn time in state i as the first state, during the $k^{th}$ visit, $k = 1, \dots, \bar{N}_{\bullet}^{b}(L)$.

The estimator $\hat{a}_{ij}(L, M)$ in this case, is obtained by solving the following equation:

$$ \frac{\partial \log \mathcal{L}}{\partial a_{ij}} = \frac{N_{ij}(L, M)}{a_{ij}} + \sum_{l=1}^{L} \sum_{k=1}^{N_{ij}^{(l)}(M)} \log(F(x_{i}^{(l,k)})) \\ - \sum_{k=1}^{\bar{N}_{\bullet}^{b}(L)} \frac{F(x_{i,0}^{(k)}) \sum_{m \in E} a_{m}}{1 - (F(x_{i,0}^{(k)}))^{\sum_{m \in E} a_{m}}} = 0. \qquad (23) $$

## Censoring at the Beginning and at the End for Several Trajectories

We can combine the cases considered in Sections 3.2.2 and 3.2.3 so that all trajectories have censoring both at the beginning and at the end. Note through that, from practical point of view, it is important to properly write the case where only several trajectories have censoring at the beginning and/or at the end. This will be done in this section.

Consider the case of L trajectories where some of the sojourn times are censored either at the beginning or/and at the end. Given L censored sample paths of a semi-Markov process:

$$ \{x_0^{(l)}, j_0^{(l)}, x_1^{(l)}, j_1^{(l)}, x_2^{(l)}, \dots, j_{N'(M)}^{(l)}, u_M^{(l)}, \dots\}, l = 1, \dots, L, $$

Where:

$$ \delta_b^{(l)} = \begin{cases} 1, & \text{if the first sojourn time, of the } l^{\text{th}} \text{ trajectory, is censored,} \\ 0, & \text{if the first sojourn time, of the } l^{\text{th}} \text{ trajectory, is not censored.} \end{cases} $$

$$ \delta_e^{(l)} = \begin{cases} 1, & \text{if the last sojourn time, of the } l^{\text{th}} \text{ trajectory, is censored,} \\ 0, & \text{if the last sojourn time, of the } l^{\text{th}} \text{ trajectory, is not censored.} \end{cases} $$

The likelihood in this case can be written as:

$$ \mathcal{L} = \left( \prod_{i \in E} \alpha_{i}^{N_{i,0}(L)} \right) \left( \prod_{i,j \in E} P_{ij}^{N_{ij}^{(l)}(M)} \right) \left( \prod_{l=1}^{L} \prod_{i \in E} \prod_{k=1}^{N_{ij}^{(l)}(M)} f_{i, \bullet} \left( x_{i}^{(l,k)} \right) \right) \\ \times \left( \prod_{i \in E} \prod_{k=1}^{\bar{N}_{\bullet}^{b}(L)} \bar{W}_{i, \bullet} \left( x_{i,0}^{(k)} \right) \right) \left( \prod_{i \in E} \prod_{k=1}^{\bar{N}_{\bullet}^{b}(L)} \bar{W}_{i, \bullet} \left( x_{i,M}^{(k)} \right) \right). \qquad (24) $$

Note that, in the above likelihood, due to the appearance of the counting processes $\bar{N}_{\bullet}^{b}(L)$ and $\bar{N}_{\bullet}^{c}(L)$, we do not take into account all the trajectories but only those that have censoring at the beginning and/or at the end. All others without censoring, are included in the term $\prod_{l=1}^{L} \prod_{i \in E} \prod_{k=1}^{N_{ij}^{(l)}(M)} f_{i, \bullet} \left( x_{i}^{(l,k)} \right)$.

For the class of distributions given in (3), the logarithm of the likelihood given in (24) takes the form:

$$ \log \mathcal{L} = \sum_{i \in E} N_{i,0}(L) \log \alpha_i + \sum_{l=1}^{L} \sum_{i, j \in E} N_{ij}^{(l)}(M) \log a_{ij} \\ + \sum_{l=1}^{L} \sum_{k=1}^{N_{ij}^{(l)}(M)} \sum_{i, j \in E} a_{ij} \log(F(x_{i}^{(l,k)})) \\ + \log \left( \prod_{i, j, k} \frac{F(x_{i}^{(l,k)})}{F(x_{i}^{(k)})} \right) + \sum_{i \in E} \sum_{k=1}^{\bar{N}_{\bullet}^{b}(L)} \log \left( 1 - (F(x_{i,0}^{(k)}))^{\sum_{m \in E} a_{m}} \right) \\ + \sum_{i \in E} \sum_{k=1}^{\bar{N}_{\bullet}^{c}(L)} \log \left( 1 - (F(x_{i,M}^{(k)}))^{\sum_{m \in E} a_{m}} \right). \qquad (25) $$

The estimator $\hat{a}_{ij}(L, M)$ is obtained by solving the following equation:

$$ \frac{\partial \log \mathcal{L}}{\partial a_{ij}} = \frac{N_{ij}(L, M)}{a_{ij}} + \sum_{l=1}^{L} \sum_{k=1}^{N_{ij}^{(l)}(M)} \log(F(x_{i}^{(l,k)})) \\ - \sum_{k=1}^{\bar{N}_{\bullet}^{b}(L)} \frac{F(x_{i,0}^{(k)}) \sum_{m \in E} a_{m} \log(F(x_{i,0}^{(k)}))}{1 - (F(x_{i,0}^{(k)}))^{\sum_{m \in E} a_{m}}} \\ + \sum_{k=1}^{\bar{N}_{\bullet}^{c}(L)} \frac{F(x_{i,M}^{(k)}) \sum_{m \in E} a_{m} \log(F(x_{i,M}^{(k)}))}{1 - (F(x_{i,M}^{(k)}))^{\sum_{m \in E} a_{m}}} = 0. \qquad (26) $$

Among the estimators of $a_{ij}$, one should choose, according to the case under investigation, the appropriate expression for the determination of the estimators of $p_{ij}$, $W_i$ and $Q_{ij}$, namely:

$$ \hat{p}_{ij}(L, M) = \frac{\hat{a}_{ij}(L, M)}{\sum_{m \in E} \hat{a}_{im}(L, M)} = \frac{N_{ij}(L, M)}{N_i(L, M)}, \qquad (27) $$

$$ \hat{W}_i (t; L, M) = [F(t)]^{\sum_{m \in E} \hat{a}_{m}(L, M)} \qquad (28) $$

and:

$$ \hat{Q}_{ij}(t; L, M) = \frac{\hat{a}_{ij}(L, M)}{\sum_{m \in E} \hat{a}_{im}(L, M)} [F(t)]^{\sum_{m \in E} \hat{a}_{m}(L, M)}. \qquad (29) $$
$$ \mathcal{L} = \left( \prod_{i \in E} \alpha_{i}^{N_{i,0}(L)} \right) \prod_{i \in E} \prod_{k=1}^{\bar{N}_{\bullet}^{b}(L)} W_{i, \bullet} \left( x_{i,0}^{(k)} \right) \left( \prod_{i,j \in E} P_{ij}^{N_{ij}(L,M)} \right) \\ \times \left( \prod_{l=1}^{L} \prod_{i \in E} \prod_{k=1}^{N_{i}^{(l)}(M)} f_{i, \bullet} \left( x_{i}^{(l,k)} \right) \right), \qquad (22) $$

where, $x_{i,0}^{(k)}$ is the first censored sojourn time in state i as the first state, during the kth visit, $k = 1, \dots, \bar{N}_{\bullet}^{b}(L)$.

The estimator $\hat{a}_{ij}(L, M)$ in this case, is obtained by solving the following equation:

$$ \frac{\partial \log \mathcal{L}}{\partial a_{ij}} = \frac{N_{ij}(L, M)}{a_{ij}} + \sum_{l=1}^{L} \sum_{k=1}^{N_{ij}^{(l)}(M)} \log(F(x_{i}^{(l,k)})) \\ - \sum_{k=1}^{\bar{N}_{\bullet}^{b}(L)} \frac{F(x_{i,0}^{(k)}) \sum_{m \in E} a_{m}}{1 - (F(x_{i,0}^{(k)}))^{\sum_{m \in E} a_{m}}} = 0. \qquad (23) $$

### Censoring at the Beginning and at the End for Several Trajectories

We can combine the cases considered in Sections 3.2.2 and 3.2.3 so that all trajectories have censoring both at the beginning and at the end. Note through that, from practical point of view, it is important to properly write the case where only several trajectories have censoring at the beginning and/or at the end. This will be done in this section.

Consider the case of L trajectories where some of the sojourn times are censored either at the beginning or/and at the end. Given L censored sample paths of a semi-Markov process:

$$ \{x_0^{(l)}, j_0^{(l)}, x_1^{(l)}, j_1^{(l)}, x_2^{(l)}, \dots, j_{N'(M)}^{(l)}, u_M^{(l)}, \dots\}, l = 1, \dots, L, $$

Where:

$$ \delta_b^{(l)} = \begin{cases} 1, & \text{if the first sojourn time, of the lth trajectory, is censored,} \\ 0, & \text{if the first sojourn time, of the lth trajectory, is not censored.} \end{cases} $$

$$ \delta_e^{(l)} = \begin{cases} 1, & \text{if the last sojourn time, of the lth trajectory, is censored,} \\ 0, & \text{if the last sojourn time, of the lth trajectory, is not censored.} \end{cases} $$

The likelihood in this case can be written as:

$$ \mathcal{L} = \left( \prod_{i \in E} \alpha_{i}^{N_{i,0}(L)} \right) \left( \prod_{i,j \in E} P_{ij}^{N_{ij}^{(l)}(M)} \right) \left( \prod_{l=1}^{L} \prod_{i \in E} \prod_{k=1}^{N_{ij}^{(l)}(M)} f_{i, \bullet} \left( x_{i}^{(l,k)} \right) \right) \\ \times \left( \prod_{i \in E} \prod_{k=1}^{\bar{N}_{\bullet}^{b}(L)} \bar{W}_{i, \bullet} \left( x_{i,0}^{(k)} \right) \right) \left( \prod_{i \in E} \prod_{k=1}^{\bar{N}_{\bullet}^{b}(L)} \bar{W}_{i, \bullet} \left( x_{i,M}^{(k)} \right) \right). \qquad (24) $$

Note that, in the above likelihood, due to the appearance of the counting processes $\bar{N}_{\bullet}^{b}(L)$ and $\bar{N}_{\bullet}^{c}(L)$, we do not take into account all the trajectories but only those that have censoring at the beginning and/or at the end. All others without censoring, are included in the term $\prod_{l=1}^{L} \prod_{i \in E} \prod_{k=1}^{N_{ij}^{(l)}(M)} f_{i, \bullet} \left( x_{i}^{(l,k)} \right)$.

For the class of distributions given in (3), the logarithm of the likelihood given in (24) takes the form:

$$ \log \mathcal{L} = \sum_{i \in E} N_{i,0}(L) \log \alpha_i + \sum_{l=1}^{L} \sum_{i, j \in E} N_{ij}^{(l)}(M) \log a_{ij} \\ + \sum_{l=1}^{L} \sum_{k=1}^{\bar{N}_{\bullet}^{b}(L)} \sum_{i, j \in E} a_{ij} \log(F(x_{i}^{(l,k)})) \\ + \log\left(\prod_{i, j, k} \frac{f(x_{i}^{(l,k)})}{F(x_{i}^{(l,k)})}\right) + \sum_{i \in E} \sum_{k=1}^{\bar{N}_{\bullet}^{c}(L)} \log\left(1 - (F(x_{i,0}^{(k)})^{\sum_{m \in E} a_{m}})\right) \\ + \sum_{i \in E} \sum_{k=1}^{\bar{N}_{\bullet}^{c}(L)} \log\left(1 - (F(x_{i,M}^{(k)})^{\sum_{m \in E} a_{m}})\right). \qquad (25) $$

The estimator $\hat{a}_{ij}(L, M)$ is obtained by solving the following equation:

$$ \frac{\partial \log \mathcal{L}}{\partial a_{ij}} = \frac{N_{ij}(L, M)}{a_{ij}} + \sum_{l=1}^{L} \sum_{k=1}^{N_{ij}^{(l)}(M)} \log(F(x_{i}^{(l,k)})) \\ - \sum_{k=1}^{\bar{N}_{\bullet}^{b}(L)} \frac{F(x_{i,0}^{(k)}) \sum_{m \in E} a_{m} \log(F(x_{i,0}^{(k)}))}{1 - (F(x_{i,0}^{(k)}))^{\sum_{m \in E} a_{m}}} \\ + \sum_{k=1}^{\bar{N}_{\bullet}^{c}(L)} \frac{F(x_{i,M}^{(k)}) \sum_{m \in E} a_{m} \log(F(x_{i,M}^{(k)}))}{1 - (F(x_{i,M}^{(k)}))^{\sum_{m \in E} a_{m}}} = 0. \qquad (26) $$

Among the estimators of $a_{ij}$, one should choose, according to the case under investigation, the appropriate expression for the determination of the estimators of $p_{ij}$, $W_i$ and $Q_{ij}$, namely:

$$ \hat{p}_{ij}(L, M) = \frac{\hat{a}_{ij}(L, M)}{\sum_{m \in E} \hat{a}_{im}(L, M)} = \frac{N_{ij}(L, M)}{N_i(L, M)}, \qquad (27) $$

$$ \hat{W}_i(t; L, M) = [F(t)]^{\sum_{m \in E} \hat{a}_{im}(L, M)} \qquad (28) $$

and:

$$ \hat{Q}_{ij}(t; L, M) = \frac{\hat{a}_{ij}(L, M)}{\sum_{m \in E} \hat{a}_{im}(L, M)} [F(t)]^{\sum_{m \in E} \hat{a}_{im}(L, M)}. \qquad (29) $$
Note that for $L = 1$, $\hat{a}_{ij}(1, M) = \hat{a}_{ij}(M)$.

For the uncensored case the above estimators have explicit forms by combining (27) - (29) with (12) - (18) depending on the number of trajectories. Furthermore, the consistency of the resulted estimators in the case of no censoring can be established due to the explicit expression of the associated estimators. In such cases the following proposition holds:

### Proposition 2

For all $I, j \in E$, the estimators $\hat{a}_{ij}(M)$ of $a_{ij}$ given in (12), $\hat{a}_{ij}(L, M)$ of $a_{ij}$ given in (18) and $\hat{\alpha}_i(L, M)$ of $\alpha_i$ given in (19), are strongly consistent, i.e.:

$$ 
\begin{aligned}
\hat{a}_{ij}(M) &\xrightarrow{\frac{a.s.}{M \to \infty}} a_{ij}, \\
\hat{a}_{ij}(L, M) &\xrightarrow{\frac{a.s.}{L \to \infty}} a_{ij}, \\
\hat{\alpha}_i(L, M) &\xrightarrow{\frac{a.s.}{L \to \infty}} \alpha_i.
\end{aligned}
$$

### Proof

The proof follows along the lines of the proof of Theorem 1 (Barbu et al., 2017) by taking under consideration the convergence of the empirical estimators to the true transition probabilities of the embedded Markov chain, the law of large numbers and the fact that the MRC is regular.

The following Corollary follows immediately from the Proposition above.

### Corollary 1

For all $I, j \in E$ and $t > 0$ and under the setting of this Section, for $L$ sample paths, $L \ge 2$ of length $M$ of a semi-Markov process, the estimators of $Q_{ij}(t)$, $p_{ij}$, $W_i(t)$, $f_i(t)$ obtained by considering the estimators $\hat{a}_{ij}(L, M)$ of $a_{ij}$ given in (18) and $\hat{\alpha}_i(L, M)$ of $\alpha_i$ given in (19), are strongly consistent, as $L$ goes to infinity.

The above Corollary holds also for $L = 1$ with $\hat{a}_{ij}(M)$ given in (12) when $M$ tends to infinity.

## Special Case for the Class of Distribution Closed Under Minima

In this Section, a similar framework as in Barbu et al. (2017) is considered in order to investigate the case of censoring at the beginning for the following class of distribution functions which is closed under minima (30). More precisely, in the cited paper we have considered the case of no censoring and the case of censoring at time $M$ for both situations of one sample path and several sample paths. However, here we deal with the general case where some of the sojourn times

are censored either at the beginning and/or at the end for several trajectories.

Similarly, as in the case of class (3), the class of distributions given by:

$$ F(t; a) := 1 - (1 - F(t; 1))^a, \qquad (30) $$

is absolutely continuous w.r.t the Lebesgue measure and with density $f(t; a)$.

Members of the above class if distributions is the Geometric distribution, the Pareto distribution, the Weibull distribution, the Exponential, the Rayleigh and the Erlang truncated exponential.

The following Lemma shows that the class (30) is closed under minima (cf. Balasubramanian et al., 1991).

### Lemma 2

Let $X_1, \dots, X_N$ be independent random variables such that $X_i \sim F(x; a_i)$ which belongs to class (30). Then the distribution function $F^{(1)}$ of the minimum order statistic $X_1$ belongs also to (30).

The evolution of the model, in this case, is governed by the following characteristic: The minimum value of $T_{il}$ identifies the next visited state, assuming that the process was in state $i$. Thus, under this setting the semi-Markov kernel takes the form (cf. Barbu et al., 2017):

$$ Q_{ij}(t) = p_{ij}W_{i}(t), $$

where:

$$ p_{ij} = \mathbb{P}(J_{n-1} = j | J_{n-1} = i) = \mathbb{P}(T_{ij} \le T_{il}, \forall l | J_{n-1} = i) $$

and:

$$ W_{ij}(t) := W_{i}(t), \text{ independent of } j, $$

which represents the cdf of the sojourn time in state $i$ (unconditional to the next state to be visited).

Similarly, as before, it holds that:

$$ \sum_{j} Q_{ij}(t) = W_{i}(t). $$

### Proposition 3

According to the setup of this section, the following quantities can be derived:

$$ Q_{ij}(t) = \frac{a_{ij}}{\sum_{k \in E} a_{ik}} \left[ 1 - (1 - F(t))^{\sum_{k \in E} a_{ik}} \right], \qquad (31) $$
Note that for $L = 1$, $\hat{a}_{ij}(1, M) = \hat{a}_{ij}(M)$.

For the uncensored case the above estimators have explicit forms by combining (27) - (29) with (12) - (18) depending on the number of trajectories. Furthermore, the consistency of the resulted estimators in the case of no censoring can be established due to the explicit expression of the associated estimators. In such cases the following proposition holds:

### Proposition 2

For all $I, j \in E$, the estimators $\hat{a}_{ij}(M)$ of $a_{ij}$ given in (12), $\hat{a}_{ij}(L, M)$ of $a_{ij}$ given in (18) and $\hat{\alpha}_i(L, M)$ of $\alpha_i$ given in (19), are strongly consistent, i.e.:

$$ \hat{a}_{ij}(M) \xrightarrow{\frac{a.s.}{M \to \infty}} a_{ij}, \\ \hat{a}_{ij}(L, M) \xrightarrow{\frac{a.s.}{L \to \infty}} a_{ij}, \\ \hat{\alpha}_i(L, M) \xrightarrow{\frac{a.s.}{L \to \infty}} \alpha_i. $$

### Proof

The proof follows along the lines of the proof of Theorem 1 (Barbu et al., 2017) by taking under consideration the convergence of the empirical estimators to the true transition probabilities of the embedded Markov chain, the law of large numbers and the fact that the MRC is regular.

The following Corollary follows immediately from the Proposition above.

### Corollary 1

For all $I, j \in E$ and $t > 0$ and under the setting of this Section, for $L$ sample paths, $L \ge 2$ of length $M$ of a semi-Markov process, the estimators of $Q_{ij}(t)$, $p_{ij}$, $W_i(t)$, $f_i(t)$ obtained by considering the estimators $\hat{a}_{ij}(L, M)$ of $a_{ij}$ given in (18) and $\hat{\alpha}_i(L, M)$ of $\alpha_i$ given in (19), are strongly consistent, as $L$ goes to infinity.

The above Corollary holds also for $L = 1$ with $\hat{a}_{ij}(M)$ given in (12) when $M$ tends to infinity.

## Special Case for the Class of Distribution Closed Under Minima

In this Section, a similar framework as in Barbu et al. (2017) is considered in order to investigate the case of censoring at the beginning for the following class of distribution functions which is closed under minima (30). More precisely, in the cited paper we have considered the case of no censoring and the case of censoring at time $M$ for both situations of one sample path and several sample paths. However, here we deal with the general case where some of the sojourn times

are censored either at the beginning and/or at the end for several trajectories.

Similarly, as in the case of class (3), the class of distributions given by:

$$ F(t; a) := 1 - (1 - F(t; 1))^a, \qquad (30) $$

is absolutely continuous w.r.t the Lebesgue measure and with density $f(t; a)$.

Members of the above class if distributions is the Geometric distribution, the Pareto distribution, the Weibull distribution, the Exponential, the Rayleigh and the Erlang truncated exponential.

The following Lemma shows that the class (30) is closed under minima (cf. Balasubramanian et al., 1991).

### Lemma 2

Let $X_1, \dots, X_N$ be independent random variables such that $X_i \sim F(x; \alpha_i)$ which belongs to class (30). Then the distribution function $F^{(1)}$ of the minimum order statistic $X_{(1)}$ belongs also to (30).

The evolution of the model, in this case, is governed by the following characteristic: The minimum value of $T_{il}$ identifies the next visited state, assuming that the process was in state $i$. Thus, under this setting the semi-Markov kernel takes the form (cf. Barbu et al., 2017):

$$ Q_{ij}(t) = p_{ij}W_{i\cdot}(t), $$

where:

$$ p_{ij} = \mathbb{P}(J_{n-1} = j | J_{n-1} = i) = \mathbb{P}(T_{ij} \le T_{il}, \forall l | J_{n-1} = i) $$

and:

$$ W_{ij}(t) := W_{i\cdot}(t), \text{ independent of } j, $$

which represents the cdf of the sojourn time in state $i$ (unconditional to the next state to be visited).

Similarly, as before, it holds that:

$$ \sum_j Q_{ij}(t) = W_{i\cdot}(t). $$

### Proposition 3

According to the setup of this section, the following quantities can be derived:

$$ Q_{ij}(t) = \frac{a_{ij}}{\sum_{k \in E} a_{ik}} \left[ 1 - (1 - F(t))^{\sum_{k \in E} a_{ik}} \right], \qquad (31) $$
$$p_{ij} = \frac{a_{ij}}{\sum_{m \in E} a_{im}}, \qquad (32)$$

$$W_{i, \bullet}(t) = 1 - [1 - F(t)]^{\sum_{m=1}^{N} \alpha_{im}} \qquad (33)$$

and:

$$f_{i, \bullet}(t) = \sum_{m=1}^{N} a_{im} (1-F(t))^{\sum_{k=1}^{N} \alpha_{ik}} \frac{f(t)}{1-F(t)}. \qquad (34)$$

Consider the case of $L$ trajectories where some of the sojourn times are censored either at the beginning or/and at the end. Recall that $\delta_b^{(l)}$ and $\delta_e^{(l)}$ are indicators of censored for the beginning and for the end respectively. Given $L$ censored sample paths of a semi-Markov process:

$$\{x_0^{(l)}, j_0^{(l)}, x_1^{(l)}, j_1^{(l)}, x_2^{(l)}, \dots, j_{N'(M)}^{(l)}, u_M^{(l)}\}, l = 1, \dots, L,$$

then the associated likelihood:

$$\mathcal{L} = \left( \prod_{i \in E} \alpha_i^{N_{i,0}(L)} \right) \left( \prod_{i,j \in E} P_{ij}^{\sum_{l=1}^{L} N_{ij}^{(l)}(M)} \right) \left( \prod_{l=1}^{L} \prod_{i \in E} \prod_{k=1}^{N_{ij}^{(l)}(M)} f_{i, \bullet}(x_i^{(l,k)}) \right) \times \left( \prod_{i \in E} \prod_{k=1}^{\bar{N}_i^{(L)}} \bar{W}_{i, \bullet}(x_i^{(k)}) \right) \left( \prod_{i \in E} \prod_{k=1}^{\bar{N}_i^{(L)}} \bar{W}_{i, \bullet}(x_{i,M}^{(k)}) \right). \qquad (35)$$

For the class of distributions given in (30), the above likelihood takes the form:

$$\mathcal{L} = \left( \prod_{i \in E} \alpha_i^{N_{i,0}(L)} \right) \left( \prod_{l=1}^{L} \prod_{j,l \in E} a_{ij}^{N_{ij}^{(l)}(M)} \right) \times \prod_{i,j,k} \left[ (1-F(x_i^{(l,k)}))^{\sum_{m \in E} a_m} \frac{f(x_j^{(l,k)})}{1-F(x_j^{(l,k)})} \right] \times \left( \prod_{i \in E} \prod_{k=1}^{\bar{N}_i^{b}(L)} \left[ (1-F(x_{i,0}^{(k)})^{\sum_{m \in E} a_m}) \right] \left( \prod_{j \in E} \prod_{k=1}^{\bar{N}_j^{b}(L)} \left[ (1-F(x_{j,M}^{(k)})^{\sum_{m \in E} a_m}) \right] \right) \right]. \qquad (36)$$

Therefore, the estimator $\hat{a}_{ij}(L, M)$ in this case becomes:

$$\hat{a}_{ij}(L, M) = - \frac{N_{ij}(L, M)}{B_i(L, M) + C_i^b(L, M) + C_i^e(L, M)}, \qquad (37)$$

where:

$$B_i(L, M) := \sum_{l=1}^{L} \sum_{k=1}^{N_{ij}^{(l)}(M)} \log(1-F(X_i^{(l,k)})),$$

$$C_i^b(L, M) := \sum_{k=1}^{\bar{N}_i^b(L)} \log(1-F(X_{i,0}^{(k)})).$$

and:

$$C_i^e(L, M) := \sum_{k=1}^{\bar{N}_i^e(L)} \log(1-F(X_{i,M}^{(k)})).$$

The estimator for the initial law, as in the previous class, is described by:

$$\hat{a}_i(L, M) = \frac{N_{i,0}(L)}{L}. \qquad (38)$$

### Remark 1

In the case of no censoring, the estimator in (37) is reduced to the estimator given in Barbu et al. (2017):

$$\hat{a}_{ij}(L, M) = - \frac{N_{ij}(L, M)}{\sum_{l=1}^{L} \sum_{k=1}^{N_{ij}^{(l)}(M)} \log(1-F(X_i^{(l,k)}))} \qquad (39)$$

Note that the proper form of the estimators given in Barbu et al. (2017), in the case of $L$ trajectories with censoring at time $M$ is as follows:

$$\hat{a}_{ij}(L, M) = - \frac{N_{ij}(L, M)}{B_i(L, M) + C_i(L, M)},$$

where, $C_i(L, M)$ is in place of $C(L, M)$.

The consistency of the above estimators is provided in the Proposition below.

### Proposition 4

For all $i, j \in E$, the estimators $\hat{a}_{ij}(L, M)$ of $a_{ij}$ given in (37) is strongly consistent, i.e.:

$$\hat{a}_{ij}(L, M) \xrightarrow{a.s.} a_{ij}. $$

The proof is omitted due to the similarity with the proof of Theorem 1 (Barbu et al., 2017) and Proposition 2 at the end of Section 3.

## Markov Renewal Function and Semi-Markov Transition Matrix

The Markov renewal function and the semi-Markov transition function are furnished and discussed in this section.

The Markov renewal function $\Psi_{ij}(t)$, $i, j \in E$, $t \ge 0$, is defined as (cf. Limnios and Opri□an, 2001):

$$\begin{aligned} \Psi_{ij}(t) &:= \mathbb{E}_i [N_j(t)] = \sum_{n=0}^{\infty} Q_{ij}^{(n)}(t) \\ &= \sum_{n=0}^{\infty} \sum_{k \in E} \int_0^{\infty} Q_{ik}(ds) Q_{kj}^{(n-1)}(t-s). \end{aligned} \qquad (40)$$
$$p_{ij} = \frac{a_{ij}}{\sum_{m \in E} a_{im}}, \qquad (32)$$

$$W_{i, \bullet}(t) = 1 - [1 - F(t)]^{\sum_{m=1}^{N} \alpha_{im}} \qquad (33)$$

and:

$$f_{i, \bullet}(t) = \sum_{m=1}^{N} a_{im} (1-F(t))^{\sum_{k=1}^{N} \alpha_{im}} \frac{f(t)}{1-F(t)}. \qquad (34)$$

Consider the case of L trajectories where some of the sojourn times are censored either at the beginning or/and at the end. Recall that $\delta_b^{(l)}$ and $\delta_e^{(l)}$ are indicators of censored for the beginning and for the end respectively. Given L censored sample paths of a semi-Markov process:

$$\{x_0^{(l)}, j_0^{(l)}, x_1^{(l)}, j_1^{(l)}, x_2^{(l)}, \dots, j_{N'(M)}^{(l)}, u_M^{(l)}\}, l = 1, \dots, L,$$

then the associated likelihood:

$$\mathcal{L} = \left( \prod_{i \in E} \alpha_i^{N_{i,0}(L)} \right) \left( \prod_{i,j \in E} P_{ij}^{\sum_{l=1}^{L} N_{ij}^{(l)}(M)} \right) \left( \prod_{l=1}^{L} \prod_{i \in E} \prod_{k=1}^{N_{ij}^{(l)}(M)} f_{i, \bullet}(x_i^{(l,k)}) \right) \times \left( \prod_{i \in E} \prod_{k=1}^{\bar{N}_i^{(L)}} \bar{W}_{i, \bullet}(x_i^{(k)}) \right) \left( \prod_{i \in E} \prod_{k=1}^{\bar{N}_i^{(L)}} \bar{W}_{i, \bullet}(x_{i,M}^{(k)}) \right). \qquad (35)$$

For the class of distributions given in (30), the above likelihood takes the form:

$$\mathcal{L} = \left( \prod_{i \in E} \alpha_i^{N_{i,0}(L)} \right) \left( \prod_{l=1}^{L} \prod_{j,l \in E} a_{ij}^{N_{ij}^{(l)}(M)} \right) \times \prod_{l,j,k} \left[ (1-F(x_{j,l}^{(k)}))^{\sum_{m \in E} a_m} \frac{f(x_{j,l}^{(k)})}{1-F(x_{j,l}^{(k)})} \right] \times \left( \prod_{i \in E} \prod_{k=1}^{\bar{N}_i^{b}(L)} \left[ (1-F(x_{i,0}^{(k)})^{\sum_{m \in E} a_m}) \right] \left( \prod_{j \in E} \prod_{k=1}^{\bar{N}_j^{b}(L)} \left[ (1-F(x_{j,M}^{(k)})^{\sum_{m \in E} a_m}) \right] \right) \right]. \qquad (36)$$

Therefore, the estimator $\hat{a}_{ij}(L, M)$ in this case becomes:

$$\hat{a}_{ij}(L, M) = - \frac{N_{ij}(L, M)}{B_i(L, M) + C_i^b(L, M) + C_i^e(L, M)}, \qquad (37)$$

where:

$$B_i(L, M) := \sum_{l=1}^{L} \sum_{k=1}^{N_{ij}^{(l)}(M)} \log(1-F(X_{i,l}^{(k)})),$$

$$C_i^b(L, M) := \sum_{k=1}^{\bar{N}_i^b(L)} \log(1-F(X_{i,0}^{(k)})).$$

and:

$$C_i^e(L, M) := \sum_{k=1}^{\bar{N}_i^e(L)} \log(1-F(X_{i,M}^{(k)})).$$

The estimator for the initial law, as in the previous class, is described by:

$$\hat{a}_i(L, M) = \frac{N_{i,0}(L)}{L}. \qquad (38)$$

### Remark 1

In the case of no censoring, the estimator in (37) is reduced to the estimator given in Barbu et al. (2017):

$$\hat{a}_{ij}(L, M) = - \frac{N_{ij}(L, M)}{\sum_{l=1}^{L} \sum_{k=1}^{N_{ij}^{(l)}(M)} \log(1-F(X_{i,l}^{(l,k)}))} \qquad (39)$$

Note that the proper form of the estimators given in Barbu et al. (2017), in the case of L trajectories with censoring at time M is as follows:

$$\hat{a}_{ij}(L, M) = - \frac{N_{ij}(L, M)}{B_i(L, M) + C_i(L, M)},$$

where, $C_i(L, M)$ is in place of $C(L, M)$.

The consistency of the above estimators is provided in the Proposition below.

### Proposition 4

For all $i, j \in E$, the estimators $\hat{a}_{ij}(L, M)$ of $a_{ij}$ given in (37) is strongly consistent, i.e.:

$$\hat{a}_{ij}(L, M) \xrightarrow{a.s.} a_{ij}. $$

The proof is omitted due to the similarity with the proof of Theorem 1 (Barbu et al., 2017) and Proposition 2 at the end of Section 3.

## Markov Renewal Function and Semi-Markov Transition Matrix

The Markov renewal function and the semi-Markov transition function are furnished and discussed in this section.

The Markov renewal function $\Psi_{ij}(t)$, $i, j \in E$, $t \ge 0$, is defined as (cf. Limnios and Opri□an, 2001):

$$\begin{aligned} \Psi_{ij}(t) &:= \mathbb{E}_i [N_j(t)] = \sum_{n=0}^{\infty} Q_{ij}^{(n)}(t) \\ &= \sum_{n=0}^{\infty} \sum_{k \in E} \int_0^{\infty} Q_{ik}(ds) Q_{kj}^{(n-1)}(t-s). \end{aligned} \qquad (40)$$
where, $Q_{ij}^{(n)}$ denotes the $n^{th}$ convolution of $Q$ by itself, namely:

$$Q_{ij}^{(n)}(t) := \begin{cases} \sum_{k \in E} \int_{0}^{t} Q_{ik}(s) Q_{kj}^{(n-1)}(t-s) ds & n \ge 2, \\ Q_{ij}(t), & n = 1, \\ \delta_{ij} \mathbf{1}_{\{t \ge 0\}}, & n = 0. \end{cases}$$

For the class of distributions closed under maxima and according to (4), the Markov renewal function takes the form:

$$\Psi_{ij}(t) = \sum_{n=0}^{\infty} \sum_{k \in E} \int_{0}^{t} \left( \alpha_{ik}(F(s))^{\sum_{m \in E} a_m^{-1}} f(s) \right) Q_{kj}^{(n-1)}(t-s) ds. \quad (41)$$

The semi-Markov transition matrix (function) is defined as:

$$P_{ij}(t) := \mathbb{P}(Z_t = j | Z_0 = i), i, j \in E. \quad (42)$$

Limnios and Opri□an (2001) proved that the semi-Markov transition function satisfies the following Markov renewal equation (MRE):

$$P(t) = I_N - \mathrm{diag} \left( \sum_j Q_{ij}(t); i \in E \right) + (Q \star P)(t),$$

where, $I_N$ is the $N \times N$ identity matrix, ★ represents the convolution and "diag" is a diagonal matrix.

Let $W(t) := \mathrm{diag}(W_i(t); i \in E) = \mathrm{diag}(\sum_j Q_{ij}(t); i \in E) = \mathrm{diag}(Q \cdot 1_N)(t)$ be the diagonal matrix where the $(i, i)$ element is $W_i(t) = \sum_{j \in E} Q_{ij}(t)$ and $1_N = \frac{(1, \dots, 1)^T}{N}, (\quad)^T$ denotes the transposed of a vector.

Limnios and Opri□an (2001) established the unique solution of the above MRE:

$$P(t) = \left( (I_N - Q)^{(-1)} \star (I_N - W) \right) (t) = \left( \Psi \star (I_N - W) \right) (t), \quad (43)$$

where, $\Psi(t) = (\Psi_{ij}(t))_{i,j \in E}$ and it is shown that $(I_N - Q)^{(-1)}(t) = \Psi(t)$, where $\Psi^{(-1)}$ denotes the inverse convolution function.

Let us consider the case of several trajectories where we may have censoring at the beginning and/or at the end. Then, the estimators $\hat{P}_{ij}(t)$ and $\hat{\Psi}_{ij}(t)$, $i, j \in E, t > 0$, are given by:

$$\hat{P}_{ij}(t; L, M) = \left( \hat{\Psi}_{ij} \star \left( 1 - \sum_{k \in E} \hat{Q}_{jk} \right) \right) (t; L, M) \quad (44)$$

and:

$$\begin{aligned} \hat{\Psi}_{ij}(t; L, M) &= \sum_{n=0}^{\infty} \hat{Q}_{ij}^{(n)}(t; L, M) \\ &= \sum_{n=0}^{\infty} \sum_{k \in E} \int_{0}^{t} \hat{Q}_{ik}(s; L, M) \hat{Q}_{kj}^{(n-1)}(t-s; L, M) ds, \end{aligned} \quad (45)$$

where the appropriate estimator for $a_{ij}$ obtained in the previous section, is chosen for deriving $\hat{Q}_{ij}^{(n)}(t; L, M)$.

## Reliability/Survival Analysis Indicators

For the purpose of this section we assume that the state space $E$ can be split into two subsets, $U$ (containing the functioning states) and $D$ (containing the failure states), such as $E = U \cup D$ and $E = U \cap D = \emptyset$, where let say that $U = \{1, \dots, n\}$ and $D = \{n+1, \dots, N\}$. Each matrix can be partitioned according to this state space partition.

This section is devoted to the construction of estimators for a variety of reliability indicators in the case of several censored paths at the beginning and/or at the end (Limnios and Ouhbi, 2003).

### Reliability

The reliability at time $t$, $R(t)$, is defined by:

$$R(t) = \mathbb{P}(T_D > t) = \mathbb{P}(Z_s \in U, s \le t),$$

where, $T_D := \inf\{t | Z_t \in D\}$ is the lifetime of the system. The following result presents the estimator of the reliability of a semi-Markov system in terms of estimators of basic quantities of a semi-Markov process.

### Proposition 5

For a semi-Markov system, the estimator of the reliability at time $t > 0$ is given by (cf. Ouhbi and Limnios, 1996):

$$\hat{R}(t; L, M) = \hat{\alpha}_U(L, M) \hat{P}_{UU}(t; L, M) 1_n, \quad (46)$$

where, $\hat{\alpha}_U(L, M)$ is an estimator of $\alpha_U$ and $b \hat{P}_{UU}(t; L, M)$ is an estimator of $P_{UU}(t)$.

### Lemma 3

The estimator of the failure rate of the system is given by:

$$\hat{\lambda}(t; L, M) := - \frac{\hat{R}'(t; L, M)}{\hat{R}(t; L, M)}, \quad t > 0. \quad (47)$$

### Availability

The availability of the system, $A(t)$, is defined as:
where, $Q_{ij}^{(n)}$ denotes the $n^{th}$ convolution of $Q$ by itself, namely:

$$Q_{ij}^{(n)}(t) := \begin{cases} \sum_{k \in E} \int_{0}^{t} Q_{ik}(s) Q_{kj}^{(n-1)}(t-s) ds & n \ge 2, \\ Q_{ij}(t), & n = 1, \\ \delta_{ij} \mathbf{1}_{\{t \ge 0\}}, & n = 0. \end{cases}$$

For the class of distributions closed under maxima and according to (4), the Markov renewal function takes the form:

$$\Psi_{ij}(t) = \sum_{n=0}^{\infty} \sum_{k \in E} \int_{0}^{t} \left( \alpha_{ik}(F(s))^{\sum_{m \in E} a_m^{-1}} f(s) \right) Q_{kj}^{(n-1)}(t-s) ds. \quad (41)$$

The semi-Markov transition matrix (function) is defined as:

$$P_{ij}(t) := \mathbb{P}(Z_t = j | Z_0 = i), i, j \in E. \quad (42)$$

Limnios and Opri□an (2001) proved that the semi-Markov transition function satisfies the following Markov renewal equation (MRE):

$$P(t) = I_N - \mathrm{diag}\left(\sum_j Q_{ij}(t); i \in E\right) + (Q \star P)(t),$$

where, $I_N$ is the $N \times N$ identity matrix, ★ represents the convolution and "diag" is a diagonal matrix.

Let $W(t) := \mathrm{diag}(W_i(t); i \in E) = \mathrm{diag}(\sum_j Q_{ij}(t); i \in E) = \mathrm{diag}(Q \cdot 1_N)(t)$ be the diagonal matrix where the $(i, i)$ element is $W_i(t) = \sum_{j \in E} Q_{ij}(t)$ and $1_N = \frac{(1, \cdots, 1)^T}{N}, (\quad)^T$ denotes the transposed of a vector.

Limnios and Opri□an (2001) established the unique solution of the above MRE:

$$P(t) = \left( (I_N - Q)^{(-1)} \star (I_N - W) \right) (t) = (\Psi \star (I_N - W))(t), \quad (43)$$

where, $\Psi(t) = (\Psi_{ij}(t))_{i,j \in E}$ and it is shown that $(I_N - Q)^{(-1)}(t) = \Psi(t)$, where $\Psi^{(-1)}$ denotes the inverse convolution function.

Let us consider the case of several trajectories where we may have censoring at the beginning and/or at the end. Then, the estimators $\hat{P}_{ij}(t)$ and $\hat{\Psi}_{ij}(t)$, $i, j \in E, t > 0$, are given by:

$$\hat{P}_{ij}(t; L, M) = \left( \hat{\Psi}_{ij} \star \left( 1 - \sum_{k \in E} \hat{Q}_{jk} \right) \right) (t; L, M) \quad (44)$$

and:

$$\begin{aligned} \hat{\Psi}_{ij}(t; L, M) &= \sum_{n=0}^{\infty} \hat{Q}_{ij}^{(n)}(t; L, M) \\ &= \sum_{n=0}^{\infty} \sum_{k \in E} \int_{0}^{t} \hat{Q}_{ik}(s; L, M) \hat{Q}_{kj}^{(n-1)}(t-s; L, M) ds, \end{aligned} \quad (45)$$

where the appropriate estimator for $a_{ij}$ obtained in the previous section, is chosen for deriving $\hat{Q}_{ij}^{(n)}(t; L, M)$.

## Reliability/Survival Analysis Indicators

For the purpose of this section we assume that the state space $E$ can be split into two subsets, $U$ (containing the functioning states) and $D$ (containing the failure states), such as $E = U \cup D$ and $E = U \cap D = \emptyset$, where let say that $U = \{1, \dots, n\}$ and $D = \{n+1, \dots, N\}$. Each matrix can be partitioned according to this state space partition.

This section is devoted to the construction of estimators for a variety of reliability indicators in the case of several censored paths at the beginning and/or at the end (Limnios and Ouhbi, 2003).

### Reliability

The reliability at time $t$, $R(t)$, is defined by:

$$R(t) = \mathbb{P}(T_D > t) = \mathbb{P}(Z_s \in U, s \le t),$$

where, $T_D := \inf\{t | Z_t \in D\}$ is the lifetime of the system. The following result presents the estimator of the reliability of a semi-Markov system in terms of estimators of basic quantities of a semi-Markov process.

### Proposition 5

For a semi-Markov system, the estimator of the reliability at time $t > 0$ is given by (cf. Ouhbi and Limnios, 1996):

$$\hat{R}(t; L, M) = \hat{\alpha}_U(L, M) \hat{P}_{UU}(t; L, M) 1_n, \quad (46)$$

where, $\hat{\alpha}_U(L, M)$ is an estimator of $\alpha_U$ and $b \hat{P}_{UU}(t; L, M)$ is an estimator of $P_{UU}(t)$.

### Lemma 3

The estimator of the failure rate of the system is given by:

$$\hat{\lambda}(t; L, M) := - \frac{\hat{R}'(t; L, M)}{\hat{R}(t; L, M)}, t > 0. \quad (47)$$

### Availability

The availability of the system, $A(t)$, is defined as:
$$A(t) = \mathbb{P}(Z_i \in U). \qquad (48)$$

### Proposition 6

The estimator of the availability at time $t > 0$, for a semi-Markov system, is given by (Ouhbi and Limnios, 1996):

$$\hat{A}(t; L, M) = \hat{\alpha}(L, M) \hat{P}(t; L, M) 1_{N; n}, \qquad (49)$$

where, $\hat{P}(t; L, M)$ is the estimator of $P(t)$, $\hat{\alpha}(L, M)$ is the estimator of $\alpha$ and $1_{N; n} = \begin{array}{c} (1, \dots, 1, 0, \dots, 0)^T \\ \underbrace{1}_{n} \quad \underbrace{1}_{N-n} \end{array}$.

### Maintainability

The maintainability of the system, $M(t)$, is defined by:

$$M(t) = \mathbb{P}(T_U \le t) = 1 - \mathbb{P}(Z_s \in D, s \le t), \qquad (50)$$

where, $T_U := \inf\{t | Z_t \in U\}$ is the duration of repair.

### Proposition 7

The estimator of the maintainability at time $t > 0$, for a semi-Markov system, is given by (cf. Limnios and Opri□an, 2001):

$$\hat{M}(t; L, M) = 1 - \hat{\alpha}_D(L, M) \hat{P}_{DD}(t; L, M) 1_{N-n}, \qquad (51)$$

where, $\hat{\alpha}_D(L, M)$ is an estimator of $\alpha_D$ and $\hat{P}_{DD}(t; L, M)$ is an estimator of $P_{DD}(t)$.

### The Mean Time to Failure

The Mean Time To Failure (MTTF) is defined as the mean lifetime, namely:

$$MTTF := \mathbb{E}(T_D).$$

Let introduce first the mean sojourn times vector $m := (m_1, \dots, m_N)^T$, where:

$$m_i := \mathbb{E}(S_1 | J_0 = i) = \int_0^\infty 1 - W_i(t) dt = \int_0^\infty \left( 1 - \left[ F(t) \right]_{t=1}^{\sum a_{ij}} \right) dt$$

is the mean sojourn time spent in state $i$. Note that for a regular and positive recurrent MRP, we have $m_i < \infty$, $I \in E$ (cf. Limnios and Opri□an, 2001).

The estimator of $m_i$ can be obtained in two ways, firstly using an MLE considering the estimators obtained in Section 3 and secondly using the empirical estimator. We can estimate the mean sojourn time in state $i$ in two different ways, namely using a plugin MLE (considering

the estimators obtained in Section 3) or the empirical estimator, namely:

$$\hat{m}_i^{(1)}(L, M) := \int_0^\infty (1 - \hat{W}_i(t; L, M)) dt = \int_0^\infty \left( 1 - \left[ F(t) \right]_{t=1}^{\sum \hat{a}_i(L, M)} \right) dt$$

and:

$$\hat{m}_i^{(2)}(L, M) := \frac{\sum_{k=1}^{N_i^{(M)}} X_i^{(k)}}{N_i(L, M)},$$

where the estimators of $a_{ij}$ are obtained in Section 3. For more details concerning the MTTF of a semi-Markov system, one can see (Limnios and Opri□an, 2001).

### Proposition 8

Let a semi-Markov system, where the matrix $(I_n - \hat{p}_{UU}(M))$ is nonsingular. The estimator of MTTF can be obtained by one of the following two expressions:

$$\begin{align*}
\widehat{MTTF}^{(1)}(L, M) &= \hat{\alpha}_U(L, M) (I_n - \hat{p}_{UU}(L, M))^{-1} \hat{m}_U^{(1)}(L, M), \\
\widehat{MTTF}^{(2)}(L, M) &= \hat{\alpha}_U(L, M) (I_n - \hat{p}_{UU}(L, M))^{-1} \hat{m}_U^{(2)}(L, M),
\end{align*}$$

where, $\hat{p}_{UU}(L, M)$ is the estimator of $p_{UU}$; $\hat{\alpha}_U(L, M)$ is an estimator of $\alpha_U$, $\hat{m}_U^{(1)}(L, M)$ and $\hat{m}_U^{(2)}(L, M)$ are the restrictions to set $U$ of $\hat{m}_U^{(1)}(L, M)$ and $\hat{m}_U^{(2)}(L, M)$, respectively.

Following a similar procedure, one can obtain the estimator of the Mean Time To Repair (MTTR), defined as $MTTR := \mathbb{E}(T_U)$.

### Corollary 2

For all $i, j \in E$ and $t > 0$ and under the setting of Section 3, for $L$ sample paths, $L \ge 2$ of length $M$ of a semi-Markov process, the estimators of $\Psi_{ij}(t)$, $P_{ij}(t)$, $R(t)$, $\lambda(t)$, $A(t)$, $M(t)$, MTTF, obtained by considering the estimators $\hat{a}_{ij}(L, M)$ of $a_{ij}$ given in (18) and $\hat{\alpha}_i(L, M)$ of $\alpha_i$ given in (19), are strongly consistent, as $L$ tends to infinity.

### Simulations

#### Estimation

A series of simulations in $R$ is analyzed in order to evaluate the accuracy of the proposed procedures. For this purpose we consider $L$ sample paths of a semi-Markov process, where we assume that the observation time $M$ is set to be 1000 and the sojourn times follow the
$$A(t) = \mathbb{P}(Z_i \in U). \qquad (48)$$

### Proposition 6

The estimator of the availability at time $t > 0$, for a semi-Markov system, is given by (Ouhbi and Limnios, 1996):

$$\hat{A}(t; L, M) = \hat{\alpha}(L, M) \hat{P}(t; L, M) 1_{N; n}, \qquad (49)$$

where, $\hat{P}(t; L, M)$ is the estimator of $P(t)$, $\hat{\alpha}(L, M)$ is the estimator of $\alpha$ and $1_{N; n} = \begin{array}{c} (1, \dots, 1, 0, \dots, 0)^T \\ \underbrace{1}_{n} \quad \underbrace{1}_{N-n} \end{array}$.

### Maintainability

The maintainability of the system, $M(t)$, is defined by:

$$M(t) = \mathbb{P}(T_U \le t) = 1 - \mathbb{P}(Z_s \in D, s \le t), \qquad (50)$$

where, $T_U := \inf\{t | Z_t \in U\}$ is the duration of repair.

### Proposition 7

The estimator of the maintainability at time $t > 0$, for a semi-Markov system, is given by (cf. Limnios and Opri□an, 2001):

$$\hat{M}(t; L, M) = 1 - \hat{\alpha}_D(L, M) \hat{P}_{DD}(t; L, M) 1_{N-n}, \qquad (51)$$

where, $\hat{\alpha}_D(L, M)$ is an estimator of $\alpha_D$ and $\hat{P}_{DD}(t; L, M)$ is an estimator of $P_{DD}(t)$.

### The Mean Time to Failure

The Mean Time To Failure (MTTF) is defined as the mean lifetime, namely:

$$MTTF := \mathbb{E}(T_D).$$

Let introduce first the mean sojourn times vector $m := (m_1, \dots, m_N)^T$, where:

$$m_i := \mathbb{E}(S_1 | J_0 = i) = \int_0^\infty 1 - W_i(t) dt = \int_0^\infty \left( 1 - \left[F(t)\right]_{t=1}^{\sum a_{ij}} \right) dt$$

is the mean sojourn time spent in state $i$. Note that for a regular and positive recurrent MRP, we have $m_i < \infty$, $I \in E$ (cf. Limnios and Opri□an, 2001).

The estimator of $m_i$ can be obtained in two ways, firstly using an MLE considering the estimators obtained in Section 3 and secondly using the empirical estimator. We can estimate the mean sojourn time in state $i$ in two different ways, namely using a plugin MLE (considering

the estimators obtained in Section 3) or the empirical estimator, namely:

$$\hat{m}_i^{(1)}(L, M) := \int_0^\infty (1 - \hat{W}_i(t; L, M)) dt = \int_0^\infty \left( 1 - \left[F(t)\right]_{t=1}^{\sum \hat{a}_i(L, M)} \right) dt$$

and:

$$\hat{m}_i^{(2)}(L, M) := \frac{\sum_{k=1}^{N_i^{(M)}} X_i^{(l,k)}}{N_i(L, M)},$$

where the estimators of $a_{ij}$ are obtained in Section 3. For more details concerning the MTTF of a semi-Markov system, one can see (Limnios and Opri□an, 2001).

### Proposition 8

Let a semi-Markov system, where the matrix $(I_n - \hat{p}_{UU}(M))$ is nonsingular. The estimator of MTTF can be obtained by one of the following two expressions:

$$\begin{align*} \widehat{MTTF}^{(1)}(L, M) &= \hat{\alpha}_U(L, M) (I_n - \hat{p}_{UU}(L, M))^{-1} \hat{m}_U^{(1)}(L, M), \\ \widehat{MTTF}^{(2)}(L, M) &= \hat{\alpha}_U(L, M) (I_n - \hat{p}_{UU}(L, M))^{-1} \hat{m}_U^{(2)}(L, M), \end{align*}$$

where, $\hat{p}_{UU}(L, M)$ is the estimator of $p_{UU}$; $\hat{\alpha}_U(L, M)$ is an estimator of $\alpha_U$, $\hat{m}_U^{(1)}(L, M)$ and $\hat{m}_U^{(2)}(L, M)$ are the restrictions to set $U$ of $\hat{m}_U^{(1)}(L, M)$ and $\hat{m}_U^{(2)}(L, M)$, respectively.

Following a similar procedure, one can obtain the estimator of the Mean Time To Repair (MTTR), defined as $MTTR := \mathbb{E}(T_U)$.

### Corollary 2

For all $i, j \in E$ and $t > 0$ and under the setting of Section 3, for $L$ sample paths, $L \ge 2$ of length $M$ of a semi-Markov process, the estimators of $\Psi_{ij}(t)$, $P_{ij}(t)$, $R(t)$, $\lambda(t)$, $A(t)$, $M(t)$, MTTF, obtained by considering the estimators $\hat{a}_{ij}(L, M)$ of $a_{ij}$ given in (18) and $\hat{\alpha}_i(L, M)$ of $\alpha_i$ given in (19), are strongly consistent, as $L$ tends to infinity.

## Simulations

### Estimation

A series of simulations in $R$ is analyzed in order to evaluate the accuracy of the proposed procedures. For this purpose we consider $L$ sample paths of a semi-Markov process, where we assume that the observation time $M$ is set to be 1000 and the sojourn times follow the
Power Function distribution that belongs to the class of distributions closed under maxima, namely:

$$f(t; a_{ij}) = \frac{a_{ij}}{c^{a_{ij}}} t^{a_{ij}-1}, \quad 0 \le t \le c, \ c > 0,$$

where, c is the scale parameter and is taken to be 100. The initial distribution α is considered to follow the discrete Uniform distribution with parameters 1 and N. In this work we study several cases, like:

* Uncensored sample paths

* Censored sample paths at the end (50% censored rate for the times associated with the end of the trajectory)

* Censored sample paths at the beginning (50% censored rate for the times associated with the beginning of the trajectory)

* Censored sample paths at the beginning and/or at the end (50% censored rate for the times associated with the beginning/end of the trajectory)

The above cases can also be studied in the case of one trajectory. Variations of the above cases are also possible. Furthermore, one can consider other distributions for the sojourn times that belong to one of the two classes of distributions and investigate a variety of scenarios like the ones above. For the purpose of the simulation study the values are arbitrarily chosen in Table 1.

## Several Uncensored Sample Paths

One way of illustrating the accuracy of the estimators is by providing the Squared Errors (S.E.) of the estimators for several values of L. The results are shown in Table 2. Observe that the accuracy for the transition probabilities of the embedded Markov chain is higher than that of the transition rates. As expected the squared errors are extremely good even for very small sample sizes.

Observe that the larger the number of trajectories, the smaller the squared errors of the estimators of $a_{ij}$ and $p_{ij}$. On the other hand, according to Table 3, the smaller the value of t the most accurate the estimators of the transition probabilities of the semi-Markov process. In other words, we have more accurate predictions as we get closer to the present.

## Several Censored Sample Paths at the Beginning and/or at the End

Let us first explain, the way we simulate the censoring at the beginning. Using a Bernoulli we randomly choose the trajectories that have censored sojourn time in the first visited state. For the trajectories that do have censoring at the beginning,

we randomly cut the interval that have been computed as the first sojourn time in two parts using the Uniform distribution with parameters 0 and the first sojourn time that has been computed. The second part is considered to be the censored sojourn time in the first visited state. Of course, one can consider modifications of the above procedure.

As for the trajectories that have censored sojourn time in the last visited state, the same procedure is followed.

It should be noted that the censoring rate which is taken to be equal to 50% refers not to all sojourn times but only to the times associated with the beginning and/or the end of the trajectory. Consequently, the overall percentage of all sojourn times is much less than 50%.

The estimated values of the parameters for L = 1000 rejection with censoring at the beginning and/or at the end one pointed in the Table 4. Table 5 gives a comparison between the estimators of $a_{ij}$ and $p_{ij}$ obtained for different values of L. The estimators approach the true value as L increases. In addition, Table 6 shows that we obtain good estimators for values of t approaching the upper limit and even better estimators, in terms of the squared errors for small values of t. The estimators in the central part of the time interval appear to be less accurate as compared with the ones at the end points of the intervals.

## One Censored Sample Path at the Beginning and at the End

Table 7 provides the estimated values of $a_{ij}$, $p_{ij}$ in the special case of one censored sample path at the beginning and at the end, where the observation time M is set to be 100000. For the specific example the number of visited states is 74031.

The accuracy of the estimators is studied using the square errors for several values of M. The results for M = 1000, 10000 and 100000 are shown in Tables 8 and 9.

The conclusions are similar to the ones in the case of several censored paths. According to Table 8 as M increases we obtain much better estimators. Table 9 gives better estimators for small values of t with a small reduction of the squared errors while we approach the upper limit of the time variable.

<table><caption>**Table 1:** Transition rates $a_{ij}$ and transition probabilities $p_{ij}$ used for the simulation study.</caption><thead><tr><th></th><th>1</th><th>2</th><th>3</th></tr></thead><tbody><tr><th scope="row">a<sub>ij</sub></th><td></td><td></td><td></td></tr><tr><th scope="row">1</th><td>0</td><td>0.5</td><td>1</td></tr><tr><th scope="row">2</th><td>1.2</td><td>0</td><td>0.9</td></tr><tr><th scope="row">3</th><td>1.4</td><td>1.5</td><td>0</td></tr><tr><th scope="row">p<sub>ij</sub></th><td></td><td></td><td></td></tr><tr><th scope="row">1</th><td>0</td><td>0.333</td><td>0.667</td></tr><tr><th scope="row">2</th><td>0.5714</td><td>0</td><td>0.4286</td></tr><tr><th scope="row">3</th><td>0.4828</td><td>0.5172</td><td>0</td></tr></tbody></table>
Power Function distribution that belongs to the class of distributions closed under maxima, namely:

$$f(t; a_{ij}) = \frac{a_{ij}}{c^{a_{ij}}} t^{a_{ij}-1}, \quad 0 \le t \le c, \ c > 0,$$

where, c is the scale parameter and is taken to be 100. The initial distribution α is considered to follow the discrete Uniform distribution with parameters 1 and N. In this work we study several cases, like:

* Uncensored sample paths

* Censored sample paths at the end (50% censored rate for the times associated with the end of the trajectory)

* Censored sample paths at the beginning (50% censored rate for the times associated with the beginning of the trajectory)

* Censored sample paths at the beginning and/or at the end (50% censored rate for the times associated with the beginning/end of the trajectory)

The above cases can also be studied in the case of one trajectory. Variations of the above cases are also possible. Furthermore, one can consider other distributions for the sojourn times that belong to one of the two classes of distributions and investigate a variety of scenarios like the ones above. For the purpose of the simulation study the values are arbitrarily chosen in Table 1.

## Several Uncensored Sample Paths

One way of illustrating the accuracy of the estimators is by providing the Squared Errors (S.E.) of the estimators for several values of L. The results are shown in Table 2. Observe that the accuracy for the transition probabilities of the embedded Markov chain is higher than that of the transition rates. As expected the squared errors are extremely good even for very small sample sizes.

Observe that the larger the number of trajectories, the smaller the squared errors of the estimators of $a_{ij}$ and $p_{ij}$. On the other hand, according to Table 3, the smaller the value of t the most accurate the estimators of the transition probabilities of the semi-Markov process. In other words, we have more accurate predictions as we get closer to the present.

## Several Censored Sample Paths at the Beginning and/or at the End

Let us first explain, the way we simulate the censoring at the beginning. Using a Bernoulli we randomly choose the trajectories that have censored sojourn time in the first visited state. For the trajectories that do have censoring at the beginning,

we randomly cut the interval that have been computed as the first sojourn time in two parts using the Uniform distribution with parameters 0 and the first sojourn time that has been computed. The second part is considered to be the censored sojourn time in the first visited state. Of course, one can consider modifications of the above procedure.

As for the trajectories that have censored sojourn time in the last visited state, the same procedure is followed.

It should be noted that the censoring rate which is taken to be equal to 50% refers not to all sojourn times but only to the times associated with the beginning and/or the end of the trajectory. Consequently, the overall percentage of all sojourn times is much less than 50%.

The estimated values of the parameters for L = 1000 rejection with censoring at the beginning and/or at the end one pointed in the Table 4. Table 5 gives a comparison between the estimators of $a_{ij}$ and $p_{ij}$ obtained for different values of L. The estimators approach the true value as L increases. In addition, Table 6 shows that we obtain good estimators for values of t approaching the upper limit and even better estimators, in terms of the squared errors for small values of t. The estimators in the central part of the time interval appear to be less accurate as compared with the ones at the end points of the intervals.

## One Censored Sample Path at the Beginning and at the End

Table 7 provides the estimated values of $a_{ij}$, $p_{ij}$ in the special case of one censored sample path at the beginning and at the end, where the observation time M is set to be 100000. For the specific example the number of visited states is 74031.

The accuracy of the estimators is studied using the square errors for several values of M. The results for M = 1000, 10000 and 100000 are shown in Tables 8 and 9.

The conclusions are similar to the ones in the case of several censored paths. According to Table 8 as M increases we obtain much better estimators. Table 9 gives better estimators for small values of t with a small reduction of the squared errors while we approach the upper limit of the time variable.

<table><caption>**Table 1:** Transition rates $a_{ij}$ and transition probabilities $p_{ij}$ used for the simulation study.</caption><thead><tr><th></th><th>1</th><th>2</th><th>3</th></tr></thead><tbody><tr><th scope="row">a<sub>ij</sub></th><td></td><td></td><td></td></tr><tr><th scope="row">1</th><td>0</td><td>0.5</td><td>1</td></tr><tr><th scope="row">2</th><td>1.2</td><td>0</td><td>0.9</td></tr><tr><th scope="row">3</th><td>1.4</td><td>1.5</td><td>0</td></tr><tr><th scope="row">p<sub>ij</sub></th><td></td><td></td><td></td></tr><tr><th scope="row">1</th><td>0</td><td>0.333</td><td>0.667</td></tr><tr><th scope="row">2</th><td>0.5714</td><td>0</td><td>0.4286</td></tr><tr><th scope="row">3</th><td>0.4828</td><td>0.5172</td><td>0</td></tr></tbody></table>
<table><caption>**Table 2:** S.E. for the estimators of $a_{ij}$ and $p_{ij}$ in the case of no censoring, for various values of $L$</caption><thead><tr><th>S.E.\L</th><th>5</th><th>10</th><th>100</th><th>1000</th></tr></thead><tbody><tr><th>ŷ<sub>ij</sub> (L, M)</th><td>3.70943×10<sup>-1</sup></td><td>2.21032×10<sup>-1</sup></td><td>1.31127×10<sup>-2</sup></td><td>7.240581×10<sup>-3</sup></td></tr><tr><th>p̂<sub>ij</sub> (L, M)</th><td>4.24745×10<sup>-2</sup></td><td>1.79203×10<sup>-2</sup></td><td>5.74847×10<sup>-5</sup></td><td>8.24954×10<sup>-6</sup></td></tr></tbody></table>

<table><caption>**Table 3:** S.E. for the estimators of the semi-Markov transition probabilities in the case of no censoring ($L = 1000$), for various values of $t$</caption><thead><tr><th>S.E.\t</th><th>1</th><th>25</th><th>50</th><th>75</th><th>100</th></tr></thead><tbody><tr><th>P̂<sub>ij</sub> (t; L, M)</th><td>3.19490×10<sup>-8</sup></td><td>4.57773×10<sup>-5</sup></td><td>1.10313×10<sup>-4</sup></td><td>1.84713×10<sup>-4</sup></td><td>6.43798×10<sup>-4</sup></td></tr></tbody></table>

<table><caption>**Table 4:** Estimated values of the transition rates $a_{ij}$, transition probabilities $p_{ij}$ and initial law $a_i$ in the case of $L = 1000$ trajectories with censoring at the beginning and/or at the end</caption><thead><tr><th></th><th>1</th><th>2</th><th>3</th></tr></thead><tbody><tr><th>ŷ<sub>ij</sub> (L, M)</th><td></td><td></td><td></td></tr><tr><th>1</th><td>0</td><td>0.4625</td><td>0.9789</td></tr><tr><th>2</th><td>1.0893</td><td>0</td><td>0.8407</td></tr><tr><th>3</th><td>1.2890</td><td>1.3778</td><td>0</td></tr><tr><th>p̂<sub>ij</sub> (L, M)</th><td></td><td></td><td></td></tr><tr><th>1</th><td>0</td><td>0.3209</td><td>0.6791</td></tr><tr><th>2</th><td>0.5644</td><td>0</td><td>0.4356</td></tr><tr><th>3</th><td>0.4833</td><td>0.5167</td><td>0</td></tr><tr><th>i</th><td></td><td></td><td></td></tr><tr><th>a<sub>i</sub> (L, M)</th><td>0.319</td><td>0.355</td><td>0.326</td></tr></tbody></table>

<table><caption>**Table 5:** S.E. for the estimators of $a_{ij}$ and $p_{ij}$ in the case of censoring at the beginning and/or at the end, for various values of $L$</caption><thead><tr><th>S.E.\L</th><th>5</th><th>10</th><th>100</th><th>1000</th></tr></thead><tbody><tr><th>ŷ<sub>ij</sub> (L, M)</th><td>2.423613</td><td>2.98703×10<sup>-1</sup></td><td>6.50080×10<sup>-2</sup></td><td>4.48959×10<sup>-2</sup></td></tr><tr><th>p̂<sub>ij</sub> (L, M)</th><td>2.67957×10<sup>-2</sup></td><td>1.95020×10<sup>-2</sup></td><td>7.11892×10<sup>-2</sup></td><td>4.09628×10<sup>-4</sup></td></tr></tbody></table>

<table><caption>**Table 6:** S.E. for the estimators of the semi-Markov transition probabilities in the case of censoring at the beginning and/or at the end, for various values of $t$.</caption><thead><tr><th>S.E.\t</th><th>1</th><th>25</th><th>50</th><th>100</th></tr></thead><tbody><tr><th>P̂<sub>ij</sub> (t; L, M)</th><td>1.61768×10<sup>-7</sup></td><td>5.61013×10<sup>-5</sup></td><td>2.23401×10<sup>-3</sup></td><td>9.09540×10<sup>-4</sup></td></tr></tbody></table>

<table><caption>**Table 7:** The estimated values of the transition rates $a_{ij}$ and the transition probabilities $p_{ij}$ in the case of 1 trajectory with censoring at the beginning and at the end</caption><thead><tr><th></th><th>1</th><th>2</th><th>3</th></tr></thead><tbody><tr><th>ŷ<sub>ij</sub> (M)</th><td></td><td></td><td></td></tr><tr><th>1</th><td>0</td><td>0.4864</td><td>1.0332</td></tr><tr><th>2</th><td>1.2421</td><td>0</td><td>0.9254</td></tr><tr><th>3</th><td>1.4786</td><td>1.6130</td><td>0</td></tr><tr><th>p̂<sub>ij</sub> (M)</th><td></td><td></td><td></td></tr><tr><th>1</th><td>0</td><td>0.3201</td><td>0.6799</td></tr><tr><th>2</th><td>0.5731</td><td>0</td><td>0.4269</td></tr><tr><th>3</th><td>0.4783</td><td>0.5217</td><td>0</td></tr></tbody></table>

<table><caption>**Table 8:** S.E. for the estimators of $a_{ij}$ and $p_{ij}$ in the case of 1 trajectory with censoring at the beginning and at the end, for various values of the observation time $M$</caption><thead><tr><th>S.E.\M</th><th>1000</th><th>10000</th><th>100000</th></tr></thead><tbody><tr><th>ŷ<sub>ij</sub> (M)</th><td>7.71</td><td>1.88320×10<sup>-1</sup></td><td>2.26453×10<sup>-2</sup></td></tr><tr><th>p̂<sub>ij</sub> (M)</th><td>4.32255×10<sup>-1</sup></td><td>1.73480×10<sup>-2</sup></td><td>3.97106×10<sup>-4</sup></td></tr></tbody></table>
<table><caption>**Table 2:** S.E. for the estimators of $a_{ij}$ and $p_{ij}$ in the case of no censoring, for various values of $L$</caption><thead><tr><th>S.E.\L</th><th>5</th><th>10</th><th>100</th><th>1000</th></tr></thead><tbody><tr><th>ŷ<sub>ij</sub> (L, M)</th><td>3.70943×10<sup>-1</sup></td><td>2.21032×10<sup>-1</sup></td><td>1.31127×10<sup>-2</sup></td><td>7.240581×10<sup>-3</sup></td></tr><tr><th>p̂<sub>ij</sub> (L, M)</th><td>4.24745×10<sup>-2</sup></td><td>1.79203×10<sup>-2</sup></td><td>5.74847×10<sup>-5</sup></td><td>8.24954×10<sup>-6</sup></td></tr></tbody></table>

<table><caption>**Table 3:** S.E. for the estimators of the semi-Markov transition probabilities in the case of no censoring ($L = 1000$), for various values of $t$</caption><thead><tr><th>S.E.\t</th><th>1</th><th>25</th><th>50</th><th>75</th><th>100</th></tr></thead><tbody><tr><th>P̂<sub>ij</sub> (t; L, M)</th><td>3.19490×10<sup>-8</sup></td><td>4.57773×10<sup>-5</sup></td><td>1.10313×10<sup>-4</sup></td><td>1.84713×10<sup>-4</sup></td><td>6.43798×10<sup>-4</sup></td></tr></tbody></table>

<table><caption>**Table 4:** Estimated values of the transition rates $a_{ij}$, transition probabilities $p_{ij}$ and initial law $a_i$ in the case of $L = 1000$ trajectories with censoring at the beginning and/or at the end</caption><thead><tr><th></th><th>1</th><th>2</th><th>3</th></tr></thead><tbody><tr><th>ŷ<sub>ij</sub> (L, M)</th><td></td><td></td><td></td></tr><tr><th>1</th><td>0</td><td>0.4625</td><td>0.9789</td></tr><tr><th>2</th><td>1.0893</td><td>0</td><td>0.8407</td></tr><tr><th>3</th><td>1.2890</td><td>1.3778</td><td>0</td></tr><tr><th>p̂<sub>ij</sub> (L, M)</th><td></td><td></td><td></td></tr><tr><th>1</th><td>0</td><td>0.3209</td><td>0.6791</td></tr><tr><th>2</th><td>0.5644</td><td>0</td><td>0.4356</td></tr><tr><th>3</th><td>0.4833</td><td>0.5167</td><td>0</td></tr><tr><th>i</th><td></td><td></td><td></td></tr><tr><th>a<sub>i</sub> (L, M)</th><td>0.319</td><td>0.355</td><td>0.326</td></tr></tbody></table>

<table><caption>**Table 5:** S.E. for the estimators of $a_{ij}$ and $p_{ij}$ in the case of censoring at the beginning and/or at the end, for various values of $L$</caption><thead><tr><th>S.E.\L</th><th>5</th><th>10</th><th>100</th><th>1000</th></tr></thead><tbody><tr><th>ŷ<sub>ij</sub> (L, M)</th><td>2.423613</td><td>2.98703×10<sup>-1</sup></td><td>6.50080×10<sup>-2</sup></td><td>4.48959×10<sup>-2</sup></td></tr><tr><th>p̂<sub>ij</sub> (L, M)</th><td>2.67957×10<sup>-2</sup></td><td>1.95020×10<sup>-2</sup></td><td>7.11892×10<sup>-2</sup></td><td>4.09628×10<sup>-4</sup></td></tr></tbody></table>

<table><caption>**Table 6:** S.E. for the estimators of the semi-Markov transition probabilities in the case of censoring at the beginning and/or at the end, for various values of $t$.</caption><thead><tr><th>S.E.\t</th><th>1</th><th>25</th><th>50</th><th>100</th></tr></thead><tbody><tr><th>P̂<sub>ij</sub> (t; L, M)</th><td>1.61768×10<sup>-7</sup></td><td>5.61013×10<sup>-5</sup></td><td>2.23401×10<sup>-3</sup></td><td>9.09540×10<sup>-4</sup></td></tr></tbody></table>

<table><caption>**Table 7:** The estimated values of the transition rates $a_{ij}$ and the transition probabilities $p_{ij}$ in the case of 1 trajectory with censoring at the beginning and at the end</caption><thead><tr><th></th><th>1</th><th>2</th><th>3</th></tr></thead><tbody><tr><th>ŷ<sub>ij</sub> (M)</th><td></td><td></td><td></td></tr><tr><th>1</th><td>0</td><td>0.4864</td><td>1.0332</td></tr><tr><th>2</th><td>1.2421</td><td>0</td><td>0.9254</td></tr><tr><th>3</th><td>1.4786</td><td>1.6130</td><td>0</td></tr><tr><th>p̂<sub>ij</sub> (M)</th><td></td><td></td><td></td></tr><tr><th>1</th><td>0</td><td>0.3201</td><td>0.6799</td></tr><tr><th>2</th><td>0.5731</td><td>0</td><td>0.4269</td></tr><tr><th>3</th><td>0.4783</td><td>0.5217</td><td>0</td></tr></tbody></table>

<table><caption>**Table 8:** S.E. for the estimators of $a_{ij}$ and $p_{ij}$ in the case of 1 trajectory with censoring at the beginning and at the end, for various values of the observation time $M$</caption><thead><tr><th>S.E.\M</th><th>1000</th><th>10000</th><th>100000</th></tr></thead><tbody><tr><th>ŷ<sub>ij</sub> (M)</th><td>7.71</td><td>1.88320×10<sup>-1</sup></td><td>2.26453×10<sup>-2</sup></td></tr><tr><th>p̂<sub>ij</sub> (M)</th><td>4.32255×10<sup>-1</sup></td><td>1.73480×10<sup>-2</sup></td><td>3.97106×10<sup>-4</sup></td></tr></tbody></table>
## Influence of Taking into Account the Censoring

We would like now to examine the case of estimating the parameters where in the target case we have censoring (for example at the end). We take into account two different cases: (i) The case of estimating the parameters using the formula with censoring at the end; (ii) and estimating the parameters using (incorrectly) the formula without censoring, under the assumption that the first and the last observation times (0 and M) are jump times.

This is done in order to investigate the influence of taking into account the censoring. For this procedure we consider both statistical settings with one trajectory and several trajectories.

The average number of visited states in $L = 10$ trajectories for the values of $M$ in Table 10 is 8.5, 15.8, 150.1, 1483.3 and 14825.5 respectively. The average length of trajectories (the average total non-censored sojourn time) is 471.4447, 967.3908, 9964.861, 99950.99 and 999956.5 respectively. Observe that, in the cases where the censored part represents approximately more than 1% of the average number of visited states, the target case gives better estimators than the case where we do not take into account the censored part. However, when the censored part represents less than 1% of the average

number of visited states, the contribution of the censored part is not significant.

The number of visited states for the values of $M$ in Table 11 is 7, 19, 147 and 1467 respectively. The length of the trajectory (the total non-censored sojourn time) is 421.7443, 999.8391, 9937.624 and 99988.39 respectively. Observe that, in the cases where the censored part represents the 12% of the number of visited states, the target case gives better estimators than the case where we do not take into account the censored part. However, when the censored part represents less than 12% of the number of visited states, the contribution of the censored part is not significant. Note that the huge difference between the two cases for $M = 500$ is predictable since the censored part is not to so small as compared to the total trajectory.

## Simulations: Reliability

Let us consider the complete (full) case of several sample paths with censoring at the beginning and/or at the end in order to investigate the behavior of several reliability indices that have been proposed in Section 6.

According to Table 12 the Reliability estimator performs better for values of $t$ that belong to the second and third quarter of its domain. However, as for the estimators of Availability and Maintainability, there is no significant difference for the entire spectrum of the time interval.

<table><caption>**Table 9:** S.E. for the estimators of the semi-Markov transition probabilities in the case of censoring at the beginning and/or at the end, for various values of $t$</caption><thead><tr><th>S.E.\t</th><th>1</th><th>25</th><th>50</th><th>100</th></tr></thead><tbody><tr><td>P̂<sub>ij</sub> (t; M)</td><td>1.15749×10<sup>-8</sup></td><td>7.84353×10<sup>-5</sup></td><td>5.87928×10<sup>-3</sup></td><td>5.695643×10<sup>-4</sup></td></tr></tbody></table>

<table><caption>**Table 10:** S.E. for the estimators of $a_{ij}$ in the case of $L = 1000$ trajectories with censoring at the end, for several values of the observation time $M$</caption><thead><tr><th>S.E.\M</th><th>500</th><th>1000</th><th>10000</th><th>100000</th><th>1000000</th></tr></thead><tbody><tr><td>â<sub>ij</sub> (L, M) (case (i))</td><td>0.6119373</td><td>0.04237282</td><td>0.02595444</td><td>0.001830896</td><td>0.0001634836</td></tr><tr><td>â<sub>ij</sub> (L, M) (case (ii))</td><td>0.6571087</td><td>0.0511309</td><td>0.02698702</td><td>0.001829154</td><td>0.0001637393</td></tr></tbody></table>

<table><caption>**Table 11:** S.E. for the estimators of $a_{ij}$ in the case of 1 trajectory with censoring at the end, for several values of the observation time $M$</caption><thead><tr><th>S.E.\M</th><th>500</th><th>1000</th><th>10000</th><th>100000</th></tr></thead><tbody><tr><td>â<sub>ij</sub> (M) (case (i))</td><td>8.754049</td><td>2.637813</td><td>0.3570539</td><td>0.05331928</td></tr><tr><td>â<sub>ij</sub> (M) (case (ii))</td><td>18.75966</td><td>2.6378166</td><td>0.3603736</td><td>0.05330249</td></tr></tbody></table>

<table><caption>**Table 12:** S.E. for Reliability indices in the case of several trajectories ($L = 1000$) with censoring at the beginning and/or at the end, for several values of $t$</caption><thead><tr><th>S.E.\t</th><th>1</th><th>25</th><th>50</th><th>100</th></tr></thead><tbody><tr><td>R̂(t; L, M)</td><td>5.26994×10<sup>-5</sup></td><td>7.72747×10<sup>-6</sup></td><td>1.60258×10<sup>-6</sup></td><td>3.18534×10<sup>-5</sup></td></tr><tr><td>Â(t; L, M)</td><td>5.27137×10<sup>-5</sup></td><td>2.34896×10<sup>-5</sup></td><td>2.45494×10<sup>-5</sup></td><td>2.83508×10<sup>-5</sup></td></tr><tr><td>M̂(t; L, M)</td><td>5.37922×10<sup>-5</sup></td><td>8.83381×10<sup>-5</sup></td><td>1.829589×10<sup>-4</sup></td><td>2.25083×10<sup>-5</sup></td></tr></tbody></table>
## Influence of Taking into Account the Censoring

We would like now to examine the case of estimating the parameters where in the target case we have censoring (for example at the end). We take into account two different cases: (i) The case of estimating the parameters using the formula with censoring at the end; (ii) and estimating the parameters using (incorrectly) the formula without censoring, under the assumption that the first and the last observation times (0 and M) are jump times.

This is done in order to investigate the influence of taking into account the censoring. For this procedure we consider both statistical settings with one trajectory and several trajectories.

The average number of visited states in $L = 10$ trajectories for the values of $M$ in Table 10 is 8.5, 15.8, 150.1, 1483.3 and 14825.5 respectively. The average length of trajectories (the average total non-censored sojourn time) is 471.4447, 967.3908, 9964.861, 99950.99 and 999956.5 respectively. Observe that, in the cases where the censored part represents approximately more than 1% of the average number of visited states, the target case gives better estimators than the case where we do not take into account the censored part. However, when the censored part represents less than 1% of the average

number of visited states, the contribution of the censored part is not significant.

The number of visited states for the values of $M$ in Table 11 is 7, 19, 147 and 1467 respectively. The length of the trajectory (the total non-censored sojourn time) is 421.7443, 999.8391, 9937.624 and 99988.39 respectively. Observe that, in the cases where the censored part represents the 12% of the number of visited states, the target case gives better estimators than the case where we do not take into account the censored part. However, when the censored part represents less than 12% of the number of visited states, the contribution of the censored part is not significant. Note that the huge difference between the two cases for $M = 500$ is predictable since the censored part is not to so small as compared to the total trajectory.

## Simulations: Reliability

Let us consider the complete (full) case of several sample paths with censoring at the beginning and/or at the end in order to investigate the behavior of several reliability indices that have been proposed in Section 6.

According to Table 12 the Reliability estimator performs better for values of $t$ that belong to the second and third quarter of its domain. However, as for the estimators of Availability and Maintainability, there is no significant difference for the entire spectrum of the time interval.

<table><caption>**Table 9:** S.E. for the estimators of the semi-Markov transition probabilities in the case of censoring at the beginning and/or at the end, for various values of $t$</caption><thead><tr><th>S.E.\t</th><th>1</th><th>25</th><th>50</th><th>100</th></tr></thead><tbody><tr><td>P̂<sub>ij</sub> (t; M)</td><td>1.15749×10<sup>-8</sup></td><td>7.84353×10<sup>-5</sup></td><td>5.87928×10<sup>-3</sup></td><td>5.695643×10<sup>-4</sup></td></tr></tbody></table>

<table><caption>**Table 10:** S.E. for the estimators of $a_{ij}$ in the case of $L = 1000$ trajectories with censoring at the end, for several values of the observation time $M$</caption><thead><tr><th>S.E.\M</th><th>500</th><th>1000</th><th>10000</th><th>100000</th><th>1000000</th></tr></thead><tbody><tr><td>â<sub>ij</sub> (L, M) (case (i))</td><td>0.6119373</td><td>0.04237282</td><td>0.02595444</td><td>0.001830896</td><td>0.0001634836</td></tr><tr><td>â<sub>ij</sub> (L, M) (case (ii))</td><td>0.6571087</td><td>0.0511309</td><td>0.02698702</td><td>0.001829154</td><td>0.0001637393</td></tr></tbody></table>

<table><caption>**Table 11:** S.E. for the estimators of $a_{ij}$ in the case of 1 trajectory with censoring at the end, for several values of the observation time $M$</caption><thead><tr><th>S.E.\M</th><th>500</th><th>1000</th><th>10000</th><th>100000</th></tr></thead><tbody><tr><td>â<sub>ij</sub> (M) (case (i))</td><td>8.754049</td><td>2.637813</td><td>0.3570539</td><td>0.05331928</td></tr><tr><td>â<sub>ij</sub> (M) (case (ii))</td><td>18.75966</td><td>2.6378166</td><td>0.3603736</td><td>0.05330249</td></tr></tbody></table>

<table><caption>**Table 12:** S.E. for Reliability indices in the case of several trajectories ($L = 1000$) with censoring at the beginning and/or at the end, for several values of $t$</caption><thead><tr><th>S.E.\t</th><th>1</th><th>25</th><th>50</th><th>100</th></tr></thead><tbody><tr><td>R̂(t; L, M)</td><td>5.26994×10<sup>-5</sup></td><td>7.72747×10<sup>-6</sup></td><td>1.60258×10<sup>-6</sup></td><td>3.18534×10<sup>-5</sup></td></tr><tr><td>Â(t; L, M)</td><td>5.27137×10<sup>-5</sup></td><td>2.34896×10<sup>-5</sup></td><td>2.45494×10<sup>-5</sup></td><td>2.83508×10<sup>-5</sup></td></tr><tr><td>M̂(t; L, M)</td><td>5.37922×10<sup>-5</sup></td><td>8.83381×10<sup>-5</sup></td><td>1.829589×10<sup>-4</sup></td><td>2.25083×10<sup>-5</sup></td></tr></tbody></table>
## Conclusion

As mentioned earlier the main objective of this work is the proposal of parsimonious modeling for multi-state systems, considering also a semi-Markov framework. This is a useful and powerful tool applicable to diverse scientific fields such as economics and finance, biomedicine, geosciences, engineering, etc. As future work we wish to implement the proposed methodology to geosciences and in particular to seismology using real seismic data. In such a case the seismic zones are considered as the states of a semi-Markov process and the proposed methodology is employed for estimating the transition probabilities of seismic events transit from a specific seismic zone to another one.

## Acknowledgment

The work of the third co-author was done while he held a postdoc position at the University of Rouen, Normandy, France. (financed by the project MOUSTIC).

## Author's Contributions

All authors equally contributed in this work.

## Ethics

This article is original and contains unpublished material. The corresponding author confirms that all of the other authors have read and approved the manuscript and no ethical issues involved.

## References

Balasubramanian, K., M.I. Beg and R.B. Bapat, 1991. On families of distributions closed under extrema. Sankhya A, 53: 375-388.

Barbu, V.S., C. Bérard, D. Cellier, M. Sautreuil and N. Vergne, 2018. An R package for estimation and simulation of discrete-time semi-Markov models. R J., 10: 1-21. DOI: 10.32614/RJ-2018-050

Barbu, V.S., A. Karagrigoriou and A. Makrides, 2017. Semi-Markov modelling for multi-state systems. Meth. Comput. Applied Prob., 19: 1011-1028. DOI: 10.1007/s11009-016-9510-y

Carreira, D.J. and K.H. Chu, 1985. Stress-strain relationship for plain concrete in compression. J. Am. Concrete I., 82: 797-804. DOI: 10.14359/10390

Carreira, D.J. and K.H. Chu, 1986. Stress-strain relationship for plain concrete in tension. J. Am. Concrete I., 83: 21-28. DOI: 10.14359/1756

Courtney, T.H., 2005. Mechanical Behavior of Materials. 2nd Edn., Waveland Press, USA.

El-Neveihi, E. and F. Proschan, 1984. Degradable systems: A survey of multistate system theory. Comm. Statist. Theory Meth., 13: 405-432. DOI: 10.1080/03610928408828694

Limnios, N. and G. Oprițan, 2001. Semi-Markov processes and reliability. Birkhäuser, Boston.

Limnios, N. and B. Ouhbi, 2003. Empirical Estimators of Reliability and Related Functions for Semi-Markov Systems. In: Mathematical and Statistical Methods in Reliability, Lindqvist, B.H. and K.A. Doksum (Eds.), World Scientific, Singapore, pp: 469-484. DOI: 10.1142/9789812795250_0030

Limnios, N. and B. Ouhbi, 2006. Nonparametric estimation of some important indicators in reliability for semi-Markov processes. Stat. Methodol., 3: 341-350. DOI: 10.1016/j.stamet.2005.10.003

Limnios, N., B. Ouhbi and A. Sadek, 2005. Empirical estimator of stationary distribution for semi-Markov processes. Comm. Statist. Theory Methods, 34: 987-995. DOI: 10.1081/STA-200054441

Lisnianski, A., I. Frenkel and Y. Ding, 2010. Multi-state System Reliability Analysis and Optimization for Engineers and Industrial Managers. 1st Edn., Springer, London, ISBN-10: 1849963207, pp: 393

Lisnianski, A. and G. Levitin, 2003. Multi-state System Reliability: Assessment, Optimization and Applications. 1st Edn., World Scientific, Singapore, ISBN-10: 9812383069, pp: 358.

Natvig, B., 1982. Two suggestions of how to define a multistate coherent system. Adv. Applied Probab., 14: 434-455. DOI: 10.2307/1426529

Ouhbi, B. and N. Limnios, 1996. Non-parametric estimation for semi-Markov kernels with application to reliability analysis. Applied Stoch. Models Data Anal., 12: 209-220. DOI: 10.1002/(SICI)1099-0747(199612)12:4<209::AID-ASM284>3.0.CO;2-T
## Conclusion

As mentioned earlier the main objective of this work is the proposal of parsimonious modeling for multi-state systems, considering also a semi-Markov framework. This is a useful and powerful tool applicable to diverse scientific fields such as economics and finance, biomedicine, geosciences, engineering, etc. As future work we wish to implement the proposed methodology to geosciences and in particular to seismology using real seismic data. In such a case the seismic zones are considered as the states of a semi-Markov process and the proposed methodology is employed for estimating the transition probabilities of seismic events transit from a specific seismic zone to another one.

## Acknowledgment

The work of the third co-author was done while he held a postdoc position at the University of Rouen, Normandy, France. (financed by the project MOUSTIC).

## Author's Contributions

All authors equally contributed in this work.

## Ethics

This article is original and contains unpublished material. The corresponding author confirms that all of the other authors have read and approved the manuscript and no ethical issues involved.

## References

Balasubramanian, K., M.I. Beg and R.B. Bapat, 1991. On families of distributions closed under extrema. Sankhya A, 53: 375-388.

Barbu, V.S., C. Bérard, D. Cellier, M. Sautreuil and N. Vergne, 2018. An R package for estimation and simulation of discrete-time semi-Markov models. R J., 10: 1-21. DOI: 10.32614/RJ-2018-050

Barbu, V.S., A. Karagrigoriou and A. Makrides, 2017. Semi-Markov modelling for multi-state systems. Meth. Comput. Applied Prob., 19: 1011-1028. DOI: 10.1007/s11009-016-9510-y

Carreira, D.J. and K.H. Chu, 1985. Stress-strain relationship for plain concrete in compression. J. Am. Concrete I., 82: 797-804. DOI: 10.14359/10390

Carreira, D.J. and K.H. Chu, 1986. Stress-strain relationship for plain concrete in tension. J. Am. Concrete I., 83: 21-28. DOI: 10.14359/1756

Courtney, T.H., 2005. Mechanical Behavior of Materials. 2nd Edn., Waveland Press, USA.

El-Neveihi, E. and F. Proschan, 1984. Degradable systems: A survey of multistate system theory. Comm. Statist. Theory Meth., 13: 405-432. DOI: 10.1080/03610928408828694

Limnios, N. and G. Oprițan, 2001. Semi-Markov processes and reliability. Birkhäuser, Boston.

Limnios, N. and B. Ouhbi, 2003. Empirical Estimators of Reliability and Related Functions for Semi-Markov Systems. In: Mathematical and Statistical Methods in Reliability, Lindqvist, B.H. and K.A. Doksum (Eds.), World Scientific, Singapore, pp: 469-484. DOI: 10.1142/9789812795250_0030

Limnios, N. and B. Ouhbi, 2006. Nonparametric estimation of some important indicators in reliability for semi-Markov processes. Stat. Methodol., 3: 341-350. DOI: 10.1016/j.stamet.2005.10.003

Limnios, N., B. Ouhbi and A. Sadek, 2005. Empirical estimator of stationary distribution for semi-Markov processes. Comm. Statist. Theory Methods, 34: 987-995. DOI: 10.1081/STA-200054441

Lisnianski, A., I. Frenkel and Y. Ding, 2010. Multi-state System Reliability Analysis and Optimization for Engineers and Industrial Managers. 1st Edn., Springer, London, ISBN-10: 1849963207, pp: 393

Lisnianski, A. and G. Levitin, 2003. Multi-state System Reliability: Assessment, Optimization and Applications. 1st Edn., World Scientific, Singapore, ISBN-10: 9812383069, pp: 358.

Natvig, B., 1982. Two suggestions of how to define a multistate coherent system. Adv. Applied Probab., 14: 434-455. DOI: 10.2307/1426529

Ouhbi, B. and N. Limnios, 1996. Non-parametric estimation for semi-Markov kernels with application to reliability analysis. Applied Stoch. Models Data Anal., 12: 209-220. DOI: 10.1002/(SICI)1099-0747(199612)12:4<209::AID-ASM284>3.0.CO;2-T