# Semi-Markov Reliability Models with Recurrence Times and Credit Rating Applications

Authors: Guglielmo D'Amico; Jacques Janssen; Raimondo Manca
Journal: Journal of Applied Mathematics and Decision Sciences
 (2009)

---

## Abstract

We show how it is possible to construct efficient duration dependent semi-Markov
reliability models by considering recurrence time processes. We define generalized reliability indexes and we show how it is possible to compute them. Finally, we describe a possible application in the study of credit rating dynamics by considering the credit rating migration as a reliability problem.


---

## Body

## 1. Introduction

Homogeneous semi-Markov processes (HSMPs) were defined by [1, 2]. A detailed analysis of HSMP was given in [3, 4]. Engineers have used these processes to analyze mechanical, transportation, and informative systems. One of the most important engineering applications of HSMP is in reliability; see [3, 5, 6]. Generalized semi-Markov processes have been proposed and insensitivity phenomenon displayed by stochastic models from the areas of reliability and telephone engineering has been investigated; see [7–10].In its basic form, a reliability problem consists of the analysing the performance of a system that moves randomly in time between many possible states. The state set is partitioned into two subsets. The first subset that is formed by the states in which the system is working and in the second subset by all states in which the system is down are present.Supposing that next state depends only on the last one (the future depends only on the present) the problem can be confronted by means of Markov processes. In discrete time Markov chain environment the distribution functions (d.f.) of the waiting times between transitions are geometric while in continuous time they are negative exponentials. Usually, the transitions happen after random durations in general are not described by memoryless distribution functions (exponential or geometric). This is the reason why HSMP fits better then the Markov one in reliability problems; it offers the possibility of being able to use any distribution function.Semi-Markov processes have been proposed for reliability and performability evaluation of systems; see [6, 11, 12]. In [13], how to apply homogeneous and nonhomogeneous semi-Markov processes in reliability problems is shown.In this paper, in order to study the duration dependence we attach the backward and forward recurrence time processes to the HSMP and we consider them simultaneously at the beginning and at the end of the considered time interval.These processes have been analyzed by several authors; see [4, 5, 14–18]. They allow us to have complete information as regards the waiting times scenarios. In fact(i)
initial backward times take into account the time at which the system went in the state even if the arrival time is before the beginning of the studied time horizon;(ii)
initial forward times consider the time at which the first transition after the beginning of the studied time will happen;(iii)
final backward times take into account the time at which the last transition before the end of the considered time interval is done;(iv)
final forward allows us to consider the time at which the system will exit from the state occupied at the final time.In this way, the use of the initial and final backward and forward processes gives us the possibility of constructing all the waiting time scenarios that could happen in the neighbours of the initial and final observed times. Since HSMP uses no memoryless d.f., different values assumed by recurrence processes change the transition probabilities of the HSMP. Consequently it is possible to define generalized reliability measures which depend on the values assumed by the HSMP and recurrence time processes at starting and ending times.The usefulness of the results is illustrated in the applicative section on the credit risk rating dynamic which is one of the most important problems in financial literature. Fundamentally it consists of computing the default probability of a firm going into debt. The literature on this topic is very wide but the reader can refer to [19–22].In order to evaluate credit risk, international big organisations like Fitch, Moody’s, and Standard & Poor’s give different ranks to firms which agree to be evaluated.Each firm receives a “rating” representing an evaluation of the “reliability” of its capacity to reimburse debt. Clearly, the lower the rating, the higher the interest rate the evaluated firm should pay.The rating level changes in the time and one way to model it is by means of Markov processes as in [23]. In this environment Markov models are called “migration models.” The poor fitting of Markov process in credit risk environment was outlined in [24–26].In our opinion, the credit rating problem can be included in the more general problem of the reliability of a stochastic system as already highlighted in [27–30]. Indeed, rating agencies, through the assessment of a rating, estimate the reliability of the firm issuing a bond.In this paper we generalize the results of [27, 28] and another step is made in order to establish a link between credit rating and reliability models. Moreover the duration dependence of the rating evolution can be fully captured by means of recurrence times.Section2 will present a short description of HSMP. In Section 3 the backward and forward recurrence time processes are presented and their general distributions are determined. Section 4 is devoted to the initial and final backward semi-Markov reliability models. Section 5 presents the credit risk model with complete information as regards the waiting times. In the last section some concluding remarks are given.


## 2. Homogeneous Semi-Markov Processes

We follow the notation given in [4]. On a complete probability space (Ω,ℑ,P)we define two random variables:(i)
Jn,n∈ℕ with state space I={1,…,m}representing the state at the nth transition.(ii)
Tn,n∈ℕ with ℝ+ as state space representing the time of the nth transition.The process(Jn,Tn) is supposed to be a homogeneous Markov renewal process (HMRP) of kernel Q=[Qij(t)] so that(2.1)Qij(t)=P[Jn+1=j,Tn+1-Tn≤t∣Jn=i].
We know that(2.2)pij≐P(Jn=j∣Jn-1=i)=limt→∞Qij(t),i,j∈I,t∈ℝ+,
soP=[pij] is the transition matrix of the embedded Markov chain {Jn}. Furthermore, it is necessary to introduce the probability that the process will leave state i in a time t:(2.3)Hi(t)≐P[Tn+1-Tn≤t∣Jn=i]=∑j∈IQij(t).
It is possible to define the distribution function of the waiting time in each state i, given that the state j, successively occupied, is known as(2.4)Gij(t)≐P[Tn+1-Tn≤t∣Jn=i,Jn+1=j]={Qij(t)pij,ifpij≠0,1,ifpij=0.
The main difference between a continuous time Markov process and an HSMP is in the distribution functions Gij(t). In a Markov environment this function has to be a negative exponential function of parameter λij; on the other hand, in the semi-Markov case the distribution functions Gij(t) can be of any type. This fact means that in order to consider the duration effects we can use the functionsGij(t). The HSMP Z=(Z(t),t∈ℝ+) represents, for each waiting time, the state occupied by the system, that is,(2.5)Z(t)=JN(t),whereN(t)=max{n:Tn≤t}.
The HSMP transition probabilities are defined in the following way:(2.6)ϕij(t)=P[Z(t)=j∣Z(0)=i,TN(0)=0].
They are obtained solving the following evolution equations (see among others [4]):(2.7)ϕij(t)=δij(1-Hi(t))+∑β∈I∫0tQ̇iβ(ϑ)ϕβj(t-ϑ)dϑ,
where δij represents the Kronecker symbol.The first part of formula (2.7), δij(1-Hi(t)), gives the probability of the system not having transitions up to time t given that it entered in state i at time 0.In the second part,∑β=1m∫0tQ̇iβ(ϑ)ϕβj(t-ϑ)dϑ,Q̇iβ(ϑ)dϑ represents the probability of the system not making the next transition in [ϑ,ϑ+dϑ) moving from state ito state β. After the transition, the system will go to state j following one of all possible trajectories that go from state β to state j in a timet-ϑ.


## 3. Backward and Forward Recurrence Time Processes

In this section we present some generalizations of the HSMP transition probabilities (2.7) using the recurrence time processes.Recurrence time processes have been investigated by many authors. For example, in [14, 15] the backward process at starting time was used to determine the asymptotic distribution of an ergodic HSMP. In [5] the backward processes were considered both at starting and arriving times in the transition probabilities but, to the authors’ knowledge, a complete study such as that given in the next subsection has never been presented.Given the HMRP(Jn,Tn), we define the following stochastic processes of recurrence times:(3.1)B(t)=t-TN(t),F(t)=TN(t)+1-t.
The process B(t) is called the backward recurrence time(or age)process and F(t)F the forward (or residual time) recurrence time process (see [4]).The recurrence time processes complement the semi-Markov to a Markov process with respect toℑt+≡σ{Z(τ),F(τ),τ∈[0,t]},t≥0and for this reason they are often called auxiliary processes. Then for any bounded I×ℝ+-measurable function f(x,t),s≤t, it results that(3.2)E[f(Z(t),F(t))∣ℑs+]=E[f(Z(t),F(t))∣Z(s),F(s)].
For (3.2) see [16].Figure1 presents an HSMP trajectory in which we reported the recurrence time processes. At time sthe HSMP is in state Z(s)=i, it entered in this state with its last transition at time Tn=s-vthen the initial backward value is B(s)=v.The process makes next transition at time Tn+1=s+u, then the initial forward value is F(s)=u. At time t+s the HSMP is in state Z(t+s)=j, it entered in this state with its last transition at time Th-1=t+s-v′ then the initial backward value is B(t+s)=v′.The process makes next transition at time Th=t+s+u′, then the initial forward value is F(t+s)=u′.Figure 1
Trajectory of backward and forward SMP.Our objective, in this section, is to define and compute transition probabilities that are constrained at initial times and at final time t+sby the recurrence time processes. To be more precise, given the information(Z(s)=i,B(s)=v,F(s)=u), we want to compute the probability of having (Z(t+s)=j,B(t+s)=v′,F(t+s)=u′).In order to clarify the presentation, we show, firstly some particular cases.(a) Letϕbij(v;t)=P[Z(t)=j∣Z(0)=i,B(0)=v]be the transition probability with initial backward value v. It denotes the probability of being in state j after t periods given that at present the process is in state i and it entered into this state with the last transition v periods before.Using the relation(3.3){Z(t)=i,B(t)=v}={JN(t)=i,TN(t)=t-v,TN(t)+1>t},
it can be proved that(3.4)ϕbij(v;t)=δij(1-Hi(t+v))1-Hi(v)+∑k∈I∫0tQ̇ik(τ+v)1-Hi(v)ϕkj(t-τ)dτ.
An explanation of (3.4) can be provided. The term (1-Hi(t+v))/(1-Hi(v)) represents the probability of remaining in state i for t+v times given that the process will stay in state i for v times. This probability contributes to ϕbij(v;t) only if i=j.The second term∑k∈I∫0t(Q̇ik(τ+v)/(1-Hi(v)))φkj(t-τ)dτ expresses the probability of a trajectory making provision for the entrance into the state k after τ periods given that the process remained in state i for v times; then the transition in state j in the remaining time  t-τfrom state k has to be carried out. This holds for all states k∈I and timesτ∈[0,t].Note that ifv=0,ϕbij(v;t)=ϕij(t) recovering the ordinary HSMP transition probabilities (2.7).(b) Letϕfij(u;t)=P[Z(t)=j∣Z(0)=i,F(0)=u] be the transition probability with fixed initial forward value u. It denotes the probability of being in state j after a time t given that, at time zero, the process entered into state i and it makes next transition just at time u into whatever state(3.5)ϕfij(u;t)=∑k∈IdQik(u)dHi(u)ϕkj(t-u).
The term dQik(u)/dHi(u) is the Radon-Nikodym derivative of Qik(·) with respect toHi(·). It expressesdQik(u)/dHi(u)=P[Jn+1=k∣Jn=i,Tn+1-Tn=u]. In this way, in (3.5) the entrance into all possible states k with next transition at time u is considered and then, from that state the process has to occupy state j at time t.(c) LetϕFij(u;t)=P[Z(t)=j∣Z(0)=i,F(0)>u] be the transition probability with initial certain waiting value u. It denotes the probability of being in state j at time t given that, at time zero, the process entered into state i and it makes next transition after time u into any state, then we are sure that, up to time u, the process is still in state i(3.6)ϕFij(u;t)=δij(1-Hi(t))1-Hi(u)+∑k∈I∫0t-uQ̇ik(τ+u)1-Hi(u)ϕkj(t-u-τ)dτ.
The first term δij(1-Hi(t))/(1-Hi(u)) gives us the probability of remaining up to time t in state i given that the process will stay in that state at least up to time u. The second term considers the possibility of evolving with the next transition into any state at any time τ∈(u,t] given no movement up to time u. After the transition at timeτ, the system will be in the state j at time t following one of the possible trajectories.(d) LetϕijB(;v′,t)=P[Z(t)=j,B(t)≤t-v′∣Z(0)=i]be the transition probability with final backward value v′.It denotes the probability of being in statej at time t given that the process entered into state j with its last transition within the interval [v′,t] given that it entered at time zero in state i(3.7)ϕijB(;v′,t)=δij(1-Hi(t))1{v′=0}+∑k∈I∫0tQ̇ik(τ)ϕkjB(;v′-τ,t-τ)dτ.
The term (1-Hi(t)) gives us the probability of remaining from time zero up to time t in state i; this probability contributes to ϕijB(;v′,t) only if i=jand v′=0. In fact, the system will be in state j at time t without any transition in the time interval [0,t] only if i=j, then the backward value at time t has, necessarily, to be equal to the length of the interval that is B(t)=t this implies B(t)=t-v′=t⇔v′=0. The second term considers the possibility of evolving with the next transition into any state at any time τ∈[0,t] and then we have to consider all possible trajectories that will bring the system into state j in the remaining time t-τwith its last transition into state j at a time that belongs to the interval[v′-τ,t-τ]. In this way, the final backward will be less or equal to t-τ-(v′-τ)=t-v′ as required in relation (3.7).(e) LetϕijF(;t,u′)=P[Z(t)=j,F(t)≤u′-t∣Z(0)=i]be the transition probability with final forward value u′.It denotes the probability of being in statej at time t and of making the next transition in the time interval (t,u′] given that the process entered at time zero into state i(3.8)ϕijF(;t,u′)=δij(Hi(u′)-Hi(t))+∑k∈I∫0tQ̇ik(τ)ϕkjF(;t-τ,u′-τ)dτ.
The first term(Hi(u′)-Hi(t)) gives us the probability of exiting for the first time from state i in the time interval (t,u′]; this probability contributes to ϕijF(;t,u′) only if i=j. The second term considers the possibility of evolving with the next transition into any state at any timeτ∈[0,t] and then we have to consider all possible trajectories that will bring in state j in the remaining time t-τ with a final forward ofu′-τ.


### 3.1. The General Distributions of the Auxiliary Processes

There are a lot of cases in which combinations of the previous equations can be considered. In this subsection, we explain the equations for the two general cases.(f) LetϕbfijBF(v,u;v′,t,u′)=P[Z(t)=j,B(t)≤t-v′,F(t)≤u′-t∣Z(0)=i,B(0)=v,F(0)=u]be the transition probability with initial and final backward and forward values v, u, v′, u′. It denotes the probability of being in state j at time t and of entering into that state in the time interval [v′,t] and of making next transition in (t,u′] given that at present the process is in state i and it entered into this state with the last transition v periods before and it remained until time u where a transition took place.It results that(3.9)ϕbfijBF(v,u;v′,t,u′)=∑k∈IdQik(v+u)dHi(v+u)ϕkjBF(;v′-u,t-u,u′-u),
where(3.10)ϕijBF(;v′,t,u′)=P[Z(t)=j,B(t)≤t-v′,F(t)≤u′-t∣Z(0)=i]=δij(Hi(u′)-Hi(t))1{v′=0}+∑k∈I∫0tQ̇ik(τ)φkjBF(;v′-τ,t-τ,u′-τ)dτ.
Equation (3.10) is composed of two parts, the first expresses the probability of exiting for the first time from state i in the time interval (t,u′] this contributes to ϕijBF(;v′,t,u′) when i=j then the backward at time t must be equal to t periods, that is, B(t)≡t-v′=t⇔v′=0; the second term considers the entrance into any state k at any time τ∈(u,t] and then the system has to be in state j at time t with final backward and forward values at a maximum of t-v′ and u′-t.Consequently, relation (3.9) states that to compute ϕbfijBF(v,u;v′,t,u′) it is enough to consider the probability ϕkjBF(;v′-u,t-u,u′-u) using a random starting distribution represented by the Radon-Nikodym derivativedQik(v+u)/dHi(v+u).(g) LetϕbFijBF(v,u;v′,t,u′)=P[Z(t)=j,B(t)≤t-v′,F(t)≤u′-t∣Z(0)=i,B(0)=v,F(0)>u]be the transition probability with initial backward and starting certain waiting forward and final backward and forward values v, u, v′, u′. It denotes the probability of being in state j at time t and of entering that state in the time interval [v′,t] and of making the next transition in (t,u′] given that at present the process is in state i and it entered into this state with the last transition v periods before and it remained in this state at least until time u. It results that(3.11)ϕbFijBF(v,u;v′,t,u′)=δij[Hi(u′+v)-Hi(t+v)1-Hi(u+v)]1{v′=-v}+∑k∈I∫0t-uQ.ik(τ+u+v)1-Hi(u+v)φkjBF(;v′-u-τ,t-u-τ,u′-u-τ)dτ.
The term [(Hi(u′+v)-Hi(t+v))/(1-Hi(u+v))] gives us the probability of exiting from state i for the first time in the interval (t,u′] given that at present the process is in state i and it entered into this state with the last transition v periods before and up to time u (i.e., for u+v periods) no new transitions were carried out by the process. This probability is part of ϕbFijBF(v,u;v′,t,u′) if i=j; consequently, the backward process at time t has to be equal tot+v, that is,B(t)≡t-v′=t+v⇔v′=-v.The other term considers the entrance into the statek at timeτ∈(u,t], given that at present the process is in state i and it entered into this state with the last transition v periods before with no movement from that state up to time u; then all possible trajectories that will bring the system into state j at time t with a final backward and forward of, respectively, t-v′ and u′-t given the entrance into state k at time τ have to be considered.


## 3.1. The General Distributions of the Auxiliary Processes

There are a lot of cases in which combinations of the previous equations can be considered. In this subsection, we explain the equations for the two general cases.(f) LetϕbfijBF(v,u;v′,t,u′)=P[Z(t)=j,B(t)≤t-v′,F(t)≤u′-t∣Z(0)=i,B(0)=v,F(0)=u]be the transition probability with initial and final backward and forward values v, u, v′, u′. It denotes the probability of being in state j at time t and of entering into that state in the time interval [v′,t] and of making next transition in (t,u′] given that at present the process is in state i and it entered into this state with the last transition v periods before and it remained until time u where a transition took place.It results that(3.9)ϕbfijBF(v,u;v′,t,u′)=∑k∈IdQik(v+u)dHi(v+u)ϕkjBF(;v′-u,t-u,u′-u),
where(3.10)ϕijBF(;v′,t,u′)=P[Z(t)=j,B(t)≤t-v′,F(t)≤u′-t∣Z(0)=i]=δij(Hi(u′)-Hi(t))1{v′=0}+∑k∈I∫0tQ̇ik(τ)φkjBF(;v′-τ,t-τ,u′-τ)dτ.
Equation (3.10) is composed of two parts, the first expresses the probability of exiting for the first time from state i in the time interval (t,u′] this contributes to ϕijBF(;v′,t,u′) when i=j then the backward at time t must be equal to t periods, that is, B(t)≡t-v′=t⇔v′=0; the second term considers the entrance into any state k at any time τ∈(u,t] and then the system has to be in state j at time t with final backward and forward values at a maximum of t-v′ and u′-t.Consequently, relation (3.9) states that to compute ϕbfijBF(v,u;v′,t,u′) it is enough to consider the probability ϕkjBF(;v′-u,t-u,u′-u) using a random starting distribution represented by the Radon-Nikodym derivativedQik(v+u)/dHi(v+u).(g) LetϕbFijBF(v,u;v′,t,u′)=P[Z(t)=j,B(t)≤t-v′,F(t)≤u′-t∣Z(0)=i,B(0)=v,F(0)>u]be the transition probability with initial backward and starting certain waiting forward and final backward and forward values v, u, v′, u′. It denotes the probability of being in state j at time t and of entering that state in the time interval [v′,t] and of making the next transition in (t,u′] given that at present the process is in state i and it entered into this state with the last transition v periods before and it remained in this state at least until time u. It results that(3.11)ϕbFijBF(v,u;v′,t,u′)=δij[Hi(u′+v)-Hi(t+v)1-Hi(u+v)]1{v′=-v}+∑k∈I∫0t-uQ.ik(τ+u+v)1-Hi(u+v)φkjBF(;v′-u-τ,t-u-τ,u′-u-τ)dτ.
The term [(Hi(u′+v)-Hi(t+v))/(1-Hi(u+v))] gives us the probability of exiting from state i for the first time in the interval (t,u′] given that at present the process is in state i and it entered into this state with the last transition v periods before and up to time u (i.e., for u+v periods) no new transitions were carried out by the process. This probability is part of ϕbFijBF(v,u;v′,t,u′) if i=j; consequently, the backward process at time t has to be equal tot+v, that is,B(t)≡t-v′=t+v⇔v′=-v.The other term considers the entrance into the statek at timeτ∈(u,t], given that at present the process is in state i and it entered into this state with the last transition v periods before with no movement from that state up to time u; then all possible trajectories that will bring the system into state j at time t with a final backward and forward of, respectively, t-v′ and u′-t given the entrance into state k at time τ have to be considered.


## 4. Continuous Time Homogeneous Waiting Times Complete Knowledge Semi-Markov Reliability Model

There are a lot of semi-Markov models in reliability theory; see, for example, [5, 6, 11–13, 31]. The nonhomogeneous case was presented in [13]. More recently, in [28] a nonhomogeneous backward semi-Markov reliability model was presented.In this section, we will generalize these reliability models taking into account initial and final backward and forward processes all together in a homogeneous environment.Let us consider a reliability system that can be at every timet in one of the states ofI={1,…,m}. The stochastic process of the successive states of the system is denoted byZ={Z(t),t≥0}.The state set is partitioned into setsU and D, so that(4.1)I=U∪D,∅=U∩D,U≠∅,U≠I.
The subset U contains all “good” states in which the system is working and subset D all “bad” states in which the system is not working well or has failed.The classic indicators used in reliability theory are the following ones.(i)The pointwise availability function  A giving the probability that the system is working at time t regardless of what has happened on (0,t]:(4.2)A(t)=P[Z(t)∈U].(ii)The reliability function  R giving the probability that  the system was always working from time 0 to time t:(4.3)R(t)=P[Z(u)∈U:∀u∈(0,t]].(iii)The maintainability function  M giving the probability that the system will leave the set D within the time t being in D at time 0:(4.4)M(t)=1-P[Z(u)∈D,∀u∈(0,t]].
Considering the generalizations presented in the previous section we give the following new definitions.(i′) The pointwise homogeneous waiting times complete knowledge availability function with fixed initial forwardAbfiBF giving the probability that the system is working at time t, given that the entrance into the state Z(0)=i with v as initial backward time and that the process moves from this state just at time u. Furthermore, we require the system to enter into the working state Z(t) at time TN(t)≥v′ and to remain in this state until the time of the next transition TN(t)+1≤u′:(4.5)AbfiBF(v,u;v′,t,u′)=P[Z(t)∈U,B(t)≤t-v′,F(t)≤u′-t∣Z(0)=i,B(0)=v,F(0)=u].(i′′) The pointwise homogeneous waiting times complete knowledge availability function  with starting certain waiting  forward timeAbFiBF giving the probability that the system is working on time t, conditioned by the entrance into the state Z(0)=i with v as initial backward time and that the process does not move from this state up to the time u. Furthermore, we require the system to enter the working state Z(t) at time t≥TN(t)≥v′ and to remain in this state until the time of the next transition t<TN(t)+1≤u′:(4.6)AbFiBF(v,u;v′,t,u′)=P[Z(t)∈U,B(t)≤t-v′,F(t)≤u′-t∣Z(0)=i,B(0)=v,F(0)>u].(ii′) The homogeneous waiting times complete knowledge reliability function  with fixed initial forwardRbfiBF giving the probability that the system was always working for a time t, given that the entrance into the state Z(0)=i was v time periods before 0 and the next transition is at time u. Furthermore, it is assumed that the system did the N(t)th transition at time t≥TN(t)≥v′ and the (N(t)+1)th at timet<TN(t)+1≤u′:(4.7)RbfiBF(v,u;v′,t,u′)=P[Z(h)∈U∀h∈(0,t],B(t)≤t-v′,F(t)≤u′-t∣Z(0)=i,B(0)=v,F(0)=u].(ii′′) The homogeneous waiting times complete knowledge reliability function  with starting certain waiting  forward timeRbfiBF giving the probability that the system was always working for a time t, given that the entrance into the state Z(0)=i was v time periods before 0 and the next transition is after time u. Furthermore, it is assumed that the system did the N(t)th transition at time t≥TN(t)≥v′ and the (N(t)+1)th at time t<TN(t)+1≤u′:(4.8)RbFiBF(v,u;v′,t,u′)=P[Z(h)∈U∀h∈(0,t],B(t)≤t-v′,F(t)≤u′-t∣Z(0)=i,B(0)=v,F(0)>u].(iii′) The homogeneous waiting times complete knowledge maintainability functionMbfiBF giving the probability that the system will leave the set D going into an up state, at least once, within the time t and thatt≥TN(t)≥v′, t<TN(t)+1≤u′ given that at present the process is in state i∈Dand it entered this state with the last transition v periods before and at time u a new transition occurs(4.9)MbfiBF(v,u;v′,t,u′)=P[TN(t)≥v′,TN(t)+1≤u′,∃h∈(0,t]:Z(h)∈U∣Z(0)=i∈D,B(0)=v,F(0)=u].(iii′′) The homogeneous waiting times complete knowledge maintainability function  with starting certain waiting  forward timeMbFiBF giving the probability that the system will leave the set D going into an up state within the time t and that t≥TN(t)≥v′, t<TN(t)+1≤u′ given that at present the process is in state i∈Dand it entered into this state with the last transition v periods before and after the time u a new transition occurs(4.10)MbFiBF(v,u;v′,t,u′)=P[TN(t)≥v′,TN(t)+1≤u′,∃h∈(0,t]:Z(h)∈U∣Z(0)=i∈D,B(0)=v,F(0)>u].
The probabilities (4.5), (4.6), (4.7), (4.8), (4.9), and (4.10) can be computed as follows.(i)
The pointwise availability functions (4.5) and (4.6) are, respectively,(4.11)AbfiBF(v,u;v′,t,u′)=∑j∈UϕbfijBF(v,u;v′,t,u′),(4.12)AbFiBF(v,u;v′,t,u′)=∑j∈UϕbFijBF(v,u;v′,t,u′).(ii)
The reliability functions (4.7) and (4.8) are, respectively,(4.13)RbfiBF(v,u;v′,t,u′)=∑j∈UϕbfijBFR(v,u;v′,t,u′),(4.14)RbFiBF(v,u;v′,t,u′)=∑j∈UϕbFijBFR(v,u;v′,t,u′),
whereϕbfijBFR(v,u;v′,t,u′) and ϕbFijBFR(v,u;v′,t,u′)are the solutions to (3.9) and (3.11) with all the states in D that are absorbing.To compute these probabilities all the states of the subsetD are changed into absorbing states through the following transformation of the semi-Markov kernel:(4.15)pij=δijifi∈D.(iii)
The maintainability function (4.9) and (4.10) are, respectively,(4.16)MbfiBF(v,u;v′,t,u′)=∑j∈UϕbfijBFM(v,u;v′,t,u′),(4.17)MbFiBF(v,u;v′,t,u′)=∑j∈UϕbFijBFM(v,u;v′,t,u′),
whereϕbfijBFM(v,u;v′,t,u′) and ϕbFijBFM(v,u;v′,t,u′) are, respectively, the solution to (3.9) and (3.11) with all the states in U that are absorbing.In this case, all the states of the subsetU are changed into absorbing states through the transformation of the semi-Markov kernel:(4.18)pij=δijifi∈U.
The here-defined reliability indexes are able to assess different probabilities depending on the backward and forward process values. This makes it possible to obtain, for example, a complete knowledge of the variability of the survival probabilities of the system depending on the waiting time scenario.


## 5. The Homogeneous Waiting Time with Complete Knowledge for Semi-Markov Reliability Credit Risk Models

The rating migration problem can be situated in the reliability environment. The rating process, done by the rating agency, states the degree of reliability of a firm’s bond.The problem of the poor fit of the Markov process in a credit risk environment was outlined in [24–26]. The duration effect of rating transitions is one of the main problems which demonstrate the inadequacy of Markov models. The probability of changing rating depends on the time that a firm remains within the same rating class (see [32]). This problem can be solved satisfactorily by means of HSMP; see [27, 28, 33]. In fact, as already explained, in HSMP the transition probabilities are a function of the waiting time spent in a state of the system.The knowledge of the waiting times around the beginning and the end of the considered interval is of fundamental relevance in credit rating migration modelling. Indeed, the solutions of the evolution equations (3.9) and (3.11) consider the duration time inside the starting and the arriving states.In the next two subsections, we will present two semi-Markov reliability credit risk models.To construct a semi-Markov model, it is necessary to construct the embedded Markov chain (2.2) and to find the d.f. of waiting times (2.4). The embedded Markov chain constructed by real data of Standard & Poor’s rating agency was given in [34] and it is reported in the next sub-section. This matrix is aperiodic and irreducible and has two down states D and NR. In the following sub-section the case in which the default state is supposed to be absorbing is studied and the No-Rating state is not considered. Under these hypotheses, the embedded Markov chain is mono-unireducible; see [35].


### 5.1. The Irreducible Case with Two Down States

For example the rating agency Standard & Poor’s considers 9 different classes of rating and the No Rating state, so we have the following set of states:(5.1)I1={AAA,AA,A,BBB,BB,B,CCC,D,NR}.
The ratings express the creditworthiness of the rated firm. The creditworthiness is the highest for the rating AAA, assigned to firm extremely reliable with regard to financial obligations, and decrease towards the rating D which expresses the occurrence of payment default on some financial obligation. A table showing the financial meaning of the Standard & Poors rating categories is reported in [21]. As a matter of example, the rating B is assigned to firm vulnerable to changes in economic conditions currently showing the ability to meet its financial obligations.The first 7 states are working states (good states) and the last two are the nonworking states. The two subsets are the following:(5.2)U={AAA,AA,A,BBB,BB,B,CCC},D1={D,NR}.
By solving the different evolution equations we obtain the following results.(1.1′) ϕbfijBF(v,u;v′,t,u′) represents the probabilities of being in the state j after a time t starting in state i with an initial backward time v, an initial forward time u, with v′≤TN(t)≤t<TN(t)+1≤u′.(1.1′′) ϕbFijBF(v,u;v′,t,u′) represents the probabilities of being in the state j after a time tstarting in the state i with an initial backward time v, a starting certain forward time u, with v′≤TN(t)≤t<TN(t)+1≤u′.Both the results take into account the different probabilities of changing state as a function of all the possible entrance and exit times in the starting and arriving states. In the first case, the exit will be just at timeu and in the second case after a time u.(1.2′) AbfiBF(v,u;v′,t,u′)=∑j∈UϕbfijBF(v,u;v′,t,u′) represents the probability of the system having, at time t, an up rating given that it entered the state i with an initial backward time v and exited from i at time u (forward initial time).(1.2′′) AbFiBF(v,u;v′,t,u′)=∑j∈UϕbFijBF(v,u;v′,t,u′) represents the probability of the system having, at time t, an up rating given that it entered the state i with an initial backward time v and exited from i after time u (certain waiting forward time).(2.1′) ϕbfijBFR(v,u;v′,t,u′) represents the probability of being in the state j after a time t starting in the state i with an initial backward time v, an initial forward time u, with v′≤TN(t)≤t<TN(t)+1≤u′given that the two down states are considered absorbing.(2.1′′) ϕbFijBFR(v,u;v′,t,u′) represents the probability of being in the state j after a time t starting in the state i∈U with an initial backward time v, a starting certain forward time u, with v′≤TN(t)≤t<TN(t)+1≤u′ given that the two down states are considered absorbing.(2.2′) RbfiBF(v,u;v′,t,u′)=∑j∈UϕbfijBFR(v,u;v′,t,u′) represents the probability that the system was always up in the time interval (0,t] given that it entered the state i∈U with an initial backward time v and it exits from i at time u (forward initial time) considering the two down states as absorbing.(2.2′′) RbFiBF(v,u;v′,t,u′)=∑j∈UϕbFijBFR(v,u;v′,t,u′) represents the probability that the system was always up in the time interval (0,t] with an initial backward time v and it exited from i after time u (certain waiting forward time) considering the two down states as absorbing.(3.1′) ϕbfijBFM(v,u;v′,t,u′) represents the probabilities of being in the state j after a time t starting in the state i∈D with an initial backward time v, an initial forward time u, with v′≤TN(t)≤t<TN(t)+1≤u′given that all the up states are considered absorbing.(3.1′′) ϕbFijBFM(v,u;v′,t,u′) represents the probabilities of being in state j after a time t starting in the state i∈D with an initial backward time v, a starting certain forward time u, with v′≤TN(t)≤t<TN(t)+1≤u′ given that all the up states are considered absorbing.(3.2′) MbfiBF(v,u;v′,t,u′)=∑j∈UϕbfijBFM(v,u;v′,t,u′) represents the probability that the system at time t has an up rating given that it entered into the state i∈D with an initial backward time v and it exited from i at time u (forward initial time) given that all the up states are considered absorbing.(3.2′′) MbFiBF(v,u;v′,t,u′)=∑j∈UϕbFijBFM(v,u;v′,t,u′) represents the probability that the system at time t has an up rating given that it entered the state i∈D with an initial backward time v and exited from i after time u (certain waiting forward time) given that all the up states are considered absorbing.The maintainability functionM has a precise financial meaning. It assesses the probability of a firm leaving state D within time t through reorganization. In fact, if the firm is reorganized the rating agency will give a new rating which evaluates the new financial situation. In the No Rating state, the re-entrance of the firm in the bond market will imply a new rating evaluation.In Table1 the embedded Markov chain obtained considering all the transitions of the historical data base of S&P is given. This matrix was presented in [34].Table 1
Embedded Markov chain obtained by the S&P data.
StatesAAAAAABBBBBBCCCDNRAAA0.3285080.1011050.0365080.0218980.0105500.0059240.0014310.0005660.493510AA0.0517660.4649720.1838820.0244960.0085930.0042370.0014320.0004480.260174A0.0218470.1005150.4530240.1270870.0119710.0065840.0020740.0007910.276107BBB0.0150100.0256070.1320550.3784570.1230320.0184110.0052840.0014050.300736BB0.0069660.0161080.0304750.1935860.2772700.1836210.0179460.0027570.271272B0.0039830.0072450.0154500.0245190.1558470.3286310.1609330.0129070.290485CCC0.0019600.0053900.0111480.0149450.0182530.1269140.3417860.2235700.256033D0.0048920.0077460.0101920.0228290.0228290.0521810.0986550.0966160.684060NR0.0703020.0850190.1167810.0897680.0444200.0386080.0120000.0025020.540599


### 5.2. The Default as Absorbing Case

In many credit risk migration models, the NoRating state is ignored and the Default is considered as an absorbing state. Under these hypotheses, the embedded Markov chain of the semi-Markov process has only two classes of states. The first is a transient class and the second is an absorbing class. The absorbing class is constituted by only one state and all the elements of the main diagonal of the matrix are always greater than zero. This kind of matrices (and the corresponding processes) is called monounireducible; see [35].The set of states becomes(5.3)I2={AAA,AA,A,BBB,BB,B,CCC,D},
and the partition in up and down states is(5.4)U={AAA,AA,A,BBB,BB,B,CCC},D2={D}.
The embedded Markov chain is reported in Table 2.Table 2
S&P embedded Markov chain with default as an absorbing state.
StatesAAAAAABBBBBBCCCDAAA0.6485970.1996190.0720810.0432350.0208290.0116960.0028250.001117AA0.0699710.6284880.2485480.033110.0116150.0057270.0019360.000605A0.030180.1388540.6258160.175560.0165370.0090950.0028640.001093BBB0.0214660.036620.1888490.5412230.1759460.026330.0075570.00201BB0.0095590.0221040.0418190.2656490.3804850.2519750.0246270.003784B0.0056130.0102120.0217760.0345570.2196520.4631770.2268210.018192CCC0.0026350.0072450.0149840.0200890.0245350.1705910.4594110.30051D00000001In this case, it results that reliability and availability correspond. Indeed, the only down state is absorbing and if the system at timet is available it means that it never went into the default state and so it remained for all the observation time in a up state, which is the reliability definition. Furthermore, maintainability does not make sense because it is not possible to exit from the default state.The following results can be obtained(1.1′) ϕbfijBF(v,u;v′,t,u′) represents the probabilities of being in the state j after a time t starting in the state i with an initial backward time v, an initial forward time u, with v′≤TN(t)≤t<TN(t)+1≤u′.(1.1′′) ϕbFijBF(v,u;v′,t,u′) represents the probabilities of being in the state j after a time t starting in the state i with an initial backward time v, starting certain forward time u, with v′≤TN(t)≤t<TN(t)+1≤u′.Both the results take into account the different probabilities of changing state during the permanence of the system in the same state considering all the possible entrance and the exit times in the starting and arriving states. In the first case, the exit will be just at timeu, in the second case after a time u.(1.2′) AbfiBF(v,u;v′,t,u′)=RbfiBF(v,u;v′,t,u′)=∑j∈UϕbfijBF(v,u;v′,t,u′)represents the probability that the system at time t has an up rating given that it entered the state i∈U with an initial backward time v and it exits from i at time u (forward initial time).(1.2′′) AbFiBF(v,u;v′,t,u′)=RbFiBF(v,u;v′,t,u′)=∑j∈UϕbFijBF(v,u;v′,t,u′)represents the probability that the system at time t has an up rating given that it entered the state i∈U with an initial backward time v and it exits from i after time u (certain waiting forward time).Remark 5.1.
We wish to mention that the Markov matrices that are given yearly in the Standard & Poor’s publications always have greater elements on the main diagonal compared to the matrices that are presented in this paper. The reason for this is that, in a semi-Markov environment the transitions occur only when there is a real check on the state. In a credit risk environment this means that a transition is computed if and only if the rating agency assigns a new rating, given that the firm already has a rating. In the S&P transition Markov chain, if in a year there was no rating evaluation of a firm, it is supposed that the firm is still in the same state. Then the rating agency, in the construction of the transition matrix, considers a “virtual” transition (transition from that state into the same state). This implies that the number of virtual transitions is very high and the Markov chain, almost everywhere, becomes diagonally dominant. In the embedded Markov chain of the SMP, the virtual transitions are possible, but they happen when the rating agency gives a new rating which is equal to the previous one.
We think another reason for the superior performance of the semi-Markov environment as compared to that of the Markov one is the fact that the former only considers the transitions of the rating process which actually occurred.


## 5.1. The Irreducible Case with Two Down States

For example the rating agency Standard & Poor’s considers 9 different classes of rating and the No Rating state, so we have the following set of states:(5.1)I1={AAA,AA,A,BBB,BB,B,CCC,D,NR}.
The ratings express the creditworthiness of the rated firm. The creditworthiness is the highest for the rating AAA, assigned to firm extremely reliable with regard to financial obligations, and decrease towards the rating D which expresses the occurrence of payment default on some financial obligation. A table showing the financial meaning of the Standard & Poors rating categories is reported in [21]. As a matter of example, the rating B is assigned to firm vulnerable to changes in economic conditions currently showing the ability to meet its financial obligations.The first 7 states are working states (good states) and the last two are the nonworking states. The two subsets are the following:(5.2)U={AAA,AA,A,BBB,BB,B,CCC},D1={D,NR}.
By solving the different evolution equations we obtain the following results.(1.1′) ϕbfijBF(v,u;v′,t,u′) represents the probabilities of being in the state j after a time t starting in state i with an initial backward time v, an initial forward time u, with v′≤TN(t)≤t<TN(t)+1≤u′.(1.1′′) ϕbFijBF(v,u;v′,t,u′) represents the probabilities of being in the state j after a time tstarting in the state i with an initial backward time v, a starting certain forward time u, with v′≤TN(t)≤t<TN(t)+1≤u′.Both the results take into account the different probabilities of changing state as a function of all the possible entrance and exit times in the starting and arriving states. In the first case, the exit will be just at timeu and in the second case after a time u.(1.2′) AbfiBF(v,u;v′,t,u′)=∑j∈UϕbfijBF(v,u;v′,t,u′) represents the probability of the system having, at time t, an up rating given that it entered the state i with an initial backward time v and exited from i at time u (forward initial time).(1.2′′) AbFiBF(v,u;v′,t,u′)=∑j∈UϕbFijBF(v,u;v′,t,u′) represents the probability of the system having, at time t, an up rating given that it entered the state i with an initial backward time v and exited from i after time u (certain waiting forward time).(2.1′) ϕbfijBFR(v,u;v′,t,u′) represents the probability of being in the state j after a time t starting in the state i with an initial backward time v, an initial forward time u, with v′≤TN(t)≤t<TN(t)+1≤u′given that the two down states are considered absorbing.(2.1′′) ϕbFijBFR(v,u;v′,t,u′) represents the probability of being in the state j after a time t starting in the state i∈U with an initial backward time v, a starting certain forward time u, with v′≤TN(t)≤t<TN(t)+1≤u′ given that the two down states are considered absorbing.(2.2′) RbfiBF(v,u;v′,t,u′)=∑j∈UϕbfijBFR(v,u;v′,t,u′) represents the probability that the system was always up in the time interval (0,t] given that it entered the state i∈U with an initial backward time v and it exits from i at time u (forward initial time) considering the two down states as absorbing.(2.2′′) RbFiBF(v,u;v′,t,u′)=∑j∈UϕbFijBFR(v,u;v′,t,u′) represents the probability that the system was always up in the time interval (0,t] with an initial backward time v and it exited from i after time u (certain waiting forward time) considering the two down states as absorbing.(3.1′) ϕbfijBFM(v,u;v′,t,u′) represents the probabilities of being in the state j after a time t starting in the state i∈D with an initial backward time v, an initial forward time u, with v′≤TN(t)≤t<TN(t)+1≤u′given that all the up states are considered absorbing.(3.1′′) ϕbFijBFM(v,u;v′,t,u′) represents the probabilities of being in state j after a time t starting in the state i∈D with an initial backward time v, a starting certain forward time u, with v′≤TN(t)≤t<TN(t)+1≤u′ given that all the up states are considered absorbing.(3.2′) MbfiBF(v,u;v′,t,u′)=∑j∈UϕbfijBFM(v,u;v′,t,u′) represents the probability that the system at time t has an up rating given that it entered into the state i∈D with an initial backward time v and it exited from i at time u (forward initial time) given that all the up states are considered absorbing.(3.2′′) MbFiBF(v,u;v′,t,u′)=∑j∈UϕbFijBFM(v,u;v′,t,u′) represents the probability that the system at time t has an up rating given that it entered the state i∈D with an initial backward time v and exited from i after time u (certain waiting forward time) given that all the up states are considered absorbing.The maintainability functionM has a precise financial meaning. It assesses the probability of a firm leaving state D within time t through reorganization. In fact, if the firm is reorganized the rating agency will give a new rating which evaluates the new financial situation. In the No Rating state, the re-entrance of the firm in the bond market will imply a new rating evaluation.In Table1 the embedded Markov chain obtained considering all the transitions of the historical data base of S&P is given. This matrix was presented in [34].Table 1
Embedded Markov chain obtained by the S&P data.
StatesAAAAAABBBBBBCCCDNRAAA0.3285080.1011050.0365080.0218980.0105500.0059240.0014310.0005660.493510AA0.0517660.4649720.1838820.0244960.0085930.0042370.0014320.0004480.260174A0.0218470.1005150.4530240.1270870.0119710.0065840.0020740.0007910.276107BBB0.0150100.0256070.1320550.3784570.1230320.0184110.0052840.0014050.300736BB0.0069660.0161080.0304750.1935860.2772700.1836210.0179460.0027570.271272B0.0039830.0072450.0154500.0245190.1558470.3286310.1609330.0129070.290485CCC0.0019600.0053900.0111480.0149450.0182530.1269140.3417860.2235700.256033D0.0048920.0077460.0101920.0228290.0228290.0521810.0986550.0966160.684060NR0.0703020.0850190.1167810.0897680.0444200.0386080.0120000.0025020.540599


## 5.2. The Default as Absorbing Case

In many credit risk migration models, the NoRating state is ignored and the Default is considered as an absorbing state. Under these hypotheses, the embedded Markov chain of the semi-Markov process has only two classes of states. The first is a transient class and the second is an absorbing class. The absorbing class is constituted by only one state and all the elements of the main diagonal of the matrix are always greater than zero. This kind of matrices (and the corresponding processes) is called monounireducible; see [35].The set of states becomes(5.3)I2={AAA,AA,A,BBB,BB,B,CCC,D},
and the partition in up and down states is(5.4)U={AAA,AA,A,BBB,BB,B,CCC},D2={D}.
The embedded Markov chain is reported in Table 2.Table 2
S&P embedded Markov chain with default as an absorbing state.
StatesAAAAAABBBBBBCCCDAAA0.6485970.1996190.0720810.0432350.0208290.0116960.0028250.001117AA0.0699710.6284880.2485480.033110.0116150.0057270.0019360.000605A0.030180.1388540.6258160.175560.0165370.0090950.0028640.001093BBB0.0214660.036620.1888490.5412230.1759460.026330.0075570.00201BB0.0095590.0221040.0418190.2656490.3804850.2519750.0246270.003784B0.0056130.0102120.0217760.0345570.2196520.4631770.2268210.018192CCC0.0026350.0072450.0149840.0200890.0245350.1705910.4594110.30051D00000001In this case, it results that reliability and availability correspond. Indeed, the only down state is absorbing and if the system at timet is available it means that it never went into the default state and so it remained for all the observation time in a up state, which is the reliability definition. Furthermore, maintainability does not make sense because it is not possible to exit from the default state.The following results can be obtained(1.1′) ϕbfijBF(v,u;v′,t,u′) represents the probabilities of being in the state j after a time t starting in the state i with an initial backward time v, an initial forward time u, with v′≤TN(t)≤t<TN(t)+1≤u′.(1.1′′) ϕbFijBF(v,u;v′,t,u′) represents the probabilities of being in the state j after a time t starting in the state i with an initial backward time v, starting certain forward time u, with v′≤TN(t)≤t<TN(t)+1≤u′.Both the results take into account the different probabilities of changing state during the permanence of the system in the same state considering all the possible entrance and the exit times in the starting and arriving states. In the first case, the exit will be just at timeu, in the second case after a time u.(1.2′) AbfiBF(v,u;v′,t,u′)=RbfiBF(v,u;v′,t,u′)=∑j∈UϕbfijBF(v,u;v′,t,u′)represents the probability that the system at time t has an up rating given that it entered the state i∈U with an initial backward time v and it exits from i at time u (forward initial time).(1.2′′) AbFiBF(v,u;v′,t,u′)=RbFiBF(v,u;v′,t,u′)=∑j∈UϕbFijBF(v,u;v′,t,u′)represents the probability that the system at time t has an up rating given that it entered the state i∈U with an initial backward time v and it exits from i after time u (certain waiting forward time).Remark 5.1.
We wish to mention that the Markov matrices that are given yearly in the Standard & Poor’s publications always have greater elements on the main diagonal compared to the matrices that are presented in this paper. The reason for this is that, in a semi-Markov environment the transitions occur only when there is a real check on the state. In a credit risk environment this means that a transition is computed if and only if the rating agency assigns a new rating, given that the firm already has a rating. In the S&P transition Markov chain, if in a year there was no rating evaluation of a firm, it is supposed that the firm is still in the same state. Then the rating agency, in the construction of the transition matrix, considers a “virtual” transition (transition from that state into the same state). This implies that the number of virtual transitions is very high and the Markov chain, almost everywhere, becomes diagonally dominant. In the embedded Markov chain of the SMP, the virtual transitions are possible, but they happen when the rating agency gives a new rating which is equal to the previous one.
We think another reason for the superior performance of the semi-Markov environment as compared to that of the Markov one is the fact that the former only considers the transitions of the rating process which actually occurred.


## 6. Conclusions

This paper introduces, for the first time to the authors’ knowledge, initial and final backward and forward processes in a continuous time homogeneous semi-Markov environment at the same time.By means of this new approach a generalization of the transition probabilities of an HSMP is given and we show how it is possible to consider the time spent by the system in the starting state and in the final state. The waiting time inside the starting state is managed by means ofinitial backward and forward times. The time spent in the last state of the considered horizon is studied by means of the final backward and forward times.The obtained results are used to derive generalized reliability measures and we show how it is possible to compute them.An application to credit risk problems, which is considered as a particular aspect of the more general context of the reliability of a system, is illustrated. In this way, the paper may also serve the purpose of inviting stochastic modelling engineers into a new field. However, the model could also be useful for solving other reliability problems.In the last part of the paper, the Markov chains embedded in the homogeneous semi-Markov processes obtained by the historical Standard and Poor’s database are presented. The difference between the obtained transition matrices and the ones that are provided by Standard and Poor’s agency is outlined. The authors also explain why the matrices obtained are more reliable compared to those of Standard and Poor’s.Future work includes the construction of(i)
the discrete time version of this model,(ii)
the related algorithm and computer program,(iii)
the nonhomogeneous model,(iv)
the related algorithm and computer program.Furthermore, we hope to apply the models to the mechanical reliability context in the near future.



---

*Source: 625712-2009-11-19.xml*