# Process Mining IPTV Customer Eye Gaze Movement Using Discrete-Time Markov Chains +
## 3 Methodology
### Fitting into a DTMC Model
#### 3.4.1 Markov Definitions
In this study, the trajectory refers to the path of people's gaze movements on the TV screen. Based on the Markov Property, if the transition matrix \(A=(a_{ij})\) is known, we can find the probability of any trajectory \(\{s_{1}\), \(s_{2}\), \(s_{3}\),..., \(s_{n-1}\), \(s_{n}\}\) by multiplying together the starting distribution and all subsequent single-step probabilities. The calculation is shown: \[ \begin{array}{ll}P(s_{0},s_{1},s_{2},\ \ldots,s_{n})&=P(X_{0}=s_{0},\ X_{1}=s_{1},\ X_{2}=s_{2}, \ \ldots,X_{n}=s_{n})\\ &=P(X_{0}=s_{0})\times a_{01}\times a_{12}\times a_{n-2,n-1}\times a_{n-1,n} \end{array} \] (10)  
Therefore, in this case study, we provide a method for determining the probability of the \"most likely trajectory\" (the trajectory is formed of the most probable condition at each stage). According to Equation (8), in the \(n^{\text{th}}\) step, let \(\pi_{n}\) denote the probability distribution over the states, \(s_{n}\)\({}^{max}\) denotes the state of the highest probability, then we have:  
\[P(X_{n}=s_{n}{}^{max})=\text{argmax}\{\pi_{n}\} \tag{11}\]  
Let \(p^{max}\) denote the probability of the'most likely trajectory' in \(n\) steps, then can calculate it as follows:  
\[p^{max}=\prod\
olimits_{i=0}^{n}\text{argmax}\{\pi_{i}\} \tag{12}\]  
Meanwhile, the state \(s_{n}\)\({}^{max}\) in each step \(n\) comprises a set of states \(S^{max}\), which represents the sequence of states from \(X_{0}\). Thus, \(S^{max}\) is the most likely trajectory shown below:  
\[S^{max}=\{s_{0}{}^{max},s_{1}{}^{max},s_{2}{}^{max},\dots,s_{n}{}^{max}\} \tag{13}\]  
Similarly, ref. [29] introduces the same concept about maximum likelihood trajectories but calculates them in a different way. * First Passage Time  
First passage time [30] is also named hitting time, referring to the time to reach a certain state (if the states communicate). Given the initial state \(X_{0}=S_{i}\), the total number \(T_{ij}\) of steps taken by the Markov Chain from state \(S_{i}\) to reach state \(S_{j}\) for the first time is the first passage time from \(S_{i}\) to \(S_{j}\). The commonly used quantity related to the first passage time is the mean first passage time. Let \(T_{ij}\) be the first passage time from state \(S_{i}\) to \(S_{j}\), so we define \(m_{ij}\) as the mean first passage time correspondingly, which is represented as follows [31]:  
\[m_{ij}=\mathbb{E}\big{[}T_{ij}\big{|}X_{0}=S_{i}\big{]} \tag{14}\]  
* Transition Matrix Segmentation  
If state \(j\) is an absorbing state such that it has \(m\) transient states and \(n\) absorbing states, then the transition matrix \(A\) can be separated into sections \(Q\), \(R\), \(0\) and \(I_{n}\) written as [58]:  
\[A= \begin{bmatrix}Q&R\\ 0&I_{n}\end{bmatrix}  \tag{15}\]  
where:  
* \(Q\) is an \(m\times m\) matrix;
* \(R\) is an \(m\times n\) matrix;
* \(0\) is an \(n\times m\) matrix of zeros;
* \(I_{n}\) is an \(n\times n\) identity matrix [59; 60; 61; 62]. This segmentation of the transition matrix of an absorbing chain can allow for further calculations, including the Fundamental Matrix \(N=\{n_{ij}\}\), such that the element \(n_{ij}\) for N provides the expected number of times the process is in transient state \(S_{j}\), given that the chain began in transient state \(S_{i}\). The Fundamental Matrix \(N\) is the inverse of \((I-Q)\) and can therefore be calculated by the equation as follows [59; 60; 61; 62]:  
\[N=(I-Q)^{-1} \tag{16}\]  
* Expected Time to Absorption  
Furthermore, if we let \(t_{i}\) represent the expected number of steps before the Markov chain is absorbed, given that the chain began in state \(i\), then the column vector \(t\) (with an \(i\)th entry of \(t_{i}\)) can be calculated as:  
\[t=Nc \tag{17}\]where \(c\) is a column vector of 1 s, e.g., if \(N\) is a \(3\times 3\) matrix, then \(c\) will be a \(3\times 1\) matrix:  
\[c= \begin{bmatrix}1\\ 1\\ 1\end{bmatrix}