# Process Mining IPTV Customer Eye Gaze Movement Using Discrete-Time Markov Chains +
## 3 Methodology
### Fitting into a DTMC Model  
#### 3.4.1 Markov Definitions  
Considering that the eye tracker records gaze movement at a particular frequency, which makes it a discrete-time process, discrete-time Markov chains (DTMC) are chosen to model the trajectory of participants' gaze movement on the screen in our study. Before going deep into the modelling approach, we would like to provide a brief description of several commonly discussed terms in a DTMC and relevant properties which are primarily used in our study. * DTMC The concept of a Markov Model uses a system with a set of'states' \(S\) = {\(S_{1}\), \(S_{2}\),..., \(S_{n}\)} where n is the number of states in the system, so for example, if \(S\) = {A, B, C, D, E, F... } \(S_{1}\) = A, \(S_{2}\) = B, etc. [54]. In a first-order DTMC, the probability of the current state is based solely on the previous state of the Markov chain. That is, where the time instants associated with state changes are \(t\) = 0, 1, 2, 3... and the actual state at a given time \(t\) (\(t\) \(\geq\) 1) is denoted as \(q_{t}\), the probability of arriving at a given state, given its previous state, can be calculated as follows: \[P\big{(}q_{t}=S_{j}\mid q  
This means a transition matrix containing the sets of \(a_{ij}\) can be formed with the probabilities where each element of position \((i,j)\) in the matrix stands for the transition probability \(a_{ij}\), meaning in each row that a given state \(q_{t-1}\) will go to the next state \(q_{t}\). And each row will sum to 1 [54].  
* Classification of States In a Markov Chain, two states, \(i\) and \(j\), are said to be communicating if they are accessible from each other. In other words, if the probabilities in the transition matrix \(a_{ij}>0\) and \(a_{ji}>0\), then these states are said to be communicating. If all states communicate with each other in the Markov chain, then it is said to be irreducible, and it has only one communicating class, while Markov chains with multiple communicating classes are said to be reducible.  
Moreover, a state is said to be recurrent if, upon leaving the state, the chain is certain to return to that state (with a probability of one); however, if the probability of returning to that state is less than one, the state is said to be transient [57]. This can be defined as:  
\[f_{ii}=P(X_{n}=i,\ for\ some\ n\ \geq 1|X_{0}=i) \tag{6}\]  
and state \(i\) will be recurrent if \(f_{ii}=1\). However, it will be transient if \(f_{ii}<1\). If a state exists such that when the state is entered, it is then impossible to leave, and the probability of remaining within that state is equal to one. This is known as an absorbing state, denoted in the transition matrix as \(a_{ii}=1\). * Distribution of States In each step, \(t=0\), \(1\), \(2\), \(3\)..., the distribution over all states in the whole system can be represented by a row vector \(x\) (\(\sum_{i}x_{i}=1\), \(x_{i}\geq 0\)), and the state is denoted by \(X_{t}\). When \(t=0\), we call \(x\) an initial state vector, denoted by \(\pi_{init}\): \[\pi_{init}=[P(X_{0}=S_{1}),P(X_{0}=S_{2}),\ \ldots,\ P(X_{0}=S_{n})]\] (7)  
In a DTMC, let \(A\) be the transition matrix, and \(\pi_{init}\) be the initial state vector, then the probability distribution of \(X_{n}\) after \(n\) steps is \(\pi_{n}\):  
\[\pi_{n}=\pi_{init}A^{n} \tag{8}\]  
In a Markov Model that is finite, irreducible and aperiodic, an equilibrium can be reached after a long-run period known as the 'Steady State'. This is a vector \(\pi\) for a transition matrix \(A\) such that:  
\[\pi=\pi A \tag{9}\]  
which can be solved analytically or through computation. Alternatively, to calculate the stationary distribution over time, \(A^{n}\) can be calculated as \(n\rightarrow\infty\). * Trajectory A trajectory of a Markov Chain is a particular set of values for \(X_{0}\), \(X_{1}\), \(X_{2}\),.... Generally, if we refer to the trajectory \(\{S_{1}\), \(S_{2}\), \(S_{3}\),... }, we mean that \(X_{0}=S_{1}\), \(X_{1}=S_{2}\), \(X_{2}=S_{3}\),...