## 2 Materials and Methods
### Data Analysis
#### 2.2.2 Model Development  
During the model development phase, the rules of the Markov chain and its related methods were used [36; 37; 38; 39; 40; 41; 42; 43]. The developed behavior model \(\lambda(A,B,\pi)\) with two states and three parameters A, B, and \(\pi\) is as follows:  
* State transition probability distribution: \(A=\left\{a_{ij}\right\}=\left[ \begin{array}{cc}a_{11}&a_{12}\\ a_{21}&a_{22}\end{array} \right]\)
* Observation symbol probability distribution: \(B=\left\{b_{j}(v_{k})\right\}=\left[ \begin{array}{cc}b_{1}(v_{1})&b_{1}(v_{ 2})&b_{1}(v_{3})\\ b_{2}(v_{1})&b_{2}(v_{2})&b_{2}(v_{2})\end{array} \right]\)
* Initial state distribution: \(\pi=\left\{\pi_{i}\right\}=\left[ \begin{array}{cc}1&1\\ 1&1\end{array} \right]\)  
Adopting the classic model notation for a hidden Markov model (HMM) from [PERSON] and [PERSON] [36], the HMM is characterized by five elements, namely \(N\), \(M\), \(A\), \(B\), and \(\pi\). These are described as follows. \(N\) is the number of states in the model. The individual states and the state at time \(t\) are denoted as \(S=\left\{S_{1},S_{2},\cdots,S_{N}\right\}\) and \(q_{t}\), respectively. \(M\) is the number of observation symbols. The individual observation symbols are denoted as \(V=\left\{v_{1},v_{2},\cdots,v_{M}\right\}\). \(A\) is the state transition probability distribution, \(A=\left\{a_{ij}\right\}\), where \(a_{ij}=P\big{[}q_{i+1}=S_{j}\big{|}q_{t}=S_{i}\big{]}\), \(1\leq i,j\leq N\). The term \(a_{ij}\) represents the transition probability from state \(S_{i}\) to state \(S_{j}\), and is subject to the constrains of \(\sum_{j}^{N}a_{ij}=1\) and \(a_{ij}\geq 0\). \(B\) is the observation symbol probability distribution in state \(j\): \(B=\left\{b_{j}(k)\right\}\), where \(b_{j}(k)=P\big{[}v_{k}\) at \(t|q_{t}=S_{j}\big{]}\), \(1\leq j\leq N\), and \(1\leq k\leq M\). The term \(b_{j}(k)\) represents the probability of state \(j\) at time \(t\) with observation symbol \(v_{k}\), and is subject to the constraints of \(\sum_{k}^{M}b_{j}(k)=1\) and \(b_{j}(k)\geq 0\). Finally, \(\pi\) is the initial state probability distribution \(\pi=\left\{\pi_{i}\right\}\), where \(\pi_{i}=P[q_{1}=S_{i}]\), \(1\leq i\leq N\). The term \(\pi_{i}\) represents the probability of \(S_{i}\) being the initial state at time \(t=1\). Given appropriate values of \(N\), \(M\), \(A\), \(B\), and \(\pi\), the HMM can be used to generate an observation sequence \(O=O_{1}O_{2}\cdots O_{T}\) in which each observation \(t\) is a symbol from \(V\), and \(T\) is the number of \begin{table}
\begin{tabular}{c c c} \hline \hline
**Performance Level** & **Symbol** & **Rule** \\ \hline Low & Poor & \(PL_{\text{ }}3\) = 1/3 \\ Moderate & Average & 1/3 \(<=PL_{\text{ }}<2/3\) \\ High & Excellent & \(PL_{\text{ }}>=2/3\) \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Rules for symbolic performance levels. observations in the sequence. Given a finite observation sequence of hidden states, all the possible transition probabilities and symbol output probabilities can be multiplied at each transition to calculate the overall likelihood of all the output symbols produced in the transition path up to that point. Summing all such transition paths, one can then compute the likelihood of the HMM generating the sequence. The compact notion \(\lambda=(A,B,\pi)\) can be used to represent the complete parameter set of the model. With HMMs, there are three basic problems that must be addressed, namely (i) evaluation, (ii) decoding, and (iii) learning. The first problem is evaluation, i.e., the probability \(P(|\lambda)\) of the model \(\lambda=(A,B,\pi)\) producing a given sequence \(O=O_{1}O_{2}\cdots O_{T}\). The evaluation problem can be solved using the forward-backward dynamic programming algorithm [37; 38]. The forward variable \(\alpha_{t}(j)\) is defined as the probability of the partial observation sequence \(O_{1}O_{2}\cdots O_{t}\) and state \(S_{i}\) at time \(t\), given the model \(\lambda\).