## 3 Methodology
### The Proposed Multimodal Hidden Markov Model (MHMM)  
Building on the foundational principles of HMMs, this research proposes an MHMM specifically designed for the dynamic assessment of human proficiency.  
The conceptual architecture of the proposed MHMM is illustrated in Figure 1. The model's key innovation is its two-layer structure.  
The model is structured into two layers. The Hidden Layer contains the unobservable proficiency states (Novice, Intermediate, and Expert), which must be inferred. The Observation Layer consists of directly measurable data streams: Heart Rate Variability (HRV), Task Completion Time (TCT), and NASA Task Load Index (NASA-TLX). Arrows within the hidden layer represent Transition Probabilities (A) (Equation (2)), modeling how proficiency evolves. Dashed arrows from the hidden to the observation layer represent Emission Probabilities (B) (Equation (3)), linking proficiency states to the data they are likely to produce.  
Figure 2, this flowchart visually outlines the MHMM framework for real-time human proficiency assessment. It details the steps from initial multimodal data collection (HRV, TCT, NASA-TLX), and preprocessing (normalization, handling missing values), through the parameterization of the MHMM (initial state, transition, and emission probabilities). It highlights the crucial decision point of incorporating contextual factors (task complexity, fatigue) to either utilize an expert-driven model or a trained model, ultimately leading to the inference of latent proficiency states using the Viterbi Algorithm and the output of valuable transition probabilities. This systematic approach underpins the dynamic and  
HRV, TCT, and NASA-TLX values at time \(t\), given the system is in proficiency state \(s_{j}\), is factored as shown in Equation (4):  
\[P\big{(}HRV_{t},TCT_{t},TXL_{t}\big{|}q_{t}=s_{j}\big{)}=P\big{(}HRV_{t}\big{|}q _{t}=s_{j}\big{)}.P\big{(}TCT_{t}\big{|}q_{t}=s_{j}\big{)}.P\big{(}TLX_{t}\big{|} q_{t}=s_{j}\big{)} \tag{4}\]  
This assumption means that, for any given level of proficiency, the HRV value is considered independent from the TCT and NASA-TLX score at that moment. The main reason for using this assumption is to make the model simpler and easier to work with. It reduces the number of parameters the model needs, which makes training faster and running the model more efficient.  
Of course, in real life, elements such as HRV, TCT, and perceived effort often change together, even when someone's skill level stays the same. For example, frustration might raise HRV, decelerate task time, and increase the sense of effort simultaneously. However, HMMs are generally quite good at handling small violations of this independence assumption. In practice, the different types of data (physiological, behavioral, and subjective) often show clear enough patterns for each skill level-Novice, Intermediate, Expert-that the model can still tell the difference between them, even if some overlap exists. In this approach, each type of data acts somewhat similar to an independent \"signal\" about the person's skill level. As long as these signals are mostly reliable and distinct for each level, the model can be expected to work reasonably well.