# TSA-GRU: A Novel Hybrid Deep Learning Module for Learner Behavior Analytics in MOOCs
## 4 Experimental Results and Comparaison
### Implementation of the TSA-GRU for Learner Behavior Analytics  
The TSA-GRU model was developed using several configurations and tested for its ability to predict student response times from sequential interaction data. The model is  
\begin{table}
\begin{tabular}{l l} \hline \hline
**Hyperparameter** & **Value** \\ \hline Number of Attention Heads & 4 \\ Hidden Dimension (Embedding) & 128 \\ Learning Rate & 0.001 \\ Batch Size & 64 \\ Number of Epochs & 5 \\ Optimizer & Adam \\ Dropout Rate & Not used \\ Loss Function & Binary Cross-Entropy \\ Sequence Length & 10 \\ GRU Hidden Units & 128 \\ Weight Initialization & Xavier Normal \\ Sparsity Threshold & 50\% of max attention score \\ Total Parameters & 166,017 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Hyperparameter settings for TSA-GRU model.  
comprised of a combination of TSA and GRU, capturing broad behavioral patterns as well as the temporal characteristics present in the context of learner interactions.  
The simulated sequential interaction data serving as input are structured as a series of numerically encoded features that represent student behavior over time. By combining a TSA-Attention module for global feature refinement with a GRU layer for modeling sequential dependencies, TSA-GRU effectively predicts the expected response time for the next interaction. Hyperparameter tuning across various configurations was performed to optimize predictive accuracy while ensuring robust generalization across multiple student datasets.  
The TSA-GRU model includes the following key components (Figure 2):  
* Linear Embedding Layer: The input sequences (10 steps with multiple numerical features per step) are projected into a 128-dimensional space using a linear embedding layer. With 640 learnable parameters, this layer enables the model to capture complex nonlinear relationships among features.
* TSA-Attention Module: The embedded features are refined through linear transformations of queries, keys, and values. A sparse selection mechanism, based on binary masks and thresholding, filters out irrelevant interactions and enhances the most pertinent temporal patterns.
* Query transformation: 16,512 parameters
* Key transformation: 16,512 parameters
* Value transformation: 16,512 parameters
* Output transformation: 16,512 parameters  
The total number of parameters in the TSA-Attention module is therefore 66,048, enabling effective feature selection and the extraction of global behavioral representations.
* GRU Module for Sequential Dependency Modeling: The refined features from the TSA module are passed to a GRU layer. Using update and reset gates, the GRU captures temporal dependencies in student behavior, retaining important historical information while discarding less useful patterns. This module contains 99,072 tunable parameters.
* Fusion and Prediction Layer: Outputs from both the TSA-Attention and GRU modules undergo global average pooling to reduce sequence length. The pooled outputs (each 128-dimensional) are concatenated into a 256-dimensional vector. This vector is passed through a fully connected layer (with 257 parameters) to predict the normalized response time for the next interaction.