## 3 System Architecture
### Reinforcement Learning Optimization Module  
The Reinforcement Learning Optimization Module is responsible for dynamically adjusting the fuzzy membership function thresholds to ensure that the classification of students as Fast, Moderate, or Slow learners remains accurate over time. Unlike a static fuzzy logic system where thresholds (T\({}_{low}\), T\({}_{mid}\), and T\({}_{max}\)) are predefined and fixed, this module enables the system to evolve based on student behavior, ensuring a more personalized learning experience. By applying Reinforcement Learning (RL), the system continuously evaluates the effectiveness of UI adaptations and modifies the fuzzy membership thresholds accordingly. This ensures that fuzzy weights remain relevant to different learning speeds rather than using manually defined parameters that may not generalize well across learners. The RL agent's goal is to optimize the classification boundaries of the Fuzzy Inference Module so that adaptive UI modifications occur at the right time. The agent operates in an episodic environment, where each episode corresponds to a student's session interacting with the educational software. The state of the environment at any time step t is represented as follows:  
\[S_{t}=\left(x_{1}^{t},T_{low}^{t},T_{mid}^{t},\ T_{max}^{t}\right)\]  
where:  
* \(x_{1}^{t}\) is the time spent on an exercise at time t.
* \(T_{low}^{t},T_{mid}^{t}\) and \(T_{max}^{t}\) are the current fuzzy membership thresholds at time t. The action space consists of modifications to these thresholds: \[A_{t}=\left(\Delta T_{low},\Delta T_{mid},\Delta T_{max}\right)\] Each action adjusts the fuzzy logic membership functions by increasing or decreasing the values of the thresholds, allowing the system to refine when UI adaptations occur. At any time step, the agent of RL chooses an action, A\({}_{t}\), to adjust the fuzzy membership thresholds. The goal is to optimize UI interventions, ensuring that students receive help when needed but not too soon or too late. The action space is discrete and defined as follows:  
\[A_{t}\in\left\{(\pm 0.5,\ 0,\ 0),\ (0,\ \pm 0.5,\ 0),\ (0,\ 0,\ \pm 0.5),\ (\pm 1,\ 0,\ 0),\ (0,\ \pm 1,\ 0),\ (0,\ 0,\ \pm 1)\right\}\]  
where each action modifies one of the fuzzy membership thresholds by a small (\(\pm 0.5\)) or large (\(\pm 1.0\)) adjustment. These specific increment values were selected based on the scale of the fuzzy input domain (0-20 min), ensuring that adjustments are large enough to be meaningful but small enough to avoid instability. The \(\pm 0.5\) step enables fine-tuning when thresholds are close to optimal, while \(\pm\)1.0 allows for quicker adaptation when greater deviations are needed. This design was informed by pilot testing and ensures a practical balance between precision and convergence speed. The RL agent employs an \(\varepsilon\)-greedy policy to achieve a balance between exploration and exploitation:  
* The parameter \(\varepsilon\) represents the probability that the agent chooses a random action to encourage exploration. * The parameter \(1-\varepsilon\) is the probability that it selects the action that maximizes the expected future rewards, promoting exploitation. \[A_{t}=\underset{A^{\prime}}{\text{argmax}}Q\big{(}S_{t},\ A^{\prime}\big{)}\]  
where Q(S\({}_{t}\)A\({}^{\prime}\)) denotes the expected reward associated with taking action A\({}^{\prime}\) in state S\({}_{t}\). To guide the RL agent, a reward function R\({}_{t}\) is designed to ensure that fuzzy weight adjustments lead to improved learning experiences. The reward function consists of four main components:  
\[R_{t}=\lambda_{1}\Big{(}T_{before}-T_{after}\Big{)}+\lambda_{2}U-\lambda_{3} \underline{\sum}|\Delta T|-\lambda_{4}H\]  
where:  
* Tbefore and Tafter: Measure learning efficiency improvement (time spent before and after fuzzy threshold adjustments). * U: Represents user engagement and performance-based feedback, which can be implicit (e.g., improvement in performance) or explicit (e.g., a satisfaction rating).