## 3 Methodology
### Data Collection Methods
#### 3.4.2 Data Sources and Collection Tools  
The pre-tests and post-tests were designed as 50-question assessments, each lasting approximately 60 min, and comprised a mix of 70% multiple-choice questions (e.g., selecting correct programming syntax or solving algebraic equations) and 30% short-answer questions (e.g., explaining a physics concept or composing a brief paragraph). The pre-tests assessed foundational knowledge specific to each theme--Computer Science (e.g., basic programming constructs), Mathematics (e.g., algebraic operations), Physics (e.g., mechanics principles), and English Language (e.g., grammar and vocabulary)--to establish a baseline. The post-tests, while maintaining the same format and difficulty level within each theme, shifted focus to applied understanding (e.g., debugging code, solving multi-step math problems, applying physics laws, or writing structured essays). Although the pre-tests and post-tests were theme-specific and not identical across disciplines (e.g., Math vs. English), they were standardized within each subject to ensure consistency in measuring progress, with question banks validated by subject-matter experts for reliability and relevance. To ensure data integrity and comprehensiveness, multiple data sources were employed, including automated system logs, pre- and post-study assessments, surveys, and instructor observations. 1. System-Generated Learning Logs  
The AI-driven system recorded every student interaction, including response accuracy, time spent on tasks, feedback utilization, and engagement frequency. Logs capture data at five-minute intervals, allowing for a fine-grained analysis of student learning behavior. 2. Pre- and Post-Study Assessments  
To measure the impact of AI-driven feedback, students were administered two standardized assessments: one before implementation and another at the end of the study. These assessments evaluated conceptual understanding, problem-solving skills, and knowledge retention as shown in Table 5.  
To align with the study's focus on STEM and Language Education, the pre- and post-test instruments were specifically designed for each content area. For Computer Science, pre-tests assessed baseline knowledge of programming fundamentals (e.g., variables, conditionals) with questions like \"Identify the correct syntax for a loop\", while post-tests evaluated application (e.g., \"Write a short program to solve a given problem\"). In Mathematics, pre-tests covered algebraic foundations (e.g., solving linear equations), with post-tests testing multi-step problem-solving (e.g., word problems involving quadratics). For Physics, pre-tests focused on conceptual mastery of mechanics (e.g., \"State Newton's First Law\"), and post-tests required application (e.g., \"Calculate the velocity of an object given force and mass\"). In the English Language, pre-tests measured grammar and vocabulary (e.g., \"Correct the sentence structure\"), while post-tests assessed composition skills (e.g., \"Write a paragraph using specified vocabulary\"). Each 50-question test combined multiple-choice (70%) and short-answer (30%) formats, was developed with input from subject experts and underwent pilot testing to ensure validity and reliability across mixed-ability learners. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Assessment Type** &
\begin{tabular}{c} **Number of** \\ **Questions** \\ \end{tabular} & **Topics Covered** & **Score Range** \\ \hline Pre-Study Test & 50 & Baseline Conceptual Knowledge & 0–100\% \\ Post-Study Test & 50 & Applied Understanding & 0–100\% \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Pre- and post-study assessment information. To capture student perceptions of AI-driven feedback, structured surveys were conducted in weeks 10 and 20.