# Enhancing Online Learning Through Multi-Agent Debates for CS University Students
## 4 Results  
### Data Analysis  
According to the pre-test and post-test results, as shown in Table 4, the mean score on the pre-test questions was below 70%. We limited our analysis to the single and multiple-choice questions, as grading the open-ended questions proved more challenging. Instead, we employed a word cloud analysis to examine the open-ended questions. After reviewing the lecture videos, students under all conditions demonstrated learning effectiveness. The learning method using MAD achieves the greatest improvement in outcomes. We employed a paired t-test and [PERSON]'s \(d\)[75] to analyze test correctness scores before and after \"video-only\" [\(t(29)=-3.946\), \(p<0.001\), 95% CI [\(-22.724\), \(-7.210\)], [PERSON]'s \(d=-0.720\)], \"video-based Q&A\" [\(t(30)=-2.765\), \(p=0.005\), 95% CI [\(-16.320\), \(-2.454\)], [PERSON]'s \(d=-0.497\)], and \"video-based Q&A + MAD\" [\(t(28)=-3.260\), \(p=0.001\), 95% CI [\(-31.108\), \(-7.099\)], [PERSON]'s \(d=-0.605\)] interventions. These findings indicate that the \"video-based Q&A + MAD\" approach is more effective than \"video-only\" and \"video-based Q&A\" methods for online learning. These results support H1. Question samples and SPSS 29.0.1.0 results can be found at [[https://osf.io/4](https://osf.io/4) vt9h/]([https://osf.io/4](https://osf.io/4) vt9h/), accessed on 26 November 2024. Those in condition 2 exhibited the smallest improvement, whereas those in condition 3 showed the greatest improvement. It is generally believed that the use of large language models demonstrates an advantage over traditional video-watching methods. Table 4 indicates that the post-test performance of students in condition 3 shows a relative improvement of 29.78%, compared to 14.70% for students in condition 2, and 23.39% for students in condition 1. The quiz questions consisted of a mix of true/false questions and single and multiple-choice questions. A possible explanation is that the video-based Q&A approach might encourage students to take shortcuts, as they may not watch the video attentively and, instead, rely on asking questions directly to learn. This behavior could result in overlooking some video content, thereby leading to lower scores. In contrast, the \"video-based Q&A + MAD\" approach likely enhances students' interest in learning through debates. The presentation of affirmative viewpoints may prompt students to question specific knowledge points, extending their study time and fostering a more comprehensive understanding of the video content. The pre-test and post-test questions are presented in Appendix A. As shown in Figure 6, we used violin plots to present the specific distributions of the scores of the three groups of samples in pre-test and post-test. It can be observed that the score distributions of the video-based Q&A + MAD method before and after the pre-test and post-test show significant differences, and these differences are much more pronounced compared to those of the other two groups. Therefore, we conducted a further analysis from the perspective of students' video-viewing duration. Specifically, we examined the variations in the duration of students' engagement with online lecture videos under different scenarios, with and without the assistance of large language models. By comparing these durations in Figure 7, we can discern students' interest in utilizing large language models and determine whether the use of such models can enhance students' interest in learning videos. As shown in Figure 7, there is significant variation in the time students spent watching the video in the \"traditional learning\" condition. Some students spent only a few minutes, while others spent over