# Enhancing Online Learning Through Multi-Agent Debates for CS University Students
## 3 Materials and Methods
### Design of Evaluation System
#### 3.1.1 LLM RAG-Based Video Understanding Function  
Videos inherently exhibit multimodal properties, encompassing both visual and audio features. The visual features can be categorized into image captioning, object detection, and optical character recognition (OCR) tasks. We extracted keyframes from the video using CLIP features and used the BLIP model to generate image captions, while employing GRiT and RAM++ for object detection. Additionally, we utilized Whisper to transcribe the audio into text. These multimodal features provide the foundational data for subsequent QA tasks, as shown in Figure 3.  
The method proposed in this paper is compared with the current state-of-the-art approaches, including VideoChat2 ([[https://github.com/OpenGVLab/Ask-Anything](https://github.com/OpenGVLab/Ask-Anything)]([https://github.com/OpenGVLab/Ask-Anything](https://github.com/OpenGVLab/Ask-Anything)), accessed on 5 September 2024) and the open-source VLog project. Unlike VideoChat2, which primarily utilizes visual features, our approach incorporates additional modalities, such as audio and OCR features. As a result, our method demonstrates a superior ability to capture finer details in video content. Figure 4a shows that current state-of-the-art methods for video understanding, such as VideoChat2 (Videoshat2, can only answer questions related to the visual content of the video but struggle with finer details. In contrast, the multi-modal video analysis method based on the VLog project ([[https://github.com/showlab/VLog](https://github.com/showlab/VLog)]([https://github.com/showlab/VLog](https://github.com/showlab/VLog)), accessed on 27 August 2024) can answer detailed questions about the video, though not always with full accuracy. The results presented in Figure 4b build on the VLog project, where ChatGPT is replaced with the free online \"WizardLM-2\" LLM API by Together.ai ([[https://api.together.ai/](https://api.together.ai/)]([https://api.together.ai/](https://api.together.ai/)), accessed on 27 August 2024), along with English-to-Chinese translation using the pre-trained \"Helsinki-NLP/opus-mt-en-zh\" model. WizardLM-2's capabilities are very close to those of cutting-edge proprietary models such as GPT-4-1106-preview and significantly ahead of all other open-source models. However, as shown in Figure 4c, our system, enhanced with OCR recognition, significantly improves accuracy in capturing detailed information within the video content. Compared to existing systems, our system accurately extracts video information, then provides it to the RAG module for answer generation.  
Figure 3: Experimental procedure. From left to right, the diagram illustrates the feature processing, RAG workflow, and debate workflow of the AI-empowered video-based online learning system. The multi-agent debate module operates independently from the QA system while also complementing it, both of which serve the purpose of supporting students in self-learning.  
Figure 4: Evaluation of results from different video understanding methods. (**a**) The result of VideoChat2. (**b**) The result of VLog. (**c**) The result of our video-MAD system.