# A Survey on MLLMs in Education: Application and Future Directions
## 3 Applications of MLLMs in Education
### Adaptive Learning Platforms
#### 3.1.2 Technology Used: MLLMs in Adaptive Learning  
The incorporation of MLLMs into adaptive learning platforms represents a convergence of advanced AI technologies and educational methodologies. MLLMs utilize transformer-based architectures [11], which employ self-attention mechanisms to process sequential and non-sequential data across different modalities [19]. Extensions like Vision Transformer (ViT) [20] and Audio Spectrogram Transformer (AST) [20] enable processing of visual and auditory data, respectively.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Application Area** & **Technologies Used** & **Examples of Applications** & **Case Study** \\ \hline Adaptive Learning Platforms & MLLMs with transformer architectures (ViT, AST) & Cognii, Carnegie Learning’s & Integration of MLLMs in Duelingo \\ \hline Virtual Tutors and Chatbots & Transformer-based language models (GPT-3, GPT-4), VisualGPT, DeepSpeech, Tacotron 2, ADS & Squirrel AI Learning, Duolingo, Watson Tutor, Woebot [35] & Implementation of LOVA3 in Virtual Tutoring Systems \\ \hline Intelligent Content Creation & Transformer-based multimodal models (GPT-4, BLIP-2), DALL-E and DALL-E 2, NLG (GPT-3, T5) & Automated textbook generation, quiz and assessment generation, interactive simulations, multimedia content creation, language translation and localization & Automated Quiz Generation Using GPT-3 \\ \hline AI-powered LMS & GPT-4 and CLIP, natural language interfaces, Wav2 Vec 2.0, Tacotron 2 & Coursera, edX, Udemy, Knewton’s Alta & Integration of MLLMs in Coursera \\ \hline Insight and Predictive & GPT-4, CLIP, computer vision models (OpenFace), speech and audio processing models (Wav2 Vec 2.0) & Early warning systems, sentiment and emotion analysis, adaptive feedback generation, curriculum and institutional design insights, collaborative skills assessment & Early Warning System Using MLLMs \\ \hline Grading and Assessment Tools & NLP and NLU (GPT-4, BERT), computer vision and image recognition (CLIP, ViT), speech and audio processing (Wav2 Vec 2.0) & E-Rater by ETS, MOSS, CodeRunner, Duolingo English Test, tools for multimodal assignment evaluation & Reducing the Cost of Short-Answer Scoring with MLLMs \\ \hline \hline \end{tabular} The table summarizes key applications of MLLMs in education.  
\end{table}
Table 2: Applications of MLLMs in education.  
Techniques for integrating data from multiple modalities are crucial. Models like CLIP [14] and UNITER [24] align visual and textual representations in a shared embedding space, facilitating cross-modal understanding. Adaptive platforms use algorithms that analyze learner data to adjust content delivery. MLLMs enhance this by interpreting complex multimodal inputs, such as facial expressions, speech patterns, and handwriting [5].  
Moreover, MLLMs can generate contextually relevant feedback and explanations in natural language, improving learner comprehension and engagement [8]. By leveraging these technologies, adaptive learning platforms can offer more nuanced and effective personalization, catering to individual learner profiles.