# Multi-Scale Audio Spectrogram Transformer for Classroom Teaching Interaction Recognition
## 5 Conclusions  
In this paper, we propose a model MAST for audio scene recognition and transform the classroom interaction detection problem into a classification problem of classroom speech interactivity. We design multi-channel spectrogram features to effectively characterize the multi-dimensional features of audio and to explore the detailed features of audio; by introducing a multi-scale Transformer network, we strengthen the model's ability to compute the global attention of the audio spectrogram at different scales; finally, the Time-Frequency Domain Local Enrichment Module captures the temporal dynamic features and \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Method** & **Year** & **Params.** & **Silent** & **Teaching** & **Interaction** & **Average** \\ \hline PANN [52] & 2020 & 81M & 0.932 & 0.813 & 0.834 & 0.851 \\ AST [25] & 2021 & 87M & 0.924 & 0.819 & 0.825 & 0.847 \\ MAST & 2022 & 36M & 0.932 & 0.851 & 0.861 & 0.875 \\ MAST (pretrain) & 2022 & 36M & 0.951 & 0.904 & 0.918 & 0.921 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Classification accuracy for each category on the CSID validation set. Figure 11: Confusion matrix based on the CSID dataset. (**a**) MAST. (**b**) MAST (pretrain). concurrent features of the sound, enhancing the local modeling at the time and frequency domain levels. The MAST algorithm proposed in this paper can accurately identify classroom interaction scenes with higher accuracy and robustness for generic audio classification datasets. Experiments under the evaluation protocol of common scene classification datasets AudioSet and ESC-50, show that MAST achieves the accuracy of advanced methods compared with CNN and traditional Transformer. Ablation experimental analysis shows that multi-channel features and Time-Frequency Domain Local Enrichment Module are important modules. At the same time, the use of the Multi-scale Transformer reduces the number of model parameters and achieves the goal of further lightweight and efficiency of the model while ensuring the accuracy of classroom interaction recognition in the self-built dataset, which is more in line with the requirements of practical production applications. While the results are encouraging, the application of this work is also limited. Firstly, classroom audio is susceptible to interference from other classroom or outdoor factors, making it difficult to obtain data for the ideal collection of classroom interactions. Secondly, our method only considers teacher-student verbal interactions during classroom interactions. However, in practice, classroom interactions may include a small number of teacher-student behavioral interactions. The traditional lecture-based classroom differs from the interactive practice classroom in that the interaction between teachers and students in the classroom is verbal knowledge exchange, and the traditional lecture-based classroom can be judged by verbal interaction. This research has the following potential applications and further developments in the classroom scenario. By deploying audio capture devices in the classroom connected to a backend server and porting interaction recognition algorithms, classroom interactions can be recorded and presented automatically. The education system completes an automatic analysis and evaluation based on the frequency and proportion of interactions within the classroom at each period. If the proportion and frequency of weak interactions exceed certain thresholds, the teacher is advised to improve the teaching strategy of the classroom. In future work, we will consider combining classroom behavior recognition algorithms with learning emotion recognition algorithms. Adding teacher-student interaction behavior and emotional states in learning to classroom assessments enables smart classroom algorithms with more integrated assessment results and richer use of classroom scenarios. **Author Contributions:** Supervision, J.F. ; conceptualization, F.L. and J.F. ; formal analysis, F.L. and J.F. ; methodology, F.L. ; software, F.L.