# A Novel Method for Cross-Modal Collaborative Analysis and Evaluation in the Intelligence Era
## 2 Literature Review  
Research on cross-modal learning analysis could be dated back a decade, and the academic community continues to focus on its development. In 2012, scholars raised the idea of applying cross-modal data such as text, video, and audio in the field of cross-modal interaction research, a new direction for the research of cross-modal learning interaction, which put cross-modal learning as the core status of educational evaluation. In 2012, the International Conference on Cross-Modal Interaction (ICMI) established the frontier status of cross-modal learning analysis by organizing relevant scholars to participate in the seminar on \"Cross-Modal Learning Analysis\" for the first time in the form of workshops [1], declaring that the aim of this research field is to combine cross-modal analysis technology with learning science research to help researchers better understand the mechanism of learning. Since 2014, with the gradual development of blended learning, learning analysis, and big data technologies, a large amount of online and offline learning data have become available for identifying patterns such as learning emotions by extracting and analyzing cross-modal feature data. In 2016, the International Conference on Learning Analysis and Knowledge (LAK) set up a data challenge workshop for cross-modal learning andanalysis, organized relevant scholars to participate in the practical work of cross-modal data analysis, and explored the development direction of learning analysis research supported by cross-modal data [2]. The application of artificial intelligence technology has been widely spread in cross-modal learning analysis since 2018. By applying artificial intelligence to education, [PERSON] [3] proposed to use cross-modal machine learning to create and replicate human cognition, analyze and design artificial artifacts based on artificial intelligence technology through cross-modal learning, and ultimately support, help, and expand human cognitive abilities. The American Educational Research Association (AERA) annual meeting in 2019 proposed to use cross-modal data such as text, charts, audio, and video to construct a research paradigm based on cross-modal narratives to seek evidence of fairness in educational research [4]. In terms of specific research content, [PERSON] et al. [5] analyzed the interpretability problem of cross-modal learning behavior and used the HDRBM (Hybrid Deep Restricted Boltzmann Machine), a deep neural network model, to obtain the implicit physiological and emotional characteristics of students. This research used the Bayesian network and the junction tree algorithm to analyze the interpretability of the results, which has a certain effect on improving the interpretability of learning behavior analysis. From a cross-modal measurement perspective, [PERSON] et al. [6] designed and developed a computational model framework for assessing learning status. This framework is capable of capturing, analyzing, and measuring complex human behavior as well as analyzing noisy, unstructured, and cross-modal data. The analysis process utilizes cross-modal data, including audio, video, and activity log files, to construct an Analytic Hierarchy Process for modeling the temporal dynamics of human behavior. [PERSON] et al. [7] used transponders to collect students' voice data to analyze the continuity and motivation of group collaboration. [PERSON] et al. [8] used an ensemble feature extraction method to develop a new cross-modal information extraction agent. In particular, the developed method exploited joint tri-modal features (text, audio, and video) to infer and aggregate semantic and emotional information on user-generated cross-modal data. In terms of analysis methods, [PERSON] [9] used a cross-modal sequence classifier to analyze the expressions of laughter in the process of multi-party dialogue in the natural environment.