# Artificial Intelligence in Intelligent Tutoring Robots: A Systematic Review and Design Guidelines
## 3 Artificial Intelligence Techniques for Designing Intelligent Tutor Robots
### Multi-Modal Perception of Students
#### 3.1.1 Multi-Modal Data Fusion  
The first phase of initiating an effective cognitive perception-planning-action loop is to acquire sufficient information concerning the environments and the students. As shown in Figure 2, the ITR may use multi-modal perception by utilizing multiple information channels, e.g., audio, visual, tactile, electromagnetic, and electroencephalograph sensors. These sensors may outperform and augment human-tutor sensory capabilities. Pre-processing steps are demanded before extracting information from the raw signals. First, it is required to remove the noise and interference, while preserving the useful information. Then the signals and data gleaned from multiple independent channels should be aligned in the spatial-temporal space to avoid incorrect correlations [24]. In the context of the ITR design, multi-modal data fusion is not specifically explored, but the topic is well-investigated and still a hot one in the context of data-mining and robotics. Due to space limitations, the rest of the section would not aim at a thorough review of the state-of-art data fusion. Instead, it reviews recent application driven surveys on data fusion and their great potential applications in the intelligent tutor design. * Pixel-level data fusion: In Reference [25], the authors provide a review of contributions on pixel-level data fusion for multiple visual sensors, with applications to remote sensing, medical diagnosis, surveillance, and photography. Although pixel-level data fusion is seldom devoted to intelligent educational applications, the fusion methods and quality measures design may be transferred to the design of ITRs. For example, matting methods may be used for multiple image fusion of moving objects captured by the moving visual sensor mounted on the ITR in dynamic scenarios, and recent popularized depth image sensor like Kinect may capture both posture and video data of the learners by fusing multi-channel 2D visual images and depth maps, which provides more information than single 2D visual sensory data for estimating learners' status [26; 27]. * Human action recognition: In Reference [28], a survey focusing on data fusion in applications of human action recognition is given. The authors compare the pros and cons of combining inertial sensory data with traditional visual sensors. Human action recognition may find its wide range of applications such as video analytics, robotics, and human-computer interaction. Therefore, it may be directly adopted in intelligent tutor design. For instance, in a complex tutoring scenario of much occlusion and moving objects, the inertial sensory data may complement the visual sensory data with a limited field of view using a support vector machine and hidden-Markov model. An example of applying human action recognition for second language ITR in Reference [29] uses hidden-Markov models and k-means algorithms to model and annotate Kinect sensory data, which is then fed to train action-recognition classifiers. * Affective computing: In Reference [30], the authors discuss in detail the recent contributions in affective computing, and the methods are evolving from uni-modal analysis to multi-modal fusion. Affective computing is an inter-disciplinary research field devoted to endow machines with cognitive capabilities to recognize, interpret, and express emotions and sentiments. Hence, it may be applied directly in ITR for students' emotion and sentiment interpretation that helps the analysis of learning style and knowledge level. From audio-visual sensory data, the ITR may facilitate a series of metabolic variables, e.g., heart monitoring and eye tracking, to gather information about the emotion as well as the level of engagement and attention of the students. Multiple kernel learning and deep convolutional neural network methods may be adopted for the sentiment detection of students. However, the state-of-the-art work in ITRs relies on uni-modal, mostly visual data to derive affective information.