# Investigation on the Design of Anthropomorphic Oral Presentation Assistant Training System

Authors: Xinbo Zhao; Fei Fei
Journal: Mobile Information Systems
 (2022)

---

## Abstract

Oral presentation is a formal talk on a specific topic given to the audience through presentation tool. With the widespread use of oral presentations in a variety of professions, the ability to deliver an oral presentation using PowerPoint is becoming increasingly important. However, when a speaker practices alone, the presenter is unable to communicate with the audience, which means that the emotional input is unaffected, resulting in the training effect. A presentation assistant training system is designed using the anthropomorphic design concept which consists anthropomorphic appearance sensor to enable presentation practitioners to receive emotional feedback in the anthropomorphic appearance and further sensory immersion, thereby improving the training effect. The results of a comparative grouping experiment with 60 students at a university reveal that the system based on anthropomorphic design aspects has a greater practice effect. The design of the system has viable importance to improve oral presentation ability which also opens up for the improvement of the system, product research, and development.


---

## Body

## 1. Introduction

In the modern world, presentations are the most common mode of communication. Industry executives utilize presentations as a normal practice to make a stronger impression on their colleagues or clients and successfully communicate their messages [1]. Various presentation tools have become widely available as a result of the rapid growth of Internet application technology. There are few related technologies such as screen sharing, document sharing, and slide synchronization which are emerged in recent days. There has been a lot of study on technology that supports oral presentations, as well as the usage of virtual agents to help presenters prepare and give their presentations [2]. Recently, technologies like metaverse are emerged as digital landscape, helpful for educational institutions. Students and teachers can meet up in virtual reality via their virtual reality headsets regardless of their physical location. For those who are willing to pursue it, such functionality can lead to improved education [3].With the continuous advancement of world economic globalization, the importance of interpersonal communication efficiency and communication ability is becoming increasingly important in the fields of education, business, scientific research, and so on. Relevant education for the purpose of cultivating communication ability also emerges [4]. Effective interpersonal communication can improve labor force performance and overall organizational productivity [5]. Many of the current challenges in business presentations are attributable to outdated methodologies. Since its release in 1984, PowerPoint has evolved little, and it no longer fits into the new enterprises that are surrounded by evolving technologies. Nowadays, IoT-based system offers students better access to all from learning materials to communication channels, and they give educators the ability to assess student learning progress in real time [6–8].In recent years, the oral presentation mode using Microsoft PowerPoint has been widely used as a basic means of communication. At the same time, with the maturity of distance education technology and the influence of COVID-19, online communication has become more and more common. There has been work proposed by Nordin et al. [9], which focuses on improving the speaker’s ability to speak. However, despite this, there is almost no effective way to enable presentation practitioners to improve their presentation ability [10]. For oral public speaking, due to the lack of experience of speakers, they often rely too much on content memory and lack of confidence in oral presentation, and most presentation practitioners pay more attention to language organization and other links, and have no time to take into account sound performance and physical performance, so as to affect the evaluation of the audience [11]. This situation is more obvious for beginners and non-native speakers. The audience believed that the speaker lacked confidence and necessary preparation, which directly led to the poor overall evaluation of the presentation. In order to improve the presentation level, an oral presentation assistant training system based on anthropomorphic emotional feedback is designed to solve the problems encountered by presentation practitioners in the process of practicing alone and lay the foundation for product in the future. The major contributions of this article are to(i)
Study about the present situation of presentation assisted practice(ii)
Design oral presentation assistant training system based on anthropomorphic emotional feedback to solve the issues faced by presenters(iii)
Perform detailed experimentation to support the effectiveness and efficiency of the proposed systemThe remaining sections the article are structured as follows. Section2 discusses the state-of-the-art presentation assisted practice. Section 3 briefs about system concepts pertaining to oral presentation assistance training system. The system design and experimental results for presentation assistant training system are presented in Section 4. Section 5 concludes the research work.


## 2. Current Situation of Presentation Assisted Practice

Delivering oral presentations is common in higher institutions and organizations all around the world. In many disciplines, presentations play an important part in students’ learning and are frequently used as a form of assessment. There is a plethora of studies related to an oral presentation done by the researchers. Ferreira [12] discussed the implementation of technologies in the augmented way in which traditional classroom teaching is being augmented with contemporary technologies, together with course content acquisition, self-assessment, web searches, and discussion. This method is very useful to enhance the effectiveness of learning by establishing link with experts, diverse opinions, and dialogue.Many institutions are currently working on creating a cutting-edge physical-digital learning environment to help students improve their conversation and presenting skills. With state-of-the-art technology like touch panel discussion tables, digital posters, and an interactive wall-sized whiteboard, the setting ensures an efficient debate among users. It features a data mining system for recording, summarizing, and annotating discussions held within our facilities. Nagao et al. [13] created a digital poster writing tool, which is a unique tool for producing interactive digital posters that can be viewed on our digital poster presentation system. The efficiency of employing the facilities, such as the data mining system and the digital poster creation tool, is demonstrated by the evaluation findings.Peer marking is a crucial skill for students to learn since it helps them grasp the learning and assessment process. This approach is becoming more popular in medical education, especially for formative assessment. Peer marking in summative assessment, on the other hand, is not generally used since many teachers are concerned about biased marking by their peers’ students. Steverding et al. [14] studied about if marking summative peer assessments may increase peer-marking reliability. The peer-marking outcomes of a summative assessment of oral presentations of two cohorts of students were compared in a retrospective analysis. One group of students was told that their peer marks would be compared to a benchmark consisting of the average of examiner marks, and that their final test results would be made up of these scores, as well as peer and examiner marks. The other group of students had only recently been informed that their final exam scores will be determined by the examiner and peer marks.Tsang [15] proposed a novel method to raise the awareness of oral presenting (delivery) abilities in the context of self-directed learning. This work also examines and reports on the creation of a presentation skills inventory, as well as how learners’ awareness was raised through classroom discussion and the inventory.The students’ experiences of using the presentation in their studies were investigated by Hyll et al. [16] using a qualitative study approach. A semi-structured interview guide was used to interview students. The interviews were verbatim transcribed, and qualitative content analysis was used to analyze them. The analysis yielded two primary themes. First, the students felt as if they were in charge of their own learning: the presentation allowed for more freedom in studying and encouraged student participation in the learning process. Second, the presentation was part of a superficial learning process: students perceived it as a supplement to other educational activities, but noted that there were no pedagogical contacts, preventing the presentation’s information from being placed in a larger context.Although there has been numerous research on the use of PowerPoint Presentations (PPP) in various sectors, few have looked at the effects of various aspects of multimodal PowerPoint presentations on foreign language students’ language comprehension. Gordani and Khajavi [17] studied about how PowerPoint-supported (PPS) lectures affect foreign language university students’ immediate comprehension and long-term retention of content. Because of the following reasons, the study’s findings will be important to scholars and practitioners in the field of language instruction. First, PP lectures are commonly utilized in language classrooms, with the majority of pupils studying the slides for improved comprehension. Second, the impacts of PPP have not been thoroughly investigated, and teachers are unfamiliar with them.Ishino et al. [18] presented a robot lecture, in which a robot takes the place of a human lecturer and reconstructs their nonverbal behavior to improve the quality of their presentation. They created a model of nonverbal conduct in lecture to help with this reconstruction. This work also shows how a rebuilt robot lecture system can accurately imitate nonverbal behavior of human lecturers. Furthermore, this work describes a case study involving 36 participants and the system, the goal of which was to see if robot lecture with reconstruction was more effective at controlling learners’ attention and better at helping them understand the lecture contents than video lecture by a human or robot lecture with simple reproduction. Table 1 presents the summary of the state-of-the-art approaches.Table 1
Summary of various state-of-the-art approaches for oral presentation assistant training system.
ApproachAuthorYearDescriptionProsConsStudy on the implementation of technologies to augment the classroom.Ferreira [12]2012This study has three phases such as qualitative tradition concurrent phase and analysis of data to analyze the teaching and learning.Useful to improve teaching and student engagement.Ascertaining the cultural factors that contribute to adoption of technologies is not investigated.Physical-digital learning environment for discussion and presentation skills trainingNagao et al. [13]2015This tool designed an advanced physical-digital learning platform to train students for enhancing their discussion and presentation skills. This tool utilizes a data mining system that efficiently accounts, summarizes, and marks discussions held inside this facility.Students have the opportunities to improve the presentation skills using digital poster presentation system. Presentation evaluation system will be capable of assessing each presentation based on listener feedback, recorded audio-visual data, and interaction with the facilities.This tool is not useful in studying the body movement, gestures, posture, etc., which is common in evaluating presentations.Determining student learning outcomes using hypothesis studyManderson et al. [19]2015This work tries to identify connection between using the video platform, gaining confidence, and improving public speaking skills. This study also identified a correlation between the grades given by peers (on video) and the grades given by instructors (in class).This approach is used to identify usefulness of a video presentation environment in teaching public speaking.There may be some spillover effect when we try to have the video assignments occur around their in-class presentations. Survey responses are always troubled with issues, mainly when the teacher is also the researcher and students may sense that they need to respond a certain way even if they have been guaranteed that their participation or lack of participation in surveys will have no bearing on their course grades.Pilot study on marking of summative peer assessmentSteverding et al. [14]2016The retrospective analysis of peer marking of summative assessments of a teaching unit on presentation skills is presented.Peer marking is possible in certain summative assessments, and students generally assess their peers objectively.This work is not addressing the “friendly marking” problem (i.e., students can be prejudiced by apprehensions over recognizing the work of peers and subsequently award biased higher scores)Raising awareness of oral presentation skills in the context of self-regulated learning.Tsang [15]2017This work aims to promote self-regulated learning. This work discussed an assembling of an inventory of presentation skills and how learners’ awareness was raised through classroom conversation and the inventory.This work established the abundance of co-constructing learning and assessment materials in classrooms. This also supports how educators play an authoritative role in facilitating learners’ development.An inventory is created by a group of ethnologically similar learners in a specific educational setting; some of the stuffs may be inappropriate in other contexts, hence, creating concerns in generalizability.Discover the students’ learning experiences while using the Prezi presentationHyll et al. [16]2019This work performs in-depth exploration of the students’ experiences of using the presentation in their educations. With the help of semi-structured interview guide, students were interviewed. These interviews were recorded verbatim and examined using qualitative content analysis.This work can be a useful part of and a complement to blended learning.This is a small-scale study conducted within a specific subject.Qualitative descriptive method to assess speaking and presentation skillsIndriani [20]2020This method recorded 34 students’ speaking and presentation practices by using video camera and received the feedback about the same to improve and evaluate what they have done. This information is investigated descriptively with supporting theories.This method is used to create opportunities for learners and their peers to assess their language and presentation skills.This work is not suitable for improving the students’ speaking and presentation skills that needs a collaborative work of students.Presentation trainerYi et al. [21]2020This approach includes a presentation impression prediction system which uses support vector machine and Markov random field, or using multimodal neural network, to forecast audiences’ impressions for speech videos. This approach also has a presentation slide analysis system, which uses convolutional neural network and global average pooling to assess the design of slides.This approach is used to automatically evaluate the presentation with the help of machine learning models.The ways to handle the noises in the data and overfitting are not investigated.Study on the effect of PowerPoint-supported (PPS) lecturesGordani and Khajavi [17]2020This work aims to study the effect of lectures on instantaneous comprehension and longer-term retention of the content by foreign language university students.This is useful for researchers and practitioners in the field of language teaching for better comprehension. This work is the systematic way to investigate the effects of PPSThis work does not address the existence of students with diverse learning styles and preferences. Impact of sessions on courses with a different landscape such as the ones in need of a lot of graphics, images, illustrations, and numbering is not addressed.Robot lecture systemIshino et al. [18]2022This work proposes a robot lecture system, which enhances lecture by human and rebuilds lecture behavior conducted in the lecture. A novel lecture behavior model is designed with the help of such reconstruction.The robot lecture with reconstruction encouraged learner’s understanding of slide guts more than the video lecture and the robot lecture with simple reproduction.This work lacks in detecting learning states to dynamically change lecture behavior for interactive lecture.


### 2.1. Oral Presentation Ability Survey

With the acceleration of global economic integration, speaking ability has become a necessary social skill in order to better participate in world communication. Due to the wide use of English, the form and evaluation criteria of modern presentations are also deeply influenced by Europe and America. Due to the influence of history and culture on East Asia, the frequency of looking directly into each other’s eyes and the range of body movements in communication are often low, resulting in the presentation effect. Moreover, for inexperienced speakers, their performances in their presentations are often unsatisfactory. A random sampling survey of 493 students at a domestic university shows that less than 30% of them have experience in oral presentations in class; more than 90% of the students surveyed did not interact with the audience, including eye contact, during the presentation; during the presentation, more than 60% of the students will experience negative emotions such as tension and low self-confidence. More than 70% of the respondents recognized that the presentation practice was limited to the content of PPT pages and presentations and did not pay attention to the problems of nonverbal communication, such as facial expressions in the presentation; more than 90% of the respondents believe that speaking ability is an important skill for college students. If there is a corresponding training system, more than 80% of the respondents will try to use it.


### 2.2. Oral Presentation Aids

PowerPoint is widely used as a tool for various types of oral presentation and has become an international standard. The current research directions of presentation assistance are discussed below.


#### 2.2.1. Assistant Technology for Presentation Evaluation

Research on presentation evaluation has always been a field of concern for researchers. It includes research on presentation skills, the comprehensibility of presentation, and the correlation between the audience’s evaluation data onto presentation. The expert-based artificial presentation evaluation method is widely used, among which, the more commonly used method is to use the evaluation item table for evaluation at the presentation site or when watching a video [19].In order to effectively evaluate students’ presentations, Indriani [20] uses qualitative description methods, recorded presentation videos, reflection logs, and e-Google forms to evaluate students’ presentations and presentation skills. Among them, eye contact, body posture, and sound performance are considered to be important factors related to oral presentation skills.At present, there are few effective automatic evaluation tools for oral presentations. Yi et al. [21] developed a system called “presentation controller,” which includes a presentation impression prediction subsystem using support vector machines and Markov random field, a presentation impression prediction subsystem using multimodal neural networks, and a presentation slide analysis subsystem using convolutional neural network evaluation design. It is used to evaluate the performance of speakers and provide feedback related to impressions.


#### 2.2.2. Assistant Technology for Presentation Interaction

Presentation interactive assistant technology is mainly carried out by speakers to better interact with the audience in order to complete presentation activities. Different from the traditional way of interactive presentation by operating the mouse and keyboard, it can be controlled by sensors, gestures, and some specific actions, which will make the whole presentation process more natural. Aiming at the problem that the interaction between the speaker and the audience becomes more and more difficult in an online presentation, Kimura and others divided the online presentation into four categories: “with applause,” “no applause,” “only gesture/mark,” and “with applause and gesture/mark.” Then, the effect of applause is compared with the emotional information read from everyone’s faces through micro-expression. The results show that due to the influence of applause and gestures, there are significant differences in the moods of the two meetings, which play a positive role in improving the mood in online presentations [22]. Savitha implements a system that uses static gestures, uses the webcam connected to the computer as the input device, and uses the thinning method to obtain the number of parameters for fingers. Based on the recognized static gesture, it is used to control the slide. This method does not require other devices or databases [23].


#### 2.2.3. Auxiliary Technology for Presentation Documents

Building visual documentation to enhance the audience’s understanding of oral presentation slides is an important skill in educational and professional environments. Many junior undergraduates are not familiar with the basic principles of effective slide design to improve the audience’s understanding. Based on the method of experiential learning and peer teaching, Hammond improves students’ presentation ability and confidence by designing PowerPoint slides. Most students can meet the learning standards and provide conditions for the further development of activities [24]. In addition, Yamada et al. [25] conducted research on the assistant training system that can also edit the slide content on-site in the presentation mode, which enables the speaker to better explain and record the feedback from the audience at any time.


### 2.3. Evaluation Elements of Presentation Assistant System

The process of PowerPoint-based presentation practice is shown in Figure1. First, we need to complete the content of relevant presentation topics, including PowerPoint composition and design. After the presentation content is determined, we will start to practice, find problems in continuous practice, and then adjust the page content and presentation content accordingly.Figure 1
Oral presentation practice process.As a vital part of presentation, physical performance and sound performance need corresponding ability. Therefore, training to help speakers improve their expression skills is very important. In particular, speaking beginners must be able to reflect on and improve their unpopular physical and vocal performance in speaking practice. From the literature survey, it is found that “eye contact” and “body orientation” belong to two important evaluation indicators in communication. Based on the “volume” of the presentation performance evaluation project, consider “cadence,” “presentation speed,” and “accurate and clear pronunciation.” Among them, some take presentation expression as the research object, focusing on presentation speed, pause, and sound cadence [26]. It is very important to be able to hear the speaker’s voice, so “sound size” is considered as the basic evaluation item of presentation expression.Although physical expression is considered an embodiment of nonverbal expression, it is well known that gestures as nonverbal expression vary from country to country. International conferences and other presentations are usually held in English, so modern presentation evaluation indicators often follow the habits of English-speaking countries. On the contrary, due to culture, some people think that the gestures, expressions, and intonation of oriental people (Japanese and Chinese) are not attractive enough. Therefore, it is necessary to consider nonverbal expressions such as body movements as part of physical expression in order to make attractive presentations at international conferences on English.


## 2.1. Oral Presentation Ability Survey

With the acceleration of global economic integration, speaking ability has become a necessary social skill in order to better participate in world communication. Due to the wide use of English, the form and evaluation criteria of modern presentations are also deeply influenced by Europe and America. Due to the influence of history and culture on East Asia, the frequency of looking directly into each other’s eyes and the range of body movements in communication are often low, resulting in the presentation effect. Moreover, for inexperienced speakers, their performances in their presentations are often unsatisfactory. A random sampling survey of 493 students at a domestic university shows that less than 30% of them have experience in oral presentations in class; more than 90% of the students surveyed did not interact with the audience, including eye contact, during the presentation; during the presentation, more than 60% of the students will experience negative emotions such as tension and low self-confidence. More than 70% of the respondents recognized that the presentation practice was limited to the content of PPT pages and presentations and did not pay attention to the problems of nonverbal communication, such as facial expressions in the presentation; more than 90% of the respondents believe that speaking ability is an important skill for college students. If there is a corresponding training system, more than 80% of the respondents will try to use it.


## 2.2. Oral Presentation Aids

PowerPoint is widely used as a tool for various types of oral presentation and has become an international standard. The current research directions of presentation assistance are discussed below.


### 2.2.1. Assistant Technology for Presentation Evaluation

Research on presentation evaluation has always been a field of concern for researchers. It includes research on presentation skills, the comprehensibility of presentation, and the correlation between the audience’s evaluation data onto presentation. The expert-based artificial presentation evaluation method is widely used, among which, the more commonly used method is to use the evaluation item table for evaluation at the presentation site or when watching a video [19].In order to effectively evaluate students’ presentations, Indriani [20] uses qualitative description methods, recorded presentation videos, reflection logs, and e-Google forms to evaluate students’ presentations and presentation skills. Among them, eye contact, body posture, and sound performance are considered to be important factors related to oral presentation skills.At present, there are few effective automatic evaluation tools for oral presentations. Yi et al. [21] developed a system called “presentation controller,” which includes a presentation impression prediction subsystem using support vector machines and Markov random field, a presentation impression prediction subsystem using multimodal neural networks, and a presentation slide analysis subsystem using convolutional neural network evaluation design. It is used to evaluate the performance of speakers and provide feedback related to impressions.


### 2.2.2. Assistant Technology for Presentation Interaction

Presentation interactive assistant technology is mainly carried out by speakers to better interact with the audience in order to complete presentation activities. Different from the traditional way of interactive presentation by operating the mouse and keyboard, it can be controlled by sensors, gestures, and some specific actions, which will make the whole presentation process more natural. Aiming at the problem that the interaction between the speaker and the audience becomes more and more difficult in an online presentation, Kimura and others divided the online presentation into four categories: “with applause,” “no applause,” “only gesture/mark,” and “with applause and gesture/mark.” Then, the effect of applause is compared with the emotional information read from everyone’s faces through micro-expression. The results show that due to the influence of applause and gestures, there are significant differences in the moods of the two meetings, which play a positive role in improving the mood in online presentations [22]. Savitha implements a system that uses static gestures, uses the webcam connected to the computer as the input device, and uses the thinning method to obtain the number of parameters for fingers. Based on the recognized static gesture, it is used to control the slide. This method does not require other devices or databases [23].


### 2.2.3. Auxiliary Technology for Presentation Documents

Building visual documentation to enhance the audience’s understanding of oral presentation slides is an important skill in educational and professional environments. Many junior undergraduates are not familiar with the basic principles of effective slide design to improve the audience’s understanding. Based on the method of experiential learning and peer teaching, Hammond improves students’ presentation ability and confidence by designing PowerPoint slides. Most students can meet the learning standards and provide conditions for the further development of activities [24]. In addition, Yamada et al. [25] conducted research on the assistant training system that can also edit the slide content on-site in the presentation mode, which enables the speaker to better explain and record the feedback from the audience at any time.


## 2.2.1. Assistant Technology for Presentation Evaluation

Research on presentation evaluation has always been a field of concern for researchers. It includes research on presentation skills, the comprehensibility of presentation, and the correlation between the audience’s evaluation data onto presentation. The expert-based artificial presentation evaluation method is widely used, among which, the more commonly used method is to use the evaluation item table for evaluation at the presentation site or when watching a video [19].In order to effectively evaluate students’ presentations, Indriani [20] uses qualitative description methods, recorded presentation videos, reflection logs, and e-Google forms to evaluate students’ presentations and presentation skills. Among them, eye contact, body posture, and sound performance are considered to be important factors related to oral presentation skills.At present, there are few effective automatic evaluation tools for oral presentations. Yi et al. [21] developed a system called “presentation controller,” which includes a presentation impression prediction subsystem using support vector machines and Markov random field, a presentation impression prediction subsystem using multimodal neural networks, and a presentation slide analysis subsystem using convolutional neural network evaluation design. It is used to evaluate the performance of speakers and provide feedback related to impressions.


## 2.2.2. Assistant Technology for Presentation Interaction

Presentation interactive assistant technology is mainly carried out by speakers to better interact with the audience in order to complete presentation activities. Different from the traditional way of interactive presentation by operating the mouse and keyboard, it can be controlled by sensors, gestures, and some specific actions, which will make the whole presentation process more natural. Aiming at the problem that the interaction between the speaker and the audience becomes more and more difficult in an online presentation, Kimura and others divided the online presentation into four categories: “with applause,” “no applause,” “only gesture/mark,” and “with applause and gesture/mark.” Then, the effect of applause is compared with the emotional information read from everyone’s faces through micro-expression. The results show that due to the influence of applause and gestures, there are significant differences in the moods of the two meetings, which play a positive role in improving the mood in online presentations [22]. Savitha implements a system that uses static gestures, uses the webcam connected to the computer as the input device, and uses the thinning method to obtain the number of parameters for fingers. Based on the recognized static gesture, it is used to control the slide. This method does not require other devices or databases [23].


## 2.2.3. Auxiliary Technology for Presentation Documents

Building visual documentation to enhance the audience’s understanding of oral presentation slides is an important skill in educational and professional environments. Many junior undergraduates are not familiar with the basic principles of effective slide design to improve the audience’s understanding. Based on the method of experiential learning and peer teaching, Hammond improves students’ presentation ability and confidence by designing PowerPoint slides. Most students can meet the learning standards and provide conditions for the further development of activities [24]. In addition, Yamada et al. [25] conducted research on the assistant training system that can also edit the slide content on-site in the presentation mode, which enables the speaker to better explain and record the feedback from the audience at any time.


## 2.3. Evaluation Elements of Presentation Assistant System

The process of PowerPoint-based presentation practice is shown in Figure1. First, we need to complete the content of relevant presentation topics, including PowerPoint composition and design. After the presentation content is determined, we will start to practice, find problems in continuous practice, and then adjust the page content and presentation content accordingly.Figure 1
Oral presentation practice process.As a vital part of presentation, physical performance and sound performance need corresponding ability. Therefore, training to help speakers improve their expression skills is very important. In particular, speaking beginners must be able to reflect on and improve their unpopular physical and vocal performance in speaking practice. From the literature survey, it is found that “eye contact” and “body orientation” belong to two important evaluation indicators in communication. Based on the “volume” of the presentation performance evaluation project, consider “cadence,” “presentation speed,” and “accurate and clear pronunciation.” Among them, some take presentation expression as the research object, focusing on presentation speed, pause, and sound cadence [26]. It is very important to be able to hear the speaker’s voice, so “sound size” is considered as the basic evaluation item of presentation expression.Although physical expression is considered an embodiment of nonverbal expression, it is well known that gestures as nonverbal expression vary from country to country. International conferences and other presentations are usually held in English, so modern presentation evaluation indicators often follow the habits of English-speaking countries. On the contrary, due to culture, some people think that the gestures, expressions, and intonation of oriental people (Japanese and Chinese) are not attractive enough. Therefore, it is necessary to consider nonverbal expressions such as body movements as part of physical expression in order to make attractive presentations at international conferences on English.


## 3. System Concept

### 3.1. Personification Communication

Anthropomorphic communication has been widely used in various industries. With the development of social media, since 2014, major technology companies have launched virtual images to communicate with consumers. These forms of communication can help both sides establish positive emotional feedback. Lijun Xu and others concluded that nonverbal symbols based on anthropomorphic smiling faces can help people communicate their emotions and product marketing [27]. In the field of tourism robots, it also has a positive impact on the promotion of customer value co-creation intention [28]. Bing Dwen, the mascot of the Beijing Winter Olympic Games, also displayed the charming image of the 2022 Winter Olympic Games.


### 3.2. Emotional Interaction Theory

Communication between people is a process of information and knowledge transmission and emotional exchange through language, body movements, expressions, and other means. In the context of human-computer interaction, to make the machine replace humans to some extent and have the effect of harmonious interaction with users so that the machine has the same behavioral characteristics as humans are the focus of current research. Among them, Professor Picard of MIT initiated related research such as “emotional computing.” In addition, from the perspective of embodied theory, Li and others explained and established the visual design paradigm of virtual assistance in the field of human-computer interaction on three levels: virtual personality shaping, visual expression of nonverbal behavior, and social visual feedback [29].


### 3.3. Anthropomorphic Emotional Feedback

By means of anthropomorphic communication and integrating the image of affinity and easing tension, we can give users positive interaction. This method is similar to AI Xing’s learning theory of timely recognizing learners’ actions and expressions with the help of various intelligent technologies and giving feedback to realize the dynamic interaction between people and the environment [30]. Applying this method of presentation practice can obtain the same learning effect, improve practice efficiency, and promote the improvement of the practitioners’ presentation level.


## 3.1. Personification Communication

Anthropomorphic communication has been widely used in various industries. With the development of social media, since 2014, major technology companies have launched virtual images to communicate with consumers. These forms of communication can help both sides establish positive emotional feedback. Lijun Xu and others concluded that nonverbal symbols based on anthropomorphic smiling faces can help people communicate their emotions and product marketing [27]. In the field of tourism robots, it also has a positive impact on the promotion of customer value co-creation intention [28]. Bing Dwen, the mascot of the Beijing Winter Olympic Games, also displayed the charming image of the 2022 Winter Olympic Games.


## 3.2. Emotional Interaction Theory

Communication between people is a process of information and knowledge transmission and emotional exchange through language, body movements, expressions, and other means. In the context of human-computer interaction, to make the machine replace humans to some extent and have the effect of harmonious interaction with users so that the machine has the same behavioral characteristics as humans are the focus of current research. Among them, Professor Picard of MIT initiated related research such as “emotional computing.” In addition, from the perspective of embodied theory, Li and others explained and established the visual design paradigm of virtual assistance in the field of human-computer interaction on three levels: virtual personality shaping, visual expression of nonverbal behavior, and social visual feedback [29].


## 3.3. Anthropomorphic Emotional Feedback

By means of anthropomorphic communication and integrating the image of affinity and easing tension, we can give users positive interaction. This method is similar to AI Xing’s learning theory of timely recognizing learners’ actions and expressions with the help of various intelligent technologies and giving feedback to realize the dynamic interaction between people and the environment [30]. Applying this method of presentation practice can obtain the same learning effect, improve practice efficiency, and promote the improvement of the practitioners’ presentation level.


## 4. System Design and Experimental Analysis

In this section, the system composition, information structure model, system design, and experimental results for presentation assistant training system are discussed.


### 4.1. System Composition, Information Structure Model, and Construction Framework

As shown in Figure2, the system is composed of a noncontact body sensing sensor “Xiaofei” (hereinafter referred to as “Xiaofei”), a high-performance laptop, and an external LCD.Figure 2
System composition diagram.In order to make the speaker feel immersed in the real venue when practicing alone, a sensor “Xiaofei” with anthropomorphic appearance is placed at the audience position of the presentation practice site (Figure3). “Xiaofei” can give corresponding feedback according to the speaker’s performance. If the speaker’s performance is excellent, Xiaofei will show a smiling face. In addition, “Xiaofei” can also show expressionless, crying face, and surprise in order to achieve the effect of interaction between the speaker and him.Figure 3
“Xiaofei” with anthropomorphic appearance.


### 4.2. Construction of Information Structure Model in Presentation Assistant Training System

The system is mainly composed of four modules: sound feedback, visual feedback, data recording, and PPT operation (Figure4). Among them, “sound detection function” includes the detection and display of the speaker’s voice size, oral language, and tonal cadence; “visual detection function” includes real-time performance of speaker’s limb angle, visual detection of sound waveform, real-time detection, and display of volume; “data recording function” is mainly used to convert the speaker’s body and sound signals collected by the sensor into numerical values through A/D transformation, and record and save them in the system background database; “PPT operation function” mainly realizes assistant functions such as page turning before and after PPT, designated page switch, page marking, and timer.Figure 4
System function module diagram.


### 4.3. System Design Expression and Experimental Results

The interactive interface of the system is shown in Figure5. The interface is mainly divided into two parts. On the left is the data collected by the sensor in real time (the resolution of real-time video image is 640 × 480), including video, audio, and limb movement data. When the inappropriate limb and sound performance lasts for a certain time, a dialog box will pop up in real time to prompt the speaker to improve. On the right is the PPT page control area. Based on the system, the comparative experiments are carried out in the two cases of no application and application of “Xiaofei.” 60 students were randomly selected from a university and randomly divided into two groups of 30 people in each group to participate in two different experiments.Figure 5
System interaction interface.


#### 4.3.1. Detection times

Table2 shows the number of speakers and occurrence times corresponding to each detection index in the experiment. Through Wilcoxon signed-rank test, the number of voice detection and visual detection in the three indicators was compared, and no significant difference was found. In the experiment, whether there is “Xiaofei” or not, it does not affect the presentation level of relevant experimental participants. In other words, the presentation level of presentation practitioners will not be significantly different due to the change of practice environment, which is also in line with the objective law of ability training.Table 2
Evaluation results of evaluators.
Detection itemsNo “Xiaofei”“Xiaofei”No. of experimentsTimes (Avg)No. of experimentsTimes (Avg)Loud voicing306.5306.4Forward (head orientation)181.9172.1Forward (body orientation)92.281.9


#### 4.3.2. Test Effect without “Xiaofei” Participation

Table3 shows the test results without the participation of “Xiaofei.” Before the system feedback, the value of the test item “loud voicing” was 0.55 and then increased to 0.62. It was analyzed by the paired t-test method. The results showed that there was a significant difference (P < 0.01). In all 136-feedback data, the effective number of times when the volume exceeds 0.70 is 119, and the effective rate is 88%. There was little change between 7.4 degrees before “forward (head orientation)” feedback and 7.9 degrees after feedback, and there was no significant difference in paired t-test (P = . 25). In all 34 times of data, the effective times were 23 times and the effective rate was 68%. 16.3 degrees before “forward (body orientation)” feedback changed to 12.0 degrees after feedback, which was analyzed by paired t-test, and there was a significant difference (P < 0.05). In all 20 times of data, the effective times were 19 times and the effective rate was 95%. It can be concluded that the training for “loud” and “forward (body orientation)” in the test item is effective, but it has no effect on the test item “forward (head orientation).”Table 3
Detection effect without “Xiaofei.”
Detection itemsBeforeAfterEffective/countLoud voicing∗∗0.550.62119/136Forward (head orientation)7.4 deg.7.9 deg.23/34Forward (body orientation)∗16.3 deg.12.0 deg.19/20Pairedt-test: ∗P < .05; ∗∗P < .01.


#### 4.3.3. Test Effect of “Xiaofei” Participation

Table4 shows the test results of “Xiaofei” participation. The volume value of “loud voicing” before detection was 0.57 and after detection was 0.65. There was significant difference in the results by using paired t-test (P < 0.01). In all 134 times of data, the effective times were 123 times and the effective rate was 92%. For the test item “forward (head orientation),” the included angle between the front and the front was reduced from 10.7 degrees before the test of 4.8 degrees after the test. The results were analyzed by paired t-test, and there was a significant difference (P < 0.01). In all 35 times of data, the effective times were 33 times and the effective rate was 94%. The test item “forward (body orientation)” decreased from 15.0° before the test of 5.7 degrees after the test. The results were analyzed by paired t-test, and there was significant difference (P < 0.01). In all 15 times of data, the effective times were 14 times and the effective rate was 93%. It is concluded that visual detection is effective against three types of detection items: loud voicing, forward (head orientation), and forward (body orientation).Table 4
Detection effect with “Xiaofei.”
Detection itemsBeforeAfterEffective/countLoud voicing∗∗0.570.65123/134Forward (head orientation)∗∗10.7 deg.4.8 Deg.33/35Forward (body orientation)∗∗15.0 deg.5.7 deg.14/15Pairedt-test:∗P < .05; ∗∗P < .01.


#### 4.3.4. Comparison of Test Results with or without the Participation of “Xiaofei”

Table5 shows the comparison results of the test results with or without the participation of “Xiaofei.” It is observed that for the detection index “loud voicing,” the experimental environment with or without the participation of “Xiaofei” has a significant effect. For the detection index of “forward (head orientation),” the experiment with “Xiaofei” showed significant effectiveness, while the experiment without “Xiaofei” showed invalid results. For the detection index of “forward (body orientation),” although the experiments with and without “Xiaofei” showed certain effectiveness, the effectiveness of the experiments with “Xiaofei” was higher than that without “Xiaofei.” In conclusion, compared with the system without the introduction to “Xiaofei,” the oral presentation assistant practice system with sensor “Xiaofei” based on anthropomorphic emotional feedback has improved the practice effect and has the possibility of further improvement and final realization of product.Table 5
Comparison of detection effect with or without “Xiaofei.”
Detection items“Xiaofei”No “Xiaofei”Loud voice∗∗Effective∗∗Effective∗∗Forward (head orientation)∗∗Effective∗∗InvalidForward (body orientation)∗∗Effective∗∗Effective∗Pairedt-test:∗P < .05; ∗∗P < .01.


## 4.1. System Composition, Information Structure Model, and Construction Framework

As shown in Figure2, the system is composed of a noncontact body sensing sensor “Xiaofei” (hereinafter referred to as “Xiaofei”), a high-performance laptop, and an external LCD.Figure 2
System composition diagram.In order to make the speaker feel immersed in the real venue when practicing alone, a sensor “Xiaofei” with anthropomorphic appearance is placed at the audience position of the presentation practice site (Figure3). “Xiaofei” can give corresponding feedback according to the speaker’s performance. If the speaker’s performance is excellent, Xiaofei will show a smiling face. In addition, “Xiaofei” can also show expressionless, crying face, and surprise in order to achieve the effect of interaction between the speaker and him.Figure 3
“Xiaofei” with anthropomorphic appearance.


## 4.2. Construction of Information Structure Model in Presentation Assistant Training System

The system is mainly composed of four modules: sound feedback, visual feedback, data recording, and PPT operation (Figure4). Among them, “sound detection function” includes the detection and display of the speaker’s voice size, oral language, and tonal cadence; “visual detection function” includes real-time performance of speaker’s limb angle, visual detection of sound waveform, real-time detection, and display of volume; “data recording function” is mainly used to convert the speaker’s body and sound signals collected by the sensor into numerical values through A/D transformation, and record and save them in the system background database; “PPT operation function” mainly realizes assistant functions such as page turning before and after PPT, designated page switch, page marking, and timer.Figure 4
System function module diagram.


## 4.3. System Design Expression and Experimental Results

The interactive interface of the system is shown in Figure5. The interface is mainly divided into two parts. On the left is the data collected by the sensor in real time (the resolution of real-time video image is 640 × 480), including video, audio, and limb movement data. When the inappropriate limb and sound performance lasts for a certain time, a dialog box will pop up in real time to prompt the speaker to improve. On the right is the PPT page control area. Based on the system, the comparative experiments are carried out in the two cases of no application and application of “Xiaofei.” 60 students were randomly selected from a university and randomly divided into two groups of 30 people in each group to participate in two different experiments.Figure 5
System interaction interface.


### 4.3.1. Detection times

Table2 shows the number of speakers and occurrence times corresponding to each detection index in the experiment. Through Wilcoxon signed-rank test, the number of voice detection and visual detection in the three indicators was compared, and no significant difference was found. In the experiment, whether there is “Xiaofei” or not, it does not affect the presentation level of relevant experimental participants. In other words, the presentation level of presentation practitioners will not be significantly different due to the change of practice environment, which is also in line with the objective law of ability training.Table 2
Evaluation results of evaluators.
Detection itemsNo “Xiaofei”“Xiaofei”No. of experimentsTimes (Avg)No. of experimentsTimes (Avg)Loud voicing306.5306.4Forward (head orientation)181.9172.1Forward (body orientation)92.281.9


### 4.3.2. Test Effect without “Xiaofei” Participation

Table3 shows the test results without the participation of “Xiaofei.” Before the system feedback, the value of the test item “loud voicing” was 0.55 and then increased to 0.62. It was analyzed by the paired t-test method. The results showed that there was a significant difference (P < 0.01). In all 136-feedback data, the effective number of times when the volume exceeds 0.70 is 119, and the effective rate is 88%. There was little change between 7.4 degrees before “forward (head orientation)” feedback and 7.9 degrees after feedback, and there was no significant difference in paired t-test (P = . 25). In all 34 times of data, the effective times were 23 times and the effective rate was 68%. 16.3 degrees before “forward (body orientation)” feedback changed to 12.0 degrees after feedback, which was analyzed by paired t-test, and there was a significant difference (P < 0.05). In all 20 times of data, the effective times were 19 times and the effective rate was 95%. It can be concluded that the training for “loud” and “forward (body orientation)” in the test item is effective, but it has no effect on the test item “forward (head orientation).”Table 3
Detection effect without “Xiaofei.”
Detection itemsBeforeAfterEffective/countLoud voicing∗∗0.550.62119/136Forward (head orientation)7.4 deg.7.9 deg.23/34Forward (body orientation)∗16.3 deg.12.0 deg.19/20Pairedt-test: ∗P < .05; ∗∗P < .01.


### 4.3.3. Test Effect of “Xiaofei” Participation

Table4 shows the test results of “Xiaofei” participation. The volume value of “loud voicing” before detection was 0.57 and after detection was 0.65. There was significant difference in the results by using paired t-test (P < 0.01). In all 134 times of data, the effective times were 123 times and the effective rate was 92%. For the test item “forward (head orientation),” the included angle between the front and the front was reduced from 10.7 degrees before the test of 4.8 degrees after the test. The results were analyzed by paired t-test, and there was a significant difference (P < 0.01). In all 35 times of data, the effective times were 33 times and the effective rate was 94%. The test item “forward (body orientation)” decreased from 15.0° before the test of 5.7 degrees after the test. The results were analyzed by paired t-test, and there was significant difference (P < 0.01). In all 15 times of data, the effective times were 14 times and the effective rate was 93%. It is concluded that visual detection is effective against three types of detection items: loud voicing, forward (head orientation), and forward (body orientation).Table 4
Detection effect with “Xiaofei.”
Detection itemsBeforeAfterEffective/countLoud voicing∗∗0.570.65123/134Forward (head orientation)∗∗10.7 deg.4.8 Deg.33/35Forward (body orientation)∗∗15.0 deg.5.7 deg.14/15Pairedt-test:∗P < .05; ∗∗P < .01.


### 4.3.4. Comparison of Test Results with or without the Participation of “Xiaofei”

Table5 shows the comparison results of the test results with or without the participation of “Xiaofei.” It is observed that for the detection index “loud voicing,” the experimental environment with or without the participation of “Xiaofei” has a significant effect. For the detection index of “forward (head orientation),” the experiment with “Xiaofei” showed significant effectiveness, while the experiment without “Xiaofei” showed invalid results. For the detection index of “forward (body orientation),” although the experiments with and without “Xiaofei” showed certain effectiveness, the effectiveness of the experiments with “Xiaofei” was higher than that without “Xiaofei.” In conclusion, compared with the system without the introduction to “Xiaofei,” the oral presentation assistant practice system with sensor “Xiaofei” based on anthropomorphic emotional feedback has improved the practice effect and has the possibility of further improvement and final realization of product.Table 5
Comparison of detection effect with or without “Xiaofei.”
Detection items“Xiaofei”No “Xiaofei”Loud voice∗∗Effective∗∗Effective∗∗Forward (head orientation)∗∗Effective∗∗InvalidForward (body orientation)∗∗Effective∗∗Effective∗Pairedt-test:∗P < .05; ∗∗P < .01.


## 4.3.1. Detection times

Table2 shows the number of speakers and occurrence times corresponding to each detection index in the experiment. Through Wilcoxon signed-rank test, the number of voice detection and visual detection in the three indicators was compared, and no significant difference was found. In the experiment, whether there is “Xiaofei” or not, it does not affect the presentation level of relevant experimental participants. In other words, the presentation level of presentation practitioners will not be significantly different due to the change of practice environment, which is also in line with the objective law of ability training.Table 2
Evaluation results of evaluators.
Detection itemsNo “Xiaofei”“Xiaofei”No. of experimentsTimes (Avg)No. of experimentsTimes (Avg)Loud voicing306.5306.4Forward (head orientation)181.9172.1Forward (body orientation)92.281.9


## 4.3.2. Test Effect without “Xiaofei” Participation

Table3 shows the test results without the participation of “Xiaofei.” Before the system feedback, the value of the test item “loud voicing” was 0.55 and then increased to 0.62. It was analyzed by the paired t-test method. The results showed that there was a significant difference (P < 0.01). In all 136-feedback data, the effective number of times when the volume exceeds 0.70 is 119, and the effective rate is 88%. There was little change between 7.4 degrees before “forward (head orientation)” feedback and 7.9 degrees after feedback, and there was no significant difference in paired t-test (P = . 25). In all 34 times of data, the effective times were 23 times and the effective rate was 68%. 16.3 degrees before “forward (body orientation)” feedback changed to 12.0 degrees after feedback, which was analyzed by paired t-test, and there was a significant difference (P < 0.05). In all 20 times of data, the effective times were 19 times and the effective rate was 95%. It can be concluded that the training for “loud” and “forward (body orientation)” in the test item is effective, but it has no effect on the test item “forward (head orientation).”Table 3
Detection effect without “Xiaofei.”
Detection itemsBeforeAfterEffective/countLoud voicing∗∗0.550.62119/136Forward (head orientation)7.4 deg.7.9 deg.23/34Forward (body orientation)∗16.3 deg.12.0 deg.19/20Pairedt-test: ∗P < .05; ∗∗P < .01.


## 4.3.3. Test Effect of “Xiaofei” Participation

Table4 shows the test results of “Xiaofei” participation. The volume value of “loud voicing” before detection was 0.57 and after detection was 0.65. There was significant difference in the results by using paired t-test (P < 0.01). In all 134 times of data, the effective times were 123 times and the effective rate was 92%. For the test item “forward (head orientation),” the included angle between the front and the front was reduced from 10.7 degrees before the test of 4.8 degrees after the test. The results were analyzed by paired t-test, and there was a significant difference (P < 0.01). In all 35 times of data, the effective times were 33 times and the effective rate was 94%. The test item “forward (body orientation)” decreased from 15.0° before the test of 5.7 degrees after the test. The results were analyzed by paired t-test, and there was significant difference (P < 0.01). In all 15 times of data, the effective times were 14 times and the effective rate was 93%. It is concluded that visual detection is effective against three types of detection items: loud voicing, forward (head orientation), and forward (body orientation).Table 4
Detection effect with “Xiaofei.”
Detection itemsBeforeAfterEffective/countLoud voicing∗∗0.570.65123/134Forward (head orientation)∗∗10.7 deg.4.8 Deg.33/35Forward (body orientation)∗∗15.0 deg.5.7 deg.14/15Pairedt-test:∗P < .05; ∗∗P < .01.


## 4.3.4. Comparison of Test Results with or without the Participation of “Xiaofei”

Table5 shows the comparison results of the test results with or without the participation of “Xiaofei.” It is observed that for the detection index “loud voicing,” the experimental environment with or without the participation of “Xiaofei” has a significant effect. For the detection index of “forward (head orientation),” the experiment with “Xiaofei” showed significant effectiveness, while the experiment without “Xiaofei” showed invalid results. For the detection index of “forward (body orientation),” although the experiments with and without “Xiaofei” showed certain effectiveness, the effectiveness of the experiments with “Xiaofei” was higher than that without “Xiaofei.” In conclusion, compared with the system without the introduction to “Xiaofei,” the oral presentation assistant practice system with sensor “Xiaofei” based on anthropomorphic emotional feedback has improved the practice effect and has the possibility of further improvement and final realization of product.Table 5
Comparison of detection effect with or without “Xiaofei.”
Detection items“Xiaofei”No “Xiaofei”Loud voice∗∗Effective∗∗Effective∗∗Forward (head orientation)∗∗Effective∗∗InvalidForward (body orientation)∗∗Effective∗∗Effective∗Pairedt-test:∗P < .05; ∗∗P < .01.


## 5. Conclusion and Future Directions

In recent years, more and more emphasis has been placed on product experience and user interaction in product design. In addition, the products of presentation assistance application systems in the market are still blank and there are few patents. The existing traditional presentation training methods are often based on experience. Practitioners often cannot accurately evaluate their own practice effect, cannot objectively evaluate their own presentation level, and cannot meet the application of presentation practitioners, especially for beginners, resulting in poor practice effect. This research integrates the knowledge of design, psychology, and information technology, designs the appearance of the system, and demonstrates the effectiveness of the system through experiments. This study designed a set of oral presentation assistant practice systems using anthropomorphic noncontact somatosensory sensors and tried to integrate emotional elements such as “happiness, anger, sadness, and joy,”, which has practical significance in the design and development of intelligent teaching tool products and the promotion of practical related research in the future. In the future, it is considered to apply the real-time feedback interface for immersive optimization so that presentation practitioners can carry out corresponding nonverbal interactions such as limbs and expressions through the anthropomorphic emotional feedback real-time assistant practice system, so as to improve the oral presentation.



---

*Source: 3719010-2022-07-16.xml*