A study of mediated communication enhancement systems employing alignment techniques between remote sites

In the past, the communication circle among human beings was so limited in terms of distance, time, and people. Today, the communication circle is almost unlimited enhanced by information technologies. People can communicate with other people over the globe beyond barriers of distance, time and/or systems. This is a result of the large number of advanced communication systems realized with new media and technologies which are available these days. However, the network-based communication environment provided by the new technologies is different from the traditional face-to-face communication in many aspects, such as reality, affinity, sympathy, mutuality, simplicity, availability, usability, and efficiently. We can accept the current communication systems as it is and change our communication behavior in accordance with the systems, which is what we are doing even though it is not preferable for everyone. Alternatively, we can develop novel technologies to implement new systems to help people communicate over the network that are as good as or even better than face-to-face communication. Replicating face-to-face environment is a challenge considering limitation of the current technologies, the entire user behavior and huge application fields. Therefore, this growing research field is worth to investigate in order to enhance and implement new mediated communication systems to facilitate the people’s communication. This dissertation discusses the empirical investigation and implementation of enhanced mediated communication systems between two sites, focusing on communication technologies between participants situated in different alignments, i.e., spatial alignment, surrounding alignment, and chronological alignment. Three experiments have been carried out to evaluate three alignment techniques in three different communications domains. The results showed that participants’ communicational behavior was significantly affected by the alignment, resulting in improved physical motion transference, enhanced conversation, and a heightened sense of other participants’ presence. The experiment in the first communication domain was carried out as a learning task of physical body motion in a Mixed Reality environment. Mixed Reality refers to the process of merging computer generated graphics onto a real world scene to produce new environments and visualizations where physical and digital objects coexist and interact in real time. In this domain, the participants’ body movement and rotation is one important aspect of nonverbal communication. Conveying an appropriate physical body posture is very important to achieve a better communication. In this research, the author investigated the spatial alignment and the effects on learning outcome in the Mixed Reality environment. Spatial alignment refers to the alignment of spatial aspects such as position, orientation, physical distances between people, body proxemics, body angling, etc. In this research, first, the author confirmed that the posture of a teacher body affects learning efficiency, when some teacher settings are more comfortable and easy to watch than others. A sequence of physical task learning experiments has been conducted using the Mixed Reality technology. The result suggested that the virtual teacher’s close side view is the optimal view for learning physical tasks that include significant single hand movements. However, when both hands are used, or body rotates around, a rotation angle adjustment becomes necessary. Based on these facts, the author proposed a novel automatic adjustment method “Motion Adaptive Virtual Teacher (MAVT)” governing the virtual teacher’s horizontal rotation angle, so that a learner can easily observe the important body motions. The method revealed effective for the motions that gradually reposition the most important moving part. This research activity indicates that spatial alignment of other person’s position and rotation is important for achieving optimal physical task learning. The second communication domain is related to videoconferencing. In daily life, numerous factors affect the effectiveness of our communication. The surrounding is one of these factors. Common surrounding factors that affect communication are background, lighting, distractions, distance, viewing angle, noise, etc. Surrounding alignment refers to the alignment of surrounding aspects such as environment or setting, space, dimension, border, background, location, etc. In face-to-face communication, the shared surrounding plays an important role in smoothing the communication process. However, in mediated communication, surroundings of the participants are different, which may affect communication process. Aligning the surrounding may help enhancing mediated communication. In this research, the author investigated the background effect in video-conferencing communication. To achieve unified surrounding, he proposed a novel method, which is to superimpose the real-time image of a person at a remote site onto surrounding image of the local site. This technology makes a local user feels as though the other person at a remote site sits in the local site. The technology has been implemented as the “Being Here System (BHS)”. The system was evaluated through an experiment and it was revealed that the proposed technology significantly enhanced communication by improving the perceived presence of other person, in comparison with conventional videoconferencing. This research also investigated user’s communication behavior in the implemented high presence video-conferencing, in comparison with a conventional video-conferencing. User’s communication behavior, verbal and nonverbal, is likely to be affected. The analysis of recorded video of system evaluation experiment revealed that the proposed system significantly affected user’s communication behavior such as speaking turn taking, speech overlapping, total speech time, and number of individual gaze off, while no significant effects were found in terms of the percentage of individual user speech and percentage of gaze off. This research showed that surrounding alignment of the background view significantly enhances participants’ communication behavior. The third communication domain is related to time shifted remote dining. Real time communication between people in remote sites requires that both persons are available at the same time and ready to communicate. This condition is not always easy to fulfill in the global networked environment because of time zone differences and other such contingent factors. One solution may lie in time shifted communication, or, in other words, delayed communication, which may affect fundamental communication process between people. In a time shifted environment, synchronization between participants is a challenge. This study proposed to minimize the synchronization gap through close chronological alignment. Chronological alignment refers to the alignment of chronological aspects that relate to events arrangement in the order of time. In this research, we investigated the effects of events synchrony in a time shifted tele-dining application. We proposed a system “KIZUNA” enabling people to enjoy a meal together in a virtual environment involving the transmission of recorded video messages. The system helps synchronize a video message with the dining pace of a participant who dines a meal watching the video message through synchronizing the start of dining sessions and adapting the displayed video’s playback speed to the difference in dining progress between local and remote person. This is likely to create a sort of real time behavioral mimicry which enhances communication and increases enjoyment while dining. A validation experiment revealed that the proposed KIZUNA adaptation method enhanced diners’ communication behavior and perceived presence of other person, in comparison with conventional time-shifted tele-dining. This result suggests a promising future for the KIZUNA system. This research activity shows that chronological alignment of events significantly enhances participants’ subjective impressions in communication. The results of the research shown in this dissertation revealed significant enhancement of communication between spatially or temporally separated individuals, suggesting a promising new approach to the design of mediated communication systems. In particular, the results suggest that focusing on alignment between remote sites is of great value in enhancing communication.