Pseudo Real-Time Live Event: Virtualization forNonverbal Live Entertainment and Sharing
Yamazaki Yusuke1, 2, Shirai Akihiko1
12GREE VR Studio Laboratory, JapanTokyo Institute of Technology, Japan

Corresponding author: Yamazaki Yusuke, z.yusuke.yamazaki@gree.net
Keywords: Nonverbal interaction – Pseudo real-time – Live entertainment – VibeShare

Abstract
We have developed a pseudo real-time live system that allows users to share nonverbal interactions with pastor future audiences. To test this system, we added nonverbal “Entities” such as sound and visual effects to anonline video. We experimented with three conditions: (1) the Entities were added to only the first half of thevideo, (2) the Entities were added to only the second half, and (3) no Entities were added. The results showedthat more participants were attracted to the experiment under the first condition than the others, and participants
spontaneously interacted with the added Entities. This suggests that pseudo real-time interaction enhances theuser experience for online video audiences.

1. Introduction
This paper aims to contribute to computer-human interaction in the post-Covid-19 era. Video conference toolsand broadcast services solve distance and physical restrictions in online classrooms, international conferences,and live performances. However, the interaction between the performers and the audiences is limited to sending
emoticons and questions and answers via text comments. These limited interactions, further limited by timezone differences, lack some nonverbal elements experienced at live events: the presence, applause, cheers, andvibes from audiences. To address the loss of nonverbal engagement, we developed VibeShare (GREE VR
Studio Laboratory, 2021), which converts nonverbal input into multimodal feedback. VibeShare enables
nonverbal, real-time live interaction between the performers and audiences (SHIRAI et al., 2019).
This paper proposes a pseudo real-time live system (Figure 1), which enables asynchronous nonverbal
communication and reports how it affects the user experience of watching online video content.

Figure 1: Concept of “pseudo real-time live.” The audiences watching a traditional real-time live event can only
see the reactions of those present. The audiences watching pseudo real-time live can enjoy the reactions of past
participants and feel like they are participating with them, even if they are the sole participant at that particulartime.

2. Evaluation
To examine the effect of our concept on user experience, we prepared the video shown in Figure 2 andexperimented with three conditions: (1) the recorded sound effects and emoticons (hereinafter called Entities)
were added to only the first half of the video (the condition called FH), (2) the Entities were added to only thesecond half (SH), and (3) no Entities were added (NO). An experimental web page (https://vibeshareex2101.herokuapp.com/), including the video and Entities, was developed in JavaScript. The URL was shared
as an experiment with anonymous participants via mailing lists and Twitter without explaining the aim. The
participants were able to drop out of the experiment if they felt bored. We defined “completions” as instancesof participants watching the video to the end.

Figure 2. Depiction of the video and Entities. The top part shows the Entity that occurs when a player touches thescreen or the recorded Entities play. The bottom part shows the sequence of the video. FH had Entities atappropriate times, such as at the end of the greeting message and on beat with the music. SH had Entities appearing
in the timings of the snare notes.

Figure 3. The number of taps associated with the video time code. The red series shows the number of recorded
Entities. The blue series shows the total number of taps from all participants. “CR” stands for completion rate,
the percentage of participants who watched the video to the end. “SR” stands for the second-half tap rate. “Ave.”stands for the average number of taps per participant.

3. Discussion and Conclusion
In Figure 3, the highest completion rate (CR) of FH suggests that the Entities that appeared in the first half
probably attracted the participants and prevented them from dropping out. The second-half tap rate (SR) was
also highest in FH, even though there were no recorded Entities in the second half. This indicates that therecorded Entities helped familiarize participants with the proposed experience and encouraged them to feel
free to tap and enjoy making the Entities. These results suggest that the proposed experience, which expresses
the past presence of others by Entities, even if asynchronously, positively attracts the users. We expect thepseudo real-time live system can redesign the interaction between online audiences.

4. References
GREE VR Studio Laboratory. 2021. VibeShare. Accessed May 12, 2021 fromhttps://vr.gree.net/lab/demo/vibeshare/.
Akihiko SHIRAI et al. 2019. Global Bidirectional Remote Haptic Live Entertainment by Virtual Beings.
Retrieved January 11, 2021 from https: //sa2019.siggraph.org/attend/real-time-live/session/111.