DEIM Forum 2017 I3-1

A VR Application for Rehearsing and Improving Presentation
Yixuan He†Hayato Yamana†‡†Graduate School of Fundamental Science and Engineering, Waseda University, 3-4-1 Okubo, Shinjuku-ku Tokyo,Japan‡Faculty of Science and Engineering, Waseda University, 3-4-1 Okubo, Shinjuku-ku Tokyo, Japan‡National Institute of Informatics, 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo, Japan
E-mail: {pandaxuan, yamana}@yama.info.waseda.ac.jp

Abstract: In this paper we propose a presentation rehearsal application in Virtual Reality (VR) that could be used to help
users practice, improve their presentation skills, and overcome stage fright. In this application, users can import their own
materials and thus they can see the slides during their speech in virtual environment. When they finish the rehearsal, a scorewill be given as a feedback. The whole presentation is evaluated by the user’s stage usage and interaction with audience, suchas eye contacts. Additionally, an eye tracking heat map will also be generated. It will allow users to see where and how theyinteract within the virtual environment. Recently, in Virtual Reality Applications, there are several domains show great
potentials: Video Games, Tourism, Education, etc. Especially, Interactive Learning in VR gradually becomes more useful andapproachable compared to traditional education ways. It is proved to be more effective to enable users to get more involved in
the learning process. In this research, our goal is not only to provide the user a place to practice presentation despite thelimitations of time and space, but also to help them overcome the performance anxiety and improve the quality. To the best of
our knowledge, our paper is the first to research on the “Visual” impact of presentation in Virtual Reality. “Visual” means how
the presenter uses the space and interacts with the audience. In order to compare users’ pre and post-practice, we evaluate oursystem by self-assessment and expert-assessment.
Keyword:

Virtual Reality, Application, Presentation, Public Speaking, Stage Frightpracticing in VR is not

1. INTRODUCTION
Presentations gradually become common and importantdependent on people andscheduling, place availability and costs [1].

in our everyday life. Besides the context and layout of theAccording to the research, there are mainly threematerials, nonverbal communication , such as volume andaspects when defining a good presentation: “Verbal,”

speed of voice, eye contacts, stage usage , are key elements“Vocal,” and “Visual” 4 . Verbal represents words peopleof a successful presentation. However, millions of peoplechoose, vocabulary, grammar, etc. Vocal includes tone ofsuffer from performance anxiety, commonly called “stagevoice, pitch range, loudness, etc. Visual includes postures,fright” 1 , which causes trembling hands and voice, lacking

gestures, eye contacts, etc. 5 . Researchers also identify

eye contacts and interactions. One of the best ways to deal

communication as 55% Visual, 38% Vocal, and 7%

with stage fright is to practice more 2 . However, practice isVerbal. 5 It indicates that when we make speaking, thelimited to the space and time. Because of lacking of properwords we choose only have a small impact on theplaces and time to practice, people are seeking to find apresentation quality, the way we convey the message

solution using technology. Unable to simulate nervousness

matters much more. Especially the visual effect, forand presence, traditional platforms, such as computers andexample how we look when we say it , is a dominant

phones, failed to give a satisfying solution. With theelement for a good presentation.

ability to provide an immersion, virtual reality is therefore
being expected to solve this kind of problems 3 . Virtual
reality gives a solution, which works well to provide usersa presence. And at the same time, unlike

other 1ways,

2 ht t p s :/ / b lo g. b u ff er a p p . co m/ w ha t - ha p p ens - t o- ou r - br a ins - w h en- w e- ha v e- s t a g e- f r ig ht - t he- s ci en c e- of - p u b li c - s p ea k in g
3 ht t p :/ / w w w. vr p h ob ia . co m/ t h er a p y. ht m
4 ht t p :/ / s u cc es s f u l l y - s p ea k in g. c o m/ b lo g/ t h e - t hr ee - vs - o f - c o mmu n ica t io n

1 ht t p :/ / w w w. w eb md. c o m/ a nx i et y - p a ni c/ gu id e/ s t a g e- f r i g ht - p er f or ma nc e - a nx i et y# 2

5 ht t p :/ / w w w. ni c ko ls . u s /p r es ent a/ o ns . p df

In previous research, there existed many researchpossible in their work.

focusing on “Verbal”, using natural language processing to

In Tarid’s work [5]，they discussed several technologies

analyze the context of the speech, using image processing

in a forensic psychiatry context. They present a work in

to analyze the layout, color, and quantity of the contents

the development of a multimodal brain -computer interface

on the presentation materials [2]. Some of them are(BCI) combining neuroimaging and eye trac king with VR

focusing on “Vocal”. Researchers do audio analysis todevices. In their work, SensoMotric Instrument (SMI)categorize utterance duration, pitch, filled pause, and so

Eye-Tracking Glasses V2.0 is used to track the gaze in theon by using the input from headset microphone. They also

3D environment. The Volfoni Glasses are synchronizedbuild a real-time alarm system as a feedback to the user

with the VR System’s display using a Volfoni ActiveHub[3]. Due to the difficulty of collecting data, such as stage

IR100. They do not use any VR device, instead, ausage data, eye tracking data, there is a few research

modified version of Volfoni Active Eyes Glasses is set on

focusing on “Visual.” In [4], they integrate signals

top of the SMI ETG glasses to enable stereoscopic vision.

obtained from variety of sources, such as the gestures andIn Nikolaos’s work [6], they present a gaze-controlled

voice. Facial expressions are collected as videos by using

Multimedia User Interface and developed six applications

cameras. The data are analyzed using image processing.

including mail composing, multimedia viewing for VR

In our research, HTC Vive head -mounted headset isheadsets. They mentioned that now various companie s,

used. Lighthouse, which is HTC Vive’s tracking system, issuch as SensoMotoric Instruments, provide eye -tracking

used to collect position data. For eye tracking data, there

add-on to the Oculus Rift Development Kit 2 (DK2) HMD.

existed several ways: 1) using eye-tracker glasses, 2)

Withinusing eye-tracker for VR, and 3) estimation. However,embedded in a VR setup controlled via eye tracking can beglasses cannot be fit into VR, while eye-tracker (SMI) 1 fordesigned, implemented and evaluated. In their approach,

VR is very expensive. There is also one specialized VR,

they used twin-CCD binocular eye-tracker by Arrington

FOVE 2 , which is said to be the world ’s first eye tracking

Research to record the eye-tracking data.

VR headset. However, it is still under developing.

good 3D Characters that meet their needs, they developed

Therefore, we will collect eye-tracking data by using anARViPL Character Designer, which is an in-house Unity

estimation method, which we will introduce in Chapter 3.

3D plugin. Together with Singular Inversion FaceGen

In our research, we are mainly focusing on Visual. We
evaluate the presentation based on the usage of stage, andthis,immersive

User

Interfaceparadigms

And to model

SDK, it can generate full bodies.

eye contacts, and how well the presenter interact s with the2.2. Research on evaluating presentation
In [2], Kazutaka proposed an automatedaudience. To our best knowledge, our proposal is thequantitative evaluation of presentation materials. They

state-of-the-art approach to build presentation rehearsalevaluated by analyzing the quantity of the contents, fontapplication in VR.

size, and how well the font color matches with theIn the reminder of the paper, Section 2 describes thebackground color. The score of font size is given as:prior works related to virtual reality and evaluation of𝐻𝑚 𝑓𝑝 𝑙𝑚𝑖𝑛

good presentation. We present our approach in Section 3.

𝐻𝑝 𝑙𝑓𝑚𝑖𝑛And we conclude the paper and make a discussion on
future work in Section 4.

basic≥1(1)where 𝐻𝑚 is the Height/Length of the display, 𝐻𝑝 isthe Vertical resolution of the display, 𝑓𝑝 represents the2. RELATED WORK

number of pixel of the font, l represents the distance2.1. Research on Virtual Reality development

between audience and display, 𝑙𝑚𝑖𝑛 equals to 0.4m, andIn

jujunjun110’swork 3 ,theybuild

anA-Frame

𝐹𝑚𝑖𝑛 equals to 0.003m.

application, which is a web framework for building virtual

They provide two ways 2 to evaluate how well the fontreality experiences. It enables users to have highlycolor and background color matches. The first is toimmersion when viewing in VR headset. Space 3D Scan

compare the Contrast, and the formula is given as follows:

Camera and Omnidirectional Camera are used . They
managed to provide a good immersion. However, it only
enables the user to see from one fixed point. Also,interaction between environment and the user is not

0.299×𝑅𝑑 + 0.587×𝐺𝑑 + 0.114×𝐵𝑑 ≥ 125(2)

The other is to calculate the color-difference:
1 ht t p :/ / w w w. s miv is i o n. c o m/ en/ ga z e - a n d- ey e- t r a cki n g- s ys t ems / p r o du ct s / ey e - t r a cki n g- ht
c- v i v e. ht ml
2 ht t p s :/ / w w w. g et f ov e. c o m/
3 ht t p :/ / vr - la b . v oya g egr ou p . co mmax(𝑅1 ,𝑅2) – min(𝑅1 ,𝑅2 )+max(𝐺1 , 𝐺2 ) – min(𝐺1 , 𝐺2 )3.1. Device and Platform+max( 𝐵1 , 𝐵2 ) – min( 𝐵1 , 𝐵2 ) ≥500

(3)

In this research, our experiments are implemented onHTC

Vivehead-mounted

headset.

Resolution

isMathieu et al. [4] compared three feedback strategies

2160×1200. Eye sight range is 110°. Game Engine

to improve public speaking: a non -interactive virtual

Unity3D 5.5 is used to develop the application. We assume

audience, direct visual feedback, and nonverbal feedback

that the classroom, developed by AutoCAD, in VR asfrom

shown in Fig.1.

aninteractive

virtualaudience.

They

take

self-assessment questionnaires, expert assessments, eyecontacts, and avoidance of pause fillers into account whenevaluate the speaking. And they found that among thesethree conditions, interactive virtual audience works thebest in: (1) increasing engagement and challenges, (2)improving public speaking.
As

described

above,

thereexist

severalresearchworking on helping users improve their speaking based on“Verbal” and “Volume”, but there is only a few research on
evaluating “Visual”, eye contacts, interactions, and usageof the stage due to the difficulties of colleting such data.

Fig.1 Structure of Classroom in AutoCAD

On the other hand, the innovation of VR, which has theability to provide an immersion, has a great potential.

3.2. Interaction with Materials in VR

In our paper, we will present a novel approach and anapplication that could help users practice, evaluate, andimprovetheirpresentation

basedontheir“Visual”

performance.

3. METHODOLOGY
The final goal of this research is to build a virtual
reality application that can help users practice andimprove their presentation by giving them practical advice
based on their “Visual” performance.

Fig.2 Whole process of importing materials
Users’ presentation materials are imported into VR

The task of our research can be divided as the following

environment as shown in Fig.2. Firstly, we convert all thesteps: 1) Construct a place that meets normal presentationmaterials, such as slides, into images, then each image isroom standards, and build it into Virtual Reality, 2) insaved as one file. Then we attach the materials’ files to theorder to make it similar to real situations , how to importboard object. Secondly, we change materials of the board

user’s materials into Virtual environment and how to makeobject to realize the slide-change.

it to be possible to interact with, 3) since there is yet littleControllers, which are packaged in HTC Vive, are used

research on eye tracking in Virtual Reality, ho w toto imitate the functions of laser pointer. User can change

estimate eyes’ gazing data, 4) how to present an eye

slides in virtual environment by simply pushing down thetracking heat map, which is aimed to give the users atriggers on controllers. However, there has one limitation,reference and an advice on eye contacts performance

which is that animations cannot be used any more due toduring the presentation.

the conversion into pictures.

In this paper, we present our approach solving problems1) to 3), and we will continue our research on 4). The
limitation of this research comes from the inaccurate eye
tracking data, since inthis paper,

we ignore eye

movements, and we assume that the head orientation
equals to the eyes orientation.

Fig.3 Slide-Change in virtual environment

Fig.5 Output of collision positions

We could find that the left screen and the right screen
are not exactly same. It is due to the phenomenon called“parallax,” based on which our brains can perceive depth.

3.4. Interaction with audience
In this paper, we use balls to represent audience,temporary. Our goal is to examine whether the presenter

3.3. Estimation of Eye Tracking Data

could give enough eye contacts with each audience.

Position Data of user’s head can be easily collected byCurrently, when the cursor, which means his/her eye gaze,using lighthouse. For eye-tracking data, to simplify theand the ball collide, the ball will be changed to anotherestimation, we assume that the position of the screenmaterial to show him/her that the system detect yourcenter, which means the position of head, is close to theeye-gaze. This is for the situation for example when there

position of eyes, i.e., eye gaze.

is some student sitting in the corner sleeping, whether theThe following steps are adopted for eye -tracking.

professor could make a n otice or give a warn is a very1) Adding a cursor at the center of the screen in VR as

importantelement

when

evaluating

histeaching

shown in Fig.5 (could be transparent, blue here forperformance. In the future, we will change balls to human

explanation).

models, and change materials to animations, such as2) Collecting all the positions where the cursor and thesleeping, listening, and so on.

objects in VR collide, which is similar to the collision ofeye gaze and objects. For example, if the cursor (user ’seye gaze) collides with the square, which is one of theobjects, the position of the square is collected as shown in
Fig.4.
3) Refining the results by further dividing each object
into several parts, i.e., fine grain level. When the cursor(user’s eye gaze) collides with any part of an object, thistime instead of collecting the position of the whole objects,
now we only collect the position data we of the part being

Fig.6 Material change when collisions occur

watched. For example, as shown in Fig.5, we divide thesquare into three parts, and the user is looking at the left
side. Then we can obtain and collect the position of theleft side part,3.5. Eye Tracking Heat Map
To generate an eye-tracking heat-map, we are now
considering two approaches: 1) using a third-party heat4) Dividing the objects further into more parts until
units in Unity3D.

map, 2) changing the color and transparency of thetransparent blocks when collision occurs. The former may
require some additional work. While if adopting the latter,
we may need to make many more blocks but it could be apretty good real-time feedback.

Fig.4 Mirror screen from VR
As shown in Fig.4, if there is a collision of cursor (eyegaze) and block (object)

3.6. Evaluation of the presentation
We evaluate the whole presentation by analyzing theeye tracking data, whether the presenter gives enough eye
contacts to every corner of the room, and stage usage,
whether the presenter has made good use of the space.
However, we have not decided the weight of theparameters.

4. CONCLUSION and FUTURE WORKIn this paper, we

present a VR application forpresentation rehearsal and improvement. We manage toimportusers’

presentationmaterials

intovirtualenvironment,

andwealsomanagetodetect

theapproximate position that the user is looking at. In orderto improve the system, there are also some future works todo. We need to do more survey and try to implement 3D
eye-tracking heat map in Virtual Reality. We also need to
do some test and assign proper value of weight to eachparameterwhenevaluating

thepresentation.

Lastly,instead of using balls, we need to find or make 3D
audience models and animations.

REFERENCE[1] Mathieu Chollet, Torsten Wortwein, Louis -Philippe
Morency, Ari Shapiro, Stefan Scherer, Exploring
Feedback Strategies to Improve Public Speaking: An
Interactive
VirtualAudience&Framework,
UbiComp’15, 2015
[2] 栗 原 一 貴 , 加 藤 公 一 , 大 浦 弘 樹 , SlideChecker:

プ レ ゼ ン テ ー シ ョ ン 資 料 の 基 礎 的 な 定 量 的 自 動
評 価 手 法 , 2009[3] Kazutaka Kurihara, Masataka Goto, Jun Ogata,
Yosuke Matsusaka, Takeo Igarashi, Presentation
Sensei: A Presentation Training System using Speech
and Image Processing, ICMI’07[4] Mathieu Chollet, Torsten Wortwein, Louis -Philippe
Morency, Ari Shapiro, Stefan Scherer, Exploring
Feedback Strategies to Improve Public S peaking: An
Interactive
VirtualAudience
Framework,
UbiComp’15, 2015[5] Tarid Boukhalfi, Christian Joyal, Stephane Bouchard,Sarah Michelle Neveu, Patrice Renaud, Tools andTechniques for Real-time Data Acquisition andAnalysis in Brain Computer Interface studies usingqEEG and Eye Tracking in Virtual Reality
Environment, International Federation of AutomaticControl Hosting by Elsevier Ltd. 2015[6] Nikolaos Sidorakis, George Alex Koulieris, Katerina
Mania, Binocular Eye-Tracking for the Control of a
3D Immersive Multimedia User Interface , IEEE,
2015s as References. Instead, if you want to refersuch web contents, just put them at footnote.