# e2-Mask: Design and Implementation of a Mask-Type Display to Support Face-to-Face Communication

Akino Umezawa, Yoshinari Takegawa, Keiji Hirtata

► To cite this version:

Akino Umezawa, Yoshinari Takegawa, Keiji Hirtata. e2-Mask: Design and Implementation of a Mask-Type Display to Support Face-to-Face Communication. 16th International Conference on Entertainment Computing (ICEC), Sep 2017, Tsukuba City, Japan. pp.88-93, 10.1007/978-3-319-66715-7_10 hal-01771294

HAL Id: hal-01771294

https://inria.hal.science/hal-01771294

Submitted on 19 Apr 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# e2-Mask: Design and Implementation of a Mask-type Display to Support Face-to-Face Communication

Akino Umezawa, Yoshinari Takegawa, and Keiji Hirtata

Dept. of System Information Science, Future University Hakodate, Hakodate, Hokkaido 041-8655, Japan
{p4417001,yoshi,hirata}@fun.ac.jp
http://hiratakelab.jp/

**Abstract.** In Face-to-Face communication, his or her face is an important part that decides the impression of a person because a person estimates the personality of the interlocutor from the features of the face of the interlocutor. Therefore, we made a hypothesis own impression and personality of giving to others by changing the own face freely. Then we proposed the e2-Mask that a mask-type display that synchronizes with facial expressions and substitutes an avatar for a human face.

**Keywords:** Face-to-Face Communication, Avatar, Cognitive Psychology, Display

## 1 Introduction

There is a relationship between people's visual appeal and the impression of their inner personality [1]. In particular, this relationship is strong in the initial stage of interpersonal relationships [2]. The psychologist Mehrabian mentioned that first impressions are formed from 7% language information, 38% auditory information and 55% visual information [3]. Also, Secord mentioned that people estimate a person's character from the characteristics of his or her face [4]. Therefore, the better the impression of a person's face, he or she can have a positive character such as being kind and friendly to interlocutor. Also, people change the impression of the face according to the situation, such as making up makeup or shaving a beard. In conventional research, there are some systems that change face virtually. Yamamoto et al. implemented a system to enable people to relax during public speaking by substituting the audience's faces with an avatar of a smiley pumpkin [5]. However, this system aims to change the face of the audience and cannot change the user's own face when the audience sees the user's face. Then we think that if we can change the face of the user seen by an unspecified number of others, we can operate the impression and atmosphere of the user more flexibly and strongly.

Therefore, we proposed the system that a mask-type display that substitutes a person's face with an avatar. We call the proposed system the e2-Mask. The
user's face is substituted with the avatar synchronously with the facial expres-
sion. Also, the e2-Mask can show the motion of the user's eyes. We think that
using the e2-Mask cannot only operate the impression of the user's face but
also support face-to-face communication. In a conversation with the user and
an interlocutor, the impression of the using's looks can be improved by using
the e2-Mask. Then interlocutor can assume that the user is positive character.
Therefore, the e2-Mask will have effects that relax and increase the number of
utterances. In addition, eye contact will be easier because the user talks through
the avatar.

## 2 Related research

In conventional research, there are some systems that change people's face virtually. Here we will introduce the three such systems. The first system is the ChameleonMask, used for telepresence [6]. The ChameleonMask shows the face of remote user on a display. In telepresence, an agent can act on behalf of a remote user by wearing the ChameleonMask. Therefore, the ChameleonMask gives a sense of presence to a conversation between a remote user and interlocutor. In contract, the e2-Mask shows the avatar on the display and is used in FTF communication. The second system is the Agencyglass, which is a system, in the form of sunglasses, that acts as a substitute for the user's eyes [7]. The Agencyglass a liquid crystal plate of the same size as the eye is placed on the lens of the sunglasses, then the motion of the user's eyes, recorded in advance, is shown on the liquid crystal display. The Agencyglass shows the prerecorded motion of the user's eyes. However, the e2-Mask shows an avatar synchronized with expressions in real time. The third system is a support system for presentations [5]. This system proposed a method of overlapping positive responses on audiences, and implemented a system which overlaps an image of a smiling pumpkin on each audience member using see-through HMD. This system is used in presentations. However, the e2-Mask is used in FTF communication such as a conversation by turns.

## 3 System structure

We design and implement a mask-type display that substitutes a person's face with an avatar<fnref n="1" />. FTF communication involves scenarios such as a daily conversation, a meeting and an interview. As a situation in which the e2-Mask can be used, there is an interview. For example, an interviewee is nervous by the atmosphere of interview and an interviewer who look craggy. Therefore, the interviewee cannot speak own opinion and understand interviewer 's questions. We think that the interviewer uses e2-Mask to substitute the face of the interviewer with an impressive avatar for the interviewee, so that the interviewee can ease tension and make it easy to talk. The e2-Mask is assumed for use in FTF communication as mentioned above. Therefore, we set up two design policies.

<footnote n="1">¹ https://drive.google.com/open?id=0B5YrczXNVwFVRkVSbDVMSl9QZHM</footnote>
(1) Transmissibility: The e2-Mask usage scenario is FTF communication such as a daily conversation, a meeting or an interview. Therefore, we need to design the mask so that the user and the interlocutor can visually recognize each other's gestures, figures and expressions. Especially, facial expression is an effective means to transmit and read feelings such as joy or surprise [8]. Thus the e2-Mask changes the expression of the avatar in real time based on the user's expression, to transmit the user's feeling to the interlocutor at once. In addition, by covering the entire head like a costume, the voice of the e2-Mask user and the voice of the interlocutor are not heard. Therefore, we should consider that each other's voice is transmitted to the other.

(2) Wearability: The e2-Mask is used in various situations, as described above. In improving the wearability, we should consider how to prevent the user's daily activity, such as shaking hands, holding luggage, moving up and down stairs and walking, from being impeded when wearing the e2-Mask. Thus it is important that the user can use the e2-Mask hands-free and that the e2-Mask is small and lightweight.

We implemented the prototype of the e2-Mask based on these two design policies. Figure 1 shows the appearance of someone wearing the e2-Mask. Figure 2 shows the structure of the system. The weight of the e2-Mask is 3.9 kg. Table 1 shows the manufacturer's name, specifications and number of the devices used. The reason for using this display is that the size fits people's faces and it is light. The reason for using this web camera is that it is small and it recognizes even if the distance between the lens and the face is short. The following is an explanation of the composition of the e2-Mask.

(1) The facial expression output unit: The e2-Mask uses the Facerig to substitute an avatar for a human face. The Facerig is software that can reflect a person's face recognized by the web camera onto an avatar and stream facial expressions to synchronize with the avatar (Figure 2). Also, it is possible to select the avatar according to the preference of the user and interlocutor because the Facerig has over the 55 kinds of avatar. The display shows picture of avatar that attached user's expression obtained from the facial expression recognition camera to the avatar by using the Facerig.

(2) The visual field acquisition unit: The e2-Mask uses the display and the web camera to secure the user's view. The display for securing visual field shows the image from the visual recognition camera.

(3) Each fixed part: Figure 4 shows each fixed part of the e2-Mask.

(a) The fixing of the facial expression output unit and the visual field acquisition unit: The facial expression output unit and the visual field acquisition unit were put into display cases (the upper left in Figure 4). After that, the two displays were combined back to back. Also, the lower part of the display case of the facial expression output unit was fixed the facial expression recognition camera. The upper part of the display case of the visual field acquisition unit was fixed the visual recognition camera.
(b) The fixing of arm: Between the facial expression output unit and the visual acquisition unit is fixed an acrylic board with a hole of a size through which the arm can pass. We fixed to the arm, the facial expression output unit and the visual acquisition unit by passing the arm through the hole and fixing it to the acrylic board (the upper center in Figure 4). Then the arm and the backpack were connected by fixing the arm and the backpack with metal fittings (the upper right in Figure 4).

(c) The fixed part of the mobile battery and the mini PC: An acrylic plate of sufficient size to fit the mobile battery and mini PC was fixed to the backpack. The mobile battery and mini PC were attached to the plate with double-sided tape (the lower in Figure 4).

<figure><img src="image_2.png" /><figcaption>Fig. 1. Wearing the prototype of the proposed system</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>Fig. 2. System configuration</figcaption></figure>
<table><caption>**Table 1.** Information on the devices used in the prototype of the proposed system</caption><thead><tr><th>Device</th><th>Manufacture name</th><th>Spec</th><th>Number</th></tr></thead><tbody><tr><td>Display</td><td>IGZO-LCD panel</td><td>7inch, 41g</td><td>2</td></tr><tr><td>The facial expression recognition camera</td><td>ELP-USB500W05G-L170</td><td>170 degree megapixel fisheye lens</td><td>1</td></tr><tr><td>The view recognition camera</td><td>ELP-USBFHD01M-L21-J</td><td>Wide angle full HD, 2.1 miri lens</td><td>1</td></tr><tr><td>The mobile battery</td><td>Poweradd 23000mAh mobile battery</td><td>Maximum output: DC 9/12/16/19/20V, 4.5A, USB1 5V/2.5A, USB2 5V1A</td><td>1</td></tr><tr><td>The mini PC</td><td>IntelBOXNUC6I5SYK</td><td>Number of USB port: 4 Speed of CPU: 2.8GHz</td><td>1</td></tr><tr><td>The display case</td><td>SHARP 7 inch IGO-LCD Acrylic panel kit</td><td>Only for IGZO-LCD panel, panel protection</td><td>2</td></tr><tr><td>The fixing arm</td><td>HANGNBTS</td><td>1.3 meter length</td><td>1</td></tr></tbody></table>

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Facial expression recognition</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Each fixed part of the system</figcaption></figure>
4 Conclusions

We proposed the e2-Mask that a mask-type display to support FTF communication. At this stage, we set up design policies and implemented the prototype of the e2-Mask. We are going to evaluation experiment.

References

1. K. Dion, E. Berscheid and E. Walster: What is beautiful is good. Journal of Personality and Social Psychology, 24, 285-290, 1972.

2. Fiske, S. T., Cox, M. G. Person concepts:The effects of targets familiarity and descriptive purpose on the process of describing others. Journal of Personality, 47, 136-161, 1979.

3. Mehrabian A. Nonverbal Betrayal of Feeling of Feeling. Journal of Experimental Research in Personality, 5, 64-73, 1971.

4. Secord, P. F. Facial features and inference processes in interpersonal perception. In R. Tagiuri L. Petrullo (Eds.), Person perception and interpersonal behavior. Stanford: Stanford University Press, 1958.

5. K. Yamamoto, K. Kassai, I. Kuramoto and Y. Tsujino. Presenter Supporting System with Visual-Overlapped Positive Response on Audiences. In Advances in Affective and Pleasurable Design, Springer International Publishing, pp.87-93, Florida, USA, 7 2016.

6. ChameleonMask: Embodied Physical and Social Telepresence using Human Surrogates. Kana Misawa Junchi Rekimoto. Retrieved April 8, 2018, from https://lab.rekimoto.org/projects/chameleonmask.

7. Mass Appeal -Hirotaka Osawa's Glasses Give a Fuck Even If You Don't (2014). Chris moore. Retrieved April 8, 2018 from https://massappeal.com/hirotaka-osawas-glasses-give-a-fuck-even-if-you-dont.

8. Ekman, P. Facial expressions of emotion; new findings, new questions. American Psychological Society, 3, 34-38, 1992.