# Smile Like Hollywood Star: Face Components as Game Input

Jaewook Jung, Hanbit Park, Shin Kang, Minsoo Hahn

► To cite this version:

Jaewook Jung, Hanbit Park, Shin Kang, Minsoo Hahn. Smile Like Hollywood Star: Face Components as Game Input. 9th International Conference on Entertainment Computing (ICEC), Sep 2010, Seoul, South Korea. pp.487-489, 10.1007/978-3-642-15399-0_68. hal-01055593

HAL Id: hal-01055593

https://inria.hal.science/hal-01055593

Submitted on 13 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Smile like Hollywood Star: Face Components as Game Input

Jaewook Jung, Hanbit Park, Shin Kang and Minsoo Hahn

Digital Media Laboratory, KAIST,
517-10 Dogok-dong, Gangnam-gu, Seoul 135-854, S. Korea
{jaewookjung, han_bit, kangshin12, mshahn2}@kaist.ac.kr

**Abstract.** Most of the commercialized games have used controller by manipulated hands or feets. Recently, some studies tried to facial expressions or emotion as game input, but it didn't directly manipulate a game. It adjusted the difficulty of game. In this study, we suggest a new type of game input interface using face components as input, directly, and present one offline game and two online games by using this method.

**Keywords:** Game interface, Face components, Smile.

## 1 Introduction

Many offline games use joystick as game input and most of the online games use a keyboard and a mouse as game input. Some researches tried to adopt facial expression as game input indirectly [1-3]. Those studies measured the emotional state from the facial expressions and adjusted the level of difficulty in game. It is not the direct input for game.

In this study, we present the face component recognition (FCR) interface. In addition to the interface, we suggest one offline game, *Smile like Hollywood Star*, controlled by FCR, and apply FCR to two online games, Tetris and Kart-Rider.

## 2 Face Component Recognition Interface

FCR interface consists of one webcam, one normal PC, and one keyboard signal generation board. Most of the online games require the keyboard signal in hardware level because of security. Therefore, we implemented the signal generation board to occur the real key-code signal in Figure 1.

The working process of FCR interface has five steps. At first, it detects the face region by Haarcascade method from the captured frames. At second, it detects face components, eyes and mouth. At third, it sets up the face components as the region of interest (ROI). At fourth, it tracks the ROI and recognizes the input states, the smile, the Ho, the wink with right eye, and the wink with left eye in Figure 2. At fifth, PC sends the recognized input state to the keyboard signal generation board by RS-232C.
Finally, the board enters the keyboard signal according to the received command into
PC.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Keyboard Signal Generation Board</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>Fig. 2. Face Components as game input
(a) Smile, (b) Ho, (c) Wink with left eye, and (d) Wink with right eye.</figcaption></figure>

3 Game Description

We have suggested one offline game, *Smile like a Hollywood Star*. It uses face
components as game input, directly. Figure 3-(a) shows the screen shot of *Smile like
Hollywood Star*. The game is very simple and intuitive. Hollywood star's photo is
displayed and a user mimics the face expression of the displayed photo like the
Hollywood star. When the feature points, face components, are similar between the
player and the star, the player gets the game score. The game score is proportional to
the completion time.

Besides offline game, we have applied FCR interface to two commercialized
online games in Korea. One is Tetris [4] in Figure 3-(b) and the other is Kart-Rider
[5] in Figure 3-(c). In Tetris, the smile rotates a block, the Ho fast drops a block, the
wink with right eye moves a block to the right, and the wink with left eye moves a
block to the left. In Kart-Rider, the smile drives the Kart to forward, the Ho drives the
Kart to backward, the wink with right eye turns right the Kart, and the wink with left
eye turns left the Kart.
<figure><img src="image_4.png" /><figcaption>Fig. 3. Games with Face Components Recognition (FCR) Interface
(a) *Smile like Hollywood star* (b) Kart-Rider serviced by NEXON, and (c) Tetris by NHN</figcaption></figure>

3 Conclusion

In this study, we suggest new game interface that uses face components as game input,
directly. By adopting the suggested FCR interface, we have presented one new types
of game, *Smile like Hollywood Star*, and FCR interface has applied to two online
games.

FCR interface and three games have three purposes. One is that the users have
more natural smiling face. Usually, the East Asian is impassive than the Westerner,
because they feel awkward to express an emotion [6]. By iteratively playing the game
with FCR interface, we expect that the user feels more comfortable to reveal an
emotion and have dynamic facial expression including a smiling face. Another
purpose is reducing the level of stress and promoting a positive emotion by smile,
even though it is artificially smile. The other is the game input style for mobility
impaired people and hand impaired people.

References

1. Bernhaupt, R., Boldt, A., Mirlacher, T., Wilfinger, D., Tscheligi, M.: Using emotion in games: emotional flowers. In: Proceedings of the international conference on Advances in computer entertainment technology, pp. 41--48, ACM(2007)

2. Lankes, M., Riegler, S., Weiss, A., Mirlacher, T., Pirker, M., Tscheligi, M.: Facial expressions as game input with different emotional feedback conditions. In: Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology, pp. 253--256, ACM (2008)

3. Obaid, M., Han, C., Billinghurst, M.: Feed the Fish: an affect-aware game. In: Proceedings of the 5th Australasian Conference on Interactive Entertainment, ACM(2008).

4. HanGame Tetris: http://tetris.hangame.com

5. Nexon KartRider: http://kart.nexon.com/kart/page/Main.aspx?URL=Home/index

6. Kayan, S., Fussell, S. R., Setlock, L. D.: Cultural differences in the use of instant messaging in Asia and North America. In: Proceedings of the 20th anniversary conference on Computer supported cooperative work, pp.525--528, ACM (2006)