# Emotional Expression of Korean Dance Assisted by a Virtual Environment System

Taeyoung Uhm, Hanhoon Park, Mi-Hee Lee, Un-Mi Kim, Jong-Il Park

► To cite this version:

Taeyoung Uhm, Hanhoon Park, Mi-Hee Lee, Un-Mi Kim, Jong-Il Park. Emotional Expression of Korean Dance Assisted by a Virtual Environment System. Second IFIP TC 14 Entertainment Computing Symposium (ECS) / Held as Part of World Computer Congress (WCC), Sep 2010, Brisbane, Australia. pp.208-213, 10.1007/978-3-642-15214-6_21. hal-01056340

HAL Id: hal-01056340

https://inria.hal.science/hal-01056340

Submitted on 18 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Emotional Expression of Korean Dance Assisted by a Virtual Environment System

Taeyoung Uhm<fnref n="1" />, Hanhoon Park<fnref n="2" />, Mi-Hee Lee<fnref n="3" />, Un-Mi Kim<fnref n="3" />, and Jong-Il Park<fnref n="1" />,

**Abstract.** Korean dance, which is one of the traditional Korean performing arts, has been a common way of expressing Korean peculiar emotion including "han", "heung", and "mut". This emotional presentation discriminates Korean dance from Western dance. In this paper, we try to understand what "han" is and assist the emotional expression of a representative Korean dance "salpuri" with empathy for "han" using a virtual environment system. Qualitative and quantitative analysis results demonstrate that "han" is related to being more dynamic and expressive and better empathy and expression of a dancer can be expected by using the virtual environment system.

**Keywords:** Virtual environment system, Korean dance, emotional expression assistance, qualitative and quantitative analysis

## 1 Introduction

The traditional Korean arts involve Korean peculiar and implicit emotions and characteristics. In particular, Korean dance is the most typical way for expressing these emotions and characteristics. However, it has been directly passed down only through personal communication among people. Therefore, there has been a difficulty in training and expressing Korean dance with empathy for the emotions and characteristics that cannot be explained by words. Therefore, it would be very helpful if a system of assisting to express Korean dance visually and aurally and analyzing the expression is provided. Thus, in this paper, we construct a virtual environment system that:

- is a three-planes CAVE-like projection-based system and provides visual and aural information, i.e. video and music related to Korean dance, thus enhances strongly the sense of immersion of the performer.

- includes a motion analysis system that was devised for analyzing and recognizing emotional performance.

and demonstrate its effectiveness through qualitative and quantitative analysis.

<footnote n="1">¹ Mixed Reality Laboratory, Department of Electronics and Computer Engineering, Hanyang University, Seoul, Korea</footnote>

<footnote n="2">² NHK Science & Technology Research Laboratories, Tokyo, Japan</footnote>

<footnote n="3">³Research Institute of Korean Traditional Dance, Department of Dance, Hanyang University, Seoul, Korea</footnote>

<footnote>{uty02, hanuni}@mr.hanyang.ac.kr, {A029019, kimunmi, jipark}@hanyang.ac.kr</footnote>
## 1.1 Related Work

Research for training using virtual environment has been important for helping boost motivation to exercise and train sports skill and has been mainly focused on motion analysis from user's body parts [2, 3]. In the medical field, Pieper et al. [4] and Maekawa et al. [5] made it possible for user to train medical surgery and treatment by using virtual environment system.

However, existing systems were only for helping users train physical skills using virtual environment. To the best of our knowledge, there has been no virtual environment system that assists to train emotional expression, in particular from Korean traditional dance performance.

# 2 Korean Peculiar Emotions and Korean Dance "salpuri"

In fact, it is not easy to define Korean emotions with several specific words because they are complicated, delicate, mixed, and so on. However, Korean peculiar emotions which are expressed by Korean dance are usually categorized into three classes ("han": close to understated sorrow, "heung": close to excitement, "mut": close to spruce-up). In this paper, we deal with "han" which is the most representative Korean peculiar emotion, which is started as sorrowful regret and ended in the excess of mirth by Korean dance performance.

Korean dance is roughly formed A-B type which is changed from static A to dynamic B, or A-B-A' type which is ended another static A' for becoming light-hearted and is suitable for expressing "han". The "salpuri" that is originally aimed at exorcism is a typical Korean dance performance of A-B-A' type. Therefore, we use "salpuri" to understand "han".

<figure><img src="image_2.png" /><figcaption>Fig. 1. Our virtual environment system.</figcaption></figure>

# 3 Virtual Environment System

A virtual environment system was created as shown in Fig. 1. The system has three sides (rear screens) where video images are projected by three rear projectors, and it is surrounded by speakers and lighting lamps, and thus can provide an immersive virtual
environment. A single camera is used for calibrating the system and capturing the scene. The captured images are used for the motion analysis which will be explained in Section 3.2. The projectors and camera were synchronized with the v-sync signal of video output from a laptop. The synchronization is required for user region extraction which will be explained in Section 3.1. The geometric calibration and radiometric calibration [1] were done once in an initial step.

## 3.1 User Region Extraction

In our environment system, a single camera image (C) consists of two regions: one is a projection region ($R_O$) and the other is a non-projection region ($R_X$). And, each region consists of a foreground region (a part of user region) and a background region as follows.

$$C = R_O + R_X + N \quad (1)$$

where $R_O = F_O + B_O$, $R_X = F_X + B_X$.

Here, F, B, N indicate the foreground, the background, and the environmental noise, respectively. This is also illustrated in Fig. 2. The separation between $R_O$ and $R_X$ can be done by the geometric calibration [6] that informs us where the projected background is in the camera images. From the two regions, user region is extracted precisely and in real-time by a cooperative method [7]. Figure 3 shows the results of extracting the real dancer's region in our virtual environment system.

<figure><img src="image_3.png" /><figcaption>Fig. 2. User region extraction.</figcaption></figure>

## 3.2 Motion Analysis

After extracting user region, we extract low-level features, that describe the 2D motion of a rectangle circumscribing the user region in 3D space which consists of space-, time-, and energy-axis based on Laban's theory, from the user region and outliers are eliminated using a spatial normalization and clipping method [8]. By comparing the distribution of the low-level features extracted from different dance performances, we can recognize how different the dance performances are.
<figure><img src="image_4.png" /><figcaption>Fig. 3. Results of extracting user region using a cooperative method [7].</figcaption></figure>

# 4 Experimental Results

We exploited the virtual environment system to analyze the characteristics of Korean dance. Its effectiveness in learning Korean dance is also tested. Six students, who have never learned any Korean dance performance, learned the "salpuri" intensively in a short period. When their learning curve was almost saturated, they performed the "salpuri" in the system twice as shown in Fig. 4. At the first trial, the virtual environment system was turned off. At the second trial, the system was turned on.

Background music for "salpuri" and "janggu" (double-headed drum with a narrow waist in the middle) beats were played and performed to provide rhythmical tune during performance. For the second trial, the real sounds such as the lapping of the waves or the rustle of leaves were provided additionally through the virtual environment system.

## 4.1 Qualitative Analysis

We deeply interviewed the students on the spot and had a general discussion about the validity of the virtual environment system. Through a hermeneutical approach, we could know that the virtual environment system had two important positive effects:

1. With the virtual environment system, the students were easily and more absorbed in an emotional expression.

2. The application of virtual environment system helped the students express well "han" inside them with empathy and finally clear up the sadness.
<figure><img src="image_5.png" /><figcaption>Fig. 4. Camera images of capturing "salpuri" performed by six students.</figcaption></figure>

## 4.2 Quantitative Analysis

We analyzed the difference between spatial distributions of low level features extracted from two sets of performances performed by each student. Figure 5 shows a part of results that is the distribution of activity, velocity, and acceleration in the horizontal and vertical movement of their performing region. In the result, their second performance with our system on was more dynamic and expressive than the first performance. We also asked several dance experts to evaluate the performances and found that the ones with the virtual environment system obtained better scores. Therefore, we arrive at the conclusion that expressing "han", which includes both sorrow and mirth, with empathy is related to being more dynamic and expressive and the virtual environment system is useful for dance training.

## 5 Conclusion

In this paper, we tried to understand the Korean peculiar emotion "han" and assisted to express well it by performing the Korean traditional dance "salpuri" in a virtual environment system. From the qualitative and quantitative analysis results, we could know that "han" was related to being more dynamic and expressive and the virtual environment system was useful. In the future, we will focus on more detailed quantitative analysis of the emotional characteristics of "salpuri". For this research, we currently consider to use bio-signal sensors and motion sensors.

**Acknowledgments.** This research is supported by Ministry of Culture, Sports and Tourism(MICST) and Korea Creative Content Agency(KOCCA) in the Culture Technology(CT) Research & Development Program (2010).
<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Example of showing the distribution difference of low-level features between different emotional performances.</figcaption></figure>

References

1. Park, H., Lee, M.-H., Kim, S.-J., Park, J.-I.: Surface-Independent Direct-Projected Augmented Reality. In: ACCV 2006, pp. 892--901. (2006)

2. IJsselsteijn, W., de Kort, Y., Westerink, J., de Jager, M., Bonants, R.: Fun and Sports: Enhancing the Home Fitness Experience. In: ICEC 2004, vol. 3166, pp. 46--56. (2004)

3. Tsuji, T., Sumida, Y., Kaneko, M., Sadao, K.: A Virtual Sports System for Skill Training. Journal of Robotics and Mechatronics, 13, 2, 168--175 (2001)

4. Pieper, S.D., Delp, S., Rosen, J., Fisher, S.S.: Virtual Environment System for Simulation of Leg Surgery. In: SPIE, vol. 1457, pp. 188. (1991)

5. Maekawa, Y., Ishiguro, K., Yasuo, O.: Simplified Apparatus for the Abdominal Breathing Exercise. The Journal of Japanese Physical Therapy Association, 22, 7, 413--416 (1995)

6. Pingali, G., Hampapur, A.: Geometric Image Masking for Segmenting Dynamic Projected Imagery. In: International Conference on Image Processing. (2003)

7. Uhm, T. Park, H., Lee, M.-H., Park, J.-I.: User Region Extraction from Dynamic Projected Background for Virtual Environment System. In: The 2nd International Conference on Digital Image Processing, pp. 75462I-1--6. (2010)

8. Uhm, T., Park, H., Park, J.-I., Kim, U.-M.: Quantitative Analysis of Korean Dance Performance Based on a Vision-Based Emotion Recognition System. In: International Symposium on Ubiquitous VR, pp. 113--114. (2006)