# DecoTouch: Turning the Forehead as Input Surface for Head Mounted Display

Koki Yamashita, Yuta Sugiura, Takashi Kikuchi, Maki Sugimoto

► To cite this version:

Koki Yamashita, Yuta Sugiura, Takashi Kikuchi, Maki Sugimoto. DecoTouch: Turning the Forehead as Input Surface for Head Mounted Display. 16th International Conference on Entertainment Computing (ICEC), Sep 2017, Tsukuba City, Japan. pp.481-484, 10.1007/978-3-319-66715-7_68. hal-01771274

HAL Id: hal-01771274

https://inria.hal.science/hal-01771274

Submitted on 19 Apr 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# DecoTouch: Turning the Forehead as Input Surface for Head Mounted Display

Koki Yamashita*¹, Yuta Sugiura¹, Takashi Kikuchi¹, Maki Sugimoto¹

¹ Keio University, Kanagawa, Japan
zenki@imlab.ics.keio.ac.jp

**Abstract.** In this paper, we propose a new interaction technique for controlling applications by touching the forehead that the skin deformation is detected by an enhanced Head-Mounted Display(HMD). This technique envisions supporting commands and controls for applications as map applications and photo viewers without carrying additional controlling device which removes the user's eye from the display. In this technique, the skin deformation is measured by attaching light sensors to the HMD. The deformation of the skin is caused by the user touching the forehead with the finger. In this paper, we support the recognition of multiple gestures with a Support Vector Machine (SVM).

**Keywords:** HMD • Gesture • Recognition.

## 1 Introduction

Head-Mounted Displays (HMD allow users' to get visual information without restrict-
ing our body movement. HMD enables us to see the computer graphics both indoor and
outdoor which can support our everyday life, for example, showing the map when walk-
ing outside. Recently, applications that overlaps computer graphics to the actual view
are released for entertainment use which are suitable for HMD. Therefore, the need of
HMD is increasing.

As a fundamental function of the HMD, visual information is presented through a
transmissive display fixed in front of the user's sight. Manipulation of HMD is usually
performed by mobile devices when using it outdoor. However, when using mobile de-
vice, the user must look at the device, which removes the user's sight from the display.
Also, it is necessary to carry the HMD and the controlling device separately. Therefore,
new type of input method is required. There are input methods that attach sensors to the
back of the hands or arms use the surface of the skin as touch surface [1]. In this way,
user can input information without changing the gaze direction, but this does not solve
the problem of carrying additional device.

We propose a new interaction technique by touching the forehead, which enables user
to input information without removing the sight from the display In this technique, we
attach light sensors to the frame of the HMD and measure the shape deformation of the
skin caused by the user touching forehead with the finger (Fig.1). Using the acquired
sensor data, the directions of the touch on the forehead can be recognized. The sensors
are attached to the HMD itself; therefore, additional controlling device is unnecessary.
There are also research that utilize forehead skin deformation as an input [2], by moving
the muscles of the eyebrows. Our research differs from this research as our method uses
deformation of the skin caused by the touch of forehead with fingers.

<figure><img src="image_2.png" /><figcaption>Fig. 1. The deformation of the skin on the forehead.</figcaption></figure>

2 DecoTouch

2.1 Principle

In our studies, we measure the deformation of the skin with some light sensors on the
frame of the HMD. This light sensor, called Photo reflective sensor, consists of Infra-
red LED and phototransistor. This sensor is generally used for measuring the distance
from the sensor to the object. We attached the sensors to the frame of the HMD in front
of the forehead and measure the distance from the HMD to the forehead. When the user
touches the forehead, the deformation of the skin is occurred. As a result, the distance
from the sensor to the forehead is changed, and touch gesture can be recognized.

<figure><img src="image_3.png" /></figure>
<caption>**Fig. 2.** The principle of the DecoTouch</caption>

## 2.2 Implementation

### Hardware

For our recognition system, we attached some light sensors to the front frame of the MOVERIO BT-300 made by EPSON (Fig. 3). In order to get vertical and horizontal deformation of the forehead, we fixed light sensors over two rows. We adopted SG-105 of Codency as a light sensor. Light sensors is connected to the Micro Controller(Arduino Pro Mini, 3.3V), and sensor data is transmitted to the PC through XBee.

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** The appearance of the DecoTouch</figcaption></figure>

### Recognition

Our system performs gesture recognition using the acquired sensor data. We used Support Vector Machine (SVM) which is one of supervised machine learning as gesture recognition. In addition, PSVM: Support Vector Machine for Processing (PSVM) library was used for implementation[3]. This also outputs the result as a probability. Before gesture recognition, we prepare a gesture data set. Our system starts accumulating the learning data by recording the sensor data when the forehead is pulled to the direction of up, down, right or left, and also recording the sensor data when the forehead is not pulled. After collecting sensor data 20 times for each direction, it becomes possible to recognize the gesture that pulls the forehead up, down, left or right, and also the gesture of no touch.

### User Trial

We conducted a user study to evaluate the precision accuracy of our system. The participants were instructed to perform the following 5 gestures: 1. No touch, 2. Up pull, 3. Left pull, 4. Right pull, 5. Down pull. We gathered training data 20 times as sensor data for each state and performed ten-fold cross validation for gesture recognition. The participants included 5 men in their 20s. The average accuracy was 80 percent (standard deviation was 12.5 percent). In particular, there are a lot of misrecognition about the
gesture 1(No touch) and the gesture 2 (Up pull), and the gesture 1(No touch) and the
gesture 5(Down pull), which is considered to be caused by the lack in the number of
sensors. Adding another sensor row would increase accuracy of the recognition.

3 Conclusion

In this paper, we proposed a new input method for HMD by using forehead as a touch surface. The light sensors attached to the HMD measure the deformation of the forehead when touched with finger. In the user study, we recognized five directional gestures with 80% accuracy (standard deviation was 12.5 percent).

References

1. Masa Ogata, Yuta Sugiura, Yasutoshi Makino, Masahiko Inami, and Michita Imai. SenSkin: adapting skin as a soft interface. In Proceedings of the 26th annual ACM symposium on User interface software and technology. ACM, 539-544. (2013)

2. Hiromi Nakamura, Homei Miyashita. Control of augmented reality information volume by glabellar fader. In Proceedings of the 1st Augmented Human International Conference ACM, Article 20, 3 pages.(2010)

3. PSVM:Support Vector Machine for Processing, http://makematics.com/code/psvm/