# Development of Wall Amusements Utilizing Gesture Input

Takahisa Iwamoto, Atsushi Karino, Masayuki Hida, Atsushi Nishizaki, Tomoyuki Takami

► To cite this version:

Takahisa Iwamoto, Atsushi Karino, Masayuki Hida, Atsushi Nishizaki, Tomoyuki Takami. Development of Wall Amusements Utilizing Gesture Input. 9th International Conference on Entertainment Computing (ICEC), Sep 2010, Seoul, South Korea. pp.499-501, 10.1007/978-3-642-15399-0_72. hal-01055588

HAL Id: hal-01055588

https://inria.hal.science/hal-01055588

Submitted on 13 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Development of Wall Amusements
## Utilizing Gesture Input

Takahisa Iwamoto¹, Atsushi Karino¹, Masayuki Hida¹,
Atsushi Nishizaki¹ and Tomoyuki Takami¹

¹ Faculty of Information Science and Arts, Osaka Electro-Communication University,
1130-70 Kiyotaki, Shijonawate, Osaka, 575-0063 JAPAN
{w06018ti, w06039ak, w06094mh, w05098an}@dg.osakac.ac.jp, takami@isc.osakac.ac.jp

**Abstract.** Focusing on an infrared camera and a near-infrared radar, we have developed a system for new amusements which we can operate by gesture input toward a screen or moving our bodies in front of a screen projected on a wall. The infrared camera is used for operations performed by means of gestures, and the near-infrared radar is used for operations performed by larger movement of a human body by detecting the position of the person or the state of his feet near the floor surface. A screen is projected on a wall by an ultra-short throw projector. This system can be set up anywhere if there is a certain sized wall.

**Keywords:** Multi-touch, Digital game, Infrared camera, Infrared radar

## 1 Introduction

Studies of gesture input have been proceeding as a method of intuitive operations. Examples for the studies include touch panels, graphics tablets, Apple's Magic Mouse for computers and game operations by Wii remote control toward a TV screen. However, these examples of gesture input are basically used within a restricted range, and unsuitable for large-scale amusements such as physical amusements. Consequently, focusing on an infrared camera and a near-infrared radar, we have developed a system for new amusements which we can operate by gesture input toward a screen or moving our bodies in front of a screen projected on a wall.

## 2 Devices used for gesture input

An infrared camera and a near-infrared radar are used as devices for gesture input. They have advantages that they are easy to set up because of their compact sizes and they have sufficient processing capacities. The infrared camera is used for operations performed by means of gestures, and the near-infrared radar is used for operations performed by larger movement of a human body by detecting the position of the person or the state of his feet near the floor surface.
An infrared camera, OptiTrack FLEX V100 is used. The data of three-dimensional coordinates, sizes and areas of markers are obtained by irradiating infrared light from around a lens on the camera and taking the reflected infrared light from the surface of the markers. An image taken by the camera has only the markers and the other part of the image except the markers such as background and others becomes "transparent". Processing of the captured image is immediately performed and gesture input is immediately reflected in output image of an application. Image capture is performed 100 times per second, a view angle is 45-degree and the range of capture is 3 cm to 6 m when a 1-inch standard marker is used (figure 1).

The near-infrared radar rotates laser beams 36 times per second with a 0.36-degree pitch in fan-shaped flat space with radius 4 m and central angle 240-degree, and can acquire the data of directions and the distances to objects by detecting the reflected laser beams from the surface of the objects. The positions of the objects are calculated from these data (figure 1).

<figure><img src="image_2.png" /><figcaption>Fig. 1. An infrared camera and markers(left) and data detected by a near-infrared radar(right).</figcaption></figure>

## 3 System Structure

A screen is projected on a wall by an ultra-short throw projector. This system can be set up anywhere if there is a certain sized wall. Further, as it is assumed that an operator stands in front of a screen, there is a possibility that a body silhouette appears on the screen when normal projectors are used, however, if an ultra-short throw projector is used, the silhouette does not almost appear on the screen because the main body of the projector can be placed almost adhered to a wall.

The infrared camera is fixed with a tripod and installed in the position at the same height or a little higher than the operating position. The camera is connected to a personal computer with an USB cable and captured images are sent to a server using API. The server performs image processing, calculates the coordinates, height, width and area of each detected marker, lumps the data together and sends it to an application, and the data is utilized for the application. The near-infrared radar is set up on a floor and detects the standing position and other data of an operator. Figure 2 shows the system configuration.
<figure><img src="image_3.png" /><figcaption>Fig. 2. System configuration.</figcaption></figure>

The recognition of the position of a person standing on the floor is performed by placing the near-infrared radar parallel to the floor. The radar can recognize the shapes of the person as follows; if both feet are detected by the radar, the person is standing, if one foot is detected by the radar, the person is standing on one foot and if the radar does not detect anything, the person is jumping. Squat is recognized by placing two radars and sensing at different heights. This system can be available for several people at the same time.

## 4 An example of applications

A product example of applications using gesture input by an infrared camera is shown in Figure 3. This is an amusement in which cards displayed on a wall are moved by gestures. The card overlapped with a marker goes away at random and an image hidden behind these cards appear.

<figure><img src="image_4.png" /><figcaption>Fig. 3. A product example of an application using gesture input by an infrared camera.</figcaption></figure>