Z-touch: A multi-touch system for detecting spatial gestures near the tabletop
Yoshiki Takeoka
The University of Tokyo
yoshikinoko@gmail.com

Takashi Miyaki
The University of Tokyo

Jun Rekimoto
The University of Tokyo
Sony Computer Science Laboratory

Figure 1: These two pictures have been captured simultaneously. (a): Tabletop of Z-touch. Multilayered line laser modules are placed at theedge of the tabletop panel. (b): Captured depth map of the hand near the top panel of Z-touch shown in (a). RGB channels correspond to theLaser Plane: R-Highest, G-Middle, B-Lowest (very close to the surface of the tabletop). The image at the bottom of (b) is that of the thumb
not touching to the surface, so the silhouette is in a pale red color.

Abstract
We introduce Z-touch, a multi-touch table that can detect spatial
hand gestures near the tabletop surface. This system uses a combination of a multilayered infrared (IR) laser plane and a high-speed
camera. Multilayered IR line laser modules are synchronized withthe shutter signal of the high-speed camera. This system allows
user interaction on the basis of the posture of the user’s fingers nearthe tabletop surface without the use of special markers or deviceson the fingers.
Keywords: multi-touch, hand gesture interaction
Figure 2: System architecture.

1

Introduction

Since multi-touch systems detect multiple contact points on a tabletop surface, user interaction is restricted to a 2D planar surface.
However, users often forget this restriction and interact with thesystem using 3D gestures near the tabletop. When carrying out
physical interactions such as manipulating some objects, we simultaneously control multiple points and continuous parameters. Thisis because the most of the parameters are dependent on “how thefingers/hands touch to the objects.” In the same context, we believethat it is very important to take into account the interaction using
the postures of fingers when designing multi-touch systems. We
may use finger posture in applications such as those for controlling
3DCG objects and effects of music. We introduce Z-touch, a novel
multi-touch table that is capable of detecting posture and position ofmultiple objects (fingers) near the tabletop. Users can interact withthe contents displayed on Z-touch by using their bare hands without wearing special markers. [Benko and Wilson 2009] introduced
the concept of 3D hover gesture interaction in front of the projection screen surface. They used a depth sensing camera to detect 3Dinteraction. However, we use a multilayered laser plane synchronized with the shutter signal of a high-speed camera to obtain aclear image of the depth map near the surface (Figure1(b)).

controlled by synchronization with the shutter signal of high-speed
camera. The IR laser is reflected by the objects in the middle ofthe top panel and is captured by the high-speed camera embedded
under the surface (Figure1(b)). We used Pointgrey Grasshopper
which can capture 8bit gray scale VGA image at 200fps. Eight setsof triple line laser modules are placed at center of edge and vertexof the top panel (Figure1(a)). The IR laser plane is calibrated along
the distance between each laser plane (11.5mm) and is parallel tothe top panel. The lowest laser plane is very close to the surfaceof the top panel. The top panel (control area) is 420 mm squareglass panel having a thickness of 6mm. This panel is covered by aprojection sheet, the DILAD screen sheet, which only diffuses visible light from the rear projector to the surface of the top panel butallows reflected IR light to pass through.

2References

System DesignFigure 2 shows the system architecture of Z-touch. The key components of Z-touch are multilayered IR line laser modules and ahigh-speed camera. The appearance of each of IR laser plane is3Conclusion

We introduced Z-touch, that combines multi-touch and 3D hand
gesture detection near the surface of a tabletop, using multilayered
IR line lasers synchronized with the shutter signal of a high-speedcamera.

B ENKO , H., AND W ILSON , A. D. 2009. Depthtouch: Using depthsensing camera to enable freehand interactions on and above theinteractive surface. Tech.Report, MSR–TR–2009–23.