Haptic Editor
Sho Kamuro ∗The University of Tokyo

Yuta Takeuchi †Keio University

Abstract
In this project, we propose an interactive content creation and editing system for haptic-enabled 3D content by drawing shapes in theair and copying and pasting surface textures. To achieve realistic
haptic interaction, we create a data structure for haptic content using three kinesthetic layers and a tactile layer. The user creates
haptic 3D models by drawing geometries through aerial sketching;
painting compliance and friction values on the layers; and copyingand pasting the vibrotactile surface textures of real objects onto thevirtual model. We present a user-friendly haptic interface to enable
these functions.
CR Categories:
H.1.2 [Information Systems]: Models andPrinciples—User/Machine Systems, Human Factors H.5.2 [UserInterface]: Haptic I/O—;
Keywords: haptic interaction, haptic interface, 3D modeling

1Introduction

A number of haptic interfaces have been proposed thus far. For
the popularity of haptic technologies to reach the next stage, a creation system for haptic-enabled content is required. At present, themarket offers a number of commercially available visual editors
for creating and editing images or 3D models, such as LightWave[NewTech, Inc.] and 3ds Max [Autodesk, Inc.]. Standard methods
for the creation of visual content allow the user to copy and paste
colors or visual textures from one place to another, as with the eyedropper tool in Photoshop [Adobe Systems, Inc.]. Based on image
textures obtained from real objects using cameras, content with realistic surface features can be achieved. I/O brush [Ryokai et al.
2004] enables copying and pasting of various visual images fromthe real world to the virtual world.

Kouta Minamizawa ‡Keio University

Susumu Tachi §Keio University

and some researchers have proposed devices for haptic scanning as
WHaT [Lang and Andrews 2011]. WHaT enables the user to obtaintextures by interactively scanning the surfaces of objects. However,
although these methods reveal fine-grained surface details, they do
not enable direct design or allow the user to test the sensations oftouching the content. For realistic representation of the texture ofvirtual surfaces, vibrotactile feedback is more effective than force
feedback because vibrations can be generated easily and rapidly
with simple actuators. In addition, vibrotactile feedback can beeasily integrated with force feedback as reported in [Nomura et al.
2010]; forces and vibrations are provided on the basis of the surface
properties of virtual objects calculated from photometric images ofreal objects. In [Minamizawa et al. 2012], Minamizawa et al. proposed a simple method of vibrotacitile feedback using a voice coilto generate vibrations and a microphone to record the vibrations of
haptic experiences.
In this project, we propose a data structure of haptic contents for realistic haptic interactions and an editing system for creating hapticenabled 3D content. As shown in Figure 1, the user can create
haptic 3D content by editing haptic properties on a layer editor andtouch the created content to test sensations using a pen-shaped haptic interface. To allow the user to design haptic content by using
various realistic haptic textures, we realize copying and pasting of
haptic textures from tactile materials in the real world to the surfaceof 3D content in the virtual world.

Haptic sensation represents integration of considerable amount ofsensory information, therefore, it is difficult to design haptic- enabled content. Although some research on 3D modeling systems using haptic interfaces has been conducted [Foskey et al.
2005][Fiorentino et al. 2006], these works do not focus on thecreation of models designed for haptic interactions. To constructrealistic haptic content, it is important to obtain detailed surface
textures of real objects. There are a numbers of studies on surface shape reconstruction, including [Johnson and Adelson 2009],∗ e-mail:kamuro@tachilab.org† e-mail:yuppon@tachilab.org‡ e-mail:kouta@tachilab.org

Figure 1: Concept Sketch of Haptic Editor.

§ e-mail:tachi@tachilab.org

2

Proposed Method

2.1

Structure of Haptic 3D Content

To generate realistic haptic experiences, we have focused on humanhaptic perception to create a structure for haptic-enabled content.
Human haptic sensations include tactile sensations of the skin andkinesthetic sensations produced at the joints of fingers and arms.
Although most conventional display and rendering methods for
Copyright is held by the author / owner(s).
SIGGRAPH Asia 2012, Singapore, November 28 – December 1, 2012.
ISBN 978-1-4503-1757-3/12/0011

haptic interactions use a unitary physical stimulus, multiple stimuli, each targeting tactile or kinesthetic sensation provide more effective and realistic sensations of touching virtual objects with richhaptic features.
Based on this hypothesis, we propose a data structure for hapticcontent, which is composed of four layers (see Figure 2). The geometry layer represents geometries of haptic content. The compliance layer and friction layer represent distributions of the values ofcompliance and friction respectively. These three layers are usedfor rendering kinesthetic feedback. In addition, the tactile layer
represents the distribution of vibrotactile textures, which is used for
rendering of tactile feedback.
Since compliance and friction are one-dimensional values on eachpoint of the surface, compliance layer and friction layer are treatedas gray scale images to describe distributions of each parameter.
On the other hand, to realize vibrotactile feedback based on tactile
layer, multiple vibrotactile textures are prepared and tactile layer
contains a distribution of IDs of the textures.

Figure 3: Flow of Creation of Haptic 3D Content.

To provide sensations of touching 3D haptic content, at first, aphysics simulator for 3D interactions detects a contact between aproxy and the geometry. Based on the detected contact point, values of compliance and friction are obtained by referring to layers.
These values are used for calculation of contact forces. The ID ofvibrotactile texture is also obtained by referring to tactile layer. The
vibration of texture is provided based on the speed of stroking toward the surface of the geometry. By changing the playing speedof vibration, uniform sensations of tactile textures are realized.

Figure 4: Ungrounded Pen-shaped Haptic Interface.

Figure 2: Proposed Data Structure of Haptic 3D Content.

2.2

Editor for Haptic 3D Content

The flow of haptic 3D content creation is shown in Figure 3. First,
the user draws the geometry of the content through aerial sketchingwith a pen-shaped interface. 3D shapes are generated on the basis of
our sketch-based shape creation method proposed in [Kamuro et al.
2011]. Then, the user paints haptic parameters on layers mapped tosurface meshes of geometries. Vibrotactile textures for the tactile
layer are obtained by scanning materials in the real world using thehaptic interface as a scanner. An audio microphone is attached at
the tip of the interface to record vibration sounds as vibrotactile textures [Minamizawa et al. 2012]. The user can design tactile textures
for their content easily by using this interactive scanning method.
During these processes, the user can check the content by touchingits display on a 3D monitor.

Figure 5: Vibrotactile Texture Obtained Through Scanning.

3System Construction

3.1

Interface for Cration and ExperienceWe constructed a simple and easy-to-use haptic interface (shown inFigure 4) for the creation and experience of haptic content by inte-grating the force-display method proposed in [Kamuro et al. 2009]
with vibrotactile feedback. The device is composed of two parts:a grip and a base. A DC motor built into the base generates thetranslation of the grip, which applies forces on the users fingers.
Moreover, a voice coil motor built in the grip provides vibrations tothe fingers. These components are used for kinesthetic and tactile
feedback, respectively. In addition, this haptic interface works as
a haptic scanner. In the tip of the grip, a microphone is attached
for interactive scanning of real objects. The user can scan targetobjects with the tip of the interface and obtain vibrotactile textures,as shown in Figure 5.

3.2

Figure 7: Example of Sketch-based Geometry Creation.

System Overview

Figure 6 shows the appearance of the constructed system. The
user wears 3D glasses for the 3D monitor and grasps the penshaped haptic interface. Positions and poses of the users head andthe interface are tracked by an optical tracking system (OptiTrack
V120: trio, NaturalPoint, Inc.). Sketching gestures are tracked andrecorded, and 3D geometries are generated in a simulation environment constructed on the basis of a physical simulation engine(PhysX, nViria Corp.), as shown in Figure 7.
A layer editor (shown in Figure 8) is displayed on the tablet monitor. Gray scale paintings are used to edit compliance and friction
layers that represent distributions of compliance and friction values.
To edit the tactile layer, the user paints IDs of vibrotactile texturesby palette painting. The user can scan tactile textures from real objects and the sccaned textures are integrated with images and used
as tactile palettes. The result of the painting is also shown on thesurfaces of the corresponding geometries displayed on the 3D monitor, as shown in Figure 9. Moreover, the user can check the createdhaptic content by touching the image and getting haptic feedback
with the interface. When the user touches the 3D content, the position of the contact between the content and the proxy in the simulation environment is sent to the layer editor. The layer editor extracts
the values of friction and compliance and the ID of the tactile texture at the corresponding point of each layer. Contact and frictional
forces are calculated on the basis of these values as well as the simulation, and they are generated by driving the motor of the hapticinterface. Textural vibrations are provided by the voice coil on thebasis of the scanned vibrotactile texture and the stroking velocity
toward the surfaces. These separated feedback methods result in
the realistic sensations of touching textured objects, produced in an
efficient manner.

Figure 8: Layer Editor for Haptic Content. (Top: friction layer,Bottom left: compliance layer, Bottom right: tactile layer)

Figure 9: Reflection of Editing on 3D Content. (Top: friction layer,Bottom left: compliance layer, Bottom right: tactile layer)
Figure 6: Constructed Haptic Editor System.

4Applications

References

Our simple editing system provides an efficient and intuitive development environment for designers of 3D haptic content. They
can design the desired content while editing and experiencing theeffects of such content using a simple haptic interface. Furthermore, the users can enjoy creating content with their own hands inthe comfort of their homes. Such user-generated-content will serve
as a foundation for the popularization of haptic technologies in thefuture.

F IORENTINO , M., M ONNO , G., AND U VA , A. 2006. Cad interface in virtual reality: Issues and solutions. In Proceedingsof the Workshop on Virtual Reality in Product Engineering andRobotics: Technology and Applications.

One possible example of the practical uses of the proposed system
is haptic-enabled online shopping. For this experience, designers
would prepare various types of textures that can be used for customizable products. Users, sitting in their homes, would be able
to select and test their favorite textures by touching the products
virtually via the haptic interface, as shown in Figure 10.

J OHNSON , M. K., AND A DELSON , E. H. 2009. Retrographic
sensing for the measurement of surface texture and shape. In
Computer Vision and Pattern Recognition (CVPR), 1070–1077.

F OSKEY, M., OTADUY, M. A., AND L IN , M. C. 2005. Artnova:
Touch-enabled 3d model design. In Proceedings of ACM SIGGRAPH 2005, Courses.

K AMURO , S., M INAMIZAWA , K., K AWAKAMI , N., AND TACHI ,S. 2009. Pen de touch. In ACM SIGGRAPH 2009 EmergingTechnologies.
K AMURO , S., M INAMIZAWA , K., K AWAKAMI , N., AND TACHI ,S. 2011. 3d haptic modeling system using ungrounded penshaped kinesthetic display. In Proceedings of IEEE Virtual Reality 2011.
L ANG , J., AND A NDREWS , S. 2011. Measurement-based modeling of contact forces and textures for haptic rendering. IEEETransactions on Visualization and Computer Graphics 17, 3.
M INAMIZAWA , K., K AKEHI , Y., NAKATANI , M., M IHARA , S.,
AND TACHI , S. 2012. Techtile toolkit. In Laval Virtual 2012
Revolution.
N OMURA , K., S AKAGUCHI , Y., Y IN , X., AND TANAKA , H.
2010. Estimation and rendering tactile feeling information
based on photometric images. In Proceedings of IEEE International Conference on Computational Science and Its Applications(ICCSA 2010).

Figure 10: Haptic-enabled Online Shopping.

5Conclusion

In this project, we propose a structure for haptic 3D content withmultiple layers for realistic haptic feedback. We constructed aninteractive editing system for creating and experiencing the proposed content by using an ungrounded pen-shaped haptic interface.
The system enables a user to interactively create haptic content bydrawing shapes in the air, painting haptic properties on surfaces,and copying and pasting textures from the real world to the virtualworld. Moreover, using the system, the user can touch and enjoy
their created 3D content as though it existed in the real world.

Acknowledgements
This project is supported by JST-CREST Haptic Media Project and
JSPS Fellowships.

RYOKAI , K., M ARTI , S., AND I SHII , H. 2004. I/o brush: Drawing
with everyday objects as ink. In Proceedings of CHI 2005.