HaptoMapping (EuroHaptics 2020, SIGGRAPH ASIA Etech 2020, IEEE TVCG 2021)
Channel: SENSlab
Date: 2022-01-28

Miyatake, Yamato, Takefumi Hiraki, Daisuke Iwai, and Kosuke Sato. "HaptoMapping: Visuo-Haptic Augmented Reality by Embedding User-Imperceptible Tactile Display Control Signals in a Projected Image." IEEE Transactions on Visualization and Computer Graphics (2021).

Abstract:
This paper proposes HaptoMapping, a projection-based visuo-haptic augmented reality (VHAR) system, that can render visual and haptic content independently and present consistent visuo-haptic sensations on physical surfaces. HaptoMapping controls wearable haptic displays by embedded control signals that are imperceptible to the user in projected images using a pixel-level visible light communication technique. The prototype system is comprised of a high-speed projector and three types of haptic devicesfinger worn, stylus, and arm mounted. The finger-worn and stylus devices present vibrotactile sensations to a users fingertips. The arm-mounted device presents stroking sensations on a users forearm using arrayed actuators with a synchronized hand projection mapping. We identified that the developed systems maximum latency of haptic from visual sensations was 93.4 ms. We conducted user studies on the latency perception of our VHAR system. The results revealed that the developed haptic devices can present haptic sensations without user-perceivable latencies, and the visual-haptic latency tolerance of our VHAR system was 100, 159, 500 ms for the finger-worn, stylus, and arm-mounted devices, respectively. Another user study with the arm-mounted device discovered that the visuo-haptic stroking system maintained both continuity and pleasantness when the spacing between each substrate was relatively sparse, such as 20 mm, and significantly improved both the continuity and pleasantness at 80 and 150 mm/s when compared to the haptic only stroking system. Lastly, we introduced four potential applications in daily scenes. Our system methodology allows for a wide range of VHAR application design without concern for latency and misalignment effects.