Robots as Haptic and Locomotor Interfaces for the Blind.

To ensure independence, an assistive shopping device must provide the blind shopper with interfaces to the supermarket’s haptic and locomotor spaces. In the absence of vision, the egocentric and allocentric frames are known to align best in the haptic space. Robots can function as interfaces to the haptic and locomotor spaces to the extent that they can eliminate the need of frame alignment in the locomotor space and cue the shopper to the salient features of the environment sufficient for product retrieval. Research on spatial cognition and navigation of the visually impaired distinguishes two spatial categories: locomotor and haptic (Ungar 2000). The haptic space is defined as the immediate space around the individual that can be sensed by touch or limb motion without any bodily translation. The locomotor space is defined as a space whose exploration requires locomotion. In the absence of vision, the egocentric (self to object) and allocentric (object to object) frames of reference align best in the haptic space. As the haptic space translates with the body, lack of vision causes the frames to misalign in the locomotor space, which negatively affects action reliability. Giving the visually impaired equal access to everyday environments entails designing interfaces to the haptic and locomotor spaces of those environments that either eliminate the necessity of alignment or enable the visually impaired to align the frames when necessary. What environments should be targeted first? In cataloguing the most functionally difficult environments for the visually impaired, Passini and Proulx (Passini & Proulx 2000) top their list with shopping complexes. Grocery shopping is an activity that presents a barrier to independence for many visually impaired individuals who either do not go grocery shopping at all or depend on sighted guides, e.g., store staffers, spouses, and friends. Can robots function as effective interfaces to the haptic and locomotor spaces in the supermarket? We believe that they can. Neither guide dogs nor white canes can effectively interface to locomotor spaces, because they cannot help their users with macronavigation, which requires topological knowledge of the environment. Sighted guides do ensure the reliable maneuCopyright c © 2007, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. Figure 1: RoboCart. vering of the haptic space, but only at the expense of independence. Loss of independence translates into loss of privacy. Our experience with visually impaired individuals in our previous robot-assisted shopping experiments convinced us that many of them are not willing to use store staffers when shopping for personal hygiene items, medicine, and other products that require discretion (Kulyukin, Gharpure, & Nicholson 2005). A Trichotomous Spatial Ontology Spatial ontologies come about when we attempt to categorize space according to the ways we interact with it (Tversky et al. 1999). Freundschuh and Egenhofer (Freundschuh & Egenhofer 1997) give a comprehensive review of previous work on categorization of space and distinguish six categories based on manipulability, locomotion, and size and use their ontology to describe previous ontologies of space in the geography literature. We contribute to this line of research a trichotomous ontology of space in a supermarket. This trichotomy is an extension of the dichotomous ontology (haptic vs. locomotor) currently used by many researchers on spatial cognition of the visually impaired. Our trichotomy is certainly incomplete. We developed it solely for the purposes of describing the interactions between blind shoppers and RoboCart, our robotic shopping assistant shown in Figure 1. RoboCart assists the shopper in two stages. It first guides the shopper into the vicinity of the desired product and then instructs the shopper on how to maneuver the haptic space within that vicinity. In the first stage, RoboCart interfaces the shopper to the locomotor space, guiding the shopper to the required aisle section and interacting with other shoppers by asking them to yield the way when a passage is blocked. In the second stage, RoboCart cues the shopper to some salient features of the environment near the haptic interface through voice instructions grounded in the shopper’s egocentric frame of reference. The design of RoboCart reflects this dual interface functionality insomuch as the device consists of twomodules: locomotor and haptic. The locomotor module consists of a Pioneer 2DX mobile robotic base from ActivMedia, Inc. upon which a wayfinding toolkit is fitted in a PVC pipe structure. A shopping basket is mounted upon the PCV structure, as shown in Figure 1. The PVC structure includes a firm handle at the lower abdomen level. A 10-key numeric keypad is attached to the handle for giving instructions to RoboCart. Using the numeric keypad, the shopper can either browse through a list of products or enter a product number. Once the shopper confirms the selection, RoboCart guides the shopper to the vicinity of the product. The haptic module consists of a wireless omni-directional barcode reader. The reader is ergonomically modified to help the shopper align the reader with the shelf. After RoboCart brings the shopper in the vicinity of the product, RoboCart uses the shopper’s egocentric frame of reference to instruct the shopper through synthetic speech on how to find the product, e.g. Honey Nut Cheerios is on the top shelf to your right. The shopper finds the shelf and uses the barcode to scan the barcodes on that shelf. The product name of each scanned barcode is read to the shopper. As the robot moves, the rigid PVC handle gives the shopper haptic feedback about the robot’s maneuvers. To describe how the visually impaired shopper interacts with the supermarket space using RoboCart, we introduce the category of target space. The target space is the shoppercentric subspace of the locomotor space in which the shopper perceives the target product to be. The target space is always defined with respect to a specific target product. Haptic cues in the target space act as external reference points during the shopper’s maneuvering of the haptic space in the target space until the haptic space contains the product.