# Pointing Devices for Wearable Computers

Authors: Andrés A. Calvo; Saverio Perugini
Journal: Advances in Human-Computer Interaction
 (2014)

---

## Abstract

We present a survey of pointing devices for wearable computers, which are body-mounted devices that users can access at any time. Since traditional pointing devices (i.e., mouse, touchpad, and trackpoint) were designed to be used on a steady and flat surface they are inappropriate for wearable computers. Just as the advent of laptops resulted in the development of the touchpad and trackpoint, the emergence of wearable computers is leading to the development of pointing devices designed for them. However, unlike laptops, since wearable computers are operated from different body positions under different environmental conditions for different uses, researchers have developed a variety of innovative pointing devices for wearable computers characterized by their sensing mechanism, control mechanism, and form factor. We survey a representative set of pointing devices for wearable computers using an “adaptation of traditional devices” versus “new devices” dichotomy and study devices according to their control and sensing mechanisms and form factor. The objective of this paper is to showcase a variety of pointing devices developed for wearable computers and bring structure to the design space for wearable pointing devices. We conclude that a de
facto pointing device for wearable computers, unlike laptops, is not likely to emerge.


---

## Body

## 1. Introduction

Aninput device allows a user to interact with a computer by converting human motor responses into signals a computer processes. Apointing device is a type of input devices that allows a user to interact with a computer by moving a cursor on a monitor to select icons and trigger desired actions. The de facto pointing device for desktop computers is a mouse, which was invented by Douglas England in 1967 [1]. To manipulate the cursor, a user moves a mouse on a flat surface; the cursor’s motion is proportional to the relative change in position of the mouse. A similar pointing device designed for desktop computers is atrackball—an upside-down mouse that allows a user to control the cursor by rotating the ball in the direction of desired cursor motion. Unlike most hardware devices, the mouse has endured for several decades [2]. However, a mouse is ineffective if a user is unable to operate it on a flat and steady surface. Thus, the emergence of an increase in the use of laptops resulted in the development of touchpads and isometric joysticks.Atouchpad is a small rectangular pad that responds to touch and is found on a laptop; moving a finger across a touchpad moves the cursor in the same direction [2]. Anisometric joystick senses the force vector applied to it to determine which direction and how fast the cursor has been moved [2]. It is mounted on the center of a laptop computer’s keyboard and is referred to generally as atrackpoint. A touchpad and/or trackpoint is embedded into the chassis of a laptop allowing a user to operate each while viewing the laptop’s monitor. A trackpoint is especially suitable for small laptops since it has a tiny footprint (i.e., it is much smaller than a touchpad) [1]. The touchpad and trackpoint are the de facto pointing devices for laptops.Awearable computer is a body-mounted device that a user can access at any time. The adjective “wearable” refers to “the use of the human body as a support environment for the product” [3]. The portability of wearable computers allows them to be used not only pervasively, but also while engaged in other activities (e.g., in the battlefield or operating room) [4]. Since traditional pointing devices (i.e., mouse, touchpad, trackpoint, and trackball) were designed to be used with desktop and laptop computers, which were not intended to be used in concurrence with other activities, they are inappropriate for wearable computers. Just as the advent of laptops resulted in the development of the touchpad and trackpoint, the emergence of wearable computers is leading to the development of pointing devices designed for them.People operate wearable computers from different body positions under different environmental conditions for different uses. Possible positions include standing, walking, and prone. Possible environmental conditions include illumination, acoustic noise, and temperature. Possible uses include surgery, combat, and sports. These aspects (i.e., positions, environmental conditions, and uses) may change as a user operates a wearable computer [4]. We summarize this contextual landscape for pointing devices for wearable computers as(1)
body
positions
×
environmental
conditions
×
uses
.
This wide cross-product has led to the development of a variety of innovative wearable pointing devices.We survey a representative set of pointing devices for wearable computers and classify them using an “adaptation of traditional devices” versus “new devices” dichotomy. Adaptations of traditional devices consist of traditional pointing devices adapted for use with wearable computers. In contrast, we refer to pointing devices for wearable computers that are not adaptations of traditional devices as new devices. We study these devices according to their control and sensing mechanisms and form factor.The objective of this paper is to showcase a rich variety of pointing devices, with diverse characteristics, developed for wearable computers and bring structure to the design space for wearable pointing devices. We conclude that a de facto pointing device for wearable computers, unlike laptops, is not likely to emerge.This paper is organized as follows. Section2 discusses the three primary characteristics from which we survey devices: control and sensing mechanisms and form factor. Section 3 covers adaptations of traditional pointing devices while Section 4 surveys a variety of new devices. We identify tradeoffs in the design of wearable pointing devices and conclude in Section 5.


## 2. Characteristics of Pointing Devices

We characterize wearable pointing devices based on their control and sensing mechanisms and form factor. Note that distinct implementations of the same type of device may each use a different sensing mechanism. For instance, optical and scroll ball mice use different sensors but are referred to as mice nonetheless.


### 2.1. Control Mechanisms

The control mechanism of a pointing device refers to the movements made by a user to manipulate the cursor. The control mechanisms we consider are head-tracking, finger-tracking, wrist-tracking, forearm-tracking, and hand-operated.Head-tracking devices allow a user to tilt or rotate her head to move the cursor. While these devices have the advantage of allowing a user to operate the device hands-free, they require head movements to control the cursor and, thus, constrain gaze. Finger-tracking devices allow a user to control the cursor by moving his fingers, usually in the direction of intended motion. Since fingers usually have a high level of dexterity, these devices allow a user to precisely control the cursor. Wrist-tracking devices allow users to control the cursor using wrist movements. Forearm-tracking devices allow a user to control the cursor with her forearm independently of wrist or finger movement. Pointing devices that a user operates with his hand but do not explicitly track finger or wrist position are referred to here as hand-operated devices.Pointing devices can have other control mechanisms. For instance, a pointing device can use eye tracking [5]. However, pointing devices based on eye tracking are currently more appropriate for desktop, as opposed to wearable computers, because the sensors are large and the accuracy is poor in mobile environments [6] and, thus, are not treated further here. Pointing devices can also be tongue-operated [7] or controlled by an EEG [8] or EMG [9]. We do not consider these here because they are generally invasive (i.e., either in a user’s mouth or sensors are cumbersome with current technology).The control mechanism of a pointing device is an important design choice since it designates the muscle groups that manipulate the pointing device and, as a result, control the cursor. These muscle groups determine the upper bound of the bandwidth of the device as measured using Fitts’ law [10] and, therefore, must be selected with consideration of the device intended uses and body positions of a user [11]. In other words, the muscle groups that control the cursor must provide the pointing device with sufficient bandwidth for typical pointing tasks of the intended uses and body positions of a user.


### 2.2.  Sensing Mechanisms

Pointing devices utilize sensors, which output the magnitude of a physical quantity as an electrical signal, to measure the motor responses of a user and move the cursor accordingly. Since sensors determine the movements that a device is sensitive to, the choice of control mechanism is dependent on the choice of sensing mechanism. A deficient choice in the selection of a sensor can lead to usability problems [2]. The four sensing mechanisms we consider are acoustic sensors, cameras, inertial sensors, and electromechanical sensors (see the bottom of Figure 2).A device that employs an acoustic sensor measures the propagation delay between an acoustic source and the corresponding sensor to determine the separation between them. Either the source or the sensor is placed on a user, who manipulates the cursor by changing the relative position between the source and sensor.A device that employs a camera utilizes image processing to track a user’s movement and move the cursor accordingly. For such a device to operate effectively, with the exception of an optical mouse, a user’s fingers must be within the purview of the camera in an illuminated environment.An inertial sensor that measures changes in linear acceleration is called an accelerometer while one that measures angular velocity is called a gyroscope [1]. These sensors are available as small integrated circuits and, thus, allow pointing devices to have an effective form factor (i.e., in this case, small in size). Inertial sensors also consume low power and are inexpensive and self-contained (i.e., they do not require any external components). Thus, they have an advantage over acoustic sensors, which require external components (e.g., an emitter).An electromechanical sensor translates a mechanical quantity, such as an object’s position or displacement, into an electrical signal. For example, a rotary encoder is an electromechanical sensor which converts a shaft’s angular position into a digital or analog signal.The sensing mechanism of a pointing device for wearable computers is an important design choice since it designates what physical quantity a user manipulates to move the cursor. For instance, gyroscopes allow a user to move the cursor with rotational motion while cameras allow a user to move the cursor with displacement, position, or orientation.


### 2.3.  Form Factor

Form factor is the physical arrangement and configuration of a device. Since people operate wearable computers from different body positions under different environmental conditions for different uses, the form factor of a wearable pointing device should not hinder a user’s mobility or impede the execution of other tasks. The general areas of the human body that are unobtrusive for wearable objects are the collar; rear of the upper arm; forearm; rear, side, and front rib cage; waist and hips; thigh; shin; and top of the foot [3]. Even though this list does not include the wrist or fingers because, due to their high dexterity, they are an advantageous area on which to mount sensors, wrists or fingers increase the upper bound of a device bandwidth at the expense of increased obtrusiveness, which is acceptable especially when a sensor is compact. We consider two categories for the form factor of pointing devices for wearable computers: hand-held and body-mounted.


## 2.1. Control Mechanisms

The control mechanism of a pointing device refers to the movements made by a user to manipulate the cursor. The control mechanisms we consider are head-tracking, finger-tracking, wrist-tracking, forearm-tracking, and hand-operated.Head-tracking devices allow a user to tilt or rotate her head to move the cursor. While these devices have the advantage of allowing a user to operate the device hands-free, they require head movements to control the cursor and, thus, constrain gaze. Finger-tracking devices allow a user to control the cursor by moving his fingers, usually in the direction of intended motion. Since fingers usually have a high level of dexterity, these devices allow a user to precisely control the cursor. Wrist-tracking devices allow users to control the cursor using wrist movements. Forearm-tracking devices allow a user to control the cursor with her forearm independently of wrist or finger movement. Pointing devices that a user operates with his hand but do not explicitly track finger or wrist position are referred to here as hand-operated devices.Pointing devices can have other control mechanisms. For instance, a pointing device can use eye tracking [5]. However, pointing devices based on eye tracking are currently more appropriate for desktop, as opposed to wearable computers, because the sensors are large and the accuracy is poor in mobile environments [6] and, thus, are not treated further here. Pointing devices can also be tongue-operated [7] or controlled by an EEG [8] or EMG [9]. We do not consider these here because they are generally invasive (i.e., either in a user’s mouth or sensors are cumbersome with current technology).The control mechanism of a pointing device is an important design choice since it designates the muscle groups that manipulate the pointing device and, as a result, control the cursor. These muscle groups determine the upper bound of the bandwidth of the device as measured using Fitts’ law [10] and, therefore, must be selected with consideration of the device intended uses and body positions of a user [11]. In other words, the muscle groups that control the cursor must provide the pointing device with sufficient bandwidth for typical pointing tasks of the intended uses and body positions of a user.


## 2.2.  Sensing Mechanisms

Pointing devices utilize sensors, which output the magnitude of a physical quantity as an electrical signal, to measure the motor responses of a user and move the cursor accordingly. Since sensors determine the movements that a device is sensitive to, the choice of control mechanism is dependent on the choice of sensing mechanism. A deficient choice in the selection of a sensor can lead to usability problems [2]. The four sensing mechanisms we consider are acoustic sensors, cameras, inertial sensors, and electromechanical sensors (see the bottom of Figure 2).A device that employs an acoustic sensor measures the propagation delay between an acoustic source and the corresponding sensor to determine the separation between them. Either the source or the sensor is placed on a user, who manipulates the cursor by changing the relative position between the source and sensor.A device that employs a camera utilizes image processing to track a user’s movement and move the cursor accordingly. For such a device to operate effectively, with the exception of an optical mouse, a user’s fingers must be within the purview of the camera in an illuminated environment.An inertial sensor that measures changes in linear acceleration is called an accelerometer while one that measures angular velocity is called a gyroscope [1]. These sensors are available as small integrated circuits and, thus, allow pointing devices to have an effective form factor (i.e., in this case, small in size). Inertial sensors also consume low power and are inexpensive and self-contained (i.e., they do not require any external components). Thus, they have an advantage over acoustic sensors, which require external components (e.g., an emitter).An electromechanical sensor translates a mechanical quantity, such as an object’s position or displacement, into an electrical signal. For example, a rotary encoder is an electromechanical sensor which converts a shaft’s angular position into a digital or analog signal.The sensing mechanism of a pointing device for wearable computers is an important design choice since it designates what physical quantity a user manipulates to move the cursor. For instance, gyroscopes allow a user to move the cursor with rotational motion while cameras allow a user to move the cursor with displacement, position, or orientation.


## 2.3.  Form Factor

Form factor is the physical arrangement and configuration of a device. Since people operate wearable computers from different body positions under different environmental conditions for different uses, the form factor of a wearable pointing device should not hinder a user’s mobility or impede the execution of other tasks. The general areas of the human body that are unobtrusive for wearable objects are the collar; rear of the upper arm; forearm; rear, side, and front rib cage; waist and hips; thigh; shin; and top of the foot [3]. Even though this list does not include the wrist or fingers because, due to their high dexterity, they are an advantageous area on which to mount sensors, wrists or fingers increase the upper bound of a device bandwidth at the expense of increased obtrusiveness, which is acceptable especially when a sensor is compact. We consider two categories for the form factor of pointing devices for wearable computers: hand-held and body-mounted.


## 3. Adaptations of Traditional Pointing Devices

Researchers have primarily adapted the form factor of traditional pointing devices, which were originally designed for desktop or laptop computers, to operate with wearable computers.


### 3.1. Adaptations of the Mouse

A traditional optical mouse uses cameras while a scroll ball mouse uses electromechanical sensors; both use a hand-operated control mechanism. Davis tested the feasibility of an optical mouse and a wireless mouse with a scroll ball as hand-held pointing devices for wearable computers [12]. The optical mouse was used by establishing a tracking surface against a user’s body while the scroll ball mouse was used against a book held by the user’s other hand [12]. Both of these devices have a hand-held form factor and the same sensing and control mechanisms as the mouse on which they are based.Davis concluded that the optical mouse can be used against a user’s chest or abdomen while walking or against the thigh while standing still. Her experiments reveal that the optical mouse is susceptible to ambient motion (i.e., the noise incurred by walking).


### 3.2.  Adaptations of the Trackball

A traditional trackball uses electromechanical sensors and has a hand-operated control mechanism. The4D Off-Table Hand Track mouse, shown in Figure 1(a), is a trackball adapted to operate as a hand-held device [13]. It has a hand-held form factor and the same sensing and control mechanisms as a trackball. A user holds the track mouse in his hand and operates the trackball with his thumb. He rotates the ball with his thumb to move the cursor. The device contains two buttons above the trackball used for primary and secondary clicks (by default for right-handed users, primary and secondary clicks correspond to left and right clicks, resp.). The track mouse also provides a trigger for a user to perform drag-and-drop operations—a user holds the trigger with his index finger while moving the trackball with his thumb (the trigger is also a redundant button for primary clicks). This device is easier to operate in mobile settings than a traditional mouse [1]. Zucco et al. evaluated the track mouse and compared it to three other pointing devices. See the bottom of Section 4.3 for an overview of the results of this usability evaluation.Adaptations of traditional pointing devices. (a) The4D Off-Table Hand Track mouse, (b) theRemotePoint, (c) theTwiddler2, and (d) theEasyCat Touchpad. (a)–(d) from [14, 25–27], respectively.
(a)
(b)
(c)
(d)Figure 2
Directed graph depicting the conceptual design landscape for new wearable pointing devices from control (top) and sensing (bottom) mechanism perspectives. The form factor of each device is given in the bottom half of each rectangular node (center).


### 3.3.  Adaptations of the Trackpoint

A traditional trackpoint uses electromechanical sensors and has a hand-operated form factor. TheRemotePoint, shown in Figure 1(b), has a hand-held form factor and the same sensing and control mechanisms as a trackpoint. It contains a trackpoint that a user operates with his thumb and a button similar to a trigger that a user presses with his index finger to perform a primary click [14].MacKenzie evaluated theRemotePoint using the pointing test of the ISO 9241-9 standard [15]. They tested 12 participants and found that the throughput (which quantifies overall speed and accuracy) of theRemotePoint is low when compared to a desktop mouse. Nine of the twelve participants rated theRemotePoint as “very difficult” or “difficult” to use.TheTwiddler2, shown in Figure 1(c), has a hand-held form factor and the same sensing and control mechanisms as the trackpoint. It contains an embedded trackpoint and straps around the back of a hand [13]. TheTwiddler2 also includes a keyboard for text entry. When a user moves its trackpoint, theTwiddler2 enters “mouse mode” and two of the keys previously designated for keyboard entry function as primary and secondary clicks. Zucco et al. evaluated theTwiddler2 and compared it to the three other pointing devices. See the bottom of Section 4.3 for an overview of the results of this usability evaluation.


### 3.4.  Adaptations of the Touchpad

TheEasyCat Touchpad, shown in Figure 1(d), is a small touchpad that uses electromechanical sensors and has a hand-operated control mechanism [13]. Operating it requires both of a user’s hands. The device contains buttons for both primary and secondary clicks. A study by Thomas et al. determined that mounting the touchpad on the front of a user’s thigh is most appropriate for sitting, kneeling, and standing while mounting the touchpad on a user’s forearm is best while prone [16]. Moreover, this study revealed that if only one touchpad position must be used for these four positions, the forearm position is best. Based on these results, Zucco et al. mounted the device on a user’s forearm with elastic Velcro to evaluate its performance in subsequent study [13]. In this configuration, the device has a body-mounted form factor. Zucco et al. evaluated theEasyCat Touchpad and compared it to the three other pointing devices. See the bottom of Section 4.3 for an overview of the results of this usability evaluation.


## 3.1. Adaptations of the Mouse

A traditional optical mouse uses cameras while a scroll ball mouse uses electromechanical sensors; both use a hand-operated control mechanism. Davis tested the feasibility of an optical mouse and a wireless mouse with a scroll ball as hand-held pointing devices for wearable computers [12]. The optical mouse was used by establishing a tracking surface against a user’s body while the scroll ball mouse was used against a book held by the user’s other hand [12]. Both of these devices have a hand-held form factor and the same sensing and control mechanisms as the mouse on which they are based.Davis concluded that the optical mouse can be used against a user’s chest or abdomen while walking or against the thigh while standing still. Her experiments reveal that the optical mouse is susceptible to ambient motion (i.e., the noise incurred by walking).


## 3.2.  Adaptations of the Trackball

A traditional trackball uses electromechanical sensors and has a hand-operated control mechanism. The4D Off-Table Hand Track mouse, shown in Figure 1(a), is a trackball adapted to operate as a hand-held device [13]. It has a hand-held form factor and the same sensing and control mechanisms as a trackball. A user holds the track mouse in his hand and operates the trackball with his thumb. He rotates the ball with his thumb to move the cursor. The device contains two buttons above the trackball used for primary and secondary clicks (by default for right-handed users, primary and secondary clicks correspond to left and right clicks, resp.). The track mouse also provides a trigger for a user to perform drag-and-drop operations—a user holds the trigger with his index finger while moving the trackball with his thumb (the trigger is also a redundant button for primary clicks). This device is easier to operate in mobile settings than a traditional mouse [1]. Zucco et al. evaluated the track mouse and compared it to three other pointing devices. See the bottom of Section 4.3 for an overview of the results of this usability evaluation.Adaptations of traditional pointing devices. (a) The4D Off-Table Hand Track mouse, (b) theRemotePoint, (c) theTwiddler2, and (d) theEasyCat Touchpad. (a)–(d) from [14, 25–27], respectively.
(a)
(b)
(c)
(d)Figure 2
Directed graph depicting the conceptual design landscape for new wearable pointing devices from control (top) and sensing (bottom) mechanism perspectives. The form factor of each device is given in the bottom half of each rectangular node (center).


## 3.3.  Adaptations of the Trackpoint

A traditional trackpoint uses electromechanical sensors and has a hand-operated form factor. TheRemotePoint, shown in Figure 1(b), has a hand-held form factor and the same sensing and control mechanisms as a trackpoint. It contains a trackpoint that a user operates with his thumb and a button similar to a trigger that a user presses with his index finger to perform a primary click [14].MacKenzie evaluated theRemotePoint using the pointing test of the ISO 9241-9 standard [15]. They tested 12 participants and found that the throughput (which quantifies overall speed and accuracy) of theRemotePoint is low when compared to a desktop mouse. Nine of the twelve participants rated theRemotePoint as “very difficult” or “difficult” to use.TheTwiddler2, shown in Figure 1(c), has a hand-held form factor and the same sensing and control mechanisms as the trackpoint. It contains an embedded trackpoint and straps around the back of a hand [13]. TheTwiddler2 also includes a keyboard for text entry. When a user moves its trackpoint, theTwiddler2 enters “mouse mode” and two of the keys previously designated for keyboard entry function as primary and secondary clicks. Zucco et al. evaluated theTwiddler2 and compared it to the three other pointing devices. See the bottom of Section 4.3 for an overview of the results of this usability evaluation.


## 3.4.  Adaptations of the Touchpad

TheEasyCat Touchpad, shown in Figure 1(d), is a small touchpad that uses electromechanical sensors and has a hand-operated control mechanism [13]. Operating it requires both of a user’s hands. The device contains buttons for both primary and secondary clicks. A study by Thomas et al. determined that mounting the touchpad on the front of a user’s thigh is most appropriate for sitting, kneeling, and standing while mounting the touchpad on a user’s forearm is best while prone [16]. Moreover, this study revealed that if only one touchpad position must be used for these four positions, the forearm position is best. Based on these results, Zucco et al. mounted the device on a user’s forearm with elastic Velcro to evaluate its performance in subsequent study [13]. In this configuration, the device has a body-mounted form factor. Zucco et al. evaluated theEasyCat Touchpad and compared it to the three other pointing devices. See the bottom of Section 4.3 for an overview of the results of this usability evaluation.


## 4. New Devices

In contrast to adapting traditional pointing devices for use with wearable computers, researchers have developed entirely new wearable pointing devices. We present a representative subset of these devices with an emphasis on their sensing mechanism, control mechanism, and form factor (see the directed graph in Figure2).


### 4.1.WearTrack

TheWearTrack uses acoustic sensors and has a finger-tracking control mechanism and a body-mounted form factor. To determine how a user moves the cursor, the device calculates his finger position and orientation relative to his head through the use of a head orientation tracker, a wireless acoustic emitter, and microphones; these components are mounted on a user’s body as shown in Figure 3(a) [17]. TheWearTrack design exploits proprioception—the awareness that a human possess of an object’s position with respect to her body. TheWearTrack includes three microphones on a user’s head to detect the finger-mounted acoustic emitter’s output and measure the sound’s propagation delay. This information is sufficient to compute a user’s hand position. TheWearTrack accurately calculates this position since the close range between a user’s head and hand allows for high update rates. The head tracker’s output is used to transform the hand position calculated from the microphones’ data into a position relative to a user’s head. The system requires no configuration and, thus, can be used by individuals with no knowledge of head-tracking equipment. Note that, despite using a head-tracker, theWearTrack is not a head-tracking device since a user controls the cursor with finger, as opposed to head, movements. Foxlin and Harrington did not perform a usability evaluation of theWearTrack in [17].New wearable pointing devices. (a) Components of aWearTrack equipped wearable computer. (b) Desktop of a personal computer showing the processed camera output of theGraylevel VisualGlove. A user’s index (red) and thumb (green) fingertip positions, which control the cursor’s motion, are highlighted in color while detected by the system. (c) TheGyration Ultra GT cordless optical mouse. (a)–(c) from [17, 18, 28], respectively.
(a)
(b)
(c)


### 4.2.Graylevel VisualGlove

TheGraylevel VisualGlove uses cameras and has a finger-tracking control mechanism and a body-mounted form factor. It employs a head-mounted camera to detect a user’s index and thumb fingertips with gray-level-based image-processing techniques, which are independent of scenery and color [18]. Since real-time gesture detecting image-processing techniques are computationally expensive on wearable computers, theGraylevel VisualGlove uses efficient algorithms to operate under the limited computational capabilities typical of wearable computers. The designers of theGraylevel VisualGlove tested a prototype on a personal computer since they did not have access to a wearable computer (see Figure 3(b)). The device recognizes some simple and intuitive gestures to manipulate the cursor. For instance, a user moves her index and thumb fingertips in the direction of the intended cursor motion and joins these two fingertips to perform a primary click. TheGraylevel VisualGlove also contains an infrared illuminator to allow the device to operate in dark environments.Iannizzotto et al. evaluated the accuracy of 12 participants performing four operations: selection, drag and dropping, double click, and drawing a straight line. They evaluated each operation under normal light, IR light, and normal light with a cluttered background. Overall, the double click operation had the lowest accuracy while selection had the highest accuracy. Illumination by IR light yielded the highest accuracy across all operations.


### 4.3.Gyration Ultra GT Cordless Optical Mouse

TheGyration mouse, shown in Figure 3(c), uses inertial sensors (in this case, gyroscopes) and has a hand-operated control mechanism and a hand-held form factor. The device allows a user to manipulate the cursor by holding the device and moving her wrist in the direction of intended motion [13]. It includes a trigger which must be pressed to enable cursor movement and released when cursor movement is no longer desired. TheGyration mouse also contains primary- and secondary-click buttons and a scroll wheel.Zucco et al. compared theGyration mouse with the track mouse, theTwiddler2, and theEasyCat Touchpad. The experimental task was based on the ISO 9241-9 standard for the evaluation of non-keyboard input device [15] and required participants to drag and drop a target to a destination. They tested 24 participants and found that the fastest device was theGyration mouse while the slowest was theTwiddler2. They also found that theGyration mouse was the most accurate while stationary but the least accurate while walking.


### 4.4.  Accelerometer Mouse

The accelerometer mouse uses inertial sensors and has a wrist-tracking control mechanism and a body-mounted form factor. The device consists of two inertial sensors (in this case, accelerometers) attached to the back of a user’s hands or elbows [19]. The cursor is placed at the intersection of two lines on the screen, each of which is controlled by an accelerometer. Moving the accelerometer on the right hand or elbow rotates line B about the screen’s bottom right corner as illustrated in Figure 4(a), and moving the accelerometer on the left hand or elbow rotates line A about the screen’s bottom left corner as illustrated in Figure 4(b).The accelerometer mouse places the cursor at the intersection of two lines. (a) Moving the inertial sensor (in this case, accelerometer) on the right hand or elbow rotates line B about the screen’s bottom right corner (i.e., point B). (b) Moving the inertial sensor (again, accelerometer) on the left hand or elbow rotates line A about the screen’s bottom left corner (i.e., point A). Images from [19].
(a)
(b)Tokoro et al. evaluated the accelerometer mouse by having eight participants click on a target at a random location on the screen. They found that placing the accelerometers on the hand yielded faster pointing than on the elbows. Additionally, they found that the accelerometer mouse yielded slower pointing than a trackball, an optical mouse, and a joystick, noting that these devices do not allow for hands-free operation.


### 4.5.SBIMouse

TheSBIMouse, shown in Figure 5, uses inertial sensors (in this case, gyroscopes) and has a head-tracking control mechanism and a body-mounted form factor. The inertial sensors are attached to a user’s head to detect angular velocity and move the cursor in the same direction as his head [20]. The device contains two switches attached to a user’s head that can be activated by the cheek to perform primary and secondary clicks. Santos et al. did not include an evaluation of theSBIMouse in [20].TheSBIMouse. (a) The headband contains theSBIMouse electronics. (b) A user wearing theSBIMouse—a head-mounted device. Images from [20].
(a)
(b)


### 4.6.SCURRY

TheSCURRY, shown in Figure 6(a), uses inertial sensors (in this case, gyroscopes) and has a finger-tracking control mechanism and a body-mounted form factor. It contains two inertial sensors attached to the back of a user’s hand to sense angular velocity so that she can manipulate the cursor through hand motion [21]. TheSCURRY has four finger attachments containing inertial sensors (in this case, accelerometers) used to detect click motion. A wireless module attached to a user’s wrist, also shown in Figure 6(a), helps theSCURRY communicate with a computer through radio frequency. This device is powered by AA batteries.More wearable pointing devices. (a) A user wearing the wrist and finger attachments of theSCURRY. (b) A user wearing theG3. (c) A user holding theCUPID-MZ. (a)–(c) from [21, 23, 24], respectively.
(a)
(b)
(c)Kim et al. evaluated the finger-click recognition algorithm of theSCURRY with five participants and found that the number of successful clicks with the algorithm is significantly greater than without the algorithm.


### 4.7.  Oakley, Sunwoo, and Cho’s Pointing Devices

Oakley et al. implemented pointing devices for wearable computers using anX-Sens MTi sensor pack, which contains integrated inertial sensors (here, a gyroscope and an accelerometer), and magnetometers [22]. Oakley et al. used three different sensor positions: (i) on the back of the user’s hand, which has a wrist-tracking control mechanism and a body-mounted form factor; (ii) on a user’s wrist, which has a forearm-tracking control mechanism and a body-mounted form factor; and (iii) held by a user’s hand, which has a hand-operated control mechanism and a hand-held form factor. The device continually monitors the accelerometer and gyroscope of the sensor pack to move the cursor in proportion to its movement regardless of the posture of a user.Oakley et al. evaluated these pointing devices with 12 participants on a Fitts’ task based on the ISO 9241-9 standard [15]. They concluded that although the hand-held condition yielded the best performance with statistical significance the performance in other conditions was relatively close in practice.


### 4.8.G3

TheG3, shown in Figure 6(b), uses inertial sensors (in this case, a gyroscope) and has a finger-tracking control mechanism and a body-mounted form factor. The sensor is placed on the tip of a tactical glove’s index finger to move the cursor in the same direction as the tracked index finger [23]. The device contains two buttons attached to the side of the glove’s index finger: one to perform primary clicks and one to enable movement of the cursor. Releasing the latter allows a user to move his hand without altering the position of the cursor on the screen. TheG3 is a body-mounted device since its sensor is mounted on a tactical glove.Calvo et al. evaluated theG3 on 12 participants using the multidirectional mapping task from the ISO 9241-9 standard [15]. They found that the throughput, which quantifies speed and accuracy, of theG3 was higher than that of a touchpad and trackpoint of a wearable computer but lower than that of a desktop mouse.


### 4.9.CUPID-MZ

TheCUPID-MZ, shown in Figure 6(c), uses electromechanical sensors and has a hand-operated control mechanism and a hand-held form factor. A user holds the device between her index and middle fingers and alters the position of the cursor on the screen by moving her thumb against the sensor grid at the top of the device in the direction of intended motion [24]. The device recognizes a user’s finger gestures using electromechanical sensors (in this case, a 6 × 6 grid of capacitive touch sensors) that detect finger contact by measuring the increased capacitance of a user. TheCUPID-MZ has a button for primary clicks and another for secondary clicks; a user can also perform primary clicks by tapping the device’s sensor grid. TheCUPID-MZ also contains a toggle button to enable drag mode and another to enable scroll mode. The user performs drag-and-drop operations by toggling the drag mode button and moving the cursor. Similarly, a user scrolls by toggling the scroll mode button and moving the cursor. Chatterjee and Matsuno did not include an evaluation of theCUPID-MZ in [24].


## 4.1.WearTrack

TheWearTrack uses acoustic sensors and has a finger-tracking control mechanism and a body-mounted form factor. To determine how a user moves the cursor, the device calculates his finger position and orientation relative to his head through the use of a head orientation tracker, a wireless acoustic emitter, and microphones; these components are mounted on a user’s body as shown in Figure 3(a) [17]. TheWearTrack design exploits proprioception—the awareness that a human possess of an object’s position with respect to her body. TheWearTrack includes three microphones on a user’s head to detect the finger-mounted acoustic emitter’s output and measure the sound’s propagation delay. This information is sufficient to compute a user’s hand position. TheWearTrack accurately calculates this position since the close range between a user’s head and hand allows for high update rates. The head tracker’s output is used to transform the hand position calculated from the microphones’ data into a position relative to a user’s head. The system requires no configuration and, thus, can be used by individuals with no knowledge of head-tracking equipment. Note that, despite using a head-tracker, theWearTrack is not a head-tracking device since a user controls the cursor with finger, as opposed to head, movements. Foxlin and Harrington did not perform a usability evaluation of theWearTrack in [17].New wearable pointing devices. (a) Components of aWearTrack equipped wearable computer. (b) Desktop of a personal computer showing the processed camera output of theGraylevel VisualGlove. A user’s index (red) and thumb (green) fingertip positions, which control the cursor’s motion, are highlighted in color while detected by the system. (c) TheGyration Ultra GT cordless optical mouse. (a)–(c) from [17, 18, 28], respectively.
(a)
(b)
(c)


## 4.2.Graylevel VisualGlove

TheGraylevel VisualGlove uses cameras and has a finger-tracking control mechanism and a body-mounted form factor. It employs a head-mounted camera to detect a user’s index and thumb fingertips with gray-level-based image-processing techniques, which are independent of scenery and color [18]. Since real-time gesture detecting image-processing techniques are computationally expensive on wearable computers, theGraylevel VisualGlove uses efficient algorithms to operate under the limited computational capabilities typical of wearable computers. The designers of theGraylevel VisualGlove tested a prototype on a personal computer since they did not have access to a wearable computer (see Figure 3(b)). The device recognizes some simple and intuitive gestures to manipulate the cursor. For instance, a user moves her index and thumb fingertips in the direction of the intended cursor motion and joins these two fingertips to perform a primary click. TheGraylevel VisualGlove also contains an infrared illuminator to allow the device to operate in dark environments.Iannizzotto et al. evaluated the accuracy of 12 participants performing four operations: selection, drag and dropping, double click, and drawing a straight line. They evaluated each operation under normal light, IR light, and normal light with a cluttered background. Overall, the double click operation had the lowest accuracy while selection had the highest accuracy. Illumination by IR light yielded the highest accuracy across all operations.


## 4.3.Gyration Ultra GT Cordless Optical Mouse

TheGyration mouse, shown in Figure 3(c), uses inertial sensors (in this case, gyroscopes) and has a hand-operated control mechanism and a hand-held form factor. The device allows a user to manipulate the cursor by holding the device and moving her wrist in the direction of intended motion [13]. It includes a trigger which must be pressed to enable cursor movement and released when cursor movement is no longer desired. TheGyration mouse also contains primary- and secondary-click buttons and a scroll wheel.Zucco et al. compared theGyration mouse with the track mouse, theTwiddler2, and theEasyCat Touchpad. The experimental task was based on the ISO 9241-9 standard for the evaluation of non-keyboard input device [15] and required participants to drag and drop a target to a destination. They tested 24 participants and found that the fastest device was theGyration mouse while the slowest was theTwiddler2. They also found that theGyration mouse was the most accurate while stationary but the least accurate while walking.


## 4.4.  Accelerometer Mouse

The accelerometer mouse uses inertial sensors and has a wrist-tracking control mechanism and a body-mounted form factor. The device consists of two inertial sensors (in this case, accelerometers) attached to the back of a user’s hands or elbows [19]. The cursor is placed at the intersection of two lines on the screen, each of which is controlled by an accelerometer. Moving the accelerometer on the right hand or elbow rotates line B about the screen’s bottom right corner as illustrated in Figure 4(a), and moving the accelerometer on the left hand or elbow rotates line A about the screen’s bottom left corner as illustrated in Figure 4(b).The accelerometer mouse places the cursor at the intersection of two lines. (a) Moving the inertial sensor (in this case, accelerometer) on the right hand or elbow rotates line B about the screen’s bottom right corner (i.e., point B). (b) Moving the inertial sensor (again, accelerometer) on the left hand or elbow rotates line A about the screen’s bottom left corner (i.e., point A). Images from [19].
(a)
(b)Tokoro et al. evaluated the accelerometer mouse by having eight participants click on a target at a random location on the screen. They found that placing the accelerometers on the hand yielded faster pointing than on the elbows. Additionally, they found that the accelerometer mouse yielded slower pointing than a trackball, an optical mouse, and a joystick, noting that these devices do not allow for hands-free operation.


## 4.5.SBIMouse

TheSBIMouse, shown in Figure 5, uses inertial sensors (in this case, gyroscopes) and has a head-tracking control mechanism and a body-mounted form factor. The inertial sensors are attached to a user’s head to detect angular velocity and move the cursor in the same direction as his head [20]. The device contains two switches attached to a user’s head that can be activated by the cheek to perform primary and secondary clicks. Santos et al. did not include an evaluation of theSBIMouse in [20].TheSBIMouse. (a) The headband contains theSBIMouse electronics. (b) A user wearing theSBIMouse—a head-mounted device. Images from [20].
(a)
(b)


## 4.6.SCURRY

TheSCURRY, shown in Figure 6(a), uses inertial sensors (in this case, gyroscopes) and has a finger-tracking control mechanism and a body-mounted form factor. It contains two inertial sensors attached to the back of a user’s hand to sense angular velocity so that she can manipulate the cursor through hand motion [21]. TheSCURRY has four finger attachments containing inertial sensors (in this case, accelerometers) used to detect click motion. A wireless module attached to a user’s wrist, also shown in Figure 6(a), helps theSCURRY communicate with a computer through radio frequency. This device is powered by AA batteries.More wearable pointing devices. (a) A user wearing the wrist and finger attachments of theSCURRY. (b) A user wearing theG3. (c) A user holding theCUPID-MZ. (a)–(c) from [21, 23, 24], respectively.
(a)
(b)
(c)Kim et al. evaluated the finger-click recognition algorithm of theSCURRY with five participants and found that the number of successful clicks with the algorithm is significantly greater than without the algorithm.


## 4.7.  Oakley, Sunwoo, and Cho’s Pointing Devices

Oakley et al. implemented pointing devices for wearable computers using anX-Sens MTi sensor pack, which contains integrated inertial sensors (here, a gyroscope and an accelerometer), and magnetometers [22]. Oakley et al. used three different sensor positions: (i) on the back of the user’s hand, which has a wrist-tracking control mechanism and a body-mounted form factor; (ii) on a user’s wrist, which has a forearm-tracking control mechanism and a body-mounted form factor; and (iii) held by a user’s hand, which has a hand-operated control mechanism and a hand-held form factor. The device continually monitors the accelerometer and gyroscope of the sensor pack to move the cursor in proportion to its movement regardless of the posture of a user.Oakley et al. evaluated these pointing devices with 12 participants on a Fitts’ task based on the ISO 9241-9 standard [15]. They concluded that although the hand-held condition yielded the best performance with statistical significance the performance in other conditions was relatively close in practice.


## 4.8.G3

TheG3, shown in Figure 6(b), uses inertial sensors (in this case, a gyroscope) and has a finger-tracking control mechanism and a body-mounted form factor. The sensor is placed on the tip of a tactical glove’s index finger to move the cursor in the same direction as the tracked index finger [23]. The device contains two buttons attached to the side of the glove’s index finger: one to perform primary clicks and one to enable movement of the cursor. Releasing the latter allows a user to move his hand without altering the position of the cursor on the screen. TheG3 is a body-mounted device since its sensor is mounted on a tactical glove.Calvo et al. evaluated theG3 on 12 participants using the multidirectional mapping task from the ISO 9241-9 standard [15]. They found that the throughput, which quantifies speed and accuracy, of theG3 was higher than that of a touchpad and trackpoint of a wearable computer but lower than that of a desktop mouse.


## 4.9.CUPID-MZ

TheCUPID-MZ, shown in Figure 6(c), uses electromechanical sensors and has a hand-operated control mechanism and a hand-held form factor. A user holds the device between her index and middle fingers and alters the position of the cursor on the screen by moving her thumb against the sensor grid at the top of the device in the direction of intended motion [24]. The device recognizes a user’s finger gestures using electromechanical sensors (in this case, a 6 × 6 grid of capacitive touch sensors) that detect finger contact by measuring the increased capacitance of a user. TheCUPID-MZ has a button for primary clicks and another for secondary clicks; a user can also perform primary clicks by tapping the device’s sensor grid. TheCUPID-MZ also contains a toggle button to enable drag mode and another to enable scroll mode. The user performs drag-and-drop operations by toggling the drag mode button and moving the cursor. Similarly, a user scrolls by toggling the scroll mode button and moving the cursor. Chatterjee and Matsuno did not include an evaluation of theCUPID-MZ in [24].


## 5. Conclusion: Tradeoffs and Discussion

We surveyed a representative set of wearable pointing devices and classified them at a high level into adaptations of traditional devices and new devices. We further classified devices by their sensing and control mechanisms and form factor as shown in Figure2.Table1 provides a comprehensive summary of our classification and survey. The last row of Table 1 gives frequencies for each characteristic of the pointing devices for wearable computers considered in this paper and illustrates how the characteristics are distributed among the devices surveyed here. For instance, form factors are evenly balanced: eight hand-held and eight body- mounted. Although “hand-operated” is the predominant control mechanism of the devices surveyed, this is because the control mechanism of all adaptations presented here is hand-operated, the other control mechanisms are distributed fairly even among the new devices showcased. Technology for inertial sensors has advanced to the point where they consume low power, have a small size, and are inexpensive and self-contained (i.e., they do not require any external components). Thus, we anticipate that inertial sensors will be used more frequently in pointing devices for wearable computers than the other sensing mechanisms. Our survey reveals this trend: inertial sensors are the predominant sensing mechanism among the new devices while electromechanical sensors are the most frequently used sensing mechanism among the adaptations because the touchpad, trackpoint, and trackball typically use this sensing mechanism.Table 1
Our matrix classification of the devices surveyed, from an adaptations versus new devices dichotomy. Each device is characterized by control and sensing mechanisms and form factor.
Pointing devices
Control mechanism
Sensing mechanism
Form factor
Head-track
Finger-track
Wrist-track
Forearm-track
Hand-operated
Acoustic
Cameras
Inertial
e-mech
Hand-held
Body-mounted
Adaptations
Davis’s optical mouse [12]
✗
✗
✗
✗
✓
✗
✓
✗
✗
✓
✗
Davis’s wireless mouse [12]
✗
✗
✗
✗
✓
✗
✗
✗
✓
✓
✗
Track mouse
✗
✗
✗
✗
✓
✗
✗
✗
✓
✓
✗
R
e
m
o
t
e
P
o
i
n
t
✗
✗
✗
✗
✓
✗
✗
✗
✓
✓
✗
T
w
i
d
d
l
e
r
2
✗
✗
✗
✗
✓
✗
✗
✗
✓
✓
✗
EasyCat  Touchpad
✗
✗
✗
✗
✓
✗
✗
✗
✓
✗
✓
New devices
W
e
a
r
T
r
a
c
k
✗
✓
✗
✗
✗
✓
✗
✗
✗
✗
✓
G
r
a
y
l
e
v
e
l
V
i
s
u
a
l
G
l
o
v
e
✗
✓
✗
✗
✗
✗
✓
✗
✗
✗
✓
Gyration  mouse
✗
✗
✗
✗
✓
✗
✗
✓
✗
✓
✗
Accelerometer mouse
✗
✗
✓
✗
✗
✗
✗
✓
✗
✗
✓
SBIMouse
✓
✗
✗
✗
✗
✗
✗
✓
✗
✗
✓
SCURRY
✗
✓
✗
✗
✗
✗
✗
✓
✗
✗
✓
Oakley et al.’s devices [22]
✗
✗
✓
✓
✓
✗
✗
✓
✗
✓
✓
G
3
✗
✓
✗
✗
✗
✗
✗
✓
✗
✗
✓
CUPID-MZ
✗
✗
✗
✗
✓
✗
✗
✗
✓
✓
✗
Frequency
1
4
2
1
9
1
2
6
6
8
8Each control mechanism, sensing mechanism, and form factor has an inherent set of advantages and disadvantages with respect to body positions, environmental conditions, and uses. Thus, tradeoffs emerge while designing a device for particular positions, environmental conditions, and uses. For instance, Thomas et al. required their pointing device to operate when a user is walking, standing, sitting, or prone [16]. In contrast, other pointing devices were designed to operate only when a user is standing or sitting. As another example, a benefit of head-tracking devices is that they allow a user to point without occupying her hand. On the other hand, head-tracking devices require a user to move her head to manipulate the mouse and, thus, constrain the direction of her gaze, making it difficult for her to walk. Another example of a tradeoff occurs with cameras. For a wearable pointing device using cameras to operate properly, there must be sufficient light in the environment. In the absence of natural light, pointing devices that use cameras can incorporate a light source such as an infrared emitter. Thus, designers must choose whether to constrain the device to operate in bright environments or include a light source at the expense of extra weight and power consumption. Note that aside from theSBIMouse andG3, to the best of our knowledge, none of devices surveyed here were designed for any specific uses or users (theSBIMouse was designed for users with motor impairment while theG3 was designed for military applications).While the focus of this paper is onpointing devices for wearable computers, pointing is only one of a variety of ways to interact with an application. For instance,Google Glass relies on coarse finger gestures on a trackpad or head gestures. However, specific applications that run onGlass can use its inertial sensors to implement pointing and this is an avenue of possible future research. If used in this manner,Glass operates as a wearable pointing device with a head-tracking control mechanism. For instance, a game calledSpellista challenges a user to find a word in a series of scrambled letters. To select a letter, a user must move a pointer on top of the letter with head movements.Since desktop computers are constrained to be used on a desk, the mouse became the de facto pointing device for these computers because it enables efficient pointing when a user has a flat and steady surface on which to manipulate it. The small footprint of both a touchpad and a trackpoint allows them to be easily embedded into the chassis of a laptop computer, which contributes to their ubiquitous presence in such computers. In stark contrast, no wearable pointing device is appropriate for all body positions in all environmental conditions for all uses. Thus, unlike desktop and laptop computers, there is no de facto pointing device for wearable computers, and we believe that a universal pointing device for wearable computers is not likely to emerge. However, based on our analysis, we anticipate that(i)
inertial sensors will be used more frequently in pointing devices for wearable computers than the other sensing mechanisms;
(ii)
as niche domains for wearable computers are identified, more targeted pointing devices for specific applications and users will emerge;
(iii)
applications forGoogle Glass which use its inertial sensors to implement pointing will emerge.This survey, and particularly Figure 2 and Table 1, has brought structure to the design space for wearable pointing devices, and we are optimistic that it will help developers of pointing devices for wearable computers navigate the space, including its tradeoffs and constraints, in evaluating design criteria for wearable pointing devices.



---

*Source: 527320-2014-03-24.xml*