ThumbSense: Automatic Input Mode Sensing for
Touchpad-based Interactions
Jun Rekimoto
Interaction Laboratory
Sony Computer Science Laboratories, Inc.
3-14-13 Higashigotanda Shinagawa-ku, Tokyo 141-0022, Japan
Phone: +81 3 5448 4380 Fax: +81 3 5448 4273
Mail: rekimoto@acm.org
http://www.csl.sony.co.jp/person/rekimoto.html

ABSTRACTA

B

While manipulating the touchpad, a user’s hand position
must be away from the keyboard’s home position. This effect hinders smooth switching between text entry and pointermanipulation, and is considered to be the one of the majordrawback of the touchpad against to the trackpoint. This
paper introduces ThumbSense, a new input technique aims
to solve this problem by automatically sensing users’ input
mode based on finger contact
to the touchpad. A key on thekeyboard, such as the F  key, transparently acts both as anormal key as well as a mouse button. This technique is implemented by using the sensor feature of the touchpad, and ispossible to apply most of currently available portable computers without requiring any additional hardware/sensors.

CDKeywords

Touchpad, touch-sensitive user interfaces, mode sensing.

Figure 1: Comparison of hand positions (A, B: Without
ThumbSense. The user’s hands are off from the home position. C, D: with ThumbSense. The user can transparently switch between the text (C) and mouse (D) modes.)INTRODUCTION

Touchpad is one of the most popular input devices for portablecomputers. This device detects finger position based on capacitive sensing and can be used as a mouse compatible input
device with the combination of attached buttons. However, incontrast to Trackpoint, users of touchpad are forces to change
their hand positions from the keyboard home position to theposition of the touchpad (Figure 1). Although it is not impossible to operate the touchpad from the keyboard home position by stretching thumbs toward the touchpad buttons, it isneither comfortable nor natural. In contrast, Trackpoint is designed to be operated while the user’s hands are on the homeposition. Because of this difference, users who are capableof touch-typing tend to prefer trackpoint to touchpad.
This paper proposes a new interaction technique, called ThumbSense, to allow touchpad users to manipulate it while keeping
the keyboard home position.
 The primary idea is to use keyboard keys, such as F or J keys, as mouse buttons. TheThumbSense software automatically distinguishes user’s inCopyright is held by the author/owner(s).
CHI 2003, April 5–10, 2003, Ft. Lauderdale, Florida, USA.
ACM 1-58113-637-4/02/0004.

put modes, typing and mouse operation modes, by sensing
whether the user’s thumb is contacting the touchpad surface.
ThumbSense

The touchpad senses the users finger position based on capacitive sensing. It also detects finger contact to the touchpad, and thus it can be used as a kind of touch sensors, but
this feature has not been effectively used in previous userinterface systems. The ThumbSense recognizes user input
modes using this touch sensitivity. When a user’s finger(thumb) touches the touchpad, the system automatically becomes the ”touch-mode”
 and preocesses keyboard keys. For
example, the F  and J  keys act as mouse buttons. With
this feature, users of ThumbSense can operate the touchpad
without moving their hands away from the home position.
To enter characters, users can simply lift their thumb fromthe touchpad, and F  and J  keys return to the normalkeys. This touch-sensing feature provides transparent modeswitching between the key-input and the mouse-operation
modes without requiring explicit mode change commands.
The mouse ”wheel” button can also be emulated;moving
the thumb on the touchpad while pressing the S key would
cause wheel motion events.

AAT

S3

S2

Legends:

RUU

Finger touch
Mouse Down

Mouse Up

Mouse UpF, J down
TstartS0S1
R

Any keydown

Key UpKey Down

UKey down
Other keydown

Key UpU

Key Down

TS5
A
Auto repeat

SiS4RA
Auto repeat

Events:Mouse Down
User inputs:
U: key up
T: finger touch
R: finger release
A: auto repeat

Figure 2: The state transition diagram.

Figure 3: An example of multi-button mouse operations.
One of the keyboard keys is bound to the application
switch function. By simply typing this key, it acts as thenormal task-switching key (i.e., ALT-TAB of Windows),
but sliding the thumb (finger) on the keypad, the task selection pie menu appears on the screen.
to the window that is obscured by another window. WithThumbSense, dragging can be modified by pressing otherkeys. For example, a user can start dragging operation bytouch- F , and change the window order by pressing E without releasing the F button.
IMPLEMENTATION

Figure 2 shows the state transition diagram of ThumbSense
for emulating the mouse. Basically, the system distinguishes
the key-input and the mouse modes based on finger contact. However, during the ”drag” operation, the finger is often lifted for repositioning (because of the limited area ofthe touchpad), and thus the user’s finger is temporarily awayfrom the touchpad. To consider this situation, the system enters the ”drag” mode when the user first presses the emulatedmouse buttons (i.e., corresponding keyboard keys). Whilethe user holds the mouse buttons, the input mode is fixed tothe mouse emulation mode regardless of the finger contact tothe touchpad.
INTERACTION WITH MULTIPLE-BUTTON MOUSEAs described in the previous section, ThumbSense emulates
the normal mouse device with three buttons (the third button
could be a wheel). Since the keyboard has more than threekeys, it has a potential to design interaction techniques thathas multiple buttons.
Symmetricmouse button assignment:
We can define F ,J as the leftmouse button, D , K as the right mouse button, and S , L as the wheel button. This symmetric assignment enables flexible mouse operations with either by one
hand or two hands (e.g., both mouse buttons and the touchpad can be operated by the right hand).
As a new modifier key: ”Touch” can be regarded as a newmodifier key. For example,
a user could launch the mail application by touch- M (pressing the M key while touchingthe keypad), or change the window order by touch- E , etc.
These key operations can be followed by mouse operations.
Within a drawing application,a user could directly create arectangle by touch- R  dragging (i.e., without selecting a”rect” tool). Another example is also shown in Figure 3.
Enhance mouse operations: While performing a drag operation, it is often the case that we want to drop an object

The ThumbSens system is implemented as a taskbar softwaremodule for Microsoft Windows. ½ It intercepts keyboard andmouse events, and generates and delivers appropriate events
according to the state transition as shown in Figure 2. Since
this event emulation occurs at the OS level, no modification is required to applications. The low-level APIs to thetouchpad are also used to detect finger contact to the touchpad. Currently, two major touchpad devices (Synaptics andALPS) are supported, and can be used with the most of theexisting notebook computers.
CONCLUSIONThe ThumbSense is a new interaction technique that allows
users to operate the touchpad input device without put hands
aware from the keyboard home position. This feature solves
one of the major drawbacks of the touchpad against to thetrackpoint. ThumbSense can be regarded as a kind of sensory enhanced user interfaces [1, 2]. Unlike these previous
systems, ThumbSense uses the feature of the existing input
device, and thus can be widely used with currently available
notebook computers without requiring any additional hardware or sensors.
Acknowledgements

We thank Sony Corporation and ALPS electronics Inc. forproviding us the touchpad driver information.
REFERENCES

1. Beverly L. Harrison, Kenneth P. Fishkin, Anuj Gujar,
Carlos Mochon, and Roy Want. Squeeze me, hold me,tilt me! an exploration of manipulative user interfaces.
In CHI’98 Proceedings, pages 17–24, 1998.
2. Ken Hinckley and Mike Sinclair. Touch-sensing input
devices. In CHI’99 Proceedings, pages 223–230, 1999.
½ downloadable from http://www.csl.sony.co.jp/person/rekimoto/tsense/