World Automation Congress © 2010 TSI Press.

REBO: A LIFE-LIKE UNIVERSAL REMOTE CONTROLSEIJI YAMADA*1 AND KAZUKI KOBAYASHI*2*1

National Institute of Informatics / The Graduate University for Advanced Studies (SOKENDAI)
2-1-2 Hitotsubashi, Chiyoda, Tokyo, 101-8430, Japanseiji@nii.ac.jp*2
Graduate School of Science and Technology, Shinshu University
4-17-1 Wakasato, Nagano, 380-8553, Japan
kby@shinshu-u.ac.jp

ABSTRACT— This paper describes a novel remote control operational with stroking its surface.
There are lots of remote controls in our houses for TVs, air conditioners, and so on. However, when
we use a remote control, we need to look at both our fingers and the target appliance that we would
like to control. It may be not significantly problematic for young people, but elderly people have adifficulty in manipulating remote controls with many buttons. We consider it will be comfortable forvarious people to use a remote control without looking at their fingers and pushing buttons. Also we
consider a remote control should have life-like appearance to become a more familiar artifact tousers. In this study, we propose a life-like remote control, Rebo, operational only with stroking its
surface and apply to a TV system. The developed remote control has three advantages; familiarity,
function awareness, and stroke manipulation in contrast with conventional remote controls withmany buttons. These advantages enable users to feel much familiarity by using it, to easily notice its
implemented functions, and to use it without looking at the fingers and buttons.
Key Words: universal remote control, stroke-based, life-like

1. INTRODUCTION
There are various remote controls in our houses. We have remote controls for TV, air conditioner,room light, and so on. Remote controls for appliances are commonplace devices and have been widely
spread. However there are significant problems on remote controls. For example, any remote controls
confuse us because we need to use various remote controls with different user interfaces. A universal
remote [1] that aggregates functions of various remote controls is one of technical solutions for thisproblem. Users can control various appliances by manipulating a universal remote control, not using
specific remote controls for each appliance. On the other hand, users for traditional button-based remotecontrol need to search a target button and to move the fingers to push it. It may be not a serious problem foryoung people, but elderly people have a difficulty in such manipulation for remote controls. We consider itwill be comfortable and easily operational even for elder people to use a remote control without looking at
the fingers. Such a remote control is also comfortable to young people who are familiar to home electric
appliances because of easy operation. Also we consider a remote control for appliances should have a lifelike appearance to become a more familiar artifact to users. By such an appearance, users who are not sofamiliar to appliances get to use the remote control without hesitation.
In this paper, we propose a life-like remote control, Rebo, manipulatable only with stroking its surface[2] and apply to an advanced TV system. Users only stroke its surface with the fingers to control various
home electric appliances far from them. Rebo has three advantages which conventional remote controls
have never had; familiarity, function awareness, and stroke manipulation. The life-like appearance andfacial expressions make it familiar with users. As for function awareness, we adopt action sloping [3, 4]that enables users to notice its functions. Therefore, users do not have to seek a button that they want anduse it easily.
As a realized example of the proposed remote control, we develop an advanced TV system in which
action sloping is implemented. We actually implemented some functions like channel select, power on/off
on the advanced TV system, and confirmed the execution.

2. RELATED WORK
Rebo is a device which users can control various appliances only by stroking with the fingers, and alsoa kind of touch-based remote control. In human-computer interaction and user interface, such touch-based
remote controls have been developed so far. In contrast with conventional touch-based remote controls,since Rebo utilizes users’ strokes with the fingers, they do not need to visually recognize and understandany icon, signal, and acknowledgment from a remote control. A user can concentrate on the feedback fromappliances, not a remote controls. Advantage of our approach partially resembles that of eyes-free approachfor many people including visually-disabled persons and elder people.
Zhao et al. developed a potable audio player having eyes-free menu selection which utilizes touch
input and reactive audio feedback [5]. Buil et al. also applied a touch-based manipulation to control
headphones [6]. Brewster et al. designed eyes-free multi-modal interaction for Wearable devices [7].
Particularly for using mobile and wearable computers, input or output problems occur due to limited screen
space and interaction techniques. In these studies, they overcame the problems by developing a 3D audio
radial pie menu and a sonically enhanced 2D gesture recognition system on a belt-mounted PDA.
Micire et al. proposed a multi-touch remote control for human-robot interaction [8]. They tried toapply a multi-touch table to control a physical robot agent and described analysis on users’ interaction
styles in complicated functions of the multi-touch interface.
All of these previous studies have attempted just to enrich usability of the user interface by touchbased devices like touch panels. In the view of developing touch-based interface devices, Rebo is similar tothe previous studies. However we consider Rebo’s touch-based approach is not only for usability ofinterface, but also for partial execution for function awareness as discussed later. Thus Rebo provides novel
advantage of touch-based interface in a human-adaptive system.
Osawa et al. proposed a unique approach to anthropomorphize various appliances by attaching arms
and eyes on them [9]. They actually developed artificial arms, eyes and attached them on a refrigerator, aprinter and so on. Also they conducted experiments with participants and investigated user’s behavior andinterpretation on the anthropomorphized appliances. Their approach is similar to Rebo’s one in terms ofmaking appliances more familiar to users by anthropomorphizing them, but the concrete method issignificantly different. In contrast that they attach additional equipments like arms, eyes to conventional
appliances, we try to design a novel life-like remote control of an appliance.
Partial execution, one of Rebo’s advantages, is an extension of our previous work on action sloping formaking a user to notice a robot’s function [3, 4]. In our previous work, partial execution was restricted inrobot’s physical functions like robot’s motions. In contrast with it, in this work, partial execution is
generalized to partially execute various unphysical functions including feedback expression such as an
animation on a two-dimensional surface.

3. REBO AS A LIFE-LIKE REMOTE CONTROL TESTBED
Fig. 1 shows the developed life-like remote control, Rebo. Rebo is an universal remote controls of

Fig. 1 Rebo’s appearance.

Fig. 2 Architecture of Rebo.

home electric appliances. It has a user-friendly appearance; a smooth white surface for stroking and a backside fits to users’ thighs. We have carefully designed the shape of Rebo with cooperation of an industrial
designer, one of the authors, and determined a life-like appearance shown in Fig.1. We consider this simple
and life-like appearance significantly makes users to feel much more familiarity than conventional buttonbased remote controls. This appearance also makes users interested in it and guides them to use it without
hesitation. The size of Rebo is 249 mm long, 146 mm wide and 96 mm high.

3.1 System Architecture
Fig. 2 shows architecture of Rebo system. In the current stage, a main processing device (PC) is notembedded in Rebo. In the next stage in near future, all of devices including batteries and the mainprocessing device will be embedded in the body of Rebo. Rebo has a display for facial expressions, fullcolor LED modules, and three touch devices (iPod touches) are embedded in the body. Two iPods are used
as touch sensors for sensing users’ stroking and the remaining one is used for displaying facial expressions.
The iPod touches used as touch sensors are covered with a white, soft and smooth cloth. Touch positions
are sent to a host PC through WiFi from iPod’s web browser Safari. We adopted Open Sound Control(OSC) as a communication protocol. When a user touches the browser on an iPod touch, a JavaScript
program is executed and then a PHP function sends OSC packets to a connected PC. Three full-color LED
modules embedded in Rebo provide a user with light-based feedback when s/he manipulates Rebo.
In the following subsections, we explain three advantages of Rebo: familiarity, function awareness,and stroke manipulation.

3.2 Familiarity
Rebo has a life-like appearance in Fig. 1 and facial expressions to acquire familiarity with a user. We
adopt the concept of “intermediate entity between artifacts and animate beings”. Rebo is not only a tool but
also a pet to users. The body of Rebo is covered with soft and bouncy cloth and it is comfortable for users
to touch Rebo. When the user strokes Rebo, it changes its facial expressions to inform him or her of various
emotional states of Rebo. We consider this makes the user more comfortable with Rebo. Rebo currently has
four different facial expressions like “Sleep”, “Normal”, “Select TV channels” and “Change soundvolume”. Therefore, interaction with Rebo can be the purpose of the user as well as manipulation of homeelectric appliances. We believe that this concept plays an important role for establishing familiarity
between users and Rebo.

3.3 Function Awareness
Function awareness is user’s awareness that which manipulation causes which function of a machine[3, 4]. For example, when a user purchases a new appliance which s/he has few experiences to use, s/hereads the owner’s manual before using it to understand its functions. However searching for the desired
functions and studying how to manipulate the appliance is often complicated and difficult. Therefore, a way
for a user to comprehend machine’s functions without reading the manuals would be useful.
In this work, we adopt action sloping [3, 4] that enables a user to notice its functions to realizefunction awareness. Action sloping enables machines to provide feedback that gradually changes inFig. 3 Intensity of feedback behavior.

intensity as the user carries out given actions. As for intensity of feedback behavior, it is assumed that thevolume, frequency and quality of representation are changed. Some patterns of feedback behavior that canbe used are changing lights displayed, sounds emitted, or timing of movements. Fig. 3(a) shows theconventional method for exhibiting feedback behavior. When a function of a machine is executed, feedback
is provided for the user. However, the machine provides no feedback at all if the user does not perform afunction.
In contrast, with action sloping shown in Fig. 3(b), a machine exhibits feedback behavior even when afunction is not performed. It provides feedback in response to user’s incomplete manipulation. Therefore,
action sloping enables users to understand relationship between user’s action and the machine’s functions.
For example, we can consider a case of using a TV remote as follows. When a power button is pushed(triggering action is performed), the TV turn off (feedback behavior). A user receives feedback of his/hermanipulation when the TV actually turns off. In contrast, action sloping gradually provides feedback before
the TV actually turns off. Before the user touches the button, the TV decreases the image dimensions to
express “the TV is likely to turn off”. This action sloping requires a sensor that can measure the distance
between the user’s fingers and the button. In this study, a user interacts with Rebo by stroking it. This kindof interaction manner bears a continuous action and easily achieves action sloping.
We utilize this feature and propose partial execution as a way for the user to notice functions of themachines. Partial execution is a feedback expression method that provides a meaning of users’
manipulation by showing a part of an executing function. For example, when a user strokes Rebo for ashort time to turn off the TV, Rebo decrease the image dimensions of the TV for a short time. If the user
strokes Rebo for a long time, it decreases the image dimensions and turns off the TV power eventually.
Feedback expressions of the machine strongly depend on its functions and are designed for eachapplication. We need to develop the methodology to design adequate realization of action sloping invarious domains related to a remote control.

3.4 Stroke Manipulation
Rebo has no button and a user strokes its surface to control appliance. In this kind of manipulation,s/he does not need to move the fingers precisely and not seek the button that s/he wants to push. It is more
comfortable for the user to gaze at a machine that s/he wants to control than to gaze at the remote control
because the feedback from the machine is more important than that from the remote control. Thus weconsider Rebo’s manipulation by stroking has advantage over conventional button-based remote controls.

4. A TV REMOTE CONTROL AND EXPERIMENTS
We developed Rebo, as a TV remote control based on the proposed remote control to implement allof the three advantages in the previous section. In addition, we also designed feedback behavior of the TV
and constructed an advanced TV system. The advanced TV system is a simulated TV in which a movie
player plays recorded TV programs on a PC monitor in multiple channels like a real TV. Also, traditional
remote controls can communicate the advanced TV system, and we can implement and test various action
sloping methods on the advanced TV system. Unfortunately implementation of action sloping and partial

Fig. 4 Partial execution: changing a TV channel by stroking Rebo.

Fig. 5 Sound volume control.

Fig. 6 Partial execution in power on/off.

execution on a real TV is hard, thus we need to use an advanced TV system.
Functions we implemented in the advanced TV system are channel select, sound volume change, andpower on/off. In this section, manipulation methods of Rebo and feedback behavior of the TV are described.

4.1 Channel SelectFig.4 shows a series of screenshots in which the positions of two videos are changed as a user strokes
the surface of Rebo right and left. When a user widely strokes the surface of Rebo with the fingers, a video
picture goes the outside of the TV frame and another video picture comes into the frame. When a user
strokes Rebo for a short time, a part of another video picture comes into the frame and then goes out of theframe automatically. As described earlier, we used recorded TV programs and did not use TV tuners in this
implementation. It is technically possible to capture TV programs and show them in the monitor.

4.2 Sound Volume Change
When a user strokes Rebo up and down, the volume indicator (a vertical white bar) is shown on thevideo (Fig. 5). The length of the indicator is changed as the movement distance of the fingers is changed.
This is a typical example of indication-based partial execution to express the varying sound and the bar
indication.

4.2 Power On / Off
A series of screenshots in the top of Fig. 6 shows a video being reduced in an image dimensions. Whena user touches Rebo for more than one second, the size of the video is reduced and finally disappeared. If
the user stops touching Rebo before the video is disappeared, the video is enlarged and goes to an ordinarysize.
On the other hand, a series of screenshots in the bottom of Fig.6 shows a video being enlarged. Whenthe advanced TV system has been turned off, the picture is gradually enlarged while the user touches Rebo.
When the user stops touching before the video becomes to a maximum size, it is automatically going down
and then disappeared.

4.3 Experiments
We are currently preparing to conduct experiments
to evaluate the advantage1s of Rebo. Impression on theappearance, usability, ease of finding functions, and eye
movements will be compared between Rebo and atypical TV Infrared remote (IR remote). Eye
movements measured by an eye-tracking system (Tobii
T60) will be used to investigate the manipulation
without looking at Rebo. Fig. 7 shows a snapshoot of anexperimental environment. Through these experiments,
we will evaluate Rebo’s advantages: familiarity,
function awareness, and stroke manipulation.

5. CONCLUSION
In this paper, we proposed a robot-like remote Fig. 7 Experimental environment.
control, Rebo, for controlling home electric appliances
and implemented Rebo for an advanced TV system.
Rebo has three advantages; familiarity, function awareness, and stroke manipulation. Life-like appearance
and facial expression makes it familiar with users.
We implemented four facial expressions as a life-like behavior and video picture animations as
feedback from an advanced TV system to enable users to easily notice its functions. Also we confirmed thecomplete execution of Rebo on the advanced TV system. We have a plan to conduct conducted
experiments with participants to evaluate Rebo’s advantages.

REFERENCES
1.
2.

3.

4.
5.

6.

7.

8.
9.

B. LaPlant, S. Trewin, G. Zimmermann, and G. Vanderheiden, “The universal remote console: A
universal access bus for pervasive computing,” IEEE Pervasive Computing, 2004, pp. 76-80.
K. Kobayashi, Y. Nakagawa, S. Yamada, S. Nakagawa and Y. Saito, “Rebo: A remote control withstrokes,” In Proceedings of the 18th International Workshop on Robot and Human Interactive
Communication, 2009, pp. 751-756.
K. Kobayashi, Y. Kitamura and S. Yamada, “Action sloping as a way for users to notice a robot’s
function,” In Proceedings of the 16th International Workshop on Robot and Human Interactive
Communication, 2007, pp. 445–450.
K. Kobayashi, S. Yamada, and Y. Kitamura, “Action sloping for increasing awareness of robot’s
function,” Transactions of Human Interface Society, No. 10, Vol. 1, 2008, pp. 37-46.
S. Zhao, P. Dragicevic, M. Chignell, R. Balakrishnan, and P. Baudisch, “Earpod: eyes-free menu
selection using touch input and reactive audio feedback”, In Proceedings of the SIGCHI conferenceon Human Factors in Computing Systems, 2007, pp. 1395-1404.
V. Buil, G. Hollemans, S. van de Wijdeven, “Headphones with touch control,” In Proceedings ofthe 7th International Conference on Human Computer Interaction with Mobile Devices&Services,
2005, pp. 377-378.
S. Brewster, J. Lumsden, M. Bell, M. Hall and S. Tasker, “Multimodal ’Eyes-Free’ interaction
techniques for wearable devices,” In Proceedings of the SIGCHI conference on Human factors incomputing systems, 2003, pp. 473-480.
M. Micire, JL. Drury, B. Keyes, HA, Yanco, “Multi-touch interaction for robot control,” In Proc. of
the 13th international conference on Intelligent user interfaces, 2009, pp. 425-428.
H. Osawa, R. Ohmura, and M. Imai, “Using attachable humanoid parts for realizing imaginary
intention and body image,” International Journal of Social Robotics, No. 1, Vol. 1, pp. 109-123.