<figure><img src="image_1.png" /></figure>

HAL
open science

# Emotive Expression through the Movement of Interactive Robotic Vehicles

Eric Kryski, Ehud Sharlin

► To cite this version:

Eric Kryski, Ehud Sharlin. Emotive Expression through the Movement of Interactive Robotic Vehicles. 13th International Conference on Human-Computer Interaction (INTERACT), Sep 2011, Lisbon, Portugal. pp.91-99, 10.1007/978-3-642-23765-2_7 . hal-01591835

HAL Id: hal-01591835

https://inria.hal.science/hal-01591835

Submitted on 22 Sep 2017

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Emotive Expression Through the Movement of
## Interactive Robotic Vehicles

Eric Kryski, Ehud Sharlin

University of Calgary
{epkryski, ehud}@ucalgary.ca

**Abstract.** In this paper, we discuss our ongoing design of interactive personal vehicles that exhibit behavioral constructs expressed through motion in order to improve the user's commuting experience. The behavioral, personality-like traits demonstrated by the interactive vehicles are intended to be useful and helpful, as well as to stretch beyond the effectiveness into affect and emotion, creating an overall more satisfying experience for the user. This short paper presents our design goals and approach, describes the evolution of the implementation of our personal vehicle prototypes, and outlines our current preliminary design critique evaluation findings.

**Keywords:** Social human-robot interaction (HRI), Vehicle personality, Emotional expression through motion.

## 1 Introduction

Vehicle personality may seem like an abstract concept but in reality many people already treat their vehicles as social entities which possess the ability to understand and respond to their requests. Humans can have distinct emotional connections to inanimate objects, in the same way that they may have an emotional connection to a pet or child. Emotions that are evoked by these relationships can range from anger to fear to pleasure, and can play a fundamental role in the way we operate our vehicles [1].

Furthermore, the interactions we have with our vehicles can be viewed as a reminiscence of the symbiotic relationship between humanity and its classic modes of transportation, for example, the close relationships between handlers of camels or horses. However, although we can argue that there is some emotional connection between owners and vehicles, a connection strong enough, for example, to sometime persuade owners to name their vehicle, we believe that modern interactive vehicles could elevate this emotional connection by using synthetic personality and emotive expression. Arguably, if we can bring this heightened level of emotional connection to the way we interact with our vehicles it could potentially greatly enhance our commuting experience while simultaneously solidifying humanity's 'love affair' with our vehicles [2].

The purpose of our research is to design an interactive personal vehicle that exhibits emotional expression through motion, and to discover how humans will react
to such a vehicle. That is, more specifically, what behavior types exhibited by the
vehicle motion and movement would be acceptable to humans? Which displays of
behavior are beneficial and which ones pose a negative impact on human-vehicle
interaction? Our work was inspired by previous efforts in the domain, for example it
has been shown that matching a driver's emotion to the perceived emotion of a
simulated car voice can improve the driver performance and safety [3]. Our work
however is an attempt to explore the emotional connection between human and
vehicle through a more basic and entirely non-verbal layer: emotional expression as
expressed via the vehicle motion. Our intent is not to replace other modalities such as
visual or auditory, but rather to discern what physical emotive cues could supplement
these other modalities. In this paper we examine three different classes of behavioral
constructs (*semi-autonomous*, *cooperative*, and *training or taming*), which we argue
are applicable to emotive vehicle expression through motion. We present the personal
vehicle testbed prototypes we designed and implemented in order to probe these
behaviors and their effect on users, and present our preliminary evaluation of one of
these classes, as a simple testcase of emotive expression exhibited through motion.

## 2 Background

Media and technology provide many examples of convincing displays of personality,
conveyed by seemingly inanimate objects, for example in computer games, film, and
robotics. In film, viewers have been persuaded to emotionally reflect on a caring
teapot, an inquisitive lamp, or a heartbroken robot. Even if these objects cannot talk or
do not possess any facial features audiences were still able to associate them with
personality and emotive expression. Reeves and Nass suggested that associating
human characteristics to objects is an inherently human trait [4], and Heider and
Simmel in their seminal research [5] outlined that people can relate emotion, gender,
and motivation solely by perceiving an objects motion. Since movement is
fundamental to robotic interfaces the above findings can provide crucial insight on
how to design socially acceptable robots.

Based on this previous research, recent efforts are pursuing possible mapping
between motion and emotion. In Saerbeck and Bartneck's research it was suggested
that humans perceive a robot's movements in different ways, depending on the
acceleration value and curvature [8]. Another effort presented a preliminary study
about how abstract emotive actuation, when exhibited by a robot, is perceived by
people [6], showing that people made conclusions as to what an abstract object was
feeling, its intent and even its gender, based upon how fast the object moved, in which
directions, and how quickly it accelerated.

## 3 Classes of Behavioral Expression

Since interpersonal communication relies heavily upon body language [5] it is
plausible to believe that motion is a primary contributing factor to people's perception
of emotion and intent. It seems realistic to think that in order to simulate human
emotions robotic vehicles should apply behavioral contexts that are clear and apparent to humans. This is the intent of our research: to discover which behavioral expressions, conveyed via motion, are acceptable for a vehicle, and why. We argue that physical interaction can be more subtle, but also more effective, than either visual or auditory methods. We also believe that humans are acquainted with subtleties, and that interfaces that can express these subtleties can ultimately be more effective, and enhance the interaction experience, without necessarily overruling other, less subtle, modalities. Below we introduce a proposed set of applicable behavior classes that a vehicle may be able to exhibit during emotive movement.

**Semi-Autonomous Behavior.** During the semi-autonomous behavior the vehicle navigates or transports the user where they wish or are required to go. It is merely a reactionary behavior where the primary purpose is to transport the user but also to alarm the user, for example via abrupt motion or vibration, of any information that is of importance. This personality profile allows the user to focus on other tasks as opposed to navigating, but still maintains situational awareness via subtle motion-based expression.

**Taming/Training Behavior.** In this behavioral state the vehicle initially begins as a blank slate that may or may not behave the way the user wishes. Similar to a puppy dog or a horse, the vehicle must be “broken” or trained to respond the way the user would like them to. The vehicle may, for example, move too fast for the user who prefers a more cautious style of movement. Conversely, the user may want the vehicle to move in a more aggressive manner with quick accelerations and more abrupt stops. The main goal of the taming/training behavioral type is not to be reckless or unsafe, but rather to afford the user with a sense of customization, companionship and pride of ownership akin to if one were to train a horse or a dog. It should be noted that similarly to training an animal, the training of an emotively expressive vehicle would also be performed in a secure environment before the vehicle is publicly exposed.

**Cooperative Behavior.** During this behavioral state the user and the vehicle collaborate to enhance the user's experience, and help the user fulfill a predefined set of tasks (such as reaching her meetings on time). The vehicle may exhibit portions of the semi-autonomous state or may be completely passive until the user wishes to perform some action that is not in their best interest according to the vehicle's HRI awareness [9]. The primary goal of the cooperative behavior type is to perform every action in the best interest of the user. The vehicle accomplishes this goal by utilizing a set of physical gestures or movements that subtly, or explicitly direct the user to predefined areas to complete predetermined tasks or physically removing the user from an unwanted dangerous situation.

# 4 Prototype Design

In order to practically evaluate our emotive vehicle expression through motion research question, and to examine the three behavioral types we outlined above, we designed two prototype personal robotic vehicles.

Our initial inspiration came from Segway™ but we ended up opting to use another widely used personal vehicle; an electric wheelchair. However, we did not want to
limit the discussion scope to wheelchair bound individuals. As a result, we developed two prototypes based upon an electric wheelchair platform, one to be operated in the seated position and one in the standing position (Fig. 1A, 1B). In order to support rapid prototyping and critical evaluation of our concept, rather than a sophisticated technical contribution, we decided to use the 'Wizard of Oz' prototyping approach [10] as it greatly reduced the vehicle complexity and allowed us to develop a functional prototype in a short period of time. The shortcoming of our approach is that we deferred many of the actual complex implementation challenges associated with our interaction design, and focused our contribution only on the fundamental interaction concept.

## 4.1 Mechanics

We have designed two prototype vehicles, a sitting (Fig. 1A) and a standing platform (Fig. 1B), both of which are based upon an electric wheelchair structure. We chose an electric wheelchair because it is already viewed as a personal vehicle, it was easy to modify for our needs, and electric wheelchairs have the ability to operate in many different environments. That is, they are not constricted by legal issues in the same way that a Segway™ or an automobile is.

<figure><img src="image_3.png" /><figcaption>Figure 1A. Seated Platform</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Figure 1B. Standing Platform</figcaption></figure>

In order to support our 'Wizard of Oz' prototyping approach both vehicles use a Nintendo Wiimote with Nunchuk extension as input for both the user and the wizard. Each controller is connected via Bluetooth to an onboard vehicle PC. In order to get the desired wireless range, so that the wizard is not detected by the user, we used a generic class 1 bluetooth dongle, which provided an approximate operating range of 50 meters indoors.
After receiving information from the Wiimotes, the PC communicates with the vehicles' Dimension Engineering Sabertooth 2X25A motor driver via a USB to TTL cable. For safety reasons both prototypes include a momentary "on" push button kill switch, in between the battery ground and the motor driver ground, a button that the user must keep constantly pressed using their foot in order for the vehicle to operate.

## 4.2 Wizard-User Control

Control of our prototypes were based on a "Wizard of Oz" puppeteering method, wherein the user was unaware that the vehicle was at times controlled by another human using a second Nintendo Wiimote. The intent was to allow the 'wizard' to provide input on behalf of the vehicle, deceiving the user into thinking that it is actually the vehicle that is deciding and acting out the movements. As previously mentioned we opted for the 'Wizard of Oz' approach because it allowed us to perform rapid prototyping and it greatly reduced the complexity of the vehicle while still maintaining the ability to get user feedback [10] regarding how a vehicle's motion impacts the user's perception of a vehicle's intent and emotional behavior. A negative aspect of using this technique is that the wizard needs to deceive the user, which requires considerable awareness of the user intentions and puppeteering skills, both prone to human error. This can be especially problematic when dealing with a larger robot that is responsible for transporting a human, however this risk was mitigated in part by the user's on vehicle kill switch as well as by the wizard's ability to cease the vehicle locomotion at any point.

Upon initial implementation a decision was made to give the user full control until the wizard pressed a button on their Wiimote and overrode the user's control. At first glance this seemed appropriate but it quickly became apparent that when the user had no control at all they thought that there was a malfunction or they completely gave up on controlling the machine.

As an alternative to this solution we decided to, when the wizard interjects, give the user half the power of the wizard. This way the user can, if they so choose, "fight" the vehicle to an extent, which is controlled by the wizard, but the wizard will still be able to guide the vehicle in the direction that they intend. We believe that this solution will be more readily embraced by the user as it still leaves them with some sense of control over the vehicle regardless of its emotive expression.

# 5 Evaluation

For the design critique we chose to evaluate the cooperative behavior class, primarily because this behavioral expression seemed the most practical to implement and we hypothesized that users would be able to more readily grasp the concept of a "personal assistant" or "helping robotic vehicle". As well, for the prototyping effort we believed that it was the simplest to execute and the easiest to evaluate. As such, presented are two selected possible scenarios which inspired our design, both demonstrate how the cooperative behavior type may be beneficial to a user:
**Scenario 1.** The user is at an airport and their flight has been delayed. They have decided to take their personal vehicle to peruse the shops in the airport in order to pass the time. Being preoccupied, the user is unaware of an announcement that their departure time has been moved up. The vehicle however, could be aware of the flight schedule and may begin to subtly physically nudge the user towards the direction of the departure gate. The user may decide to correct the vehicle and return to the shop but as it gets closer to the departure time the vehicle may become more persistent and assertive. Alternatively, if the user is continually correcting the vehicle's behavior, it may simply give up on its attempts and silently obey.

**Scenario 2.** The user is operating the vehicle inside a corporate building and there is an explosion. Alarms are going off, inaudible announcements are being made over the speaker system and smoke is beginning to spread. The user is now distressed and there is public panic. Since some people have reduced rational decision making ability during times of emergency or high stress [8] the vehicle would assume an authority role and immediately guide the user to safety without requiring user guidance or input. The vehicle may appear protective, brave, or confident by moving in an abrupt direct manner, thus instilling confidence in the user.

During these two very different scenarios the motive displays of emotion by the vehicle, such as being brave or helpful, will vary situationally between passive and aggressive. Consequently, even though we are dealing with the cooperative behavior class the assertiveness or passiveness of the vehicle's movement is an important factor in determining the appropriate motive emotion for each situation. As such, this is something that we took into consideration when selecting the cooperative behavior class for our prototypes.

For the preliminary evaluation of our prototypes we performed a design critique using some of our design team members acting as users. The obvious limitation of our approach was that the users were informed of our prototyping approach. However, we still believe that our findings, outlined in this section, provide important insight on how to proceed pursuing our research question.

For our preliminary design critique evaluation we decided to use the cooperative personality type in lab setting, and accordingly opted to develop a scenario where it makes sense that the vehicle would exhibit a cooperative personality. As such, we devised a scenario where the user is required to complete a certain set of pre-defined daily lab tasks, such as picking up a print out from the printer room, in an allotted period of time. We constructed this scenario based on what we thought would be a meaningful and relatively valid task for most participants, that is, many people perform similar chores on a daily basis. It should be noted however, that our participants did not integrate these vehicles into their daily lives. Our rationale for selecting the cooperative behavior class was that it required the least amount of human learning and we hypothesized that users would be able to more readily associate with a cooperative behavior. The design critique was conducted with two members of our research lab and one engineering student external to our lab, using the seated platform. All of the participants were involved and familiar with our research and hence these findings can be viewed only as a design critique, not a user study.
**5.1 Prototype Findings**

Overall the prototypes operated quite well and both vehicles can be operated
smoothly or in a jerking manner. On the mechanical end, the seated platform is quite
stable, even at high speeds, but the standing platform can be quite tippy in the lateral
direction. A larger base, lower center of gravity, or wider wheels will increase the
force it takes to tip the standing vehicle. Taking this safety concern into consideration
is why we opted to use the seated platform for the design critique.

**5.2 Interaction Findings**

After the initial trial we found that the participants were often unaware of the
vehicle's intention, which caused them to become confused, frustrated or fearful. As
such, we opted to revise the design critique scenario to get the user to perform a series
of tasks in the same fashion, except that some key information required to complete
some of the tasks, would be missing. Since the vehicle's intended behavior was to be
cooperative, we hypothesized that having a few missing items provided a more
perceivable rational motivation for why the vehicle would interject. Thus, if the
vehicle's cooperative behavior is performed convincingly the participant could
potentially increase their level of trust in the vehicle, and may be more in tune to its
gestures. Although this approach provides a reasonable way for preliminary reflection
on usability and influence of the perceived vehicle behavior on the user, it also
highlighted some shortcomings. We had found that it was imperative that the task-
related items and their locations be clearly marked and visible. When they were not,
the participants became easily confused or frustrated, regardless of the vehicle's
attempt to help. Also, we have found that our users overall reflection was that the
seated platform was sometimes inhibiting, with participants often wishing to get out
of the vehicle to look for missing items.

Furthermore, the wizard had to be extremely skilled with the joystick control and
adapting to the participant's actions. The wizard must be very aware of when the
right time to interject is and how much force to apply. If the timing is incorrect or the
force is too strong the participant may think that there is a vehicle malfunction.
Ultimately there is the very real possibility that the participant will not even
understand that the vehicle is trying to hint at anything as opposed to just behaving
abnormally.

Finally, we discovered that without an initial process of gaining trust in the vehicle the participant was generally unaware of what the movements could mean. There is some learning by the participant that should take place before the vehicle's interjection movements can become meaningful. The participant must be able to recognize that the movement by the vehicle is intentional, and discern what its intent is. Initially this is unclear and for some participants we discovered that the process is almost bound to fail, unless there is a mechanism that allows them to gain trust in the vehicle.
Conclusion

We think that the new emerging area of human vehicle interaction can benefit from
the introduction of emotive layers of interaction between the user and the vehicle. We
believe that our work on emotive expression through movement can potentially
introduce a subtle dimension to how we interact with our vehicles, and might lead to
further research that can help strengthen the bond we have with our transportation
means. Humans have had personal connections with their horses or camels for
centuries because of the simple fact these modes of transportation are alive, and
behave as such. The animal exhibits personality, has a mind of its own and expresses
emotions, but respects the power of its master. The human is responsible for the well-
being of the animal so in turn they both serve each other functionally, as well as, to
some extent, emotionally. If we can emulate that same experience with future
vehicles we believe that humans could start to develop stronger, more affectionate
bonds with their vehicles, similar to those that exist historically with other zoological
modes of transportation. We believe that such bonds can improve the commuting
experience, make it more interactive, more effective, and ultimately more enjoyable.

The implementation effort we have reported in this short paper is preliminary, and
needs to be improved. In the long run we would like to replace the Wizard of Oz with
an autonomous vehicle. Furthermore, it would be beneficial to further explore the
other proposed behavioral classes we envisioned: *semi-autonomous* and *training or
taming*.

We believe that trust will play an inherently important role in interaction with
vehicular robots. When a machine is responsible for safely transporting users, taking
control of their movement through space, trust becomes essential. Thus, in order to
build trust while expressing behavior or emotion through movement, the vehicle must
be able to convey the intent of a movement to the user. We believe that users would
then be inclined to assign some emotion to the movement of the vehicle, whether it be
angry, lazy or confused as long as they are not thinking the vehicle is simply
malfunctioning. Unfortunately, especially without familiarity, an unforeseen
movement is generally interpreted as an error. We believe that based on our
preliminary findings the main challenge facing the design of emotive expression
through motion in robotic vehicles is getting over the trust barrier. However, like any
relationship this may simply be a matter of time and patience, and repeated exposure
to the vehicle may be beneficial to users as it would increase familiarity and hopefully
eventually increase trust as well.

References

1. Sheller, M. *Automotive Emotions: Feeling the Car*. Theory Culture Society 2004 21: 221. 2004.

2. Sachs, A. *For Love of the Automobile: Looking Back into the History of Our Desires*. Berkeley, 2002.

3. Nass, et al. *Improving automotive safety by pairing driver emotion and car voice emotion*. CHI 2005

4. Reeves, B, Nass, C. *The Media Equation*. New York: Cambridge University Press. 1996.

5. Heider, F, Simmel, M. *An Experimental Study of Apparent Behaviour*. American Journal of Psychology, vol. 57, pp. 243-259. 1944.

6. Harris, J, Sharlin, E. *Exploring Emotive Actuation and Its Role in Human-Robot Interaction*, HRI 2010

7. Kowalski-Trakofler KM, Vaught C, Scharf T. *Judgment and Decision Making Under Stress: An Overview for Emergency Managers*. International Journal Emergency Management 2003; 1:278–89.

8. Saerbeck, M. and Bartneck, C. *Perception of Affect Elicited by Robot Motion*, HRI 2010

9. Drury J.L, et al. *Awareness in human-robot interactions*. IEEE Systems, Man and Cybernetics 2003

10. Dahlback N, et al. *Wizard of Oz Studies - Why and How*, in Proceedings of IUI '93, ACM, 193-200