Directing a Target Person among Multiple Users
using the Motion Effects
of an Image-Based Avatar
Tsubasa Miyauchi1 ,Masashi Nishiyama1 , and Yoshio Iwai1
Graduate School of Engineering, Tottori University
101 Minami 4-chome, Koyama-cho, Tottori, 680-8550, Japanmiyauti.tsubasa51@gmail.com

Abstract. We investigate whether an image-based avatar with motion
can smoothly direct a target person who has requested guidance. Existing methods that use an avatar generally assume a one-to-one interaction
with the user and did not fully consider how the avatar should direct atarget person, which is important in interactions with multiple users.
When the target person feels that the image-based avatar is facing themand feels that the avatar is directing them, the person judges that therequested guidance has been given. When a nontarget person feels thatthe image-based avatar is not facing them and also feels that the avatar
is not directed towards them, the nontarget person then judges that therequested guidance has not come to them. To direct the target personsmoothly, the action-type motion of the image-based avatar and the rotational motion of the video sequence itself relative to the display areimportant. We performed a subjective assessment of how the target person and the nontarget person feel about the direction by comparing thecase where motions were added with the case where motions were notadded. The results show that the image-based avatar with the actiontype and rotation-type motions is effective for the target person but is
not effective for the nontarget person.
Keywords: Image-based avatar, Motion, Direction, Target person, Nontarget person

1Introduction

We have developed an image-based avatar system [1] that interacts naturallywith users. Several image-based avatar systems have been produced over thelast decade [2, 3]. Specifically, we considered a navigation system that guides auser and consists of an avatar shown on a stationary display. The avatar serves
as an alternative to a real guide in an information center located in a public
space, e.g., at an airport or in a bus/rail station.
Before describing this system, we consider an actual navigation situation
involving people in an information center, where a real guide interacts with

II

Fig. 1. Navigation situation.

the users waiting around him/her. Figure 1 shows an example of this type ofsituation. Based on our knowledge of cognitive science [4], the people can be
categorized into the following three roles: a guide that provides directions, aparticipant that receives these directions, and a side participant who is waiting
to receive directions from the guide. A side participant becomes a participant
when they are given directions by the guide.
Here, we consider how the avatar gives information to a single side participant
when multiple side participants are present and waiting to begin interaction withthe avatar. First, multiple side participants are waiting around the avatar for
directions, as illustrated in Fig. 2 (a). In order to provide clear separation of theroles of the multiple side participants, we also newly define the following roles:
a target person who is receiving directions and a nontarget person who is notreceiving directions, as illustrated in Fig. 2 (b). When a target person feels thatthey are being given directions by the avatar, this person changes from a side
participant into a participant, as illustrated in Fig. 2 (c). The nontarget person
remains in their role as a side participant as long as they do not feel that they
are being directed by the avatar. It is important that the avatar only directs thetarget person in the presence of the nontarget person.
We now discuss how to design a method such that the avatar only directs thetarget person, even when a nontarget person is present. Existing methods [5, 6,
7, 8, 9] have used special displays with embedded motion mechanisms. However,
we cannot use this method easily in a stationary display. Caridakis et al. [10] useda stationary display with video sequences that included motion effects when theavatar spoke to the user. We believe that the motion effects provide an importantcue when the avatar and the user interact via a stationary display. Therefore, we
focused on the use of motion effects in the video sequences for the image-based
avatar. However, this method does not sufficiently consider the problem of how
to direct a target person when in the presence of a nontarget person.
In this paper, we investigate the use of motion effects in an image-based
avatar to enable this avatar to give directions smoothly to a target person in thepresence of another nontarget person. To compare methods that add motions to

III
AvatarSide
participantSide
participant

AvatarNontarget
person(a)

AvatarTargetperson(b)SideparticipantParticipant(c)Fig. 2. User role transitions in the proposed navigation system.

the image-based avatar with a method without movement of the image-based
avatar, we evaluated the following two hypotheses:
H1:A target person feels that they are facing the image-based avatar. The target
person then feels that they are being given directions by the image-based
avatar.
H2: A nontarget person does not feel that they are facing the image-based
avatar. The nontarget person thus does not feel that they are being given
directions by the image-based avatar.
We evaluated the effects of an action-type motion (where the face and body of
the avatar move to face the target person) and a rotational motion (where thedisplay rotates to face the target person).

2

Navigation Situation

We assume a situation in which the side participants stand on the right and left
sides of the participant in the information center. Figure 3 shows the flow that
occurs in this situation. At t1 , a participant is guided by an image-based avatar.
At t2 , a side participant who wants to use the image-based avatar arrives theinformation center and stands in an empty space at the side of the participant.
At t3 , another side participant arrives at the information center and stands onthe empty side opposite the first side participant. At t4 , the participant leaves
the information center after receiving the image-based avatar’s guidance. At thispoint, the side participants stand on the right and left sides of the position that
had been occupied by the participant. To direct the target person only, we add
motion effects to the image-based avatar in this situation.

IV

Side
participant

AvatarAvatarParticipantParticipant

t1

t2

AvatarAvatarParticipant

t3

Side
participantSide
participantSide
participantSide
participantt4

Fig. 3. Flow of the situation that we consider in this paper.

3

Motion Effects

3.1

Overview

To direct a target person only, we must consider what motion must be added
to the image-based avatar. Because the image-based avatar resembles the appearance of a person, the image-based avatar is then expected to interact like aperson. However, the image-based avatar is also expected to produce an interaction like that between a person and a machine because the image-based avatar isdisplayed on a display. We therefore added a guide motion and a rotating display
to the image-based avatar. We describe the details of each motion below.
First, we explain how a guide moves to direct a target person only. A guide
talks to a target person after facing that target person. It is important that theguide faces the target person. Figure 4 shows the way in which a guide moves to

V
Guide

Guide

TargetpersonNontarget
person(a)Nontarget
personTargetpersonSide
participantParticipant(b)Fig. 4. How the guide moves to direct the target person only.

direct the target person only. The target person feels that they are facing a guidebecause that guide is facing them (Fig. 4(a)). Furthermore, the target person
feels that they are being given directions because that guide is talking to them(Fig. 4(b)). In contrast, the nontarget person does not feel that they are facing
the guide because that guide is facing the target person (Fig. 4(a)). Furthermore,
the nontarget person does not feel that they are being given directions by thatguide because the guide is talking to the target person (Fig. 4(b)). There are
motions in the face, the body and the eyes in the case where the guide faces thetarget person.
Below, we consider addition of the motions of a guide to the image-based
avatar. When using a video sequence for an image-based avatar, we must consider
the Mona Lisa effect [11, 12], which occurs whenever users see an avatar thatis displayed on a flat panel. This effect causes the users to feel that an avatar
who is facing the camera is actually gazing directly at them. If the avatar faces
the camera, then both the target person and the nontarget person will feel thatthey are facing the avatar simultaneously. To alleviate the Mona Lisa effect, ourmethod uses the fact that the avatar rotates both its face and its body whilegazing at the camera. When an image-based avatar talks to a target person after
rotating its face and body, the target person then feels that they are being given
directions by the image-based avatar.
Next, we explain how to move a rotating display to direct a target person
only. The display rotates towards the target person. It is important that therotating display faces the target person. In this paper, we represent the physicalframe of the rotating display in the video sequence of the image-based avatar.
We can change the appearance of the region of the frame in which the avatar islocated using projective transformation of the video sequence. As a result, thetarget person feels that they are facing an image-based avatar. When the imagebased avatar talks to a target person after the projective transformation, thetarget person then feels that they are being given directions by the image-based
avatar.

VI

Displayθ b 0[deg] θ b
Nontarget
personNontarget
personTarget personθbTarget person

Fig. 5. Angle parameter θb for an action-type motion.

DisplayθfNontarget
person0[deg]Frame

θf

Nontarget
personθfTarget person

Target person

Fig. 6. Angle parameter θf for a rotational motion

3.2

Action-type motion

Action-type motion is used to describe the scenario where the image-based avatar
has a guiding motion added to the video sequence. In this case, the image-based
avatar rotates its face and body while its gaze remains fixed towards the camera. Note that the image-based avatar rotates its face and body in conjunction
because rotation of its face and body individually would occur only rarely in theinformation center. Figure 5 shows the angle parameter θb for an action-type
motion. θb is the angle of the rotating body. θb sets the front direction to 0
degrees. When the target person is standing on the right side of the avatar, θb
has a positive value. When the target person is standing on the left side of theavatar, θb has a negative value.
3.3

Rotational Motion

Rotational motion is used to describe the situation where the motion of a rotating
display is added to the image-based avatar in the video sequence. Changes in theappearance of the subject region in the frame and the frame itself are expressedVIIby projective transformation. Figure 6 shows the angle parameter θf of therotational motion. θf is the angle of the rotating frame. θf sets the front direction
to 0 degrees. When the target person is standing on the right side of the avatar,θf has a positive value. When the target person is standing on the left side of
the avatar, θf has a negative value.
3.4

Combination of an Action-type Motion with a Rotational
MotionNext, we need to consider combination of an action-type motion with a rotationalmotion. A rotational motion is added to an image-based avatar during an actiontype motion. This combination of the action-type motion and the rotationalmotion has an angle parameter composed of θb0 and θf0 . We set θb0 = θb /2 andθf0 = θf /2.

4

Subjective Assessment

4.1

Experimental Conditions

We performed a subjective assessment to investigate the hypotheses described in
Section 1. In addition, we performed another subjective assessment to investigate
the impression that is made by the image-based avatar. Twenty subjects (17males and 3 females, with an average age of 22.2±1.1 years) participated in thisassessment. We compared the following four motion methods for the image-based
avatar:M1: No motion effects
M2: Action-type motion
M3: Rotational motion
M4: Both motion types
Fig. 7 shows examples of these methods.
The 20 subjects were split randomly into pairs to view a video sequence ofan image-based avatar for each method. One subject was assigned the role of thetarget person and the other subject was assigned the nontarget person role. The
two subjects then stood to the side of the display. Figure 4 shows the standing
positions of the two subjects. After viewing the video sequence, the subjects
then answered the following questions:
Q1: Did you feel that you faced an image-based avatar?
Q2: Did you feel that you were given directions by an image-based avatar?
Q3: Did you feel that you interacted smoothly with the image-based avatar?
Q4: Did you feel that the image-based avatar interacted politely with you?
Q5: Did you feel that the image-based avatar interacted nicely with you?
Each subject provided a rated score using four response levels (-1.5: disagreeable;-0.5: slightly disagreeable; 0.5: fairly agreeable; 1.5: agreeable) for each question.
We also asked the reverse questions to Q1 to Q4. We set θb = 15[deg] andθf = 15[deg]. We used two-way analysis of variance (ANOVA) and the Wilcoxon
signed-rank test to evaluate the test results.

VIII

M1

M2

M3

M4
Time
Fig. 7. Image-based avatars M1 to M4 with and without motion effects for the subjective assessment.

IX

Q2

Q1

**
**

**

**

**
**

1.5

**

** ：p<0.01

**

**

1.5

1

1

0.5

0.5

0

0

−0.5

−0.5

−1

−1

−1.5

−1.5

M1

M2

M3

M4

M1

Interaction of M2 with M3
Simple main effect of M2
Simple main effect of M3

M2

M3

M4

Interaction of M2 with M3
Simple main effect of M2
Simple main effect of M3(a) Target person
Q1

Q2

*

*

1.5

*

* ：p<0.05

1.5

1

1

0.5

0.5

0

0

−0.5

−0.5

−1

−1

−1.5

−1.5

M1

M2

M3

M4

Interaction of M2 with M3
No simple main effect of M2
No simple main effect of M3

M1

M2

M3

M4

No interaction of M2 with M3
No simple main effect of M2
No simple main effect of M3(b) Nontarget person
Fig. 8. Results of subjective assessments of Q1 and Q2.

4.2

Results of subjective assessment

Figure 8 (a) shows the subjective scores for Q1 and Q2 for the target persons.
The high subjective scores indicate agreement among the target persons. Additionally, there are significant differences among M1 and the other three methods.
Therefore, we can claim that H1, as described in Section 1, is valid under thecondition that motion effects were used. Figure 8 (b) shows the subjective scoresfor Q1 and Q2 for the nontarget persons. The low subjective scores also indicateagreement among the nontarget persons. There were no significant differencesamong M1 and the other three methods. Therefore, we cannot claim that H2,

X
Q3

Q4

**
**

**

**

**

**

**

1.5

**

** ：p<0.01

**
**

1.5

1

1

0.5

0.5

0

0

−0.5

−0.5

−1

−1

−1.5

−1.5

M1

M2

M3

M4

M1

Interaction of M2 with M3
Simple main effect of M2
Simple main effect of M3

M2

M3

M4

Interaction of M2 with M3
Simple main effect of M2
Simple main effect of M3
Q5

**
**

**

**
**

1.5
1
0.5

0
−0.5
−1
−1.5

M1

M2

M3

M4

Interaction of M2 with M3
Simple main effect of M2
Simple main effect of M3

Target person
Fig. 9. Results of subjective assessments of Q3 to Q5 for the target persons.

as described in Section 1, was valid under the condition that motion effects are
used.
Figure 9 shows the subjective scores for Q3 to Q5 for the target persons. The
high subjective scores again indicate agreement among the target persons. Additionally, there are significant differences among M1 and the other three methods.
Therefore, we can claim that the image-based avatar made a good impression
under the conditions where motion effects were used. Figure 10 shows the subjective scores for Q3 and Q5 for the nontarget persons. The low subjective scoresalso indicate agreement among the nontarget persons. There were no significant
differences in this case among M1 and the other three methods. Therefore, we

XI
Q3

Q4

1.5

1.5

1

1

0.5

0.5

0

0

−0.5

−0.5

−1

−1

−1.5

−1.5

M1

M2

M3

M4

No interaction of M2 with M3
No simple main effect of M2
No simple main effect of M3

M1

M2

M3

M4

No interaction of M2 with M3
No simple main effect of M2
No simple main effect of M3
Q5

1.5
1
0.5
0
−0.5
−1
−1.5

M1

M2

M3

M4

No interaction of M2 with M3
No simple main effect of M2
No simple main effect of M3

Nontarget person
Fig. 10. Results of subjective assessments of Q3 to Q5 for the nontarget persons.

cannot claim that the image-based avatar made a good impression under theconditions where motion effects were used.

5

Conclusions

We have proposed a method to ensure that an image-based avatar only directsa specific target person. We added an action effect and a rotation effect to an
image-based avatar. We then performed a subjective assessment to compare themethods that add the effects to the image-based avatar with a method without
movement in the image-based avatar. The results of a subjective assessment
showed that a target person feels that they are being given directions by theimage-based avatar when the motion effects are used. In addition, the imagebased avatar also made a good impression on the target person during the useof motion effects. However, the results of the subjective assessment did not show

XII

that the nontarget person did not feel that they were being given directions
by the image-based avatar. In future work, we will consider motion effects inanother situation in an information center, and we will also compare motion
effects and special displays.

References[1] Miyauchi, T., Ono, A., Yoshimura, H., Nishiyama, M., Iwai, Y.: Embedding theawareness state and response state in an image-based avatar to start natural userinteraction. IEICE Transactions on Information and Systems E100-D(12) (2017)
3045–3049[2] Artstein, R., Traum, D., Alexander, O., Leuski, A., Jones, A., Georgila, K., Debevec, P., Swartout, W., Maio, H., Smith, S.: Time-offset interaction with a holocaust survivor. In: Proceedings of the 19th International Conference on Intelligent
User Interfaces. (2014) 163–168[3] Jones, A., Unger, J., Nagano, K., Busch, J., Yu, X., Peng, H.I., Alexander, O.,
Bolas, M., Debevec, P.: An automultiscopic projector array for interactive digital
humans. In: Proceedings of ACM SIGGRAPH Emerging Technologies. (2015) 6:1[4] Clark, H.H., Carlson, T.B.: Hearers and speech acts. Language 58 (1982) 332–373[5] Otsuki, M., Kawano, T., Maruyama, K., Kuzuoka, H., Suzuki, Y.: Thirdeye:
Simple add-on display to represent remote participant’s gaze direction in videocommunication. CHI ’17 Proceedings of the CHI Conference on Human Factors
in Computing Systems (2017) 5307–5312[6] Kawaguchi, I., Kuzuoka, H., Suzuki, Y.: Study on gaze direction perception of face
image displayed on rotatable flat display. CHI ’15 Proceedings of the 33rd Annual
ACM Conference on Human Factors in Computing Systems (2015) 1729–1737[7] Yankelovich, N., Simpson, N., Kaplan, J., Provino, J.: Porta-person: Telepresence
for the connected conference room. CHI’07 Extended Abstracts on Human Factors
in Computing Systems (2007)[8] Adalgeirsson, S.O., Breazeal, C.: Mebot: Arobotic platform for socially embodied
presence. In Proceedings of 5th ACM/IEEE International Conference on Humanrobot Interaction (2010) 15–22[9] Onishi, Y., Tanaka, K., Nakanishi, H.: Embodiment of video-mediated communication enhances social telepresence. In: Proceedings of the Fourth International
Conference on Human Agent Interaction. (2016) 171–178[10] Caridakis, G.: Virtual agent multimodal mimicry of humans. Language Resources
and Evaluation 41(3-4) (2007) 367–388[11] Masame, K.: Perception of where a person is looking: Overestimation and understimation of gaze direction. Tohoku Psychologica Folia 49 (1990) 33–41[12] Kendon, A.: Some functions of gaze-direction in social interaction. Acta Psychologica 26 (1967) 22–63