Proposal of a Script Language for Embodied Conversational
Agents as Asynchronous Conversational MediaToru TAKAHASHI†, ‡†Hideaki TAKEDA*, †Graduate School of Information Science, Nara Institute of Science and Technology,
8916-5 Takayama-cho, Ikoma-city, Nara 630-0101, JAPAN+81 743 72 5265
‡ATR Media Information Science Laboratories,
2-2-2 Seika-cho, Soraku-gun, Kyoto 619-0288, JAPAN+81 774 95 1486
* National Institute of Informatics,
2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo 101-8430, JAPAN+81 3 4212 2543

toru@atr.co.jp

takeda@nii.ac.jp
communities are located in the “information space.”
Our goal is to make online communities more communicativeand more social place, and we think embodied conversational
agents (ECAs) [1] are one of the most probable solutions. In an
asynchronous online community system like TelMeA [2], one ormore ECAs are employed as personal conversational media. In
this paper, we use the term Personified Media, or PM, for theECAs that function as personal conversational media.
Asynchronous conversation with PM enables people tocommunicate reflectively through multimodal behavior and toexplicitly share web pages among them.
Most script languages used to describe the behaviors ofautonomous interface agents are designed to describe a scenario
of interaction logically and consistently. That is, they arelanguages for developers of agent systems. On the other hand,
script languages for PM are languages for users to communicate.
Therefore, the languages for PM should be designed so that users
can describe utterances naturally and adequately.
In this paper, we propose a new script language named PMScript.
One PMScript description represents one utterance, andconversations are built through asynchronously exchanging thePMScript descriptions instead of the texts in the case of
newsgroups and BBS. PMScript supports and enhances naturalsocial interaction in a community, by enabling its users to create
multimodal utterances easily.

ABSTRACTIn this paper, we propose a script language for Personified Media(PM). PM is a kind of embodied conversational agent that canfunction as a personal conversational medium in asynchronous
community systems. In our community system, participants havetheir personal PM to communicate with others. Each participant
composes multimodal utterances towards his/her own PM in ascript language and submits the utterances to the community
system by turns. Because the script language of PM is not alanguage for agent system developers but for participants foronline communities, the language should have a structure thatusers can compose utterances easily and naturally.

Categories and Subject Descriptors
H.4.3 [Information Systems Application]: Communications
Applications; H.5.3 [Information Interfaces and Presentation]:Group and Organization Interfaces

General Terms
Design, Languages

Keywords
interface agents & conversational agents, agent language andenvironments, online community, asynchronous communication

1. INTRODUCTION
When communicating with others, people represent theirintentions integrating their full-body modalities such as voice,gesture, pose, gaze, and motion. Additionally, people use sharedcontext around them including information brought by the media.
In online communities formed with information systems such as
newsgroups and bulletin board system (BBS), however, people
usually communicate only by text. There, people communicate
without richly expressive faces and hands and only in words.
People implicitly share web contents by including links to web
pages; people cannot explicitly share the web pages while the2. PROPOSAL FOR A NEW SCRIPTLANGUAGE FOR PM
PMScript is currently provided in XML syntax. Figure 1 shows
the structure tree of PMScript. As the figure shows, the content of
PMScript consists of two elements, i.e., CAST element andSCRIPT element.

2.1 CAST element
The CAST element presents a list of PM and multimedia contents
that are involved in the Script element. PM of the list arepresented in the PM element and multimedia contents of the list
are presented in the CONTENT elements.

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,requires prior specific permission and/or a fee.
AAMAS’02, July 15-19, 2002, Bologna, Italy.
Copyright 2002 ACM 1-58113-480-0/02/0007…$5.00.

The CAST element has only one attribute, that is, the ID attribute.
The ID attribute is an identifier of elements. Most elements inPMScript have this attribute, and it simply shows the identifier, sowe will omit the explanation of the ID attribute hereafter.

1the user intend to express by performing this physicalrepresentation. For instance, when a “smile” animation is selected
in a utterance, the PERFORMATIVE attribute indicates that this“smile” could specify either an acceptation or a sarcasm.

PM _SCRIPTID

SCRIPTCASTACTO RPM

2.5 VERBAL Element
The VERBAL element presents linguistic representation such asballoon text and synthesized voice via PM. In addition to ID,ANIMATION, and PERFORMATIVE, the VERBAL element has
CONTENT attribute. The CONTENT attribute presents thelinguistic sentences for this representation.

CONTENTID
USERURIID
TYPE
URI

PHYSICAL

EXISTENTIAL

M EDIAIDANIM ATIO NPERFO RM ATIVEID
STATUSANIM ATIONPERFO RM ATIVEID
TARG ET
ANIM ATIO NPERFORM ATIVEVERBALM OVINGID
CO NTENT
ANIM ATIO NPERFO RM ATIVEID
TARG ET
ANIM ATIO NPERFORM ATIVE

2.6 MOVING Element
The MOVING element presents representation through themovement of the PM on the display. The MOVING element hasTARGET attribute other than ID, ANIMATION, andPERFORMATIVE. The TARGET attribute presents themovement destination point of the PM, which is specified in termsof either an ID attribute of a target PM or a target CONTENTelement, or the coordinate value of a target position on the display.

Figure 1. Structure tree of PMScript

2.7 EXISTENTIAL Element
The EXISTENTIAL element presents a change of a state of a PM,such as size of PM, appearance of PM, or the state of whether thePM is displayed or not. The EXISTENTIAL element, for example,
may make the PM bigger, reclothe the PM, or hide the PM. In
addition to ID, ANIMATION, and PERFORMATIVE, theEXISTENTIAL element has a STATE attribute. The STATE
attribute presents the state of PM after this element is performed.

2.2 PM element and CONTENT element
The PM element presents PM of a participant in this script, andthe CONTENT element presents multimodal content as well. ThePM element and CONTENT element each have three attributes:
ID, USER, and URI attributes for the PM element, and ID, TYPE,and URI attributes for the CONTENT element. The USER
attribute of the PM element presents the user of the PM. TheTYPE element of the CONTENT element presents the MIME
content type of this content. The URI attribute of both elements
presents the URI (Uniform Resource Identifier) of thisPM/content.

2.8 MEDIA Element
The MEDIA element presents representation with multimedia
contents. In addition to ID, ANIMATION, and PERFORMATIVE,the MEDIA element has TARGET attribute. The TARGET
attribute presents the target of a multimedia content to operatesuch as letter string, image, movies, web pages, and so on. Thisvalue of the TARGET attribute is designated as an ID attribute ofthe CONTENT element.

2.3 SCRIPT Element
The SCRIPT element has only one attribute, The ACTORattribute. The utterer PM of the script is designated by thisACTOR attribute in the SCRIPT element. The SCRIPT element
presents the context of the utterance as a combination of its subelements, i.e., VERBAL, PHYSICAL, MOVING, EXISTENTIAL,and MEDIA elements. Each element presents representations byPM and they are classified by modalities. Each element has ID
ANIMATION, and PERFORMATIVE attributes. The ID attributeis an identifier of each element. The ANIMATION attributedesignates PM’s animation including, for example, lip-sync
animation when PM speaks verbally and stepping animation when
PM moves around. The PERFORMATIVE attribute indicatesintention of performing the representation. The purpose of this
attribute is to clarify the intentions of speakers when they
compose the script. For instance, we suppose users select theirbehavior not from the representation of animations but from thePERFORMATIVE, that is, the intention of this behavior. The
same behavior often has different meanings in various contexts.

3. CONCLUSION
In this paper, we propose a script language for ECA that canfunction as the personal conversational media on asynchronous
community systems. The benefits of PMScript are that it (1)provides a standard protocol that enables different PM-based
community systems, and (2) provides a good structure ofutterances in order to reuse them. Future works around PMScript
is practice and evaluation. We are currently working to adapt ournew community system (cf. [2]) using PM to be able tounderstand and process PMScript, and we are going to propose to
the agent research community that the specification of PMScript
be used as the standard in order to stimulate further discussion.

4. ACKNOWLEDGEMENT
The authors wish to thank Dr. Yasuhiro Katagiri of ATR MediaInformation Science Labs for making constructive comments forimproving this paper. This research was supported in part by theTelecommunications Advancement Organization.

We explain details of each element below.

2.4 PHYSICAL Element
The PHYSICAL element presents a representation of a physicalmodality by animation, such as gestures, facial expressions, andposes. The PHYSICAL element has ID, ANIMATION, andPERFORMATIVE attributes. The ANIMATION attributeindicates the entity of the animation, which shows a sequence ofphysical movement of PM. On the other hand, thePERFORMATIVE attribute presents the user’s intention which

5. REFERENCES[1] Cassell, J., Sullivan, J., Prevost, S., and Churchill, E. (eds.),
Embodied Conversational Agents, Cambridge; MIT Press, 2000.

[2] Takahashi, T. and Takeda, H., TelMeA: An Asynchronous
Community System with Avatar-like Agents, Proc. INTERACT2001,
pp. 480-487, 2001.

2