Keywords

Abstract

Introduction
1
In these several years, a lot of studies have been done on a life-like agent like a software agent[7][6] and a robot agent[11][9]. A typical life-like agent appears on a Web shopping page and assists a user in inputting his/her order. Through the development of various life-like agents, an agent's mind , an internal state ¹ representing emotion, processing load has been recog- nized to play a very important role in making them believable to a user[2]. Thus researchers are trying to implement a mind model on an agent for making it Tomohiro YAMAGUCHI Nara National College of Technology 22 Yata-cho, Yamato-Koriyama, Nara 639-1080, Japan yamaguch@info.nara-k.ac.jp more believable[2][10]. However, even if a mind gen- eration mechanism is fully implemented on a life-like agent, there is a significant problem that mind reading between a user and an agent is difficult. For establishing effective communication between a life-like agent and a user, they need to be able to iden- tify the other’s mind from an expression and we call this task mind reading. If mind reading is impossible, they can not act human-like behaviors which signifi- cantly depend on the other's internal state. For exam- ple, a life-like agent should kindly and carefully behave to a depressed or busy user, and intuitively commu- nicate its processing load to a user through a (facial) expression.  Though mind reading is always done a- mong human, it between a life-like agent and a user becomes far more difficult.  Because design of agen- t's expressions depends on personal preference, social culture and so on. For example, Fig.1 shows various expressions and corresponding minds of MicrosoftB agents, which is known as the most popular like-like software agent.  We can easily identify minds from some expressions (Surprised, Congratulate in Fig.1), however minds from some expressions (Confused, De- cline, Process in Fig.1) may be hard to be identified. Consequently a life-like agent and a user need to ac- quire relation between an expression and a mind when they actually encounter.  We call such a mapping from an expression to a mind a mind mapping.  Since this situation occurs independently of whether an life-like agent is implemented on software or a physical robot, human-robot interaction needs to cope with this prob- lem of learning mind mapping mutually. In this paper, we propose a human-agent interac- tion framework in which a user and a life-like agent mutually acquire mind mappings each other. They play a mutual mind reading game together and grad- ually learn mind mappings each other. Instance-based learning is applied to agent's learning.  We fully imple- mented our framework on a PC with a CCD camera and eventually we make experiments for investigating mutual mind reading. Velásquez[10] proposed a emotion model based on society of mind. His model is for generating human- like emotions using a multi-agent system architecture in which each agent corresponds to a primitive emo- tion and emotions are emerged as a result of the in- teractions. However the purpose of his research is to generate emotions and moods like a human, and not to build a framework for interaction between an agent and a user. Various researches of avatars have been done inten- sively on interaction with a man[3], and they found out interesting view points about communicative agents. However their purpose is to develop avatars that can naturally communicate with a human and our one is to design interaction between a human and an agent. A lot of researches on facial expression recognition [5] have been done thus far. We can utilize these tech- niques to categorize sensed expressions. However our interest is concerned with mutual learning of mind mapping, and our research objectives is quite differ- ent from facial expression recognition. Human robot interaction have been also studied ac- tively. In particular, Ono and Imai proposed a cogni- tive model to describe how a human reads a robot mind and investigated its validity experimentally[9]. Though their work is excellent and interesting, it has no mutual learning of mind reading like our work.
2
Learning of Mind Mapping
In this section , we formalize our framework to deal with mutual mind reading between a agent and a user. First the following primitives are introduced. ● Mind state s a , sh : A variable s and sh stand- ing for a state of mind for an agent and a user respectively. A primitive mind is substituted for this variable. ● Primitive mind Eα = {ef,..}, Eh = {ef,..}: Eα and E h are sets of m elements of agent's and a user's minds respectively. We can define these primitive minds depending on a particular task. ● Primitive expression Xα = {x α, . .}, X h = {x h , . . }:  X α and X h are sets of agent's and a user's primitive expressions. ● Mind mapping Mh:x→c = {x, → e, , . . }: This means a user's many-to-one mapping from prim- itive expressions to primitive minds which was learned by an agent. M a:x→e means agent’s mind mapping learned by a user. ● Expression mapping Mh:e→z, Ma:e→z: A user's (or an agent's) one-to-many mapping from prim- itive minds to primitive expressions. ● Mind transition function Tα(c), Th(c): This func- tion determines the next mind of an agent/user depending a context c. This context c may in- clude its current mind, the other's current mind, success rates and so on. Using the above notations, we describe a frame- work in which a life-like agent a and a user h interact through expressions as shown in Fig.2. 2.1 What should be learned? With the framework described in Fig.2, we define learning of a mind mappings and mutual learning of mind mappings in the following. Learning of a mind mapping: An agent(or a user) acquires the other's mind mapping M h : x → e (or Ma:x----). Mutual learning of mind mappings :  An agent and a user mutually acquire the other's mind mapping, Mh:x→e and Mh:x→e Since a designer is able to develop an agent by him- self in practical situations, we can assume that the fol- lowing parameters which are concerned with an agent are given.  Primitive minds of a user may not be essen- tially determined by a designer. However we consider that an agent (or its designer) should determine prim- itive minds of a user because how an agent utilizes them is significantly dependent on the agent's ability. ● Primitive minds of a user and an agent. ● Primitive expressions of an agent. ● A mind transition function of an agent. Except primitive minds of a user, we give no constrain to a user. A user freely learns an agent's mind map- ping. Given the above parameters, the mutual learn- ing of mind mappings is achieved by procedures de- scribed in the next subsection.
2.2
Learning in an agent
Because a user is able to autonomously learn an agent's mind mapping in our framework, we give no restriction to a user within his/her learning. Thus we develop only learning procedures of an agent. Since primitive minds of a user are given, an agent does not need to acquire them. Also user's primitive expressions are obtained by categorizing captured im- ages with a CCD camera.  Hence if a user's primi- tive mind e h is estimated when a user's expression x h is observed, an agent acquires an instance of a us- er's mind mapping xh → eh.  After an agent stores sufficient such instances through interactions with a user, it becomes able to estimate a user's primitive mind from his/her observed primitive expression by instance-based learning[1] like a NN(nearest neigh- bor) method[4]. Agent Learning procedure ● c ∈ C , c = ( I , S ): an instance. ● Ic: an attribute vector. ● V: a set of classes v. ● S c : a sequence of latest n answer pairs. S c = [ s1 , s2, · · ·, sn] = [(v1,good), (v2,nogood), · · ·] 2. Investigate the most similar instance c sim to I new by computing the distance between the attribute vectors. 3. Determine a class ˆ v ∈ V using the following equa- tion.  Random selection is done for tie-breaking.
g( v, s) s E S c x i m where g(v,s) = 1 if s = (v,good), g(v,s) = −1 if s = ( v, nogood ), and g( v, s ) = 0 if no ( v, _ ).  If no instance in an initial period, determine ¥ at random. 4. Indicate ¥ to a user, and he/she answers YES or 5. If the answer is YES, add ( ˆ ν, good ) into S of C sim , and remove the oldest s from S if necessary. 6. If the answer is NO, add ( ¥ , nogood ) into S of csim respectively, and remove the oldest s from S if necessary. Also if the distance between csim and I ncw is over a threshold α , add a new instance ( Incw , [( ˆ v , nogood )]) to C . When an agent guess a user's mind and shows it to a user in a later mutual mind reading game, he/she an- swers by "Yes" or "No" to the estimated mind. Thus an agent needs to utilize "No" answer which is not gen- erally employed for instance-based learning. Since the number of classes (user's primitive minds) is usually over three, we can not determine which class the "No" answer is a positive instance to. Thus we modified a simple instance-based learning algorithm IBL2[1] to be able to deal with a "No" answer. When a "No" an- swer is given to an estimated primitive mind, an agent stores it as a new instance having negative evaluation to the estimated class. To deal with such negative e- valuation, an agent assigns a set of recent evaluations and estimated minds to an instance and determines its class by a majority vote. Detail procedures of agent learning are shown in the following. In all the later experiments, we set parameters as n = 2, α = 900
empirically.
Success rate and finish condition of
2.3
learning
The success rate r(e) for a primitive mind e is computed by the following equation. This suc- cess rate is also utilized to evaluate user's learn- The average value R of all r(e) is used ing. indicate the progress of learning. to ll r(e) The number of success answer pairs in Se Finish con- Sc dition for learning of an agent and a user is described as R = 1. This means recognitions of all primitive minds become complete when the condition is satis- fied.
3
Mutual Mind Reading Game
A primal objective of a mutual mind reading game is to collect instances for agent's instance-based learn- ing efficiently and broadly.  An instance is a pair of an observed user's expression and an estimated prim- itive mind. In this paper, a game in which a player estimates the other's mind state through the (facial) expression to compete for the accuracy is called a mu- tual mind reading game.  A problem of this game is that user's cognitive load becomes high. To solve this, this game is designed so that a user may enjoy it to play a part in collecting training data actively, and as results, the user's cognitive load becomes low. Another objective of a mutual mind reading game is concerned with trust and motivation[8][6]. We con- sider that it is not a good idea to give a user an agent which fully learned a user's mind mapping from the s- tart. On this matter, Schneiderman argued that such a sophisticated agent would give a user a feeling of loss of control and understanding and the user does not try to do modeling the agent[8]. Thus we believe that a user is effectively motivated through a mutual mind reading game. Procedures of a mutual mind reading game are giv- en in the following. Note that an agent tells its correct mind to a user when his/her answer is incorrect, but a user does not do so to reduce his/her cognitive load. 1. An expression of an agent is displayed to a user in GUI. 2. A user guesses agent's mind from seeing the ex- pression, and tells the mind to an agent by click- ing a button. 3. An agent replies "Yes" (the guess is correct) or (the guess is incorrect) with the correct "No" mind as judgment against the other's guess. 4. An agent sees an expression of a user by a CCD camera. 5. An agent guesses user's mind from the captured expression, and shows the mind to a user through GUI. 6. A user replies "Yes" (the guess is correct) or "No" (the guess is incorrect) as judgment against the other's guess. 7. The above procedures are repeated until a finish condition of mutual learning (described in 2.3) is satisfied.
Implementation
4
We fully implemented our framework. A system consists of a laptop computer (SONY VAIO-SR9G/K) and a CCD color camera (Creative Media:  Web- Cam Plus) with USB. The resolution of the camera We used VineLinux2.1, C is 720×680 (8bit color). and GTK+.  Also Video4Linux API was employed for image capture programming.  An experimental envi- ronment is shown in Fig.3. In a phase of agent's learning, an agent sequentially captures images of user expressions per 500ms, and obtains a stable expression.  This stable expression means continuous four images with distance less than a threshold. We experimentally set the threshold as 250. When a stable expression is obtained, the head image is used as a captured image. This mechanism allows a user to control the timing to present his/her expression to an agent. Captured image is transformed into an image with 40×30 with 8bit grey scale for an instance. Since com- putational cost depends on the size of an image, we used such a small grey image. Thus an instance is described by a vector with 256 values of 1200 dimen- sions. The similarity between instances is defined the Euclid distance. We do not employ any feature detection for describ- ing an instance.  Because large computational cost makes system response slow and neither the best fea¬ tures nor the best detection method for any (facial) expression recognition has been developed. In stead, we consider that a user adaptively forms his/her ex- pressions so that an agent can recognize them. This is user's adaptation to an agent, and agent's learning is agent's adaptation to a user. Fig.4 shows a snapshot of GUI when a user guesses agent's mind. When a user clicks the "Start" button, an agent shows its expression.  Then a user guess- es agent's mind, and clicks one of "Primitive Mind" buttons.  If a user clicks the button, an agent tell- s the judgment with the correct mind like a message in Fig.4. Also two progress bars are shown for indi- cating average success rates R (described in 2.3) of a user and an agent. A user can understand the degrees of learning progresses by seeing the progress bars. A game finishes when both of two progress bars reaches to the right edges. Fig.5 shows interface where an agent guesses user's mind.  When a user clicks a "Start agent's recognition" button, an agent begins to capture user's images. Af- ter a stable expression is captured, the four images are shown the window. Also stored instances are indicat- ed with labels and the distance between them and a captured image. Using the most similar instance, an agent guesses a user's mind and tells it to a user like Fig.5. A user answers to it by clicking "Yes" or "No" buttons.
5
Experiments
Experimental methods
5.1
We made experiments to verify mutual learning be- tween a user and an agent, and to investigate its char- acteristics. Through all the experiments, we employed eight subjects consisting of five graduate students, three staff majoring Computer Science at Tokyo Institute of Technology. We used four primitive minds and primitive expres- sions for an agent shown in Fig.6 and three primitive minds "Ordinary", "Thinking", "Decline" for a user. As the primitive minds increase, mutual learning be- comes harder.  We empirically consider the number of these primitive minds is valid for practical experi- ments. Before experiments, we briefly gave subjects the fol- lowing instructions. However we did not explain de- tail procedures of agent's learning, success rates and meanings of captured images, instance images in Fig.5. ● Rules of a mutual mind reading game. ● Explanation on GUI: meanings of two progress bars, buttons and tabs. ● Advise to affect user's expressions: it is effective to slightly rotate, tilt a head and touch a face. Due to agent's ability, fine expressions on a face is hard to be recognized. ● Three primitive minds "Ordinary", "Thinking", “Decline” for a user. Also we set an agent's mind transition function Tα(c) described in section 2 as the simplest one: ran- dom transition. This means an agent's mind changes into next mind randomly independently of context. We will improve this function later to make human learning more efficient. Under the above conditions, each of eight subject- s played a mutual mind reading game once with an agent and we investigated transitions of user's and a- gent's success rates, success rates for each primitive mind, the number of interactions until learning fin- ished and real time taken for a game. We counted an interaction by a pair of agent's guess and user's guess in a game.
5.2
Observing mutual learning
Fig.7 shows representative results for success rates of a user and an agent. The two success rates gradu- ally increases as interactions progressed, finally both of them converged to 1 and the game finished. Thus we are able to observe mutual learning of mind map- pings (described in 2.1) in the experimental results. Since a user and an agent sometimes failed to guess the other’s mind, increases of two success rates are not monotonic. In all the experiments, we observed such mutual learning of mind mappings between a user and an agent.  A single game took about 5–15 minutes, and most subjects seemed to enjoy experiments. The stored instances for three subjects s-1, s-2, s-4 are shown in Fig.8. In contrast that only three in- stances which were minimum to categorize three user's minds were stored for s-2, over five instances were s- tored for s-1 and s-4. For all the subjects, the numbers of stored instances have significant dispersion.  See- ing the expressions in the instances in Fig.8, most of them were done by tilting a head or touching a face. Instruction to affect expressions might excessively re- strict user's expressions. Since our work is in an early stage, there are some limitations and open problems. In these experiments, the number of primitive minds were relatively small. Thus we can utilize a simpler method that we direct- ly show a user a table like Fig.1 to remember mind mapping. However a user intends to feel "loss of con- trol and understanding" in such a situation as Schnei- derman claims. Thus we consider our approach of a game may outperform such a simple approach. While our method is applicable to a large number of prim- itive minds, mutual learning becomes very slow and we need additional methods to improve it.
6
Conclusion
We proposed a human-agent interaction framework in which a user and a life-like agent mutually acquire their mind mappings through a mutual mind read- ing game. For describing mind interactions between a life-like agent and a user, we defined elements of our framework and developed agent's learning procedures by using an instance-based learning method. Then, to acquire the mind mapping each other, we developed a mutual mind reading game in which a user and a life-like agent try to recognize the other's mind from the other's expression.  We implemented our frame- work and made preliminary experiments by employing subjects. As results, we found out mutual learning be- tween a user and a agent through a mutual mind read- ing game and some characteristics of mutual learning. A life-like agent in our framework is not necessary to be a software agent, may be a physical robot agent like a pet robot. Actually we are currently developing a system in which a user can interact with a SONY pet robot AIBO with a CCD camera. In such a case, an expression of an agent may be a gesture rather than a facial expression, and we need image processing like segmentation of a face region.  We consider the human- robot interaction gives us much interesting phenomena between a human and a robot agent, and we try to modify our framework to deal with them effectively.