Abstract:
For a joint performance of a dialogue between a human partner and a robot, the robot analyzes phrase and action of the partner to detect a recognized behavior of the partner and analyze a state of audience listening to utterances from the partner and the robot to detect a recognized state of the audience. A scenario describing the dialogue is stored in entries of a memory. The memory is successively referenced entry by entry and a check is made for a match between an utterance by the partner or the robot to a reaction from the audience. Responsive to a currently detected audience state, a corresponding robot behavior is determined. Preferably, possible partner&#39;s behaviors and expected audience states are mapped in a database to specified robot behaviors. The database is searched for a specified robot behavior corresponding to a currently sensed partner behavior or a currently sensed audience state.

Description:
BACKGROUND OF THE INVENTION  
       [0001]     1. Field of the Invention  
         [0002]     The present invention relates generally to robots, and more specifically to a robot for participating in a joint performance with a human partner and a method of controlling a robot. The robot is useful for jointly performing a comic dialogue with a comedian partner in a theater and for assisting a schoolteacher in a classroom.  
         [0003]     2. Description of the Related Art  
         [0004]     Recent advances in microprocessors and robotics have culminated in the development of two-footed robots known under the trade names of “Aibo” and “Asimo”. Another example of robot is a nursing robot which is described in Japanese Patent Publication 2000-353012 for nursing the aged or the handicapped. The robot has an eye whose viewing point is determined with respect to the position of a user. In response to a voice command from the user, the robot interprets the voice command at the determined viewing point and determines its direction of motion.  
         [0005]     Although the known robots are capable of recognizing human voice and of using light and sound to express their own feelings, their performance is limited to responding to only a simple phrase or action by a user. Therefore, their performance is a series of mutually unrelated behaviors.  
         [0006]     However, a need exists to provide a robot capable of participating in a joint performance of dialogue with a human partner.  
       SUMMARY OF THE INVENTION  
       [0007]     It is therefore an object of the present invention to provide a robot and a method of controlling a robot for participating in a joint performance with a human partner.  
         [0008]     According to a first aspect of the present invention, there is provided a robot comprising a first analyzer for analyzing phrase and action of a human partner to detect a recognized behavior of the partner, a second analyzer for analyzing a state of a plurality of people listening to utterances from the partner and the robot to detect a recognized state of the people, a scenario memory for storing a scenario describing a dialogue between the partner and the robot, and a processor for making reference to a portion of the scenario in the memory according to one of the recognized behavior of the partner and the recognized state of the people, and determining a behavior of the robot according to the referenced portion of the scenario.  
         [0009]     According to a second aspect, the present invention provides a robot control system for controlling a robot, comprising a first analyzer for analyzing phrase and action of a human partner to detect a recognized behavior of the partner, a second analyzer for analyzing a state of a plurality of people listening to utterances from the partner and the robot to detect a recognized state of the people, a scenario memory for storing a scenario describing a dialogue between the partner and the robot, and a processor for making reference to a portion of the scenario in the memory according to one of the recognized behavior of the partner and the recognized state of the people, and determining a behavior of the robot according to the referenced portion of the scenario.  
         [0010]     According to a third aspect, the present invention provides a method of controlling a robot, comprising the steps of (a) storing a scenario describing a dialogue between a human partner and the robot in a memory, (b) analyzing phrase and action of a human partner to detect a recognized behavior of the partner and analyzing a state of a plurality of people listening to utterances from the partner and the robot to detect a recognized state of the people, and (c) making reference to a portion of the scenario in the memory according to at least one of the recognized behavior of the partner and the recognized state of the people and determining a behavior of the robot according to the referenced portion of the scenario.  
         [0011]     According to a fourth aspect, the present invention provides a computer-readable storage medium containing a stored scenario describing a dialogue between a human partner and a robot and a program for controlling the robot, the program performing tie steps of (a) analyzing phrase and action of a human partner to detect a recognized behavior of the partner and analyzing a state of a plurality of people listening to utterances from the partner and the robot to detect a recognized state of the people, and (b) making reference to a portion of the stored scenario according to at least one of the recognized behavior of the partner and the recognized state of the people and determining a behavior of the robot according to the referenced portion of the scenario.  
       BRIEF DESCRIPTION OF THE DRAWINGS  
       [0012]     The present invention will be described in detail further with reference to the following drawings, in which:  
         [0013]      FIGS. 1A and 1B  are front and side views of an entertainment robot of the present invention, respectively;  
         [0014]      FIG. 2  is a block diagram of the robot of according to a first embodiment of the present invention, in which the robot acts as an artificial partner for jointly performing a comic dialogue with a human comedian partner;  
         [0015]      FIG. 3  is an illustration of the scenario memory of  FIG. 2  according to the first embodiment of the invention;  
         [0016]      FIG. 4  is an illustration of the action pattern memory of the robot behavior database of  FIG. 2  for comical actions to be performed by the robot;  
         [0017]      FIG. 5  is an illustration of the phrase pattern memory of the robot behavior database for showing comical phrases to be uttered by the robot;  
         [0018]      FIG. 6  is a flowchart of the operation of the processor of  FIG. 2  according to the first embodiment;  
         [0019]      FIG. 7  is an illustration of a modified scenario memory according to a modification of the first embodiment of this invention;  
         [0020]      FIG. 8  is a flowchart of the operation of the processor according to the modification of the invention;  
         [0021]      FIG. 9  is a block diagram of the robot according to a further modification of the first embodiment of the invention; and  
         [0022]      FIG. 10  is a block diagram of a robot according to a second embodiment of the present invention, in which the robot acts as an artificial assistance for a schoolteacher in a classroom for teaching a group of pupils. 
     
    
     DETAILED DESCRIPTION  
       [0023]     According to a first embodiment of the present invention, an entertainment robot is shown in  FIGS. 1A and 1B . The entertainment robot is used on the stage of a hall or theater as an artificial comedian, standing on the side of a human comedian partner. They perform a comic dialogue by exchanging comical phrases with each other on a wide range of topics including social, political and human affairs.  
         [0024]     The entertainment robot comprises a torso  1  of generally cylindrical structure with a rounded upper portion  2  having a circular opening  3  and a pair of limb portions  4  on its front side. A head  5  of spherical shape is partially submerged in the opening  3  and rotatably secured so that it can orient its face in a limited horizontal angle about a vertical axis  5 A and in a limited vertical angle about a horizontal axis  5 B. Each of the limb portions  4  extends outwards from the torso  1  and includes a driven wheel  6 . A castor  7  is fitted to the bottom of torso  1  near its rear side so that the robot is capable of moving about on the stage.  
         [0025]     Head  5  is provided with a CCD camera  8 A in the position of an eye, a thermographic sensor  8 B in the position of the other eye and a touch sensor  10  on top of the head. A microphone  9  and a loudspeaker  11  are provided on the front side of torso  1 .  
         [0026]     As shown in  FIG. 2 , the outputs of CCL camera  8 A, thermographic sensor  8 B, microphone  9  and touch sensor  10  are connected to a partner&#39;s behavior analyzer  12 . Partner&#39;s behavior analyzer  12  uses its input signals to determine what the partner has behaved by consulting a partner&#39;s feature memory  13  in which a plurality of facial features and speech patterns of the partner are stored as references. Analyzer  12  uses the thermographic sensor  8 B to analyze the feeling of the partner to detect a recognized partner&#39;s behavior.  
         [0027]     The outputs of CCD camera  8 A and microphone  9  are coupled to an audience state analyzer  14 , which determines whether the theater audience is large or small and analyzes their reaction in response to amusing phrases or serious remarks from the performers.  
         [0028]     A scenario memory  15  and a robot behavior database  16  are provided. Robot behavior database  16  includes an action pattern memory  16 A and a phrase pattern memory  16 B. The details of these memories will be described below.  
         [0029]     The robot includes a pair of head actuators  21  and  22  for respectively controlling the orientation of the head  5  in vertical and horizontal angular positions and a pair of wheel actuators  23  and  24  for moving the robot in a specified direction. Controller  20  responds to the output signal from the processor  17  to individually control these actuators.  
         [0030]     As shown in  FIG. 3 , the scenario memory  15  is divided into a plurality of entries (rows) numbered and arranged according to the scenario of a comic dialogue being played by the comedian partner and the robot.  
         [0031]     Each entry is sub-divided into a plurality of fields (columns)  31  to  34 . A specified partner&#39;s action or phrase is indicated in the field  31 , and one or more database keys (A 001  and B 001 , for example) are indicated in the field  32  for making a search through the robot database  16  for corresponding robot action and phrase. The field  33  of each entry is a right-to-speak field that indicates which of the comedian or robot has the right to speak first in that entry. The field  34  is an audience field for which the robot is programmed to consider an expected scale of audience (large or small) and an expected reaction of audience (laughter or no laughter). The symbol “x” in the audience field  34  indicates that there is no need for the robot to consider audience reaction.  
         [0032]     Robot behavior field  32  of entries numbered  3 ,  5 ,  6 , and  10  is further sub-divided into an upper sub-field and a lower sub-field. In the case of entry # 3 , the lower sub-field specifies an alternative robot&#39;s behavior to be performed when the partner fails to do according to the scenario specified in the partner&#39;s behavior field  31 . In the entries # 5 , # 6  and # 10 , the lower sub-field specifies an alternative robot&#39;s behavior to be performed when sensed audience state (scale or reaction) does not match the expected audience state specified in the audience field  34 .  
         [0033]     As shown in  FIG. 4 , the action pattern memory  16 A of robot behavior database  16  has a plurality of entries (rows) in which database keys A 001  through A 008  are mapped to corresponding patterns of robot action and sensed partner&#39;s behaviors, which will be detected by the analyzer  12  Using the database key and a sensed partner&#39;s behavior, the action pattern memory  16 A is searched for a corresponding pattern of action, which the robot is programmed to perform. The robot actions may include bowing, turning around, stepping forward and combinations of these actions.  
         [0034]     As shown in  FIG. 5 , the phrase pattern memory  16 B of robot behavior database  16  has a plurality of entries (rows) in which database keys B 001  through B 008  are mapped to corresponding phrase patterns of the robot and sensed partner&#39;s behaviors, which will be detected by the analyzer  12 . Using the database key and a sensed partner&#39;s behavior, the phrase pattern memory  16 B is searched for a corresponding phrase pattern, which the robot is programmed to utter.  
         [0035]     In the above example,  FIG. 3  shows phrase keys B 001  to B 007  of  FIG. 4 . In the entries of the scenario memory  15 , action keys specified in  FIG. 5  may be combined with the phrase keys to indicate a desired robot behavior.  
         [0036]     The operation of processor  17  proceeds according to the flowchart of  FIG. 6 .  
         [0037]     Processor  17  initially sets a pointer of the scenario memory  15  to the first entry and begins with step  101  by examining the right-to-speak field  33  of the entry of the scenario memory  15  indicated by the pointer. At step  102 , the processor  17  determines which side has the right to speak, the comedian partner or the robot. If the partner has the right to speak, flow proceeds from step  102  to step  103  to examine the audience field  34  of the current entry and flow proceeds to step  104  to determine whether it is necessary to consider the audience reaction or state. If the audience field  34  is marked with the symbol “x”, it is not necessary to consider the audience state and flow proceeds to step  105  to compare a partner&#39;s behavior (uttered phrase) detected by the analyzer  12  to the phrase specified in the partner&#39;s behavior field  31  of the current entry to detect a match (step  106 ). If they match, flow proceeds from step  106  to step  121  to check to see if the current entry is the last entry. If not, flow proceeds to step  122  to advance the pointer to the next entry. If the decision at step  106  is negative, flow proceeds to step  107  to read a robot behavior data from the lower sub-field of robot&#39;s behavior field  32 .  
         [0038]     If the partner fails to behave according to the scenario specified in the partner&#39;s behavior field  31  of the current entry, the decision at step  106  is negative and robot behavior data is read from the lower sub-field of robot behavior field  32  for urging the partner to perform the specified behavior. If the partner has failed to perform the specified behavior a predetermined number of times (step  108 ), flow proceeds to step  121 . Otherwise, flow proceeds to step  109  to perform the specified behavior, and flow returns to step  105  to repeat the process.  
         [0039]     If the audience field  34 , examined at step  103 , contains an expected audience state, the decision at step  104  is negative and flow proceeds to step  111  to turn the robot face towards the audience to sense the audience state by the audience state analyzer  14  and compares the sensed audience to the expected audience state for a match (step  112 ). If they match, flow proceeds to step  121 . Otherwise, flow proceeds to step  113  to turn the robot face toward the partner to sense his behavior by the partner&#39;s behavior analyzer  12  and makes a search through the robot behavior database  16  for a corresponding robot behavior to be performed, using the sensed partner&#39;s behavior as a key. If a corresponding robot behavior is specified in the database  16  (step  114 ), flow proceeds to step  120  to perform a robot behavior specified in the robot behavior field  32  (or robot behavior database  16 ). Otherwise, flow proceeds to step  121 .  
         [0040]     If the right-to-speak field  33 , referenced at step  101 , indicates that the robot has the right to speak first in the current entry, flow proceeds from step  102  to step  115  to examine the audience field  34 . If the audience field  34  contains an expected audience state, the decision at step  116  is negative and flow proceeds to step  117  to turn the robot face toward the audience to detect the audience state by the analyzer  14  and compares the sensed audience state to the expected state for a match (step  118 ). If they match, flow proceeds to step  120 . Otherwise, flow proceeds to step  119  to read a key from the lower sub-field of the robot behavior field  32 . Flow proceeds to step  120  to perform the robot behavior specified in the database  16  corresponding to the retrieved key.  
         [0041]     The following is a description of the operation of the processor  17  according to the contents of scenario memory  15  with reference to  FIGS. 3, 4  and  5 .  
         [0042]     First, the processor initializes the address pointer to “1” and examines the right-to-speak field  33  of entry # 1  of the scenario memory  15  (step  101 ) and proceeds to step  102 . In the entry # 1 , the right to speak is on the partner side and the robot&#39;s human partner is supposed to utter a self-introduction phrase as indicated in the partner&#39;s behavior field  31  and the processor proceeds to steps  103 ,  104  to examine the audience field  34 . Since the audience field of entry # 1  is marked with the symbol “x”, flow proceeds to step  105  to turn the robot face toward the partner to sense his behavior (i.e., the uttered self-introduction phrase) by the analyzer  12  and compares the partner&#39;s behavior to the phrase specified in the partner&#39;s behavior field  31  for a match. If they match, flow proceeds through steps  121  and  122  and returns to step  101  to address the entry # 2  of the scenario memory. Therefore, the processor performs the scenario of entry # 1  by following steps  101 - 106 ,  121  and  122  (as indicated as pass  1  in  FIG. 3 ).  
         [0043]     In the entry # 2 , the right to speak is on the robot side (steps  101 ,  102 ). The processor proceeds from step  102  to step  115  to examine the audience field  34 . Since it is marked with the symbol “x”, the decision at step  116  is affirmative and the processor proceeds to step  120  to perform the robot behavior specified in the robot behavior field  32 . Since the key B 001  is marked in the robot behavior field  32 , the robot is instructed to utter a self-introduction phrase such as “My name is PAPERO”, and then proceeds through steps  121  and  122 , incrementing the address pointer to “3” before returning to step  101 . As a result, the processor performs the entry # 2  by following steps  101 ,  102 ,  115 ,  116 ,  120 ˜ 122  (as indicated as pass  2  in  FIG. 3 )  
         [0044]     In the entry # 3 , the right to speak is on the partner side (steps  101 ,  102 ) and the partner is supposed to perform the behavior specified in the field  31 . In this case, the partner is supposed to slap the robot on the head (i.e., the touch sensor  10 ). The processor proceeds to steps  103 ,  104  to examine the audience field  34 . Since the audience field of entry # 3  is marked with the symbol “x”, flow proceeds to step  105  to compare the partner&#39;s behavior (i.e., the slapping on the robot&#39;s head) to the action specified in the partner&#39;s behavior field  31  for a match (step  106 ). If they match, flow proceeds from step  106  to step  122  via step  121  to increment the address pointer by one and returns to step  101 . Therefore, the processor follows the pass  1  as in the case of entry # 1  if the partner&#39;s behavior matches the specified scenario. If the partner fails to make a slapping action, the decision at step  106  is negative and the processor reads an urging phrase (“Slap My Head”) from the database  16  corresponding to the key B 002  specified in the robot&#39;s behavior field  32  (step  107 ) and flow proceeds through step  108  to step  109  to utter the retrieved urging phrase, and flow returns to step  105 . As a result, when the partner fails to behave according to the specified scenario the processor performs steps  101 ˜ 109 ,  121  and  122  (pass  3 ).  
         [0045]     In the entry # 4 , the right to speak is again on the partner side (steps  101 ,  102 ) and the partner is supposed to utter a greeting phrase such as “Happy to See All of You!” The processor proceeds to steps  103 ,  104  to examine the audience field  34 . Audience field of entry # 4  is again marked with the symbol “x”. Thus, flow proceeds to step  105  to compare the partner&#39;s utterance (i.e., the greeting phrase) to the phrase specified in the partner&#39;s behavior field  31  for a match (step  106 ). If they match, flow proceeds from step  106  to step  122  via step  121  to increment the address pointer to “5” and returns to step  101 . If the partner forgets to utter the greeting phrase, the processor performs steps  101 - 109 ,  121  and  122  (pass  3 ) to read an urging phrase from the database  16  corresponding to the key B 003  specified in the robot&#39;s behavior field  32  of entry # 3  to urge the robot&#39;s partner to utter the missing phrase.  
         [0046]     In the entry # 5 , the right to speak is on the robot side (steps  101 ,  102 ). The processor proceeds from step  102  to step  115  to examine the audience field  34 , which contains an expected state of audience (large audience). The decision at step  116  is therefore negative and flow proceeds to step  117  to compare the sensed audience size (i.e., the number of people) to the expected large audience for a match (step  118 ). If they match, flow proceeds to step  120  to read a reactive phrase (“Hello, Everybody”) from the database  16  corresponding to the key B 004  specified in the upper sub-field of robot behavior field  32  and utter the retrieved phrase. At step  122 , the processor increments the address pointer to “6” before returning to step  101 . In this example, steps  101 ,  102 ,  115 ˜ 118 ,  120 ˜ 122  (pass  4 ) are executed.  
         [0047]     If the sensed audience is small, the decision at step  118  is negative and flow proceeds to step  119  to make a search through the database  16  for one of the reactive behaviors using the output of the audience state analyzer  14  and performs the detected reactive behavior (step  120 ). In this case, steps  101 ,  102 ,  115 ˜ 122  (pass  5 ) are executed.  
         [0048]     In the entry # 6 , the right to speak is on the partner side (steps  101 ,  102 ) and the partner is supposed to utter an opening phrase such as “Today, We Are Going to Perform a Comic Dialogue!” The processor proceeds to steps  103 ,  104  to examine the audience field  34 , which indicates large audience. Since the decision at step  104  is negative, flow proceeds to step  111  to compare the sensed audience state to the expected large audience for a match (step  112 ). If they match, flow proceeds through step  121  to step  122  to increment the address counter to “7”. If the comparison at step  111  indicates that the sensed audience is small, the decision at step  112  is negative and flow proceeds to step  113  to make a search through the database  16  for a reactive robot behavior using the partner&#39;s behavior currently sensed by the partner&#39;s behavior analyzer  12  as a key. If a corresponding reactive robot behavior is detected (step  114 ), flow proceeds to step  120  to perform the detected behavior.  
         [0049]     In the entry # 7 , the right to speak is on the robot side (steps  101 ,  102 ). The processor proceeds from step  102  to step  115  to examine the audience field  34 , which contains an “X” mark. The decision at step  116  is therefore affirmative and flow proceeds to step  120  to read a comical phrase from the database  16  corresponding to the key B 005  specified in the robot behavior field  32  and utter the retrieved comical phrase. At step  122 , the processor increments the address pointer to “8” before returning to step  101 . In this example, steps  101 ,  102 ,  115 ,  116 ,  120 ˜ 122  (pass  2 ) are executed.  
         [0050]     In the entry # 8 , the right to speak is again on the robot side (steps  101 ,  102 ). The processor proceeds from step  102  to step  115  to examine the audience field  34 , which contains a “laughter” mark as an expected audience reaction. The decision at step  116  is negative and flow proceeds to step  117  to compare the output of the audience state analyzer  14  to the expected audience reaction for a match (step  118 ). If the robot&#39;s comical phrase of entry # 7  has appealed to the audience, causing them to laugh, the decision at step  118  is affirmative, and flow proceeds to step  120  to read a reactive phrase from the database  16  corresponding to the key B 006  specified in the robot behavior field  32  and utters the retrieved comical phrase. In this example, steps  101 ,  102 ,  115 ˜ 118 ,  120 ˜ 122  (pass  4 ) are executed, incrementing the address pointer to “9”.  
         [0051]     If the robot&#39;s comical phrase of entry # 7  has not appealed to the audience, causing them to remain silent, the decision at step  118  for the current entry # 8  is negative, and flow proceeds to step  119  to read reactive motion data from the database  16  corresponding to the audience reaction sensed by the audience state analyzer  14  and performs the retrieved reactive motion (step  120 ). In this example, steps  101 ,  102 ,  115 - 122  (pass  5 ) are executed.  
         [0052]     In the entry # 9 , the right to speak is on the partner side (steps  101 ,  102 ) and the partner is supposed to utter a comical phrase. The processor proceeds to steps  103 ,  104  to examine the audience field  34 , which indicates a laughter mark. Since the decision at step  104  is negative, flow proceeds to step  111  to compare the sensed audience state to the expected laughing state of the audience for a match (step  112 ). If the partner&#39;s comical phrase during the current entry is appealing to the audience, the decision at step  112  is affirmative and flow proceeds through step  121  to step  122  to increment the address pointer to the next. If the current partner&#39;s comical phrase is not appealing to the audience, the decision at step  112  is negative and flow proceeds to step  113  to make a search through the database  16  for a reactive robot behavior using the partner&#39;s behavior currently sensed by the partner&#39;s behavior analyzer  12  as a key. If a corresponding reactive robot behavior is detected (step  114 ), flow proceeds to step  120  to perform the detected behavior.  
         [0053]     Scenario memory  15  is modified to include further entries # 11 , # 12 , and # 13  as shown in  FIG. 7  and the flowchart of  FIG. 6  is modified as shown in  FIG. 8 .  
         [0054]     In  FIG. 7 , the audience field  34  of each entry further includes a symbol “o” which indicates that the processor  17  should recognize the analyzed audience state. Depending on the recognized state, the processor determines its reactive phrase or motion.  
         [0055]     The operation of the processor according to the flowchart of  FIG. 8  proceeds as follows when the processor addresses the entries # 11 , # 12  and # 13  in succession.  
         [0056]     In the entry # 11 , the right to speak is on the partner side (steps  101 ,  102 ) and the partner is supposed to utter a comical phrase. The processor proceeds to steps  103 ,  104  to examine the audience field  34 , which contains the symbol “o”. Since the decision at step  104  is negative, flow proceeds to step  119  to make a search through the database  16  for a corresponding robot behavior by using the audience state currently analyzed by the analyzer  14  as a key. At step  120 , the processor performs the detected robot behavior.  
         [0057]     If the comedian partner utters some comments on the audience and their atmosphere, the robot will analyze the audience reaction to that comments and acts in accordance with the analyzed audience reaction.  
         [0058]     In the entry # 12 , the right to speak is on the robot side (steps  101 ,  102 ) and the robot proceeds to steps  115 ,  116  to examine the audience field  34 , which contains the symbol “x”. Since the decision at step  116  is affirmative, flow proceeds to step  120  to read a comical phrase from the database  16  corresponding to the key B 006  specified in the robot behavior field  32  and utters the retrieved comical phrase.  
         [0059]     In the entry # 13 , the right to speak is again on the robot side (steps  101 ,  102 ) and the robot proceeds to steps  115 ,  116  to examine the audience field  34 , which contains the symbol “o”. Since the decision at step  116  is negative, flow proceeds to step  119  to make a search through the database  16  for a corresponding robot behavior by using the audience state currently analyzed by the analyzer  14  as a key. At step  120 , the processor performs the detected robot behavior.  
         [0060]     In this way, enhanced reality can be given to the stage performance of a comic dialogue by a comedian and a robot by adaptively controlling the robot to different audience reactions.  
         [0061]      FIG. 9  illustrates a modification of the first embodiment of this invention. In this modification, the robot is additionally provided with a wireless interface  40  with which it establishes communication with a wireless LAN  41 . To the wireless LAN  41  is connected a server  42 , a microphone  43  and a CCD camera  44 . CCD camera  44  is located in such a position that it can take the whole view of the audience and the microphone  43  is located near the audience to pick up their voice. Server  42  is used to store data indicating the number of sold tickets and the social classes of the audience. The stored data is transmitted from the server  42  via the wireless LAN  41  to the robot where the wireless interface  40  receives the transmitted data for application to the audience state analyzer  14 . The outputs of the microphone  43  and camera  44  are also transmitted through the wireless LAN  41  to the robot as additional audience state data to the audience state analyzer  14 .  
         [0062]     It is seen from the foregoing description, the operation of the robot can be summarized as the following six modes: 
        In mode A, if (1) the partner has the right to speak, (2) no consideration of the audience is necessary and (3) the behavior of the partner is specified as A(x), then no behavior is performed by the robot if the recognized partner&#39;s behavior matches A(x). Otherwise, a specified robot behavior is performed.     In mode B, if (1) the robot has the right to speak, (2) no consideration of the audience is necessary and (3) no behavior of the partner is specified, then a predetermined behavior R(x) is performed by the robot and the program proceeds to the next entry;     In mode C, if (1) the robot has the right to speak, (2) consideration of the audience is necessary and (3) no behavior of the partner is specified, then a predetermined behavior R(x) is performed by the robot if the recognized state of the audience matches the expected state. Otherwise, the robot behavior database  16  is referenced using the recognized audience state;     In mode D, if (1) the partner has the right to speak, (2) consideration of the audience is necessary and (3) the behavior of the partner is specified as A(x), then no behavior is performed by the robot if the recognized state of the audience matches the expected state. Otherwise, the robot behavior database  16  is searched with the recognized partner&#39;s behavior as a key to perform a corresponding robot behavior.        
 
         [0067]     In a second embodiment of the present invention, the robot is used as assistance for a schoolteacher in a classroom for teaching a group of school children. As shown in  FIG. 10 , the robot includes a teacher&#39;s behavior analyzer  52  to which the outputs of the microphone  9 , cameras  8  and touch sensor  10  and a teacher&#39;s feature memory  53  are supplied in a manner similar to the first embodiment. A pupils state analyzer  54  is provided to receive the outputs of the microphone  9  and cameras  8  to determine the reaction of the pupils in the classroom. Similar to the first embodiment, scenario memory  55  and robot behavior database  56  are connected to the processor  57 .  
         [0068]     In this embodiment, the robot is additionally provided with a teaching manual memory  58  for supplying the processor  57  with teaching materials including both textual and pictorial data. Scenario memory  55  includes a plurality of phrases of a scenario to be uttered by the teacher and the robot in a manner similar to the previous embodiment for teaching a particular subject and may further include the identity of a teaching material to be displayed on the video screen in the classroom. Processor  57  transmits a teaching material supplied from the memory  58  to a wireless interface  59 , which communicates with a wireless LAN  60  at intervals in response to a timing control signal, which may be stored in the scenario memory  55 .  
         [0069]     In the classroom, a video screen  61  is provided for displaying the teaching material supplied from the robot. Preferably, a plurality of computer terminals  62  are placed respectively on the pupil&#39;s desks. The teaching materials from the robot may also be displayed on the individual computer terminals  62 . The pupils use the computer terminals  62  to input their answer in response to a teacher&#39;s question. Such input data are transmitted through the wireless LAN  61  to the robot, where the wireless interface  59  repeats it to the pupils state analyzer  54 .  
         [0070]     In the classroom, the robot stands in a position close to the schoolteacher. They perform a dialogue according to a scenario stored in the scenario memory  55 . The dialogue will proceed in a different way depending on the pupils&#39; reaction to what is being taught on a subject or how the subject is explained by the teacher. In this manner, an element of entertainment can be incorporated into the classroom teaching at appropriate timing so that the pupils keep their attention for an extended period of time on a subject of the type likely to cause distraction.