A multi-party conversational agent includes a computing platform having a hardware processor and a memory storing a software code. The hardware processor is configured to execute the software code to identify a first predetermined expression for conversing with a group of people, and to have a group conversation, using the first predetermined expression, with at least some members of the group. The hardware processor is configured to further execute the software code to identify, while having the group conversation, a second predetermined expression for having a dialogue with at least one member of the group, and to interrupt the group conversation to have the dialogue, using the second predetermined expression, with the at least one member of the group.

BACKGROUND

Advances in artificial intelligence have led to the development of a variety of devices providing one of several dialogue-based interfaces, such as GOOGLE HOME™, AMAZON ALEXA™, and others. However, one shortcoming of all such existing systems is their inability to engage in natural, fluid conversations with more than one person at a time. Moreover, although existing conversational agents offer some degree of user personalization, for example tailoring responses to an individual user's characteristics or preferences, that personalization remains limited by the transactional design of conventional conversational agents. That is to say, their transactional functionality makes it unnecessary for conventional conversational agents to remember more than a few past interactions and a limited set of predefined keywords, such as user names and basic user preferences. Thus, there is a need in the art for a conversational agent capable of interacting with multiple users concurrently in an emotionally sophisticated and engaging manner.

SUMMARY

There are provided multi-party conversational agents and methods for their use, substantially as shown in and/or described in connection with at least one of the figures, and as set forth more completely in the claims.

DETAILED DESCRIPTION

The present application discloses automated multi-party conversational agents and methods for their use that address and overcome the deficiencies in the conventional art. It is noted that, as used in the present application, the terms “automation,” “automated”, and “automating” refer to systems and processes that do not require the participation of a human administrator. Although, the conversation topics and the predetermined expressions used by the multi-party conversational agent disclosed herein are programmed into software code by a human author, the selection and use of those resources to initiate and continue a multi-party conversation or dyadic dialogue is performed in an automated process. Thus, the methods described in the present application may be performed under the control of hardware processing components of the disclosed multi-party conversational agents.

It is further noted that, as used in the present application, the term “conversational agent” refers to a non-human communicative entity rendered in hardware and software that is designed for expressive interaction with one or more human users. In some use cases, a conversational agent may be instantiated as a virtual character rendered on a display and appearing to watch and listen to a user in order to have a conversation with the user. In other use cases, a conversational agent may take the form of a machine, such as a robot for example, appearing to watch and listen to a user in order to converse with the user. Alternatively, a conversational agent may be implemented as an automated voice response (AVR) system, or an interactive voice response (IVR) system, for example.

FIG. 1shows an exemplary multi-party conversational agent, according to one implementation. As shown inFIG. 1, multi-party conversational agent100includes computing platform102having hardware processor104, input module130, output module108, and system memory106implemented as a non-transitory storage device. According to the present exemplary implementation, system memory106stores software code110and user history database120including user histories122,124,126, and128.

As further shown inFIG. 1, multi-party conversational agent100is implemented within a use environment including at least group150having group members152,154, and156(also hereinafter “users152,154, and156”), as well as user158who may not presently be a member of group150. Moreover, and as also shown inFIG. 1, in some implementations, multi-party conversational agent100may be communicatively coupled to one or more information aggregators115(hereinafter “information aggregator(s)115”) via communication network132and network communication links133.

Information aggregator(s)115may correspond to one or more databases and/or one or more knowledge bases, for example, from which multi-party conversational agent100can obtain information relevant to a conversation with group150. For example, multi-party conversational agent100may obtain current weather, sports, or news information from information aggregator(s)115for use in conversing with group150. As noted above, information aggregator(s)115may be accessible to multi-party conversational agent100via communication network132and network communication links133. Communication network132may be a packet-switched network such as the Internet, for example. Alternatively, communication network132may take the form of a wide area network (WAN), a local area network (LAN), or another type of limited distribution or private network.

It is noted that, although the present application refers to software code110as being stored in system memory106for conceptual clarity, more generally, system memory106may take the form of any computer-readable non-transitory storage medium. The expression “computer-readable non-transitory storage medium,” as used in the present application, refers to any medium, excluding a carrier wave or other transitory signal that provides instructions to hardware processor104of computing platform102. Thus, a computer-readable non-transitory medium may correspond to various types of media, such as volatile media and non-volatile media, for example. Volatile media may include dynamic memory, such as dynamic random access memory (dynamic RAM), while non-volatile memory may include optical, magnetic, or electrostatic storage devices. Common forms of computer-readable non-transitory media include, for example, optical discs, RAM, programmable read-only memory (PROM), erasable PROM (EPROM), and FLASH memory.

FIG. 2Ashows a more detailed diagram of input module230suitable for use in multi-party conversational agent100, inFIG. 1, according to one implementation. As shown inFIG. 2A, input module230includes multiple sensors234, one or more microphones235(hereinafter “microphone(s)235”), analog-to-digital converter (ADC)236, and may include transceiver231. As further shown inFIG. 2A, sensors234of input module230may include radio-frequency identification (RFID) sensor234a, facial recognition (FR) sensor234b, automatic speech recognition (ASR) sensor234c, object recognition (OR) sensor234d, one or more cameras234e(hereinafter “camera(s)234e”), and one or more weather sensor(s)234f(hereinafter “weather sensor(s)234f”). Input module230corresponds in general to input module130, inFIG. 1. Thus, input module130may share any of the characteristics attributed to input module230by the present disclosure, and vice versa.

It is noted that the specific sensors shown to be included among sensors234of input module130/230are merely exemplary, and in other implementations, sensors234of input module130/230may include more, or fewer, sensors than RFID sensor234a, FR sensor234b, ASR sensor234c, OR sensor234d, camera(s)234e, and weather sensor(s)234f. Moreover, in other implementations, sensors234may include a sensor or sensors other than one or more of RFID sensor234a, FR sensor234b, ASR sensor234c, OR sensor234d, camera(s)234e, and weather sensor(s)234f. It is further noted that camera(s)234emay include various types of cameras, such as red-green-blue (RGB) still image and video cameras, RGB-D cameras including a depth sensor, and infrared (IR) cameras, for example. It is also noted that weather sensor(s)234fmay include temperature sensors, humidity and/or precipitation sensors, and barometric pressure sensors, to name a few examples.

When included as a component of input module130/230, transceiver231may be implemented as a wireless communication unit enabling multi-party conversational agent100to obtain data from information aggregator(s)115via communication network132and network communication links133. For example, transceiver231may be implemented as a fourth generation (4G) wireless transceiver, or as a 5G wireless transceiver configured to satisfy the IMT-2020 requirements established by the International Telecommunication Union (ITU).

FIG. 2Bshows a more detailed diagram of output module208suitable for use in multi-party conversational agent100, inFIG. 1, according to one implementation. As shown inFIG. 2B, output module208includes at least Text-To-Speech (TTS) module237and one or more audio speakers238(hereinafter “audio speaker(s)238”). As further shown inFIG. 2B, in some implementations, output module208may include one or more mechanical actuators239(hereinafter “mechanical actuator(s)239”). It is noted that, when included as a component or components of output module208, mechanical actuator(s)239may be used to produce facial expressions by multi-party conversational agent100, and/or to articulate one or more limbs or joints of multi-party conversational agent100. Output module208corresponds in general to output module108, inFIG. 1. Thus, output module108may share any of the characteristics attributed to input module208by the present disclosure, and vice versa.

It is noted that the specific components shown to be included in output module108/208are merely exemplary, and in other implementations, output module108/208may include more, or fewer, components than TTS module237, speaker(s)238, and mechanical actuator(s)239. Moreover, in other implementations, output module108/208may include a component or components other than one or more of TTS module237, speaker(s)238, and mechanical actuator(s)239.

FIG. 3shows exemplary software code310suitable for execution by hardware processor104of multi-party conversational agent100, inFIG. 1, according to one implementation. As shown inFIG. 3, software code310may include authoring interface312, dialogue manager314, conversational agent database316, and user history communication module318. As shown inFIG. 3, user history communication module318is configured to be communicatively coupled to user history database120, inFIG. 1, while dialogue manager314is configured to be communicatively coupled to input module130/230and output module108/208. As further shown inFIG. 3, authoring interface312of software code310is interactively linked to conversational agent database313, and dialogue manager314is interactively linked to user history communication module318, as well as to conversational agent database316.

Software code310, inFIG. 3, corresponds in general to software code110, inFIG. 1, and those corresponding features may share any of the characteristics attributed to either corresponding feature by the present disclosure. That is to say, like software code310, software code110may include features corresponding respectively to authoring interface312, dialogue manager314, conversational agent database316, and user history communication module318.

FIG. 4Ashows exemplary authoring interface412provided by software code110/310, according to one implementation. As shown inFIG. 4A, authoring interface412enables an author or a programmer of multi-party conversational agent100to select topic460of a conversation presently being authored, as well as to identify what are deemed important variables of the conversation using priority panel464. Also shown inFIG. 4Ais dialogue tree or dialog graph462linking the objectives of the conversation being authored, i.e., “intents out”466with expected answers, i.e., “intents in”468from one or more members of group150engaged in conversation with multi-party conversational agent100.

It is noted that multi-party conversational agent100extends the traditional concept of dialogue trees to allow an author or a programmer of multi-party conversational agent100to create loops, thereby producing directed graphs. As a result, the feature “dialog tree” will hereinafter be referred to more generally as “dialog graph.” It is further noted that, as used in the present application, the terms “intent” or “intents” may refer to verbal expressions or non-verbal behaviors by multi-party conversational agent100, as well the members of group150. Referring toFIG. 2A, intents on the part of the members of group150in the form of verbal expressions may be identified by performing natural language understanding (NLU) on speech sensed by ASR sensor234c. Intents on the part of the members of group150in the form of non-verbal behaviors may be identified by multi-party conversational agent100through use of camera(s)234eand/or FR sensor234b. It is also noted that authoring interface412corresponds in general to authoring interface312, inFIG. 3, and those corresponding features may share any of the characteristics attributed to either corresponding feature by the present disclosure.

FIG. 4Bshows exemplary authoring interface412displaying conversational flow470for multi-party conversational agent100, according to one implementation. It is noted that any features inFIG. 4Bidentified by reference numbers identical to those shown inFIG. 4Acorrespond respectively to those previously described features and may share any of the characteristics attributed to them above.FIG. 4Bshows list of topics460, each topic corresponding to one or more previously authored dialogue graphs, represented inFIG. 4Aby exemplary dialogue graphs462a,462b,462c, and462d(hereinafter “dialogue graphs462a-462d”). Conversational flow470can be created by an author or a programmer of multi-party conversational agent100by linking terminal nodes of a particular dialogue to the root of another using flow-edges472. For example, a conversation beginning with dialogue graph462acontinues to dialogue graph462cif dialogue graph462aterminates at the response “bad” from one or more members of group150. By contrast, that conversation continues to dialogue graph462b, and may continue further to dialogue graph462dwhen dialogue graph462aterminates at the response “good.”

Authoring interface312/412gives an author or a programmer of multi-party conversational agent100an interactive user interface where they can design conversational flow470from dialogue graphs462a-462d. Conversational flow470describes the way a conversational topic is advanced to achieve each one of its goals (e.g., Greeting, HowAreYous, etc.) As shown inFIG. 4A, each goal corresponds to a dialogue graph, e.g., dialogue graph462, spanning multiple nodes. Each node represents one intent, and it could be either (a) an intent from one or more members of group150(e.g., AnswerYes, AnswerNo), (b) an internal state (e.g., UserAnsweredYes, UserAnsweredNo), or (c) an intent from one or more members of group150that is remembered by multi-party conversational agent100from a previous interaction.

Referring toFIG. 1,FIG. 3, andFIG. 4Bin combination, an example of a remembered intent is shown by the transition from dialogue graph462bto dialogue graph462d. In that use case, initiating dialogue graph462bfor topic460labeled “dancing” results in dialogue manager314utilizing user history communication module318to obtain a user history for one or more members of group150and thereby “remember” that the group member or members have previously informed multi-party conversational agent100that dancing is liked. The presence of that previously obtained information in memory, i.e., in user database120stored in system memory106, allows multi-party conversational agent100to transition seamlessly to dialogue graph462d, rather than asking the default predetermined question for initiating dialogue graph462b, i.e., “Do you like to dance?”

In order to create a new conversational flow corresponding to conversational flow470, an author or a programmer of multi-party conversational agent100can use authoring interface312/412to drag-and-drop dialogue graphs corresponding to topics460, and then connect them using flow-edges472. Additionally, as shown byFIG. 4A, an author or a programmer can define variables that dictate what is of importance at each stage of the conversation. These variables can be selected to prioritize mood and emotion, for example, as well as any additional memory variables that multi-party conversational agent100has access to (e.g., time of day, group member's name). As shown inFIG. 4B, conversational flow470is guided using flow-edges472described above. Together the authoring features described above provide a simple and effective way of designing conversations that can interact with groups of people over a variety of topics. Moreover, authoring interface312/412enables the explicit distinction between dyadic dialogues between multi-party conversational agent100and a single member of group150(e.g., greetings), and group conversations with some or all members of group150. This difference is important in a multi-party setting as it enables multi-party conversational agent100to dynamically switch between addressing group150and one of its members152,154, or156, making for a more naturalistic conversation.

FIG. 5Ashows an exemplary diagram of dialogue manager314included in software code110/310utilizing a conversation queue to coordinate a multi-party conversation, according to one implementation. As shown inFIG. 5A, dialogue manager514includes conversation queue574being utilized by conversation manager578, as well as interrupt stack576, which is presently not in use. As noted above, dialogue manager514corresponds in general to dialogue manager314, inFIG. 3. Thus, dialogue manager314may share any of the characteristics attributed to dialogue manager514by the present disclosure, and vice versa.

The exemplary implementation shown inFIG. 5Adepicts a use case in which multi-party conversational agent100converses with a group having two members identified for the purposes ofFIG. 5Aand subsequentFIG. 5Bas “User A” and “User B.” Referring toFIG. 1, it is noted that User A corresponds to any one of members152,154, or156of group150, while User B corresponds to any other member of group150.

According to the exemplary use case shown inFIG. 5A, conversation manager578utilizes conversation queue574to continue a User B thread of a group conversation in which both of User A and User B are participants. Moreover, the User B thread processed by dialogue manager514inFIG. 5Ais a thread from one of two group conversations being conducted concurrently. That is to say, User A conversation thread #1 and User B conversation thread #2 are part of a group conversation directed to the topic “dancing”, while User A conversation thread #3 and User B conversation thread #4 are part of a group conversation directed to the topic “movies.” As shown inFIG. 5A, User B replies to a prompt from multi-party conversational agent100with the answer “yes”573that corresponds to what was expected in conversation thread #2. As a result, conversation manager578forwards new intent575and queues new conversation thread #N+1 for User B.

FIG. 5Bshows exemplary dialogue manager of514utilizing interruption stack576to coordinate a multi-party conversation, according to one implementation. It is noted that any features inFIG. 5Bidentified by reference numbers identical to those shown inFIG. 5Acorrespond respectively to those previously described features and may share any of the characteristics attributed to them above.FIG. 5Bdepicts the processing of non-verbal intent577, i.e., new User C joining the group originally including only User A and User B. As shown inFIG. 5Bconversation manager578creates interruption #1 and adds it to interruption stack576, which causes dialogue manager514to initiate Greet User procedure579.

In other words, according to the exemplary use case depicted inFIG. 5B, during a group conversation that multi-party conversational agent100is having with group150including only User A and User B, multi-party conversational agent100senses a change in the composition of group150. In response, multi-party conversational agent100identifies a predetermined expression based on the sensed change in the composition of group150, and interrupts the group conversation to have a dyadic dialogue with the new member of group150, i.e., User C.

Conversation manager578oversees driving the interaction of multi-party conversational agent100with group150. Dialogue manager514can use conversation manager578to keep track of who the audience, i.e., speaker and addressee, and the current conversation threads (markers on who has said what and their expected responses). In addition, dialogue manager514can use conversation manager to respond in a consistent manner to multi-party interaction dynamics that typically overwhelm conventional solutions. For example, dialogue manager514can use conversation manager578to maintain continuity and relevance of conversational threads as one or group members depart during a group conversation, despite addition of one or more new members to a group, or in the presence of verbal interruptions or interruptions in the form of gestures by group members.

The advantages of the present multi-party conversational agent solution are achieved as the result of a two-step approach. First, as noted above, non-verbal behaviors are included as additional user intents. That inclusion of non-verbal behaviors as user intents advantageously enables mapping of user behaviors to updates to the internal state of multi-party conversational agent100. Second, conversational flow is modeled within two structures: (1) a queue of conversation threads, and (2) a stack of interruptions. The conversation queue includes a priority queue that holds conversation objects (representing each one of the active conversation threads) indexed by expected response intents. Conversation objects maintain a record of expected response intents, origin intent, topic, and addressee. During an interaction, a user's intents are checked against the queue and if a result is found, i.e., a reply to previous topic is received, that conversation object is used as the pivoting point for carrying on with the group conversation.

The interruption stack holds records of the current interruptions. Dialogue manager514uses conversation manager578to process interruptions in a last-in first-out manner until no interruptions remain. The two-step approach described above provides the tools to construct a more natural multi-party group conversation. For example, and as described above, multi-party conversational agent100can have a group conversation with a group of users, can interrupt itself when a new person arrives, can greet and ask the name of the new person in a dyadic dialogue with the new person only, and can resume the group conversation by rephrasing or summarizing what was being said before the interruption.

The functionality of software code110/310will be further described by reference toFIG. 6.FIG. 6shows flowchart680presenting an exemplary method for use by multi-party conversational agent100, according to one implementation. With respect to the method outlined inFIG. 6, it is noted that certain details and features have been left out of flowchart680in order not to obscure the discussion of the inventive features in the present application.

Referring toFIG. 6, with further reference toFIGS. 1 and 2, flowchart680begins with identifying a first predetermined expression for conversing with a group of people, i.e., group150inFIG. 1(action681). The first predetermined expression for conversing with group150may be identified by software code110/310, executed by hardware processor104of multi-party conversational agent100, based on data received from input module130/230and using dialogue manager314/514, as well as one or more of conversation agent database316and user history communication module318.

For example, input module130/230may detect the presence of group150based on audio inputs received from group150by microphone(s)235and/or image capture and interpretation performed using one or more of sensor(s)234. Data received by software code110/310from input module130/230may be processed to identify the first predetermined expression for conversing with group150. In one implementation, dialogue manager314/514may utilize user history communication module318to communicate with user history database120to determine if one or more users included in group150is remembered by multi-party conversational agent100.

As noted above by reference toFIG. 1,FIG. 3, andFIG. 4Bin combination, an example of a remembered intent is shown by the transition from dialogue graph462bto dialogue graph462d. In that use case, initiating dialogue graph462bfor topic460labeled “dancing” results in dialogue manager314/514utilizing user history communication module318to obtain a user history for one or more users included in group150and thereby “remember” that the user or users have previously informed multi-party conversational agent100that dancing is liked. The presence of that previously obtained information in memory, i.e., in user database120stored in system memory106, allows multi-party conversational agent100to transition seamlessly to dialogue graph462d, rather than asking the default predetermined question for initiating dialogue graph462b, i.e., “Do you like to dance?”

Although the remembered intent example discussed above applies to transitions among dialogue graphs462included in conversational flow470, such reliance on memory can also be used to select an appropriate root for initiating conversational flow470with group150dynamically based on a user history for one or more users included in group150. That is to say, in some implementations, the first predetermined expression for conversing with group150may be identified in action681based on a user history of at least one member of group150.

Regarding user history database120, according to the present novel and inventive principles every user-intent may be assigned a memory-intent counterpart that collects the answers to queries received from a particular user during one or more previous interactions, such as group conversations including the user, or dyadic dialogues between the user and multi-party conversational agent100. When a conversation includes a memory-intent, dialogue manager314/514can check what is remembered for the user by utilizing user history communication module318to obtain user history data stored in user history database120. If relevant user history data is obtainable, dialogue manager314/514may automatically traverse the dialogue graphs included in a particular predetermined conversational flow authored using authoring interface312/412to fill in conversation fields for which memory data has been obtained. If there is no memory data available for any member of group150, the conversational flow continues along its authored path. This approach advantageously results in conversations that engage with users in a personalized manner over multiple interactions.

It is noted that the data describing previous interactions and retained in user history database120is exclusive of personally identifiable information (PII) of users with whom multi-party conversational agent100has interacted. Thus, although multi-party conversational agent100is typically able to distinguish an anonymous user with whom a previous conversation or dyadic dialogue has occurred from anonymous users having no previous interaction experience with multi-party conversational agent100, user history database does not retain information describing the age, gender, race, ethnicity, or any other PII of any user with whom multi-party conversational agent100converses or otherwise interacts.

Flowchart680continues with having a group conversation, using the first predetermined expression identified in action681, with at least some members of group150(action682). In some use cases, as noted above, multi-party conversational agent100may not recognize any of users152,154, or156included in group150as a remembered user. In those cases, dialogue manager314/514may identify the first predetermined expression in action681as the root of a conversational flow specifically authored as a greeting to a group of new users. Thus, in some implementations, action682may include having a group conversation with all of the users included in group150.

In other use cases, as discussed above, multi-party conversational agent100may recognize some of users152,154, or156included in group150as remembered users, but not others. For example, multi-party conversational agent100may remember previous interactions with users152and154, but none with user156. In that case, dialogue manager314/514may identify the first predetermined expression in action681as the root of a conversational flow having conversation fields dynamically filled with data obtained from a user history of one or both of users152and154. Thus, in some implementations, action682may include having a group conversation with some, but not all, of the users included in group150.

Action682may be performed by software code110/310, executed by hardware processor104of multi-party conversational agent100, and using conversation queue574and conversation manager578of dialogue manager314/514, as well as output module108/208. For example, text data output by dialogue manager314/514may be translated to speech using TTS module237and speaker(s)238. Moreover, in some implementations, the group conversation may include facial expressions and/or movements, such as gestures by multi-party conversational agent100, for example, produced using mechanical actuator(s)239.

In some implementations, the first predetermined expression used in action682may be associated with a response criterion that must be met before multi-party conversational agent100will continue the group conversation. By way of example, a response criterion for the first predetermined expression may be “respond by all,” meaning that the group conversation is continued only after all users included in group150respond to the first predetermined expression. Alternatively, a response criterion for the first predetermined expression may be “respond by any,” meaning that the group conversation is continued when any user included in group150responds to the first predetermined expression. As yet another alternative, a response criterion for the first predetermined expression may be “no response necessary” when the first predetermined expression is merely declarative, or when it is a rhetorical question, for example. Thus, in some implementations, hardware processor104may execute software code110/310to identify a response criterion for the first predetermined expression, and to continue the group conversation when the response criterion is satisfied.

Flowchart680continues with identifying, while having the group conversation, a second predetermined expression for having a dialogue with at least one member of group150(action683). The second predetermined expression may be identified in action683by software code110/310, executed by hardware processor104of multi-party conversational agent100, based on the data received from input module130/230and using interruption stack576and conversation manager578of dialogue manager314/514, as well as one or more of conversation agent database316and user history communication module318.

In some implementations, action683may occur in response to multi-party conversational agent100sensing a change in the composition of group150while having the group conversation. In those implementations, the second predetermined expression may be identified in action683based on the sensed change. The sensed change in the composition of group150may be a departure of a member of group150or an addition of a new member to group150. For example, referring toFIG. 1, where group150may initially have included user158, in addition to users152,154, and156, action683may be in response to sensing the departure of user158from group150during the group conversation. Conversely, where group150initially includes users152,154, and156, but not user158, action683may be in response to sensing the addition of user158to group150during the group conversation. A change to the composition of group150may be sensed by software code110/310, executed by hardware processor104of multi-party conversational agent100, based on data received from input module130/230and using dialogue manager3141514.

For example, in use cases in which the second predetermined expression is identified in action683based on a sensed change in the composition of group150, and the sensed change is a departure of a member of group150or an addition of a new member to group150, as described above, the second predetermined expression identified in action683may acknowledge the departure or the addition. In a use case in which user158joins group150during the group conversation, for instance, dialogue manager314/514may utilize user history communication module318to communicate with user history database120to determine if user158is remembered by multi-party conversational agent100, so as to identify a greeting appropriate to the interaction history of user158with multi-party conversational agent100.

That is to say, in some implementations, the second predetermined expression may be identified in action683based on a user history of at least one member of group150. As noted above, such an approach advantageously results in interactions that engage with users in a personalized manner over multiple sessions. If there is no memory data available for new user158, the second predetermined expression may be a generic greeting or a request that user158identify himself or herself.

Flowchart680continues with interrupting the group conversation to have a dialogue, using the second predetermined expression, with at least one member of group150(action684). In use cases in which a change in the composition of group150motivates the dialogue, and change is the departure of a group member during the group conversation, action684may correspond to multi-party conversational agent100interrupting the group conversation to remark on the departure of the group member to one or more remaining members of group150. Conversely, in such use cases in which the change in the composition of group150is the addition of a new group member during the group conversation, action684may correspond to multi-party conversational agent100interrupting the group conversation to have a dyadic dialogue with the new group member only, greeting the new group member or welcoming the new group member to group150.

Action684may be performed by software code110/310, executed by hardware processor104of multi-party conversational agent100, and using dialogue manager314/514and output module108/208. For example, text data output by dialogue manager314/514may be translated to speech using TTS module237and speaker(s)238. Moreover, in some implementations, the dialogue may include facial expressions and/or movements, such as gestures by multi-party conversational agent100, for example, produced using mechanical actuator(s)239.

In some implementations, the method outlined by flowchart680may conclude with action684. However, as shown inFIG. 6, in some implementations, flowchart680may continue with resuming the group conversation after having the dialogue in action684(action685). In one such implementation, for example, multi-party conversational agent100may resume the group conversation with group150and provide a status summary of the group conversation prior to the dialogue in action684. For example, multi-party conversational agent100may resume the group conversation by rephrasing or summarizing what was being said before the interruption to the group conversation.

Action685may be performed by software code110/310, executed by hardware processor104of multi-party conversational agent100, and using conversation queue574and conversation manager578of dialogue manager314/514, as well as output module108/208. For example, and as noted above, text data output by dialogue manager314/514may be translated to speech using TTS module237and speaker(s)238. Moreover, in some implementations, the resumed group conversation may include facial expressions and/or movements, such as gestures by multi-party conversational agent100, for example, produced using mechanical actuator(s)239.

In some implementations, flowchart680can continue and conclude with updating or creating a user history for members of group150based on one or both of the group conversation of actions682,683and685and the dialogue of action684(action686). As noted above, the creation and updating of user histories, such as user histories122,124,126, and128stored in user history database120, advantageously results in multi-party conversational agent100engaging with users in a personalized manner over multiple dialogues and/or group conversations. Action686may be performed by software code110/310, executed by hardware processor104of multi-party conversational agent100, and using dialogue manager314/514and user history communication module318.

As noted above, in some implementations, flowchart680can conclude with action64, while in other implementations, flowchart680may include one or both of additional actions685and686. Moreover, it is further noted that actions685and686, when performed, may occur in any order. It is also noted that subsequent to authoring of dialogue graphs462and conversational flows470through use of authoring interface312/412, actions681through684, or actions681through684followed by one or both of actions685and686, may be performed in an automated process from which human involvement, other than the participation of group150, may be omitted.

Thus, the present application discloses automated multi-party conversational agents and methods for their use that address and overcome the deficiencies in the conventional art. In contrast to conventional multi-party solutions, the multi-party conversational agents disclosed by the present application do not model group conversations merely as multiple concurrent dyadic dialogues. Instead, the present solution provides a framework for keeping track of a conversation as it evolves among a group, managing both verbal and non-verbal interruptions, and capable of engaging all group members in a fluent and consistent manner. When compared with the present state-of-the-art, the present solution provides a faster and more tractable way for authors attempting to manually craft and consider all possible conversational pathways for various multi-party interaction scenarios. The automatic handling of interruptions, user histories, and group versus dyadic topics are particularly effective in creating natural interaction experiences that would previously have been impossible or excessively costly to produce.