Debrief mode for capturing information relevant to meetings processed by a virtual meeting assistant

In one embodiment, the interactive virtual meeting assistant implements a meeting debrief post-processing operation. For a given meeting that the interactive virtual meeting assistant participated in, a meeting post-processing engine enables one or more participants of the meeting to associate a debrief with the meeting. The debrief may be an audio recording, a video recording, text, or any file. The meeting post-processing engine stores the debrief in data stores and provides access to the debrief via the meeting GUI associated with the meeting. The meeting post-processing engine also processes the debrief to generate tasks to be assigned to participants or other entities and/or schedule reminders to be provided to the participants or other entities. The debrief may be private, such that only the participant who provided the debrief may subsequently access the debrief.

BACKGROUND OF THE INVENTION

Field of the Invention

Embodiments of the present invention relate generally to interactive virtual assistants, and more specifically, to a debrief mode for capturing information relevant to a meeting processed by a virtual assistant meeting.

Description of the Related Art

Interactive virtual assistants are becoming more commonplace to provide an easy way for a user to accomplish the user's goals via a computer. In conventional approaches, interactive virtual assistants may receive a voice input from the human user, parse the words spoken by the user, and cause an action requested by the user to be performed. For instance, the user may request ordering a particular item from a shopping website or making a phone call to a person whose contact information is stored in a contact database, and, in response, the interactive virtual assistant causes a computer to order the requested item or causes a telephone to make the requested call.

However, conventional approaches have not effectively integrated interactive virtual assistants into a meeting environment. There are several technical challenges when integrating interactive virtual assistants into a meeting environment. For example, there are a variety of disparate platforms and services for conducting meetings. Because such platforms and services do not typically follow a standard, creating an interactive virtual assistant that is able integrate with the disparate platforms and services is technically challenging. Further, given that a meeting typically has at least two vocally active participants, providing a useful and complete set of features to the meeting participants requires sophisticated natural language and audio processing operations.

DETAILED DESCRIPTION

System Overview

FIG. 1illustrates a system100configured to implement one or more aspects of the present invention. As shown, system100includes, without limitation, a computing device110coupled via dial-in infrastructure networks140to multiple meeting participants150(0) to150(m).

As shown, the computing device110includes, without limitation, a processor120, input/output (I/O) devices125, and a memory130. Processor120may be any technically feasible form of processing device configured to process data and execute program code. Processor120could be, for example, a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), and so forth. I/O devices125may include devices configured to receive input or provide output, including, for example, a keyboard, a mouse, a display, and so forth.

Memory130may be any technically feasible storage medium configured to store data and software applications. Memory130may be, for example, a hard disk, a random-access memory (RAM) module, a read-only memory (ROM), and so forth. As also shown, the memory130includes, without limitation, an interactive virtual meeting assistant132, which is a software application that, when executed by the processor120, causes processor120to execute an interactive virtual meeting assistant application. The interactive virtual meeting assistant132may include any technically feasible type of virtual meeting assistant, such as the Eva™ application from Voicera, Inc.

Dial-in infrastructure networks140may be any technically feasible network or set of interconnected communication links that enable the interactive virtual meeting assistant132, executed by processor120, to participate in a meeting with one or more meeting participants150(0) to150(m). As shown, the dial-in infrastructure may include, without limitation, one or more telephone line connections or one or more computer connections, such as a local area network (LAN), wide area network (WAN), the World Wide Web, or the Internet, among others. The dial-in infrastructure networks140may also enable the interactive virtual meeting assistant132to access other information via the networks, such as by accessing information via the World Wide Web, or the Internet, among others.

Meeting participants150(0) to150(m) represent one or more human and/or computer participants in a meeting environment. Each of the one or more meeting participants150(0) to150(m) may be connected to other meeting participants and the interactive virtual meeting assistant132, as executed by processor120, via any technically feasible device that forms a connection to other meeting participants, such as a telephone, smartphone, computing device, or personal data assistant, among others. The connections linking the meeting participants150(0) to150(m) may be any technically feasible communication link(s), including, without limitation, communication links included in the dial-in infrastructure networks140, and/or one or more other external communication links, including, without limitation, one or more telephone line connections or one or more computer connections, such as a local area network (LAN), wide area network (WAN), the World Wide Web, or the Internet, among others.

AlthoughFIG. 1shows the interactive virtual meeting assistant132stored in the memory130of the computing device110, in alternative embodiments, the interactive virtual meeting assistant132may be stored in-part or entirely in the memory130and/or any technically feasible memory device internal to or external to the computing device110, including any memory device coupled to the computing device110through a wired connection, a wireless connection, a network connection, and so forth.

As described in greater detail below in conjunction withFIGS. 2-6, the interactive virtual meeting assistant132receives an invitation to participate in a meeting environment, parses text information included in or derived from the invitation to determine meeting information necessary for the interactive virtual meeting assistant132to join the meeting, and either joins the meeting based on the meeting information determined from parsing the received text information, or returns an error message indicating that the meeting information could not be correctly determined from parsing the received text information. When the meeting concludes, the interactive virtual meeting assistant132performs one or more post-processing operations to generate metrics, tasks, trigger notifications, etc. associated with the meeting.

Interactive Virtual Meeting Assistant

FIG. 2illustrates interactive virtual meeting assistant200, which is a more detailed illustration of the interactive virtual meeting assistant132ofFIG. 1, according to various embodiments of the present invention. As shown, interactive virtual meeting assistant200includes, without limitation, a data receiving engine210, a communications engine220, a scheduling engine230, data stores260, an in-meeting engine270, and a meeting post-processing engine280.

Data receiving engine210may receive data from a user requesting to integrate the interactive virtual meeting assistant200into a meeting environment. Types of data received from the user via the data receiving engine210may include, without limitation, a telephone call or other voice information received from the user, text or other data received from the user via a website associated with the interactive virtual meeting assistant200, text or other data received from the user via an application associated with the interactive virtual meeting assistant200, or an email, calendar invite, or other message containing text information received from the user, among others.

For voice data received via a telephone connection or other voice connection, the data receiving engine210may convert the received voice data into text via a voice-recognition mechanism (not shown) for converting voice to text information. Such voice recognition mechanisms would be well-understood by persons of skill in the art and are not described in further detail herein. In various embodiments, for data received via the data receiving engine210that incudes text information or is converted into text information, such as voice data converted into text via a voice-recognition mechanism, the text information may include text that is formatted according to a predetermined format, or the text information may include text that is free-form, as written by the user, and does not follow any predetermined format.

Communications engine220may perform actions to connect the interactive virtual meeting assistant200to the meeting environment via the dial-in infrastructure networks140, based on the data received from the user via the data receiving engine210. In various embodiments, data may be received via the data receiving engine210to cause the communications engine220to integrate the interactive virtual meeting assistant200into a meeting environment immediately. In other embodiments, data may be received via the data receiving engine210to cause the communications engine220to integrate the interactive virtual meeting assistant200into a meeting environment at a later time.

In various embodiments, data received via the data receiving engine210seeking to integrate the interactive virtual meeting assistant200into a meeting immediately may include data received via a voice connection, via a website connection, or via another form of application, among others. In such circumstances, where the interactive virtual meeting assistant200is to be integrated into the meeting environment immediately, the interactive meeting assistant200may be integrated into the meeting environment using the data received via the data receiving engine210, without retrieving any additional information about the meeting. For instance, the communications engine220may form a connection to integrate the interactive meeting assistant200into the meeting environment immediately in response to data associated with the user pushing one or more buttons on a telephone via a telephone connection. In other embodiments, the communications engine220may form a connection to integrate the interactive meeting assistant200into the meeting environment immediately in response to receiving data via a data connection, such as via a website or via an application, where the received data causes a connection to the meeting environment to be activated.

In other embodiments, data may be received via the data receiving engine210seeking to invite the interactive virtual meeting assistant200to connect to a meeting at a later time. For instance, the data receiving engine210may receive an email via an email inbox, where the data receiving engine210may include an email server or may access an external email server to access received emails. In other embodiments, the data receiving engine210may receive a message, such as a calendar invitation, via a calendar application. In various embodiments, the received emails or messages may include information about how to connect to the meeting in a body of a message, as an attachment to the message, and so forth. In various embodiments, the data receiving engine210may receive a message or other data signal via any kind of application or data connection. The received data, including an email, message, attachment, or other data signal, may include text information in any kind of electronic format, such as hypertext markup language (HTML), plaintext, or any other kind of data that represents text in electronic form, among others.

In such circumstances, where the interactive virtual meeting assistant200is to be integrated into the meeting environment at a later time, the scheduling engine230may parse the received data to determine information about the meeting, such as the scheduled time and date of the meeting, the location of the meeting, a description of the meeting, a title or summary of the meeting, an inviter or organizer of the meeting, and so forth. Determining the scheduled time and location of the meeting enables the communications engine220to connect the interactive virtual meeting assistant200to the meeting environment at the scheduled time and location.

In particular, the scheduling engine230may receive the text information from the data receiving engine210and perform one or more pre-processing operations so that the text information is normalized and formatted according to a standardized format that can be recognized by the text parsing mechanisms included in the scheduling engine230. The scheduling engine230may parse the received pre-processed text information to determine information about the meeting. The information about the meeting includes, but is not limited to, a summary or title of the meeting, an inviter or organizer of the meeting, and how the interactive virtual meeting assistant200may join the meeting. Information enabling the interactive virtual meeting assistant200to join the meeting environment includes any passcodes, passwords, or meeting identification codes, and so forth. In addition, the information may include network or telephone service information to be used to connect the interactive virtual meeting assistant200to the meeting. For instance, a particular network or telephone service may use specified network protocols or a specified type of telephone connection, and the interactive virtual meeting assistant200will use this specified service information to be able to connect to the meeting.

In one embodiment, the scheduling engine230also determines information related to a description of the meeting. For instance, the information related to the description of the meeting may be any technically feasible type of information describing the meeting, such as, without limitation, the expected duration of the meeting, the expected number of participants in the meeting, whether there will be any breaks in the meeting, whether any refreshments will be provided at the meeting, or any materials to be presented or discussed in the meeting, and so forth.

In one embodiment, the received text information may be received in the form of an email message, where the email message includes a file as an attachment that complies with a particular meeting information format or protocol. In such circumstances, the owner of the meeting may be determined from the attached file. For such an attachment, the attached file may include, without limitation an “organizer” field indicating the organizer or owner of the meeting, and a “sent by” field indicating the party that sent the invitation to the meeting. The scheduling engine230may determine that the owner of the meeting is the party identified in the “organizer” field. If the attached file does not include an “organizer” field, the scheduling engine230may determine that the owner of the meeting is the party identified in the “sent by” field. Accordingly, the “organizer” and “sent by” fields may be parsed in priority order, giving a higher priority to the “organizer” field. For instance, the owner of the meeting may be identified in the “organizer” field, whereas a third party, such as an administrator or an assistant to the owner of the meeting, may be the party that actually sends the email with the attached file.

In other embodiments, as would be well-understood to persons skilled in the art, for cases where there is no file attached to the email or insufficient information included in the attached file, the email itself may include one or more fields relating to the party that sent the email message. For instance, the email message may include, without limitation, a “reply-to” field, a “sender” field, and a “from” field, among others. Similar to the case of the “organizer” and “sent by” fields within the attached file, the “reply-to,” “sender,” and “from” fields may be parsed in priority order, giving top priority to the “reply-to” field, next priority to the “sender” field, and lower priority to the “from” field. For instance, the “reply-to” field indicates where replies to the email are directed, which is likely to be the owner of the meeting. By contrast, the “sender” or “from” fields may indicate the account from which the email was sent, which is not necessarily the account of the owner of the meeting. For instance, the “sender” or “from” fields may indicate the account of an email administrator or an assistant to the owner of the meeting. Accordingly, the scheduling engine230may cause the received text information to be parsed to identify the owner of the meeting.

Once the scheduling engine230determines the information related to the meeting, the scheduling engine230stores the information (also referred to herein as the “meeting parameters”) in the data stores260. When the meeting time approaches, the scheduling engine230connects the meeting assistant200to the meeting via the connection information included in the meeting parameters.

The in-meeting engine270operates as a background process when the meeting assistant200connects to a meeting through the dial-in infrastructure140. The in-meeting engine270actively performs natural language processing operation on audio data captured from the meeting to identify and act on commands spoken by one or more meeting participants150directing the meeting assistant to perform tasks related to the meeting. Such commands include taking notes, performing actions internal or external to the meeting, associating a particular point in time of the meeting with an action to be taken during a meeting post-processing step. The in-meeting engine270may also independently identify points in time during the meeting where the assistant200performs an action even though an explicit command has not been provided by a meeting participant150. Such actions include memorializing when meeting participants join/leave a meeting, terminating the connection to the meeting upon determining that the meeting has ended, and determining that a quality of the connection to the meeting has fallen below a threshold and performing one or more remedial operations.

In various embodiments, the interactive virtual meeting assistant200may store information in data stores260. As shown, data stores260include, without limitation, meeting parameters262, meeting metadata264, and meeting recordings266. In various embodiments, meeting parameters262may store the multiple different fields of meeting information, i.e., the meeting parameters, that enable the interactive virtual meeting assistant200to join the meeting environment. In various embodiments, meeting metadata264stores information generated during the meeting, as described in herein. In some embodiments, meeting recordings266stores audio and other recordings of the actual meeting environment generated during the meeting, as described in detail herein. In addition, the data stores260may include any technically feasible other data stores for storing additional information that is generated by or would be useful for the interactive virtual meeting assistant200. In some embodiments, the data stored in the data store260may be stored as part of the interactive virtual meeting assistant200, or anywhere else in memory120, or any technically feasible memory, such as an external memory connected to the computer100or remotely-accessible by the computer100.

The meeting post-processing engine280performs one or more post-processing operations subsequent to the meeting assistant200having participated in the meeting. Post-processing operations include identifying any commands received by the meeting assistant200during the meeting that need to be acted upon. These commands include transmitting notifications to meeting participants and/or other entities within an organization, setting up follow-up meetings, creating and transmitting tasks to meeting participants and/or other entities. Post-processing operations also include processing the audio of the meeting recording to generate metrics, tasks, meeting summaries, meeting notes, highlights, etc.

In one embodiment, the meeting post-processing engine280generates a meeting graphical user interface (GUI) associated with a meeting in which the meeting assistant200participated. The meeting GUI is available for viewing and manipulation by one or more meeting participants150and/or other entities. The meeting GUI includes, but is not limited to, a mechanism to playback the meeting recording associated with the meeting, a list of follow-up tasks generated for the meeting, and/or any information generated by the post-processing operations.

FIG. 3is a timing diagram illustrating actions performed by the interactive virtual meeting assistant200to integrate the interactive virtual meeting assistant into a meeting environment, according to various embodiments of the present invention. Although the actions performed are described in conjunction withFIGS. 1 and 2, persons skilled in the art will understand that any system configured to perform the method steps, in any order, falls within the scope of the present invention.

As shown, a timing diagram300begins at step302, where the interactive virtual meeting assistant200receives from one of the meeting participants150(0) to150(m) an invitation to cause the interactive virtual meeting assistant200to be integrated into the meeting environment. As described herein, the interactive virtual meeting assistant200may include a data receiving engine210for receiving data in the form of the invitation requesting to integrate the interactive virtual meeting assistant200into the meeting environment. As described herein, the invitation may include data that includes text information or includes information from which text information may be derived, including, without limitation, data in the form of a telephone call or other voice information received from the user, text or other data received from the user via a website associated with the interactive virtual meeting assistant200, text or other data received from the user via an application associated with the interactive virtual meeting assistant200, or an email or other message containing text information received from the user, among others.

At step304, the interactive virtual meeting assistant200parses the data included in the received invitation to determine one or more meeting parameters associated with the meeting that enable the interactive virtual meeting assistant200to join the meeting. In various embodiments, as described herein, the interactive virtual meeting assistant200may include a scheduling engine230for parsing the text information included in or derived from the received information to generate the one or more meeting parameters.

At step306, the interactive virtual meeting assistant200causes the parsed meeting parameters to be stored in data stores260. At step308, the interactive virtual meeting assistant200joins the meeting environment based on the stored meeting parameters. In various embodiments, the communications engine220may cause the interactive virtual meeting assistant200to join the meeting using the stored meeting parameters. For instance, the communications engine220may use stored telephone number information to cause the interactive virtual meeting assistant200to join a meeting by telephone, or use stored website information to cause the interactive virtual meeting assistant200to join a meeting via a website.

At step310, during the meeting, the interactive virtual meeting assistant200detects and processes natural language commands given by the meeting participants150(0) to150(m). In various embodiments, the interactive virtual meeting assistant200may process commands spoken by the meeting participants150(0) to150(m) to cause the interactive virtual meeting assistant200to perform some meeting-related action. For instance, the meeting participant may speak details of a command to cause the interactive virtual meeting assistant200to perform a meeting related action, such as making a note, memorializing an action item, and so forth.

At step312, the interactive virtual meeting assistant200stores meeting metadata resulting from the natural language commands generated during the meeting in the data stores260. In various embodiments, the interactive virtual meeting assistant200may record meeting metadata resulting from these commands by the meeting participants150(0) to150(m) into the meeting metadata data stores264. For instance, the meeting metadata may include the notes or action items generated during the meeting, such as an action item to send a particular email, to generate a reminder about a deadline, and so forth.

At step314, the interactive virtual meeting assistant200causes the meeting to be recorded. In various embodiments, the interactive virtual meeting assistant200may cause an audio and/or video recording of a meeting to be recorded. In some embodiments, the audio and/or video recording may be stored in the meeting recordings data stores266. At step316, the interactive virtual meeting assistant200causes the meeting recording to be stored in the meeting recordings data stores266.

At step318, the interactive virtual meeting assistant200causes one or more post-processing actions to be performed based on the stored meeting metadata and/or the stored meeting recordings. In various embodiments, the interactive virtual meeting assistant200may include a meeting post-processing engine280that performs these meeting post-processing functions on meeting metadata stored in the meeting metadata stores264, such as by sending an email or a reminder as specified in a note or action item generated during the meeting, and so forth. In addition, in various embodiments, the meeting post-processing engine280may perform data analysis or other post-processing operations on audio or video meeting recordings that are stored in meeting recordings data stores266.

Meeting Debrief Post-Processing Operation

In one embodiment, the meeting post-processing engine280implements a meeting debrief operation. In particular, for a given meeting that the interactive virtual meeting assistant200participated in or is scheduled to participate in, the meeting post-processing engine280enables one or more participants150of the meeting to associate a debrief with the meeting. The debrief may be an audio recording, a video recording, text, or any file. The meeting post-processing engine280stores the debrief in the data stores260and provides access to the debrief via the meeting GUI associated with the meeting. The debrief may be private, such that only the participant150who provided the debrief may subsequently access the debrief. Alternatively, in some embodiments, the debrief may be public or partially public, such that another participant150or entity different from the participant150who provided the debrief may subsequently access the debrief.

FIG. 4is a flow diagram illustrating a method for associating a debrief with a meeting processed by the interactive virtual meeting assistant200, according to various embodiments of the present invention. Although the actions performed are described in conjunction withFIGS. 1 and 2, persons skilled in the art will understand that any system configured to perform the method steps, in any order, falls within the scope of the present invention.

The method400begins at step402, where the interactive virtual meeting assistant200determines that, for a given meeting processed by the interactive virtual meeting assistant200, a debrief setting associated with a meeting participant150has been activated. The debrief setting is stored in the data stores260in a user profile associated with the meeting participant150. In one embodiment, the debrief setting is a global setting such that the setting applies to every meeting in which the meeting participant150is a participant. In another embodiment, the debrief setting is a meeting-specific setting such that the setting applies to only one meeting in which the meeting participant150is a participant. In yet another embodiment, the debrief setting is specific to particular type of meeting (e.g., team meeting, external meeting, all-hands meeting, etc.) such that the setting applies to only meetings of the particular type in which the meeting participant150is a participant.

At step404, the interactive virtual meeting assistant200determines a contact mechanism for contacting the meeting participant150to request the debrief for the meeting. The contact mechanism may be a telephone number, an email address, a social media handle, a messaging service identifier, or any other technically feasible mechanism for contacting the meeting participant150. In one embodiment, the contact mechanism may be stored in the data stores260in a user profile associated with the meeting participant150. In one embodiment, the user initiates the communication with the virtual meeting assistant200to provide the debrief for the meeting without being expressly requested for providing the debrief.

At step406, the interactive virtual meeting assistant200electronically transmits a request to the meeting participant150via the contact mechanism for providing the debrief of the meeting. In one embodiment, the request includes a uniform resource locator (URL) or other type of link to an application that enables the meeting participant150to record or otherwise provide the debrief.

At step408, the interactive virtual meeting assistant200receives the debrief from the meeting participant150in response to the request. At step410, the interactive virtual meeting assistant200generates a private association between the meeting and the debrief. The private association indicates that only the meeting participant150may subsequently access the debrief. In one embodiment, the interactive virtual meeting assistant200stores the debrief in the data stores260in conjunction with the meeting. The debrief may be associated with various metadata of the meeting including, without limitation, the name and email address of meeting participants, the title of the meeting, the agenda of the meeting, the audio recording of the meeting, and/or any public or private notes associated with the meeting.

At step412, the interactive virtual meeting assistant200generates a GUI associated with the meeting for presentation to the meeting participant150that includes the debrief. Further, the interactive the interactive virtual meeting assistant200may provide (either through a request from or a push to) the debrief to external systems. These external systems may be collaboration tools, information management systems, customer relationship management (CRM) systems, etc. The debrief provided to such external systems may be provided in a format that enables users of those systems to perform actions, such as view, interact with, share, and/or store, in association with the debrief.

In one embodiment, the interactive virtual meeting assistant200generates a transcription of the debrief and includes the transcription in the GUI associated with the meeting. In addition, the interactive virtual meeting assistant200evaluates the content of the debrief to perform one or more post-processing operations. These include identifying any commands received by the meeting assistant200during the meeting that need to be acted upon. The commands may include transmitting notifications to meeting participants and/or other entities within an organization, setting up follow-up meetings, creating and transmitting tasks to meeting participants and/or other entities.

FIG. 5illustrates an example of a request transmitted by the interactive virtual meeting assistant200for providing the debrief, according to various embodiments of the present invention. As shown, the meeting participant150receives a text message that includes a link502to an application that enables the meeting participant150to record or otherwise provide the debrief. In one embodiment, the text message is transmitted to the meeting participant150via a short message service based on a telephone number associated with the meeting participant.

FIG. 6illustrates an example of a meeting GUI600that includes a debrief view602provided to the meeting participant150, according to various embodiments of the present invention. As shown, the debrief view602includes a transcription of a debrief provided by the meeting participant150. The debrief view602also enables a user of the meeting GUI600to replay the debrief if the debrief is an audio or a video recording.

Advantageously, the debrief post-processing operation is automatically triggered subsequent to a meeting concluding. Consequently, the debrief post-processing operation enables a meeting participant150to provide personal follow-ups or record reactions close in time to the meeting concluding even if the meeting participant150is on the go. Further, privately associating a debrief with a meeting allows for the debrief to only be accessed by the meeting participant150should the meeting participant want to keep the debrief private. In such a manner, a meeting GUI generated for a meeting includes both information available to all the participants of the meeting and information (such as the debrief) available to only the participant that provided the information or indicated that such information be kept private.

The invention has been described above with reference to specific embodiments. Persons of ordinary skill in the art, however, will understand that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention as set forth in the appended claims. The foregoing description and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.