System for meeting facilitation

A method, system and computer program product for facilitating a meeting is implemented in a server computer and provides a graphical user interface to one or more client devices. The server displays a mind map of a center topic on the clients. The server collects statements of participants, and extracts words and phrases therefrom, to identify topics that are currently being discussed. The server determines whether a transition has occurred between the topics, and then calculates a correlation between the topics that are currently being discussed and the center topic. The server determines a location of the topics that are currently being discussed on the mind map, based on the calculated correlation, and displays an updated mind map on the clients. The server generates a warning on the clients when the server determines that the meeting proceedings are being retarded, based on the calculated correlation.

BACKGROUND

The present invention relates generally to a system for meeting facilitation. There are a number of issues related to meeting facilitation.

Generally, acceleration of the meeting process is a prerequisite, although this may not be the case with meetings for deeper discussions and seeking common understandings. Nonetheless, the goal for most meetings is to decide on something, i.e., make a decision, for most participants and, in particular, the persons in charge. In such a meeting, it is necessary to avoid any irrelevant discussion.

There exist some technologies relating to meeting facilitation. However, none of these technologies have any functions that explicitly attempt avoidance of irrelevant discussions.

SUMMARY

The invention provided herein has a number of embodiments useful, for example, in implementing a system and method for facilitating a meeting in a server computer and providing a graphical user interface to one or more client devices connected thereto, wherein:

the server computer displays a mind map of a center topic on the client devices, the center topic serving as a theme of the meeting, and the center topic being registered with the server computer in advance of the meeting;

the server computer collects statements of participants during the meeting, the collected statements being subjected to analysis by the server computer to extract words and phrases therefrom in a time-series manner, and the extracted words and phrases being defined as one or more topics that are currently being discussed in the meeting;

the server computer analyzes the extracted words and phrases to determine whether a transition has occurred between the topics, and then calculates a correlation between the topics that are currently being discussed and the center topic;

the server computer determines a location of the topics that are currently being discussed on the mind map, based on the calculated correlation, and displays an updated mind map on the client devices; and

the server computer generates a warning on the client devices when the server computer determines that the meeting proceedings are being retarded, based on the calculated correlation.

In addition, the system and method include, but are not limited to, the following:

the server computer generates the warning on the client devices when the server computer determines that the calculated correlation exceeds a threshold;

the server computer generates the warning on the client devices when the server computer determines that the topics that are currently being discussed are repeated or recursive topics;

the server computer displays the transition on the mind map as an arrow representing an order of occurrence of the topics that are currently being discussed;

the server computer designates the topics that are currently being discussed as the center topic; and/or

the server computer records a duration of each of the collected statements made by the participants, as well as an identity of the participant who made the collected statements.

DETAILED DESCRIPTION

Overview

The present invention is a computer-implemented system for meeting facilitation that more explicitly achieves avoidance of irrelevant discussions through the following computer-implemented functions:A function for, in the process of the meeting, specifying a center topic for discussion among the participants of the meeting, and visually representing the center topic on a mind map displayed to participating devices.A function for, in the process of the meeting, defining topics from words and phrases extracted from statements made by participants during the meeting.A function for, in the process of the meeting, calculating a correlation between the topics and the center topic, and visually representing the correlation on the mind map.A function for, in the process of the meeting, generating a warning in response to occurrence of an event regarded as indicating that the meeting proceedings are retarded.A function for, in the process of the meeting, issuing a warning in the process of the meeting if a topic having a weak correlation with the center topic, which is an agenda that has been specified in advance, is discussed for a predetermined period of time or longer.A function for, in the process of the meeting, tracking the transition relationships of topics and generating a warning in response to repeated or recursive occurrence of multiple existing topics or in response to formation of a loop of these topics.A function for, in the process of the meeting, computing and explicitly presenting a new topic that should be the center topic of the discussions in the meeting from among the topics that have been discussed during the meeting.

Utilization of these functions provided by the present invention makes it possible to facilitate smoother proceedings of the meeting.

Hardware and Software Environment

With reference now toFIG. 1, a pictorial representation of a system100for facilitating meetings is presented in which the invention may be implemented. The system100not only facilitates meetings, it helps in avoiding irrelevant discussions during the meeting.

The system100includes a network102, which is the medium used to provide communications links between various devices and computers connected together within the system100. In the depicted example, the network102may be a local area network (LAN), wide area network (WAN), the Internet, or another network.

In the depicted example, a server computer104is connected to the network102along with a data storage unit106. In addition, client devices108,110,112are connected to the server computer104via the network102. These client devices108,110,112may be, for example, desktop computers108, laptop or notebook computers110, smartphones112and other devices.

In the depicted example, the server computer104provides data and programs to client devices108,110,112. Specifically, the server computer104implements the functionality of the system for facilitating meetings100and for providing a suitable graphical user interface accessible by users of the participating client devices108,110,112. These aspects of the system100are described in more detail below.

System Configuration

FIG. 2illustrates a configuration200of components for facilitating meetings, according to one example. In one embodiment, each of the following components is implemented on the server computer104.

Participant Information Registration Unit202and Center Topic Registration Unit204are configured to register participant information (202) and an initial value of a center topic for a meeting (204), which are then stored as advance registered information206.

Voice Input Unit208, Voice Analysis Unit210, Word and Phrase Extraction Unit212, and Elapsed Time Computation Unit214are configured to received voice input from the client devices108,110,112(208), in order to analyze statements made by participants during the meeting (210), by extracting words and phrases from the statements (212), that are stored into and retrieved from one or more topic dictionaries216, wherein the words and phrases are searched using an external search engine218with Internet access220, and then associated with a computed elapsed time (214), all of which is stored in a cache for map creation222.

Topic Correlation Calculation Unit224, Warning Calculation Unit226, Mind Map Creation Unit228, and Display and Warning Output230are configured to calculate a correlation between a center topic's main phrase and a new topic associated with extracted words and phrases (224), to calculate a warning based on the correlation as compared to a threshold (226), to create a mind map based on the correlation (228), and to display the mind map as well as the warning (230), on the client devices108,110,112.

System Operation

In one embodiment, a center topic which serves as the theme of the meeting and participant information are registered and stored with the server computer104in advance of the meeting. At the start of the meeting, the participating client devices108,110,112connect to the server computer104via the network102.

During the meeting, the server computer104displays a mind map of the center topic on the client devices108,110,112. In addition, the server computer104displays a presentation package and optionally the participant information on the screens of the client devices108,110,112.

The server computer104collects statements of the participants during the meeting via microphones of the client devices108,110,112. The server computer104records a duration of each of the collected statements made by the participants, as well as the identity of the participant who made each of the collected statements.

The collected statements are subjected to voice analysis, including voice recognition, by the server computer104to extract words and phrases therefrom in a time-series manner. These extracted words and phrases are defined as one or more topics that are currently being discussed in the meeting.

The server computer104analyzes the extracted words and phrases to determine whether a transition has occurred between the topics, i.e., between a previous topic and a new topic. If so, the server computer104creates a data structure representing a relationship between the new topic and the previous topic, and the data structure is stored as a linked list in memory. The server computer104also identifies the new topic as the topic or topics that are currently being discussed.

The server computer104calculates a correlation between the topics that are currently being discussed and the center topic.

The server computer104determines a location of the topics that are currently being discussed on the mind map, based on a magnitude of the calculated correlation, and displays an updated mind map on the client devices108,110,112. The mind map may be updated as it changes, or periodically at the expiration of a predetermined period of time.

The relationship between the new topic and previous topic is displayed on the mind map as an arrow or directed link representing an order of occurrence of the topics that are currently being discussed.

When the following conditions are satisfied, it is determined that the proceeding of the discussion has been retarded, and the server computer104generates a visual and/or audio warning on the client devices108,110,112:The topics that are currently being discussed have a weak correlation with the center topic and this state continued for a predetermined period of time.The topics that are currently being discussed are repeated or recursive topics and this state continued for a predetermined period of time.The relationship between the new topic and previous topic forms a loop and this state remains for a predetermined period of time.

As the meeting proceeds, the topics that are currently being discussed may become a center of discussion, based on the relationship between the topics that have been discussed and the elapsed times dedicated to these topics during the meeting. If this occurs, and the topics that are currently being discussed are different from the center topic that was specified in advance, then the topics that are currently being discussed may be designated as the center topic.

These and other aspects of the system for facilitating meetings are described in more detail below.

The System and its User Interface

FIG. 3Ashows a typical environment for using the system for facilitating meetings, wherein one or more participating client devices108,110,112are connected to the server computer104implementing system. In this example, the client devices include two laptops110(one of which is connected to a projector) in a meeting room with multiple participants, as well as a smartphone112at another location used by a single participant, all of which are connected to the server computer104. In each instance, voice information of the participants is collected by microphones of the participating client devices108,110,112and transmitted to the server computer104.

FIG. 3Bshows a graphical user interface300presented by the server computer104on each of the participating client devices108,110,112. Typically, the graphical user interface300includes a presentation package302that is shared with participants, as well as the mind map304of the meeting. Moreover, the user interface may include video and/or audio of the meeting, as well as video and/or audio cues (“beep”)306to identify specific events occurring in the meeting.

Each participant is allowed to confirm by “Map Info” not only the presentation package302, but also information on the center topic and respective topics discussed. One display mode can be selected from a set of display modes by the client devices108,110,112. The warning306is visually represented via “Map Info” or issued as warning beeps or the like.

Voice Analysis and the Extraction of Words and Phrases in a Time-Series Manner.

FIGS. 4A, 4B and 4Cillustrate the voice analysis performed by the server computer104, wherein words and phrases are extracted in a time-series manner.

FIG. 4Ashows the text output400from the voice analysis of the statements made in the meeting, wherein the statements comprise:

Statements in the Meeting:

B: Don't you think that sourness of strawberry is important?

C: Instead I would like to pay attention to sweetness of the cake.

The underlined words and phrases are extracted by the server computer104in order to define the topics that are currently being discussed during the meeting.

In addition to extracting the words and phrases from the statements made by the participants of the meeting, the server computer104also measures the duration of each of the statements and identifies the participant making the statement.

FIG. 4Bshows the time-series402of the words and phrases extracted by the voice analysis, including the identity of the participant making the statement and the measured duration of each of the statements. Each participant is identified as a separate row containing their statements, with images at the beginning of the row representing the participant and the microphone (MIC A, MIC B and MIC C) associated with the participant. The words and phrases are shown as nodes (with word nodes positioned inside their associated phrase nodes) with arrows connecting between the nodes to show the sequence of the statements. (The term “Nodogoshi” is defined as sensation of liquid passing through the throat of someone as he/she drinks it.)

FIG. 4Cshows a mind map404of the topics discussed during the meeting, as a timeline based on the time-series of the words and phrases extracted by the voice analysis, and as a distance D1, D2, D3from the center topic, which is represented by a line with an arrow indicating the flow of the timeline. The timeline begins at time t=0.00, i.e., the start of the meeting, and flows from left to right for the duration of the meeting. The topics being discussed are shown as nodes, and their calculated correlation with the center topic are represented by their distance from the line representing the center topic. The arrows connecting between the nodes show the sequence of the topics.

Mind Map Display of the Center Topic and Topics Currently being Discussed

FIG. 5shows how the server computer104creates the mind map500to indicate both the center topic and the topic currently being discussed.

The center topic is first specified on the map500, wherein the center topic is displayed as a first node502with a main phrase having content similar to that of the center topic. In this example, the main phrase comprises “New strawberry cake.” In one embodiment, the first node502for the center topic is arranged at a central (vertical) position of the map500.

When a new topic is introduced as the topic currently being discussed, another node504,506,508is created on the map500by the server computer104. A main phrase is extracted from the collected statements, and the node504,506,508is displayed containing the main phrase. In this example, the main phrase comprises “Sourness of strawberry” for a second node504, the main phrase comprises “Sweetness of cake” for a third node506, and the main phrase comprises “Company X” for a fourth node508, which are created in that order.

The server computer104creates and displays each node502,504,506,508having a size representative of the number of times the main phrase appears in the collected statements. The server computer104displays each node504,506,508having a position on the map500based on the topic's calculated correlation with the node502for the center topic, wherein a node each node504,506,508having a stronger calculated correlation with the node502for the center topic is arranged at a position closer to the central position of the node502for the center topic.

Optionally, the server computer104displays each node502,504,506,508with a color assigned to one or more of the participants, e.g., the color assigned to the participant who first uttered the main phrase of each node502,504,506,508, or the color assigned to the participant who utters the main phrase of each node502,504,506,508the most times, or the mix of colors assigned to the participants who uttered the main phrase of each node502,504,506,508, or the speaker(s) who satisfies some other criteria.

The server computer104may or may not display each node502,504,506,508having a link to one or more of the other nodes502,504,506,508, wherein the link is an arrow510,512,514between the nodes502,504,506,508that represents the order of the collected statements that contain the main phrases for the nodes502,504,506,508, and the width of the arrow510,512,514is representative of the frequency of the collected statements made by one or more the participants that contain the main phrase of the linked node502,504,506,508. In this example, the arrow510between the nodes502,504is in both directions indicating a recursive or repeated discussion of the topics associated with both of the nodes502,504; the arrow512between the nodes504,506is in both directions indicating a recursive or repeated discussion of the topics associated with both of the nodes504,506; and the arrow514between the nodes506,508is in one direction indicating a transition between the topic associated with the node506to the topic associated with the node508. Also in this example, the arrow510has a width greater than the arrows512and514, which are of equal width, wherein the width is representative of the frequency of the collected statements made by one or more the participants that contain the main phrase of the linked node502,504,506,508.

Finally, the server computer104may display the nodes504,506,508with an annotation (e.g., “30 sec”, “30 sec”, “10 sec”) showing a duration of the collected statements made by one or more the participants that contain the main phrase of the node504,506,508.

Mind Map Display and the Generation of Warnings

FIG. 6shows how the server computer104creates the mind map600to generate one or more warnings.

In this example, the server computer104displays the center topic on the map600as a first node containing the phrase “New strawberry cake.”

The server computer104displays a new topic on the map600as a second node containing the phrase “Sourness of strawberry.” The server computer104displays the second node with a color (or mix of colors) assigned to one or more of the participants. The server computer104displays the second node having an annotation showing a duration of the collected statements made by one or more the participants that contain the main phrase of the node. The server computer104displays a directed link between the first and second nodes that represents the order of the collected statements that contain the main phrases for the nodes, and the width of the link is representative of the frequency of the collected statements made by one or more of the participants that contain the main phrases of the linked nodes.

The server computer104displays a new topic on the map600as a third node containing the phrase “Sweetness of cake.” The server computer104displays the third node with a color (or mix of colors) assigned to one or more of the participants. The server computer104displays the third node having an annotation showing a duration of the collected statements made by one or more the participants that contain the main phrase of the node. The server computer104displays a directed link between the second and third nodes that represents the order of the collected statements that contain the main phrases for the nodes, and the width of the link is representative of the frequency of the collected statements made by one or more the participants that contain the main phrases of the linked nodes.

Together, the first, second and third nodes comprise a state, indicated by the dashed lines602, where the same words and phrases (“strawberry”, “cake”) are repeated. Anything outside this state602would comprise a warning area where digression from the center topic has occurred, resulting in the server computer104generating a video and/or audio warning to the participating client devices108,110,112indicating that the meeting proceedings are being retarded.

For example, the server computer104displays a new topic on the map600as a fourth node containing the phrase “Company X.” The system server computer104the fourth node with a color (or mix of colors) assigned to one or more of the participants. The server computer104displays the fourth node having an annotation showing a duration of the collected statements made by one or more the participants that contain the main phrase of the node. The server computer104displays a directed link between the third and fourth nodes that represents the order of the collected statements that contain the main phrases for the nodes, and the width of the link is representative of the frequency of the collected statements made by one or more the participants that contain the main phrases of the linked nodes.

The server computer104displays a new topic on the map600as a fifth node containing the phrase “Company Y.” The server computer104displays the fifth node with a color (or mix of colors) assigned to one or more of the participants. The server computer104displays the fifth node having an annotation showing a duration of the collected statements made by one or more the participants that contain the main phrase of the node. The server computer104displays a directed link between the fourth and fifth nodes that represents the order of the collected statements that contain the main phrases for the nodes, and the width of the link is representative of the frequency of the collected statements made by one or more the participants that contain the main phrases of the linked nodes.

The server computer104displays a new topic on the map600as a sixth node containing the phrase “New Swiss roll.” The server computer104displays the sixth node with a color (or mix of colors) assigned to one or more of the participants. The system server computer104the sixth node having an annotation showing a duration of the collected statements made by one or more the participants that contain the main phrase of the node. The server computer104displays a directed link between the fifth and sixth nodes that represents the order of the collected statements that contain the main phrases for the nodes, and the width of the link is representative of the frequency of the collected statements made by one or more the participants that contain the main phrases of the linked nodes.

The server computer104displays a new topic on the map600as a seventh node containing the phrase “Macha.” The server computer104displays the seventh node with a color (or mix of colors) assigned to one or more of the participants. The server computer104displays the seventh node having an annotation showing a duration of the collected statements made by one or more the participants that contain the main phrase of the node. The server computer104displays a directed link between the sixth and seventh nodes that represents the order of the collected statements that contain the main phrases for the nodes, and the width of the link is representative of the frequency of the collected statements made by one or more the participants that contain the main phrases of the linked nodes.

The server computer104displays a new topic on the map600as an eighth node containing the phrase “Macha Latte.” The server computer104displays the eighth node with a color (or mix of colors) assigned to one or more of the participants. The server computer104displays the eighth node having an annotation showing a duration of the collected statements made by one or more the participants that contain the main phrase of the node. The server computer104displays a directed link between the seventh and eighth nodes that represents the order of the collected statements that contain the main phrases for the nodes, and the width of the link is representative of the frequency of the collected statements made by one or more the participants that contain the main phrases of the linked nodes.

The server computer104displays a new topic on the map600as a ninth node containing the phrase “Nodogoshi”. The server computer104displays the ninth node with a color (or mix of colors) assigned to one or more of the participants. The server computer104displays the ninth node having an annotation showing a duration of the collected statements made by one or more the participants that contain the main phrase of the node. The server computer104displays a directed link between the eighth and ninth nodes that represents the order of the collected statements that contain the main phrases for the nodes, and the width of the link is representative of the frequency of the collected statements made by one or more the participants that contain the main phrases of the linked nodes.

The server computer104displays a new topic on the map600as a tenth node containing the phrase “Color.” The server computer104displays the tenth node with a color (or mix of colors) assigned to one or more of the participants. The server computer104displays the tenth node having an annotation showing a duration of the collected statements made by one or more the participants that contain the main phrase of the node. The server computer104displays a directed link between the ninth and tenth nodes that represents the order of the collected statements that contain the main phrases for the nodes, and the width of the link is representative of the frequency of the collected statements made by one or more the participants that contain the main phrases of the linked nodes.

Together, the eighth, ninth and tenth nodes, based on the associated words and phrases, comprise a state, indicated by the dashed lines604, of considerable digression from the center topic. Anything inside this state604would result in the server computer104generating a more urgent video and/or audio warning to the participating client devices108,110,112indicating that the meeting proceedings are being retarded.

Thereafter, the server computer104displays a new topic on the map600as an eleventh node containing the phrase “Company Z.” The system server computer104the eleventh node with a color (or mix of colors) assigned to one or more of the participants. The server computer104displays the eleventh node having an annotation showing a duration of the collected statements made by one or more the participants that contain the main phrase of the node. The server computer104displays a directed link between the tenth and eleventh nodes that represents the order of the collected statements that contain the main phrases for the nodes, and the width of the link is representative of the frequency of the collected statements made by one or more the participants that contain the main phrases of the linked nodes.

The server computer104displays a new topic on the map600as a twelfth node containing the phrase “Coffee.” The server computer104displays the twelfth node with a color (or mix of colors) assigned to one or more of the participants. The server computer104displays the twelfth node having an annotation showing a duration of the collected statements made by one or more the participants that contain the main phrase of the node. The server computer104displays a directed link between the eleventh and twelfth nodes that represents the order of the collected statements that contain the main phrases for the nodes, and the width of the link is representative of the frequency of the collected statements made by one or more the participants that contain the main phrases of the linked nodes. The server computer104also displays a directed link between the twelfth and second nodes that represents the order of the collected statements that contain the main phrases for the nodes, and the width of the link is representative of the frequency of the collected statements made by one or more the participants that contain the main phrases of the linked nodes.

Time-Series Display of the Center Topic and Topics Currently Being Discussed

FIG. 7shows how the server computer104creates a mind map700that is a time-series display to indicate both the center topic and the topic currently being discussed, as an alternative to the mind map. Note thatFIG. 7uses the same data as shown inFIG. 6, but displays the data in a different view mode as compared toFIG. 6.

In this example, the mind map700displays the nodes on a graph, wherein the x-axis corresponds to time and y-axis corresponds to distance from the center topic. Note that a threshold may be specified for the distance from the center topic, e.g., a distance of 5 on the y-axis, and any nodes above that threshold, e.g., the eighth, ninth and tenth nodes in the state of considerable digression, would result in the server computer104generating a visual and/or audio warning to the participants.

Mind Map Display and the Computation of a New Center Topic

FIG. 8shows how the server computer104creates a mind map800with a new center topic designated using the topic currently being discussed, using the same data as shown inFIG. 6. Specifically,FIG. 8shows that the eighth topic, identified by the node with the main phrase “Macha Latte” and the dashed lines802, has become the actual center topic of the meeting, which is indicated by the server computer104causing the display of the eighth node to be emphasized by blinking, or the changing of colors, or some other visual or audio emphasis.

Extracting Words and Phrases from Consecutive Statements and Generating a Topic

FIG. 9illustrates how the server computer104uses voice analysis, including voice recognition, to extract words and phrases from consecutive statements900made by one or more of the participants, and then generates a topic from the extracted words and phrases. Specifically, the server computer104treats the consecutive statements900made by one of the participants as a set of statements with a measured elapsed time.

In this example, the set of statements comprise:“The new product scheduled to be released this time is a strawberry cake. Let's select the strawberry cake from the three types of items. We have handled various kinds of strawberry cakes, so I think that you have your own opinions about it.”
and the measured elapsed time is 60 seconds. One or more words and phrases are extracted from the set of statements, which are indicated by underlines above, using generally known methodologies.

Examples of generally known methodologies include: (1) extraction of nouns by morphological analysis; and (2) extraction of keywords by natural language processing, such as term frequency-inverse document frequency (TF-IDF). Other generally known methodologies may be used as well.

Creating a Mind Map Using the Extracted Words and Phrases

In one example, the server computer104uses a graph description language, such as DOT, to create a mind map. DOT supports both directed and undirected graph types, wherein an undirected graph shows simple relations between objects and a directed graph shows relationships between nodes. Various attributes can be applied to graphs, nodes and edges in DOT, wherein these attributes can control aspects such as color, shape, and line styles.

The server computer104uses the following information to create the mind map:Two phrases: A, B.The relationship of the two phrases: A→B, or B→A. (Note that they are two different relationships.)Respective frequencies of appearance of the phrases A and B as such.Frequency of appearance of the A⇔B relationship.Elapsed time information.

The server computer104calculates the correlation between the two phrases, which is described below. The server computer104then creates a one-dimensional representation of the relationship using DOT, namely, the distance between the two phrases.

Calculation of the Correlation Between Phrases

A number of different methods may be used by the server computer104to calculate the correlation between phrases. In one example, this is used to calculate the correlation between the center topic's main phrase and an extracted phrase.

In one method, a bag-of-words (BOW) model is used with Google™ search to calculate the correlation between the center topic's main phrase and an extracted phrase. The bag-of-words model creates a simplified representation for natural language processing, wherein a text (such as a phrase) is represented as the bag (multiset) of its words, disregarding grammar and even word order, but keeping multiplicity, and the frequency of occurrence of each word is used as a feature.

However, term frequencies are not necessarily the best representation for the text. Common words like “the”, “a”, “to” are almost always the terms with highest frequency in the text. Thus, having a high raw count does not necessarily mean that the corresponding word is more important. To address this problem, one of the most popular ways to “normalize” the term frequencies is to weight a term by the inverse of document frequency (IDF).

Thereafter, the server computer104uses a particular topic dictionary (A, B X) or an external search engine, such as Google™ search, to calculate the correlation, wherein a larger value indicates a stronger correlation. For example, the following results may be obtained by the server computer104using Google™ search:

In another method, the distance between the words in each phrase is obtained by Word2Vec™ and/or EMD (Earth Mover Distance).

Word2Vec™ is a group of related models created by Google™ that are used to produce word embeddings. These models are shallow, two-layer neural networks that are trained to reconstruct linguistic contexts of words. Word2Vec™ takes as its input a large corpus of text and produces a vector space, typically of several hundred dimensions, with each unique word in the corpus being assigned a corresponding vector in the space. Word vectors are positioned in the vector space such that words that share common contexts in the corpus are located in close proximity to one another in the space.

EMD is a measure of the distance between two probability distributions over a region D. More generally, the EMD is used in pattern recognition to compare generic summaries or surrogates of data records called signatures. A typical signature consists of list of pairs ((x1,m1), . . . (xn,mn)), where each xi is a certain “feature” (e.g., letter in a text, etc.), and mi is “mass” (how many times that feature occurs in the record). Alternatively, xi may be the centroid of a data cluster, and mi the number of entities in that cluster. To compare two such signatures with the EMD, one must define a distance between features. The EMD between two signatures is then the minimum cost of turning one of them into the other.

In another method, the magnitude of the correlation between two phrases can be calculated using the method described in the publication Matt J. Kusner, Yu Sun, Nicholas I. Kolkin, Kilian Q. Weinberger, “From Word Embeddings To Document Distances,” Proceedings of the 32nd International Conference on Machine Learning, Lille, France, 2015. JMLR: W&CP volume 37. 2015, which is incorporated by reference herein.

FIG. 10visually depicts the calculations1000performed by the server computer104of the correlation between the center topic's main phrase (“A strawberry cake as new product”) and two phrases labeled as Phrase A (“Macha Latte is my favorite drink”) and Phrase B (“7-11 Roll cake is Popular”), wherein the calculations1000are performed using any of the methods described above. The correlation between the center topic's main phrase and each Phrase A, B is calculated using the distances between the individual words of the Phrase A, B, and then summing the distances between the individual words to obtain the distance between the Phrase A, B and the center topic's main phrase. In this example, Phrase A has a distance of 2.32 from the center topic's main phrase, and Phrase B has a distance of 1.21 from the center topic's main phrase.

Computing the Center Topic from the Previous Topics

The server computer104also computes the center topic from the previous topics in the discussion using the following formula:

Rel⁡(wi)=α⁡(wi)⁢∑j∈tp⁡(mtg)⁢(tfijNi)
wherein:wi: a certain topic (i),α(wi): proportion of elapsed time consumed by the certain topic (i) to the time elapsed of the entire meeting,tp(mtg): a set of topics for which statements continued for a predetermined period of time (e.g., 60 seconds) or longer,Ni: frequency of appearance of the certain topic (i),tfij: frequency of appearance of the certain topic (i) and another topic (j), andRel(wi): a relative weight of the topic, with the largest value being designated as the center topic.

In one example, the server computer104generates a set of topics tp(mtg) for which statements continued for a predetermined period of time (e.g., 60 seconds) or longer. In this example, the set of topics tp(mtg) comprises {“strawberry cake,” “sourness,” “sweetness,” “macha,” “macha latte”}.

The server computer104computes the frequency of appearance Niof a certain topic (i). In this example, the server computer104performs a Google™ search using set of topics tp(mtg) and the number of items found is recorded. In this example, the number of items found comprises {“strawberry cake”: 709,000, “sourness”: 20,100,000, “sweetness”: 27,300,000, “macha”: 52,200,000, “macha latte”: 930,000}.

The server computer104computes the frequency of appearance tfijof the certain topic (i) and another topic (j). In this example, the server computer104performs a Google™ search using a combination of elements taken from the topics set tp(mtg). In this example, the combination of elements taken from the topics set tp(mtg) comprises phrases such as “strawberry cake sourness,” “strawberry cake sweetness,” or the like, and the number of items found comprises (“strawberry cake sourness”: 131,000+“strawberry cake sweetness”: 122,000+“strawberry cake macha”: 286,000+“strawberry cake macha latte”: 19,300)/“strawberry cake”: 709,000=0.79

The server computer104multiplies the above value by α(wi), which has elapsed time as a parameter. In this example, if the period of time in which the topic of the “strawberry cake” was discussed is 8 minutes and the time elapsed of the entire meeting is 30 minutes, then 8/30=0.27 will result and thus the value that indicates the correlation is 0.27×0.79=0.21.

The server computer104performs the above computation for all the elements belonging to the topics set tp(mtg).

The server computer104then compares the values of the individual elements with each other, and then computes and determines the center topic in the previous discussions as the topic having the largest value.

Flowchart: Voice Analysis and Phrase Relation Information Computation

FIG. 11is a flowchart of the voice analysis and phrase relation information computation.

Block1100represents the server computer104receiving voice input from the client devices108,110,112and performing voice analysis (i.e., speech-to-text) on the voice input.

Block1102represents the server computer104performing word and phrase attribute extraction, based on the degree of importance.

Block1104is a decision block the represents the server computer104determining whether the extracted word or phrase is to be analyzed. If so, control transfers to Block1106; otherwise, the logic terminates.

Block1106represents the server computer104calculating the relationship of the extracted word or phrase with a cached previous word or phrase. In this step, the server computer104also calculates a direction between the words or phrases, an accumulated weighting, and an elapsed time.

Block1108is a decision block the represents the server computer104determining whether speaker information is included. If so, control transfers to Block1110; otherwise, control transfers to Block1112.

Block1110represents the server computer104assigning the speaker information, i.e., identifying the participant associated with the statements.

Block1112represents the server computer104storing the relationship data and updating the previous phrases and words information.

Thereafter, the logic terminates.

Flowchart: Generation of Phrase and Map Information

FIG. 12is a flowchart illustrating the generation of phrase and map information.

Block1200represents the server computer104calculating inter-node information, namely, the direction, which is indicated by an arrow, and the weight, which is the width of the arrow.

Block1202represents the server computer104performing a detection of the keyword map loop structure.

Block1204is a decision block the represents the server computer104determining the thresholds of the number of times of loop and the elapsed time. If so, control transfers to Block1206; otherwise, control transfers to Block1208.

Block1206represents the server computer104generating a visual and/or audio warning.

Block1208represents the server computer104computing the distance between the center topic and the node.

Block1210is a decision block the represents the server computer104determining the thresholds of the elapsed time and the distance. If so, control transfers to Block1206to generate a warning; otherwise, control transfers to Block1212.

Block1212represents the server computer104computing the relationship between the topics and computing the center topic, which may be different from the previous center topic.

Thereafter, Block1214represents the server computer104drawing a map for display.

Computer System

With reference now toFIG. 13, a block diagram illustrating a computer system1300is depicted in which aspects of an embodiment of the invention may be implemented. Computer system1300may comprise a server computer104and/or client devices108,110,112.

In one example, computer system1300employs a peripheral component interconnect (PCI) local bus architecture, although other bus architectures may be used. Processor1302and main memory1304are connected to PCI local bus1306through PCI bridge1308, which also may include an integrated memory controller and cache memory for processor1302. Additional connections to PCI local bus1306may be made through direct component interconnection or through add-in boards. In the depicted example, local area network (LAN) adapter1310, small computer system interface (SCSI) host bus adapter1312, and expansion bus interface1314are connected to PCI local bus1306by direct component connection. In contrast, audio adapter1316, graphics adapter1318, and audio/video adapter1319are connected to PCI local bus1306by add-in boards inserted into expansion slots.

Expansion bus interface1314provides a connection for a keyboard and mouse adapter1320, modem1322, and additional memory1324. SCSI host bus adapter1312provides a connection for hard disk drive1326, tape drive1328, and CD-ROM drive1330. Typical PCI local bus implementations will support three or four PCI expansion slots or add-in connectors.

An operating system runs on processor1302and is used to coordinate and provide control of various components within data processing system1300inFIG. 13. The operating system may be a commercially available operating system. An application programming interface (API) may run in conjunction with the operating system and provide calls to the operating system from programs executing on data processing system1300. Instructions for the operating system, the API, and the programs are located on storage devices, such as hard disk drive1326, and may be loaded into main memory1304for execution by processor1302.

Those of ordinary skill in the art will appreciate that the hardware inFIG. 13may vary depending on the implementation. Other hardware or software may be used in addition to or in place of the hardware and software depicted inFIG. 13.

The depicted examples described herein are not meant to be limiting in any way. For example, data processing system1300may be a standalone or rack-mounted server; a desktop, laptop or notebook computer; or a smartphone or other portable device. Further, the present invention may reside on any data storage medium (i.e., floppy disk, compact disk, hard disk, tape, ROM, RAM, etc.) used by a computer system.

Those skilled in the art will recognize many modifications may be made to this configuration without departing from the scope of the present invention. Specifically, those skilled in the art will recognize that any combination of the above components, or any number of different components, including computer programs, peripherals, and other devices, may be used to implement the present invention, so long as similar functions are performed thereby.

Computer Program Product

CONCLUSION

This concludes the description of the various embodiments of the present invention. The descriptions of the various embodiments of the present invention have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. Since many embodiments of the invention can be made without departing from the spirit and scope of the invention, the invention resides in the claims hereinafter appended.