Dialog supporting apparatus that selects similar dialog histories for utterance prediction

A dialog supporting apparatus is provided which can support an on-going dialog so that the dialog is smoothly completed irrespective of the other party in the dialog. The dialog supporting apparatus includes utterance receiving units which receive the utterances of the dialog participants and output utterance information for identifying an utterance. Moreover, the dialog supporting apparatus includes utterance outputting units which output the utterance information from the other participant in the dialog as an utterance, and an utterance processing unit which transforms the utterance identified by the utterance information into an other utterance form. Furthermore, the dialog supporting apparatus includes a dialog history selection unit which selects an optimum dialog history in a dialog history database and an utterance prediction unit which predicts utterances in the dialog based on the selected dialog history.

BACKGROUND OF THE INVENTION

(1) Field of the Invention

The present invention relates to a dialog supporting apparatus which supports an on-going dialog made between people.

(2) Description of the Related Art

Conventionally, a translation device has been developed with a purpose of supporting an on-going dialog in different languages respectively spoken by travelers and local people at travel destinations abroad or the like. A representative example is a translation apparatus which is obtained by providing a translation scheme based on text and translation of example sentences and example usages, on a small information processing apparatus such as a PDA (Personal Digital Assistance). Such an apparatus is provided with thousands of example usages in order to cover general travel conversation, and requires a user to select a desired example usage by viewing the list of example usages. Hence, the apparatus has a usability problem when it comes to actual use. Especially in the case where the apparatus has a small display for displaying a list of example usages and thus the number of example usages which can be viewed at one time is small, this problem is more noticeable. In addition, assuming a general use status of a translation apparatus, example usages corresponding to several sentences must be used in the dialog with the other party in great many cases. Thus, it takes more time than expected to complete an on-going dialog by means of a translation apparatus. Therefore, in order to achieve a final purpose of supporting an on-going dialog made between people, there is a need to add a supplementary function for enabling a user to immediately select a desired example usage from among the list of large number of example usages.

As a method for solving this problem, there has been provided an approach for narrowing down candidate next utterances of a user using example dialog models or a corpus of conversation training histories (for example, refer to Japanese Laid-open Patent Application No. 2003-30187).

SUMMARY OF THE INVENTION

Narrowing down candidate next utterances based on past dialog history of a user of the translation device is effective in the case where the utterances of the other party are included in the utterances. In addition, the narrowing down of candidate next utterances based on a virtual dialog which has been previously uttered in training or typical dialog patterns is effective in the case where the other party utters in compliance with the dialog pattern as expected by the user. However, it is common that dialog patterns vary among people. Here is an example case where a traveler starts a dialog with a train station attendant in order to reserve a train ticket. In response to the traveler's utterance of “I'd like to reserve a train ticket”, a train station attendant may start the dialog with an utterance relating to the date and time of a train ride saying “What day would you like to take train?” and another train station attendant may start the dialog with an utterance relating to the destination of a train saying “Where would you like to go?”. Such being the case, there is a problem that the narrowing-down of the candidates utterances fails depending on the other party in an on-going dialog. An additional problem is that inappropriate narrowing-down confuses dialog participants, resulting in increasing the time to complete the dialog, contrary to the purpose.

The present invention has been conceived in view of these circumstances. An object of the present invention is to provide a dialog supporting apparatus which can support a dialog so that the dialog is smoothly completed irrespective of who the other party is.

In order to achieve the above-described object, the dialog supporting apparatus of the present invention supports an on-going dialog made by dialog participants. The dialog supporting apparatus includes: a dialog history selection unit which selects a dialog history from a dialog history database, based on the information identifying the dialog participants and the situation of the dialog; and an utterance prediction unit which predicts a next utterance of one of the dialog participants in the dialog, based on the dialog history selected by the dialog history selection unit. Here, the dialog history database stores dialog histories of the dialog participants and people other than the dialog participants.

The dialog supporting apparatus of the present invention enables a user to easily select example usages from among the candidate next utterances of the user. Thus, it eliminates the necessity for the other party to wait, and therefore the dialog supported by the dialog supporting apparatus can be smoothly advanced.

The disclosure of Japanese Patent Application No. 2004-296775 filed on Oct. 8, 2004 including specification, drawings and claims is incorporated herein by reference in its entirety.

The disclosure of PCT application No. PCT/JP2005/018429, filed on Oct. 5, 2005, designating the United States of America, including specification, drawings and claims is incorporated herein by reference in its entirety.

DETAILED DESCRIPTION OF THE INVENTION

The dialog supporting apparatus, of a first embodiment of the present invention, supports an on-going dialog made by dialog participants. The dialog supporting apparatus includes: a dialog history selection unit which selects a dialog history from a dialog history database, based on the information identifying the dialog participants and the situation of the dialog; and an utterance prediction unit which predicts a next utterance of one of the dialog participants in the dialog, based on the dialog history selected by the dialog history selection unit. Here, the dialog history database stores dialog histories of the dialog participants and people other than the dialog participants.

This makes it possible to search the histories, which include his or her own dialog histories, the dialog histories of the other party and dialog histories of people other than the dialog participants, for the history which is most similar to the dialog which is likely to be made next, and to predict the next dialog based on the history. Therefore, it becomes possible to support the dialog so that the dialog can be smoothly completed regardless of who the other party is.

Here, in a first aspect of the present invention, it is preferable that the dialog history selection unit of the dialog supporting apparatus extracts, as a standard dialog history, a dialog history which is most similar in the situation of the dialog from among dialog histories of each dialog participant, and that it selects a dialog history having a high degree of similarity with respect to each standard dialog history.

This makes it possible to predict the next utterance of a dialog participant based on the dialog history which has a high degree of similarity to the standard dialog history.

In addition, in a first aspect of the present invention, the dialog supporting apparatus may further include: an utterance receiving unit which receives an utterance of one of the dialog participants; an utterance processing unit which transforms the utterance received by the utterance receiving unit into the other utterance form; and an utterance outputting unit which outputs the utterance of the other utterance form transformed by the utterance processing unit.

This makes it possible to support an on-going dialog where different languages such as Japanese and English are used.

Note that the present invention can be realized not only as a dialog supporting apparatus like this, but also as a dialog supporting method having steps corresponding to the unique units which are provided to the dialog supporting apparatus like this, and as a program causing a computer to execute these steps. Of course, the program can be distributed through a recording medium such as a CD-ROM and a communication medium such as the Internet.

The embodiments of the present invention will be described with reference to the figures.

FIG. 1is a block diagram showing the configuration of a first embodiment of the dialog supporting system of the present invention.

The dialog supporting system is intended for supporting dialog between people. As shown inFIG. 1, the dialog supporting system includes a dialog supporting apparatus100, and a server apparatus200which is connected to the dialog supporting apparatus100through a network300.

The server apparatus200includes a dialog history database201and a communication unit202. The dialog history database201stores various types of dialog histories. The communication unit202communicates with the dialog supporting apparatus100through the network300.

On the other hand, as shown inFIG. 1, the dialog supporting apparatus100includes: an utterance receiving unit101; an utterance outputting unit102; an utterance receiving unit103; an utterance outputting unit104; an utterance processing unit105; an utterance prediction unit106; a dialog history selection unit107; and a communication unit108.

The utterance receiving unit101receives an utterance of a dialog participant1and outputs utterance information for identifying the utterance. The utterance outputting unit102outputs the utterance information from another dialog participant, who is the dialog participant2, to the dialog participant1as an utterance. Likewise, the utterance receiving unit103receives an utterance of the dialog participant2and outputs the utterance information. The utterance outputting unit104outputs the utterance information from the dialog participant1to the dialog participant2as an utterance. The utterance processing unit105transforms the utterances, which are identified by the utterance information outputted from the utterance receiving unit101and the utterance information outputted from the utterance receiving unit103, into the other party's utterance form.

Before a dialog is started, the utterance prediction unit106obtains the authentication information of the dialog participant1and the dialog participant2and the situation of the dialog from the dialog participants. In addition, after the dialog is started, the utterance prediction unit106predicts an utterance in the dialog and displays it on a display panel of the dialog supporting apparatus before the utterance receiving unit101receives an utterance from the dialog participant1or before the utterance receiving unit103receives an utterance from the dialog participant2.

The dialog history selection unit107searches the dialog history database201of the server apparatus200for the authentication information obtained by the utterance prediction unit106, and selects the optimum dialog history from among the dialog history database201. The communication unit108communicates with the server200through a network300.

Next, here will be described how the dialog supporting system configured in the manner described above, operates in the case of supporting dialog in different languages. Here, it is assumed that the dialog participant1speaks Japanese, and the dialog participant2speaks English.FIG. 2is a diagram showing an example of the interrelationship among the utterance numbers, the utterances of the dialog participant1and the utterances of the dialog participant2.

The utterance receiving unit101transforms the received utterance of the dialog participant1into corresponding utterance information. The corresponding utterance information is, for example, an utterance number inFIG. 2. The utterance processing unit105translates an utterance (Japanese) of the dialog participant1into the corresponding utterance (English) of the dialog participant2, and outputs the corresponding character string. For example, in the case where an utterance number1is inputted, the utterance processing unit105outputs the character string of “May I help you?” to the utterance outputting unit104as utterance information.

In contrast, the utterance receiving unit103transforms the received utterance of the dialog participant2into the corresponding utterance information. The utterance information is, for example, an utterance number inFIG. 2. The utterance processing unit105translates an utterance (English) of the dialog participant2into the corresponding utterance (Japanese) of the dialog participant1, and outputs the corresponding character string. For example, in the case where the utterance number1is inputted, the utterance processing unit105outputs the character string of

to the utterance outputting unit102as the utterance information. After that, in order to simplify the following description considering the language directions, the utterance number1inputted by the dialog participant1is abbreviated as J1and an utterance number2inputted by the dialog participant2is abbreviated as E1.

The utterance receiving unit101allows the dialog participant1to directly select an utterance from the list of Japanese utterances, and outputs the utterance number as the utterance information. In addition, the utterance receiving unit103allows the dialog participant2to directly select an utterance from the list of English utterances, and outputs the utterance number as the utterance information. As for the configuration of the utterance receiving unit101or the utterance receiving unit103, it may be realized by the processing of projecting the speech of each dialog participant to one of the utterance numbers using a speech recognition processing. In addition, it is realized by the processing of allowing a dialog participant to input a transcription or a character string corresponding to the pronunciation of the utterance through a keyboard and projecting the speech to one of the utterance numbers.

FIGS. 3A to 3Eeach is a diagram showing an example dialog history stored in the dialog history database201. Dialog histories are histories of utterances made in the past by dialog participants, and in each dialog history the utterances of the dialog participants are placed in time sequence. Each utterance corresponds to one of the utterances defined inFIG. 2. There is a need to distinguish even the same utterance number depending on a dialog participant. Thus, J is assigned to the utterances made in Japanese and E is assigned to the utterances made in English, and the J and E each is defined as a label of each utterance in the dialog history. After that, in order to simplify the following description, a dialog history is described using column labels. For example, the dialog history d1is described as d1: E1, J2, E3, J4, E5, J6, E7and E8. The authentication information of the dialog participants who made the dialog and the topic of the dialog (the situation of the dialog) are recorded in the dialog history. For example, information indicating that the utterances assigned with J are Taro's, the utterances assigned with E are Tom's, and the topic relates to a hotel.

As shown inFIG. 4, here will be described an example case where the dialog participant1is a Japanese traveler, the dialog participant2is a clerk of a hotel who speaks English, and the topic for the dialog is to check in the hotel.

FIG. 5is a flow chart showing the operation flow of the utterance prediction unit106. The utterance prediction unit106identifies the dialog participants and the topic of the dialog which are needed for predicting utterances first (Step S401). For example, in the dialog supporting apparatus100which is realized as a PDA as shown inFIG. 6, the information concerning the dialog participants is obtained by allowing the participants to input directly through a touch panel the names of the dialog participant1and the dialog participant2, and the topic of the dialog. The authentication information of the dialog participants may be obtained by a method for automatically obtaining it using fingerprint authentication and individual information provided in obtaining a credit card, other than the method for manual inputs. In addition, as for obtaining the topic of the dialog, the topic of the dialog may be obtained by a method for automatically associating with the automatically-obtainable information relating to the position where a PDA is placed (the information includes the information relating to the location such as a restaurant, hospital and hotel).

The utterance prediction unit106requests the dialog history selection unit107to generate a prediction stack, when the dialog start button601is pressed as shown inFIG. 7(Step S402).

FIG. 8is a flow chart showing the operation flow of the dialog history selection unit107. The dialog history selection unit107determines the dialog history dr1of the dialog participant1and the dialog history dr2of the dialog participant2which are included in the dialog history database201, based on the information concerning the authentication information of the dialog participants and the topic of the dialog which have been notified by the utterance prediction unit106(Step S701). In other words, the dialog history dr1is determined by searching the dialog history corresponding to the authentication information of the dialog participant1and the topic of the dialog, and the dialog history dr2is determined by searching the dialog history corresponding to the authentication information of the dialog participant2and the topic of the dialog. For example, in the case where a dialog history shown in one ofFIGS. 3A to 3Eis included in the dialog history database201, it is determined that the dialog history dr1is the dialog history d1and the dialog history dr2is the dialog history d3. This is because the utterances assigned with J in the dialog history d1are Taro's, the utterances assigned with E in the dialog history d3are Hanako's, and the topic of the dialog relates to a hotel. The dialog history d5is not selected because the topic of the dialog relates to a rented car while the utterances assigned with J in the dialog history d5are Taro's. In the case where there is no corresponding dialog history in the dialog history database201, it should be noted that the processing can be continued by searching the most similar dialog history disregarding the topic of the dialog.

Next, the dialog history selection unit107calculates scores, which can be obtained by the equation of “score(d)=r(d|dr1)+r(d|dr2)”, of the dialog histories d's other than the dialog histories dr1and dr2which are included in the dialog history database201(Step S702). Here, r(da|db) is a degree of similarity of the dialog history da to the dialog history db, and the degree of similarity is defined by the following equation801shown inFIG. 9A. An assembly of successive utterances is defined as utterance block. Additionally, size(d) is the number of utterance blocks included in the dialog history, and size(b) is the number of utterances included in the utterance block b. For example, in an example of calculating a degree of similarity802inFIG. 9, since the dialog history da is made up of four utterance blocks, size(da) is 4. In addition, since as for the numbers of the utterances included in the respective utterance blocks, size(b1) is 5, size(b2) is 2, size(b3) is 4, and size(b4) is 1. Thus, r(da|db) is approximately 0.424. According to the above equation, the greater the value of a degree of similarity r, the higher the degree of similarity of a dialog history becomes. In addition, a degree of similarity is proportional to the mean of the numbers of utterances included in the respective assemblies, and a degree of similarity becomes greater in proportion to an increase in the number of utterances included in an utterance block.

FIG. 10is a diagram showing an algorithm of a dynamic programming for adjusting the number of utterances in a dialog history. The calculation of the earlier-mentioned degree of similarity is performed after adjusting the numbers of utterances of the two dialog histories to the same number using this dynamic programming.FIG. 11shows an example where the dynamic programming ofFIG. 10is applied to the dialog history d4in order to calculate a degree of similarity r(d4|d1) of the dialog history d4to the dialog history d1. In the adjusted dialog history d4′, the utterances assigned with a label of φ are empty utterances added so as to adjust the number of utterances in the dialog history d4′ to the number of utterances in the dialog history d1. The degree of similarity r(d4|d1) is calculated as the value of the degree of similarity r(D4′|d1).

FIG. 12is a diagram showing an algorithm for determining utterance blocks. Basically, utterance blocks are determined so that the numbers of the utterance blocks included in the two dialog histories become a minimum.

First, the length of the utterance string A and the length of the utterance string B are adjusted to the same length using the dynamic programming (Step S1201). The number of utterances included in the utterance string A (utterance string B) is assumed to be m (Step S1202). Next, 1 is substituted to i (Step S1203). Whether or not A[i] is present in the utterance string B is judged (Step S1204). Note that A[i] shows the ith utterance in the utterance string A. In addition, as for φ, when A[i]=φ and B[j]=φ, it is not regarded as A[i] equals to B[j]. As a result of this judgment, in the case where A[i] is present in the utterance string B (Step S1204: YES), the utterance is assumed to be B[j] (Step S1205). After that, the maximum n which satisfies the condition that utterances A[i] to A[i+n] is the same as utterances B[j] to B[j+n] is obtained, and each of them is assumed to be a block (Step S1206). Next, i+n+1 is substituted to i (Step S1207).

In the case where the judgment on whether or not A[i] is present in the utterance string B showed that A[i] is not present in the utterance string B (Step S1204: NO), A[i] is assumed to be a block (Step S1208). After that, i+1 is substituted to i (Step S1209).

Next, whether i>m or not is judged (Step S1210). In the case where the judgment showed that i is not greater than m (Step S1207: NO), the processing step of judging whether or not A[i] is present in the utterance string B and the following processing steps (Steps S1204to S1207) are repeated. On the other hand, in the case where the judgment showed that i is greater than m (Step S1207: YES), the processing is completed.

FIG. 13is a diagram showing that the dialog history database201stores the dialog histories shown inFIGS. 3A to 3Eand that the results of calculating the scores, each of which is obtained by the equation of “score(d)=r(d|dr1)+r(d|dr2)”, of all the dialog histories d's other than the dialog history dr1and the dialog history dr2, when the dialog history dr1=the dialog history d1, and the dialog history dr2=the dialog history d3.

Next, the dialog history selection unit107selects the dialog history d=the dialog history dmax which provides the maximum score(d) (Step S703). In the case of the example shown inFIG. 13, the dialog history d2is selected.

The dialog history selection unit107applies the algorithm for determining utterance blocks shown inFIG. 12to the dialog history dmax as well as the dialog history dr1and the dialog history dr2, divides the dialog history dmax into utterance blocks so as to generate a prediction stack (Step S704). In the processing of identifying a continuous string of utterances (Step S1206), it should be noted that a restriction that no continuous string of utterances can be made over a boundary between predetermined utterance blocks. For example, in the case where the dialog history dmax=the dialog history d2is divided into the dialog history dr1=the dialog history d1and the dialog history dr2=the dialog history d3, the dialog history d2: E1; E8; J2; E3; J4; E5; J6and E7is divided by the dialog history d1first into three utterance blocks of the dialog history d2: E1; E8; and (J2; E3; J4; E5; J6and E7). The three utterance blocks are further divided by the dialog history d3next into five utterance blocks of the dialog history d2: E1; E8; J2, (E3and J4), and (E5, J6and E7). The dialog history selection unit107notifies the utterance prediction unit106of this dialog history d2as a prediction stack.

In the algorithm for determining utterance blocks shown inFIG. 12performed by the dialog history selection unit107, the processing for judging whether or not an utterance is included in both of the two dialog histories is made (Steps S1204and S1206). However, it should be noted that the judgment may be made by matching based on surface expressions (character expression) of an utterance in the processing of natural language or by matching based on content words. For example, in the case where one of the dialog history includes the utterance of

and the other dialog history includes the utterance of

it is possible to judge these utterances as the same utterance included in both the two dialog histories. This is because the surface expressions of the utterances are similar to each other even though these utterances are assigned with different utterance numbers. This is true of the case where one of the dialog histories includes the utterance of “Thank you.” and the other dialog history includes the utterance of “Thank you very much.”. Furthermore, here is the case where one of the dialog histories includes the utterance of

and the other dialog history includes the utterance of

In this case, it is possible to judge these utterances as the utterance included in both the two dialog histories as long as

are defined as content words. This is because many content words are commonly included in the dialog histories. In addition, these judgment methods may be used in combination. To employ a flexible utterance judgment like this makes it possible to search an appropriate dialog history even in the case where no appropriate dialog history can be searched only by means of matching based on utterance numbers.

Here, the flow chart shown inFIG. 5will be described again.

After receiving an utterance prediction stack from the dialog history selection unit107, the utterance prediction unit106judges whether or not each of the prediction stack is empty (Step S403). In the case where the prediction stack is empty for the reason that the dialog history selection unit107cannot select any appropriate dialog history in the dialog history database201, or for another reason (Step S403: YES), the utterance prediction unit106completes the processing without performing any utterance prediction action.

On the other hand, in the case where the prediction stack is not empty (Step S403: NO), the utterance prediction unit106displays the opening utterance of the prediction stack as the candidate next utterance (Step S404).FIG. 14is a diagram showing the status of the display of the dialog supporting apparatus100at the time when the dialog is started. The upper half of the display area is the utterance input and output area1301intended for the dialog participant1and the lower half of the display area is the utterance input and output area1302intended for the dialog participant2. An exit button1303is the button pressed at the time when the dialog is completed. The dialog participant1and the dialog participant2can select an arbitrary utterance from among all the utterances defined inFIG. 2, using the example usage list1304and the example usage list1305, respectively, and can notify the meaning of the utterance to the other dialog participant. The utterances transformed by the utterance processing unit105are respectively outputted to one of the utterance outputting unit1306of the dialog participant1and the utterance outputting unit1307of the dialog participant2. After that, these utterances are read out using speech synthesis processing installed in a PDA or the like at the same time when they are displayed. The utterances for the dialog participant1which are predicted by the utterance prediction unit106are displayed on the prediction display area1308, and the utterances for the dialog participant2which are predicted by the utterance prediction unit106are displayed on the prediction display area1309. The dialog participant1and the dialog participant2can omit the operation for searching an example usage list1304and an example usage list1305for the utterance to be selected, by selecting the utterance displayed on the prediction display area1308and the prediction display area1309, respectively. In other words, the dialog participants1and2can input utterances immediately.

In the case where a prediction stack is generated as in the above-described case, the utterance prediction unit106displays the utterance of “May I help you?” which is the opening utterance in the prediction stack E1, E8, J2, (E3, J4), and (E5, J6, E7) on the prediction display area1309. The dialog participant2may select an utterance from the example usage list1305. However, since the desired utterance has already been displayed on the prediction display area1309, the dialog participant2selects it as shown inFIG. 15. The utterance E1is translated into Japanese in the utterance processing unit102, and the utterance of

is notified to the utterance participant1.

At this time, the utterance prediction unit106judges whether or not the utterance is inputted by one of the dialog participants (Step S405). When the utterance is inputted by the dialog participant (Step S405: YES), the utterance prediction unit106searches the prediction stack for a matching utterance starting with the opening utterance (Step S406), and judges whether or not there is a matching utterance (Step S407). In the case where there is a matching utterance (Step S407: YES), the utterance prediction unit106judges whether or not the matching utterance is the opening utterance of the prediction stack (Step S408). In the case where it is the opening utterance (Step S408: YES), it deletes the opening utterance of the prediction stack so as to update the prediction stack (Step S409). On the other hand, in the case where the utterance is not the opening utterance (Step S408: NO), it moves the block including the matching utterance to the opening part of the prediction stack, deletes the matching utterance and the utterances placed before the matching utterance so as to update the prediction stack (Step S410). After that, the utterance prediction unit106returns to the processing of judging whether or not the prediction stack is empty (Step S403).

Since the utterance of a dialog participant is E1in the above example, the prediction stack is updated to E8, J2, (E3, J4), and (E5, J6, E7). After that, it returns to the processing of judging whether or not the prediction stack is empty (Step S403). Since the prediction stack is still not empty, the opening utterance E8of “Please fill in this form.” is displayed on the prediction display area1309as shown inFIG. 16, and an utterance by one of the users is waited for.

The dialog participant2may select an utterance from the example usage list1305. However, the desired utterance has already been displayed on the prediction display area1309. In the case where the dialog participant2selects the prediction display area1309as shown inFIG. 17, the opening utterance E8is translated into Japanese by the utterance processing unit102, and the utterance of

is notified to the dialog participant1. Likewise, the utterance prediction unit106updates the prediction stack to J2, (E3, J4), and (E5, J6, E7), and displays the utterance J2of

on the prediction display area1308as shown inFIG. 18.

The dialog participant1may select an utterance from the example usage list1304. However the desired utterance has already been displayed on the prediction display area1308. In the case where the dialog participant1selects the prediction display area1308as shown inFIG. 19, the utterance J2is translated into English by the utterance processing unit102, and the utterance of “Check-in, please.” is notified to the dialog participant2. Likewise, the utterance prediction unit106updates the prediction stack to (E3, J4), and (E5, J6, E7), and displays the utterance E3of “Have you made reservation?” on the prediction display area1309. In this way, the dialog participant1and the dialog participant2do not have to search the example usage list1304or the example usage list1305for a desired utterance, and smoothly communicate their intentions with each other. Therefore, it is possible to reduce the time to complete an on-going dialog.

The example case described next is the case where the dialog participant2does not select the utterance E3of “Have you made reservation?” which is predicted by the utterance prediction unit106on the prediction display area1309inFIG. 20, but selects the utterance E5of “Do you prefer smoking room?” by searching the example usage list1305for the utterance E5as shown inFIG. 21.

The utterance prediction unit106searches the prediction stack for the utterance E5, the utterance E5is not the opening utterance of the prediction stack. Therefore, the utterance prediction unit106moves the utterance block including the utterance E5to the opening part of the prediction stack so as to update the prediction stack to (E5, J6, E7), and (E3, J4). After that, it deletes the utterance E5so as to update the prediction stack to (J6, E7), and (E3, J4). Therefore, in the processing of displaying the candidate next utterance (Step S404), the utterance J6of

is displayed on the prediction display area1308as a candidate prediction utterance as shown inFIG. 22. After that, as shown inFIG. 23, the dialog participant1selects the utterance J6of

on the prediction display area1308and continues the dialog smoothly. In this way, even in the case where a dialog participant does not select the utterance predicted by the utterance prediction unit106, this embodiment makes it possible to flexibly support the dialog by predicting the future dialog which is made at the time of not selecting the predicted utterance.

Here, the effect of the present invention is quantitatively verified. In general, the degree of advancement of a dialog is proportional to the degree of similarity of dialog histories of dialog participants. In order to represent a degree of similarity as a specific number, the definition ofFIG. 9is used. In the case where the dialog participant1and the dialog participant2complete the dialog by selecting only the utterances displayed on the prediction display area1308and the prediction display area1309by the utterance prediction unit106, a dialog history2403shown inFIG. 24is formed. The dialog history2403is equivalent to the dmax (the dialog history d2inFIG. 3) determined by the dialog history selection unit107. The degree of similarity between the dialog history2401of the dialog participant1(the dialog history dr1determined by the dialog history selection unit107, that is, the dialog history d1inFIG. 3A) and the dialog history2402of the dialog participant2(the dialog history dr2determined by the dialog history selection unit107, that is, the dialog history d3inFIG. 3B) is 0.12. Compared to this, the degree of similarity between the dialog history2403and the dialog history2401is 0.41, and the degree of similarity between the dialog history2403and the dialog history2402is 0.16. Considering this, it is obvious that the dialog according to the prediction for which dmax is used will advance more smoothly than the dialog according to one of the dialog history dr1and the dialog history dr2. In addition, inFIG. 20, in the case where the dialog participant2continues and completes the dialog without selecting utterances predicted by the utterance prediction unit106, the dialog history2404is formed. The degree of similarity of the dialog history2404and the dialog history2401is 0.16, and the degree of similarity of the dialog history2404and the dialog history2402is 0.16. Considering this, it is obvious also in this use that the dialog according to one of the dialog history dr1and the dialog history dr2will advance more smoothly than the dialog according to the prediction for which dmax is used.

The already-described case is the case of assuming that the dialog participant1speaks Japanese and the dialog participant2speaks English in the dialog supporting system. Here will be described the case of assuming that the dialog participant1speaks Chinese and the dialog participant2speaks English.FIG. 25is a diagram showing an example of the interrelationship between the utterance numbers, the utterances of the dialog participant1and the utterances of the dialog participant2.

The utterance receiving unit101transforms the received utterance of the dialog participant1into the corresponding utterance information. The utterance information is, for example, an utterance number inFIG. 2. The utterance processing unit105translates the utterance (Chinese) of the dialog participant1into the corresponding utterance (English) of the dialog participant2, and outputs the corresponding character string. For example, in the case where the utterance number1is inputted, the utterance processing unit105outputs the character string of “May I help you?” to the utterance outputting unit104as the utterance information.

In contrast, the utterance receiving unit103translates the received utterance of the dialog participant2into the corresponding utterance information. The utterance information is, for example, an utterance number. The utterance processing unit105transforms the utterance (English) of the dialog participant2into the corresponding utterance (Chinese) of the dialog participant1and outputs the character string. For example, in the case where the utterance number1is inputted, the utterance processing unit105outputs the following character string to the utterance outputting unit102as the utterance information.

After that, in order to simplify the following description considering the language directions, the utterance number1inputted by the dialog participant1is abbreviated as C1and the utterance number1inputted by the dialog participant2is abbreviated as E1.

The utterance receiving unit101allows the dialog participant1to directly select an utterance from the list of Chinese utterances inFIG. 25, and outputs the utterance number as the utterance information. In addition, the utterance receiving unit103allows the dialog participant2to directly select an utterance from the list of English utterances inFIG. 25, and outputs the utterance number as the utterance information. As for the configuration of the utterance receiving unit101or the utterance receiving unit103, it may be realized by the processing of projecting the speech of a dialog participant to one of the utterance numbers using a speech recognition processing. In addition, it is realized by the processing of allowing a dialog participant to input a transcription or a character string corresponding to the pronunciation of the utterance through a keyboard and projecting the speech to one of the utterance numbers.

Each ofFIGS. 26A to 26Eis a diagram showing an example of a dialog history stored in the dialog history database201. A dialog history is a history of utterances made in the past by dialog participants, and in the dialog history the utterances of the dialog participants are placed in time sequence. Each utterance corresponds to one of the utterances defined inFIG. 25. There is a need to distinguish even the same utterance number depending on a dialog participant. Thus, C is assigned to the utterances made in Chinese and E is assigned to the utterances made in English, and the C and E each is defined as a label of the utterance in the dialog history. After that, in order to simplify the following description, a dialog history is described using column labels. For example, the dialog history d6is described as d6: E1, C2, E3, C4, E5, C6, E7and E8. The authentication information of the dialog participants who made the dialog and the topic of the dialog (the situation of the dialog) are recorded in the dialog history. For example, information indicating that the utterances assigned with C are Taro's, the utterances assigned with E are Tom's, and the topic relates to a hotel.

As shown inFIG. 4, here will be described an example case where the dialog participant1is a traveler who speaks Chinese, the dialog participant2is a clerk of a hotel who speaks English, and the topic for the dialog is to check in the hotel. Note that the operation flows of the utterance prediction unit106and the dialog history selection unit107are the same as the operation flows described above. Thus, they will be described with reference to flow charts shown inFIG. 5andFIG. 8, but the descriptions of the same parts are omitted here.

The utterance prediction unit106identifies the dialog participants and the topic of the dialog which are needed for predicting utterances first (Step S401). For example, in the dialog supporting apparatus100which is realized as a PDA or the like as shown inFIG. 27, the information concerning the dialog participants is obtained by allowing the participants to input directly through a touch panel the names of the dialog participant

1and the dialog participant

2, and the topic

of the dialog.

When the dialog start button

601is pressed as shown inFIG. 28, the utterance prediction unit106requests the dialog history selection unit107to generate a prediction stack (Step S402).

The dialog history selection unit107determines the dialog history dr1of the dialog participant1and the dialog history dr2of the dialog participant2which are included in the dialog history database201, based on the information concerning the authentication information of the dialog participants and the topic of the dialog which are notified by the utterance prediction unit106(Step S701). For example, in the case where a dialog history shown in one ofFIGS. 26A to 26Eis present in the dialog history database201, it is determined that the dialog history dr1is the dialog history d6and the dialog history dr2is the dialog history d8. This is because the utterances assigned with C in the dialog history d6are Taro's, the utterances assigned with E in the dialog history d8are Hanako's, and the topic of the dialog relates to a hotel. The dialog history d5is not selected because the topic of the dialog relates to a rented car while the utterances assigned with J in the dialog history d5are Taro's.

Next, the dialog history selection unit107calculates scores, which can be obtained by the equation “score(d)=r(d|dr1)+r(d|dr2)”, of the dialog histories d's other than the dialog histories dr1and dr2which are included in the dialog history database201. After that, the dialog history selection unit107selects the dialog history d=the dialog history dmax which provides the maximum score(d) (Step S703). Here, the dialog history d7is assumed to be selected.

The dialog history selection unit107applies the algorithm for determining the utterance blocks ofFIG. 12to the dialog history dmax as well as the dialog history dr1and the dialog history dr2, divides the dialog history dmax into utterance blocks so as to generate a prediction stack (Step S704). For example, in the case where the dialog history dmax=the dialog history d7is divided into the dialog history dr1=the dialog history d6and the dialog history dr2=the dialog history d8, the dialog history d7: E1; E8; C2; E3; C4; E5; C6and E7is divided by the dialog history d6first into three utterance blocks of the dialog history d7: E1; E8; and (C2; E3; C4; E5; C6and E7). The three utterance blocks are further divided by the dialog history d8next into five utterance blocks of the dialog history d7: E1; E8; C2, (E3and C4), and (E5, C6and E7). The dialog history selection unit107notifies the utterance prediction unit106of this dialog history d7as a prediction stack.

The utterance prediction unit106which received an utterance prediction stack generated in this way from the dialog selection unit107judges whether or not each of the prediction stacks is empty (Step S403). In the case where the prediction stack is empty for the reason that the dialog history selection unit107cannot select any appropriate dialog history in the dialog history database201or for another reason (Step S403: YES), the utterance prediction unit106completes the processing without performing any utterance prediction action.

On the other hand, in the case where the prediction stack is not empty (Step S403: NO), the utterance prediction unit106displays the opening utterance of the prediction stack as the candidate next utterance (Step S404).FIG. 29is a diagram showing the status of the display of the dialog supporting apparatus100at the time when the dialog is started. The upper half of the display area is the utterance input and output area1301intended for the dialog participant1and the lower half of the display area is the utterance input and output area1302intended for the dialog participant2. An exit

button1303is the button pressed when the dialog is completed. The dialog participant1and the dialog participant2can select an arbitrary utterance from among all the utterances defined inFIG. 25using the example usage list1304and the example usage list1305, respectively, and can notify the meaning of the utterance to the other dialog participant. The utterances translated by the utterance processing unit105are respectively outputted to one of the utterance outputting unit1306of the dialog participant1and the utterance outputting unit1307of the dialog participant2. After that, these utterances are read out using speech synthesis processing installed in a PDA or the like at the same time when they are displayed. The utterances for the dialog participant1which are predicted by the utterance prediction unit106are displayed on the prediction display area1308, and the utterances for the dialog participant2which are predicted by the utterance prediction unit106are displayed on the prediction display area1309. The dialog participant1and the dialog participant2can omit the operation for searching an example usage list1304and an example usage1305for the utterance to be selected, by selecting the utterances displayed on the prediction display area1308and the prediction display area1309, respectively. In other words, the dialog participants1and2can input utterances immediately.

In the case where a prediction stack is generated as in the above-described case, the utterance prediction unit106displays the utterance of “May I help you?” which is the opening utterance in the prediction stack E1, E8, C2, (E3, C4), and (E5, C6, E7) on the prediction display area1309. The dialog participant2may select an utterance from the example usage list1305. However, since the desired utterance has already been displayed on the prediction display area1309, the dialog participant2selects it as shown inFIG. 30. The utterance E1is translated into Chinese in the utterance processing unit102, and the utterance of

is notified to the utterance participant1.

At this time, the utterance prediction unit106judges whether or not the utterance is inputted by a dialog participant (Step S405). When the utterance is inputted by the dialog participant (Step S405: YES), the utterance prediction unit106searches the prediction stack for a matching utterance starting with the opening utterance (Step S406), and judges whether or not there is a matching utterance (Step S407). In the case where there is a matching utterance (Step S407: YES), the utterance prediction unit106judges whether or not the matching utterance is the opening utterance of the prediction stack (Step S408). In the case where it is the opening utterance (Step S408: YES), it deletes the opening utterance of the prediction stack so as to update the prediction stack (Step S409). On the other hand, in the case where the utterance is not the opening utterance (Step S408: NO), it moves the block including the matching utterance to the opening part of the prediction stack, deletes the matching utterance and the utterances placed before the matching utterance so as to update the prediction stack (Step S410). After that, the utterance prediction unit106returns to the processing of judging whether or not the prediction stack is empty (Step S403).

Since the utterance of a dialog participant is the utterance E1in the above example, the prediction stack is updated to E8, C2, (E3, C4), and (E5, C6, E7). After that, it returns to the processing of judging whether or not the prediction stack is empty (Step S403). Since the prediction stack is still not empty, the opening utterance E8of “Please fill in this form.” is displayed on the prediction display area1309as shown inFIG. 31, and an utterance by one of the users is waited for.

The dialog participant2may select an utterance from the example usage list1305. However, the desired utterance has already been displayed on the prediction display area1309. In the case where the dialog participant2selects the prediction display area1309as shown inFIG. 32, the utterance E8is translated into Chinese by the utterance processing unit102, and the utterance of

is notified to the dialog participant1. Likewise, the utterance prediction unit106updates the prediction stack to C2, (E3, C4), and (E5, C6, E7), and displays the utterance of

The dialog participant1may select an utterance from the example usage list1304. However, the desired utterance has already been displayed on the prediction display area1308. In the case where the dialog participant1selects the prediction display area1308as shown inFIG. 34, the utterance C2is translated into English by the utterance processing unit102, and the utterance of “Check-in, please.” is notified to the dialog participant2. Likewise, the utterance prediction unit106updates the prediction stack to (E3, C4), and (E5, C6, E7), and displays the utterance E3of “Have you made reservation?” on the prediction display area1309. In this way, the dialog participant1and the dialog participant2do not have to, respectively, search the example usage list1304and the example usage list1305for a desired utterance and smoothly communicate their intentions with each other. Therefore, it is possible to reduce the time to complete the dialog.

As shown inFIG. 36, it should be noted that a dialog supporting apparatus according to a second embodiment can be configured to have a history registration unit401in addition to the configuration shown inFIG. 1so that the utterance processing unit105can record all the pieces of utterance information made between dialog to participants and that the history registration unit401can store the record in the dialog history database201. In this way, it becomes possible to increase the number of dialog histories which are effective in the future utterance prediction. For example, as shown inFIGS. 37 and 38, this operation is executed at the time when a dialog participant instructs the end of the dialog. Especially, in the case where the dialog is made without according to the prediction of the utterance prediction unit106, the dialog is not present in the dialog history database201in many cases. Therefore, to store the dialog makes it possible to perform the next utterance prediction with an improved accuracy.

In addition, the dialog supporting apparatus of the second embodiment can be configured so that it includes a speech recognition unit501and a speech recognition unit502as shown inFIG. 39in place of the utterance receiving unit101and the utterance receiving unit103in the configuration shown inFIG. 1. In this case, it becomes possible to improve the accuracy of speech recognition by preferentially selecting the vocabularies related to the utterances to be predicted by the utterance prediction unit106among the recognition target vocabularies of the speech recognition unit501and the speech recognition unit502. Conceivably related vocabularies include: the prediction utterances themselves which are outputted by the utterance prediction unit106; sentences related to the prediction utterances; words included in the prediction utterances; and words associated with the prediction utterances.

In addition, as shown inFIGS. 40A,40B,41A and41B, it is also possible to further improve its usability by enabling each dialog participant to use a PDA implementing a dialog supporting apparatus. Here, dialog supporting apparatuses are implemented on a PDA2301used by the dialog participant1and a PDA2301used by the dialog participant2, respectively. A button2303and a button2304are intended for enabling the respective dialog participants1and2to view the prediction development of the dialog. More specifically, when one of these buttons is pressed, the utterance prediction unit106displays the utterances made so far and all the contents of the current prediction stack.FIGS. 42A and 42Beach is a diagram showing an example (Japanese and English) of prediction development of the dialog. It is assumed that the utterances made so far and the contents of the prediction stack are translated into the language of each dialog participant and displayed. Here, J1, J8, J2, (J3, J4), and (J5, J6, J7) are displayed on the prediction development display area2401, and E1, E8, E2, (E3, E4) and (E5, E6, E7) are displayed on the prediction development display area2402.

In addition,FIGS. 43A and 43Beach is a diagram showing an example (Chinese and English) of prediction development of the dialog. It is assumed inFIGS. 43A and43B that the utterances made so far are E1, E8and C2and that the current prediction stack is (E3, C4), and (E5, C6, E7). The utterances made so far and the contents of the prediction stack are translated into the respective languages of the dialog participants and displayed. Here, C1, C8, C2, (C3, C4), and (C5, C6, C7) are displayed on the prediction development display area2401, and E1, E8, E2, (E3, E4) and (E5, E6, E7) are displayed on the prediction development display area2402.

To show dialog prediction development like this to the dialog participants enables the dialog participants to know an utterance of the other dialog participant. Since the dialog participants can organize their thoughts before making an utterance, they can handle the dialog supporting apparatus more accurately.

A case of Japanese and English and another case of Chinese and English are taken as examples described in the second embodiment. However, other languages such as French can also be the case described in the second embodiment, in other words, the present invention does not depend on languages.

The dialog supporting apparatus of the present invention has a function for inputting utterances of dialog participants smoothly. It is useful as a translation application software or the like of mobile phones and mobile terminals. In addition, it is applicable in the use of public town terminals and guidance terminals and the like. Further, it is applicable in the use of, for example, a chat system where typical sentences are used.