Input apparatus, input method and input program

An input apparatus which presents examples suitable to users, including an example storage module to store a plurality of example expressions; an edit storage module to store edits when a user edits the plurality of example expressions; a presentation example determining module to determine one or more presentation examples to be presented to the user from the plurality of example expressions stored in the example storage module; an edit adapting module to edit the presentation examples determined by the presentation example determining module based on the edits stored in the edit storage module; a display control module to present the presentation examples determined by the presentation example determining module to the user; and an entry accepting module to accept one of the presentation examples as a selected example when one of the presentation examples is selected by the user, or to receive edits from the user to one of the presentation examples and accept the edited presentation example as the selected example.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority from prior Japanese Patent Application No. 2005-284282, filed Sep. 29, 2005, the entire contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an input apparatus, an input method and an input program for providing an example expression to a user that may be suitable to the user's usage history.

2. Description of the Related Art

In recent years, the practical use of human interfaces utilizing speech input has expanded. For instance, a speech-operated system exists in which a user provides speech input to enter a particular preset command, and the system recognizes the speech input to execute the operation corresponding to the command, thus allowing the user to operate the system by speech. In addition, a system is available in which a user speaks a sentence, and the system analyzes and converts the sentence to a character string, thus allowing the creation of text by speech input.

Furthermore, the following systems also exist: a speech-to-speech translation system which translates speech input in a first language into a second language and outputs the translated speech to support interlingual communications, and a spoken dialog system which is intended to allow for spoken interaction between a user and the system.

In these systems, speech recognition technology is used which converts a speech signal spoken by a user to digital data, and compares the data with predetermined data patterns to recognize the speech.

In speech recognition, incorrect recognition is likely to occur because of the influence of noise in the surrounding environment, as well as differences in speech quality, volume and speech speed of a particular user. In addition, since a user has to provide a speech input again if the first speech input is incorrectly recognized, significant effort can be required to operate such systems. Furthermore, when a given sentence is accepted, subsequent processing loads such as for comparison processing using accepted sentences are increased due to the additional data, and this can cause more difficulty in obtaining sufficient accuracy in the speech recognition process.

In Patent Reference 1 (Jpn. Pat. Appln. KOKAI No. 2000-200275), an input apparatus is proposed in which examples are presented to a user including descriptions of speech input entries, the user references the examples, and the user partially edits one of the examples to create a speech input. According to this method, since the entries can be restricted to the examples presented to the user or the partially edited example, the speech recognition process can be performed with a high degree of accuracy. In addition, since the user can reference the examples to obtain the guidance of the speech input entry descriptions, speech entry processing can be performed more easily.

However, the method of Patent Reference 1 has the limitation that the examples to be presented to the user are already prepared in the input apparatus, and therefore suitable candidates cannot be presented for all users such as those having differences in gender, ages, occupations and travel experiences.

Thus, in most cases, it is necessary to partially edit an example using this method, and thus a user has to make a speech input almost all the time. In addition, the method requires speech recognition that is sometimes performed incorrectly. Thus, a user may have to perform multiple iterations of speech reentry when incorrect speech recognition cannot be reduced.

The invention has been made in view of these circumstances.

SUMMARY OF THE INVENTION

According to embodiments of the present invention, an input apparatus includes: an example storage module to store a plurality of example expressions; an edit storage module to store edits when a user edits the plurality of example expressions; a presentation example determining module to determine one or more presentation examples to be presented to the user from the plurality of example expressions stored in the example storage module; an edit adapting module to edit the presentation examples determined by the presentation example determining module based on the edits stored in the edit storage module; a display control module to present the presentation examples determined by the presentation example determining module to the user; and an entry accepting module to accept one of the presentation examples as a selected example when one of the presentation examples is selected by the user, or to receive edits from the user to one of the presentation examples and accept the edited presentation example as the selected example.

In addition, the invention is an input method and an input program, which can implement the apparatus.

According to embodiments of the present invention, an editing done by a user is reflected to present an example having a suited entry description for every user. Accordingly, such advantages can be exerted that the necessity to make an editing input for an example is reduced and operation efforts done by the user can be reduced.

DETAILED DESCRIPTION OF EMBODIMENTS

Hereinafter, the best modes of an input apparatus, an input method and an input program according to embodiments of the invention will be described in detail with reference to the drawings.

First Embodiment

In a first embodiment, an input apparatus according to the invention will be described as a part of a speech translation apparatus which translates Japanese into English using example-based translation. The possible combinations of original language and target language are not, however, limited to Japanese and English. The invention can be adapted to any combination of languages.

In addition, the input apparatus is not limited to a speech translation apparatus. Embodiments of the invention can be adapted to any device with functionality to present examples with predetermined entry descriptions and to accept an example selected by a user from the presented examples. For instance, the input apparatus according to embodiments of the invention can be adapted to input interfaces such as a car navigation system and a home electrical product control system.

A speech translation apparatus according to the first embodiment stores an edit when a user edits an example expression stored in a storage module. Before the example expression is presented to the user, the stored example expression is edited according to the edits stored in the storage module that correspond to the example expression.

FIG. 1shows a block diagram depicting the configuration of a speech translation apparatus100according to the first embodiment. As shown in the drawing, the speech translation apparatus100has an example storage module111, an edit storage module112, a location information acquiring module113, an input/output device114, a use condition detecting module101, a presentation example determining module102, an edit adapting module103, a display control module104, an example selection accepting module105, a speech input accepting module106, an analyzing module107, and a translation and output module108.

The example storage module111stores examples of translations which are pairs of semantically equivalent Japanese and English example expressions, and can be comprised of typical storage modules such as a HDD (Hard Disk Drive), an optical disc, a memory card, or a RAM (Random Access Memory).

FIG. 2shows an illustration depicting an exemplary data structure of example translations to be stored in the example storage module111. As shown in the drawing, the example storage module111associates an example ID which uniquely identifies an example translation, including a Japanese and an English expression that have the same meaning. The example storage module111may also store situations where the expressions are used.

The type of situation where the corresponding example expression is used is stored in the situation column. For instance, “TRANSPORT” represents a situation relating to the use of transportation, “FAMILY” represents a situation relating to families and relatives, and “SIGHTSEEING” represents a situation relating to sightseeing.

The examples denoted by example ID's p01to p14in the drawing are examples which are associated with words that are used independently of the stored situations, and these are identified by the symbol “-” in the situation column. Conversely, the examples denoted by example ID's q01to q07in the drawing are examples related to various situations in which the apparatus can be used. Words included in a sentence can be terminated with a symbol “#” followed by a semantic attribute (semantic class) which denotes the meaning of the word or words. As shown, the symbols “[” and “]” denote the words referred to by the “#” symbol and thus corresponding to the semantic class.

For instance, “#CITY” is a semantic class denoting a city, “#PERSON” is a semantic class denoting a person, “#POI” is a semantic class denoting a point of interest, and “#AIRPORT” is a semantic class denoting an airport.

The portion of the expression corresponding to the semantic class is the portion that is likely to be edited by the edit adapting module103, as described later. In addition, in the examples shown in the drawing, the semantic class is assigned only to noun expressions, but this configuration may be expanded to other parts of speech.

The example storage module111stores the correspondence between individual portions of Japanese and English expressions. Thus, it is possible to determine which portion of an expression in one language corresponds to which portion of an expression in the other language.

The edit storage module112stores an edit when a user edits an example presented to the user. The editing storage module112can be comprised of typical storage modules such as a HDD, an optical disc, a memory card or a RAM.

FIG. 3shows an illustration depicting an exemplary data structure of edits stored in the edit storage module112. As shown in the drawing, the edit storage module112associates an edit ID which uniquely identifies an edit, and associates the edit with a source example ID representing the example ID of the source example which was edited, a subject expression, an edited result expression, and a semantic class.

Here, the subject expression denotes the original wording of the part of the source example that has been edited. The edited result expression denotes the modified wording used to replace the subject expression. In the “semantic class” column, a semantic class is stored when there is a semantic class corresponding to the subject expression.

For instance, suppose that for an example ID=q01shown inFIG. 2, an edit is obtained where a Japanese user edits the Japanese words for “New York” and replaces them with the Japanese words for “Los Angeles”. In this case, “q01” is the source example ID, the Japanese words for “New York” are the subject expression, and the words “Los Angeles” are the edited result expression. In addition, since the semantic class “#CITY” for the Japanese words “New York” of the subject expression can be obtained from the example ID=q01in the example storage module111, this value is also set in the semantic class column in the edit storage module112. This is shown inFIG. 3as an edit corresponding to an edit ID=r01.

The edits stored in the edit storage module112can be deleted in accordance with a predetermined elapsed time period, a change of locations, a change of users, etc. In addition, a plurality of edit storage modules112may be prepared in order to substitute for the edit storage module112according to a predetermined elapsed time period, a change of locations, a change of users, etc.

The location information acquiring module113acquires location information about the latitude and longitude of the speech translation apparatus100with a GPS reception function. However, the location information is not necessarily limited to the latitude and longitude. Any information may be used as long as the information can identify where the speech translation apparatus100is located.

In addition, the location information acquiring module113may be configured to obtain location information from an access point of a wireless LAN (Local Area Network), detected when a communication module (not shown) is connected to the LAN. The location information acquiring module113may also be configured to obtain location information through a detecting module (not shown) from a wireless tag or a bar code containing location information.

The input/output device114has a display such as a liquid crystal display which displays an operation screen and examples determined by the presentation example determining module102, an input device for input from a button, a keyboard, and a speech input microphone, and an output device such as a speaker which outputs a translation created by the translation and output module108in synthesized speech in the target language.

In addition, the input/output device114may include a touch panel which serves as the display as well as the input device. Thus, a user can touch the touch panel to select an example from a plurality of the examples presented on the display.

The use condition detecting module101detects a use condition for the speech translation apparatus100from location information acquired by the location information acquiring module113. Here, the use condition is information that specifies the situation in which the speech translation apparatus100is being used. For example, in the situation where a user is using transportation, “use of transportation” is detected as the use condition, in the situation where a user visits a tourist attraction, “sightseeing” is detected as the use condition, and in the situation in which a user talks about family, “family” is detected as the use condition.

In addition, a plurality of preset use conditions may be presented to a user, in this case the use condition detecting module101detects the use condition selected by the user.

The presentation example determining module102acquires the example associated with situation information relating to the detected use condition from the example storage module111, and determines that this example is to be presented to a user.

For instance, when the use condition detecting module101detects the use condition where the speech translation apparatus100is being used in the airport, the presentation example determining module102selects the example associated with the situation “TRANSPORT” relating to the use of transportation. In the example shown inFIG. 2, the examples of example ID's q01to q04are determined as the presentation examples to present to the user.

The edit adapting module103refers to the edits stored in the edit storage module112, and edits the presentation example or examples determined by the presentation example determining module102. There are two methods described below for editing a presentation example.

In the first method, when the presentation example includes the expression described in the column of the subject expression in the edit storage module112, the portion of the presentation example corresponding to the subject expression is replaced by the corresponding edited result expression.

In the second method, when the presentation example includes an expression with the same semantic class as the semantic class described in the semantic class column in the edit storage module112, the portion of the presentation example corresponding to the semantic class is replaced by the edited result expression associated with the semantic class.

The display control module104presents the operation screen and the example determined by the presentation example determining module102on the display of the input/output device114.

The example selection accepting module105accepts a selection by a user of one of the presentation examples presented on the display of the input/output device114. For instance, as described above, the input/output device114can be configured to allow a user to select the presentation example with the touch panel that also serves as the display and the input device.

The method of accepting the presentation example is not limited to those described above. For instance, a button associated with each presentation example may be pressed to accept a selection of the presentation example. Alternately, a symbol such as a cursor or mark may identify one of the examples, and a button or dial provided to move the symbol, and the press of a button used to select the presentation example identified by the symbol.

The speech input accepting module106receives speech input from a user through a speech input microphone, converts the received speech to an electric signal that can be processed by the analyzing module107, and outputs the signal. More specifically, speech input accepting module106receives speech, converts the speech to an electric signal by A/D (analog/digital) conversion, and outputs digital data converted to a PCM (pulse code modulation) representation, for instance. These processes can be implemented with existing methods for digitizing speech.

The example selection accepting module105and the speech input accepting module106can correspond to an entry accepting module according to embodiments of the invention.

The analyzing module107receives a speech signal in the form of digital data from the speech input accepting module106, and executes a speech recognition process. In addition, the analyzing module107compares the presentation example with the speech recognition result and detects the unmatched portion. The analyzing module107specifies the unmatched portion of the presentation example as the edited portion. The analyzing module107then outputs edits corresponding to the presentation example, i.e., edits that associate the expression in the edited portion before editing with the expression after editing.

More specifically, the analyzing module107performs frequency analysis using the FFT (fast Fourier transform), and outputs characteristic information (for instance, a spectrum) necessary for speech recognition for every predetermined section of input speech in a time series. For instance, a predetermined section can be a character string such as words, a phoneme sequence such as accent phrases, and units such as phonemes and syllables. This process can be implemented by conventional speech analysis technology.

Furthermore, the analyzing module107compares the characteristic information with a speech analysis pattern stored in a speech recognition dictionary (not shown) to compute scores showing the similarity between the patterns in the dictionary and the spoken input, and outputs recognition candidates having the highest scores. Generally used methods such as Hidden Markov Models (HMM), Dynamic Programming (DP), and Neural Networks (NN) may be employed.

In addition, in order to improve recognition accuracy, a statistical language model, for example an N-gram model, can be used with the recognition candidates, and the candidate with the highest probability can estimated and selected.

In addition, in the first embodiment, the speech recognition process can be performed by selection of the presentation example by a user, or the presentation example can be partially edited and spoken for speech input. Therefore, in comparison to speech recognition based only on a speech input, the speech recognition process can be performed with a higher degree of accuracy.

More specifically, the analyzing module107performs a comparison between the phoneme sequence created from words forming each of the presentation examples and a phoneme sequence candidate previously obtained from speech recognition by a technique such as DP used before. Then, the source example is determined by the degree of matching in comparison to the candidate, and the unmatched portion between the presentation example and the speech recognition result is detected, so the edited portion of the presentation example can be specified.

The translation and output module108creates an English translation of the Japanese expression input by a Japanese user, and outputs the English translation on the display. In addition, the English translation can be output by speech; in this case the translation and output module108synthesizes speech from the English translation, and outputs synthesized speech to an output device such as the speaker.

Methods generally known can be adapted for the speech synthesis process, including a text-to-speech system using speech synthesis by editing speech segments, format speech synthesis, etc.

Next, a speech translation process performed by the speech translation apparatus100according to the first embodiment will be described.FIG. 4shows a flow chart depicting the overall flow of the speech translation process according to the first embodiment.

First, the use condition detecting module101detects a use condition, and stores it in a temporary storage module such as a RAM (not shown) (Step S401). The stored use condition is referenced for determining the presentation examples to be presented by the presentation example determining module102.

Subsequently, the example storage module111and the edit storage module112are referenced to determine an example for presentation, and a presentation example creation process is performed to create an edited example reflecting the edits (Step S402). The details of the presentation example creation process will be described later.

Subsequently, the display control module104displays the example created in the presentation example creation process on the display of the input/output device114(Step S403). Then, the example selection accepting module105determines whether a user directly selects the example (Step S404). To directly select the example means that the user selects the example on the touch panel, or selects the example by pressing down the corresponding button without editing the example displayed on the screen.

When the user directly selects the example (Step S404: YES), a translation process1is performed to create a translated sentence corresponding to the selected example (Step S407). The detail of the translation process1will be described later.

When it is determined that the user has not directly selected the example (Step S404: NO), the speech input accepting module106determines whether the user provides a speech input (Step S405).

When it is determined that a speech input is provided (Step S405: YES), the analyzing module107analyzes the input speech to extract editing information (Step S408). More specifically, the input speech undergoes speech recognition to analyze which presented example the input speech corresponds to, and, whether a portion to be edited exists in this presentation example. When a portion to be edited exists, the subject expression (that is, the expression before editing) and the edited result expression (that is, the expression after editing) are acquired. In addition, the semantic class associated with the subject expression is acquired from the example storage module111.

As described above, the source example for editing is specified, and edits can be extracted that associate the source example ID (i.e., the presentation example ID), the subject expression, the edited result expression, and the semantic class with one another.

Subsequently, the analyzing module107registers the extracted edits in the edit storage module112(Step S409). Thus, the edits can be referenced in subsequent processes, and the example stored in the example storage module111can be edited.

Subsequently, a translation process2is performed to create a translated sentence corresponding to the speech input (Step S410). The details of the translation process2will be described later.

After performing the translation process1at Step S407or the translation process2at Step S410, the translation and output module108outputs the translation to the display of the input/output device114or the output device such as the speaker (Step S411).

In addition, the number of examples to be output may be limited in accordance with the frequency of use of the example. Thus, a more suitable example can be presented for the user, and the effort required to select an example can be reduced. For example, when the display area in a mobile device is small, it is possible to eliminate some of the effort required to scroll the screen to confirm all the examples.

When it is determined that a speech input has not been made at Step S405(Step S405: NO), the use condition detecting module101determines whether the use condition is changed (Step S406).

When the use condition is not changed (Step S406: NO), the process returns to the determination process for determining whether the user directly selects the example, and the process is repeated (Step S404).

When the use condition is changed (Step S406: YES), or after the translation is output at Step S411, the process returns to Step S402for creating and displaying another example, and the process is repeated.

Next, the detail of the presentation example creation process shown at Step S402will be described.FIG. 5shows a flow chart depicting the overall flow of the presentation example creation process according to the first embodiment.

First, the presentation example determining module102initializes an example register which stores the created example (Step S501). The example register is contained in the presentation example determining module102and is a storage area in a temporary storage module such as a RAM.

Subsequently, the presentation example determining module102acquires the example matched with the use condition from the example storage module111(Step S502). The acquired example is denoted by R1to Rm (m is the number of acquired examples).

For instance, suppose that the location information acquiring module113acquires location information indicating that the place at which the speech translation apparatus100is being used is an airport or other transportation station by the GPS reception function. Then, the use condition detecting module101detects that the use condition is “use of transportation”.

Suppose further that the example storage module111stores the examples shown inFIG. 2. Then, for the examples associated with “TRANSPORT,” that is, the situation relating to the detected use condition “use of transportation,” the presentation example determining module102acquires the examples of the example ID's q01to q04from the example storage module111.

Subsequently, the presentation example determining module102acquires all the edits from the edit storage module112(Step S503). In addition, the acquired edits are denoted by T1to Tp (p is the number of edits acquired).

Subsequently, the presentation example determining module102initializes an example counter j which counts the examples after they are processed (Step S504). The counter is initialized by setting j equal to 1.

Subsequently, the presentation example determining module102determines whether all the examples have been processed (Step S505), i.e., presentation example determining module102determines whether the example counter j is greater than the number m of acquired examples.

When all the examples have not been processed (Step S505: NO), an edit counter k is initialized which counts the edits after the edits have been processed (Step S506). Edit counter k is initialized by setting k to 1.

Subsequently, the presentation example determining module102determines whether all the edits are processed (Step S507). More specifically, it is determined whether the edit counter k is greater than the number p of acquired edits.

If all the edits are not processed (Step S507: NO), edits are adapted to perform an edited example creation process to create an edited example (Step S508). The details of the edited example creation process will be described later.

After performing the edited example creation process, the edit counter k is incremented by 1 (Step S509), the process returns to the determination process for determining whether all the edits have been processed, and the process is repeated (Step S507).

When it is determined that all the edits have been processed at Step S507(Step S507: YES), the presentation example determining module102determines whether the edited example is added in the edited example creation process (Step S510).

When the edited example is not added (Step S510: NO), the presentation example determining module102adds an example Rj in the example register (Step S511). If example Rj is edited, the edited example is added to the example register. It is necessary to add the example Rj itself to the example register when the example Rj is not edited.

In addition, when an identical example already exists in the example register, the presentation example determining module102does not add the example to the example register.

If the edited example is added at Step S510(Step S510: YES), or if the example Rj is added to the example register at Step S511, the presentation example determining module102increments the example counter by 1 (Step S512). After the example counter is incremented by 1, the presentation example determining module102returns to the determination process for determining whether all the examples have been processed, and the process is repeated (Step S505).

When it is determined that all the examples have been processed at Step S505(StepS505: YES), the presentation example determining module102determines the presentation examples from the example register, and outputs the presentation examples (Step S513).

Next, the details of the edited example creation process shown at Step S508will be described.FIG. 6shows a flow chart depicting the overall flow of the edited example creation process according to the first embodiment.

First, the edit adapting module103determines whether a subject expression S of edit Tk appears in an example Rj (Step S601). When Rj contains the subject expression S (Step S601: YES), an edited example Rj′ is added to the example register. The portion corresponding to the subject expression S in the example Rj is replaced by an edited result expression T in editing information Tk to form Rj′ (Step S602).

For instance, suppose that the example expressions of the example ID's q01to q04shown inFIG. 2are acquired, and the edits for edit ID's r01and r02as shown inFIG. 3are acquired. Suppose further that the edited example creation process is performed where the example counter j=1, and the edit counter k=1. The subject expression S of edit Tk is the subject expression “New York (in Japanese)” in the edit represented by the edit ID=r01.

The example Rj corresponds to example ID=q01“I'll go to [New York#CITY] (in Japanese)”. Since the subject expression “New York (in Japanese)” appears in the example, this portion is replaced by the edited result expression “Los Angeles (in Japanese)” in the edit represented by the edit ID=r01, and edited example Rj′, “I'll go to Los Angeles (in Japanese)” is added to the example register.

If the subject expression S of edit Tk does not appear in the example Rj at Step S601(Step S601: NO), the edit adapting module103determines whether a semantic class C of edit Tk appears in the example Rj (Step S603).

If so, (Step S603: YES), an edited example Rj″ is added to the example register, where in RJ″ the portion represented by the semantic class C in the example Rj is replaced by the edited result expression T in edit Tk (Step S604).

Based on the same premise as described above, suppose that the edited example creation process is performed where the example counter j=2, and the edit counter k=1. The semantic class C of edit Tk is the semantic class “#CITY” in the edit represented by the edit ID=r01.

In addition, the example Rj corresponds to example ID =q02, “What time does it arrive at [Chicago #City]? (in Japanese)”. Since the semantic class “#CITY” appears in the example, this portion is replaced by the edited result expression “Los Angeles (in Japanese)” in the edit represented by the edit ID=r0I, and the edited example Rj″ “What time does it arrive at Los Angeles? (in Japanese)” is added to the example register.

If the semantic class C of edit Tk does not appear in the example Rj at Step S603(Step S603: NO), or after the edited example is added to the example register at Step S602or Step S604, the edited example creation process is ended.

Since it is necessary to acquire the source example for editing in the translation process (described later), at Steps S602and S604, information that can identify the source example for editing is associated with the edited example and output to the example register.

As described above, in the first embodiment, the examples registered beforehand in the example storage module111can be presented, and the edit storage module112can be referenced, to edit an example to reflect previous edits. Therefore, each users' history can be used to produce examples suitable to the users.

Next, the details of the translation process1shown at Step S407will be described. The translation process1uses the same edits discussed above when the Japanese expression is converted to the English expression, and thus creates the correct translation corresponding to the selected Japanese example.

FIG. 7shows a flow chart depicting the overall flow of the translation process1according to the first embodiment.

First, the translation and output module108acquires the source example for editing (hereinafter, referred to as an example Y0) of the example selected by a user (hereinafter, referred to as an example Y1) (Step S701), and determines whether the example Y0is edited to create an example Yi (Step S702).

As described above, since information that identifies the source example for editing is output to the example register, this information can be referenced to acquire the example Y0. In addition, whether to store information that identifies the source example for editing can determine whether the example Y0is edited to create the example Yi.

If the example Y0is not edited to create the example Yi (Step S702: NO), that is, when the user selects the presentation example with no editing, the translation and output module108acquires the English expression of the example Yi from the example storage module111, and creates it as a translation (Step S707).

When it is determined that the example Y0is edited to create the example Yi (Step S702: YES), the translation and output module108acquires the subject expression (hereinafter, referred to as subject expression O) and the edited result expression (hereinafter, referred to as edited result expression M) of the edited portion (Step S703).

For instance, when the example expression “I'll go to [New York#CITY] (in Japanese)” is edited to “I'll go to Los Angeles (in Japanese)”, the subject expression O is the words “New York (in Japanese)”, and the edited result expression M is the words “Los Angeles (in Japanese)”.

Subsequently, the translation and output module108acquires the example translation (hereinafter, referred to as example translation Pj) with the edited result expression M in the Japanese expression column from the example storage module111(Step S704). For instance, when the edited result expression M is the words “Los Angeles (in Japanese)” and the example translations shown inFIG. 2are stored in the example storage module111, the example translation with example ID =p02is acquired.

Subsequently, the translation and output module108acquires the English expression of the example translation Pj (hereinafter, referred to as English expression Ej) from the example storage module111(Step S705). In the example described above, the words “Los Angeles (in English)” are acquired.

Subsequently, the translation and output module108creates a translation. The English expression of the source example Y0(hereinafter, referred to as an English expression E0) has a portion corresponding to the subject expression O. This portion is replaced by the English expression Ej to create the translation (Step S706). In the example described above, the English expression E0is “I'll go to New York”, and the translation “I'll go to Los Angeles (in English)” is created by replacing the words “New York” with the English expression Ej (“Los Angeles”).

If there are multiple edited portions, the processes from Step S703to Step S706are performed for all the edited portions.

After the translation and output module108creates the translation at Step S706or Step S707, the translation and output module108ends the translation process1.

Next, the details of the translation process2shown at Step S410will be described. The translation process2uses the same edits used when the user input edits the presented example and applies these edits to the English expression, thus creating the correct translation corresponding to the selected Japanese example.

FIG. 8shows a flow chart depicting the overall flow of the translation process2according to the first embodiment.

First, the translation and output module108acquires edits output from the analyzing module107(hereinafter, referred to as edits W) (Step S801), and determines whether the presentation example is edited by a user (Step S802).

The edits W include the source example for editing (hereinafter, referred to as example Yw), the subject expression, that is the expression before being edited (hereinafter, the subject expression Ow), and the edited result expression, that is, the expression after being edited (hereinafter, the edited result expression M). In addition, when the user does not edit the presented example, the edits W include information indicating that no editing has been done. Thus, the translation and output module108references to this information to determine whether there are edits.

If the presentation example is not edited at Step S802(Step S802: NO), that is, when the user selects the example by speech input with no editing to the presentation example, the translation and output module108acquires the English expression of the example Yw from the example storage module111, and creates it as the translation (Step S806).

If the presentation example is edited (Step S802: YES), the translation and output module108acquires the example translation with an edited result expression M in the Japanese expression column (hereinafter, referred to as example Yz) from the example storage module111(Step S803). For instance, when the edited result expression M is the words “Los Angeles (in Japanese)” and the example translations shown inFIG. 2are stored in the example storage module111, the example translation for example ID=p02is acquired.

Subsequently, the translation and output module108acquires the English expression of the example Yz (hereinafter, referred to as English expression Ez) from the example storage module111(Step S804). In the example described above, the words “Los Angeles (in English)” are acquired.

Subsequently, the translation and output module108creates a translation. The English expression of the source example Yw (hereinafter, English expression Ew) has a portion corresponding to the subject expression Ow. This portion is replaced by the English expression Ez (Step S805). In the example described above, the English expression Ew is “I'll go to New York.” The translation “I'll go to Los Angeles (in English)” is created by replacing the words “New York” with the English expression Ez (“Los Angeles”).

In addition, when there are multiple edited portions, the processes from Step S803to Step S805are performed for all the edited portions.

After the translation and output module108creates the translation at Step S805or Step S806, it ends the translation process2.

Next, a specific example of the speech translation process will be described that is executed in accordance with the procedures described above.FIG. 9shows an illustration depicting exemplary items of data processed in the speech translation process.

Suppose the example storage module111stores the examples shown inFIG. 2and the edit storage module112does not store edits.

Further, suppose that a user specifies use of transportation as the use condition, and the use condition detecting module101detects this (Step S401). In this case, for the example having “TRANSPORT” of the related situation in the situation column, the example translations for example ID's q01, q01, q02, q03and q04are acquired from the example storage module111(Step S502). Since the edit storage module111is empty, edits are not acquired (Step S503).

Since there are no edits, the edited example is not added in the edited example creation process (Step S508), and the four acquired examples are added to the example register as they are (Step S511).

Consequently, four examples shown in (a1) to (a4) inFIG. 9are displayed on the display of the input/output device114(Step S403).

Now suppose that a user makes a speech input to partially edit (a1) to create the sentence “I'll go to Los Angeles (in Japanese).”

In this case, the speech input accepting module106determines that a speech input is made (Step S405: YES), and the analyzing module107extracts edits (Step S408).

(b1) shown inFIG. 9shows edits output by the analyzing module107. In this example, the source example ID q01corresponds to (a1). For the edits, the subject expression is the words “New York (in Japanese)”, and the edited result expression is the words “Los Angeles (in Japanese)”.

The edits are output to the edit storage module112(Step S409). In the example above, the same edits for the edit ID r01inFIG. 3are output. (c1) shown inFIG. 9shows the edits output to the edit storage module112. In addition, since the words “New York (in Japanese)” in the edited portion is identified with the semantic class “#CITY” for example ID=q01, “#CITY” is set in the semantic class column of edits for edit ID=r01.

In the translation process2(Step S410), edits shown in (b1) inFIG. 9are acquired (Step S801). In addition, since an edit exists (Step S502: YES), the example translation for example ID=p02is acquired from the example storage module111(Step S803). This example translation is acquired because it has the words “Los Angeles (in Japanese)” from the edited result expression in the Japanese expression column.

Furthermore, the words “Los Angeles,” i.e., the English expression for example ID=p02, are acquired from the example storage module111(Step S804). The words “I'll go to New York” are the English expression for example ID=q01, and example ID=q01the source example. Thus, “New York,” which is the portion of the English expression corresponding to the subject expression “New York (in Japanese)” is replaced by the words “Los Angeles (in English).” In this manner, “I'll go to Los Angeles (in English)” is created as the translation (Step S805).

Consequently, the translation shown in (d1) inFIG. 9is displayed on input/output device114(Step S411). Instead of displaying the translation, the translation may be synthesized as speech, and output to the speaker.

As described above, the user provides a speech input so that the presentation example is partially edited. The edits are based on the user's intentions, to expressions chosen using the use condition, and in this manner the user can obtain a translation.

After that the translation is obtained, the process continues, and the presentation example creation process is performed again (Step S402). Here, since the edit storage module112stores edits, edits shown in (c1) inFIG. 9are acquired (Step S503). Since the use condition has not changed, the example translations for example ID's q01, q02, q03, and q04are acquired from the example storage module111in the manner discussed above (Step S502).

In this case, since edits exist, the edited example creation process is executed (Step S508). For example ID=q01, the words “New York (in Japanese)” appear as the subject expression (Step S601: YES), this portion is replaced by the edited result expression, i.e., the words “Los Angeles (in Japanese)” (Step S602). In (e1) shown inFIG. 9, the result is shown.

For example ID=q02, the subject expression “New York (in Japanese)” does not appear (Step S601: NO). On the other hand, the sentence “What time does it arrive at [Chicago#CITY] (in Japanese)” includes the same semantic class as the semantic class “#CITY” of the edit. Therefore, the word “[Chicago#CITY] (in Japanese)” is replaced by the words “Los Angeles (in Japanese)” (Step S603: YES, Step S604). In (e2) shown inFIG. 9, the result is shown.

After this, similar processes continue. However, since example ID=q03and example ID=q04do not have expressions matching the subject expression or expressions matching the semantic class, the examples are added to the example register as they are (Step S511).

Consequently, four examples as shown in (e1) to (e4) inFIG. 9are displayed on the input/output device114(Step S403).

As described above, an example translation can be represented that has the same edits as previously applied edits, and thus the necessity for a user to edit an example translation can be reduced. In addition, the speech input and the speech recognition process for editing are eliminated, and thus user effort can be reduced.

Suppose that the example shown in (e2) inFIG. 9is matched with an edit input by a user next time. In this case, it is unnecessary for the user to make a speech input, and the user can select the example through the touch panel or by pressing a button.

In this case, the example selection accepting module105determines that the user directly selects the example (Step S404: YES), and the translation process1is executed (Step S407).

In the translation process1, example ID=q02is acquired as the source example for editing of (e2) inFIG. 9(Step S701). In addition, since an edit is made (Step S702: YES), the subject expression “Chicago (in Japanese)” and the edited result expression “Los Angeles (in Japanese)” are acquired (Step S703).

Subsequently, example ID=p02is acquired from the example storage module111, which has the words “Los Angeles (in Japanese),” i.e., the edited result expression, in the Japanese expression column (Step S704).

The words “Los Angeles” are acquired from the example storage module111, as the English expression of the example ID=p02(Step S705). Because the English expression “What time does it arrive at Chicago#CITY?” is the source example corresponding to example ID=q02, the word “Chicago (in English)” is replaced by the words “Los Angeles (in English)”, and “What time does it arrive at Los Angeles? (in English)” is created as the translation (Step S706).

Consequently, the translation shown in (f1) inFIG. 9is displayed on the input/output device114(Step S411).

Thus, an example translation that has not been used before can be edited to be an expression very likely to be selected by the user. This translation can be prepared in advance for presentation according to the use condition. This solves the problem of example-based interfaces that examples suitable for particular users cannot be prepared in advance. In addition, since the user is more likely to directly select the presented example, the number speech inputs can be reduced. This reduces the number of limitations on acceptable environments for use of the apparatus. More specifically, the apparatus can function effectively in environments which have levels of noise which are not conducive for speech input.

In the example described above, the description of speech entry is based on the premise that the entry is selected from the presented examples. Alternately, speech can be input and a speech entry can be generated using general speech recognition. This method can be combined with the example-based translation described above, as well as with rule-based translation. Further, characters may be handwritten and input for character recognition instead of inputting speech.

As described above, in the speech translation apparatus100according to the first embodiment, when a user edits example translations, the edits are stored. When the example translation is subsequently presented to the user, the stored edits are used for editing the example translation beforehand, and the edited example translation can be presented for the user. Therefore, example translations with edits suitable to individual users can be presented to each user, the users have to make fewer edits decreased, and the effort for each user can be reduced.

Second Embodiment

A speech translation apparatus according to a second embodiment is an apparatus that references a storage module which stores a semantic relationship between two expressions, in addition to the edits stored in the storage module, edits the example translation, and then presents it to a user.

FIG. 10shows a block diagram depicting a speech translation apparatus1000according to the second embodiment. As shown in the drawing, the speech translation apparatus1000has an example storage module111, an edit storage module112, a location information acquiring module113, an input/output device114, a semantic relationship storage module1015, a use condition detecting module101, a presentation example determining module102, an edit adapting module1003, a display control module104, an example selection accepting module105, a speech input accepting module106, an analyzing module107, and a translation and output module108.

The second embodiment is different from the first embodiment in that the semantic relationship storage module1015is added and the functionality of the edit adapting module1003is modified. Since the other configurations and functions are the same as those inFIG. 1which is a block diagram depicting the configuration of the speech translation apparatus100according to the first embodiment, like elements are designated with the same numerals and signs, and their description is omitted here.

The semantic relationship storage module1015stores the semantic relationship between expressions stored in the example storage module111, which can be comprised of typical storage modules such as a HDD, an optical disc, a memory card, or a RAM.

FIG. 11shows an illustration depicting an exemplary data structure of the semantic relationship stored in the semantic relationship storage module1015. As shown in the drawing, the semantic relationship storage module1015stores a knowledge ID which uniquely identifies the semantic relationship and associates it with a conceptual expression1, a conceptual expression2, and a relationship with between the conceptual expressions1and2.

In the conceptual expression1and conceptual expression2columns, expressions that identify concepts having a particular semantic relationship are stored. In addition, in the relationship column, the semantic relationship between conceptual expression1and conceptual expression2is stored.

Here, the symbol “%” identifies the title of the semantic relationship. When meaning expressed by each of the semantic relationships is expressed in a sentence in Japanese word order such as “<conceptual expression1> no <relationship> ha <conceptual expression2> (in Japanese)”, this corresponds to (“<relationship> of <conceptual expression1> is <conceptual expression2>” in English). For instance, the semantic relationship of knowledge ID=s04has the meaning “<Los Angeles> no <main sightseeing spot> ha <Chinese Theater> (in Japanese)” corresponding to (“<main sightseeing spot> of <Los Angeles> is <Chinese Theater>” in English). Thus, the semantic relationship of knowledge ID=s06has the meaning “<Los Angeles> no <main sightseeing spot> ha <Hollywood> (in Japanese)” corresponding to(“<main sightseeing spot> of <Los Angeles> is <Hollywood>” in English).

In addition, the data structure of the semantic relationship stored in the semantic relationship storage module1015is not limited to the structure shown inFIG. 11. The semantic relationships may be stored as a concept thesaurus. Further, in the semantic relationships may be stored in predicate form such as “time (eat (John, apple), today)”, or the semantic relationships may be described in an intermediate language.

Similar to the edit adapting module103according to the first embodiment, the edit adapting module1003references edits stored in the edit storage module112, and edits the presentation example determined by the presentation example determining module102. There is a third editing method described below, in addition to two methods described according to the first embodiment.

The third method works by replacing an expression F1of an example translation X with an expression F2. This replacement is performed when the edit storage module112has an edit L that modifies an expression E1to another expression E2, and the semantic relationship storage module1015has a stored semantic relationship R between expressions E1and F1and the same stored semantic relationship R exists between expressions E2and F2. This occurs when the example translation X, determined to be a presentation example, includes the expression F1.

A speech translation process performed by the speech translation apparatus1000according to the second embodiment will be described next. The second embodiment is different from the first embodiment in that an edit is made by applying the third method in an edited example creation process. The descriptions of the other processes are the same as those of the first embodiment, and the explanation is omitted.

FIG. 12shows a flow chart depicting the overall flow of the edited example creation process according to the second embodiment.

The editing processes from Step S1201to Step S1204for the first method and the second method are the same as the processes from Step S601to Step S604in the speech translation apparatus100according to the first embodiment, and the explanation is omitted.

If the semantic class C of edit Tk does not appear in an example translation Rj at Step S1203(Step S1203: NO), the edit adapting module1003acquires the semantic relationship including a subject expression S of edit Tk as conceptual expression1(hereinafter, referred to as a semantic relationship Rs) from the semantic relationship storage module1015(Step S1205).

Subsequently, the edit adapting module1003acquires the semantic relationship that has an edited result expression T for edit Tk as conceptual expression1(hereinafter, a semantic relationship Rt) from the semantic relationship storage module1015(Step S1206).

Subsequently, the edit adapting module1003determines whether the relationship column for the acquired semantic relationship Rs matches the relationship column for the semantic relationship Rt (Step S1207). When there is no match, (Step S1207: NO), the edited example creation process is ended.

When the relationship column for the acquired semantic relationship Rs matches the relationship column for the semantic relationship Rt (Step S1207: YES), the edit adapting module1003determines whether the conceptual expression2for the semantic relationship Rs appears in the example Rj (Step S1208).

When the conceptual expression2for the semantic relationship Rs appears in the example Rj (Step S1208: YES), the edit adapting module1003adds to the example register an edited example translation Rj′″ by replacing the conceptual expression2in the example Rj with the conceptual expression2for the semantic relationship Rt (Step S1209).

When the conceptual expression2for the semantic relationship Rs does not appear in the example Rj (Step S1208: NO), the edited example creation process is ended.

Next, a specific example of the speech translation process executed in accordance with the procedures described above will be described.FIG. 13is an illustration depicting exemplary data items processed in the speech translation process.

Here, the premise is that the translation shown in (f1) inFIG. 13is output by procedures similar to those described inFIG. 9in the first embodiment. In addition, the semantic relationship storage module1015is considered to store the semantic relationships as shown inFIG. 11.

Suppose a user changes the use condition to “sightseeing” (Step S406). Then, to present an example corresponding to the changed use condition, the presentation example creation process is performed (Step S402). In this case, for the example having the related situation “SIGHTSEEING” in the situation column, example ID's q06and q07are acquired from the example storage module111(Step S502).

In (g1) and (g2) inFIG. 13, the acquired examples are shown. In addition, the edit for edit ID=r01is acquired from the edit storage module112(Step S503).

For example ID=q06, the words “New York (in Japanese)” are the subject expression of the acquired edit, but do not appear in the example translation (Step S1201: NO), and the semantic class of the edit, “#CITY”, does not appear in the example translation (Step S1203: NO). Thus an editing process is performed by referring to the semantic relationship storage module1015.

The words “New York (in Japanese)” are the subject expression. The semantic relationship with the subject expression in the conceptual expression1column is acquired from the semantic relationship storage module1015, in this case the semantic relationship corresponding to knowledge ID =s01(Step S1205). The words “Los Angeles (in Japanese)” are the edited result expression. The semantic relationships with the edited result expression in the conceptual expression1column are subsequently acquired from the semantic relationship storage module1015, in this case the semantic relationships with knowledge ID's=s04, s05, and s06.

The relationship column for the semantic relationship with knowledge ID=s01matches the relationship column for each knowledge ID=s04, s05, and s06(Step S1207: YES). Next it is determined whether the conceptual expression2for the knowledge ID=s01, i.e. the words “Empire State Building (in Japanese),” appear in example ID=q06(Step S1208).

Example ID=q06includes the sentence “Where is the Empire State Building? (in Japanese)”, which in turn includes the words “Empire State Building (in Japanese),” i.e., the conceptual expression2for knowledge ID=s01(Step S1208: YES). Thus, three example expressions are added to the example register by replacing the conceptual expression for knowledge ID=s01with the conceptual expression2for each of knowledge ID's =s04, s05, and s06, respectively (Step S1209).

In contrast, example ID=q07is not edited, and the source example is output as is. Consequently, the examples shown in (h1) to (h4) inFIG. 13are added in the example register.

When the same example translation is edited to present a plurality of edited results, a single presentation example may be presented so that the user may select only the edited portion from a plurality of candidates as shown in (i1) inFIG. 13.

Thus, even when a user selects the use condition “sightseeing” for the first time, presentation examples that are highly likely to be selected by the user can be created and presented to the user. Thus, user efforts to operate the apparatus can be reduced.

In addition, the example expressions shown in (h1) to (h3) inFIG. 13can be stored beforehand in the example storage module111, and suitably edited and presented to the user. Even for an expression with a semantic class that a user has never used before, in this case “#POI,” the example expression can be suitably edited and presented to the user.

As described above, in the speech translation apparatus according to the second embodiment, the semantic relationship between two expressions can be used to edit an example expression, and the example expression can be presented to the user. Therefore, it is unnecessary to exhaustively prepare example translations or expressions for various user attributes such as gender, age, occupation, and travel experiences. Instead, the example expressions and translations can be edited by matching them with the attributes of the user to whom they will be presented. Therefore, the number of edits to the presentation examples are reduced, with a corresponding reduction in user effort.

The input program operating in the input apparatus according to the first and second embodiments can be installed in a ROM (Read Only Memory).

The input program operating in the input apparatus according to the first and second embodiments may be configured to be recorded and offered in a file in a format suitable for a computer readable recording medium such as a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk Recordable), or a DVD (Digital versatile Disk).

In addition, the input program operating in the input apparatus according to the first and second embodiments may be configured by storing the program in a computer connected to a network such as the Internet, and downloaded through the network. Further, the input program operating in the input apparatus according to the first and second embodiments may be offered or distributed through a network such as the Internet.

The input program operating in the input apparatus according to the first and second embodiments can be configured with the modules described above (the use condition detecting module, the presentation example determining module, the edit adapting module, the display control module, the example selection accepting module, the speech input accepting module, the analyzing module, and the translation and output module). A CPU (Central Processing Unit) can read the input program out of the ROM and run the program to load one or more of the modules a main storage, and the individual modules can thus be created on the main storage.

As described above, the input apparatus, the input method and input program according to embodiments of the invention are suitable for the input apparatus which presents examples and specifies a suited entry description from the presented examples.