Patent Description:
A speaker may use a voice control function to control an electronic apparatus. The voice control function refers to a function of controlling the electronic apparatus to perform a specific operation based on a recognition result when a speaker plans on the specific operation of the electronic apparatus and utters a voice corresponding to the specific operation.

However, when there is an error in the recognition result, the electronic apparatus may perform an operation mismatched with utterance intention. Therefore, the electronic apparatus needs to perform the specific operation corresponding to the utterance intention even though an error is present in the recognition result.

<CIT>, cited under Article <NUM>(<NUM>) EPC, discloses method including: receiving a first speech, and extracting a first text from the received first speech, in response to detecting that extraction of the first text includes errors such that a request associated with the first speech is unprocessable, storing the extracted first text, receiving a second speech and extracting a second text from the received second speech, in response to detecting that the request is processable using the extracted second text, detecting whether a similarity between the first and second texts is greater than a similarity threshold, and whether the second speech is received within a predetermined time duration of receiving the first speech, and when the similarity is greater than the threshold, and the first and second speech signals are received within the time duration, storing the first text in association with the second text.

Accordingly, an aspect of the disclosure is to provide an electronic apparatus, a method of controlling the electronic apparatus, a voice recognition system comprising an electronic apparatus and a recording medium stored with a computer program, in which a specific operation corresponding to utterance intention is performed even though there is an error in a recognition result.

According to an aspect, there is provided an electronic apparatus as set out in claim <NUM>. Optional features are set out in claims <NUM> to <NUM>.

According to another aspect, there is provided a method as set out in claim <NUM>.

Optional features are set out in claims <NUM> to <NUM>.

According to another aspect, there is provided a voice recognition system as set out in claim <NUM>.

Optional features are set out in claim <NUM>.

According to another aspect, there is provided a recording medium as set out in claim <NUM>.

According to the disclosure, there are provided an electronic apparatus, a method of controlling the electronic apparatus, a voice recognition system comprising an electronic apparatus and a recording medium stored with a computer program, in which a specific operation corresponding to utterance intention is performed even though there is an error in a recognition result.

Below, embodiments of the disclosure will be described in detail with reference to accompanying drawings. In the description of the following embodiments, elements illustrated in the accompanying drawings will be referenced, and like numerals or symbols set forth in the drawings refer to like elements having substantially the same function. In the disclosure, at least one among a plurality of elements refers to not only all the plurality of elements but also both each one of the plurality of elements excluding the other elements and a combination thereof.

<FIG> is a block diagram of an electronic apparatus according to an embodiment of the disclosure. As shown in <FIG>, an electronic apparatus <NUM> may include a signal receiver <NUM>, an image processor <NUM>, a voice receiver <NUM>, and a processor <NUM>. As necessary, the electronic apparatus <NUM> may further include at least one of a display <NUM>, a storage <NUM> and a communicator <NUM>. The electronic apparatus <NUM> may be embodied by an apparatus without limitations as long as it can perform a specific operation corresponding to a voice of a speaker. For example, the electronic apparatus <NUM> may be embodied by a set-top box, a smart TV, a smartphone, a smart watch, a wearable device, a tablet computer, a personal computer (PC), etc..

The signal receiver <NUM> receives an image signal through at least one channel. The signal receiver <NUM> may be variously embodied according to the formats of the image signal to be received and the types of electronic apparatus <NUM>. For example, the signal receiver <NUM> may be embodied by a tuner that receives a radio frequency (RF) broadcast signal from a broadcasting station.

The image processor <NUM> processes a received image signal by a preset image processing process. For example, the image processor <NUM> may perform decoding, de-interlacing, scaling, noise reduction, detail enhancement, and the like image processing processes, but the processes of the image processor <NUM> are not limited to these examples. The image processor <NUM> may be embodied by a system-on-chip into which various functions are integrated, or an image processing board to which individual elements capable of performing the processes independently of each other are mounted.

The voice receiver <NUM> receives a voice uttered by a speaker. The voice receiver <NUM> receives a voice of a speaker as a kind of voice signal, and the voice signal of the speaker may be extracted by a frequency analysis or the like preprocessing process because the received voice signal may include various noises besides the voice of the speaker targeted for voice recognition. The voice receiver <NUM> may be embodied by a microphone.

The processor <NUM> may control general elements of the electronic apparatus <NUM> so that the electronic apparatus <NUM> can perform predetermined operations. The processor <NUM> may apply a voice recognition process to a voice received in the voice receiver <NUM> while the electronic apparatus <NUM> performs a predetermined operation. The processor <NUM> may obtain a recognition result through the voice recognition process, and control the electronic apparatus <NUM> to perform another operation based on the obtained recognition result. The processor <NUM> may be embodied by a central processing unit (CPU) or a micro-processor.

The display <NUM> displays an image based on an image signal output from the image processor <NUM>. There are no limits to the types of display <NUM>, and the display <NUM> may be embodied by various types such as a plasma display panel (PD), a liquid crystal display (LCD), a light emitting diode (LED), an organic light emitting diode (OLED), a flexible display, etc..

The storage <NUM> refers to a nonvolatile memory such as a flash memory, a hard disk, etc., which is configured to store an image signal, a voice signal, a program, an application and the like related to the operations of the electronic apparatus <NUM>. The communicator <NUM> may connect and communicate with an external apparatus <NUM> or an external voice recognition server <NUM>. The communicator <NUM> may use various communication standards, such as Wi-Fi, Bluetooth, ZigBee, Wi-Fi Direct (WFD), ultra-wideband (UWB), infrared Data Association (IrDA), Bluetooth low energy (BLE), near field communication (NFC), etc..

<FIG> is a flowchart showing a method of controlling the electronic apparatus of <FIG>. As shown in <FIG>, the electronic apparatus <NUM> performs a predetermined operation under control of the processor <NUM> (S210). The electronic apparatus <NUM> obtains a recognition result by applying a voice recognition process to a voice received in the voice receiver <NUM> during the predetermined operation (S220).

Further, when the obtained recognition result matches at least one pseudo recognition result previously defined corresponding to the operation of the electronic apparatus <NUM>, the electronic apparatus <NUM> may perform control based on the standard recognition result previously defined corresponding to the pseudo recognition result (S230). Thus, the electronic apparatus <NUM> can perform a specific operation based on the utterance intention even though there is an error in the recognition result.

<FIG> is a block diagram of a processor in the electronic apparatus of <FIG>. As shown in <FIG>, the electronic apparatus <NUM> of <FIG> includes the processor <NUM>, and the processor <NUM> may include a voice recognizer <NUM>, a voice corrector <NUM>, and a function implementor <NUM>. Because there are no limits to the elements of the processor <NUM>, at least one of these elements may be excluded or another element may be added to the processor <NUM>.

The voice recognizer <NUM> may include an acoustic model, a language model, a dictionary, etc. The voice recognizer <NUM> receives a voice signal from the voice receiver <NUM>, and uses an acoustic model <NUM>, and a language model & dictionary <NUM> to apply a voice recognition process to the received voice signal, thereby outputting a recognition result.

The acoustic model <NUM> may include information for detecting candidate pronunciation from characteristic information of the voice signal. The candidate pronunciation may be detected in units of ward, but not limited thereto. Alternatively, the candidate pronunciation may be detected in various units of phoneme, phone, etc. The acoustic model <NUM> may be generated based on a lot of voice signals by a statistical method, based on voice signals of many random people, or based on voice signals collected from a specific speaker. Therefore, the acoustic model <NUM> may be individually applied according to speakers during the voice recognition process.

The language model & dictionary <NUM> may be configured to store a dictionary about pronunciation and a language model, i.e. a grammatical model used by the voice recognizer <NUM> during the voice recognition process. The speakers may be different in use language, dialect, grammatical structure, word, etc. according to nationalities or residential areas, ages, linguistic habits, etc. Therefore, the language model & dictionary <NUM> may include one or more language models and one or more dictionaries.

However, due to diversity, variability, nonnormality, irregularity, etc. of an uttered voice, there may be misrecognition or the like error in the recognition result even though the voice recognizer <NUM> broadly employs the acoustic model <NUM> and the language model & dictionary <NUM> to apply the voice recognition process to the received voice signal.

The voice corrector <NUM> obtains a recognition result from the voice recognizer <NUM>, and corrects an error of the obtained recognition result. The voice corrector <NUM> may employ different methods of correcting the error of the recognition result according to whether operation information is obtained or not. The operation information refers to information about operations performed by the electronic apparatus <NUM>, and may include a menu, a program, an application, etc. executed in the electronic apparatus <NUM>. In other words, when the electronic apparatus <NUM> performs a predetermined operation and it is thus possible to obtain information about the operation, the voice corrector <NUM> may correct an error in the recognition result to correspond to the operation of the electronic apparatus <NUM>.

In more detail, when it is impossible for the voice corrector <NUM> to obtain information about a predetermined operation of the electronic apparatus <NUM>, the voice corrector <NUM> may use a universal correction model to correct an error in the recognition result. The universal correction model <NUM> is based on weighted finite state transducers (WFST), which may for example include a grammar WFST that expresses a sentence based on a relationship between words, a dictionary WFST that expresses words based on context independent phones, and a context WFST that converts the context independent phones into context dependent phones.

When it is possible for the voice corrector <NUM> to obtain information about a predetermined operation of the electronic apparatus <NUM>, the voice corrector <NUM> may use a situation correction model <NUM> to correct an error in the recognition result. The situation correction model refers to a unique correction model associated with a predetermined operation of the electronic apparatus <NUM>, and is previously defined adaptively for the operation. The previous definition adaptive for the operation means that the situation correction model previously defined adaptively for a predetermined operation of the electronic apparatus <NUM> is upgradable by learning a recognition result of an uttered voice, a score of the recognition result, etc. The upgrading of the situation correction model may include that the situation correction model is modified by a speaker.

The function implementor <NUM> may control the electronic apparatus <NUM> to implement a specific operation based on a recognition result corrected by the voice corrector <NUM>. The function implementor <NUM> may control general elements of the electronic apparatus <NUM> if needed for implementing the specific operation.

<FIG> illustrates a voice control function in the electronic apparatus of <FIG>. The reference numeral of '<NUM>' in <FIG> indicates that the electronic apparatus <NUM> of <FIG> is tuned to a "stock news channel" and displays a program of "stock news about AA company" on the display <NUM>. The program of "stock news about AA company" may be displayed on the display <NUM> in response to selection of a menu related to the program or execution of the related application. However, the program of "stock news about AA company" is merely an example, and the electronic apparatus <NUM> may display various programs of various channels without limitations.

When a voice of "stock price information" uttered by a speaker is received during an operation related to the program of "stock news about AA company", the electronic apparatus <NUM> may obtain a recognition result of "stock price information" and display a program of "stock price information about AA company" based on the recognition result. However, the speaker may not clearly and correctly utter "stock price information" because the use language, the dialect, the grammatical structure, the word, etc. may be varied depending on the nationalities or residential areas, the ages, the linguistic habits, etc. of the speaker. For example, the speaker plans on the program of "stock price information about AA company" and utters "stock price information", but the electronic apparatus <NUM> may misrecognize "stop price information".

Below, it will be described that the electronic apparatus <NUM> according to an embodiment displays the program of "stock price information about AA company" based on the utterance intention even though a speaker's utterance intention of "stock price information" is misrecognized as "stop price information".

The electronic apparatus <NUM> may previously define unique situation correction models respectively associated with predetermined operations. Because there are no limits to the operations of the electronic apparatus <NUM>, the electronic apparatus <NUM> may define a plurality of situation correction models respectively corresponding to the operations. The electronic apparatus <NUM> may store the situation correction models in the storage <NUM>.

For example, as shown in <FIG>, the electronic apparatus <NUM> may previously define the situation correction model for a context of "stock service". The situation correction model for "stock service" may define "stop price information", "operation information", etc. as pseudo recognition results, and define "stock price information" as a standard recognition result corresponding to the pseudo recognition result.

Because the electronic apparatus <NUM> displays the program of "stock news about AA company" on the display <NUM>, the situation correction model for the "stock service" may be used in correcting the recognition result from the voice recognition process performed during the operation related to the "stock". The operations related to "stock" may include execution of a menu, a program, and an application related to the "stock", reproduction of an image or a sound related to the "stock", execution of a "home trading system", etc..

When the electronic apparatus <NUM> obtains a recognition result of "stop price information" through the voice recognition process while displaying the program of "stock news about AA company", the electronic apparatus <NUM> may correct the recognition result of "stop price information" with reference to the situation correction model of the "stock service" among the plurality of situation correction models because the current operation is related to "stock news".

The electronic apparatus <NUM> may identify whether the recognition result of "stop price information" matches the pseudo recognition result of the situation correction model of the "stock service". Because "stop price information" is previously defined in the pseudo recognition result, the electronic apparatus <NUM> may identify that the recognition result of "stop price information" matches the pseudo recognition result of "stop price information". The electronic apparatus <NUM> may correct the recognition result of "stop price information" into the standard recognition result of "stock price information" corresponding to the pseudo recognition result of "stop price information".

Further, as indicated by the reference numeral of '<NUM> in <FIG>, the electronic apparatus <NUM> may display the program of "stock price information about AA company" based on "stock price information" corrected matching the utterance intention. In other words, the electronic apparatus <NUM> may perform a specific operation matching the utterance intention even though there is an error in the recognition result.

When the electronic apparatus <NUM> implements a voice control function based on the obtained recognition result of "stop price information" without reference to the situation correction model of the "stock service" about the current operation, the electronic apparatus <NUM> may display "stop price news about AA company" mismatched with the utterance intention as indicated by the reference numeral of '<NUM>' in <FIG>.

<FIG> is a flowchart showing a method of controlling the electronic apparatus of <FIG>. As shown in <FIG>, when a speaker utters a voice (S501), the electronic apparatus <NUM> may apply the voice recognition process to the uttered voice to obtain a recognition result (S502). The electronic apparatus <NUM> identifies whether it is possible to obtain the operation information (S503). The operation information refers to operation about a predetermined operation performed by the electronic apparatus <NUM>. When it is possible to obtain the operation information, the electronic apparatus <NUM> may identify a situation model corresponding to a situation model corresponding to operation information among a plurality of situation models (S504). The electronic apparatus <NUM> may be storing situation correction models corresponding to predetermined operations or receive the situation correction models from the outside. The electronic apparatus <NUM> may identify whether the recognition result matches the pseudo recognition result of the identified situation model (S505). When the recognition result is matched with the pseudo recognition result of the identified situation model, the electronic apparatus <NUM> may correct the recognition result with the standard recognition result corresponding to the pseudo recognition result (S507). However, when the recognition result is mismatched with the pseudo recognition result of the identified situation model, the electronic apparatus <NUM> may identify whether the recognition result matches the standard recognition result of the identified situation model (S506). When the recognition result matches the standard recognition result of the identified situation model, the electronic apparatus <NUM> may correct the recognition result with the standard recognition result (S508). When the recognition result is corrected with the standard recognition result (S507, S508), the electronic apparatus <NUM> performs control based on the corrected standard recognition result (S510).

On the other hand, when it is impossible to obtain the operation information, the electronic apparatus <NUM> may correct the recognition result based on the universal correction model (S509). The universal correction model may be based on the WFST. When the universal correction model is used, the electronic apparatus <NUM> may perform control based on the recognition result of which a sentence pattern error is corrected (S510).

<FIG> is a block diagram of a system including the electronic apparatus of <FIG>. As shown in <FIG>, the system may include the electronic apparatus <NUM> of <FIG>, and a voice recognition server <NUM>. The voice recognition server <NUM> may include a processor <NUM>, a storage <NUM>, and a communicator <NUM> for communication with the electronic apparatus <NUM>.

The voice recognition server <NUM> may perform some functions of the electronic apparatus <NUM> of <FIG>. The voice recognition server <NUM> may previously define unique situation correction models respectively associated with predetermined operations to be performed in the electronic apparatus <NUM>, under control of the processor <NUM>. Because there are no limits to the operations of the electronic apparatus <NUM>, the voice recognition server <NUM> may define a plurality of situation correction models respectively corresponding to the operations to be performed by the electronic apparatus <NUM>. The electronic apparatus <NUM> may store the situation correction models in the storage <NUM>. The voice recognition server <NUM> may store the plurality of situation correction models in the storage <NUM>.

For example, when the voice recognition server <NUM> defines the situation correction model about the "stock service", the voice recognition server <NUM> may define "stop price information", "operation information", etc. as the pseudo recognition results, and define "stock price information" as the standard recognition result corresponding to the pseudo recognition results. With regard to the voice received while the electronic apparatus <NUM> performs the operation related to "stock", the voice recognition server <NUM> may correct the recognition result subjected to the voice recognition process. To this end, the voice recognition server <NUM> may use the situation correction model of "stock service" corresponding to the operation related to "stock" performed by the electronic apparatus <NUM>.

As indicated by the reference numeral of '<NUM>' in <FIG>, it will be assumed that the electronic apparatus <NUM> displays the program of "stock news about AA company". The voice recognition server <NUM> may receive a voice signal through the electronic apparatus <NUM>. The voice recognition server <NUM> may apply the voice recognition process to the voice signal and obtain a recognition result of "stop price information". The voice recognition server <NUM> may obtain operation information that the current operation of the electronic apparatus <NUM> is related to "stock news", and thus correct the error of the recognition result of "stop price information" with reference to the situation correction model of the "stock service" among the plurality of situation correction models.

In other words, the voice recognition server <NUM> may identify whether the recognition result of "stop price information" matches the pseudo recognition result of the situation correction model of the "stock service". Because "stop price information" is previously defined in the pseudo recognition result, the voice recognition server <NUM> may identify that the recognition result of "stop price information" matches the pseudo recognition result of "stop price information. " The voice recognition server <NUM> may correct the recognition result of "stop price information" into the standard recognition result of "stock price information" corresponding to the pseudo recognition result of "stop price information".

Further, the voice recognition server <NUM> may transmit data about the standard recognition result of "stock price information" to the electronic apparatus <NUM>. Then, as indicated by the reference numeral of '<NUM>' in <FIG>, the electronic apparatus <NUM> may display the program of "stock price information about AA company" based on the standard recognition result of "stock price information". In other words, the system may perform a specific operation matching the utterance intention even though there is an error in the recognition result.

When the voice recognition server <NUM> cannot obtain the operation information about the current operation of the electronic apparatus <NUM>, the voice recognition server <NUM> may correct the recognition result based on the universal correction model. The correction of the recognition result based on the universal correction model is equivalent to that described with reference to <FIG>, and thus descriptions thereof will be omitted.

Meanwhile, the voice control function may be variously shared between the electronic apparatus <NUM> and the voice recognition server <NUM> according to design methods. For example, the electronic apparatus <NUM> autonomously applies the voice recognition process to the uttered voice, and provide the recognition result obtained by the voice recognition process to the voice recognition server <NUM>.

Alternatively, the electronic apparatus <NUM> may transmit a voice signal to a speech-to-text (STT) sever, and receive a text corresponding to the voice signal from the STT server. The electronic apparatus <NUM> may performs the voice recognition process based on the received text, thereby obtaining the recognition result. As necessary, the STT server may transmit the text corresponding to the voice signal to not the electronic apparatus <NUM> but the voice recognition server <NUM>, and the voice recognition server <NUM> may perform the voice recognition process to obtain the recognition result.

<FIG> illustrates that the electronic apparatus of <FIG> receives a voice through an external apparatus. The electronic apparatus <NUM> may receive an uttered voice through the voice receiver <NUM>, but may receive a digital signal about the voice from an external apparatus <NUM> as indicated by the reference numeral of '<NUM>' in <FIG>. In other words, when the external apparatus <NUM> receives the uttered voice, converts the uttered voice into the digital signal about the voice and transmits the digital signal to the electronic apparatus <NUM>, the electronic apparatus <NUM> may perform the voice recognition process based on the digital signal about the voice. Operations of the electronic apparatus <NUM> subsequent to the voice recognition process are equivalent to those described with reference to <FIG>, and thus descriptions thereof will be omitted.

The external apparatus <NUM> may be embodied by a remote controller, a smartphone, or the like portable mobile apparatus. When the external apparatus <NUM> is embodied by the potable mobile apparatus, the external apparatus <NUM> may install an application therein for a function of controlling the electronic apparatus <NUM> like the remote controller, or a function of performing the voice recognition process like the electronic apparatus <NUM> of <FIG>. In particular, when the external apparatus <NUM> executes the application for the voice recognition process to apply the voice recognition process to the uttered voice, the electronic apparatus <NUM> may receive the recognition result of the voice recognition process from the external apparatus <NUM>. Operations of the electronic apparatus <NUM> subsequent to the reception of the recognition result are equivalent to those described with reference to <FIG>, and thus descriptions thereof will be omitted.

The communicator <NUM> of the electronic apparatus <NUM> may use various communication standards, for example, Wi-Fi, Bluetooth, IrDA, etc. to communicate with the external apparatus <NUM>. To communicate with the external apparatus <NUM>, the electronic apparatus <NUM> may use the communicator <NUM> for communication with the voice recognition server <NUM>, but may use a separate communicator.

Claim 1:
An electronic apparatus comprising:
a voice receiver configured to receive a voice of a speaker; and
a processor configured to:
perform a predetermined operation,
obtain a recognition result by applying a voice recognition process to the voice received in the voice receiver during the predetermined operation,
select at least one correction model corresponding to the performed operation among a plurality of correction models respectively provided according to a plurality of operations,
identify whether the obtained recognition result matches at least one previously defined pseudo recognition result of the at least one selected correction model, and
perform control based on a standard recognition result previously defined corresponding to the at least one pseudo recognition result, based on the obtained recognition result matching the pseudo recognition result previously defined corresponding to the predetermined operation.