Patent Description:
To compute and process information in accordance with certain processes, an electronic apparatus typically includes a central processing unit (CPU), a chipset, a memory, and like electronic components for computation. Such an electronic apparatus may be classified variously in accordance with what information will be processed therein. For example, the electronic apparatus is classified into an information processing apparatus such as a personal computer, a server, or the like for processing general information, and an image processing apparatus for processing image information. The image processing apparatus, the display apparatus, the information processing apparatus, or like electronic apparatus performs functions that a single entity is specifically designed to perform. Further, a plurality of electronic apparatuses are connected to one another to form a system of a single unit network, so that the electronic apparatuses can collaborate with one another under this system, thereby performing various extended functions that a single entity (e.g., special-purpose device) cannot perform. As an example of this concept, there are Internet of Things (IoT) devices.

Among many user interface (UI) environments for operating the electronic apparatus, a voice recognition method has recently been in the limelight. In particular, the voice recognition method has been adopted more actively in the operation of systems that include multiple electronic apparatus components. Although there is little difference in details according to how the system or apparatus is materialized, the voice recognition method generally undergoes processes of converting a voice signal input through a microphone into a text, determining a command corresponding to the converted text, and executing an operation directed by the determined command in the electronic apparatus. However, it is possible for an operation unintended by a user to be performed in such an environment of carrying out a voice command.

For example, a voice recognition apparatus for voice recognition may receive a signal from a malicious user, such as a hacker, instead of a signal caused by a user speech. Since the voice recognition apparatus cannot distinguish between a signal from a user and a signal from the hacker, the signal from the hacker may be subjected to the voice recognition and thus a corresponding operation may be implemented. Therefore, in conventional cases, a hacker may transmit a signal with a malicious command or the like to and thus hack into the electronic apparatus or the system, thereby operating the electronic apparatus or the system overriding the user's original intent.

Accordingly, in terms of security and safety, it is important for the electronic apparatus or the system, which applies the voice recognition to a voice command caused by a user speech, to determine whether the voice command matches the user's intent. As references, <CIT>) and <CIT>) are cited, which disclose a system and method for voice actuation with contextual learning for intelligent machine control and a voiceprint authentication method and apparatus, respectively.

According to an aspect of an exemplary embodiment, there is provided an electronic apparatus including: a receiver configured to receive a voice command of a user; and a processor configured to determine a switching probability from a first state corresponding to a current point of time over to a second state corresponding to the voice command, based on the switching probability previously set between a plurality of states related to the electronic apparatus, and selectively perform an operation corresponding to the voice command in accordance with the determined switching probability. Thus, the electronic apparatus can prevent a command caused by a hacker from being executed instead of a voice command normally issued by a user, thereby improving security.

The processor further allows execution of an operation corresponding to the voice command de when it is determined that the determined switching probability is greater than a preset threshold value, and performs preset authentication procedure with regard to the voice command when it is determined that the determined switching probability is less than or equal to the threshold value. Thus, the electronic apparatus can easily determine the switching probability.

The processor allows execution of the operation corresponding to the voice command when it is determined that the authentication procedure is successfully passed, and prevents execution of the operation and informs the user when the authentication procedure is not successfully passed. Thus, the electronic apparatus can prevent not a hacker attack but the execution of the voice command from being unconditionally blocked.

The processor may generate a user interface (UI) for a user to authenticate the voice command, display the UI on at least one of a display of the electronic apparatus and a display apparatus communicating with the electronic apparatus, and determine whether to pass the authentication procedure based on inputs to the UI.

The processor may generate a user interface (UI) for guiding the user to pronounce a word, display the UI on at least one of a display of the electronic apparatus and a display apparatus communicating with the electronic apparatus, and determine whether to pass the authentication procedure based on a voice input by pronouncing the word displayed on the UI.

The processor may randomly generate the word. Thus, the electronic apparatus can enhance security by preventing words for authentication from leaking out by the hacker.

The electronic apparatus further includes a storage configured to store the switching probability. The processor adjusts a value of the switching probability stored in the storage in accordance with results from selectively performing the operation corresponding to the voice command. Thus, the electronic apparatus can provide a voice command execution environment optimized according to users.

The processor adds add a preset first value to the value of the switching probability stored in the storage when the operation corresponding to the voice command is performed, and subtracts a preset second value from the value of the switching probability stored in the storage when the operation corresponding to the voice command is not performed.

The processor may determine the first state based on at least one of sensed information collected through the electronic apparatus, a history of user inputs to the electronic apparatus, and a history of operations performed by the electronic apparatus.

According to an aspect of another exemplary embodiment, there is provided a method of controlling an electronic apparatus, the method including: receiving a voice command of a user; determining a switching probability from a first state corresponding to a current point of time over to a second state corresponding to the voice command, based on the switching probability previously set between a plurality of states related to the electronic apparatus; and selectively performing an operation corresponding to the voice command in accordance with the determined switching probability.

The performing of the operation includes allowing execution of the operation corresponding to the voice command when it is determined that the determined switching probability is greater than a preset threshold value; and performing an authentication procedure with regard to the voice command when it is determined that the determined switching probability is less than or equal to the threshold value.

The performing of the authentication procedure includes allowing execution of the operation corresponding to the voice command when it is determined that the authentication procedure is successfully passed; and preventing execution of the operation and informing the user when the authentication procedure is not successfully passed.

The performing of the authentication procedure may include: generating a user interface (UI) for the user to authenticate the voice command, displaying the UI on at least one of a display of the electronic apparatus and a display apparatus communicating with the electronic apparatus; and determining whether to pass the authentication procedure based on inputs to the UI.

The performing of the authentication procedure may include: generating a user interface (UI) for guiding the user to pronounce a word, displaying the UI on at least one of a display of the electronic apparatus and a display apparatus communicating with the electronic apparatus; and determining whether to pass the authentication procedure based on a voice input by pronouncing the word displayed on the UI.

The electronic apparatus may randomly generate the word.

The method further includes adjusting a value of previously stored switching probability in accordance with results from selectively performing the operation corresponding to the voice command.

The adjusting of the value of the previously stored switching probability includes may adding a preset first value to the value of the previously stored switching probability when the operation corresponding to the voice command is performed; and subtracting a preset second value from the value of the previously stored switching probability when the operation corresponding to the voice command is not being performed.

The determining of the switching probability may include determining the first state based on at least one of sensed information collected through the electronic apparatus, a history of user inputs to the electronic apparatus, and a history of operations performed by the electronic apparatus.

Below, exemplary embodiments will be described in detail with reference to accompanying drawings. The following descriptions of the exemplary embodiments are made by referring to elements shown in the accompanying drawings.

<FIG> shows an example of a system according to an exemplary embodiment.

As shown in <FIG>, the system according to an exemplary embodiment includes a plurality of electronic apparatuses <NUM>. The plurality of electronic apparatuses <NUM> is not limited to one kind of apparatus, and include a unit entity or thing of various kinds, types and functions. For example, the electronic apparatus <NUM> may be implemented as a television (TV), an electronic photo frame, a portable multimedia assistant, or like display apparatus for displaying an image; a set-top box, or like image processing apparatus for processing an image signal without directly displaying an image; a washing machine, a refrigerator, an air conditioner, or like home appliances; a printer, a copying machine, a scanner, or like office machine; a sensing system including a sensor; a light bulb; an electric heater; and other apparatuses. Besides such an apparatus stationarily installed on a certain surface, the electronic apparatus <NUM> may be implemented as a wearable device, a mobile device, or the like.

Within the system, each electronic apparatus <NUM> may perform wired or wireless communication with other electronic apparatuses <NUM>. In the system, two electronic apparatuses <NUM> may be connected to each other by a peer-to-peer method, i.e. may be directly connected one-to-one. However, there are more limits to the one-to-one connection as the number of electronic apparatuses <NUM> increases. In this regard, the system may include a communication relay apparatus <NUM> connecting with the plurality of electronic apparatuses <NUM>.

The communication relay apparatus <NUM> may be implemented in various forms according to communication protocols for connection between the plurality of electronic apparatuses <NUM>. For example, the communication relay apparatus <NUM> may be an access point (AP) or a hub in case of a local area network (LAN) system, and may be a router or a gateway in case of a wide area network (WAN) system.

With this system, when a user inputs a speech to a certain electronic apparatus <NUM>, the system applies a voice process (e.g., speech recognition) to a voice signal corresponding to the speech, and makes an operation be implemented corresponding to a result of the voice process. Herein, the operation may be implemented in the electronic apparatus <NUM> to which a user iputs the speech, but may be alternatively imolemented by another electronic apparatus <NUM> included in the system.

That is, the system operates as follows. A user speech of "turn on the light" is input to a microphone provided in a mobile apparatus <NUM>. The mobile apparatus <NUM> transmits a voice signal corresponding to the user speech to a voice recognition apparatus <NUM>. The voice recognition apparatus <NUM> analyzes the voice signal to generate a control signal that corresponds to "turn on the light," and transmits the control signal to a lamp <NUM> at which the control signal is aimed. The lamp <NUM> is turned on in response to the instruction of the control signal received from the voice recognition apparatus <NUM>.

In this exemplary embodiment, descriptions will be made on the assumption that the voice recognition apparatus <NUM> performs the voice process with regard to a user's speech from the electronic apparatus <NUM> to which the speech is input. Alternatively, the voice process is performed in the electronic apparatus <NUM>, to which a user's speech is input, in accordance with designs of the system or apparatus. Therefore, this exemplary embodiment is not construed as limiting the present disclosure. Herein, the term "voice recognition apparatus" is used for an apparatus for performing the voice process, and thus does not limit the implementation of the apparatus. The voice recognition apparatus <NUM> is one among various electronic apparatuses, including a server.

In this exemplary embodiment, communication between the voice recognition apparatus <NUM> and the electronic apparatus <NUM> is performed via the communication relay apparatus <NUM>. Alternatively, the voice recognition apparatus <NUM> and the electronic apparatus <NUM> directly connect and communicate with each other. Further, a voice recognition process is performed in one voice recognition apparatus <NUM> or is shared among a plurality of voice recognition apparatuses <NUM>.

Below, the voice recognition process of the voice recognition apparatus <NUM> will be described.

<FIG> is a block diagram showing a process of processing a user speech in a voice recognition apparatus according to an exemplary embodiment.

As shown in <FIG>, the system includes a first electronic apparatus <NUM>, a second electronic apparatus <NUM>, a first voice recognition apparatus <NUM>, and a second voice recognition apparatus <NUM>. In this exemplary embodiment, the voice recognition process is shared between the first voice recognition apparatus <NUM> and the second voice recognition apparatus <NUM>. However, this is merely an example, selected by taking an efficiency of the voice recognition process into account, and does not limit the present disclosure. In accordance with designs, the first voice recognition apparatus <NUM> and the second voice recognition apparatus <NUM> is integrated into one apparatus, or the first electronic apparatus <NUM> performs the voice recognition process instead of at least one of the first voice recognition apparatus <NUM> and the second voice recognition apparatus <NUM>.

When receiving a user speech through a microphone <NUM> at operation S210, the first electronic apparatus <NUM> transmits an analog voice signal corresponding to the user speech to the first voice recognition apparatus <NUM> at operation S220.

The first voice recognition apparatus <NUM> includes a speech-to-text (STT) processor <NUM> configured to perform an STT process. When the first voice recognition apparatus <NUM> receives the voice signal from the first electronic apparatus <NUM>, at operation S230 the STT processor <NUM> performs the STT process to extract a text by converting the voice signal into a digital signal, and transmits the extracted text to the first electronic apparatus <NUM>.

At operation S240, the first electronic apparatus <NUM> transmits the text received from the first voice recognition apparatus <NUM> to the second voice recognition apparatus <NUM>.

The second voice recognition apparatus <NUM> includes a database (DB) <NUM> configured to store commands corresponding to preset operations, and an analysis processor <NUM> configured to analyze a context of a text and retrieve the command corresponding to the text from the database <NUM>. The context refers to a meaningful and logical relationship between words in a sentence, phrases, and clauses. That is, the analysis processor <NUM> analyzes content of a text and determines what command the text corresponds to.

At operation S250, the second voice recognition apparatus <NUM> uses the analysis processor <NUM> to analyze the text received from the first electronic apparatus <NUM>, and retrieves the command corresponding to the text from the database <NUM> in accordance with analysis results.

At operation S260, the second voice recognition apparatus <NUM> transmits the retrieved command to the apparatus in which the command is executed. For example, when the command instructs the second electronic apparatus <NUM> to operate, the second voice recognition apparatus <NUM> transmits the command to the second electronic apparatus <NUM>, so that the second electronic apparatus <NUM> can perform an operation corresponding to the command. On the other hand, when the command instructs the first electronic apparatus <NUM> to operate, the second voice recognition apparatus <NUM> transmits the command to the first electronic apparatus <NUM>.

With these operations, the system selects a command corresponding to a user speech through voice recognition, and makes an operation directed by the selected command be performed.

The STT processor <NUM> and the analysis processor <NUM> may be a processor, a central processing unit (CPU), a microprocessor, a processing circuit, a processing board or like hardware element, or may be an application to be executed by the hardware element.

By the way, a hacker hacks into such a structure for the voice recognition process during a certain time window between the operation where a user makes an input to the microphone <NUM> and the operation where the analysis processor <NUM> analyzes the text. For example, instead of the speech normally input to the microphone <NUM>, the voice signal is transmitted from the hacker to the first voice recognition apparatus <NUM> or the text is transmitted from the hacker to the second voice recognition apparatus <NUM>. In addition, the first voice recognition apparatus <NUM> or the second voice recognition apparatus <NUM> is hacked so that the STT processor <NUM> can receive a voice signal from a hacker or the analysis processor <NUM> can receive a text from a hacker.

In this case, the command eventually selected by the analysis processor <NUM> reflects a hacker's intention instead of a user's intention. The operation based on the command generated without reflecting a user's intention threatens the user's security and safety.

Therefore, the analysis processor <NUM> according to an exemplary embodiment determines whether a command is normal, i.e. whether the command matches a user's intention when the command is selected corresponding to the text. A method of determining whether the command matches a user's intention will be described later. When it is determined that the command matches a user's intention, the analysis processor <NUM> transmits the command to the second electronic apparatus <NUM> so that an operation can be performed corresponding to the command.

On the other hand, when it is determined that the command does not match a user's intention, the analysis processor <NUM> prevents the operation corresponding to the command from being automatically performed and implements subsequent processes. Such subsequent processes may be achieved by various methods, and descriptions thereof will be described later.

<FIG> and <FIG> are a flowchart of showing a control method of the voice recognition apparatus. <FIG> corresponds to the claimed invention when followed by <FIG>.

As shown in <FIG>, at operation <NUM> the voice recognition apparatus obtains a text corresponding to a user speech.

At operation <NUM>, the voice recognition apparatus analyzes content of the obtained text, and selects the command corresponding to the text.

At operation <NUM>, the voice recognition apparatus calls (e.g., accesses) a preset state model. The state model defines a plurality of states previously set in the system, and a value of a switching probability between the states.

Herein, the state of the system refers to a state about at least one among at least one electronic apparatus within the system, a location where the system operates, and a user who uses the system. For example, the state of the system includes a state about a user's activities, a state about a user's position within a location where the system is installed, a state about an operation of the electronic apparatus, a state about a location where the system is installed, etc. The states of the system will be described later in detail.

At operation <NUM>, the voice recognition apparatus determines a first state corresponding to a current point of time among the plurality of states in the state model. To determine the first state at a current point of time, the voice recognition apparatus takes sensed information collected in the electronic apparatus of the system, a history of user inputs, a history of operations, etc. into account.

At operation <NUM>, the voice recognition apparatus determines a second state corresponding to a selected command among the plurality of states in the state model. The second state corresponding to the command refers to the state of the system when the system operates based on the command.

At operation <NUM>, the voice recognition apparatus determines a probability of switching over from the first state to the second state.

As shown in <FIG>, at operation <NUM> the voice recognition apparatus determines whether the value of the previously determined probability is relatively high or low when compared against a threshold. The voice recognition apparatus uses various methods to determine whether the probability is high or low. Examples of these methods will be described later.

When it is determined that the value of the probability is relatively high, the voice recognition apparatus allows the operation corresponding to the command to be performed at operation <NUM>. For example, the voice recognition apparatus identifies a target, at which the operation corresponding to the command is aimed, within the system, and transmits the command to the identified target.

On the other hand, when it is determined that the value of the probability is relatively low, at operation <NUM> the voice recognition apparatus implements authentication procedures for authenticating the execution of the command. This authentication procedures include general operations for allowing a user to confirm whether the command is normally caused by her speech. The authentication procedures are achieved by various methods, and will be described later.

At operation <NUM>, the voice recognition apparatus determines whether the authentication procedures are passed.

When the voice recognition passes the authentication procedures, the voice recognition apparatus returns to the operation <NUM>, thereby performing the operation corresponding to the command.

On the other hand, when the voice recognition fails the authentication procedures, at operation <NUM>, the voice recognition apparatus prevents the operation corresponding to the command from being performed and informs a user that the corresponding operation is not performed. A user is informed by various methods such as a user interface (UI) or a warning message, etc..

Thus, the voice recognition apparatus according to one exemplary embodiment determines whether the command based on the text is caused by the user speech, and allows or prevents the execution of the command.

In this regard, a recognition processing apparatus of the related art has used the following method in order to determine whether a predetermined command is caused by a user's speech. The recognition processing apparatus of the related art determines whether an identified text includes a preset specific word, and determines that a word or sentence input following the specific word is caused by a user's speech when the identified text includes the specific word. However, even in this case, it is difficult or impossible to defend against hacking when the preset specific word is exposed to a hacker.

On the contrary, the voice recognition apparatus according to this exemplary embodiment employs a previously designed model to determine a probability that the command will be normally caused by a user's speech. By preventing the command, the probability of which is low, from being automatically executed, it is possible to effectively circumvent an attack of a hacker on the voice recognition system.

Below, the previously designed state model will be described.

<FIG> shows an example of a partial design of state models referred to by the voice recognition apparatus according to an exemplary embodiment.

As shown in <FIG>, the voice recognition apparatus calls a state model <NUM>. When a command is selected corresponding to a text of a user speech, the voice recognition apparatus identifies the first state corresponding to the current point of time and the second state corresponding to the command in the called state model <NUM>. <FIG> merely shows a very simple state model <NUM> for describing an embodiment, and the state model <NUM> designed in practice includes many states and be more complicated.

The state model <NUM> basically includes two elements. One element is a plurality of states <NUM>, <NUM>, <NUM>, and <NUM>, and the other element is a switching probability between any two of the states <NUM>, <NUM>, <NUM>, and <NUM>. Since the switching probability is basically established in between two of the states <NUM>, <NUM>, <NUM>, and <NUM>, the state model <NUM> has to include at least two states <NUM>, <NUM>, <NUM>, and <NUM>. The state model <NUM> is designed by a manufacturer in accordance with previous experiments, data, etc., or is input by a user.

Each of the states <NUM>, <NUM>, <NUM>, and <NUM> refers to a state of one or more electronic apparatuses within the voice recognition system. In more detail, the states <NUM>, <NUM>, <NUM>, and <NUM> show states of all objects related to the voice recognition system. For example, the states <NUM>, <NUM>, <NUM>, and <NUM> includes a state about activities of a user who uses the system, a state about a location where the electronic apparatus within the system is installed, or a state about an operation of the electronic apparatus in the system. However, information for determining the state about a user or a location is basically collected by the electronic apparatus in the system through a sensing, input or like method. Therefore, the states <NUM>, <NUM>, <NUM>, and <NUM> defined in the state model <NUM> show the states of the system or the states of the electronic apparatuses in the system.

The states <NUM>, <NUM>, <NUM>, and <NUM> are defined by one or more parameters. There are various design methods depending on which parameters are used. For example, one of the states <NUM>, <NUM>, <NUM>, and <NUM> includes three kinds of parameters such as an activity showing content of each operation of the states <NUM>, <NUM>, <NUM>, and <NUM>; a time showing time during which each operation of the states <NUM>, <NUM>, <NUM>, and <NUM> is performed: a location at which each operation of the states <NUM>, <NUM>, <NUM>, and <NUM> is performed; etc..

For example, a Sleep state <NUM> indicates a state that a user sleeps in a bed from <NUM> a. to <NUM> a. In this case, the Sleep state <NUM> has an activity parameter of "Sleep," a time parameter of "<NUM> a. to <NUM> a. ," and a location parameter of "bed.

A WatchingTV state <NUM> indicates a state in which a TV is turned on in a living room from <NUM> p. to <NUM> p. In this case, the WatchingTV state <NUM> has an activity parameter of "watching TV," a time parameter of "<NUM> p. to <NUM> p. ," and a location parameter of "livingroom.

An OpenDoor state <NUM> indicates a state in which a front door is opened from <NUM> a. to <NUM> p. In this case, the OpenDoor state <NUM> has an activity parameter of "opening the front door," a time parameter of "<NUM> a. to <NUM> p. ," and a location parameter of "front Door.

A ListeningRadio state <NUM> indicates a state that a radio is turned on in a bedroom from <NUM> p. to <NUM> a. In this case, the ListeningRadio state <NUM> has an activity parameter of "listening radio," a time parameter of "<NUM> p. to <NUM> a. ," and a location parameter of "bedroom.

Like this, the states <NUM>, <NUM>, <NUM>, and <NUM> in the state model <NUM> indicate operation states of at least one among one or more electronic apparatuses, users, and locations. The voice recognition apparatus identifies a certain state <NUM>, <NUM>, <NUM>, or <NUM> based on information collected through one or more electronic apparatuses in the system, among the plurality of states <NUM>, <NUM>, <NUM>, and <NUM>.

For example, the electronic apparatus, which includes a sensor for sensing a user's position, can determine the user's position through the sensor. Further, the electronic apparatus, which includes a clock or is capable of obtaining information about a current time through a network, can determine the current time. Further, the electronic apparatus generates information about the current operation state in itself and transmit the information to another apparatus. Further, the electronic apparatus performs a specific operation in response to a user's input, store a history of a user's inputs and corresponding operations, and transmit the stored history to another apparatus.

The voice recognition apparatus selects one of the states <NUM>, <NUM>, <NUM>, and <NUM> within the state model <NUM>, based on various pieces of collected information.

By the way, the state model <NUM> includes the switching probabilities established between the plurality of states <NUM>, <NUM>, <NUM>, and <NUM>. The switching probability quantifies a probability of switching from a predetermined state <NUM>, <NUM>, <NUM>, or <NUM> over to another state <NUM>, <NUM>, <NUM>, or <NUM>.

For example, for convenience of description, one among the plurality of states corresponding to the current point of time will be called the first state, and one corresponding to the execution of the command will be called the second state. When a probability of switching over from the first state to the second state has a relatively high value, this switching is more likely to occur, and it is probable that the command is normally caused by a user's speech. On the other hand, when a probability of switching over from the first state to the second state has a relatively low value, this switching is less likely to occur, and it is improbable that the command is normally caused by a user's speech.

Among the plurality of states <NUM>, <NUM>, <NUM>, and <NUM>, the first state and the second state are different states or the same state.

The voice recognition apparatus has one or more preset threshold values. When a value of a probability of switching over from the first state to the second state is taken from the state model <NUM>, the voice recognition apparatus determines whether the value is higher than the threshold value. The voice recognition apparatus determines that the switching probability is high when it is determined that the value is higher than the threshold value, and determines that the switching probability is low when it is determined that the value is lower than the threshold value. The threshold value is set in the state model <NUM>, set by the voice recognition apparatus itself, or set by a user.

For example, suppose that the first state corresponding to the current point of time is the WatchingTV state <NUM>, and the second state directed by the command is the OpenDoor state <NUM>. In this case, the voice recognition apparatus draws a value of <NUM> from the state model <NUM> as a probability of switching over from the WatchingTV state <NUM> to the OpenDoor state <NUM>. The voice recognition apparatus compares the probability value with the threshold value. When the threshold value is <NUM>, the probability value <NUM> is higher than the threshold value. Thus, the voice recognition apparatus determines that the probability of switching over from the WatchingTV state <NUM> to the OpenDoor state <NUM> is relatively high, and determines that the command is likely caused by a normal input. In accordance with determination results, the voice recognition apparatus allows the command to be executed.

In addition, suppose that the first state corresponding to the current point of time is the WatchingTV state <NUM>, and the second state directed by the command is the Sleep state <NUM>. In this case, the voice recognition apparatus takes a value of <NUM> as a probability of switching over from the WatchingTV state <NUM> to the Sleep state <NUM>, and compares the drawn probability value with the threshold value. Since the probability value <NUM> is lower than the threshold value, the voice recognition apparatus determines that the probability of switching over from the WatchingTV state <NUM> to the Sleep state <NUM> is relatively low, and determines that the command is caused by an abnormal input. Thus, the voice recognition apparatus puts the execution of the command on hold, and performs the authentication procedures for allowing a user to authenticate whether the command is normally input by the user.

The voice recognition apparatus allows the command to be executed when a user's authentication is performed in the authentication procedures, and blocks the execution of the command and warns a user when a user's authentication is not performed in the authentication procedures.

Herein, various methods are applied to the authentication procedures as follows.

<FIG> shows an example of a user interface (UI), which makes a request for a user to authenticate a command, displayed by the voice recognition apparatus according to an exemplary embodiment.

As shown in <FIG>, a voice recognition apparatus <NUM> is configured to display a UI <NUM>, which makes a request for a user to select whether to execute a predetermined command, when it is determined that a probability of switching over to a state caused by execution of a command is relatively low. When the voice recognition apparatus <NUM> includes a display in itself, the UI <NUM> is displayed on the display of the voice recognition apparatus <NUM>. Alternatively, when the voice recognition apparatus <NUM> cannot display the UI <NUM> in itself, UI information is transmitted to a predetermined display apparatus <NUM> of a user, so that the UI <NUM> can be displayed on the display apparatus <NUM>.

The UI <NUM> includes various pieces of information, and provide content of a command and options for confirming the execution of the command. Through the UI <NUM>, a user makes an input for instructing the command to be executed or blocked.

The display apparatus <NUM> feeds selection information based on a user's input through the UI <NUM> back to the voice recognition apparatus <NUM>. In response to the received selection information, the voice recognition apparatus <NUM> allows or blocks the execution of the command.

Alternatively, the following authentication procedures are also possible.

<FIG> shows an example of a UI, which makes a request for a user to pronounce a specific word, displayed by the voice recognition apparatus according to one exemplary embodiment.

As shown in <FIG>, a voice recognition apparatus <NUM> displays a UI <NUM>, which makes a request for a user to pronounce a predetermined word, when it is determined that a probability of switching over to a state caused by execution of a command is relatively low. The voice recognition apparatus <NUM> displays the UI <NUM> in itself, or transmits UI information to a separate display apparatus <NUM> so that the display apparatus <NUM> can display the UI <NUM>. In this embodiment, the apparatus for displaying the UI <NUM> includes a microphone <NUM>.

The UI <NUM> includes content of the command, and a text of words requested to be pronounced by a user. A user pronounces the words directed in the UI <NUM> when it is determined that the command displayed on the UI <NUM> is caused by a normal input, but does not pronounce the words directed in the UI <NUM> or makes an input for denying the execution of the command when it is determined that the command is caused by an abnormal input.

When receiving a user speech through the microphone740, the display apparatus <NUM> transmits an audio signal corresponding to the speech to the voice recognition apparatus <NUM>. The voice recognition apparatus <NUM> analyzes and determines whether the received audio signal corresponds to the words indicated in the UI <NUM>, and allows the execution of the command when they do correspond, or otherwise blocks the execution of the command.

Further, when the microphone <NUM> receives no user speech for a preset period of time or the execution of the command is denied through the UI <NUM>, the display apparatus <NUM> transmits information for instructing the command not to be executed to the voice recognition apparatus <NUM>. In response to the received information, the voice recognition apparatus <NUM> blocks the execution of the command.

Herein, the words requested to be pronounced by a user are previously designated in the voice recognition apparatus <NUM>. However, when a previously designated specific word is exposed to a hacker, the authentication procedures are disabled by the hacker. Therefore, the voice recognition apparatus <NUM> generates and use a random word through a random number algorithm to implement the authentication procedures. Thus, the voice recognition apparatus <NUM> can improve the security of the authentication procedures.

In the foregoing state model, invariable values are previously designated as the values of the switching probability between two states. The values of the switching probability between two states are varied depending on a process history of the voice recognition apparatus in accordance with the invention. Below, such an embodiment will be described.

<FIG> is a table showing a principle of adjusting each value of switching probability in a state model referred to by the voice recognition apparatus according to the invention.

As shown in <FIG>, values of a switching probability between two states among plurality of states are tabulated in the state model. This table is just given for convenience of description to simply and clearly show the principle of this exemplary embodiment, and does not necessarily imply that the voice recognition apparatus has to create and manage such a table.

In this table, "FROM" shows the first state corresponding to the current point of time, "TO" shows the second state corresponding to execution of command, and "DEFAULT VALUE" shows a value of a switching probability initially set in the state model.

As described in the foregoing exemplary embodiments, when the processes of determining whether a command is normally caused by a user speech are performed, the voice recognition apparatus adjusts the default value in accordance with process results. That is, the voice recognition apparatus compensates the value of the switching probability between the first state and the second state in the state model for an offset value corresponding to the process results. In this table, "ADJUSTED VALUE" shows a result from compensating "DEFAULT VALUE" for the offset value according to the process results, and is additionally adjusted as the processes are repeated.

In accordance with the process results, there are two kinds of offset values which includes a first value for a case where a probability of switching from the first state over to the second state is relatively high, and a second value for a case where a probability of switching from the first state over to the second state is relatively low. In this exemplary embodiment, the first value is a positive value, and the second value is a negative value. In accordance with design methods, the absolute value of the first value is equal to or different from the absolute value of the second value.

For example, the probability of switching from the WatchingTV state over to the OpenDoor state has a default value of <NUM>. With this, suppose that the first value is <NUM>, and the second value is -<NUM>. During a process for a predetermined command, the voice recognition apparatus adjusts the default value by adding <NUM> to the default value of <NUM> when it is determined that the switching probability is relatively high or the authentication procedures are passed, or by adding -<NUM> to the default value of <NUM> when it is determined that the switching probability is relatively low and the authentication procedures are not passed.

As the process continues, the voice recognition apparatus applies the cumulative compensations to the adjusted value. Thus, the value of the switching probability is not the default value but the adjusted value in accordance with the cumulative processes. Based on such an adjusted value, the voice recognition apparatus determines whether the switching probability is relatively high or not. Thus, the voice recognition apparatus undergoes a kind of learning process and thus provides an optimized state model to a user.

For example, when the threshold value is <NUM>, it is determined that the probability of switching from the WatchingTV state over to the OpenDoor state is relatively high since the default value thereof is <NUM>. However, when the default value is adjusted into <NUM> by reflecting a user's intention in the state model through a learning process according to an exemplary embodiment, it is determined that the switching probability in this case is relatively low.

On the other hand, it is determined that the probability of switching from the WatchingTV state over to the Sleep state is relatively low since the default value thereof is <NUM>. However, when the default value is adjusted into <NUM> as the processes are accumulated, it is determined that the probability in this case is relatively high.

Like this, the default value is adjusted with the cumulative results of the processes, and therefore, the voice recognition apparatus can provide more optimized results to a user.

Below, a method of controlling the voice recognition apparatus according to an exemplary embodiment will be described.

<FIG> is a flowchart showing a control method of the voice recognition apparatus according to the invention.

As shown in <FIG>, operations in this embodiment are continued from the foregoing operations shown in <FIG>. That is, this embodiment shows operations improved from the operations shown in <FIG>. An embodiment shown in <FIG> is presented as an alternative to an embodiment of <FIG> and does not form part of the claimed invention.

At operation <NUM>, the voice recognition apparatus compares the value of the probability of switching from the first state corresponding to the current point of time over to the second state corresponding to an instruction of the command with the threshold value.

At operation <NUM>, the voice recognition apparatus determines whether the probability value is lower than the threshold value.

When it is determined that the probability value is lower than the threshold value, at operation <NUM>, the voice recognition apparatus puts the operations directed by the command on hold, and performs the authentication procedures with regard to the command. The authentication procedures are equivalent to those described in the foregoing exemplary embodiment.

At operation <NUM>, the voice recognition apparatus determines whether the authentication procedures are successfully passed.

When the authentication procedures are not passed, at operation <NUM>, the voice recognition apparatus blocks the operation directed by the command, and informs a user that the operation was not performed.

At operation <NUM>, the voice recognition apparatus applies an offset value, which corresponds to the blocked operation directed by the command, to the probability value, thereby updating the probability value.

At operation <NUM>, the voice recognition apparatus stores the updated probability value.

On the other hand, when it is determined in the operation <NUM> that the probability value is higher than the threshold value or when the authentication procedures are successfully passed in the operation <NUM>, the voice recognition apparatus allows the operation directed by the command to be performed at operation <NUM>.

At operation <NUM>, the voice recognition apparatus applies an offset value, which corresponds to the allowed operation directed by the command, to the probability value, thereby updating the probability value and moving to the operation <NUM>.

Thus, the voice recognition apparatus can provide an optimized voice recognition environment to a user.

In the foregoing exemplary embodiment, the first state corresponding to the current point of time is identified as one among the plurality of preset states in the state model. However, two or more states correspond to the first state in accordance with situations. In this case, the voice recognition apparatus puts the operation directed by the command on hold, and implements the authentication procedures as described above, thereby determining whether to perform the operation.

In the foregoing exemplary embodiments, the electronic apparatus for receiving the user speech and the voice recognition apparatus for processing the voice recognition are provided as separate apparatuses. Referring to <FIG>, the first electronic apparatus includes the microphone, the first voice recognition apparatus performs the STT process, and the second voice recognition apparatus performs the analysis process with regard to the command. However, this is only one method of implementing an exemplary embodiment of the present disclosure. For example, the electronic apparatus and the voice recognition apparatus are provided as a single apparatus. That is, the voice recognition apparatus is configured to include both the microphone and the analysis processor. Alternatively, one voice recognition apparatus performs both the STT process and the analysis process.

Below, basic hardware elements of the voice recognition apparatus will be described.

<FIG> is a block diagram of the voice recognition apparatus according to an exemplary embodiment.

As shown in <FIG>, a voice recognition apparatus <NUM> according to this exemplary embodiment includes a communicator <NUM> configured to communicate with an external electronic apparatus <NUM>, a signal processor <NUM> configured to process data received in the communicator <NUM>, a user input <NUM> configured to receive a user's input, a storage <NUM> configured to store data, and a control processing unit (CPU) <NUM> configured to perform computations for processes in the signal processor <NUM> and control operations of the voice recognition apparatus <NUM>. These elements are connected to one another through a system bus.

In this exemplary embodiment, the CPU <NUM> is provided separately from the signal processor <NUM>, but it is not limited thereto. Alternatively, the CPU <NUM> is achieved by a single system on chip (SoC) integrated with the signal processor <NUM>.

The communicator <NUM> includes a communication interface circuit such as a communication chip, a local area network (LAN) card, etc. The communicator <NUM> includes an Ethernet module supporting a wired communication protocol, or a wireless communication module supporting a wireless communication protocol such as Wi-Fi, Bluetooth, etc. The communicator <NUM> transmits data received from the electronic apparatus <NUM> to the CPU <NUM> or the like, and transmit the data from the CPU <NUM> to the electronic apparatus <NUM>.

The signal processor <NUM> includes a processing board, which has one or more chipsets and circuits, a SoC, or a processor. The signal processor <NUM> analyzes a predetermined text to select or identify a corresponding command, and performs the function of the analysis processor or the like voice recognition process of as described in the foregoing embodiments.

The user input <NUM> is configured to transmit various preset events generated in response to a user's control or input to the CPU <NUM>. The user input <NUM> is variously implemented in accordance with methods of inputting information, e.g. by a physical or electronic button, a touch screen, a touch pad, etc..

The storage <NUM> is configured to store various pieces of data in accordance with processing operations of the CPU <NUM> and the signal processor <NUM>. The storage <NUM> includes a flash memory, a hard disk drive (HDD), a solid-state drive (SSD), and like nonvolatile memory, in which data is retained regardless of whether the voice recognition apparatus <NUM> is powered on or off, and a random access memory (RAM), a buffer, and like volatile memory in which data to be processed by the CPU <NUM> or the signal processor <NUM> is temporarily loaded. The DB as described above in <FIG> is stored in the storage <NUM>.

The CPU <NUM> performs central computation for the processing operations of the signal processor <NUM>, and basically takes a central role of analysis and computation of data.

With this structure, the voice recognition apparatus <NUM> can perform the voice recognition processes as described above in the foregoing exemplary embodiments.

Claim 1:
An electronic apparatus (<NUM>) comprising:
a receiver configured to receive a voice command of a user;
a storage configured to store switching probabilities previously set between a plurality of states related to the electronic apparatus; and
a processor configured to:
based on receiving the voice command through the receiver, determine a first state corresponding to a current point of time among the plurality of states,
determine a second state corresponding to the voice command, among the plurality of states, wherein the second state corresponding to the voice command indicates a state of the electronic apparatus when the electronic apparatus executes the voice command,
determine a switching probability from the determined first state corresponding to the current point of time over to the determined second state corresponding to the voice command, among the switching probabilities, and
selectively allow or prevent an execution of the voice command in accordance with the determined switching probability, wherein the processor is further configured to:
allow the execution of the voice command, and adjust a value of the switching probability of the switching probabilities stored in the storage by adding a preset first value to the value of the switching probability of the switching probabilities stored in the storage, based on the determined switching probability being greater than a threshold value,
perform an authentication procedure with regard to the voice command based on the determined switching probability being less than or equal to the threshold value,
allow the execution of the voice command based on the authentication procedure being successfully passed,
prevent the execution of the voice command and inform the user that the execution of the voice command is not performed, based on the authentication procedure not being successfully passed, and
adjust the value of the switching probability of the switching probabilities stored in the storage by adding the preset first value to the value of the switching probability of the switching probabilities stored in the storage based on the authentication procedure being successfully passed, and subtracting a preset second value from the value of the switching probability of the switching probabilities stored in the storage based on the authentication procedure not being successfully passed.