Patent Description:
In recent years, various electronic devices have been developed along with development of electronic technologies. In particular, recently electronic devices capable of providing a speech recognition service for providing a response to a user's voice by using an artificial intelligence (AI) model which implements human-level intelligence have been developed.

Meanwhile, a method for a user to call a speech recognition service may include a method for pressing a button of an electronic device or uttering a call command. In particular, the method for uttering a call command is advantageous that a user is able to call the speech recognition service while the user is far away from the electronic device, it is easier than pressing a button, and a smooth user interface (UI) may be provided to a user.

However, when the user utters the call command, the call command may be recognized only when the call command is between silences, and accordingly, there is a problem that the speech recognition service is called by recognizing the call command only when the call command is uttered after silence and followed by a silence section.

Meanwhile, when consecutively calling the speech recognition service, a user has to utter the call command for each call. In addition, in a dialogue, it is general to call a person with a pronoun rather than calling a person with his/her name every time. However, the user has to utter the call command rather than the pronoun each time when calling the speech recognition service, thereby causing sense of difference to the user. Patent publication <CIT> describes speech recognition power management.

The disclosure is made in view of the above needs and an object of the disclosure is to provide an electronic device for detecting a call command to call a speech recognition service, although a user consecutively utters a user's voice containing a call command, and a control method thereof.

In accordance with an aspect of the disclosure, there is provided an electronic device as set out in the accompanying claims.

In accordance with another aspect of the disclosure, there is provided a method for controlling an electronic device, the method as set out in the accompanying claims.

According to the various embodiments of the disclosure, it is possible to provide an electronic device for detecting a call command included in a user's voice to call a speech recognition service, even if the user consecutively utters a user's voice including a call command with natural utterance, and a control method thereof.

In addition, when calling the speech recognition service consecutively, even if a user utters a predetermined word including a pronoun, the speech recognition service may be called in the same manner as with the utterance of the call command, thereby providing natural user interface to the user.

In describing the disclosure, a detailed description of the related art or configuration may be omitted when it is determined that the detailed description may unnecessarily obscure a gist of the disclosure. In addition, the embodiments below may be changed in various forms and the scope of the technical idea of the disclosure is not limited to the embodiments below. The embodiments are provided to complete the disclosure and completely transfer the technical idea of the disclosure to those skilled in the art.

It should be noted that the technologies disclosed in this disclosure are not for limiting the scope of the disclosure to a specific embodiment, but they should be interpreted to include all modifications, equivalents and/or alternatives of the embodiments of the disclosure. In relation to explanation of the drawings, similar reference numerals may be used for similar elements.

The expressions "first," "second" and the like used in the disclosure may denote various elements, regardless of order and/or importance, and may be used to distinguish one element from another, and does not limit the elements.

In this disclosure, expressions such as "A or B", "at least one of A [and/or] B,", or "one or more of A [and/or] B," include all possible combinations of the listed items. For example, "A or B", "at least one of A and B,", or "at least one of A or B" includes any of (<NUM>) at least one A, (<NUM>) at least one B, or (<NUM>) at least one A and at least one B.

In the disclosure, unless otherwise defined specifically, a singular expression may encompass a plural expression. It is to be understood that the terms such as "comprise" or "consist of" are used herein to designate a presence of characteristic, number, step, operation, element, part, or a combination thereof, and not to preclude a presence or a possibility of adding one or more of other characteristics, numbers, steps, operations, elements, parts or a combination thereof.

If it is described that a certain element (e.g., first element) is "operatively or communicatively coupled with/to" or is "connected to" another element (e.g., second element), it should be understood that the certain element may be connected to the other element directly or through still another element (e.g., third element). On the other hand, if it is described that a certain element (e.g., first element) is "directly coupled to" or "directly connected to" another element (e.g., second element), it may be understood that there is no element (e.g., third element) between the certain element and the another element.

Also, the expression "configured to" used in the disclosure may be interchangeably used with other expressions such as "suitable for," "having the capacity to," "designed to," "adapted to," "made to," and "capable of," depending on cases. The expression "configured to (or set to)" does not necessarily refer to a device being "specifically designed to" in terms of hardware. Instead, under some circumstances, the expression "a device configured to" may refer to the device being "capable of" performing an operation together with another device or component. For example, the phrase "a processor configured (or set) to perform A, B, and C" may refer, for example, and without limitation, to a dedicated processor (e.g., an embedded processor) for performing the corresponding operations, a generic-purpose processor (e.g., a CPU or an application processor), or the like, that can perform the operations by executing one or more software programs stored in a memory device.

An electronic device in the disclosure may be implemented as a device such as a smartphone, a tablet PC, a speaker, a mobile phone, a telephone, an electronic book reader, a desktop PC, a laptop PC, a work station, a server, a PDA, a portable multimedia player (PMP), an MP3 player, a medical device, a camera, a television, a digital video disk (DVD) player, a refrigerator, an air conditioner, a cleaner, an oven, a microwave, a washing machine, an air purifier, a set-top box, a home automation control panel, a security control panel, a media box (e.g., SAMSUNG HOMESYNC™, APPLE TV™, or GOOGLE TV™), a game console (e.g., XBOX™, PLAYSTATION™) an electronic dictionary, an electronic key, a camcorder, an electronic frame, or a wearable device. The wearable device herein may include at least one of an accessory type (e.g., a watch, a ring, a bracelet, an ankle bracelet, a necklace, a pair of glasses, a contact lens or a head-mounted-device (HMD)); a fabric or a garment-embedded type (e.g.: electronic cloth); skin-attached type (e.g., a skin pad or a tattoo); or a bio-implant type (implantable circuit).

Hereinafter, various embodiments of the disclosure will be described in detail with reference to the accompanying drawings.

<FIG> is a diagram illustrating a system for providing a speech recognition service according to an embodiment.

Referring to <FIG>, a system for providing a speech recognition service may include an electronic device <NUM> and a server <NUM>.

Referring to <FIG>, the electronic device <NUM> may be implemented as a speaker that is a device communicable with the server <NUM>. However, this is merely an example and the electronic device <NUM> may be implemented as various types of devices as described above.

The electronic device <NUM> may provide a service for performing speech recognition of a user's voice to provide information or perform an operation (hereinafter, may be referred to as a speech recognition service or an assistant service).

For this, the electronic device <NUM> may obtain a user's voice and transmit the obtained user's voice to the server <NUM> which provides the speech recognition service. In this case, the electronic device <NUM> may be connected to the server <NUM> via a wired or wireless network to transmit and receive various information.

Specifically, the electronic device <NUM> may detect a wake-up word (or call command) from the user's voice, wake the electronic device <NUM> up, when the wake-up word is detected from the user's voice, and transmit the user's voice to the server <NUM>. Herein, the waking up may refer to that the electronic device <NUM> activates (or calls) the speech recognition service.

Meanwhile, when a user's voice is received from the electronic device <NUM>, the server <NUM> may perform speech recognition regarding the received user's voice. Herein, the server <NUM> may be a server for providing a speech recognition service using an artificial intelligence (AI) agent.

For this, the server <NUM> may include a dialogue system for providing a response to a user's voice by using an artificial intelligence model.

Specifically, the server <NUM> may perform speech recognition regarding a user's voice to convert the user's voice into a text, and grasp intent and entity of the user's voice based on a speech recognition result. The server <NUM> may obtain information regarding the response to the user's voice based on a natural language understanding result and obtain a natural language as response information of the user's voice based on the obtained information. The server <NUM> may transmit the response information to the electronic device <NUM>.

For this, the server <NUM> may include an automatic speech recognition (ASR) module, a natural language understanding (NLU) module, a dialogue management (DM) module, a natural language generation (NLG) module, and the like.

Meanwhile, the electronic device <NUM> may provide a response to the user's voice based on the received response information.

For example, the electronic device <NUM> may convert a text included in the received response information into a voice through a text-to-speech (TTs) to output the voice, or display a user interface including the corresponding text via a display (not illustrated) of the electronic device <NUM>.

The above dialogue system may provide the response to the user's voice, and accordingly, the user may have a dialogue with the electronic device <NUM>.

Meanwhile, although it is the same word, users may utter the word with different utterance speeds, silences (pause between syllables or words), accents, and the like depending on linguistic habits or current situations (stammer, articulation disorder, and the like) of the users. For example, the user may consecutively utter a user's voice containing a wake-up word and a specific word (or phrase, clause, or sentence) without any silences or may utter other words first and then utter the wake-up word, rather than uttering the wake-up word first.

In this case, when detecting the wake-up word, the electronic device <NUM> according to an embodiment of the disclosure may divide the user's voice into a voice section and a silence section, and analyze the user's voice included in the voice section in a phonemic unit to detect the wake-up word.

Accordingly, the electronic device <NUM> may detect the wake-up word from the user's voice, not only when the user consecutively utters the user's voice including the wake-up word and the specific word without any silence, but also when the user utters the wake-up word after uttering other words first.

Accordingly, a range for detecting the wake-up word for calling the speech recognition service and an accuracy thereof may be improved, and the user is able to call the speech recognition service with natural utterance. Therefore, a level of satisfaction and convenience of the user may be improved.

<FIG> is a block diagram of an electronic device according to an embodiment.

Referring to <FIG>, the electronic device <NUM> may include a communicator <NUM> and a processor <NUM>.

The communicator <NUM> may communicate with an external device according to various types of communication methods. For example, the communicator <NUM> may be connected to the server <NUM> or an external electronic device via a wired or wireless network to perform the communication. Herein, the server <NUM> may include a server for providing a speech recognition service by using a trained artificial intelligence model.

In this case, the communicator <NUM> may transmit the user's voice obtained by the electronic device <NUM> to the server <NUM> and receive response information corresponding to the user's voice obtained via the artificial intelligence model from the server <NUM>.

For this, the communicator <NUM> may include at least one of a Wi-Fi chip, an Ethernet chip, a Bluetooth chip, and a wireless communication chip.

The processor <NUM> may control general operations of the electronic device <NUM>.

In particular, the processor <NUM> detects a voice section from an audio signal obtained by the electronic device <NUM>, identifies whether a wake-up word exists in the user's voice included in the detected voice section, transmits the user's voice to the server <NUM> providing the speech recognition service via the communicator <NUM>, when it is identified that the wake-up word exists in the user's voice, and provides a response to the user's voice based on the received response information, when response information regarding the user's voice is received from the server <NUM>.

Specifically, the electronic device <NUM> may obtain an audio signal from a voice uttered by the user. For example, the electronic device <NUM> may obtain the audio signal from the voice uttered by the user input via an internal microphone or an external microphone. In another embodiment, the electronic device <NUM> may receive a voice uttered by the user input to an external electronic device (not illustrated) from the external electronic device (not illustrated) and obtain the audio signal from the received voice.

In this case, the processor <NUM> may store the audio signal obtained by the electronic device <NUM> in a memory (not illustrated).

Meanwhile, the processor <NUM> detects a voice section from the audio signal obtained by the electronic device <NUM>. Specifically, the processor <NUM> may detect the voice section by a method for firstly removing a surrounding noise or radio noise from the audio signal and then removing a silence section (or non-voice section).

In this case, the processor <NUM> may detect the voice section and the silence section from the audio signal based on a level of the audio signal. Herein, the level of the audio signal may be a voltage (or power) value of the audio signal or an average voltage (or power) value. Specifically, the processor <NUM> may detect a section having a magnitude equal to or more than a predetermined level from the audio signal as the voice section and detect a section having a magnitude less than the predetermined level from the audio signal as the silence section. For example, in general, if <NUM> decibels (dB) that is a level when a person is whispering is set as a reference level, the processor <NUM> may detect a section having a magnitude equal to or more than <NUM> decibels from the audio signal as the voice section and detect a section having a magnitude less than <NUM> decibels as the silence section.

The processor <NUM> identifies (or detects) whether a wake-up word exists in the user's voice included in the detected voice section.

At this time, the wake-up word may refer to a trigger word (or call command) for executing the speech recognition service or requesting a response of the speech recognition service, and the user may utter the wake-up word in order to call the speech recognition service by a voice. Such a wake-up word may be a name or the like of an artificial intelligence for providing speech recognition service.

For this, the processor <NUM> may divide the user's voice in the phonemic unit to obtain a plurality of phonemes from the user's voice, and obtain a word configured with the obtained plurality of phonemes.

Specifically, the processor <NUM> may divide the user's voice included in the detected voice section into texts in the phonemic unit through a speech-to-text (STT) process. Herein, the phoneme may refer to a minimum unit of a sound. The processor <NUM> may search for a word with a pronunciation same as a pronunciation of a set of phonemes from a pronunciation dictionary, and obtain a word with a meaning from the obtained plurality of phonemes. Alternatively, the processor <NUM> may input the obtained plurality of phonemes to a trained artificial intelligence model to obtain a word with a meaning from the obtained plurality of phonemes. In this case, the artificial intelligence model may be a model trained to obtain a word with a meaning from the obtained plurality of phonemes by considering utterance intent of the user, a relationship between a syllable and other syllables before and after the syllable, and the like.

Hereinafter, the processor <NUM> identifies whether the wake-up word exists in the user's voice included in the voice section.

Specifically, when a word obtained as a result of analysis of the user's voice included in the voice section in a phonemic unit matches with the wake-up word or at least one of a plurality of words obtained as a result of analysis matches with the wake-up word, the processor <NUM> may determine that the wake-up word exists in the user's voice. At this time, the processor <NUM> may determine that the words match with each other, not only when the obtained word matches with the wake-up word, but also when the both words have a similarity equal to or more than a predetermined value.

For example, it is assumed that "Bixby" is the wake-up word and the user utters "Bixby, how's the weather?".

In this case, the processor <NUM> may detect a user's voice such as "Bixby,how's the weather?" from a voice section, and analyze the user's voice in a phonemic unit to obtain words such as "Bixby", "how's", and "weather" from the user's voice such as "Bixby,how's the weather?" At this time, the processor <NUM> may determine that the wake-up word exists in the user's voice such as "Bixby,how's the weather?", since "Bixby" among the obtained words matches with the wake-up word.

In another example, it is assumed that "Bixby" is the wake-up word and the user utters "Hey, Bixby, how's the weather?".

In this case, the processor <NUM> may remove a silence section, detect a user's voice such as "Hey, Bixby, how's the weather?" from each voice section, and analyze the user's voice in a phonemic unit to obtain words such as "Hey", "Bixby", "how's", and "the weather" from the user's voice such as "Hey, Bixby, how's the weather?" At this time, since the "Bixby" among the obtained words matches with the wake-up word, the processor <NUM> may determine that the wake-up word exists in the user's voice such as "Hey, Bixby, how's the weather?".

Accordingly, when some words among the user's voice match with the wake-up word, the processor <NUM> may identify that the wake-up word exists in the user's voice.

Meanwhile, the processor <NUM> may divide the user's voice in the phonemic unit to obtain a plurality of phonemes from the user's voice, and determine whether the phoneme corresponding to the wake-up word exists in the obtained plurality of phonemes. Herein, the phoneme corresponding to the wake-up word may be a first phoneme among the phonemes divided from the wake-up word in the phonemic unit.

When it is determined that the phoneme corresponding to the wake-up word exists in the obtained plurality of phonemes, the processor <NUM> may preferentially combine the plurality of phonemes obtained in sequence from the phoneme corresponding to the wake-up word to obtain a word configured with the obtained plurality of phonemes.

For example, it is assumed that "<IMG> (corresponding to Korean alphabets of Bixby)" is the wake-up word and the user utters "<IMG> (Hey, Bixby)". Herein, the phoneme corresponding to the wake-up word may be "<IMG>" that is a first phoneme among the phonemes "<IMG>, and <IMG>" divided from the predetermined wake-up word in the phonemic unit.

For reference, pronunciations corresponding to Korean consonants are same as in Table <NUM> below and pronunciations corresponding to Korean vowels are same as in Table <NUM> below.

In this case, the processor <NUM> may detect a user's voices such as "<IMG> (Hey)" and "<IMG> (Bixby)" from the voice section and analyze the user's voices in the phonemic unit to obtain phonemes such as "<IMG>, and <IMG>" and "<IMG>, <IMG>, and <IMG>'' from the user's voices. Herein, when obtaining a word using the phonemes obtained from the user's voice, the processor <NUM> may preferentially obtain the word "<IMG> (Bixby)" configured with "<IMG>, and <IMG>'' which are phonemes obtained in sequence from "<IMG>" which is a phoneme corresponding to the wake-up word. The processor <NUM> may identify that the wake-up word exists in the user's voice, since the obtained word "Bixby" matches with the wake-up word.

Accordingly, the wake-up word is preferentially obtained from the user's voice to improve a speed for identifying whether the wake-up word exists in the user's voice.

Meanwhile, when it is identified that the wake-up word exists in the user's voice, the processor <NUM> may wake the electronic device <NUM> up to provide a speech recognition service.

Specifically, when it is identified that the wake-up word exists in the user's voice, the processor <NUM> transmits the user's voice to the server <NUM> for providing the speech recognition service via the communicator <NUM>. At this time, the processor <NUM> may transmit the user's voice containing audio information or text information to the server <NUM>.

Herein, when it is identified that the wake-up word exists in the user's voice, the processor <NUM> may transmit the remaining user's voice excluding the wake-up word from the user's voice to the server <NUM>.

For example, it is assumed that the "Bixby" is the wake-up word and the user utters "Bixby,how's the weather?" In this case, the processor <NUM> may obtain words such as "Bixby", "how's", and "weather" from the user's voice. Herein, the processor <NUM> may determine that the wake-up word exists in the user's voice, since the "Bixby" matches with the wake-up word among the obtained words. At this time, the processor <NUM> may transmit "how's the weather" that is the remaining user's voice excluding the wake-up word "Bixby" to the server <NUM>.

Herein, the processor <NUM> may detect a plurality of voice sections from the audio signal, and, when a part of the user's voice included in any one voice section among a plurality of voice sections matches with the wake-up word, the processor <NUM> may transmit the remaining user's voice excluding the wake-up word from the user's voice included in any one voice section and the user's voice included in the remaining voice sections to the server <NUM>. Herein, the plurality of voice sections may be divided by a period of time for keeping a silence (or length of a silence section).

For example, it is assumed that "Bixby" is the wake-up word and the user utters "Hey, Bixby, how's the weather?" In this case, the processor <NUM> may detect a plurality of voice sections of "Hey, Bixby (first voice section)", (silence section), and "how's the weather (second voice section)" in sequence from the audio signal obtained by the electronic device <NUM>. At this time, since "Bixby" which is a part of the user's voice included in the "Hey, Bixby (first voice section)" among the detected plurality of voice sections matches with the wake-up word, the processor <NUM> may transmit "Hey" which is the remaining part excluding the wake-up word included in the "Hey, Bixby (first voice section)" and "how's the weather" that is the user's voice included in the remaining second voice section to the server <NUM>.

Meanwhile, the processor <NUM> may transmit a user's voice obtained for a predetermined period before or after a point when it is identified that the wake-up word exists in the user's voice to the server <NUM>.

For example, it is assumed that the wake-up word is "Bixby" and the user utters "Hello? Hey, Bixby, how's the weather?" In this case, the processor <NUM> may remove the silence section, detecting user's voices such as "Hello", "Hey", "Bixby", and "how's the weather" from each voice section, and analyze the user's voices in the phonemic unit to obtain words such as "Hello", "Hey", "Bixby", "how's", and "weather" from the user's voices. Herein, the processor <NUM> may identify that the wake-up word exists in the user's voice, since "Bixby" matches with the wake-up word among the obtained words.

Assuming that the user's voice obtained for the predetermined period of time before the point when "Bixby" is identified is "Hey", and the user's voice obtained for the predetermined period of time after the point when "Bixby" is identified is "weather, the processor <NUM> may transmit the user's voice containing "Hey" obtained for the predetermined period of time before the point when "Bixy" is identified and the user's voice containing "weather" obtained for the predetermined period of time after the point when "Bixy" is identified, among the remaining user's voices excluding the wake-up word "Bixby", to the server <NUM>.

Meanwhile, when response information regarding the user's voice is received form the server <NUM>, the processor <NUM> provides a response to the user's voice based on the received response information.

For this, the server <NUM> may receive a user's voice from the electronic device <NUM>, perform speech recognition regarding the user's voice to obtain response information regarding the user's voice, and transmit the response information of the user's voice to the electronic device <NUM>.

When the response information regarding the user's voice is received form the server <NUM>, the processor <NUM> may provide the response to the user's voice in at least one form of a sound, a text, an image, and a video based on the received response information or perform a specific operation (e.g., operation of turning light off in living room) as the response to the user's voice based on the received response information. At this time, the processor <NUM> may convert a text included in the received response information into a sound through text-to-speech (TTS) technology to output a sound, or display the text included in the received response information on a display (not illustrated) of the electronic device <NUM> via user interface, based on the received response information.

Meanwhile, when an audio signal is obtained within a predetermined period of time after it is identified that the wake-up word exists in the user's voice, the processor <NUM> identifies whether a predetermined word exists in the user's voice included in the voice section of the audio signal, and transmits the user's voice included in the voice section to the server <NUM> via the communicator <NUM>, when it is identified that the predetermined word exists.

Herein, the predetermined word may include a pronoun. For example, the predetermined word may include a pronoun such as you, sweetie, darling, or honey, and the user may add or change the predetermined word.

According to an embodiment described above, the user may utter the predetermined word including the pronoun to call the speech recognition service within the predetermined period of time after detecting the wake-up word, and the user may perform the speech recognition through smooth dialogue like as normal dialogue with a person. Accordingly, convenience and satisfaction of the user who consecutively calls the speech recognition service may be improved.

<FIG> and <FIG> are diagrams illustrating a method for controlling the electronic device according to an embodiment.

Referring to <FIG>, the processor <NUM> detects the voice section from the audio signal obtained by the electronic device <NUM> (S300).

Next, the processor identifies whether the wake-up word exists in the user's voice included in the detected voice section.

Specifically, the processor <NUM> may analyze the user's voice included in the detected voice section in the phonemic unit to obtain phonemes from the user's voice (S305).

The processor <NUM> may accumulate phonemes obtained until the end of the voice section to obtain a plurality of phonemes, and obtain a word configured with the obtained plurality of phonemes (S310).

The processor <NUM> may determine whether the voice section has ended (S315). At this time, a length of the silence section may be a reference for determining whether the voice section has ended. For example, when the time for keeping the silence satisfies a predetermined period of time, the processor <NUM> may determine that the voice section has ended.

As a result of the determination whether the voice section has ended, when the voice section has ended (S315, Yes), the processor <NUM> may determine whether the user's voice including the word configured with the obtained plurality of phonemes matches with the wake-up word (S320), and when the voice section has not ended (S315, NO), the processor <NUM> may obtain a plurality of phonemes from the detected voice section again (S310).

As a result of the determination whether the user's voice matches with the wake-up word, when a part of the user's voice matches with the wake-up word (S320, Yes), the processor <NUM> identifies that the wake-up word exists in the user's voice. Meanwhile, when the user's voice does not match with the wake-up word (S320, No), the processor <NUM> may perform an operation of detecting the voice section from the audio signal obtained by the electronic device <NUM> again (S300).

When it is identified that the wake-up word exists in the user's voice, the processor <NUM> transmits the user's voice to the server <NUM> providing the speech recognition service via the communicator <NUM>.

When response information regarding the user's voice is received from the server, the processor <NUM> provides a response to the user's voice based on the received response information (S330).

As another embodiment, referring to <FIG>, a case where an audio signal is obtained within a predetermined period of time from the point when it is identified that the wake-up word exists will be described.

The processor <NUM> detects a voice section from an audio signal obtained by the electronic device <NUM> within the predetermined period of time from the point when it is identified that the wake-up word exists (S340), may analyze the user's voice included in the detected voice section in the phonemic unit to obtain phonemes from the user's voice (S345), may accumulate phonemes obtained until the end of the voice section to obtain a plurality of phonemes, and may obtain a word configured with the obtained plurality of phonemes (S350).

The processor <NUM> may determine whether the voice section has ended (S355), and, when the voice section has ended (S355, Yes), determine whether the user's voice including the word configured with the obtained plurality of phonemes matches with the predetermined word (S360).

Herein, the predetermined word may include a pronoun. For example, the predetermined word may include a pronoun such as "you, sweetie, darling, or honey".

When it is identified that the predetermined word exists in the user's voice (S360, Yes), the processor <NUM> transmits the user's voice included in the voice section of the audio signal to the server <NUM> via the communicator <NUM> (S365).

When the response information regarding the user's voice is received form the sever <NUM>, the processor <NUM> provides the response to the user's voice based on the received response information (S370).

<FIG> are diagrams illustrating specific examples of determining a wake-up word according to an embodiment. Herein, it is assumed that the wake-up word is set as "Bixby".

Referring to <FIG>, the user utters "<IMG>? (Bixby, which restaurant do you recommend?)".

In this case, the processor <NUM> may detect voice sections and a silence section such as "<IMG> (Bixby) (first voice section)", (silence section), and "<IMG> (which restaurant do you recommend) (second voice section)" from the voice uttered by the user, and analyze the detected voice sections in the phonemic unit to obtain a plurality of phonemes such as "<IMG>, and <IMG>", and "<IMG>, <IMG>, and <IMG>".

The processor <NUM> may input the obtained plurality of phonemes to a trained artificial intelligence model to obtain words such as "Bixby", "recommend", "restaurant", "do" from the obtained plurality of phonemes. Since "Bixby" among the obtained words matches with the wake-up word, the processor <NUM> may determine that the wake-up word exists in the user's voice.

The processor <NUM> may transmit "which restaurant do you recommend" among remaining user's voice excluding the wake-up word to the server <NUM>. When response information is received as a result of recognition regarding "which restaurant do you recommend" from the server <NUM>, the processor <NUM> may provide a response such as "How's restaurant YYY located in XXX?" based on the received response information.

Referring to <FIG>, the user utters "<IMG> (Bixby,which restaurant do you recommend?)".

In this case, the processor <NUM> may detect "<IMG> (Bixby,which restaurant do you recommend)" as a voice section from the voice uttered by the user, and analyze the detected voice section in the phonemic unit to obtain a plurality of phonemes such as "<IMG>, <IMG>, and <IMG>".

The processor <NUM> may input the obtained plurality of phonemes to the trained artificial intelligence model to obtain words such as "Bixby", "recommend", "restaurant", and "do" from the obtained plurality of phonemes. Since "Bixby" among the obtained words matches with the wake-up word, the processor <NUM> may determine that the wake-up word exists in the user's voice.

Referring to <FIG>, the user utters "<IMG>? (What time is it, Bixby?)".

In this case, the processor <NUM> may detect voice sections and a silence section such as "<IMG> (What time is it) (first voice section)", (silence section), "<IMG> (Bixby) (second voice section)" from the voice uttered by the user, and analyze the detected voice sections in the phonemic unit to obtain a plurality of phonemes such as "<IMG>, and <IMG>", "<IMG>, and <IMG>".

The processor <NUM> may input the obtained plurality of phonemes to the trained artificial intelligence model to obtain words such as "what", "time", "is it", and "Bixby" from the obtained plurality of phonemes. Since "Bixby" among the obtained words matches with the wake-up word, the processor <NUM> may determine that the wake-up word exists in the user's voice.

At this time, the processor <NUM> may transmit "what time is it" among the remaining user's voices excluding the wake-up word to the server <NUM>. When the response information is received from the server <NUM> as a result of the recognition regarding "what time is it", the processor <NUM> may provide a response such as "The current time is <NUM> P. M" based on the received response information.

<FIG> is a diagram illustrating a specific example of determining a predetermined word according to an embodiment. Herein, it is assumed that the audio signal is obtained within the predetermined period of time after the electronic device <NUM> is waken up and it is assumed that the predetermined word is set as "hey".

Referring to <FIG>, the user utters "<IMG>? (Hey, what time is it?)" within the predetermined period of time after the electronic device <NUM> is waken up.

In this case, the processor <NUM> may detect voice sections and a silence section such as "<IMG> (Hey) (first voice section", (silence section), and "<IMG> (what time is it) (second voice section" from the voice uttered by the user, and analyze the detected voice sections in the phonemic unit to obtain a plurality of phonemes such as "<IMG>", "<IMG>, and <IMG>".

The processor <NUM> may input the obtained plurality of phonemes to the trained artificial intelligence model to obtain words such as "hey", "what", "time", and "is it" from the obtained plurality of phonemes. Since "hey" among the obtained words matches with the predetermined word, the processor <NUM> may determine that the predetermined word exists in the user's voice.

At this time, the processor <NUM> may transmit "what time is it" that is the remaining user's voice excluding the predetermined word to the server <NUM>. When the response information regarding "what time is it" is received from the server <NUM>, the processor <NUM> may provide a response such as "The current time is <NUM> P. M" based on the received response information.

<FIG> is a block diagram specifically illustrating a configuration of the electronic device according to an embodiment.

Referring to <FIG>, the electronic device <NUM> may include the communicator <NUM>, the processor <NUM>, the memory <NUM>, an input unit <NUM>, a microphone <NUM>, a speaker <NUM>, and a display <NUM>.

The communicator <NUM> may communicate with various types of external devices according to various communication methods. The communicator <NUM> may include at least one of a Bluetooth chip <NUM>, a Wi-Fi chip <NUM>, a wireless communication chip <NUM>, and an Ethernet chip <NUM>.

Herein, the communicator <NUM> may be controlled by the processor <NUM> to communicate with the server <NUM> or various external devices.

In particular, the communicator <NUM> may communicate with the server <NUM> for performing the speech recognition service via the trained artificial intelligence model. In other words, the communicator <NUM> may transmit input data including information regarding a plurality of slots to the server <NUM> and receive information regarding a natural language corresponding to the input data obtained through the artificial intelligence model from the server <NUM>.

The processor <NUM> (or controller) may control general operations of the electronic device <NUM>.

The processor <NUM> may include a RAM <NUM>, a ROM <NUM>, a graphics processor <NUM>, a main CPU <NUM>, first to n-th interfaces <NUM>-<NUM> to <NUM>-n, and a bus <NUM>. The RAM <NUM>, the ROM <NUM>, the graphics processor <NUM>, the main CPU <NUM>, and the first to n-th interfaces <NUM>-<NUM> to <NUM>-n may be connected to each other via the bus <NUM>.

The memory <NUM> may store various programs and data necessary for the operations of the electronic device <NUM>. In particular, the memory <NUM> may store at least one instruction or data related to the speech recognition service.

The memory <NUM> may include a non-volatile memory, a volatile memory, a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD). The memory <NUM> may be accessed by the processor <NUM> and reading, recording, editing, deleting, or updating of the data by the processor <NUM> may be executed. A term, memory, in the disclosure may include the memory <NUM>, a RAM <NUM> or the ROM <NUM> in the processor <NUM>, or a memory card (not illustrated) (e.g., micro SD card or memory stick) mounted on the electronic device <NUM>.

In addition, the memory <NUM> may store a program and data for configuring various images to be displayed in a display region of the display <NUM>, various programs and data to be output as a sound via the speaker <NUM>, and the like. Further, the memory <NUM> may store various artificial intelligence models of the disclosure.

The input unit <NUM> may receive various user inputs to transmit the user inputs to the processor <NUM>. The input unit <NUM> may include, for example, a touch panel, a (digital) pen sensor, or a key. The touch panel may use, for example, at least one type of an electrostatic type, a pressure-sensitive type, an infrared type, or an ultrasonic type. In addition, the touch panel may further include a control circuit. The touch panel may further include a tactile layer and may provide tactile sensation to the user. The (digital) pen sensor may be, for example, a part of a touch panel or may include a separate sheet for recognition. The key may include, for example, a physical button, an optical key, or a keypad.

The microphone <NUM> may obtain a voice. In particular, the microphone <NUM> may be provided in the electronic device <NUM>, but this is merely an embodiment, and the microphone <NUM> may be provided as an external device connected to the electronic device <NUM> electrically or via a network.

The speaker <NUM> may output not only various pieces of audio data obtained by executing various processing such as decoding, amplification, or noise filtering by an audio processor (not illustrated), but also various alerts or voice messages. Particularly, the configuration of outputting a sound may be implemented as the speaker <NUM>, but this is merely an embodiment, and the configuration may be implemented as an output terminal capable of outputting audio data.

In particular, the speaker <NUM> may output a natural language obtained via a natural language generation unit (not illustrated) as a voice message via TTS (not illustrated).

The display <NUM> may display image data processed in an image processor (not illustrated) on a display region (or display). The display region may refer to at least a part of the display <NUM> exposed to one surface of a housing of the electronic device <NUM>. The part of the display <NUM> may be combined with at least one of a front surface region, a side surface region, and a rear surface region of the electronic device <NUM> in a form of a flexible display. The flexible display may be curved, bent, or rolled through a paper-thin flexible substrate without any damages.

<FIG> is a flowchart illustrating a method for controlling the electronic device according to an embodiment.

Firstly, the electronic device <NUM> may detect a voice section from the obtained audio signal (S710).

According to an embodiment, the detecting the voice section from the audio signal may include detecting a voice section and a silence section from the audio signal based on a level of the audio signal, and, based on a part of the user's voice included in the detected voice section matching with a wake-up word, identifying that the wake-up word exists in the user's voice.

Herein, the detecting the voice section from the audio signal may include detecting a section having a magnitude equal to or more than a predetermined level from the audio signal as the voice section, and detecting a section having a magnitude less than the predetermined level from the audio signal as the silence section.

Meanwhile, the detecting the voice section from the audio signal may include detecting a plurality of voice sections from the audio signal.

Next, it is identified whether the wake-up word exists in the user's voice included in the detected voice section (S720). In particular, when a part of the user's voice matches with the wake-up word, it may be identified that the wake-up word exists in the user's voice.

According to an embodiment, the identifying whether the wake-up word exists in the user's voice may include obtaining a plurality of phonemes from the user's voice by dividing the user's voice in a phonemic unit, and, based on a word configured with the obtained plurality of phonemes matching with the wake-up word, identifying that the wake-up word exists in the user's voice.

Next, when it is identified that the wake-up word exists in the user's voice, the user's voice is transmitted to the server <NUM> providing a speech recognition service via the communicator (S730).

Herein, the transmitting the user's voice to the server <NUM> providing the speech recognition service may include transmitting a remaining user's voice excluding the wake-up word from the user's voice to the server <NUM>.

Herein, the transmitting the user's voice to the server <NUM> providing the speech recognition service may include, based on a part of the user's voice included in any one voice section among a plurality of voice sections matching with the wake-up word, transmitting a remaining user's voice excluding the wake-up word among the user's voice included in the any one voice section and a user's voice included in the remaining voice sections to the server <NUM>.

Next, when the response information regarding the user's voice is received from the server <NUM>, a response to the user's voice is provided based on the received response information (S740).

Meanwhile, according to the invention, when an audio signal is obtained within a predetermined period of time after identifying that the wake-up word exists in the user's voice, it is identified whether a predetermined word exists in a user's voice included in a voice section of the audio signal, and based on the predetermined word being identified to exist, the user's voice included in the voice section is transmitted to the server <NUM>.

Herein, the predetermined word may include a pronoun.

In this disclosure, the term "unit" or "module" may include a unit implemented with hardware, software, or firmware and may be interchangeably used with terms, for example, logic, logic blocks, parts, or circuits. The "unit" or the "module" may be a part integrally formed or a minimum unit or a part of the part performing one or more functions. For example, the module may be implemented as an application-specific integrated circuit (ASIC).

Various embodiments of the disclosure may be implemented as software including instructions stored in machine (e.g., computer)-readable storage media. The machine is a device which invokes instructions stored in the storage medium and is operated according to the invoked instructions, and may include an electronic device (e.g., electronic device <NUM>) according to the disclosed embodiments. In a case where the instruction is executed by a processor, the processor may perform a function corresponding to the instruction directly or using other elements under the control of the processor. The instruction may include a code made by a compiler or a code executable by an interpreter. The machine-readable storage medium may be provided in a form of a non-transitory storage medium. Here, the "non-transitory" storage medium is tangible and may not include signals, and it does not distinguish that data is semi-permanently or temporarily stored in the storage medium.

The methods according to various embodiments may be provided in a computer program product. The computer program product may be exchanged between a seller and a purchaser as a commercially available product. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)) or distributed online through an application store (e.g., PlayStore™). In a case of the on-line distribution, at least a part of the computer program product may be at least temporarily stored or temporarily generated in a storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

Claim 1:
An electronic device (<NUM>) comprising:
a communicator (<NUM>); and
a processor (<NUM>) configured to:
detect a first voice section from a first audio signal obtained by the electronic device;
identify whether a wake-up word exists in a user's voice included in the first voice section;
based on the wake-up word being identified to exist in the user's voice included in the first voice section, transmit the user's voice to a server (<NUM>) for providing a speech recognition service via the communicator;
based on response information regarding the user's voice being received from the server, provide a response to the user's voice based on the received response information;
wherein the processor is configured to, based on a part of the user's voice matching with the wake-up word, identify that the wake-up word exists in the user's voice; and
wherein the processor is further configured to:
detect a second voice section from a second audio signal obtained by the electronic device within a predetermined period of time after identifying that the wake-up word exists in the user's voice;
identify whether a predetermined word exists in the user's voice included in the second voice section; and
based on the predetermined word being identified to exist in the user's voice included in the second voice section, transmit the user's voice included in the second voice section to the server via the communicator.