Patent Description:
With the recent development of digital technology, various mobile electronic devices (for example, mobile communication devices, Personal Digital Assistants (PDAs), smartphones, tablet Personal Computers (PCs), and so on) capable of processing communication and personal information are available. Such electronic devices are equipped with a speech recognition function. Accordingly, when a user inputs voice, the related art electronic device may recognize the voice and output a recognition result.

Moreover, an electronic device performs server device management in order to improve the reliability of a speech recognition result. Accordingly, in relation to using a speech recognition function, an electronic device performs an operation for transmitting voice data to a server device and receiving information corresponding thereto. During this operation, the electronic device may experience significant delay. The delay may cause inconvenience during repeated operation of a speech recognition function.

<CIT> relates to a method and system for creating voice commands using an automatically maintained log interactions performed by a user. A voice command creation facility maintains a log containing indications of interactions performed by the user. When the facility receives an instruction from the user to create a command, the facility displays at least a portion of the log. The facility then receives user input identifying interactions whose indications are displayed in the log that are to be automatically repeated when the created command is invoked, and creates a command that automatically repeats the identified interactions when invoked.

<CIT> relates to adaptation methods and systems for speech systems. Methods and systems are provided for adapting a speech system. In one example a method includes: processing a spoken command with one or more models of one or more model types to achieve model results; evaluating a frequency of the model results; and selectively updating the one or more models of the one or more model types based on the evaluating.

<CIT> relates to a device having a voice or manual user interface and process for aiding with learning the voice instructions of such a device. The device has a voice and/or manual user interface for accessing its functions. The functions are accessible through a hierarchy of menus in which the branches can be short-circuited by voice instructions referred to as voice short-cuts. Means are provided for storing the sequences of interactions performed by the user and for detecting the cases in which a voice short-cut could have been used instead of the sequence of interactions. In this case, a message is sent to the user in order to inform him or her of the existence of the detected voice short-cut. The document also relates to a process for aiding with learning the voice instructions of the device using the means above.

The above and other aspects, and features and advantages of certain exemplary embodiments and aspects of the present disclosure will be more apparent from the following description, taken in conjunction with the accompanying drawings, in which:.

The following description of exemplary embodiments of the present disclosure, with reference to the accompanying drawings, is provided to assist in a comprehensive understanding of the present invention, as defined by the claims. The description includes various specific details to assist in that understanding but these are to be regarded as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope of the invention.

For example, an electronic device according to various exemplary embodiments of the present disclosure is described in more detail with reference to the accompanying drawings. Various modifications are possible in various embodiments of the present disclosure and specific embodiments are illustrated in the accompanying drawings and related detailed descriptions are provided. However, this does not limit the various embodiments of the present disclosure to a specific embodiment and it should be understood that the present disclosure covers all modifications, equivalents, and/or replacements of this disclosure provided they are within the scope of the appended claims.

With respect to the descriptions of the accompanying drawings, the same or similar reference numerals may be used to refer to the same or similar elements.

Detailed descriptions of structures, constructions, functions or processes known in the art may be omitted for clarity and conciseness, and to avoid obscuring the subject matter of the present disclosure.

Features, elements, components, integers, steps, processes, operations, functions, characteristics, properties and/or groups thereof described in conjunction with a particular aspect, embodiment, claim or example of the invention are to be understood to be applicable to any other aspect, embodiment, claim or example described herein unless incompatible therewith.

<FIG> is a block diagram of a voice command providing system according to an embodiment of the present disclosure.

Referring to <FIG>, a voice command providing system <NUM> includes an electronic device <NUM>, a network <NUM>, and a server device <NUM>.

The voice command providing system <NUM> of such a configuration transmits audio data, collected by the electronic device <NUM>, to the server device <NUM> through the network <NUM>. The server device <NUM> transmits analysis data of audio data to the electronic device <NUM>. The electronic device <NUM> registers specific audio data as a voice command on the basis of the received audio analysis data (for example, audio feature data, speech recognition data, function execution data (or function performance data), and so on). If audio data relating to a voice command is collected, the electronic device <NUM> performs a function (e.g. voice short-cut) for the voice command separately from the server device <NUM>. The above-mentioned voice command providing system <NUM> supports a prompt function call (for example, search, execution, activation, or deactivation) by registering specific audio data as an instruction in relation to a specific function execution of the electronic device <NUM>. Additionally, the voice command providing system <NUM> may establish a voice command architecture (or system) substantially by converting a recent frequently used function as a voice command according to a history of audio data that a user of the electronic device <NUM> inputs and managing the voice command. Additionally, the voice command providing system <NUM> may efficiently manage the voice command architecture (or system) by removing an unused voice command.

The network <NUM> supports a communication channel establishment between the electronic device <NUM> and the server device <NUM>. The network <NUM> may be prepared with a network device element capable of supporting a communication method of at least one communication module prepared in the electronic device <NUM>. For example, the network <NUM> may include a cellular communication related network device element (for example, a mobile base station, a mobile switching center, etc.), a Wireless Fidelity (WiFi) communication related network device element (for example, an Access Point (AP), a core network, etc.), and the like. The network <NUM> is not limited to a specific device element. The network <NUM> may be prepared with at least one device element that establishes at least one communication channel available for transmission of audio data and transmission of audio analysis data between the electronic device <NUM> and the server device <NUM>. According to an embodiment of the present disclosure, if a communication channel is established through a short-range wireless communication method or a direct communication method (e.g. a method of directly establishing a communication channel without passing through a base station), the network <NUM> may be omitted. An example of transmitting/receiving data on the basis of a form of including the network <NUM> is described below.

The server device <NUM> establishes a communication channel with the electronic device <NUM> through the network <NUM>. According to an embodiment of the present disclosure, the server device <NUM> establishes a data communication channel with the electronic device <NUM> in relation to the reception of audio data and the transmission of audio analysis data. According to an embodiment of the present disclosure, the server device <NUM> establishes a voice communication channel for receiving audio data from the electronic device <NUM> and a data communication channel for transmitting audio analysis data to the electronic device <NUM>. According to an embodiment of the present disclosure, the server device <NUM> manages and provides a voice command of the electronic device <NUM>. For example, the server device <NUM> may store information on voice commands for each of at least one electronic device <NUM> or each of at least one user. The server device <NUM> analyzes audio data that the electronic device <NUM> provides. The server device <NUM> may perform the deletion of voice commands performed on the electronic device <NUM> or the addition of a new voice command on the basis of audio analysis data. When a voice command update occurs, the server device <NUM> provides voice command update information to the electronic device <NUM>.

The electronic device <NUM> establishes a connection with the server device <NUM> through the network <NUM> and transmits audio data to the server device <NUM>. The electronic device <NUM> receives audio analysis data from the server device <NUM>. The electronic device <NUM> may generate new voice commands or remove a pre-generated voice command on the basis of the received audio analysis data. According to an embodiment of the present disclosure, the electronic device <NUM> may update a pre-generated voice command on the basis of the collected audio data. According to an embodiment of the present disclosure, the electronic device <NUM> may generate a specific voice command by an input event. According to an embodiment of the present disclosure, the electronic device <NUM> may receive a specific voice command from another electronic device or the server device <NUM> and store the received specific voice command. The electronic device <NUM> may update a stored voice command by using the collected audio data or the audio analysis data that the server device <NUM> provides. According to an embodiment of the present disclosure, the electronic device <NUM> may not update by at least one of the collected audio data and the audio analysis data in relation to a specific voice command. According to an embodiment of the present disclosure, the electronic device <NUM> manages a candidate recognition model relating to voice command generation in a voice command update operation. The voice command may be included in a recognition model (for example, a model including at least one of audio feature data and speech recognition data) and managed.

The above-mentioned electronic device <NUM> may include a communication interface <NUM>, an antenna <NUM>, an input module <NUM>, an audio module <NUM>, a display module <NUM>, a storage module <NUM>, a control module <NUM>, and a bus <NUM>.

The communication interface <NUM> establishes a communication channel with the network <NUM>. According to an embodiment of the present disclosure, the communication interface <NUM> establishes a communication channel with the network <NUM> according to a communication method such as <NUM>/<NUM>, and so on. According to an embodiment of the present disclosure, the communication interface <NUM> establishes a communication channel according to a WiFi communication method via the antenna <NUM>. The communication interface <NUM> may establish a communication channel with the server device <NUM> through the network <NUM>. In relation to this, the electronic device <NUM> may perform the connection of the server device <NUM> by using pre-stored or inputted address information. Herein, the address information, as information such as an Internet Protocol (IP) address and so on, may include identification information of the server device <NUM> registered in the network <NUM>.

According to an embodiment of the present disclosure, when performing a specific function (for example, a voice search function such as S-Voice of the Samsung Galaxy, a voice search function such as Siri of the Apple iPhone, etc.) of the electronic device <NUM>, the communication interface <NUM> may be activated automatically. Alternatively, when audio data is collected according to a specific function execution, the communication interface <NUM> may be activated. The communication interface <NUM> may transmit the collected audio data to the server device <NUM>. During this operation, the communication interface <NUM> may transmit identification information (for example, address information of the electronic device <NUM>, phone number information of the electronic device, hardware related information of the electronic device <NUM> such as Media Access Control (MAC) address, etc.) of the electronic device <NUM>.

According to an embodiment of the present disclosure, the communication interface <NUM> receives audio analysis data (for example, at least one of audio feature data, speech recognition data, and function execution data relating to transmitted audio data) from the server device <NUM>. The communication interface <NUM> delivers the received audio analysis data to at least one of the storage module <NUM> and the control module <NUM>. According to an embodiment of the present disclosure, the communication interface <NUM> receives at least one of a voice command and voice command update information from the server device <NUM>. The voice command and voice command update information that the communication interface <NUM> receives may be stored in the storage module <NUM> and the control module <NUM> or updated.

The input module <NUM> generates an input signal relating to the management of the electronic device <NUM>. For example, the input module <NUM> may include a physical key button (for example, a home key, a side key, a power key, etc.), a jog key, a keypad, and the like. The input module <NUM> may include a virtual keypad outputted to the display module <NUM> as an input device. The input module <NUM> may generate an input signal relating to the execution of a specific application (app) (for example, a voice search function, a voice command management function, etc.). According to an embodiment of the present disclosure, the input module <NUM> generates an input signal relating to a character input corresponding to a specific voice command. The input module <NUM> may generate an input signal for requesting the connection from the server device <NUM>, an input signal relating to a specific voice command reception from the server device <NUM>, and so on. The voice command may correspond to a recognition model (for example, a model including at least one of audio data, audio feature data, and speech recognition data).

The audio module <NUM> supports a function relating to the audio data output and audio data collection of the electronic device <NUM>. In relation to this, the audio module <NUM> includes a speaker for performing an audio data output and at least one microphone for supporting audio data collection. If a specific app (for example, a voice search function) of the electronic device <NUM> is executed, the audio module <NUM> may be activated automatically. According to an embodiment of the present disclosure, if power is supplied to the electronic device <NUM>, the audio module <NUM> is activated to perform audio data collection in real time or at predetermined periods (for example, an "always on" mode, which is a mode or function in which a voice search or collection function is performed as background processing). The audio data that the audio module <NUM> collects may be stored in the storage module <NUM> or may be transmitted to the server device <NUM> through the communication interface <NUM> according to a control of the control module <NUM>.

According to an embodiment of the present disclosure, when collecting specific audio data, the audio module <NUM> outputs response information on the collected audio data. For example, as a speech recognition result for the collected audio data, the audio module <NUM> outputs information that describes the type of a function performed based on the speech recognized result and information relating to the generation of a voice command corresponding to the collected audio data or the removal of a specific voice command. The audio data that the audio module <NUM> collects, as mentioned above, is transmitted to the server device <NUM> and is speech-recognized by the electronic device <NUM>. For example, when the electronic device <NUM> is unable to connect to the server device <NUM>, it performs speech recognition on the collected audio data.

According to an embodiment of the present disclosure, the audio module <NUM> outputs information relating to voice command processing, information relating to a function execution according to voice command processing, and information relating to voice command update (for example, a generation of a new voice command or a change or deletion of an existing voice command). The above-mentioned information output of the audio module <NUM> may be omitted according to a user setting.

The display module <NUM> provides at least one screen interface (or a user interface) relating to the management of the electronic device <NUM>. For example, the display module <NUM> may output a lock screen, a turn-off screen in a sleep mode state, a standby screen, a menu screen, etc. According to an embodiment of the present disclosure, the display module <NUM> outputs a screen according to a specific function execution (for example, a voice search function). The display module <NUM> outputs a screen relating to a voice search function when the voice search function is executed. When a microphone collects specific audio data, the display module <NUM> may output a screen relating to the transmission to the server device <NUM> of the collected audio data, a screen relating to audio analysis data reception from the server device <NUM>, and a screen relating to the update of a voice command by the received audio analysis data. When audio data corresponding to a specific voice command is collected, the display module <NUM> outputs a function execution screen corresponding to a corresponding voice command.

According to an embodiment of the present disclosure, the display module <NUM> outputs a virtual keypad screen relating to a character input corresponding to a voice command. The display module <NUM> may output a list screen relating to stored voice commands and a screen relating to voice command editing. The list screen relating to voice commands may include at least one voice command and information on function items to be performed when each voice command is inputted. The display module <NUM> may output an information screen for candidate recognition models relating to voice commands. Information on candidate recognition models may include audio feature data (for example, data such as audio frequency characteristics or time domain characteristics), speech recognition data (for example, data such as characters obtained through speech recognition), function execution data (for example, a specific function item executed according to audio data), and information on a weight (for example, the number of times that the same audio data is collected).

The storage module <NUM> stores information relating to the management of the electronic device <NUM>, for example, an operating system, app data according to a specific function execution, and the like. The storage module <NUM> may store data relating to a voice command providing function. For example, the storage module <NUM> may store audio data that the microphone of the audio module <NUM> collects. The storage module <NUM> may store audio analysis data that the server device <NUM> provides in correspondence to the transmitted audio data. In relation to this, the storage module <NUM> may store a voice command generation table. The storage module <NUM> may store an app relating to a voice search function. The storage module <NUM> may store various apps to be executed by a voice command. The voice command generation table stored in the storage module <NUM> may be updated by collected audio data, received audio analysis data, and update information that the server device <NUM> provides.

The bus <NUM> may support a signal flow between each of components of the electronic device <NUM>. For example, the bus <NUM> may deliver audio data that the audio module <NUM> collects to the control module <NUM> or may deliver the audio data to the communication interface <NUM> according to a control of the control module <NUM>. Additionally, the bus <NUM> may deliver audio analysis data, received through the communication interface <NUM>, to at least one of the storage module <NUM> and the control module <NUM>.

The control module <NUM> may perform the transmission and processing of data relating to the management of the electronic device <NUM>. The control module <NUM> may perform the transmission and processing of a control signal relating to the management control of the electronic device <NUM>. In relation to a voice command providing function support, the control module <NUM> may be implemented as shown in <FIG>.

<FIG> is a block diagram of the control module <NUM> and the storage module <NUM> according to an embodiment of the present disclosure.

Referring to <FIG>, the storage module <NUM> of the electronic device <NUM> according to an embodiment of the present disclosure includes a voice command generation table <NUM>, a candidate recognition model <NUM>, a recognition model <NUM>, and a voice command <NUM>.

The voice command generation table <NUM> includes audio feature data <NUM>, speech recognition data <NUM>, function execution data <NUM>, and a weight <NUM>. The audio feature data <NUM>, the speech recognition data <NUM>, and the function execution data <NUM> included in the voice command generation table <NUM> may be collected based on audio analysis data that the server device <NUM> provides. According to an embodiment of the present disclosure, the audio feature data <NUM> and the speech recognition data <NUM> may be provided by an analysis of the electronic device <NUM>. In relation to this, the electronic device <NUM> may further include a speech recognition algorithm (for example, an algorithm for performing speech recognition on audio data to output the audio data as text) and an audio feature analysis module (for example, a module for extracting the acoustic feature of audio data, for example, spectrum information of audio data). Additionally, the control module <NUM> of the electronic device <NUM> may perform a control to generate and store function execution data <NUM> by mapping a speech recognition result into a function executed on the electronic device <NUM>.

The audio feature data <NUM> may be data on an audio feature (including a frequency or time axis area feature value such as Mel Frequency Cepstral Coefficients (MFCC), Linear Prediction Coefficients (LPC), Line Spectral Pairs (LSP), autocorrelation, power spectrum, pitch, etc.) for specific audio data (for example, audio data that a user speaks). The audio feature data <NUM> may include at least one audio feature (or a sound wave frequency feature) included in each of at least one word, word-phrase, clause, phrase, sentence, paragraph, and page. The audio feature data <NUM> may be data into which an audio feature is mapped for each audio data. The audio feature data <NUM> on each audio feature may be received from the server device <NUM> and stored. Alternatively, the audio feature data <NUM> may be detected by the electronic device <NUM> and stored.

The speech recognition data <NUM> may be speech recognized text information. The control module <NUM> of the electronic device <NUM> transmits specific audio data, collected when a user speaks, to the server device <NUM> through the communication interface <NUM>. The control module <NUM> receives a speech recognition result (for example, text) corresponding to specific audio data and stores it in the storage module <NUM>. The speech recognition data <NUM> may include a text mapped into specific audio data.

The function execution data <NUM> includes function execution information of the electronic device <NUM> mapped into specific audio data. According to an embodiment of the present disclosure, the function execution data <NUM> includes specific function execution information of the electronic device <NUM>, which is obtained by analyzing audio data that the server device <NUM> receives from the electronic device <NUM>. According to an embodiment of the present disclosure, the function execution data <NUM> is data that the electronic device <NUM> maps concerning specific function execution information executed by the collected audio data.

The weight <NUM> is defined based on information on the number of times that audio data relating to each of the audio feature data <NUM>, the speech recognition data <NUM>, and the function execution data <NUM> are collected and a collection time, the frequency of use of a specific app executed by audio data, and a usage time of a specific app. According to an embodiment of the present disclosure, the weight <NUM> may have a relatively large value corresponding to the number of times that each audio data is collected. According to an embodiment of the present disclosure, the weight <NUM> may have a relatively larger value for a recent time at which audio data is collected. According to an embodiment of the present disclosure, the weight <NUM> may have a relatively large value corresponding to the number of times that audio data is collected within a set predetermined period. The weight <NUM> may be used for generating the candidate recognition module <NUM> or the recognition model <NUM> and generating the voice command <NUM> according thereto.

For example, the collected audio data may be the following information spoken in English: "Hello, how are you doing?" (first collected audio data <NUM>), "Show me the nearest restaurants. " (second collected audio data <NUM>), "Is it going to rain tomorrow?" (third collected audio data <NUM>), "How do I get to the nearest subway station?" (fourth collected audio data <NUM>), "Show me the nearest restaurants. " (fifth collected audio data <NUM>), "What is the stock price of Samsung Electronics?" (sixth collected audio data <NUM>), "Is it going to rain tomorrow?" (seventh collected audio data <NUM>), "Set an alarm for <NUM> am on Monday. " (eighth collected audio data <NUM>), "I feel like having an ice-cream. " (ninth collected audio data <NUM>), and "Show me the nearest restaurants. " (tenth collected audio data <NUM>). The control module <NUM> may configure a voice command table as shown in Table <NUM> below with respect to the audio data. Table <NUM> illustrates a table of the audio feature data <NUM>, the speech recognition data <NUM>, the function execution data <NUM>, and the weight <NUM> for collected audio data.

The candidate recognition model <NUM> may be candidate models prepared before the generation of the recognition model <NUM>. When specific audio data satisfies a specific predetermined specific condition (for example, at least one of the predetermined number of collections, collection within a specific predetermined specific time, the frequency of use of a specific app executed by audio data, and a usage time of a specific app), the candidate recognition model <NUM>, for example, may be generated based on the audio feature data <NUM>, the speech recognition data <NUM>, and the function execution data <NUM> corresponding to corresponding audio data. When the specific audio data does not satisfy a specific predetermined specific condition (for example, at least one of when there is no audio data collection during a specific predetermined specific period and when the frequency of use of a specific app executed by audio data or a usage time is less than a predetermined value), the candidate recognition model <NUM> relating to the specific audio data may be removed. According to an embodiment of the present disclosure, the processing operation for the candidate module <NUM> in a voice command providing function of the electronic device <NUM> may be omitted. In this case, the candidate recognition module <NUM> may not be stored in the storage module <NUM>.

The recognition module <NUM> is comparison information corresponding to the voice command <NUM>. The recognition module <NUM> may include information mapped into each voice command <NUM>. According to an embodiment of the present disclosure, the recognition model <NUM> may include at least one of the audio feature data <NUM> and the speech recognition data <NUM> corresponding to the specific voice command <NUM>. According to an embodiment of the present disclosure, when audio data satisfying a predetermined specific condition is collected, the recognition model <NUM> is generated based on the voice command generation table <NUM> corresponding to the corresponding audio data. According to an embodiment of the present disclosure, if the recognition model <NUM> does not satisfy a specific predetermined specific condition, it is removed. During the management of the candidate recognition model <NUM>, some of the candidate recognition models <NUM> that satisfy a specific predetermined specific condition among the candidate recognition models <NUM> may be converted into the recognition model <NUM>.

The voice command <NUM> may be mapped into each recognition model <NUM> and stored. The voice command <NUM> may be mapped into the function execution data <NUM>. When a new recognition model <NUM> is generated, the new voice command <NUM> may be added. When the recognition model <NUM> is removed, the voice command <NUM> may be removed. At least one voice command <NUM> among the voice commands <NUM> may not be removed and may be maintained according to a setting regardless of an additional defined condition. The voice command <NUM> may be a command used for a voice shortcut function. For example, if audio data corresponding to the voice command <NUM> is collected, the control module <NUM> may perform a control to execute a function mapped into the voice command <NUM>.

The control module <NUM> includes an audio data processing module <NUM>, a recognition model processing module <NUM>, and a voice command processing module <NUM>.

The audio data processing module <NUM> controls the collection and processing of audio data. For example, the audio data processing module <NUM> may perform a control to activate a microphone when a voice search function is activated. The audio data processing module <NUM> may perform a control to transmit audio data that a microphone collects to the server device <NUM>. The audio data processing module <NUM> may control an audio analysis data reception corresponding to audio data that the server device <NUM> provides. The audio data processing module <NUM> may extract the audio feature data <NUM>, the speech recognition data <NUM>, and the function execution data <NUM> from the received audio analysis data. The audio data processing module <NUM> may update each data stored in the storage module <NUM> on the basis of the extracted data.

According to an embodiment of the present disclosure, the audio data processing module <NUM> maps the audio feature data <NUM> in relation to the collected audio data. For example, when receiving the audio feature data <NUM> for audio data such as "camera," "weather," "news," "stock information," "please call mom," and so on from the server device <NUM>, the audio data processing module <NUM> maps the received audio feature data <NUM> into each audio data and stores them.

According to an embodiment of the present disclosure, the audio data processing module <NUM> maps the audio feature data <NUM> for one word of audio data such as "camera" and stores it. According to an embodiment of the present disclosure, the audio data processing module <NUM> maps audio feature data for each word of a three-word phrase such as "please call mother" into a corresponding character and stores them as one audio feature data <NUM>. The audio feature data <NUM> may have a predetermined error range by a situation or environment in which a user speaks or a user's physical change. For example, an audio feature for "please call mother" may have a predetermined difference value according to a speaking place (for example, an open area, a confined place, a place having noise of more than a predetermined value, a place with a different barometric pressure, and the like). The audio data processing module <NUM> may map audio features within a predetermined error range into audio features for one audio data and may store a mapping result as the audio feature data <NUM>. According to an embodiment of the present disclosure, even if audio data is configured with a similar sentence, the server device <NUM> or the electronic device <NUM> may classify a sentence including another word as different audio features. For example, the audio data processing module <NUM> may classify audio data corresponding to "please call mother" and audio data corresponding to "please call sister" as different audio feature data and store them.

According to an embodiment of the present disclosure, the audio data processing module <NUM> extracts speech recognition data and audio feature data for the received audio data by using a pre-stored speech recognition algorithm and an audio feature extraction module according to a specific situation (for example, a voice search function is performed in a situation where the electronic device <NUM> is unable to establish a communication channel with the server device <NUM>) or predetermined schedule information. In relation to this, the audio data processing module <NUM> may include a speech recognizer or speech recognition processor relating to audio data speech recognition. For example, the audio data processing module <NUM> may include a speech recognizer (for example, a voice command extractor) for determining whether audio data relates to a voice shortcut function (for example, S-Voice, Siri, etc.) or what function is to be executed in a voice shortcut function execution state. The audio data processing module <NUM> may update the speech recognition data <NUM> and the audio feature data <NUM> stored in the storage module <NUM> by using the extracted data. The audio data processing module <NUM> may check a function execution state after speech recognition and may update the function execution data <NUM> on the basis of the checked function execution information.

The recognition model processing module <NUM> manages at least one of the candidate recognition model <NUM> and the recognition model <NUM>. According to an embodiment of the present disclosure, the recognition model processing module <NUM> performs a control to generate a new candidate recognition model <NUM> on the basis of the weight <NUM> of analysis data of audio data that the electronic device <NUM> collects. The recognition model processing module <NUM> may change the candidate recognition model <NUM> into the recognition model <NUM> according to a specific condition after changing a score of the candidate recognition model <NUM> in relation to the collected audio data. The recognition model processing module <NUM> may generate a new recognition model <NUM> by changing the candidate recognition model <NUM> into the recognition model <NUM> in correspondence to a specific condition. When a new recognition model <NUM> is generated, the recognition model processing module <NUM> may generate a voice command <NUM> corresponding to the recognition model <NUM>. Alternatively, a voice command <NUM> may be generated automatically based on analysis data of audio data contributed to the generation of the recognition model <NUM>. According to an embodiment of the present disclosure, the recognition model processing module <NUM> may remove the candidate recognition model <NUM> according to a specific condition. Additionally, when removing the specific recognition model <NUM>, the recognition model processing module <NUM> may remove a voice command <NUM> corresponding thereto.

According to an embodiment of the present disclosure, when the electronic device <NUM> manages a large vocabulary recognition processor (or device), speech recognition performance is possible based on a voice command list. Correspondingly, the recognition model processing module <NUM> may be integrated into the voice command processing module <NUM> so that they may be configured as one module. Similarly, the electronic device <NUM> may support voice command processing on the basis of at least one of a voice command <NUM> and a recognition model <NUM>. For example, the electronic device <NUM> may map audio data and a voice command <NUM> without recognition model management and may support voice command processing on the basis of the mapping result. Alternatively, the electronic device <NUM> may support recognition model based function processing by mapping audio data into a recognition model <NUM> and mapping the mapped recognition model <NUM> into a specific function.

The voice command processing module <NUM> determines whether audio data that the electronic device <NUM> collects corresponds to a voice command <NUM>. If the collected audio data corresponds to a voice command <NUM>, the voice command processing module <NUM> performs a control to execute a specific function mapped into the voice command <NUM>. During this operation, the voice command processing module <NUM> may determine a similarity by mutually comparing the recognition model <NUM> and the collected audio data. The voice command processing module <NUM> may detect the recognition model <NUM> of which similarity to the collected audio data is more than a predetermined value and may check a function item of the voice command <NUM> mapped into the recognition model <NUM>.

As mentioned above, the electronic device <NUM> according to an embodiment of the present disclosure may collect audio analysis data of audio data obtained in real time, at a predetermined period, or during specific app execution. The electronic device <NUM> may generate a new voice command <NUM> or update a voice command <NUM> on the basis of the audio analysis data. The electronic device <NUM> may support the generation and management of a voice command <NUM> corresponding to an app that a user uses frequently or heavily or a heavily used function.

<FIG> is a block diagram of a server device <NUM> according to an embodiment of the present disclosure.

Referring to <FIG>, a server device <NUM> according to an embodiment of the present disclosure includes a server communication module <NUM>, an antenna <NUM>, a server storage module <NUM>, and a server control module <NUM>.

The server communication module <NUM> may establish a communication channel with the network <NUM> and may establish a communication channel in correspondence to the connection of the electronic device <NUM>. The server communication module <NUM> may receive, from the electronic device <NUM>, audio data corresponding to audio data that a user speaks. The collected audio data may be delivered to at least one of the server storage module <NUM> and the server control module <NUM>. The server communication module <NUM> may transmit audio analysis data to the electronic device <NUM> according to a control of the server control module <NUM>. According to an embodiment of the present disclosure, the server communication module <NUM> may transmit, to the electronic device <NUM>, a voice command <NUM> and voice command update information relating to the specific electronic device <NUM>.

The server storage module <NUM> stores a program and data relating to a voice command providing function. For example, the server storage module <NUM> may store an Automatic Speech Recognition (ASR) algorithm <NUM> and a voice command database <NUM>. The ASR <NUM> may include an algorithm supporting speech recognition for audio data that the electronic device <NUM> transmits. The voice command database <NUM> may include information on voice commands <NUM> for each electronic device. According to an embodiment of the present disclosure, the server storage module <NUM> may store an audio data analysis program. The audio data analysis program may include, for example, an audio data collection routine (for example, programmable information designed to execute a specific function, as including an instruction or an instruction set, a function, syntax, a template, class, etc.), a data processing routine, and a processing data transmission routine.

The server control module <NUM> controls the performance of at least one of the connection of the electronic device <NUM>, the analysis of audio data and analysis data provision, and voice command management. The server control module <NUM> may include a voice command related block <NUM> as shown in <FIG>. According to various embodiments of the present disclosure, the voice command related block <NUM> may be established in the electronic device <NUM> and managed.

<FIG> is a block diagram relating to voice command processing according to an embodiment of the present disclosure.

Referring to <FIG>, the voice command related block <NUM> includes a voice search manager <NUM> and a voice command manager <NUM>. At least one block in the voice command related block <NUM> may be included in a configuration of the server control module <NUM>.

The voice search manager <NUM> includes an audio data collection module <NUM>, a data processing module <NUM>, and a processing data transmission module <NUM>.

The audio data collection module <NUM> performs a control so that the server communication module <NUM> establishes a communication channel with the electronic device <NUM>. For example, when the server communication module <NUM> receives a connection request of the electronic device <NUM>, the audio data collection module <NUM> controls communication connection processing in correspondence thereto. The audio data collection module <NUM> collects audio data that the electronic device <NUM> provides in addition to the identification information of the electronic device <NUM>. The audio data collection module <NUM> delivers audio data to the data processing module <NUM>.

The data processing module <NUM> performs analysis on the audio data that the audio data collection module <NUM> delivers. The data processing module <NUM> extracts audio feature data, speech recognition data, and function execution data from the collected audio data. In relation to this, the data processing module <NUM> may include an analysis module for analyzing the frequency spectrum of audio data. The data processing module <NUM> may manage the ASR algorithm <NUM> stored in the storage module <NUM> in relation to the speech recognition data extraction. The data processing module <NUM> may collect text information on audio data through the management of the ASR <NUM>.

The data processing module <NUM> may manage a Natural Language Understanding (NLU) module in relation to function execution data extraction. The NLU module may extract natural languages from audio data and performance requested function execution information by combining the extracted natural languages. In relation to this, the data processing module <NUM> may check type information of a function installed on the electronic device <NUM> by comparing the identification information of the electronic device <NUM> and the type information of a pre-stored installation function for each electronic device. Alternatively, the server device <NUM> may request function type information from the electronic device <NUM> to provide it to the data processing module <NUM>. The data processing module <NUM> may extract function execution information on the basis of an analysis result of natural languages included in audio data and pre-stored device specific function type information (for example, a database where at least one natural language is mapped into each function is mapped). For example, the data processing module <NUM> may extract function execution information corresponding to the collected audio data by checking an intent for query and specific various conditions, which are identified through an analysis of natural languages. The extracted function execution information is provided to the processing data transmission module <NUM>.

The processing data transmission module <NUM> delivers audio feature data, speech recognition data, and function execution information that the data processing module <NUM> provides, to the electronic device <NUM> providing audio data. In relation to this, the processing data transmission module <NUM> may receive information on connection with the electronic device <NUM> from the audio data collection module <NUM>. According to an embodiment of the present disclosure, the processing data transmission module <NUM> delivers audio feature data, speech recognition data, and function execution information to the voice command manager <NUM>.

The voice command manager <NUM> includes a processing data collection module <NUM> and a voice command management module <NUM>.

The processing data collection module <NUM> receives audio feature data, speech recognition data, and function execution information that the voice search manager <NUM> provides. The processing data collection module <NUM> may store the received audio feature data, speech recognition data, and function execution information in the server storage module <NUM> of <FIG> so that they may be identified by each electronic device <NUM> or each user. The processing data collection module <NUM> of <FIG> may determine a weight for the collected audio feature data, speech recognition data, and function execution information. For example, the processing data collection module <NUM> may determine a weight (or score) for each data on the basis of information on the number of times that specific audio data is received and a time at which specific audio data is received. According to an embodiment of the present disclosure, if audio data is received relatively frequently within a predetermined period, the processing data collection module <NUM> may set a relatively high weight. Additionally, the processing data collection module <NUM> may increase (or decrease or maintain) a predetermined weight according to the number of receptions within a predetermined period. According to an embodiment of the present disclosure, the processing data collection module <NUM> may set a relatively high weight of audio data that is received relatively recently based on the current. Alternatively, the processing data collection module <NUM> may assign a predetermined weight (for example, the size of a minus weight is increased, which is interpreted to reduce the importance as a time interval increases).

If a set weight is greater than a specific condition, the processing data collection module <NUM> generates a voice command <NUM> of audio data corresponding to a corresponding weight. The processing data collection module <NUM> delivers the generated voice command <NUM> to the voice command management module <NUM>. According to an embodiment of the present disclosure, the processing data collection module <NUM> delivers weight change information to the voice command management module <NUM>. According to an embodiment of the present disclosure, the processing data collection module <NUM> may generate or update a voice command <NUM> according to the generation and management methods of the candidate recognition model <NUM> and the recognition model <NUM> described with the electronic device <NUM>.

The voice command management module <NUM> receives the generated voice command <NUM> from the processing data collection module <NUM>. The voice command management module <NUM> may receive information on weight change from the processing data collection module <NUM>. The voice command management module <NUM> manages the voice command database <NUM> on the basis of the received voice command <NUM> and weight change information. For example, the voice command management module <NUM> may add a newly generated voice command <NUM> to the voice command database <NUM> for each electronic device. Alternatively, if the weight of a voice command <NUM> relating to specific audio data is changed and becomes less than a specific condition, the voice command management module <NUM> may remove the voice command <NUM> from the voice command database <NUM>. When a new voice command <NUM> is generated in relation to the specific electronic device <NUM>, the voice command management module <NUM> may deliver the newly generated voice command information to the electronic device <NUM>. Additionally, when an existing voice command <NUM> is removed, the voice command management module <NUM> may deliver information on the removal situation as voice command update information to the electronic device <NUM>.

According to an embodiment of the present disclosure, functions of the processing data collection module <NUM> and the voice command management module <NUM> may be performed on the electronic device <NUM>, except for a speech recognition function. For example, if a set weight (or score) is greater than a specific condition, the electronic device <NUM> generates a voice command <NUM> of audio data corresponding to a corresponding weight. The electronic device <NUM> may generate or update a voice command <NUM> according to the generation and management methods of the candidate recognition model <NUM> and the recognition model <NUM>. When receiving information on weight change, the electronic device <NUM> may manage the storage of a voice command <NUM> on the basis of the received voice command <NUM> and weight change information. For example, the electronic device <NUM> may add a newly generated voice command <NUM> to a database storing a voice command <NUM>. When a voice command <NUM> is removed from the database, the electronic device <NUM> may output an alarm for this through the audio module <NUM> or the display module <NUM>.

According to an embodiment of the present disclosure, the electronic device <NUM> may include an audio module for collecting audio data and a control module for updating a voice command <NUM> set to execute a specific function of the electronic device <NUM> through voice when specific audio data is collected, on the basis of at least one of audio feature data of the audio data, speech recognition data obtained by speech recognition of the audio data, and function execution data executed in relation to the audio data.

According to an embodiment of the present disclosure, the control module <NUM> updates the weight of at least one of pre-stored audio feature data, the speech recognition data, and the function execution data in correspondence to the audio data collection and generates the voice command <NUM> if the weight satisfies a specific condition.

According to an embodiment of the present disclosure, if the weight satisfies the specific condition, the control module <NUM> may generate a candidate recognition model <NUM> and if the weight of the candidate recognition model <NUM> satisfies a specific condition in relation to the audio data, may change the candidate recognition model <NUM> into a recognition model <NUM> corresponding to the voice command <NUM>.

According to an embodiment of the present disclosure, the control module <NUM> may register a new voice command <NUM> corresponding to the recognition model <NUM> changed from the candidate recognition model <NUM>.

According to an embodiment of the present disclosure, the control module <NUM> may update the weight of at least one of a candidate recognition model <NUM> and recognition model <NUM> having no relation with the collected audio data and may remove at least one of a specific candidate recognition model <NUM> and a specific recognition model <NUM> in correspondence to the updated weight.

According to an embodiment of the present disclosure, the control module <NUM> may remove a registered voice command <NUM> in correspondence to the recognition model <NUM> removal.

According to an embodiment of the present disclosure, the control module <NUM> may control the display of at least one of the audio feature data <NUM>, the speech recognition data <NUM>, the function execution information <NUM>, and the voice command <NUM>.

According to an embodiment of the present disclosure, the control module <NUM> may control at least one of the weight update of the audio feature data 51on the basis of the frequency of occurrence of the audio feature data, the weight update of the speech recognition data <NUM> on the basis of the frequency of occurrence of the speech recognition data, and the weight update of the function execution data <NUM> using at least one of whether to execute a specific function of the function execution data <NUM> and a specific function execution duration time of the function execution data <NUM>.

According to an embodiment of the present disclosure, the control module <NUM> may control a communication interface <NUM> for transmitting the audio data to a server device <NUM> and receiving, from the server device <NUM>, at last one of the audio feature data <NUM>, the speech recognition data <NUM>, and the function execution data <NUM>.

According to an embodiment of the present disclosure, the control module <NUM> may control a communication interface <NUM> for receiving, from the server device <NUM>, a voice command <NUM> generated based on at least one of the audio feature data <NUM>, the speech recognition data <NUM>, and the function execution data <NUM>.

<FIG> is a flowchart of a voice command providing method according to an embodiment of the present disclosure.

Referring to <FIG>, a voice command providing method according to an embodiment of the present disclosure allows the control module <NUM> of the electronic device <NUM> to perform a management or standby function in operation <NUM>. For example, the control module <NUM> may output a standby screen, a lock screen, or a specific function execution screen according to a set schedule or an input control.

In operation <NUM>, the control module <NUM> checks whether there is an event occurrence relating to voice command support function activation. For example, the control module <NUM> may check whether there is a setting relating to a voice command support function. Alternatively, the control module <NUM> may check whether an input event relating to a voice command support function occurs.

If there is a voice command support function activation setting or a voice command support function activation event occurs in operation <NUM>, the control module <NUM> checks whether there is an event occurrence relating to voice search function activation in operation <NUM>. For example, the control module <NUM> may check the occurrence of an event for selecting a function icon relating to a voice search function and an event for requesting an app execution relating to a voice search function.

According to an embodiment of the present disclosure, when a voice search function is set to be provided by default, the control module <NUM> checks a corresponding setting. If there is no voice command support function activation setting or event occurrence in operation <NUM> or if there is no voice search function activation related setting or event occurrence in operation <NUM>, the control module <NUM> controls the function execution corresponding to the occurred event type in operation <NUM>. For example, the control module <NUM> may execute a specific app according to an event type or may maintain a previous state (for example, a sleep mode state, a lock screen state, a standby screen output state, etc.) of the electronic device <NUM>.

If there is a voice search function activation related setting or event in operation <NUM>, the control module <NUM> controls audio data collection and transmission in operation <NUM>. The control module <NUM> may activate a microphone of the audio module <NUM> and collect audio data.

According to an embodiment of the present disclosure, a voice search function may be performed separately from a voice shortcut function. For example, the control module <NUM> may drive a specific search recognizer (for example, a search processor) in relation to voice search function execution. The control module <NUM> may drive a specific function performer (for example, an S-Voice processor) in relation to a voice shortcut function. Alternatively, according to an embodiment of the present disclosure, the control module <NUM> of the electronic device <NUM> may simultaneously process a voice search function and a voice shortcut function through a function performer. If audio data is collected, the control module <NUM> may transmit corresponding audio data to the server device <NUM>. In relation to this, the control module <NUM> may establish a communication channel with the server device <NUM>. The server device <NUM> may generate audio analysis data for the received audio data and transmit it to the electronic device <NUM>.

In operation <NUM>, the control module <NUM> receives audio analysis data including at least one of speech recognition data <NUM>, audio feature data <NUM>, and function execution data <NUM>. In operation <NUM>, the control module <NUM> may perform voice command processing on the basis of the received audio analysis data. According to an embodiment of the present disclosure, the control module <NUM> may update the weights of the speech recognition data <NUM>, the audio feature data <NUM>, and the function execution data <NUM>, which are stored in the storage module <NUM>, on the basis of the received audio analysis data. When audio analysis data includes analysis data for a plurality of words or word-phrases during this operation, the control module <NUM> may update (for example, increase a weight count) the weight of at least one of the audio feature data <NUM> and the speech recognition data <NUM> including a plurality of words or word-phrases. Alternatively, the control module <NUM> may update the weight of at least one of the speech recognition data <NUM> and audio feature data <NUM> corresponding to at least part of a plurality of words. Alternatively, the control module <NUM> may update the weight of at least one of the speech recognition data <NUM> and audio feature data <NUM> corresponding to each of a plurality of words.

According to an embodiment of the present disclosure, in the case of the audio feature data <NUM>, the same information may have different values according to a collection environment of audio data. For example, first information (for example, audio data corresponding to "camera") collected from a noisy place and the first information collected from a less noisy place may have audio feature data of different characteristics. The control module <NUM> may apply an error caused by a collection environment during the update operation of the audio feature data <NUM>. Alternatively, the control module <NUM> may store and manage audio data as the audio feature data <NUM> of a different characteristic. According to an embodiment of the present disclosure, during a speech recognition operation, the same value may be extracted based on a noise canceling operation and an estimation operation through a similarity comparison of a speech recognition database. The speech recognition data <NUM> may be collected as the same value with respect to the same audio data (for example, audio data corresponding to "camera").

According to an embodiment of the present disclosure, the control module <NUM> may update the weight of the function execution data <NUM> stored in the storage module <NUM> on the basis of function execution data <NUM> included in the received audio analysis data. Alternatively, the control module <NUM> may update (for example, assign a high weight or increase a weight as an execution duration time increases) the weight of the function execution data <NUM> on the basis of the execution and execution duration time of an app or corresponding function executed by function execution data <NUM>.

If at least one of the audio feature data <NUM>, speech recognition data <NUM>, and function execution data <NUM> satisfies a specific condition (for example, a condition that it is greater than a predetermined weight set for generating a voice command <NUM>), the control module <NUM> generates and may manage a voice command <NUM> for corresponding audio data.

According to an embodiment of the present disclosure, if the server device <NUM> is designed in a form of providing a voice command, the control module <NUM> may directly receive voice command update information from the server device <NUM>. In this case, the control module <NUM> may perform the update of the voice command <NUM> in the storage module <NUM> by using the voice command update information received from the server device <NUM>.

In operation <NUM>, the control module <NUM> checks whether there is a function termination related event occurrence. If the function termination related event occurs in operation <NUM>, the control module <NUM> may control at least one function termination of a voice search function and a voice command support function. If there is no function termination related event occurrence in operation <NUM>, the control module <NUM> returns to operation <NUM> and performs the subsequent operations again. The control module <NUM> may support the collection of audio data that a user uses frequently, and the command processing according thereto by managing a voice search function continuously (for example, an "always on" mode).

Referring to <FIG>, the voice command providing method performs the management or standby function of the control module <NUM> in operation <NUM>. For example, the control module <NUM> may perform a control to maintain the electronic device <NUM> in a sleep mode or output a standby screen. The control module <NUM> may provide a setting screen relating to voice search function activation or output a voice search function related function icon or menu to the display module <NUM>. Alternatively, the control module <NUM> may allocate a function button (for example, a home key) relating to voice search function execution.

In operation <NUM>, the control module <NUM> checks whether there is a voice search function activation related event occurrence or setting. If there is no voice search function activation related setting or event occurrence, the control module <NUM> proceeds to operation <NUM> and controls a function execution corresponding to an occurred event type. For example, the control module <NUM> may control the execution of a sound source playback function, a broadcast reception function, and so on according to the event type. Alternatively, if there is no event occurrence, the control module <NUM> may perform a control to maintain a function management state (for example, a sleep mode state, a specific function screen output state, and a state in which a specific function is preformed in background) of operation <NUM>.

If an event relating to voice search function activation occurs in operation <NUM>, the control module <NUM> performs audio data collection and transmission in operation <NUM>. In operation <NUM>, the control module <NUM> receives speech recognition data, audio feature data, and function execution data from the server device <NUM>.

In operation <NUM>, the control module <NUM> checks whether there is a candidate recognition model <NUM> relating to the received data. If there is a candidate recognition model <NUM> in operation <NUM>, the control module <NUM> may perform candidate recognition model test and voice command processing in operation <NUM>. For example, the control module <NUM> may perform the weight update of the candidate recognition model <NUM>. Additionally, the control module <NUM> may perform new voice command generation according to the weight update of the candidate recognition model <NUM>.

If there is no candidate recognition model <NUM> in operation <NUM>, the control module <NUM> checks whether a candidate recognition modeling condition is satisfied in operation <NUM>. For example, the control module <NUM> may check whether at least one of the audio feature data <NUM>, speech recognition data <NUM>, and function execution data <NUM> updated by audio analysis data satisfies a specific condition or they all satisfy a specific condition. If the candidate recognition modeling condition is satisfied in operation <NUM>, the control module <NUM> performs candidate recognition model generation in operation <NUM>. If the candidate recognition modeling condition is not satisfied in operation <NUM>, the control module <NUM> skips operation <NUM>.

In operation <NUM>, the control module <NUM> checks whether there is an event occurrence relating to function termination. If the function termination related event occurs in operation <NUM>, the control module <NUM> controls voice search function termination. If there is no function termination related event occurrence in operation <NUM>, the control module <NUM> returns to operation <NUM> and performs the subsequent operations again.

<FIG> is a flowchart of a voice command provision related candidate recognition model processing method according to an embodiment of the present disclosure.

Referring to <FIG>, in relation to a candidate recognition model processing method, when audio data is collected, the control module <NUM> performs a candidate recognition model test in operation <NUM>. For example, the control module <NUM> may perform the update of the weight (or score) of audio feature data <NUM>, speech recognition data <NUM>, and function execution data <NUM> stored in the storage module <NUM> on the basis of the received audio analysis data. During this operation, the control module <NUM> may perform the weight update of at least one of the audio feature data <NUM>, speech recognition data <NUM>, and function execution data <NUM> relating to the specific candidate recognition model <NUM>. During this operation, the control module <NUM> may reduce the weight of another candidate recognition model <NUM> in correspondence to a weight increase of the specific candidate recognition model <NUM>.

In operation <NUM>, the control module <NUM> checks whether a recognition model or voice command generation condition is satisfied. For example, when the weight of the specific candidate recognition model <NUM> is greater than a predetermined value, the control module <NUM> changes a corresponding candidate recognition model <NUM> into a recognition model <NUM>. If the recognition model <NUM> or voice command generation condition is satisfied in operation <NUM>, the control module <NUM> changes the candidate recognition model <NUM> into the recognition model <NUM> in operation <NUM>. Additionally, the control module <NUM> may control the voice command generation or registration corresponding to the recognition model generation or registration. For example, the control module <NUM> may generate the voice command <NUM> on the basis of the speech recognition data <NUM> and function execution data <NUM> corresponding to the changed recognition model <NUM>.

If the voice command generation condition is not satisfied in operation <NUM>, the control module <NUM> checks whether a candidate recognition model removal condition is satisfied in operation <NUM>. If the candidate recognition model removal condition is satisfied in operation <NUM>, the control module <NUM> removes the candidate recognition model <NUM>. If the candidate recognition model removal condition is not satisfied in operation <NUM>, the control module <NUM> skips operation <NUM>. For example, if a weight change of a candidate recognition model <NUM> is between a generation condition and a removal condition, the control module <NUM> may perform the weight update of a specific candidate recognition model <NUM>.

According to an embodiment of the present disclosure, the control module <NUM> performs model removal on the recognition model <NUM> additionally or substitutionally in addition to the candidate recognition model <NUM>. For example, the control module <NUM> removes the recognition model <NUM> stored in the storage module <NUM>, for example, the recognition model <NUM> corresponding to a voice command generated through the candidate recognition model <NUM> or the recognition model <NUM> corresponding to a voice command generated by a user input, according to the frequency of use and the last use time.

In operation <NUM>, the control module <NUM> performs voice command processing on the basis of the received audio analysis data. If there is a voice command <NUM>, the control module <NUM> may update (for example, weight increase) the weight of a corresponding recognition model <NUM>. During this operation, the control module <NUM> may update (for example, weight decrease) the weight of the voice command <NUM> having no relation to audio analysis data. If there is a recognition model <NUM> having a weight less than a predetermined value, the control module <NUM> may remove a corresponding recognition model <NUM>. When the recognition model <NUM> is removed, a corresponding voice command <NUM> may be removed.

According to an embodiment of the present disclosure, the voice command providing method may include collecting audio data and updating a voice command <NUM> set to execute a specific function of an electronic device through a voice command <NUM> when specific audio data is collected, on the basis of at least one of audio feature data of the audio data, speech recognition data obtained by speech-recognizing the audio data, and function execution data executed in relation to the audio data.

According to an embodiment of the present disclosure, the method includes updating the weight of at least one of pre-stored audio feature data <NUM>, the speech recognition data <NUM>, and the function execution data <NUM> in correspondence to the audio data collection and generating the voice command <NUM> if the weight satisfies a specific condition.

According to an embodiment of the present disclosure, the updating of the voice command <NUM> may include generating a candidate recognition model <NUM> if the weight satisfies the specific condition, updating the weight of the candidate recognition model <NUM> in relation to the audio data, and changing the candidate recognition model <NUM> into a recognition model <NUM> corresponding to the voice command <NUM> if the updated weight satisfies the specific condition.

According to an embodiment of the present disclosure, the updating of the voice command <NUM> may include newly registering a voice command <NUM> corresponding to the recognition model <NUM> changed from the candidate recognition model <NUM>.

According to an embodiment of the present disclosure, the updating of the voice command <NUM> may include updating the weight of at least one of a candidate recognition model <NUM> and recognition model <NUM> having no relation with the collected audio data and removing at least one of a specific candidate recognition model <NUM> and a specific recognition model <NUM> in correspondence to the weight update.

According to an embodiment of the present disclosure, the updating of the voice command <NUM> may include removing a registered voice command <NUM> corresponding to the recognition model <NUM> removal.

According to an embodiment of the present disclosure, the method may further include outputting at least one of the audio feature data <NUM>, the speech recognition data <NUM>, the function execution data <NUM>, and the voice command <NUM>.

According to an embodiment of the present disclosure, the updating of the weight may include updating the weight of the audio feature data <NUM> in correspondence to the frequency of occurrence of the audio feature data <NUM>, updating the speech recognition data <NUM> in correspondence to the frequency of occurrence of the speech recognition data <NUM>, and updating the weight of the function execution data <NUM> on the basis of whether to execute a specific function of the function execution data <NUM> and a specific function execution duration time of the function execution data <NUM>.

According to an embodiment of the present disclosure, the method may further include transmitting the audio data to a server device <NUM> and receiving, from the server device <NUM>, at last one of the audio feature data <NUM>, the speech recognition data <NUM>, and the function execution data <NUM>.

According to an embodiment of the present disclosure, the method may further include receiving, from the server device <NUM>, a voice command generated based on at least one of the audio feature data <NUM>, the speech recognition data <NUM>, and the function execution data <NUM>.

<FIG> is an illustration of a user interface relating to a voice command provision according to an embodiment of the present disclosure.

Referring to <FIG>, the control module <NUM> performs registration of a new voice command corresponding to specific audio data collection. Accordingly, the display module <NUM> may output information for guiding voice command registration as shown in a screen <NUM>. For example, when a user inputs the audio data "show me," the control module <NUM> may receive audio analysis data by transmitting corresponding audio data to the server device <NUM> or may collect audio analysis data by using a speech recognition algorithm and an audio analysis module. The control module <NUM> updates the audio feature data <NUM>, speech recognition data <NUM>, and function execution data <NUM> for the collected audio data "show me. " If a voice command generation condition is satisfied during an update operation, the control module <NUM> generates the audio data "show me" as a new voice command <NUM>. The display module <NUM> displays the function execution information "open Map" <NUM> mapped into the voice command "show me" <NUM> together. That is, the display module <NUM> may perform a notification for a voice shortcut function.

According to an embodiment of the present disclosure, when a new voice command <NUM> is generated or an event corresponding to a voice command list view occurs, the control module <NUM> may perform a control to output a voice command list screen to the display module <NUM> as shown in a screen <NUM>. The voice command list, for example, may include "Camera" <NUM>, "Call Mom" <NUM>, and "open Map" <NUM> corresponding to function execution data <NUM> "Action. " A voice command "Voice" corresponding to the function execution data <NUM> may include "Camera" and "Picture" <NUM> corresponding to the "Camera" <NUM>. Accordingly, when a user pronounces "Camera" or "Picture," the control module <NUM> may perform a control to execute a camera function. The voice command "Voice" may include "Mom," "Mother," and "Mommy" <NUM> corresponding to "Call Mom" <NUM>. The voice command "Voice" may include "show me" <NUM> corresponding to "open Map" <NUM>.

According to an embodiment of the present disclosure, if the candidate recognition model <NUM> is updated or an event relating to the view of the candidate recognition model <NUM> occurs, the control module <NUM> may provide a screen interface as shown in a screen <NUM>. A candidate recognition model view screen, for example, may include a function execution data <NUM> "Action" <NUM>, a speech recognition data <NUM> "Transcription" <NUM>, and a weight "Score Weight" <NUM>.

<FIG> is a block diagram of an electronic device according to an embodiment of the present disclosure.

Referring to <FIG>, the electronic device <NUM>, for example, may configure all or part of the above-mentioned electronic device <NUM> shown in <FIG>. Referring to <FIG>, the electronic device <NUM> includes at least one Application Processor (AP) <NUM> (for example, the control module <NUM> of <FIG>), a communication module <NUM> (for example, the communication interface <NUM> of <FIG>), a Subscriber Identification Module (SIM) card <NUM>, a memory <NUM> (for example, the storage module <NUM> of <FIG>), a sensor module <NUM>, an input device <NUM> (for example, the input module <NUM> of <FIG>), a display module <NUM> (for example, the display module <NUM> of <FIG>), an interface <NUM>, an audio module <NUM> (for example, the audio module <NUM> of <FIG>), a camera module <NUM>, a power management module <NUM>, a battery <NUM>, an indicator <NUM>, and a motor <NUM>.

The AP <NUM> may control a plurality of hardware or software components connected to the AP <NUM> by executing an operating system or an application program. The AP <NUM> may perform various data processing and calculations with multimedia data. The AP <NUM> may be implemented with a System on Chip (SoC), for example. According to an embodiment of the present disclosure, the AP <NUM> may further include a Graphics Processing Unit (GPU).

According to an embodiment of the present disclosure, various modules relating to the above-mentioned voice command provision may be implemented in the AP <NUM>. The AP <NUM> may control the collection of audio data, the transmission of audio data, the reception of audio analysis data, the update of the storage module <NUM> on the basis of audio analysis data, the update of the candidate recognition model <NUM> and the recognition model <NUM>, and the processing and management of the voice command <NUM> by performing a control of related modules.

According to an embodiment of the present disclosure, various modules relating to the voice command provision may be separately disposed in at least one of a Communication Processor (CP) and an AP <NUM>. During this operation, modules that a CP controls may be disposed to allow direct communication in the same chipset as the CP or to be connected to the CP through an external interface in terms of hardware.

According to an embodiment of the present disclosure, various modules relating to the voice command provision may be disposed in a CP. A CP may control related modules. In this case, the CP may be communicably connected to a cellular communication module and a WiFi module. In relation to this, when each of the configurations is connected via a system bus in the same chipset, when an AP is in a power saving mode, they are communicably connected to each other. When each of the configurations is configured with a different chipset, they are connected through an external interface, and communication may be possible without passing through an AP in terms of hardware.

The communication module <NUM> may perform data transmission/reception between the electronic device <NUM> (for example, the electronic device <NUM>) and the server device <NUM> connected via network. According to an embodiment of the present disclosure, the communication module <NUM> may include a cellular module <NUM>, a WiFi module <NUM>, a Bluetooth (BT) module <NUM>, a Global Positioning System (GPS) module <NUM>, a Near Field Communication (NFC) module <NUM>, and a Radio Frequency (RF) module <NUM>.

The cellular module <NUM> may provide voice calls, video calls, text services, or Internet services through a communication network (for example, Long Term Evolution (LTE), LTE Advanced (LTE-A), Code Division Multiple Access (CDMA), Wideband CDMA (WCDMA), Universal Mobile Telecommunications System (UMTS), Wireless Broadband (WiBro), or Global System for Mobile communications (GSM)). The cellular module <NUM> may perform a distinction and authentication operation on an electronic device in a communication network by using a SIM card <NUM>, for example. According to an embodiment of the present disclosure, the cellular module <NUM> may perform at least part of a function that the AP <NUM> provides. For example, the cellular module <NUM> may perform at least part of a multimedia control function.

According to an embodiment of the present disclosure, the cellular module <NUM> may further include a Communication Processor (CP). Additionally, the cellular module <NUM> may be implemented with SoC, for example. As shown in <FIG>, components such as the cellular module <NUM> (for example, a CP), the memory <NUM>, or the power management module <NUM> are separated from the AP <NUM>, but according to an embodiment of the present disclosure, the AP <NUM> may be implemented to include some of the above-mentioned components (for example, the cellular module <NUM>).

According to an embodiment of the present disclosure, the AP <NUM> or the cellular module <NUM> (for example, a CP) may load instructions or data, which are received from a nonvolatile memory or at least one of other components connected thereto, into a volatile memory and then may process them. Furthermore, the AP <NUM> or the cellular module <NUM> may store data received from or generated by at least one of other components in a nonvolatile memory.

According to an embodiment of the present disclosure, the cellular module <NUM> may be included in the communication interface <NUM> described with reference to <FIG>. The cellular module <NUM> may establish a communication channel with the server device <NUM> during an operation for performing the voice command processing of the electronic device <NUM>. Additionally, the cellular module <NUM> may transmit audio data that the electronic device <NUM> collects and may receive audio analysis data. According to an embodiment of the present disclosure, the cellular module <NUM> may receive a voice command and voice command update information from the server device <NUM>.

Each of the WiFi module <NUM>, the BT module <NUM>, the GPS module <NUM>, and the NFC module <NUM> may include a processor for processing data transmitted/received through a corresponding module. The WiFi module <NUM> and the BT module <NUM> may be included in the communication interface <NUM>. The GPS module <NUM> may be included in the communication interface <NUM>.

Although the cellular module <NUM>, the WiFi module <NUM>, the BT module <NUM>, the GPS module <NUM>, and the NFC module <NUM> are shown as separate blocks in <FIG>, according to an embodiment of the present disclosure, some (for example, at least two) of the cellular module <NUM>, the WiFi module <NUM>, the BT module <NUM>, the GPS module <NUM>, and the NFC module <NUM> may be included in one Integrated Circuit (IC) or an IC package. For example, at least some (for example, a CP corresponding to the cellular module <NUM> and a WiFi processor corresponding to the WiFi module <NUM>) of the processors respectively corresponding to the cellular module <NUM>, the WiFi module <NUM>, the BT module <NUM>, the GPS module <NUM>, and the NFC module <NUM> may be implemented with one SoC.

According to an embodiment of the present disclosure, the WiFi module <NUM> may be included in the communication interface <NUM> described with reference to <FIG>. The WiFi module <NUM> may establish a communication channel between the electronic device <NUM> and the server device <NUM>. The WiFi module <NUM> may transmit at least one of the identification information of audio data and the electronic device <NUM> or the function type information of the electronic device <NUM> to the server device <NUM>. The WiFi module <NUM> may receive at least one of audio analysis data and voice command update information from the server device <NUM>.

The RF module <NUM> may be responsible for data transmission, for example, the transmission of an RF signal. The RF module <NUM> may include a transceiver, a Power Amplifier Module (PAM), a frequency filter, or a Low Noise Amplifier (LNA). Additionally, the RF module <NUM> may further include components for transmitting/receiving electromagnetic waves in free space in a wireless communication, for example, conductors or conductive wires. Although the cellular module <NUM>, the WiFi module <NUM>, the BT module <NUM>, the GPS module <NUM>, and the NFC module <NUM> share one RF module <NUM> shown in <FIG>, according to an embodiment of the present disclosure, at least one of the cellular module <NUM>, the WiFi module <NUM>, the BT module <NUM>, the GPS module <NUM>, and the NFC module <NUM> may perform the transmission/reception of an RF signal through an additional RF module.

The SIM card <NUM> may be inserted into a slot formed at a specific position of an electronic device. The SIM card <NUM> may include unique identification information (for example, an Integrated Circuit Card Identifier (ICCID)) or subscriber information (for example, an International Mobile Subscriber Identity (IMSI)). According to an embodiment of the present disclosure, the SIM card <NUM> may be a partial configuration of the memory <NUM>. When the SIM card <NUM> is inserted into a slot, the AP <NUM> may perform the initialization operation of the SIM card <NUM>.

The memory <NUM> (for example, the storage module <NUM>) may include an internal memory <NUM> or an external memory <NUM>. The internal memory <NUM> may include at least one of a volatile memory (for example, Dynamic Random Access Memory (DRAM), Static RAM (SRAM), Synchronous DRAM (SDRAM)) and a non-volatile memory (for example, One Time Programmable Read Only Memory (OTPROM), Programmable ROM (PROM), Erasable and Programmable ROM (EPROM), Electrically Erasable and Programmable ROM (EEPROM), mask ROM, flash ROM, NAND flash memory, and NOR flash memory). According to an embodiment of the present disclosure, the internal memory <NUM> may be a Solid State Drive (SSD).

The external memory <NUM> may further include a flash drive, for example, a Compact Flash (CF) drive, a Secure Digital (SD) memory card, a Micro SD (Micro-SD) memory card, a Miniature SD (Mini-SD) memory card, an extreme Digital (xD) memory card, or a memory stick. The external memory <NUM> may be functionally connected to the electronic device <NUM> through various interfaces. According to an embodiment of the present disclosure, the electronic device <NUM> may further include a storage device (or a storage medium) such as a hard drive.

The sensor module <NUM> measures physical quantities or detects an operating state of the electronic device <NUM>, thereby converting the measured or detected information into electrical signals. The sensor module <NUM> may include at least one of a gesture sensor 940A, a gyro sensor 940B, a barometric pressure sensor 940C, a magnetic sensor 940D, an acceleration sensor 940E, a grip sensor 940F, a proximity sensor <NUM>, a color sensor <NUM> (for example, a Red, Green, Blue (RGB) sensor), a biometric sensor 940I, a temperature/humidity sensor 940J, an illumination sensor <NUM>, and an Ultra Violet (UV) light sensor <NUM>. Additionally or alternatively, the sensor module <NUM> may include an Electronic nose (E-nose) sensor, an ElectroMyoGraphy (EMG) sensor, an ElectroEncephalogram (EEG) sensor, an ElectroCardioGram (ECG) sensor, an InfraRed (IR) sensor, an iris sensor, or a fingerprint sensor. The sensor module <NUM> may further include a control circuit for controlling at least one sensor therein.

According to an embodiment of the present disclosure, the sensor module <NUM> may collect operation related sensor signals of the electronic device <NUM>. The sensor signals that the sensor module <NUM> collects may be delivered to the AP <NUM>. The AP <NUM> may perform voice command processing on the basis of the delivered sensor signals. For example, the sensor signals delivered from the sensor module <NUM> may be used as a trigger signal relating to the activation of at least one of a voice command providing function and a voice search function.

At least one sensor included in the sensor module <NUM> may be activated in relation to the management of the electronic device <NUM>. For example, a sensor signal of a sensor for detecting the grip of the electronic device <NUM> may be used as a sensor signal relating to voice search function activation. Additionally, a sensor signal of a grip detection sensor may be used as a sensor signal relating to microphone activation. The grip detection sensor may be deactivated in correspondence to a voice search function deactivation.

The input device <NUM> may include a touch panel <NUM>, a (digital) pen sensor <NUM>, a key <NUM>, or an ultrasonic input device <NUM>. The input device <NUM> may include the input module <NUM> of <FIG>.

The touch panel <NUM> may recognize a touch input through at least one of capacitive type, resistive type, infrared type, or ultrasonic type methods, for example. Additionally, the touch panel <NUM> may further include a control circuit. In the case of the capacitive type method, both direct touch and proximity recognition are possible. The touch panel <NUM> may further include a tactile layer. In this case, the touch panel <NUM> may provide a tactile response to a user. According to an embodiment of the present disclosure, the touch panel <NUM> may generate a touch event relating to the execution of at least one of a voice command providing function and a voice search function. For example, the touch panel <NUM> may provide a touch event for removing at least one of a candidate recognition model <NUM> and a recognition model <NUM> and requesting to change a specific candidate recognition model <NUM> into a recognition model <NUM> in relation to a screen interface. According to an embodiment of the present disclosure, the touch panel <NUM> may generate a touch event for activating or deactivating a screen automatic conversion function or selecting the type of a function to be applied to a screen automatic conversion function.

The (digital) pen sensor <NUM> may be implemented through a method similar or identical to that of receiving a user's touch input or an additional sheet for recognition. The key <NUM> may include a physical button, an optical key, or a keypad, for example. At least one key, for example, a home key, in the key <NUM> may generate a key event relating to voice search function activation or a key event relating to voice command providing function activation. The ultrasonic input device <NUM>, as a device checking data by detecting sound waves through a microphone (for example, a microphone <NUM>) in the electronic device <NUM>, may provide wireless recognition through an input tool generating ultrasonic signals. According to an embodiment of the present disclosure, the electronic device <NUM> may receive a user input from an external device (for example, a computer or a server) connected thereto through the communication module <NUM>.

The display module <NUM> (for example, the display module <NUM>) may include a panel <NUM>, a hologram device <NUM>, or a projector <NUM>. The panel <NUM> may include a Liquid-Crystal Display (LCD) or an Active-Matrix Organic Light-Emitting Diode (AM-OLED). The panel <NUM> may be implemented to be flexible, transparent, or wearable, for example. The panel <NUM> and the touch panel <NUM> may be configured with one module. The hologram <NUM> may show three-dimensional images in the air by using the interference of light. The projector <NUM> may display an image by projecting light onto a screen. The screen, for example, may be placed internally or externally to the electronic device <NUM>. According to an embodiment of the present disclosure, the display <NUM> may further include a control circuit for controlling the panel <NUM>, the hologram device <NUM>, or the projector <NUM>.

The interface <NUM> may include a High-Definition Multimedia Interface (HDMI) <NUM>, a Universal Serial Bus (USB) <NUM>, an optical interface <NUM>, or a D-subminiature (D-sub) connector <NUM>, for example. Additionally/alternately, the interface <NUM> may include a Mobile High-dimension Link (MHL) interface, a Secure Digital (SD) card/Multi-Media Card (MMC) interface, or an Infrared Data Association (IrDA) standard interface.

The audio module <NUM> (for example, the audio module <NUM> of <FIG>) converts sound into electrical signals and vice versa. The audio module <NUM> may process sound information inputted/outputted through a speaker <NUM>, a receiver <NUM>, an earphone <NUM>, or a microphone <NUM>. According to an embodiment of the present disclosure, the audio module <NUM> disposed in the electronic device <NUM> may collect audio data. Additionally, the audio module <NUM> may output audio data relating to function execution by a voice command.

The camera module <NUM>, as a device for capturing a still image and a video, may include at least one image sensor (for example, a front sensor or a rear sensor), a lens, an Image Signal Processor (ISP), or a flash (for example, a Light-Emitting Diode (LED) or a xenon lamp).

The power management module <NUM> manages the power of the electronic device <NUM>. According to an embodiment of the present disclosure, the power management module <NUM> may control the power supply of at least one device element of the electronic device <NUM>. For example, when the battery power of the electronic device <NUM> is less than a setting value, the power management module <NUM> may cut off the power supply of specific device elements (for example, a sensor module <NUM>, a communication module <NUM> for establishing a direct communication channel, a GPS module <NUM>, and so on). The power management module <NUM> may change the power supply of device elements activated in relation to a communication function of the electronic device <NUM> according to a communication connection release with the server device <NUM>. The power management module <NUM> may include a Power Management IC (PMIC), a charger IC, or a battery gauge, for example.

The PMIC may be built in an IC or an SoC semiconductor, for example. A charging method may be classified into a wired method and a wireless method. The charger IC may charge a battery and may prevent overvoltage or overcurrent flow from a charger. According to an embodiment of the present disclosure, the charger IC may include a charger IC for at least one of a wired charging method and a wireless charging method. The wireless charging method includes, for example, a magnetic resonance method, a magnetic induction method, or an electromagnetic method. An additional circuit for wireless charging, for example, a circuit such as a coil loop, a resonant circuit, or a rectifier circuit, may be added.

The battery gauge measures the remaining battery power amount of the battery <NUM>, or a voltage, current, or temperature of the battery <NUM> during charging. The battery <NUM> may store or generate electrical power and may supply electrical power to the electronic device <NUM> by using the stored or generated electrical power. The battery <NUM>, for example, may include a rechargeable battery or a solar battery.

The indicator <NUM> may display a specific state of the electronic device <NUM> or part thereof (for example, the AP <NUM>), for example, a booting state, a message state, or a charging state. According to an embodiment of the present disclosure, the indicator <NUM> may display a voice command providing function state, a voice search function execution state, and a voice command based function execution available state. The indicator <NUM> may display a connection state of the voice command provision based server device <NUM> The motor <NUM> may convert electrical signals into mechanical vibration. The electronic device <NUM> may include a processing device (for example, a GPU) for mobile TV support. A processing device for mobile TV support may process media data according to the standards such as Digital Multimedia Broadcasting (DMB), Digital Video Broadcasting (DVB), or media flow.

Each of the above-mentioned components of the electronic device according to an embodiment of the present disclosure may be configured with at least one component and the name of a corresponding component may vary according to the kind of an electronic device. An electronic device according to an embodiment of the present disclosure may include at least one of the above-mentioned components, may not include some of the above-mentioned components, or may further include another component. Additionally, some of components in an electronic device according to an embodiment of the present disclosure are configured as one entity, so that functions of previous corresponding components are performed identically.

The term "module" used in various embodiments of the present disclosure, for example, may refer to a unit including a combination of at least one of hardware, software, and firmware. The terms "module," "unit," "logic," "logical block," "component," or "circuit" may be interchangeably used. The term "module" may refer to a minimum unit or part of an integrally configured component. The term "module" may refer to a minimum unit performing at least one function or part thereof. The "module" may refer to a device implemented mechanically or electronically. For example, the term "module" according to various embodiments of the present disclosure may refer to at least one of an Application-Specific Integrated Circuit (ASIC) performing specific operations, Field-Programmable Gate Arrays (FPGAs), or a programmable-logic device, all of which are known or will be developed in the future.

It will be appreciated that embodiments of the present invention can be realized in the form of hardware, software or a combination of hardware and software. Any such software may be stored in the form of volatile or non-volatile storage.

It will be appreciated that the storage devices and storage media are embodiments of machine-readable storage that are suitable for storing a program or programs comprising instructions that, when executed, implement embodiments of the present disclosure. Accordingly, embodiments provide a program comprising code for implementing an apparatus, method, device or system according to any aspect, claim, embodiment or example of this specification, and a machine-readable storage storing such a program. Still further, such programs may be conveyed electronically via any medium such as a communication signal carried over a wired or wireless connection and embodiments suitably encompass the same.

For example, according to an embodiment of the present disclosure, at least part of a device (for example, modules or functions thereof) or a method (for example, operations) according to the present disclosure, for example, as in a form of a programming module, may be implemented using an instruction stored in non-transitory computer-readable storage media. When at least one processor (for example, the processor <NUM>) executes an instruction, it may perform a function corresponding to the instruction. The non-transitory computer-readable storage media may include the memory <NUM>, for example. At least part of a programming module may be implemented (for example, executed) by the Application Processor <NUM>, for example. At least part of a programming module may include a module, a program, a routine, sets of instructions, or a process to perform at least one function, for example.

The non-transitory computer-readable storage media may include magnetic media such as a hard disk, a floppy disk, and a magnetic tape, optical media such as a Compact Disc Read Only Memory (CD-ROM) and a Digital Versatile Disc (DVD), magnetooptical media such as a floptical disk, and a hardware device especially configured to store and perform a program instruction (for example, a programming module) such as a Read Only Memory (ROM), a Random Access Memory (RAM), and a flash memory. Additionally, a program instruction may include high-level language code executable by a computer using an interpreter in addition to machine readable code generated by a complier. The hardware device may be configured to operate as at least one software module to perform an operation of an embodiment of the present disclosure and vice versa.

A module or a programming module according to an embodiment of the present disclosure may include at least one of the above-mentioned components, may not include some of the above-mentioned components, or may further include another component. Operations performed by a module, a programming module, or other components according to an embodiment of the present disclosure may be executed through a sequential, parallel, repetitive or heuristic method. Additionally, some operations may be executed in a different order or may be omitted, or other operations may be added.

The terms "include," "comprise," "have," "may include," "may comprise" and "may have" used herein indicate disclosed functions, operations, or the existence of elements but do not exclude other functions, operations or elements. Additionally, in an embodiment of the present disclosure, the terms "include," "comprise," "including," or "comprising," specify a property, a region, a fixed number, an operation, a process, an element and/or a component but do not exclude other properties, regions, fixed numbers, operations, processes, elements and/or components.

Throughout the description and claims of this specification, the singular forms "a," "an," and "the" include plural referents unless the context dictates otherwise. Thus, for example, reference to "an object" includes reference to one or more of such objects.

Throughout the description and claims of this specification, language in the general form of "X for Y" (where Y is some action, process, activity, operation or step and X is some means for carrying out that action, process, activity, operation or step) encompasses means X adapted, configured or arranged specifically, but not exclusively, to do Y.

In an embodiment of the present disclosure, the expressions "A or B" or "at least one of A or/and B" may include all possible combinations of items listed together. For instance, the expressions "A or B" or "at least one of A and/or B" may include A, B, or both A and B.

The terms such as "1st," "2nd," "first," "second," and the like used herein may refer to modifying various different elements of various embodiments of the present disclosure, but do not limit the elements. For instance, such expressions do not limit the order and/or importance of corresponding components. The expressions may be used to distinguish one element from another element. For example, a first component may be referred to as a second component and vice versa without departing from the scope of the present disclosure.

In the disclosure below, when one part (or element, device, etc.) is referred to as being "connected" to another part (or element, device, etc.), it should be understood that the former can be "directly connected" to the latter, or "connected" to the latter via an intervening part (or element, device, etc.).

In an embodiment of the present disclosure, terms used in the present disclosure are used to describe specific embodiments, and are not intended to limit the scope of the present disclosure. The terms of a singular form may include plural forms unless they have a clearly different meaning in the context.

Unless otherwise indicated herein, all the terms used herein, which include technical or scientific terms, may have the same meanings that are generally understood by a person skilled in the art. In general, the terms defined in the dictionary should be considered to have the same meanings as the contextual meanings of the related art, and, unless clearly defined herein, should not be understood abnormally or as having excessively formal meanings.

According to an embodiment of the present disclosure, electronic devices may include at least one of smartphones, tablet Personal Computers (PCs), mobile phones, video phones, electronic book (e-book) readers, desktop PCs, laptop PCs, netbook computers, Personal Digital Assistants (PDAs), Portable Multimedia Players (PMPs), Moving Picture Experts Group Audio Layer <NUM> (MP3) players, mobile medical devices, cameras, and wearable devices (for example, Head-Mounted-Devices (HMDs) such as electronic glasses, electronic apparel, electronic bracelets, electronic necklaces, electronic appcessories, electronic tattoos, and smart watches).

According to an embodiment of the present disclosure, an electronic device may be smart home appliances. The smart home appliances may include at least one of, for example, televisions, Digital Video Disk (DVD) players, audio players, refrigerators, air conditioners, cleaners, ovens, microwave ovens, washing machines, air cleaners, set-top boxes, TV boxes (for example, Samsung HomeSyncTM, Apple TVTM or Google TVTM), game consoles, electronic dictionaries, electronic keys, camcorders, and electronic picture frames.

According to an embodiment of the present disclosure, an electronic device may include at least one of various medical devices (for example, Magnetic Resonance Angiography (MRA) devices, Magnetic Resonance Imaging (MRI) devices, Computed Tomography (CT) devices, medical imaging devices, ultrasonic devices, etc.), navigation devices, Global Positioning System (GPS) receivers, Event Data Recorders (EDRs), Flight Data Recorders (FDRs), vehicle infotainment devices, marine electronic equipment (for example, marine navigation systems, gyro compasses, etc.), avionics, security equipment, vehicle head modules, industrial or household robots, financial institutions' Automatic Teller Machines (ATMs), and vendors' Point of Sale (POS) devices.

In an embodiment of the present disclosure, an electronic device may include at least one of part of furniture or buildings/structures supporting call forwarding service, electronic boards, electronic signature receiving devices, projectors, and various measuring instruments (for example, water meters, electricity meters, gas meters, or radio signal measuring instruments). An electronic device according to an embodiment of the present disclosure may be one of the above-mentioned various devices or a combination thereof. Additionally, an electronic device according to an embodiment of the present disclosure may be a flexible device. Furthermore, it is apparent to those skilled in the art that an electronic device according to an embodiment of the present disclosure is not limited to the above-mentioned devices.

According to an embodiment of the present disclosure, fast function processing is supported by converting audio data that a user uses frequently into a voice command.

Additionally, according to an embodiment of the present disclosure, when audio data is converted into a voice command, additional server device management is unnecessary so that a wireless data resource is not necessary.

Additionally, according to an embodiments of the present disclosure, when voice commands are updated based on an audio data management history, the management of unnecessary voice commands is reduced and voice commands for a function that a user wants to frequently use are managed efficiently.

Claim 1:
A mobile communication device (<NUM>), comprising:
a display module (<NUM>);
a storage module (<NUM>) configured to store voice commands, audio feature data, speech recognition data, and function execution data, each corresponding to received audio data; and
a control module (<NUM>) operably connected to the display module (<NUM>) and the storage module (<NUM>),
wherein the control module (<NUM>) is configured to:
receive audio data from a user;
obtain audio analysis data based on the received audio data, the audio analysis data being at least one of audio feature data, speech recognition data, and function execution data;
update, based on the obtained audio analysis data, the stored audio feature data, speech recognition data, and function execution data, each corresponding to the received audio data;
in the case that a voice command generation condition is satisfied based on the update, generate a voice command corresponding to the received audio data and store the generated voice command; and
display, on the display (<NUM>), the generated voice command and the corresponding function to be executed by the mobile communication device (<NUM>);
wherein updating the stored audio feature data, speech recognition data, and
function execution data comprises updating a stored weight of at least one of the audio feature data, speech recognition data, and function execution data corresponding to the received audio data based on the number of times the received audio data has been received, the number being based on the obtained audio analysis data, and
wherein the voice command generation condition is satisfied when the updated weight of at least one of the audio feature data, speech recognition data, and function execution data corresponding to the received audio data is greater than a predetermined value.