Patent Description:
Electronic devices (for example, mobile terminals, smartphones, wearable devices, or the like) provide various functions. For example, smartphones provide a short-range wireless communication (for example, Bluetooth, Wi-Fi, or near field communication (NFC), or the like) function, a mobile communication (<NUM>rd generation (<NUM>), <NUM>, <NUM>, or the like) function, a music or video replay function, a shooting function, or a navigation function, in addition to a basic voice communication function.

In recent years, interests in artificial intelligence (Al) are increasing. A recognition rate of AI increases as machines are more trained and determine by themselves, and are used. AI involves an unsupervised learning model which uses an algorithm for classifying/learning features of input data by itself, and a supervised learning network model using trained data, and may be configured by element technologies which describe functions such as recognition, determination, or the like of the human brain.

The element technologies may include at least one of language understanding technology for recognizing human language/text, visual understanding technology for recognizing things like human vision, inference/prediction technology for inferring and predicting logically by determining information, knowledge representation technology for processing human experiment information as knowledge data, and operation control technology for controlling automatic driving of vehicles, motions of robots, or the like.

Such devices are e.g. disclosed in <CIT>, <CIT>, <CIT> and <CIT>.

It is common that AI is included in a relatively high-performance external electronic device (for example, a server), rather than in an electronic device, due to complicated calculation and/or excessive throughput. For example, an electronic device may receive a user input and transmit the received user input to an external electronic device, and may receive a response (for example, a result of analyzing (predicting) by AI on the user input) to the user input from the external electronic device.

Since the user input is analyzed using AI included in the external electronic device, the electronic device may have a limitation according to a communication state with the external electronic device or a load state of the external electronic device. For example, when a communication state between the electronic device and the external electronic device is poor or an excessive load is generated in the external electronic device (for example, when analysis of user input is requested from a plurality of electronic devices), the electronic device may receive the result of analyzing very late or may fail to receive the result of analyzing.

Various example embodiments of the disclosure are provided to address the above-described problems and analyze a user input at an electronic device and an external electronic device, respectively, and control at least one application and/or a predefined function using one of the results of the analyzing, based on a response time of the external electronic device.

According to the invention, an electronic device according to claim <NUM> and method of operating an electronic device according to claim <NUM> are provided.

The above and other aspects, features, and advantages of certain embodiments of the disclosure will be more apparent from the following detailed description, taken in conjunction with the accompanying drawings, in which:.

Various example embodiments of the disclosure will be described in greater detail below with reference to the accompanying drawings. Although specific embodiments are illustrated in the drawings and relevant detailed descriptions are provided, this does not limit various example embodiments of the disclosure to a specific form. For example, it may be apparent to a person skilled in the art that various changes can be made to various embodiments of the disclosure.

<FIG> is a diagram illustrating an example system for recognizing a user input according to an embodiment of the disclosure.

Referring to <FIG>, the system <NUM> according to an embodiment of the disclosure may include an electronic device <NUM> and an external electronic device <NUM>.

The electronic device <NUM> according to an embodiment of the disclosure may, for example, and without limitation, be a smartphone, a wearable device, a tablet personal computer (PC), or the like.

The electronic device <NUM> according to an embodiment of the disclosure may receive a user input through various input devices. For example, the electronic device <NUM> may receive a user voice through a microphone. According to an example embodiment, the electronic device <NUM> may receive a user input requesting recognition of an image obtained through a camera or a stored image. According to an example embodiment, the electronic device <NUM> may receive a text input through a physical keyboard or a virtual keypad. According to an example embodiment, the electronic device <NUM> may receive a writing input through an electronic pen or a touch screen.

The electronic device <NUM> according to an embodiment of the disclosure may predict a user's demand by analyzing the user input, and may control at least one application using the result of predicting. For example, the electronic device <NUM> may predict a user's demand by analyzing the user input using a first trained model <NUM> which is trained through a first recognition module <NUM>. According to an example embodiment, the first recognition module <NUM> generates first control data for controlling at least one application among one or more applications using a first recognition method.

The first recognition module (e.g., including processing circuitry and/or executable program elements) <NUM> may be an AI module (for example, an artificial neural network). The first recognition module <NUM> may have a high processing speed, but has low accuracy in comparison to a second recognition module (e.g., including processing circuitry and/or executable program elements) <NUM>. For example, the number of neural networks, a size of a used memory, and/or the number of multiply-and-accumulate (MAC) operations in the first recognition module <NUM> may be a predefined multiple or more (for example, <NUM> times) smaller than in the second recognition module <NUM>. The first recognition module <NUM> may be included in an electronic device of a low specification or a low performance (for example, a smartphone, a tablet PC). The first trained model <NUM> may be data that is trained by the first recognition module <NUM> through an AI algorithm such as, for example, and without limitation, machine learning, neural network, gene, deep learning, classification algorithm, or the like.

The electronic device <NUM> according to an embodiment of the present disclosure may transmit at least part of the user input to the external electronic device <NUM>. For example, the electronic device <NUM> may transmit at least part of the user input to the external electronic device <NUM> through a communication module (not shown), such that the external electronic device <NUM> generates second control data using a second recognition method based at least on the user input. The second control data may be data for controlling at least one application among one or more applications.

The electronic device <NUM> according to an embodiment of the disclosure selects one of the first control data or the second control data, based on a time until the second control data is received from the external electronic device <NUM>, and controls at least one application based on the selected control data.

According to an example embodiment, the electronic device <NUM> selects one of the first control data or the second control data, and controls one of a first predefined function or a second predefined function based on the selected control data. For example, when the second control data is not received within a predefined time, the electronic device <NUM> may perform the first predefined function using the first control data. On the other hand, when the second control data is received within the predefined time, the electronic device <NUM> may perform the second predefined function using the second control data.

The external electronic device <NUM> according to an embodiment of the disclosure may be a device (for example, a server) which is able to communicate with the electronic device <NUM> through a wired and/or wireless network. The external electronic device <NUM> may receive at least part of the user input from the electronic device <NUM>. The external electronic device <NUM> may predict a user's demand by analyzing the received at least part of the user input, and may generate the second control data for controlling at least one application using the result of predicting, and may transmit the second control data to the electronic device <NUM>. For example, the external electronic device <NUM> may predict the user's demand by analyzing the user input using a second trained model <NUM> which is trained through the second recognition module <NUM>. According to an example embodiment, the second recognition module <NUM> may generate the second control data for controlling at least one application among one or more applications using the second recognition method.

The second recognition module <NUM> may be an AI module (e.g., including processing circuitry and/or executable program elements). The second recognition module <NUM> may have a low processing speed, but has high accuracy in comparison to the first recognition module <NUM>. For example, the number of neural networks, a size of a used memory, and/or the number of MAC operations in the second recognition module <NUM> may be a predefined multiple or more (for example, about <NUM> times) larger than in the first recognition module <NUM>. The second recognition module <NUM> may be included in an electronic device of a high specification or high performance (for example, a server). The second trained model <NUM> may be data that is trained by the second recognition module <NUM> through an AI algorithm such as, for example, and without limitation, machine learning, neural network, gene, deep learning, classification algorithm, or the like.

<FIG> is a diagram illustrating an example user cognition method of a system according to an embodiment of the disclosure.

Referring to <FIG>, an input layer according to an embodiment of the disclosure may receive a user input, and may transmit the received user input to the first recognition module <NUM> and the second recognition module <NUM>, thereby allowing the first recognition module <NUM> and the second recognition module <NUM> to analyze the user input altogether. The respective results of analyzing may be transmitted to an output layer, and one of the results may be selected by the output layer. For example, the output layer may select the second analysis result when the result of analyzing by the second recognition module <NUM> is received within a predefined time, and may select the first analysis result when the result of analyzing by the second recognition module <NUM> is not received with the predefined time.

An embodiment of the disclosure may analyze the user input through the first recognition module <NUM> included in the electronic device <NUM> and the second recognition module <NUM> included in the external electronic device <NUM> altogether, such that the analysis result can be always provided within a predefined time, and may control an application or a function according to the analysis result. A more detailed description thereof will be provided below with reference to <FIG>.

<FIG> is a diagram illustrating an example method for refining a trained model of a system according to an embodiment of the disclosure.

Referring to <FIG>, the result of analyzing by the second recognition module <NUM> may be provided to the first recognition module <NUM>. For example, when the result of analyzing by the first recognition module <NUM> and the result of analyzing by the second recognition module <NUM> are different, the output layer may provide the result of analyzing by the second recognition module <NUM>, which has high accuracy, to the first recognition module <NUM>, thereby training the first recognition module <NUM>. By doing so, the accuracy of the result of analyzing by the first recognition module <NUM> can be enhanced. A more detailed description thereof will be provided below with reference to <FIG>.

<FIG> is a block diagram illustrating an example electronic device <NUM> in a network environment <NUM> according to various embodiments.

Referring to <FIG>, the electronic device <NUM> (e.g., an electronic device <NUM> of <FIG>) in the network environment <NUM> may communicate with an electronic device <NUM> via a first network <NUM> (e.g., a short-range wireless communication network), or an electronic device <NUM> or a server <NUM> (e.g., an external electronic device <NUM> of <FIG>) via a second network <NUM> (e.g., a long-range wireless communication network).

<FIG> is a block diagram <NUM> illustrating an example program according to various embodiments of the disclosure.

Referring to <FIG>, the program <NUM> according to various embodiments. According to an embodiment, the program <NUM> may include an operating system (OS) <NUM> to control one or more resources of the electronic device <NUM>, middleware <NUM>, or an application <NUM> executable in the OS <NUM>. The OS <NUM> may include, for example, AndroidTM, iOSTM, WindowsTM, SymbianTM, TizenTM, or BadaTM. At least part of the program <NUM>, for example, may be pre-loaded on the electronic device <NUM> during manufacture, or may be downloaded from or updated by an external electronic device (e.g., the electronic device <NUM> or <NUM>, or the server <NUM>) during use by a user.

The OS <NUM> may control management (e.g., allocating or deallocation) of one or more system resources (e.g., process, memory, or power source) of the electronic device <NUM>. The OS <NUM>, additionally or alternatively, may include one or more driver programs to drive other hardware devices of the electronic device <NUM>, for example, the input device <NUM>, the sound output device <NUM>, the display device <NUM>, the audio module <NUM>, the sensor module <NUM>, the interface <NUM>, the haptic module <NUM>, the camera module <NUM>, the power management module <NUM>, the battery <NUM>, the communication module <NUM>, the subscriber identification module <NUM>, or the antenna module <NUM>.

The middleware <NUM> may provide various functions to the application <NUM> such that a function or information provided from one or more resources of the electronic device <NUM> may be used by the application <NUM>. The middleware <NUM> may include, for example, an application manager <NUM>, a window manager <NUM>, a multimedia manager <NUM>, a resource manager <NUM>, a power manager <NUM>, a database manager <NUM>, a package manager <NUM>, a connectivity manager <NUM>, a notification manager <NUM>, a location manager <NUM>, a graphic manager <NUM>, a security manager <NUM>, a telephony manager <NUM>, or a voice recognition manager <NUM>.

The application manager <NUM>, for example, may manage the life cycle of the application <NUM>. The window manager <NUM>, for example, may manage one or more graphical user interface (GUI) resources that are used on a screen. The multimedia manager <NUM>, for example, may identify one or more formats to be used to play media files, and may encode or decode a corresponding one of the media files using a codec appropriate for a corresponding format selected from the one or more formats. The resource manager <NUM>, for example, may manage the source code of the application <NUM> or a memory space of the memory <NUM>. The power manager <NUM>, for example, may manage the capacity, temperature, or power of the battery <NUM>, and determine or provide related information to be used for the operation of the electronic device <NUM> based at least in part on corresponding information of the capacity, temperature, or power of the battery <NUM>. According to an embodiment, the power manager <NUM> may interwork with a basic input/output system (BIOS) (not shown) of the electronic device <NUM>.

The database manager <NUM>, for example, may generate, search, or change a database to be used by the application <NUM>. The package manager <NUM>, for example, may manage installation or update of an application that is distributed in the form of a package file.

The connectivity manager <NUM>, for example, may manage a wireless connection or a direct connection between the electronic device <NUM> and the external electronic device. The notification manager <NUM>, for example, may provide a function to notify a user of an occurrence of a specified event (e.g., an incoming call, message, or alert). The location manager <NUM>, for example, may manage locational information on the electronic device <NUM>. The graphic manager <NUM>, for example, may manage one or more graphic effects to be offered to a user or a user interface related to the one or more graphic effects.

The security manager <NUM>, for example, may provide system security or user authentication. The telephony manager <NUM>, for example, may manage a voice call function or a video call function provided by the electronic device <NUM>. The voice recognition manager <NUM>, for example, may transmit a user's voice data to the server <NUM>, and receive, from the server <NUM>, a command corresponding to a function to be executed on the electronic device <NUM> based at least in part on the voice data, or text data converted based at least in part on the voice data. According to an embodiment, the middleware <NUM> may dynamically delete some existing components or add new components. According to an embodiment, at least part of the middleware <NUM> may be included as part of the OS <NUM> or may be implemented as another software separate from the OS <NUM>.

The application <NUM> may include, for example, a home <NUM>, dialer <NUM>, short message service (SMS)/multimedia messaging service (MMS) <NUM>, instant message (IM) <NUM>, browser <NUM>, camera <NUM>, alarm <NUM>, contact <NUM>, voice recognition <NUM>, email <NUM>, calendar <NUM>, media player <NUM>, album <NUM>, watch <NUM>, health <NUM> (e.g., for measuring the degree of workout or biometric information, such as blood sugar), or environmental information <NUM> (e.g., for measuring air pressure, humidity, or temperature information) application. According to an embodiment, the application <NUM> may further include an information exchanging application (not shown) that is capable of supporting information exchange between the electronic device <NUM> and the external electronic device. The information exchange application, for example, may include a notification relay application adapted to transfer designated information (e.g., a call, message, or alert) to the external electronic device or a device management application adapted to manage the external electronic device. The notification relay application may transfer notification information corresponding to an occurrence of a specified event (e.g., receipt of an email) at another application (e.g., the email application <NUM>) of the electronic device <NUM> to the external electronic device. Additionally or alternatively, the notification relay application may receive notification information from the external electronic device and provide the notification information to a user of the electronic device <NUM>.

The device management application may control the power (e.g., turn-on or turn-off) or the function (e.g., adjustment of brightness, resolution, or focus) of the external electronic device or some component thereof (e.g., a display device or a camera module of the external electronic device). The device management application, additionally or alternatively, may support installation, delete, or update of an application running on the external electronic device.

<FIG> is a block diagram illustrating an example electronic device according to various embodiments of the disclosure.

Referring to <FIG>, the electronic device <NUM> (for example, the electronic device <NUM> of <FIG> or the electronic device <NUM> of <FIG>) may include a processor (e.g., including processing circuitry) <NUM> (for example, the processor <NUM> of <FIG>), a memory <NUM> (for example, the memory <NUM> of <FIG>), and a communication module (e.g., including communication circuitry) <NUM> (for example, the communication module <NUM> of <FIG>).

The processor <NUM> may include various processing circuitry and control an overall operation of the electronic device <NUM>. For example, the processor <NUM> may be operatively (or electrically) connected with the memory <NUM> and the communication module <NUM> to control the respective elements of the electronic device <NUM>.

The processor <NUM> may receive a command or instructions from the memory <NUM>, and may control the respective elements according to the received command or instructions to perform various functions. The processor <NUM> may include various processing circuitry including, for example, and without limitation, a central processing unit (CPU), an application processor (AP), a micro control unit (MCU), a micro processor unit (MPU), or the like. The processor <NUM> may be include a single core processor or a multi-core processor. In another embodiment, the processor <NUM> may be a multi-processor including a plurality of processors. In still another embodiment, the processor <NUM> may further include a graphic processing unit (GPU) and/or an image signal processor. In yet another embodiment, the processor <NUM> may configured with a GPU or an image signal processor.

The processor <NUM> according to an embodiment of the disclosure may control an operation of analyzing (or recognizing) a user input. A more detailed description thereof will be provided below with reference to <FIG>, <FIG>, <FIG>, <FIG> and <FIG>.

The memory <NUM> may be operatively (or electrically) connected with the processor <NUM>. The memory <NUM> may store various programs for operating the electronic device <NUM>, and may store data generated during execution of he various programs, or downloaded data. In addition, the memory <NUM> may store various commands and/or instructions for operating the processor <NUM>.

The memory <NUM> according to an embodiment of the disclosure may store a recognition module <NUM> for analyzing a user input, and a trained model <NUM> which is trained through an AI algorithm. The recognition module <NUM> and the trained model <NUM> are similar to the first recognition module <NUM> and the first trained model <NUM>, and thus a detailed description thereof is omitted.

The memory <NUM> according to an embodiment of the disclosure may store a time (hereinafter, a default time) (for example, <NUM> second) which is predefined to wait for reception of the result of predicting (or second control data) from an external electronic device. The default time may be changed by a user.

The memory <NUM> according to an embodiment of the disclosure may store a waiting time according to accuracy of the result of predicting (first control data) by the recognition module <NUM>. For example, the waiting time may be stored as in table <NUM> presented below:.

Table <NUM> presented above is merely an example and does not limit embodiments of the disclosure.

The communication module <NUM> may include various communication circuitry and establish a communication channel with an external electronic device (for example, the external electronic device <NUM> of <FIG>). The communication module <NUM> may transmit at least part of a user input to the external electronic device under control of the processor <NUM>. In addition, the communication module <NUM> may receive an analysis result (or second control data) of analyzing at least part of the user input from the external electronic device.

Although not shown in <FIG>, the electronic device <NUM> may further include at least one other element (for example, a display, a camera, an audio module, etc.) which is equivalent to the above-described elements. The display (for example, the display device <NUM> of <FIG>) may, under the control of the processor <NUM>, notify (or inform) that reception of the second control data is delayed when a result of predicting (second control data) is not received from the external electronic device within the default time, and may display a user interface (Ul) (for example, a pop-up window) asking whether a user will keep waiting for reception. The camera may obtain an image to be analyzed through the first recognition module <NUM>. The image obtained through the camera may be transmitted to the external electronic device through the communication module <NUM>, and may be analyzed by a second recognition module (for example, the second recognition module <NUM> of <FIG>, and <FIG>).

The audio module may obtain a user's voice through an input device (for example, the input device <NUM> of <FIG>) (for example, a microphone). The user's voice obtained through the microphone may be analyzed by the first recognition module <NUM>. In addition, the user's voice obtained through the microphone may be transmitted to the external electronic device through the communication module <NUM>, and may be analyzed by the second recognition module.

According to various embodiments of the disclosure, an electronic device (e.g., an electronic device <NUM>, <NUM>, or <NUM>) may comprise a memory (e.g., a memory <NUM>, or <NUM>) configured to store one or more applications, a communication module comprising communication circuitry (e.g., a communication module <NUM> or <NUM>) configured to communicate with an external electronic device (e.g., an external electronic device <NUM> or a server <NUM>), and a processor (e.g., a processor <NUM> or <NUM>). The processor is configured to control the electronic device to: receive an input of a user, generate first control data for controlling at least one application among the one or more applications using a first recognition method based at least on the input, transmit at least part of the input to the external electronic device through the communication module, wherein the external electronic device comprising circuitry configured to generate second control data for controlling the at least one application using a second recognition method based at least on the input, identify (determine) a time that passes until the second control data is received after the at least part of the input is transmitted to the external electronic device, control the at least one application using the first control data based on the passing time satisfying a first predefined condition, and control the at least one application using the second control data based on the passing time satisfying a second predefined condition.

According to various embodiments, the first recognition method may comprise a first trained model (e.g., a first trained model <NUM>, or a trained model <NUM>) which is trained based at least on a first artificial neural network, and the second recognition method may comprise a second trained model (e.g., a second trained model <NUM>) which is trained based at least on a second artificial neural network.

According to various embodiments, the processor may be further configured to control the electronic device to: determine that the first predefined condition is satisfied, based on the second control data being received after a predefined time or the second control data not being received in response to the at least part of the input being transmitted to the external electronic device through the communication module, and determine that the second predefined condition is satisfied based on the second control data being received within the predefined time in response to the at least part of the input being transmitted to the external electronic device through the communication module.

According to various embodiments, the processor is further configured to control the electronic device to identify an accuracy of the first control data, and to adjust the predefined time based at least on the accuracy of the first control data.

According to various embodiments, the processor may be further configured to control the electronic device to determine whether the first control data and the second control data are identical to each other in response to the second control data being received after the predefined time, and to refine the first trained model using the second control data as training data of the first recognition method when the first control data and the second control data are not identical to each other.

According to various embodiments, the processor may be configured to control the electronic device to determine whether the first control data and the second control data are identical to each other in response to the second predefined condition being satisfied, and control the at least one application using the second control data in response to the first control data and the second control data not being identical to each other, and to refine the first trained model using the second control data as training data of the first recognition method.

According to various embodiments, the electronic device further comprises a display (e.g., display device <NUM>). The processor may be further configured to control the display to display a user interface configured to receive an input for requesting whether to wait for reception of the second control data based on the predefined time passing after the at least part of the input is transmitted to the external electronic device through the communication module, and wait for reception of the second data for greater than the predefined time in response to a request for waiting for reception being selected by the user through the user interface.

According to various embodiments, the processor may be further configured to control the electronic device to identify a communication state with the external electronic device, and to not transmit the at least part of the user input to the external electronic device in response to the communication state satisfying a third predefined condition, and to generate the first control data using the first recognition method.

According to various embodiments of the disclosure, an electronic device (e.g., an electronic device <NUM>, <NUM>, or <NUM>) may comprise a communication module comprising communication circuitry (e.g., a communication module <NUM> or <NUM>) configured to communicate with an external electronic device (e.g., an external electronic device <NUM> or a server <NUM>), and a processor (e.g., a processor <NUM> or <NUM>). The processor is configured to control the electronic device to: receive an input of a user, generate first control data using a first recognition method based at least on the input, transmit at least part of the input to the external electronic device through the communication module, wherein the external electronic device comprises circuitry configured to generate second control data using a second recognition method based at least on the input, identify (determine) a time that passes until the second control data is received after the at least part of the input is transmitted to the external electronic device, control a first predefined function using the first control data based on the passing time satisfying a first predefined condition, and control a second predefined function using the second control data based on the passing time satisfying a second predefined condition.

According to various embodiments, the first predefined function and the second predefined function may be identical to each other.

According to various embodiments, based on the first control data and the second control data not being identical to each other, the processor may be further configured to control the second predefined function using the second control data, and to refine the first trained model using the second control data as training data of the first recognition method.

<FIG> is a flowchart illustrating a control method of an electronic device according to a claimed embodiment of the disclosure.

Referring to <FIG>, a processor (for example, the processor <NUM> of <FIG>, or the processor <NUM> of <FIG>) of the electronic device (for example, the electronic device <NUM> of <FIG>, the electronic device <NUM> of <FIG>, the electronic device <NUM> of <FIG>) receives a user input in operation <NUM>. The user input may be received in various methods. For example, the electronic device may receive a user's voice through a microphone (for example, the input device <NUM> of <FIG>). According to an embodiment, the electronic device may receive a user input of requesting recognition of an image obtained through a camera (for example, the camera module <NUM> of <FIG>), or a stored image. According to an example embodiment, the electronic device may receive a text input through a physical keyboard or a virtual keypad. According to an example embodiment, the electronic device may receive a writing input through an electronic pen or a touch screen.

In operation <NUM>, the processor generates first control data using a first recognition method, based at least on the input. The first control data is data for controlling at least one application among one or more applications, and is generated based at least on the user input. The first recognition method enables a first recognition module (for example, the first recognition module <NUM> of <FIG>, and <FIG>, or the recognition module <NUM> of <FIG>) to analyze (or predict) the user input using a first trained model (for example, the first trained model <NUM> of <FIG>, or the trained model <NUM> of <FIG>). The first recognition module may be an AI module. The first recognition module may have a high processing speed, but has a low accuracy in comparison to the second recognition module. For example, the number of neural networks, a size of a used memory, and/or the number of MAC operations in the first recognition module may be a predefined multiple or more (for example, about <NUM> times) smaller than in the second recognition module. The first trained model may be data which is trained by the first recognition module through an AI algorithm such as machine learning, neural network, gene, deep learning, classification algorithm, etc..

In operation <NUM>, the processor transmits at least part of the user input to an external electronic device (for example, the external electronic device <NUM> of <FIG>, or the server <NUM> of <FIG>). For example, the processor transmits at least part of the user input to the external electronic device through a communication module (for example, the communication module <NUM> of <FIG>, or the communication module <NUM> of <FIG>), such that the external electronic device generates second control data using a second recognition method based at least on the user input. The second control data is data for controlling at least one application among one or more applications. The second recognition method enables a second recognition module (for example, the second recognition module <NUM> of <FIG>) to analyze (or predict) the user input using a second trained model (for example, the second trained model <NUM> of <FIG>). The second recognition module may be an AI module. The second recognition module may have a low processing speed, but has a high accuracy in comparison to the first recognition module. For example, the number of neural networks, a size of a used memory, and/or the number of MAC operations in the second recognition module may be a predefined multiple or more (for example, about <NUM> times) larger than in the first recognition module. The second trained model may be data that is trained by the second recognition module through an AI algorithm such as machine learning, neural network, gene, deep learning, classification algorithm, or the like.

Operations <NUM> and <NUM> may be performed simultaneously or almost simultaneously in response to the user input being received.

In operation <NUM>, the processor according to an embodiment of the disclosure may identify (or check or determine) a time that passes until the second control data generated using the second recognition method is received from the external electronic device. For example, the processor may count up a time that is taken from the time that at least part of the user input is transmitted to the external electronic device until the time that the second control data is received.

In operation <NUM>, the processor according to an embodiment of the disclosure may determine whether a first predefined condition or a second predefined condition is satisfied.

When the first predefined condition is satisfied (for example, when the second control data is not received within a predefined time or when the second control data is received after the predefined time) as a result of determining in operation <NUM>, the processor may control at least one application using the first control data in operation <NUM>.

When the second predefined condition is satisfied (for example, when the second control data is received within the predefined time) as a result of determining in operation <NUM>, the processor may control at least one application using the second control data in operation <NUM>.

In <FIG>, it is illustrated that the application is controlled based on the user input. However, embodiments of the disclosure are not limited thereto. According to an embodiment, when the first predefined condition is satisfied, the electronic device according to an embodiment of the disclosure may control a first predefined function using the first control data, and, when the second predefined condition is satisfied, the electronic device may control a second predefined function using the second control data. The first predefined function and the second predefined function may be the same.

<FIG> is a flowchart illustrating an example control method of an electronic device according to an embodiment of the disclosure.

Referring to <FIG>, a processor (for example, the processor <NUM> of <FIG>, or the processor <NUM> of <FIG>) of the electronic device (for example, the electronic device <NUM> of <FIG>, the electronic device <NUM> of <FIG>, the electronic device <NUM> of <FIG>) according to an embodiment of the disclosure may receive a user input in operation <NUM>. The user input may be received in various methods.

In operation <NUM>, the processor according to an embodiment of the disclosure may determine whether a communication state with an external electronic device (for example, the external electronic device <NUM> of <FIG>, the server <NUM> of <FIG>) satisfies a predefined condition (hereinafter, a third predefined condition). The third predefined condition may be a condition in which communication between the electronic device and the external electronic device is impossible, or a communication speed is very low (for example, a reception signal strength of Wi-Fi or mobile communication is less than (or less than or equal to) a predefined value (for example, about -<NUM> dBm)).

According to an embodiment of the disclosure, when the processor is set to use only a Wi-Fi network, the processor may identify a communication state with the external electronic device through a Wi-Fi network. In another example, when the processor is set to use a Wi-Fi network and a mobile communication network (Wi-Fi+cellular), the processor may identify a communication state with the external electronic device through the Wi-Fi network, first, and, when the communication through the Wi-Fi network is impossible or a reception signal strength of the Wi-Fi network is less than a predefined value (for example, about -<NUM> dBm), the processor may identify a communication state with the external electronic device through the mobile communication network.

When the communication state with the external electronic device does not satisfy the third predefined condition (for example, when communication with the external electronic device is possible) as a result of determining in operation <NUM>, the processor may perform operations <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>. Operations <NUM> to <NUM> may correspond to operations <NUM> to <NUM> of <FIG>, respectively. Accordingly, a detailed description of operations <NUM> to <NUM> will not be repeated here.

When the communication state with the external electronic device satisfies the third predefined condition (for example, when communication with the external electronic device is impossible) as a result of determining in operation <NUM>, the processor may generate first control data using a first recognition method in operation <NUM>. In this example, since communication is impossible, the processor may not transmit at least part of the user input to the external electronic device. When generation of the first control data is completed, the processor may proceed to operation <NUM>.

Referring to <FIG>, a processor (for example, the processor <NUM> of <FIG>, or the processor <NUM> of <FIG>) of the electronic device (for example, the electronic device <NUM> of <FIG>, the electronic device <NUM> of <FIG>, or the electronic device <NUM> of <FIG>) according to an embodiment of the disclosure performs operations <NUM>, <NUM>, and <NUM> similarly to operations <NUM>, <NUM>, and <NUM> of <FIG>.

In operation <NUM>, the processor according to an embodiment of the disclosure adjusts (or determine) a reception waiting time of the second control data, based at least on accuracy (for example, probability) of the first control data. For example, as shown in table <NUM>, the processor identifies the accuracy of the first control data, and, when the accuracy of the first control data is higher than or equal to about <NUM>%, the processor may adjust the reception waiting time to a default waiting time (for example, <NUM> second) times <NUM>. When the accuracy is greater than or equal to about <NUM>% and less than about <NUM>%, the processor may adjust the reception waiting time to the default waiting time times <NUM>.

When the reception waiting time is adjusted, the processor identifies a time until the second control data is received from the external electronic device (for example, the external electronic device <NUM> of <FIG>, the server <NUM> of <FIG>) in operation <NUM>.

In operation <NUM>, the processor according to an embodiment of the disclosure determines whether the second control data is received from the external electronic device within the adjusted reception waiting time.

When the second control data is not received within the determined reception waiting time (when the first predefined condition is satisfied) as a result of determining in operation <NUM>, the processor controls at least one application using the first control data in operation <NUM>.

When the second control data is received within the determined reception waiting time (when the second predefined condition is satisfied) as a result of determining in operation <NUM>, the processor controls at least one application using the second control data in operation <NUM>.

Referring to <FIG>, a processor (the processor <NUM> of <FIG>, or the processor <NUM> of <FIG>) of the electronic device (for example, the electronic device <NUM> of <FIG>, the electronic device <NUM> of <FIG>, or the electronic device <NUM> of <FIG>) according to an embodiment of the disclosure may perform operations <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> similar to operations <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> of <FIG>.

When the first predefined condition is satisfied (for example, when the second control data is not received within the predefined time) as a result of determining in operation <NUM>, the processor may control a display (for example, the display device <NUM> of <FIG>) to display a user interface (UI) (for example, a pop-up window) asking whether the user will wait reception of the second control data in operation <NUM>.

In operation <NUM>, the processor according to an embodiment of the disclosure may determine whether waiting for reception of the second control data is selected (requested). For example, when "Yes" is selected on the pop-up window, the processor may recognize that waiting for reception of the second control data is requested.

When waiting for reception is selected as a result of determining in operation <NUM>, the processor may proceed to operation <NUM>. For example, the processor may wait for reception of the second control data the predefined time more. According to an example embodiment, the reception waiting time of the second control data may be different from the predefined time.

When waiting for reception is not selected as a result of determining in operation <NUM>, the processor may control at least one application using the first control data in operation <NUM>.

Referring to <FIG>, a processor (the processor <NUM> of <FIG>, or the processor <NUM> of <FIG>) of the electronic device (for example, the electronic device <NUM> of <FIG>, the electronic device <NUM> of <FIG>, or the electronic device <NUM> of <FIG>) according to an embodiment of the disclosure may perform operations <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> similarly to operations <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> of <FIG>.

When the first predefined condition is satisfied (for example, when the second control data is not received within the predefined time) as a result of determining in operation <NUM>, the processor may control at least one application using the first control data in operation <NUM>.

In operation <NUM>, the processor according to an embodiment of the disclosure may determine whether second control data is received after the predefined time passes. When the second control data is not received after the predefined time passes as a result of determining in operation <NUM>, the processor may finish the control operation of the electronic device. On the other hand, when the second control data is received after the predefined time passes as a result of determining in operation <NUM>, the processor may determine whether the first control data and the second control data are the same in operation <NUM>.

When the first control data and the second control data are the same as a result of determining in operation <NUM>, the processor may finish the control operation of the electronic device. On the other hand, when the first control data and the second control data are not the same as a result of determining in operation <NUM>, the processor may proceed to operation <NUM>, which will be described below.

When the second predefined condition is satisfied (for example, when the second control data is received within the predefined time) as a result of determining in operation <NUM>, the processor may determine whether the first control data and the second control data are the same in operation <NUM>.

When the first control data and the second control data are the same as a result of determining in operation <NUM>, the processor may proceed to operation <NUM>, which will be described below. On the other hand, when the first control data and the second control data are not the same as a result of determining in operation <NUM>, the processor may control at least one application using the second control data in operation <NUM>, and may refine a trained model (for example, the first trained model <NUM> of <FIG>, or the trained model <NUM> of <FIG>) using the second control data in operation <NUM>. For example, the processor may control a first recognition module (for example, the first recognition module <NUM> of <FIG>, and <FIG>, or the recognition module <NUM> of <FIG>) to refine the trained model using the second control data as training data.

According to various embodiments of the disclosure, a method of an electronic device (e.g., an electronic device <NUM>, <NUM>, or <NUM>) may comprise receiving an input of a user, generating first control data for controlling at least one application among one or more applications using a first recognition method, based at least on the input, transmitting at least part of the input to an external electronic device (e.g., an external electronic device <NUM> or a server <NUM>) through a communication module (e.g., a communication module <NUM> or <NUM>), wherein the external electronic device is configured to generate second control data for controlling the at least one application using a second recognition method based at least on the input, identifying a time that passes until the second control data is received after the at least part of the input is transmitted to the external electronic device in response to the passing time satisfying a first predefined condition, controlling the at least one application using the first control data, and controlling the at least one application using the second control data in response to the passing time satisfying a second predefined condition.

According to various embodiments, the generating the first control data using the first recognition method may comprise generating the first control data using a first trained model (e.g., a first trained model <NUM>, or a trained model <NUM>) which is trained based at least on a first artificial neural network, and the generating the second control data using the second recognition method may comprise generating the second control data using a second trained model (e.g., a second trained model <NUM>) which is trained based at least on a second artificial neural network.

According to various embodiments, the first predefined condition may comprise a condition in which the second control data is received after a predefined time passes, or the second control data is not received, and the second predefined condition may comprise a condition in which the second control data is received within the predefined time.

According to various embodiments, the method further comprises identifying accuracy of the first control data, and adjusting the predefined time based at least on the accuracy of the first control data.

According to various embodiments, the method may further comprise determining whether the first control data and the second control data are identical to each other in response to the second control data being received after the predefined time passes, and refining the first trained model using the second control data as training data of the first recognition method when the first control data and the second control data are not identical to each other.

According to various embodiments, the method may further comprise determining whether the first control data and the second control data are identical to each other in response to the second predefined condition being satisfied, and in response to the first control data and the second control data not being identical to each other, controlling the at least one application using the second control data, and refining the first trained model using the second control data as training data of the first recognition method.

According to various embodiments, the method may further comprise displaying a user interface for receiving an input for requesting whether to wait for reception of the second control data based on the predefined time passing after the at least part of the input is transmitted to the external electronic device through the communication module, and in response to a request for waiting for reception being selected by the user through the user interface, waiting for reception of the second data more for the predefined time.

According to various embodiments, the method may further comprise identifying a communication state with the external electronic device, not transmitting the at least part of the user input to the external electronic device in response to the communication state satisfying a third predefined condition, and generating the first control data using the first recognition method.

Various embodiments of the disclosure can provide a result of analyzing a user input within a predetermined time, regardless of a communication (or network) state between the electronic device and the external electronic device, and/or a load state of the external electronic device, and may control an application or a function according to the result of analyzing.

The electronic devices may include, for example, and without limitation, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, a home appliance, or the like.

It is to be understood that if an element (e.g., a first element) is referred to, with or without the term "operatively" or "communicatively", as "coupled with," "coupled to," "connected with," or "connected to" another element (e.g., a second element), the element may be coupled with the other element directly (e.g., wiredly), wirelessly, or via a third element.

As used herein, the term "module" may include a unit implemented in hardware, software, firmware, or any combinations thereof, and may interchangeably be used with other terms, for example, "logic," "logic block," "part," or "circuitry".

Wherein, the term "non-transitory" simply means that the storage medium is a tangible device, but this term does not differentiate between where data is semi-permanently stored in the storage medium and where the data is temporarily stored in the storage medium.

Claim 1:
An electronic device comprising:
a memory configured to store one or more applications;
a communication module comprising communication circuitry configured to communicate with an external electronic device; and
a processor,
wherein the processor is configured to control the electronic device to:
receive an input;
generate first control data for controlling at least one application among the one or more applications by analyzing the input using a first recognition method; and
transmit at least part of the input to the external electronic device through the communication module, wherein the external electronic device is configured to generate second control data for controlling the at least one application by analyzing the input using a second recognition method having higher accuracy than the first recognition method;
characterized in that the processor is configured to
identify accuracy of the first control data;
determine a reception waiting time of the second control data as a function of the identified accuracy of the first control data;
determine whether the second control data is received from the external electronic device within the determined reception waiting time;
control the at least one application using the first control data based on the determination satisfying a first predefined condition; and
control the at least one application using the second control data based on the determination satisfying a second predefined condition.