Electronic device and method for determining device for performing task by electronic device

An electronic device is provided. The electronic device includes a communication module, a memory, and a processor configured to obtain first information for performing a task, identify a first task corresponding to the first information, identify a first device meeting a plurality of conditions corresponding to the first task among a plurality of devices, and request the first device to perform the first task via the communication module. Various other embodiments may be provided.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a National Phase Entry of PCT International Application No. PCT/KR2021/005458, which was filed on Apr. 29, 2021, and claims priority to Korean Patent Application No. 10-2020-0141858, which was filed on Oct. 29, 2020, the contents of which are incorporated herein by reference.

TECHNICAL FIELD

The disclosure relates to an electronic device for determining the optimal device for performing a task corresponding to a user utterance among a plurality of devices and a method for determining a device for performing a task by the electronic device.

BACKGROUND ART

The Internet of things (IoT) refers to a technology in which various devices with built-in sensors and communication functions are connected to the Internet and may provide the function of monitoring and controlling each of various devices connected through the Internet.

DISCLOSURE

Technical Problem

Home IoT is used to simply monitor or control individual devices.

To use various devices in the home via a voice service, the user needs to designate a specific device for performing the service, as well as what is to be performed in the service.

Although various devices in the home include a device capable of performing the service better than the specific device designated by the user, the service requested by the user may be carried out only via the specific device designated by the user.

Aspects of the disclosure are to address at least the above-mentioned problems and/or disadvantages and to provide at least the advantages described below. Accordingly, an aspect of the disclosure is to provide an electronic device for determining the optimal device for performing a task corresponding to a user utterance among a plurality of devices and a method for determining a device for performing a task by the electronic device.

Another aspect of the disclosure is to provide an electronic device for providing a home modeling service by compiling contexts gathered in the home and a method for determining a device for performing a task by the electronic device.

Another aspect of the disclosure is to provide an electronic device that may determine the overall state of the home and provide the optimal service in a home multi-modality environment to thereby increase user convenience and a method for determining a device for performing a task by the electronic device.

Technical Solution

In accordance with an aspect of the disclosure, an electronic device is provided. The electronic device includes a communication module, a memory, and a processor configured to obtain and analyze a user utterance, obtain first information for performing a task, identify a first task corresponding to the first information, identify a first device meeting a plurality of conditions corresponding to the first task among a plurality of devices, and requesting the first device to perform the first task via the communication module.

In accordance with another aspect of the disclosure, a method for determining a device for performing a task by an electronic device is provided. The method includes obtaining first information for performing the task, identifying a first task corresponding to the first information, identifying a first device meeting a plurality of conditions corresponding to the first task among a plurality of devices, and requesting the first device to perform the first task via a communication module of the electronic device.

Advantageous Effects

According to the embodiments, an electronic device may determine the optimal device capable of performing a requested service among various devices in the home and enable the optimal device to perform the service requested by the user. The electronic device may determine an additional device to allow the user to conveniently control the optimal device performing the service corresponding to the request and to conveniently control the optimal device using the additional device. The electronic device may continuously provide the user with a service by determining a device capable of performing the service among various devices in the home.

The electronic device may provide a home modeling service by compiling contexts gathered in the home, determine the overall state of the home, and provide the optimal service in the home multi-modality environment, thereby increasing user convenience.

MODE FOR DISCLOSURE

FIG.1is a block diagram illustrating an electronic device in a network environment according to an embodiments of the disclosure. Referring toFIG.1, the electronic device101in the network environment100may communicate with an electronic device102via a first network198(e.g., a short-range wireless communication network), or an electronic device104or a server108via a second network199(e.g., a long-range wireless communication network). The electronic device101may communicate with the electronic device104via the server108. The electronic device101may include a processor120, memory130, an input module150, a sound output module155, a display module160, an audio module170, a sensor module176, an interface177, a connecting terminal178, a haptic module179, a camera module180, a power management module188, a battery189, a communication module190, a subscriber identification module (SIM)196, or an antenna module197. At least one (e.g., the connecting terminal178) of the components may be omitted from the electronic device101, or one or more other components may be added in the electronic device101. Some of the components (e.g., the sensor module176, the camera module180, or the antenna module197) may be integrated into a single component (e.g., the display module160).

The program140may be stored in the memory130as software, and may include an operating system (OS)142, middleware144, or an application146.

The input module150may receive a command or data to be used by other component (e.g., the processor120) of the electronic device101, from the outside (e.g., a user) of the electronic device101. The input module150may include a microphone, a mouse, a keyboard, keys (e.g., buttons), or a digital pen (e.g., a stylus pen).

The haptic module179may convert an electrical signal into a mechanical stimulus (e.g., a vibration or motion) or electrical stimulus which may be recognized by a user via his tactile sensation or kinesthetic sensation. The haptic module179may include a motor, a piezoelectric element, or an electric stimulator.

The camera module180may capture a still image or moving images. The camera module180may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module188may manage power supplied to the electronic device101. The power management module188may be implemented as at least part of, for example, a power management integrated circuit (PMIC).

The battery189may supply power to at least one component of the electronic device101. The battery189may include a primary cell which is not rechargeable, a secondary cell which is rechargeable, or a fuel cell.

The antenna module197may transmit or receive a signal or power to or from the outside (e.g., the external electronic device). The antenna module197may include one antenna including a radiator formed of a conductor or conductive pattern formed on a substrate (e.g., a printed circuit board (PCB)). The antenna module197may include a plurality of antennas (e.g., an antenna array). In this case, at least one antenna appropriate for a communication scheme used in a communication network, such as the first network198or the second network199, may be selected from the plurality of antennas by, e.g., the communication module190. The signal or the power may then be transmitted or received between the communication module190and the external electronic device via the selected at least one antenna. In addition to the radiator, the antenna module197may include other components, such as a radio frequency integrated circuit (RFIC).

FIG.2is a view200illustrating operations for controlling a plurality of devices in a home by an electronic device according to an embodiment of the disclosure.

Referring toFIG.2, a home may include a plurality of devices221to229and an electronic device201capable of managing the plurality of devices221to229.

The electronic device201may be a primary device capable of performing a home IoT edge function, and the electronic device201may manage the plurality of devices while transmitting and receiving information to/from each of the plurality of devices221to229, with communication established with the plurality of devices221to229.

The electronic device201and the plurality of devices221to229may include all or some of the components of the electronic device101ofFIG.1.

At least one of the plurality of devices221to229may be set as a primary device capable of performing a home IoT edge function.

At least one non-portable device (e.g., TV devices201,221, and222, a refrigerator223, and an artificial intelligence (AI) speaker225) among various devices201and221to229in the home may be set as the primary device capable of performing a home IoT edge function.

The electronic device201may receive device information from each of the plurality of devices221to229and store the device information.

The device information may include device configuration information (e.g., display information, speaker information, microphone information, and voice service support information) and device state information (e.g., power on/off information, information about the function being currently performed, and location information).

Upon receiving first information for performing a task, the electronic device201may identify a first task corresponding to the first information, determine a first device capable of performing the first task based on a plurality of conditions corresponding to the first task and the device information about each of the plurality of devices221to229, and request the first device to perform the first task.

Upon receiving state information according to performing the first task from the first device, the electronic device201may identify a second task for the first device, determine a second device capable of performing the second task among the plurality of devices221to229based on a plurality of conditions corresponding to the second task and the state information about the first device, and request the second device to perform the second task.

One of portable devices among the plurality of devices221to229, such as a watch phone227or a smartphone229, may be set as the second device.

FIG.3is a block diagram300illustrating an electronic device according to an embodiment of the disclosure.

Referring toFIG.3, an electronic device301(e.g., the electronic device101ofFIG.1) may include a processor320(e.g., the processor120ofFIG.1), a memory330(e.g., the memory130ofFIG.1), a display360(e.g., the display module160ofFIG.1), and a communication module390(e.g., the communication module ofFIG.1).

The processor320may control the overall operation of the electronic device301. The processor320may include a device manager unit310that may manage a plurality of devices (e.g., the plurality of devices221to229ofFIG.2) by the electronic device301, determine an optimal first device capable of performing a first task corresponding to first information for performing a task among the plurality of devices, and determine a second device capable of performing a second task for the first device.

The device manager unit310may be included in the processor320or be configured as a separate component.

The processor320(e.g., the device manager unit310) may obtain and analyze first information for performing a task and identify the first task based on the analyzed first information.

According to an embodiment, the processor320may obtain user utterance information and, upon identifying that the obtained user utterance information is the first information, identify the first task corresponding to the user utterance information.

For example, the processor320may identify that the user utterance information, e.g., “Direct me to a Chinese restaurant,” is the first information and identify the first task (e.g., navigation).

The processor320may obtain state change information about a device in the home, identify that the obtained state change information about the device in the home is the first information, and identify the first task corresponding to the state change information about the device in the home. For example, the processor320may identify that a state change of an air conditioner device from an off state to an on state is the first information and identify the first task (e.g., providing weather information) corresponding to the first information, i.e., the state change of the air conditioner device.

The processor320may obtain preset specific time information, identify that the specific time information is the first information, and identify the first task corresponding to the specific time information. For example, the processor320may identify that 2 PM every Monday as preset is the first information and identify the first task (e.g., viewing a TV show) corresponding to the first information, i.e., the specific time information.

The processor320may obtain state change information about the home, identify that the obtained state change information about the home is the first information, and identify the first task corresponding to the state change information about the home. For example, the processor320may obtain state change information (e.g., changes in temperature, humidity, and light intensity by a reference value or more or less) about the home based on sensor information gathered via at least one sensor mounted on the plurality of devices in the home or at least separate sensor provided in the home, identify that the obtained state change information about the home is the first information, and identify the first task (e.g., outputting the information about the current change in the temperature, humidity, or light intensity of the home) corresponding to the first information, i.e., the home state change information.

The processor320may obtain location information about the user in the home and identify that the obtained user location information is the first information. For example, upon identifying that the user enters a specific one of a plurality of rooms in the home, the processor320may identify that the location information about the user in the specific room is the first information and identify the first task (e.g., outputting schedule information for the user) corresponding to the first information, i.e., the location information about the user in the home.

The processor320(e.g., the device manager unit310) may gather device information received from each of the plurality of devices (e.g., the plurality of devices221to229ofFIG.2) in the home, managed by the electronic device301, and store the device information in the memory330.

The processor320(e.g., the device manager unit310) may gather the device information (e.g., device configuration information and device state information) received from each of the plurality of devices (e.g., the plurality of devices221to229) as shown in Table 1 below and store the device information in the memory330.

As shown in Table 1, the device information about the plurality of devices (device A to device F) may include device configuration information and device state information. The device configuration information may include necessary configuration information that is not changed, such as display resolution, volume information, and voice service support information. The device state information may include information that may vary depending on the context of the device, such as power state information and user activity information. The user activity information indicates information that may be obtained by the device (e.g., device E and device F) that is held by the user or may be worn. Table 1 above is merely an example of the device information; the device information about the plurality of devices may include various information, such as location information and information about the task being currently performed.

The processor320(e.g., the device manager unit310) may identify the first task based on the first information for performing a task and identify the type of the first task.

The processor320(e.g., the device manager unit310) may identify a plurality of conditions corresponding to the first task among a plurality of conditions set per type of task.

The plurality of conditions set per type of task may include a first condition for device configuration information for performing the task, a second condition for the device state information for performing the task, and third condition (information) for the number of devices for performing the task. The first to third conditions may be sequentially applied. The first condition may denote a constraint condition that needs to necessarily be included, the second condition may denote a preference condition for selecting more suitable device for performing the task, and the third condition may denote the number of devices selected from among the plurality of devices.

The processor320(e.g., the device manager unit310) may identify a plurality of conditions corresponding to the first task among a plurality of conditions set per type of task. The processor320may determine first candidate devices that meet the first condition for performing the first task among the plurality of devices (e.g., the plurality of devices221to229ofFIG.2). The processor320may determine second candidate devices that meet the second condition for performing the first task among the first candidate devices and determine the first device that meets the third condition for performing the first task among the second candidate devices.

For example, upon obtaining a user utterance, “Direct me to a nearby Chinese restaurant,” as the first information for performing the task, the processor320may identify that the first task is of the type of task for “navigation.” The processor320may identify a plurality of conditions corresponding to the first task for “navigation,” e.g., a first condition where the context of “power” exists, a second condition where the maximum volume is preferred, and a third condition where the number of devices for performing the first task is one, among the plurality of conditions set per type of task. The processor320may determine first candidate devices (e.g., devices A, B, D, E, and F) that meet the first condition which has the context of “power” among the plurality of devices (e.g., devices A to F) based on Table 1 above. The processor320may sort the first candidate devices (e.g., devices A, B, D, E, and F) in order from the highest volume to the lowest volume and determine second candidate devices (e.g., device A>device D=device F>device B>device E) that meet the second condition. As the second candidate device (e.g., device A) meeting the second condition meets the third condition where the number of devices for performing the first task is one, the processor320may determine that device A is the first device for performing the first task.

T processor320(e.g., the device manager unit310) may previously register a method in which each of the plurality of devices (e.g., the plurality of devices221to229ofFIG.2) may perform a task according to the type of the task. For example, the processor320may previously register a method in which the plurality of devices (e.g., the plurality of devices221to229ofFIG.2) may perform a task, e.g., “navigation,” as shown in Table 2 below.

TABLE 2Method for performing the task of “navigation”Device A (221)guide via audio and screenDevice B (222)guide via audio and screenDevice C (225)guide via screenDevice B (229)guide via audio

Upon determining a first device capable of performing the first task among the plurality of devices (e.g., the plurality of devices221to229ofFIG.2), the processor320(e.g., the device manager unit310) may request the first device to perform the first task. Upon performing the first task using an audio and screen by the first device (e.g., device A) and receiving state information indicating that the first task is performed from the first device, the processor320(e.g., the device manager unit310) may gather the state information as device information and store the device information in the memory330.

After requesting the first device to perform the first task corresponding to the first information for performing a task, the processor320(e.g., the device manager unit310) may determine whether to perform a second task for the first device.

Upon receiving the state information indicating that the first task is performed from the first device, the processor320(e.g., the device manager unit310) may determine whether to perform the second task and identify the second task for the first device depending on the determination as to whether to perform the second task. The processor320(e.g., the device manager unit310) may identify a plurality of conditions (e.g., the first to third conditions) for determining whether to perform the second task, and determine to perform the second task when all or at least some of the plurality of conditions are met. The processor320may determine to perform the second task when, among the plurality of conditions, the first condition where the first task corresponding to the first information for performing a task is irrelevant to the termination of a function of the first device (e.g., a user utterance to request to terminate the function of the first device) is met. The processor320may determine to perform the second task when, among the plurality of conditions, the second condition where the number of functions (e.g., a display function, speaker function, or voice support function) that may be performed by the first device is a predetermined number or more based on the configuration information about the first device is met. The processor320may determine to perform the second task when among the plurality of conditions, the third condition where the first device is a non-portable device is met.

Upon determining to perform the second task, the processor320(e.g., the device manager unit310) may identify a second task for performing an additional service for the first task performed by the first device and/or a second task for controlling the function of the first device.

Upon determining to perform the second task for the first device after requesting the first device to perform the first task corresponding to the first information for performing a task, the processor320(e.g., the device manager unit310) may determine the second device to perform the first task. The processor320(e.g., the device manager unit310) may identify a plurality of conditions corresponding to the second task among a plurality of conditions set per type of task.

The processor320(e.g., the device manager unit310) may identify a plurality of conditions corresponding to the second task among a plurality of conditions set per type of task. The processor320may determine first candidate devices that meet the first condition for performing the second task among the plurality of devices (e.g., the plurality of devices221to229ofFIG.2). The processor320may determine second candidate devices that meet the second condition for performing the second task among the first candidate devices and determine the second device that meets the third condition for performing the second task among the second candidate devices.

For example, upon receiving state information indicating that the first task (e.g., directing to a nearby Chinese restaurant) is performed from the first device (e.g., device A221or a TV device), the processor320(e.g., the device manager unit310) may identify a second task for displaying menu information (e.g., menu information for an additional search for nearby Chinese restaurants) for performing an additional service for the first task performed by the first device and/or a second task for displaying menu information (e.g., a TV remote controller menu) for controlling the function (e.g., the TV function of device A) of the first device. The processor320may identify a first condition where the context of display exists, and user activity information is included, a second condition where the user activity information indicates a lowest level, and a third condition where the number of devices for performing the second task is one, among the plurality of conditions corresponding to the second task. The processor320may determine first candidate devices (e.g., devices F and E) which are portable devices meeting the first condition where the context of display exists and user activity information is included among the plurality of devices (devices A to F) based on Table 1 above. The processor320may determine device F (e.g., the smartphone229) meeting the second condition where the user activity information indicates a lowest level and the third condition where the number of devices for performing the second task is one, as the second device for performing the second task.

The processor320(e.g., the device manager unit310) may request the second device to perform the second task, gather state information indicating that the second task is performed, received from the second device, as device information, and store the device information in the memory330.

The processor320(e.g., the device manager unit310) may previously register a method in which each of portable devices (e.g., devices E and F) among the plurality of devices (e.g., the plurality of devices221to229ofFIG.2) may perform the second task according to the type of non-portable devices (e.g., devices A to D) as shown in Table 3 below.

TABLE 3PortableNon-portabledevicesdevicescapable ofcapable ofperformingperformingsecond taskfirst taskMethod for performing second taskdevice Edevice Adisplay menu information for performing(227)second task by device Adevice B-[No registered action]device Cdisplay menu information for performingsecond task by device Cdevice Ddisplay menu information for performingsecond task by device Ddevice Fdevice Adisplay menu information and activate voice(223)service to perform second task by device Adevice Bdisplay menu information and activate voiceservice to perform second task by device Bdevice Cdisplay menu information and activate voiceservice to perform second task by device Cdevice Ddisplay menu information and activate voiceservice to perform second task by device D

Upon receiving information about another device, as changed, for performing the first task from the second device that has been requested to perform the second task, the processor320(e.g., the device manager unit310) may request the other device, among the plurality of devices (e.g., the plurality of devices221to229ofFIG.2), to perform the first task. Upon receiving information about another device selected by the user from the second device capable of providing a service for changing the device for performing the first task, the processor320(e.g., the device manager unit310) may request the first device to terminate performing the first task and request the other device to perform the first task. The processor320may request the second device to perform the second task for the other device. For example, the processor320may request the first device (e.g., a TV device) to perform the first task and request the second device (e.g., a smartphone) to perform the second task (e.g., display a remote controller menu for controlling the TV) for the first device. Upon receiving information about another device (e.g., a refrigerator) selected by the user from the second device that may provide a service for changing the device for performing the first task, the processor320may request the first device (e.g., a TV device) to terminate performing the first task and request the other device (e.g., a refrigerator) to perform the first task. The processor320may request the second device (e.g., a smartphone) to perform the second task (e.g., display a remote controller menu for controlling the refrigerator) for the other device (e.g., a refrigerator).

The memory330may store the device information gathered by and received from the plurality of devices (e.g., the plurality of devices221to229ofFIG.2) in the home, which are managed by the electronic device301(e.g., the electronic device201ofFIG.2).

The memory330may store device information gathered by and received from the plurality of devices (e.g., the plurality of devices221to229ofFIG.2) as shown in Table 1 above.

The memory330may store a previously registered method in which the plurality of devices (e.g., the plurality of devices221to229ofFIG.2) may perform a task according to the type of the task as shown in Table 2 above.

The memory330may store a previously registered method in which portable devices (e.g., devices E and F) among the plurality of devices (e.g., the plurality of devices221to229ofFIG.2) may perform the second task according to the type of non-portable devices (e.g., devices A to D) as shown in Table 3 above.

The memory330may store a plurality of conditions set per type of task.

The display360may display the function being currently performed by the electronic device301under the control of the processor320.

The display360may display information about the first device determined to perform the first task corresponding to the first information for performing a task among the plurality of devices (e.g., the plurality of devices221to229ofFIG.2) and/or information about the second device determined to perform the second task for the first device.

The communication module390may include a plurality of communication circuits using different communication technologies.

The communication module390may include a mobile communication module (not shown) or a sub communication module (not shown) performing short-range communication with a wireless LAN. The communication module390may perform communication with an external device using at least one antenna (not shown) under the control of the processor320.

The communication module390may include at least one of a wireless LAN module (not shown) and a short-range communication module (not shown) and may include a near-field communication (NFC) communication module, Bluetooth legacy communication module, and/or a Bluetooth low energy (BLE) communication module as the short-range communication module.

FIG.4is a block diagram400illustrating a device manager unit of an electronic device according to an embodiment of the disclosure.

Referring toFIG.4, a device manager unit310may include a device gathering unit311and a device detecting unit313.

The device gathering unit311may gather device information about each of a plurality of devices (e.g., the plurality of devices221to229ofFIG.2) in the home, managed by an electronic device (e.g., the electronic device201ofFIG.2and/or the electronic device301ofFIG.3).

The device gathering unit311may gather and store the device configuration information received from each of the plurality of devices when each of the plurality of devices is connected to the electronic device.

The device gathering unit311may gather and store the device state information received from each of the plurality of devices whenever state information about each of the plurality of devices is changed.

The device gathering unit311may gather and store the device information about each of the plurality of devices as shown in Table 1 above.

The device gathering unit311may provide the device information gathered from each of the plurality of devices to the device detecting unit313.

The device detecting unit313may identify a first task corresponding to first information for performing a task, identify a first device to perform the first task among a plurality of devices (e.g., the plurality of devices221to229ofFIG.2), identify a second task for the first device, and determine a second device to perform the second task among the plurality of devices.

Upon obtaining a user utterance as the first information, the device detecting unit313may parse and analyze the user utterance and identify the first task based on the information analyzed for the user utterance.

The device detecting unit313may identify a plurality of conditions corresponding to the first task among a plurality of conditions set per type of task, determine the first device meeting the plurality of conditions corresponding to the first task among the plurality of devices, and request the first device to perform the first task.

The device detecting unit313may identify the second task for the first device performing the first task, identify a plurality of conditions corresponding to the second task among the plurality of conditions set per type of task, determine the second device meeting the plurality of conditions corresponding to the second task among the plurality of devices, and request the second device to perform the second task.

According to an embodiment of the disclosure, an electronic device (e.g., the electronic device101ofFIG.1, the electronic device201ofFIG.2, the electronic device301ofFIG.3, or the electronic device501ofFIG.5) may comprise a communication module (e.g., the communication module390ofFIG.3), a memory (e.g., the memory330ofFIG.3), and a processor (e.g., the processor122ofFIG.1or the processor320ofFIG.3) configured to obtain first information for performing a task, identify a first task corresponding to the first information, identify a first device meeting a plurality of conditions corresponding to the first task among a plurality of devices, and request the first device to perform the first task via the communication module.

The processor may be configured to store device information gathered from the plurality of devices in the memory.

The processor may be configured to identify a plurality of conditions corresponding to the first task among a plurality of conditions set per type of task and determine the first device meeting the plurality of conditions corresponding to the first task among the plurality of devices based on device information gathered from the plurality of devices stored in the memory.

The processor may be configured to identify first candidate devices meeting a first condition indicating a constraint condition for device configuration information for performing the first task among the plurality of devices, identify second candidate devices meeting a second condition indicating a preference condition for device state information for performing the first task among the first candidate devices, and identify the first device meeting a third condition for information about a number of devices for performing the first task among the second candidate devices.

The processor may be configured to gather state information indicating that the first task is performed, received from the first device, as device information and store the device information in the memory.

The processor may be configured to identify a second task for the first device, identify a second device meeting a plurality of conditions corresponding to the second task among the plurality of devices, and request the second device to perform the second task via the communication module.

The processor may be configured to identify the second task for performing an additional service for the first task performed by the first device or the second task for controlling a function of the first device.

The processor may be configured to identify the second device meeting the plurality of conditions corresponding to the second task among the plurality of devices based on device information gathered from the plurality of devices stored in the memory of the electronic device.

The processor may be configured to identify first candidate devices meeting a first condition for user activity information and device configuration information for performing the second task among the plurality of devices, identify second candidate devices meeting a second condition for device state information for performing the second task among the first candidate devices, and identify the second device meeting a third condition for information about a number of devices for performing the second task among the second candidate devices.

The processor may be configured to gather state information indicating that the second task is performed, received from the second device, as device information and store the device information in the memory of the electronic device.

FIG.5is a flowchart500illustrating operations for determining a device for performing a task by an electronic device according to an embodiment of the disclosure.

Referring toFIG.5, the operations for determining a device for performing a task may include operations501to521. At least one of operations501to521may be omitted or changed in order or may add other operations.

In operation502, an electronic device501(e.g., the electronic device101ofFIG.1, the electronic device201ofFIG.2, or the electronic device301ofFIG.3) may obtain first information for performing a task.

The electronic device501may obtain at least one of user utterance information, state change information about a device in the home, specific time information, state change information about the home, or user location information in the home, as the first information for performing a task. For example, when the first information is user utterance information, the electronic device501may issue a user utterance through the microphone of the electronic device or receive a user utterance from a home device connected via communication with the electronic device501.

In operation503, the electronic device501may identify first information for performing a task and identify a first task corresponding to the first information.

Upon identifying that the first information is user utterance information, the electronic device501may parse the user utterance and identify the first task to be performed by the user.

Upon identifying that the first information is state change information about a device in the home, the electronic device501may identify a first task (e.g., provide today's weather information) corresponding to the state change (e.g., a change of the air conditioner device from an off state to an on state) in the home device.

Upon identifying that the first information is specific time information, the electronic device501may identify a first task (e.g., viewing a TV show) to be performed at the specific time (e.g., 2 PM every Monday).

Upon identifying that the first information is state change information about the home, the electronic device501may identify a first task (e.g., outputting information about, e.g., a current change in temperature, humidity, or light intensity in the home) corresponding to the home state change information (e.g., a current change in the temperature, humidity, or light intensity of the home by a reference value or more or less).

Upon identifying that the first information is location information about the user in the home, the electronic device501may identify a first task (e.g., outputting user schedule information) corresponding to the location information (e.g., the location information about the user in a specific room) about the user in the home.

In operation505, the electronic device501may determine a first device to perform the first task among the plurality of devices.

The electronic device501may identify a plurality of conditions corresponding to the first task among a plurality of conditions set per type of task and determine the first device521meeting the plurality of conditions corresponding to the first task among the plurality of devices (e.g., the plurality of devices221to229ofFIG.2) in the home, managed by the electronic device501.

The electronic device501may determine the first device521meeting the plurality of conditions corresponding to the first task by sequentially applying the plurality of conditions corresponding to the first task.

In operation507, the electronic device501may request the first device521to perform the first task.

In operation509, the first device521may perform the first task. The first device521may perform the first task according to the request from the electronic device501.

In operation511, the first device521may transmit state information indicating that the first task is performed to the electronic device501.

The first device521may transmit state information indicating that the first task is currently performed by the first device521and/or state information indicating that the first task has been completely performed to the electronic device501.

In operation513, the electronic device501may identify a second task for the first device521.

Upon receiving the state information indicating that the first task is performed from the first device521, the electronic device501may determine whether to perform the second task and, upon determining to perform the second task, identify the second task for the first device.

The electronic device501may identify a plurality of conditions (e.g., the first to third conditions) for determining whether to perform the second task and, when all or at least some of the plurality of conditions are met, determine to perform the second task.

The plurality of conditions may include a first condition where the first task corresponding to the first information for performing a task is irrelevant to terminating the function of the first device (e.g., a user utterance requesting to terminate the function of the first device), a second condition where the number of functions (e.g., a display function, speaker function, or voice support function) that may be performed by the first device is a predetermined number or more based on the configuration information about the first device, and a third condition where the first device is a non-portable device.

Upon determining to perform the second task for the first device, the electronic device501may identify a second task for an additional service for the first device performed by the first device521and/or a second task for controlling the function of the first device521.

In operation515, the electronic device501may identify the second device523to perform the second task among the plurality of devices.

The electronic device501may identify a plurality of conditions corresponding to the second task among a plurality of conditions set per type of task and determine the second device523meeting the plurality of conditions corresponding to the second task among the plurality of devices (e.g., the plurality of devices221to229ofFIG.2) in the home, managed by the electronic device501.

The electronic device501may determine the second device523meeting the plurality of conditions corresponding to the second task by sequentially applying the plurality of conditions corresponding to the first task.

In operation517, the electronic device501may request the second device523to perform the second task.

In operation519, the second device523may perform the second task. The second device523may perform the second task according to the request from the electronic device501.

In operation521, the second device523may transmit state information indicating that the second task is performed to the electronic device501.

Upon receiving information about another device selected by the user from the second device523capable of providing a service for changing the device for performing the first task, the electronic device501may request the first device521to terminate performing the first task and request the other device to perform the first task. The electronic device501may request the second device523to perform the second task for the other device.

FIG.6is a flowchart600illustrating operations for determining a device for performing a task by an electronic device according to an embodiment of the disclosure.

Referring toFIG.6, the operations for determining a device for performing a task may include operations601to621. At least one of the operations601to621may be omitted or changed in order or may add other operations. The operations for determining a device for performing a task may be performed by the electronic device101ofFIG.1, the electronic device201ofFIG.2, the electronic device301ofFIG.3, the processor320ofFIG.3, or the electronic device501ofFIG.5.

In operation601, the electronic device301may obtain first information for performing a task.

The electronic device301may obtain at least one of user utterance information, state change information about a device in the home, specific time information, state change information about the home, or user location information in the home, as the first information for performing a task. For example, when the first information is user utterance information, the electronic device501may issue a user utterance through the microphone of the electronic device or receive a user utterance from a home device connected via communication with the electronic device501.

In operation603, the electronic device301may identify first information for performing a task and identify a first task corresponding to the first information.

Upon identifying that the first information is user utterance information, the electronic device301may parse the user utterance and identify the first task to be performed by the user.

Upon identifying that the first information is state change information about a device in the home, the electronic device301may identify a first task (e.g., provide today's weather information) corresponding to the state change (e.g., a change of the air conditioner device from an off state to an on state) in the home device.

Upon identifying that the first information is specific time information, the electronic device301may identify a first task (e.g., viewing TV show) to be performed at the specific time (e.g., 2 PM every Monday).

Upon identifying that the first information is state change information about the home, the electronic device301may identify a first task (e.g., outputting information about, e.g., a current change in temperature, humidity, or light intensity in the home) corresponding to the home state change information (e.g., a current change in the temperature, humidity, or light intensity of the home by a reference value or more or less).

Upon identifying that the first information is location information about the user in the home, the electronic device301may identify a first task (e.g., outputting user schedule information) corresponding to the location information (e.g., the location information about the user in a specific room) about the user in the home.

In operation605, the electronic device301may identify a plurality of conditions corresponding to the first task.

The electronic device601may identify a plurality of conditions corresponding to the first task among a plurality of conditions set per type of task.

In operation607, the electronic device301may determine a first device meeting the plurality of conditions corresponding to the first task among the plurality of devices.

The electronic device301may identify first candidate devices meeting the first condition for the device configuration information for performing the first task among the plurality of devices (e.g., the plurality of devices221to229ofFIG.2) in the home, managed by the electronic device301. The electronic device301may identify second candidate devices meeting the second condition for the device state information for performing the first task among the plurality of first candidate devices. The electronic device301may determine the first device meeting the third condition for the information about the number of devices for performing the first task among the second candidate devices.

In operation609, the electronic device301may request the first device to perform the first task.

In operation611, the electronic device301may receive state information indicating that the first task is performed from the first device.

The electronic device301may gather the state information received from the first device, as device information, and store the device information.

In operation613, the electronic device301may identify a second task for the first device.

Upon receiving the state information indicating that the first task is performed, the electronic device301may identify a second task for an additional service for the first device performed by the first device and/or a second task for controlling the function of the first device.

Upon receiving the state information indicating that the first task is performed from the first device, the electronic device301may determine whether to perform the second task and, upon determining to perform the second task, identify the second task for the first device.

The electronic device301may identify a plurality of conditions (e.g., the first to third conditions) for determining whether to perform the second task and, when all or at least some of the plurality of conditions are met, determine to perform the second task. The plurality of conditions may include a first condition where the first task corresponding to the first information for performing a task is irrelevant to terminating the function of the first device (e.g., a user utterance requesting to terminate the function of the first device), a second condition where the number of functions (e.g., a display function, speaker function, or voice support function) that may be performed by the first device is a predetermined number or more based on the configuration information about the first device, and a third condition where the first device is a non-portable device.

Upon determining to perform the second task for the first device, the electronic device301may identify a second task for an additional service for the first device performed by the first device and/or a second task for controlling the function of the first device.

In operation615, the electronic device301may identify a plurality of conditions corresponding to the second task.

The electronic device301may identify a plurality of conditions corresponding to the second task among a plurality of conditions set per type of task.

In operation617, the electronic device301may determine a second device meeting the plurality of conditions corresponding to the second task among the plurality of devices.

The electronic device301may identify first candidate devices meeting the first condition for the user activity information and device configuration information for performing the second task among the plurality of devices (e.g., the plurality of devices221to229ofFIG.2) in the home, managed by the electronic device301. The electronic device301may identify second candidate devices meeting the second condition for the device state information for performing the second task among the first candidate devices. The electronic device301may determine the second device meeting the third condition for the information about the number of devices for performing the second task among the second candidate devices.

In operation619, the electronic device301may request the second device to perform the second task.

In operation621, the electronic device301may receive state information indicating that the second task is performed from the second device.

The electronic device301may gather the state information received from the second device, as device information, and store the device information.

Upon receiving information about another device selected by the user from the second device capable of providing a service for changing the device for performing the first task, the electronic device301may request the first device to terminate performing the first task and request the other device to perform the first task. The electronic device may request the second device to perform the second task for the other device.

FIG.7is a flowchart700illustrating the operation of determining whether to perform a second task by an electronic device according to an embodiment of the disclosure. Referring toFIG.7, the operations for determining a device for performing a task may include operations701to709. At least one of operations701to709may be omitted or changed in order or may add other operations. The operations for determining a device for performing a task may be performed by the electronic device101ofFIG.1, the electronic device201ofFIG.2, the electronic device301ofFIG.3, the processor320ofFIG.3, or the electronic device501ofFIG.5.

In operation701, the electronic device301may identify whether, among the plurality of conditions, the first condition where the first task corresponding to the first information for performing a task is irrelevant to the termination of a function of the first device (e.g., a user utterance to request to terminate the function of the first device) is met, so as to determine to perform the second task for the first device.

Upon identifying that the first task is a task to terminate the function (or operation) of the first device in operation701, the electronic device301may determine that the first condition is not met and, in operation711, determine not to perform the second task.

Upon determining that the first condition is met in operation701, the electronic device301may identify whether, among the plurality of conditions, the second condition where the number of functions (e.g., a display function, speaker function, or voice support function) that may be performed by the first device is a predetermined number or more based on the configuration information about the first device is met in operation703.

Upon identifying that the number of functions that may be performed by the first device is a predetermined number or less in operation703, the electronic device301may determine that the second condition is not met and, in operation711, determine not to perform the second task.

Upon determining that the second condition is met in operation703, the electronic device301may identify whether, among the plurality of conditions, the third condition where the first device is a non-portable device is met in operation705.

Upon identifying that the first device is a portable device (e.g., a watch phone or a smartphone) in operation705, the electronic device301may determine that the third condition is not met and, in operation711, determine not to perform the second task.

When the third condition is met in operation705, the electronic device301may determine to perform the second task in operation707.

In operation709, upon determining to perform the second task, the electronic device301may identify the second task for the first device.

FIG.8is a view800illustrating the operation of determining a device for performing a task by an electronic device according to an embodiment of the disclosure.

Referring toFIG.8, upon obtaining a user utterance811a, e.g., “Show me photos recently taken,” as first information for performing a task, the electronic device801(e.g., the electronic device101ofFIG.1, the electronic device201ofFIG.2, or the electronic device301ofFIG.3) may analyze the user utterance and identify a first task of “displaying photos.”

The electronic device801may identify a first condition where the context of power exists, a second condition where the highest display resolution is preferred, and a third condition where the number of devices to perform the first task is one as a plurality of conditions corresponding to the first task among a plurality of conditions set per type of task.

The electronic device801may determine first candidate devices (e.g., devices A, B, D, E, and F) meeting the first condition where the context of power exists among the plurality of devices (e.g., devices A to F) in the home, managed by the electronic device801, by referring to Table 1 above. The electronic device801may determine second candidate devices (e.g., device A>device B=device F>device E) meeting the second condition in order from the highest display resolution to the lowest display resolution and having the context of display among the first candidate devices (e.g., devices A, B, D E, and F). As the second candidate device (e.g., device A) meeting the second condition meets the third condition where the number of devices for performing the first task is one, the electronic device801may determine that device A is the first device for performing the first task.

The electronic device801may request the first device821to perform the first task (811b).

The first device821may receive recent photos from a server or other devices according to a request from the electronic device801and display the recent photos on the display of the first device821.

The electronic device801may receive state information indicating that the first task is performed from the first device821(811c).

Upon receiving the state information indicating that the first task is performed from the first device821, the electronic device801may identify a second task for performing an additional service for the first task (e.g., displaying recent photos) performed by the first device821or a second task for controlling the function of the first device821. The electronic device801may identify a second task to display menu information for providing an additional service (e.g., a photo search service) for the first task (e.g., displaying recent photos) by the first device821. The electronic device801may identify a second task to display menu information for controlling the original function (e.g., TV function) by the first device821.

The electronic device801may identify a first condition where the context of display exists and user activity information exists, a second condition where the maximum volume is preferred, and a third condition where the number of devices for performing the second task is one, as a plurality of conditions corresponding to the second task, among the plurality of conditions set per type of task.

The electronic device801may determine first candidate devices (e.g., devices E to F) meeting the first condition where user activity information exists and the context of display is included among the plurality of devices (e.g., devices A to F) in the home, managed by the electronic device801, by referring to Table 1 above. The electronic device801may determine a second candidate device (e.g., device F) meeting the second condition where the maximum volume is preferred among the first candidate devices (e.g., devices E and F). As the second candidate device (e.g., device F) meets the third condition where the number of devices to perform the task is one, the electronic device801may determine that the second candidate device (e.g., device F) is the second device829.

The electronic device801may request the second device829to perform the second task (811d).

The second device829may perform the second task (e.g., displaying menu information to provide an additional service (e.g., a photo search service) for the first task (e.g., displaying recent photos) and/or displaying menu information to control the original function (e.g., TV function) by the first device821) according to a request from the electronic device801, and then transmit state information related to the second task to the electronic device801(811e). The user may control the first device821using the second device829(811f).

FIG.9is a view900illustrating the operation of determining a device for performing a task by an electronic device according to an embodiment of the disclosure.

Referring toFIG.9, upon obtaining a user utterance911a, e.g., “Direct me to a nearby Chinese restaurant,” as first information for performing a task, the electronic device901(e.g., the electronic device101ofFIG.1, the electronic device201ofFIG.2, or the electronic device301ofFIG.3) may analyze the user utterance and identify a first task of “navigation.”

The electronic device901may identify a first condition where the context of power exists and the voice service is not supportable, a second condition where the maximum volume is preferred, and a third condition where the number of devices for performing the first task is one, as a plurality of conditions corresponding to the first task, among the plurality of conditions set per type of task.

The electronic device901may determine first candidate devices (e.g., devices A, B, D, E, and F) meeting the first condition where the context of power exists among the plurality of devices (e.g., devices A to F) in the home, managed by the electronic device901, by referring to Table 1 above. The electronic device801may sort the first candidate devices (e.g., devices A, B, D, E, and F) in order from the highest volume to the lowest volume and determine second candidate devices (e.g., device A>device D=device F>device B>device E) that meet the second condition. As the second candidate device (e.g., device A) meeting the second condition meets the third condition where the number of devices for performing the first task is one, the electronic device901may determine that the second candidate device (e.g., device A921) is the first device.

The electronic device901may request the first device921to perform the first task (911b).

The first device921may receive the recent information about the direction to the Chinese restaurant closest to the user's home from a servicer or other devices according to a request from the electronic device901and perform the first task of outputting the recent information about the direction to the Chinese restaurant closest to the user's home through the speaker of the first device921.

The electronic device901may receive state information indicating that the first task is performed from the first device921(911c).

Upon receiving the state information indicating that the first task is performed from the first device921, the electronic device901may identify a second task for performing an additional service for the first task (e.g., directing to the nearest Chinese restaurant) performed by the first device921and/or a second task for controlling the function of the first device921. The electronic device901may identify a second task to display menu information for providing an additional service (e.g., an additional search service for the nearest Chinese restaurant) for the first task (e.g., directing to the nearest Chinese restaurant) by the first device921. The electronic device901may identify a second task to display menu information for controlling the original function (e.g., TV function) by the first device921.

The electronic device901may identify a first condition where the context of display exists and user activity information exists, a second condition where the current user activity information indicates the lowest level, and a third condition where the number of devices for performing the second task is one, as a plurality of conditions corresponding to the second task, among the plurality of conditions set per type of task.

The electronic device901may determine first candidate devices (e.g., devices E to F) meeting the first condition where current user activity information exists and the context of display exists among the plurality of devices (e.g., devices A to F) in the home, managed by the electronic device901, by referring to Table 1 above. The electronic device901may determine a second candidate device (e.g., device F) meeting the second condition where the current user activity information indicates the lowest level among the first candidate devices (e.g., devices E and F). As the second candidate device (e.g., device F) meets the third condition where the number of devices to perform the task is one, the electronic device901may determine that the second candidate device (e.g., device F) is the second device929.

The electronic device901may request the second device929to perform the second task (911d).

The second device929may display menu information corresponding to the second task on the display of the second device according to a request from the electronic device901.

The electronic device901may receive state information indicating that the second task is performed from the second device929(911e).

The second device929may perform the second task (e.g., displaying menu information for providing an additional service (e.g., an additional search service for the nearest Chinese restaurant) for the first task (e.g., directing to the nearest Chinese restaurant) and/or displaying menu information for controlling the original function (e.g., TV function) by the first device721) according to a request from the electronic device901, and the user may control the first device921using the second device929(9110.

In addition to the embodiments described above in connection withFIGS.8and9, the electronic device (e.g., the electronic device301ofFIG.3) may identify a state change (e.g., a change of the air conditioner device from an off state to an on state) in the home device, as the first information for performing a task and, upon identifying the first task (e.g., providing today's weather information) corresponding to the information about the state change in the home device, perform the first task via the first device (e.g., a speaker) among the plurality of devices in the home.

The electronic device (e.g., the electronic device301ofFIG.3) may identify specific time information (e.g., 2 PM every Monday) as the first information for performing a task and, upon identifying the first task (e.g., view TV show) corresponding to the specific time information, perform the first task via the first device (e.g., TV device) among the plurality of devices in the home and perform the second task (e.g., displaying a remote controller menu for controlling the TV) via the second device (e.g., a smartphone).

The electronic device (e.g., the electronic device301ofFIG.3) may identify state change information about the home (e.g., a current change in the temperature, humidity, or light intensity of the home by a reference value or more or less) as the first information for performing a task and, upon identifying the first task (e.g., outputting information about the current change in the temperature, humidity, or light intensity of the home) corresponding to the state change information about the home, perform the first task via the first device (e.g., a speaker) among the plurality of devices in the home.

The electronic device (e.g., the electronic device301ofFIG.3) may identify the information about the location of the user in the home as the first information for performing a task and, upon identifying the first task (e.g., outputting the user's schedule information) corresponding to the information about the location of the user in the home, perform the first task via the first device (e.g., a speaker) located in a specific room where the user is located among the plurality of devices in the home and perform the second task (e.g., displaying a remote controller menu for controlling the TV) via the second device (e.g., a smartphone).

FIGS.10A and10Bare views1000aand1000billustrating the operation of switching devices for performing a task by a second device according to various embodiments of the disclosure.

Referring toFIG.10A, when the user selects another device (e.g., a refrigerator) for performing a first task (e.g., displaying the photos taken yesterday) by selecting a device switch menu1003for providing a service for changing the device for performing the first task while performing a second task (e.g., displaying a remote controller menu for controlling the TV) for the first device (e.g., a TV device), a second device1023(e.g., the smartphone229ofFIG.2) may transmit information about the other device (e.g., a refrigerator) selected by the user to an electronic device (e.g., the electronic device101ofFIG.1, the electronic device201ofFIG.2, or the electronic device301ofFIG.3).

Upon receiving the information about the other device (e.g., a refrigerator) selected by the user from the second device1023, the electronic device may request the first device (e.g., a TV device) to terminate performing the first task (e.g., displaying the photos taken yesterday) and request the other device (e.g., a refrigerator) to perform the first task (e.g., displaying the photos taken yesterday). The electronic device may request the second device1023(e.g., a smartphone) to perform the second task (e.g., display a remote controller menu for controlling the refrigerator) for the other device (e.g., a refrigerator).

Referring toFIG.10B, upon receiving a request for the second task (e.g., displaying a remote controller menu for controlling the refrigerator) for the other device (e.g., a refrigerator) from the electronic device, the second device1023(e.g., the smartphone229ofFIG.2) may perform the second task (e.g., displaying a remote controller menu for controlling the TV) for the other device (e.g., a refrigerator).

According to an embodiment of the disclosure, a method for determining a device for performing a task by an electronic device may comprise obtaining first information for performing the task, identifying a first task corresponding to the first information, identifying a first device meeting a plurality of conditions corresponding to the first task among a plurality of devices, and requesting the first device to perform the first task via a communication module of the electronic device.

The method may further comprise storing device information gathered from the plurality of devices in a memory of the electronic device.

Identifying the first device may include identifying a plurality of conditions corresponding to the first task among a plurality of conditions set per type of task and determining the first device meeting the plurality of conditions corresponding to the first task among the plurality of devices based on device information gathered from the plurality of devices stored in the memory.

Identifying the first device may include identifying first candidate devices meeting a first condition indicating a constraint condition for device configuration information for performing the first task among the plurality of devices, identifying second candidate devices meeting a second condition indicating a preference condition for device state information for performing the first task among the first candidate devices, and identifying the first device meeting a third condition for information about a number of devices for performing the first task among the second candidate devices.

The method may further comprise gathering state information indicating that the second task is performed, received from the second device, as device information and storing the device information in a memory of the electronic device.

The method may further comprise identifying a second task for the first device, identifying a second device meeting a plurality of conditions corresponding to the second task among the plurality of devices, and requesting the second device to perform the second task via the communication module of the electronic device.

Identifying the second task for the first device may include identifying the second task for performing an additional service for the first task performed by the first device or the second task for controlling a function of the first device.

Identifying the second device may include identifying the second device meeting the plurality of conditions corresponding to the second task among the plurality of devices based on device information gathered from the plurality of devices stored in the memory of the electronic device.

Identifying the second device may include identifying first candidate devices meeting a first condition for user activity information and device configuration information for performing the second task among the plurality of devices, identifying second candidate devices meeting a second condition for device state information for performing the second task among the first candidate devices, and identifying the second device meeting a third condition for information about a number of devices for performing the second task among the second candidate devices.

The method may further comprise gathering state information indicating that the second task is performed, received from the second device, as device information and storing the device information in a memory of the electronic device.