Patent Description:
The disclosure relates to an input technology using a digital pen.

A mobile device is a device which is portable and includes at least one of a function of performing voice and video calls, a function of inputting or outputting information, and a function of storing data.

As the functions of such a mobile device have diversified, the mobile device has been equipped with complicated functions such as photographing of a picture or video, playback of music files or video files, games, reception of broadcasts, wireless Internet, and the like. In addition, for the convenience of a user, the mobile device has been implemented in a form that is capable of controlling the operation of the mobile device using a digital pen (e.g., a stylus pen).

<CIT> relates to a mobile device and corresponding method for determining whether a stylus pen is separated from the mobile device and to perform an operation of an application being executed on the mobile device in response to a control signal received via a communication unit of the mobile device, from the stylus pen.

Accordingly, an aspect of the disclosure is to provide an electronic device for performing different operations corresponding to user inputs according to an executed application, and may transmit guide information on an executed operation.

An electronic device may receive the user input by sensing the contact position of the digital pen or a position within a specified distance. In other words, the electronic device can perform the designated operation through a direct touching input or a non-contact hovering input. In addition, even if the buttons included in the digital pen are used, the operation that can be controlled according to the input is limited, and the types of applications (or application programs) that can be controlled can be limited.

In accordance with an aspect of the disclosure, an electronic device is provided. The electronic device includes a digital pen, a wireless communication circuit, a processor, and a memory operatively connected to the processor, wherein the memory stores instructions which, when executed, cause the processor to receive a first input through the wireless communication circuit from the digital pen, identify whether a first application or a second application is being executed based on the reception of the first input, when the first application is being executed, identify and perform a first operation associated with the first application from one or more operations corresponding to the first input, and when the second application is being executed, identify and perform a second operation associated with the second application from the one or more operations corresponding to the first input, wherein the first application is different from the second application, and wherein the first operation is different from the second operation.

In accordance with another aspect of the disclosure, a method of controlling an electronic device including a digital pen is provided. The method includes receiving a first input from the digital pen, identifying whether a first application or a second application is being executed based on the reception of the first input, when the first application is being executed, identifying and performing a first operation associated with the first application from one or more operations corresponding to the first input, and when the second application is being executed, identifying and performing a second operation associated with the second application from the one or more operations corresponding to the first input, wherein the first application is different from the second application, and wherein the first operation is different from the second operation.

The following description with reference to accompanying drawings is provided to assist in a comprehensive understanding of various embodiments of the disclosure as defined by the claims in accordance with the Protocol on the Interpretation of Article <NUM> EPC.

<FIG> is a block diagram of an electronic device in a network environment according to various embodiments of the disclosure.

Referring to <FIG>, an electronic device <NUM> may communicate with an electronic device <NUM> through a first network <NUM> (e.g., a short-range wireless communication network) or may communicate with an electronic device <NUM> or a server <NUM> through a second network <NUM> (e.g., a long-distance wireless communication network) in a network environment <NUM>. According to an embodiment, the electronic device <NUM> may communicate with the electronic device <NUM> through the server <NUM>. According to an embodiment, the electronic device <NUM> may include a processor <NUM>, a memory <NUM>, an input device <NUM>, a sound output device <NUM>, a display device <NUM>, an audio module <NUM>, a sensor module <NUM>, an interface <NUM>, a haptic module <NUM>, a camera module <NUM>, a power management module <NUM>, a battery <NUM>, a communication module <NUM>, a subscriber identification module <NUM>, or an antenna module <NUM>. According to some embodiments, at least one (e.g., the display device <NUM> or the camera module <NUM>) among components of the electronic device <NUM> may be omitted or one or more other components may be added to the electronic device <NUM>. According to some embodiments, some of the above components may be implemented with one integrated circuit. For example, the sensor module <NUM> (e.g., a fingerprint sensor, an iris sensor, or an illuminance sensor) may be embedded in the display device <NUM> (e.g., a display).

The processor <NUM> may execute, for example, software (e.g., a program <NUM>) to control at least one of other components (e.g., a hardware or software component) of the electronic device <NUM> connected to the processor <NUM> and may process or compute a variety of data. According to an embodiment, as a part of data processing or operation, the processor <NUM> may load a command set or data, which is received from other components (e.g., the sensor module <NUM> or the communication module <NUM>), into a volatile memory <NUM>, may process the command or data loaded into the volatile memory <NUM>, and may store result data into a nonvolatile memory <NUM>. According to an embodiment, the processor <NUM> may include a main processor <NUM> (e.g., a central processing unit or an application processor) and an auxiliary processor <NUM> (e.g., a graphic processing device, an image signal processor, a sensor hub processor, or a communication processor), which operates independently from the main processor <NUM>. Additionally or alternatively, the auxiliary processor <NUM> may use less power than the main processor <NUM>, or is specified to a designated function. The auxiliary processor <NUM> may be implemented separately from the main processor <NUM> or as a part thereof.

The auxiliary processor <NUM> may control, for example, at least some of functions or states associated with at least one component (e.g., the display device <NUM>, the sensor module <NUM>, or the communication module <NUM>) among the components of the electronic device <NUM> instead of the main processor <NUM> while the main processor <NUM> is in an inactive (e.g., sleep) state or together with the main processor <NUM> while the main processor <NUM> is in an active (e.g., an application execution) state. According to an embodiment, the auxiliary processor <NUM> (e.g., the image signal processor or the communication processor) may be implemented as a part of another component (e.g., the camera module <NUM> or the communication module <NUM>) that is functionally related to the auxiliary processor <NUM>.

The memory <NUM> may store a variety of data used by at least one component (e.g., the processor <NUM> or the sensor module <NUM>) of the electronic device <NUM>. For example, data may include software (e.g., the program <NUM>) and input data or output data with respect to commands associated with the software. The memory <NUM> may include the volatile memory <NUM> or the nonvolatile memory <NUM>.

The program <NUM> may be stored in the memory <NUM> as software and may include, for example, an operating system <NUM>, a middleware <NUM>, or an application <NUM>.

The input device <NUM> may receive a command or data, which is used for a component (e.g., the processor <NUM>) of the electronic device <NUM>, from an outside (e.g., a user) of the electronic device <NUM>.

The sound output device <NUM> may output a sound signal to the outside of the electronic device <NUM>. The speaker may be used for general purposes, such as multimedia play or recordings play, and the receiver may be used for receiving calls. According to an embodiment, the receiver and the speaker may be either integrally or separately implemented.

The display device <NUM> may visually provide information to the outside (e.g., the user) of the electronic device <NUM>. For example, the display device <NUM> may include a display, a hologram device, or a projector and a control circuit for controlling a corresponding device. According to an embodiment, the display device <NUM> may include a touch circuitry configured to sense the touch or a sensor circuit (e.g., a pressure sensor) for measuring an intensity of pressure on the touch.

The audio module <NUM> may convert a sound and an electrical signal in dual directions. According to an embodiment, the audio module <NUM> may obtain the sound through the input device <NUM> or may output the sound through an external electronic device (e.g., the electronic device <NUM> (e.g., a speaker or a headphone)) directly or wirelessly connected to the sound output device <NUM> or the electronic device <NUM>.

The sensor module <NUM> may generate an electrical signal or a data value corresponding to an operating state (e.g., power or temperature) inside or an environmental state outside the electronic device <NUM>. The sensor module <NUM> may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface <NUM> may support one or more designated protocols to allow the electronic device <NUM> to connect directly or wirelessly to the external electronic device (e.g., the electronic device <NUM>). According to an embodiment, the interface <NUM> may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

A connecting terminal <NUM> may include a connector that physically connects the electronic device <NUM> to the external electronic device (e.g., the electronic device <NUM>). According to an embodiment, the connecting terminal <NUM> may include, for example, an HDMI connector, a USB connector, a secure digital (SD) card connector, or an audio connector (e.g., a headphone connector).

The haptic module <NUM> may convert an electrical signal to a mechanical stimulation (e.g., vibration or movement) or an electrical stimulation perceived by the user through tactile or kinesthetic sensations.

The camera module <NUM> may shoot a still image or a video image. According to an embodiment, the camera module <NUM> may include, for example, at least one or more lenses, image sensors, image signal processors, or flashes.

According to an embodiment, the power management module <NUM> may be implemented as at least a part of a power management integrated circuit (PMIC).

According to an embodiment, the battery <NUM> may include, for example, a non-rechargeable (primary) battery, a rechargeable (secondary) battery, or a fuel cell.

The communication module <NUM> may establish a direct (e.g., wired) or wireless communication channel between the electronic device <NUM> and the external electronic device (e.g., the electronic device <NUM>, the electronic device <NUM>, or the server <NUM>) and support communication execution through the established communication channel. The communication module <NUM> may include at least one communication processor operating independently from the processor <NUM> (e.g., the application processor) and supporting the direct (e.g., wired) communication or the wireless communication. According to an embodiment, the communication module <NUM> may include a wireless communication module <NUM> (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module <NUM> (e.g., a local area network (LAN) communication module or a power line communication module). The corresponding communication module among the above communication modules may communicate with the external electronic device through the first network <NUM> (e.g., the short-range communication network such as a Bluetooth, a WiFi direct, or an infrared data association (IrDA)) or the second network <NUM> (e.g., the long-distance wireless communication network such as a cellular network, an internet, or a computer network (e.g., LAN or WAN)). The above-mentioned various communication modules may be implemented into one component (e.g., a single chip) or into separate components (e.g., chips), respectively. The wireless communication module <NUM> may identify and authenticate the electronic device <NUM> using user information (e.g., international mobile subscriber identity (IMSI)) stored in the subscriber identification module <NUM> in the communication network, such as the first network <NUM> or the second network <NUM>.

The antenna module <NUM> may transmit or receive a signal or power to or from the outside. According to an embodiment, the antenna module <NUM> may be formed of a conductor or a conductive pattern. In another embodiment, the antenna module <NUM> may include other part (e.g., a radio frequency integrated circuit (RFIC)) in addition to the conductor or the conductive pattern. According to an embodiment, the antenna module <NUM> may include one or more antennas. For example, the communication module <NUM> may select one antenna suitable for a communication method used in the communication network such as the first network <NUM> or the second network <NUM>. The signal or power may be transmitted or received between the communication module <NUM> and the external electronic device through the selected one antenna.

At least some components among the components may be connected to each other through a communication method (e.g., a bus, a general purpose input and output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)) used between peripheral devices to exchange signals (e.g., a command or data) with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device <NUM> and the external electronic device <NUM> through the server <NUM> connected to the second network <NUM>. Each of the electronic devices <NUM> and <NUM> may be the same or different types as or from the electronic device <NUM>. According to some examples, all or some of the operations performed by the electronic device <NUM> may be performed by one or more external electronic devices among the external electronic devices <NUM>, <NUM>, or <NUM>. For example, when the electronic device <NUM> performs some functions or services automatically or by request from a user or another device, the electronic device <NUM> may request one or more external electronic devices to perform at least some of the functions related to the functions or services, in addition to or instead of performing the functions or services by itself. The one or more external electronic devices receiving the request may carry out at least a part of the requested function or service or the additional function or service associated with the request and transmit the execution result to the electronic device <NUM>. The electronic device <NUM> may provide the result as is or after additional processing as at least a part of the response to the request. To this end, for example, a cloud computing, distributed computing, or client-server computing technology may be used.

<FIG> is a perspective view of an electronic device including a digital pen (or a stylus pen) according to an embodiment of the disclosure.

Referring to <FIG>, the electronic device <NUM> according to an embodiment may include the configuration shown in <FIG>, and includes a structure in which a digital pen <NUM> (e.g., a stylus pen) is inserted. The electronic device <NUM> includes a housing <NUM> and a portion of the housing, for example, a portion of a side surface 110C may include a hole <NUM>. The electronic device <NUM> may include a containing space <NUM> connected to the hole <NUM>, and the digital pen <NUM> may be inserted into the containing space <NUM>. According to the illustrated embodiment, the digital pen <NUM> may include a pressable button 201a at one end to facilitate removal of the digital pen <NUM> from the containing space <NUM> of the electronic device <NUM>. When the button 201a is pressed, a resilience mechanism (e.g., at least one spring) configured in association with the button 201a may operate to release the digital pen <NUM> from the containing space <NUM>.

<FIG> is a block diagram illustrating a digital pen according to an embodiment of the disclosure.

Referring to <FIG>, the digital pen <NUM> according to an embodiment may include a processor <NUM>, a memory <NUM>, a resonant circuit <NUM>, a charging circuit <NUM>, a battery <NUM>, a communication circuit <NUM>, an antenna <NUM>, and a trigger circuit <NUM>. In an embodiment, the processor <NUM>, at least a portion of the resonant circuit <NUM>, and/or at least a portion of the communication circuit <NUM> of the digital pen <NUM> may be configured on a printed circuit board or in a chip form. The processor <NUM>, the resonant circuit <NUM> and/or the communication circuit <NUM> may be electrically connected to the memory <NUM>, the charging circuit <NUM>, the battery <NUM>, the antenna <NUM>, or the trigger circuit <NUM>. The digital pen <NUM> according to an embodiment may be composed of only a resonant circuit and a button.

The processor <NUM> may include a customized hardware module or a generic processor configured to execute software (e.g., an application program). The processor may include a hardware component (function) including at least one of various sensors provided in the digital pen <NUM>, a data measurement module, an input/output interface, a module for managing a state or environment of the digital pen <NUM>, or a communication module, or a software element (program). For example, the processor <NUM> may include one of hardware, software or firmware, or a combination thereof. According to an embodiment, the processor <NUM> may receive a proximity signal corresponding to an electromagnetic field signal generated from the digitizer of the electronic device <NUM> through the resonant circuit <NUM>. When the proximity signal is identified, the resonant circuit <NUM> may be controlled to transmit an electro-magnetic resonance (EMR) input signal to the electronic device <NUM>.

The memory <NUM> may store information related to the action of the digital pen <NUM>. For example, the information may include information for communication with the electronic device <NUM> and frequency information related to an input action of the digital pen <NUM>.

The resonant circuit <NUM> may include at least one of a coil, an inductor, or capacitor. The resonant circuit <NUM> may be used to allow the digital pen <NUM> to generate a signal including a resonance frequency. For example, to generate the signal, the digital pen <NUM> may use at least one of an EMR scheme, an active electrostatic (AES) scheme, or an electrically coupled resonance (ECR) scheme. When the digital pen <NUM> transmits a signal in the EMR scheme, the digital pen <NUM> may generate a signal including a resonance frequency based on an electromagnetic field generated from an inductive panel of the electronic device <NUM>. When the digital pen <NUM> transmits a signal in the AES scheme, the digital pen <NUM> may generate a signal by using capacity coupling with the electronic device <NUM>. When the digital pen <NUM> transmits a signal in the ECR scheme, the digital pen <NUM> may generate a signal including a resonance frequency based on an electric field generated from a capacitive device of the electronic device. According to an embodiment, the resonant circuit <NUM> may be used to change the intensity or frequency of the electromagnetic field corresponding to the manipulating state of the user. For example, the resonant circuit <NUM> may provide a frequency for recognizing a hovering input, a drawing input, a button input, or an erasing input.

When the charging circuit <NUM> is connected to the resonant circuit <NUM> based on a switching circuit, the charging circuit <NUM> may rectify a resonance signal generated from the resonant circuit <NUM> into a direct current (DC) signal and provide the DC signal to the battery <NUM>. According to an embodiment, the digital pen <NUM> may identify whether the digital pen <NUM> is inserted into the electronic device <NUM>, by using a voltage level of the DC signal sensed by the charging circuit <NUM>.

The battery <NUM> may be configured to store power required for operation of the digital pen <NUM>. For example, the battery may include a lithium-ion battery or a capacitor, and may be rechargeable or interchangeable. According to an embodiment, the battery <NUM> may be charged by using the power (e.g., a DC signal (DC power)) supplied from the charging circuit <NUM>.

The communication circuit <NUM> may be configured to perform a wireless communication function between the digital pen <NUM> and the communication module <NUM> of the electronic device <NUM>. According to an embodiment, the communication circuit <NUM> may transmit state information and input information of the digital pen <NUM> to the electronic device <NUM> by using a local area communication scheme. For example, the communication circuit <NUM> may transmit direction information (e.g., motion sensor data) of the digital pen <NUM> obtained through the trigger circuit <NUM>, voice information input through a microphone, or information about the remaining amount of the battery <NUM> to the electronic device <NUM>. For example, the short-range communication scheme may include at least one of Bluetooth, a Bluetooth low energy (BLE), or a wireless LAN.

The antenna <NUM> may be used to transmit a signal or power to an outside (e.g., the electronic device <NUM>) or to receive the signal or power from the outside. According to an embodiment, the digital pen <NUM> may include a plurality of antennas <NUM> from which at least one antenna <NUM> may be selected that is suitable for the communication scheme. Through the at least one antenna <NUM> selected, the communication circuit <NUM> may exchange signals or power with an external electronic device.

The trigger circuit <NUM> may include at least one button or a sensor circuit. According to an embodiment, the processor <NUM> may identify the input scheme (e.g., touching or pressing) or type (e.g., an EMR button or a BLE button) of the button of the digital pen <NUM>. According to an embodiment, the sensor circuit may generate an electrical signal or data value corresponding to an internal operating state of the digital pen <NUM> or an external environmental condition. For example, the sensor circuit may include at least one of a motion sensor, a remaining battery level sensor, a pressure sensor, an optical sensor, a temperature sensor, a geomagnetic sensor, and a biosensor. According to an embodiment, the trigger circuit <NUM> may transmit a trigger signal to the electronic device <NUM> by using an input signal of the button or a signal through the sensor.

<FIG> is an exploded perspective view of a digital pen according to an embodiment of the disclosure.

Referring to <FIG>, the digital pen <NUM> may include a pen housing <NUM> constituting the outer appearance of the digital pen <NUM> and an inner assembly inside the pen housing <NUM>. In the illustrated embodiment, the inner assembly may be inserted into the pen housing <NUM> in a single assembly operation, including all of the various components mounted within the pen.

The pen housing <NUM> may include a shape elongated between a first end 400a and a second end 400b and may include a containing space <NUM> therein. The pen housing <NUM> may have an elliptical shape having a section of a long axis and a short axis, and may be formed as an elliptical pole as a whole. The containing space <NUM> of the electronic device <NUM> may have a section having an elliptical shape corresponding to the shape of the pen housing <NUM>. The pen housing <NUM> may include a synthetic resin (e.g., plastic) material and/or a metallic material (e.g., aluminum). According to an embodiment, the second end 400b of the pen housing <NUM> may be formed of a synthetic resin material.

The inner assembly may have a shape elongated corresponding to the shape of the pen housing <NUM>. The inner assembly may be roughly divided into three configurations in a longitudinal direction. For example, the inner assembly may include an ejection member <NUM> arranged at a position corresponding to the first end 400a of the pen housing <NUM>, a coil portion <NUM> arranged at a position corresponding to the second end 400b of the pen housing <NUM>, and a circuit board portion <NUM> arranged at a position corresponding to the body of the housing.

The ejection member <NUM> may include a configuration for ejecting the digital pen <NUM> from the containing space <NUM> of the electronic device <NUM>. According to an embodiment, the ejection member <NUM> may include a shaft <NUM>, an ejection body <NUM> arranged around the shaft <NUM> and constituting an entire outer appearance of the ejection member <NUM>, and a button portion <NUM>. When the inner assembly is completely inserted into the pen housing <NUM>, a portion including the shaft <NUM> and the ejection body <NUM> may be surrounded by the first end 400a of the pen housing <NUM>, and the button portion <NUM> (e.g., 201a of <FIG>) may be exposed to an outside of the first end 400a. In the ejection body <NUM>, a plurality of components (not shown), for example, cam members or elastic members, may be arranged to form a push-pull structure. In an embodiment, the button portion <NUM> may be substantially coupled to the shaft <NUM> and may reciprocate linearly with respect to the ejection body <NUM>. According to various embodiments, the button portion <NUM> may include a button having a latching structure for allowing the user to eject the digital pen <NUM> using a nail. According to an embodiment, the digital pen <NUM> may include a sensor that detects a linear reciprocating motion of the shaft <NUM>, thereby providing another input scheme.

When the inner assembly is completely inserted into the pen housing <NUM>, the coil portion <NUM> may include a pen tip <NUM> exposed to the outside of the second end 400b, a packing ring <NUM>, a multi-turn coil <NUM>, and/or a writing pressure sensing unit <NUM> for obtaining a change in pressure due to the pressing of the pen tip <NUM>. The packing ring <NUM> may include epoxy, rubber, urethane, or silicone. The packing ring <NUM> may be provided for the purpose of waterproofing and dustproofing and may protect the coil portion <NUM> and the circuit board portion <NUM> from flooding or dust. According to an embodiment, the coil <NUM> may form a resonance frequency (e.g., <NUM>) at a set frequency band and may be combined with at least one device (e.g., a capacitive element) to adjust the resonance frequency formed by the coil <NUM> in a specified range.

The circuit board portion <NUM> may include a printed circuit board <NUM>, a base <NUM> surrounding at least one surface of the printed circuit board <NUM>, and an antenna. According to an embodiment, a substrate placing portion <NUM> on which the printed circuit board <NUM> is arranged is formed on an upper surface of the base <NUM>, and the printed circuit board <NUM> is fixedly placed on the substrate placing portion <NUM>. According to an embodiment, the printed circuit board <NUM> may include upper and lower surfaces, where a variable capacitance capacitor or a switch <NUM> connected to the coil <NUM> may be arranged on the upper surface, and a charging circuit, a battery or a communication circuit may be arranged on the lower surface. The battery may include an electric double layered capacitor (EDLC). The charging circuit is located between the coil <NUM> and the battery, and may include a voltage detector circuitry and a rectifier.

The antenna may include an antenna structure <NUM> such as an example illustrated in <FIG> and/or an antenna embedded in the printed circuit board <NUM>. According to various embodiments, the switch <NUM> may be provided on the printed circuit board <NUM>. A side button <NUM> provided in the digital pen <NUM> may be used to press the switch <NUM> and may be exposed to an outside through a side opening <NUM> of the pen housing <NUM>. While being supported by a support member <NUM>, when an external force acting on the side button <NUM> is removed, the support member <NUM> may provide an elastic restoring force, so that the side button <NUM> is restored or maintained into a state where the side button <NUM> is arranged at a specified potion.

The circuit board portion <NUM> may include another packing ring such as an O-ring. For example, an O-ring made of an elastic material may be arranged on both ends of the base <NUM> to form a sealing structure between the base <NUM> and the pen housing <NUM>. In an embodiment, the support member <NUM> may partially contact the inner wall of the pen housing <NUM> around the side opening <NUM> to form a sealing structure. For example, the circuit board portion <NUM> may form a waterproof and dustproof structure similar to that of the packing ring <NUM> of the coil portion <NUM>.

The digital pen <NUM> may include a battery mounting portion <NUM>, on which a battery <NUM> is arranged, on an upper surface of the base <NUM>. For example, the battery <NUM>, which may be mounted on the battery mounting portion <NUM>, may include a cylinder-type battery.

The digital pen <NUM> may include a microphone (not shown). The microphone may be directly connected to the printed circuit board <NUM> or may be connected to a separated flexible printed circuit board (FPCB) (not shown) connected to the printed circuit board <NUM>. According to various embodiments, the microphone may be arranged at a position parallel to the side button <NUM> in the longitudinal direction of the digital pen <NUM>.

<FIG> is a block diagram illustrating a configuration of an electronic device according to various embodiments of the disclosure.

Referring to <FIG>, the electronic device <NUM> (e.g., the electronic device <NUM> of <FIG>) may include the communication module <NUM> (e.g., the wireless communication module <NUM> of <FIG>), the input device <NUM> (e.g., the input device <NUM> of <FIG>), the memory <NUM> (e.g., the memory <NUM> of <FIG>), a display <NUM> (e.g., the display device <NUM> of <FIG>), and the processor <NUM> (e.g., the processor <NUM> of <FIG>). The configuration of the electronic device <NUM> is not limited to the above configurations, and may further include the configuration illustrated in <FIG>.

According to an embodiment, the communication module <NUM> includes a wireless communication module. The communication module <NUM> may be connected to the digital pen <NUM> to transmit or receive data. According to an embodiment, the communication module <NUM> may support Bluetooth, BLE, or wireless LAN (e.g., Wi-Fi).

According to an embodiment, the input device <NUM> receives a user input. The input device <NUM> may include an inductive panel (not shown) for sensing the position of the digital pen <NUM>.

According to an embodiment, the memory <NUM> stores at least one app (or an application program). For example, the at least one app may include a camera app for controlling a plurality of cameras (e.g., the camera module <NUM> of <FIG>), a presentation app for performing a presentation function, a media app for controlling media contents, a gallery app for displaying an image stored in the memory <NUM> on the display <NUM>, and the like. According to an embodiment, the memory <NUM> may store contents to be provided to the user. For example, the contents may be media contents including at least one of an image or sound.

According to an embodiment, the display <NUM> may output an image. For example, the display <NUM> may display an image included in the contents, a graphic user interface (GUI) of an executed app, and the like. According to an embodiment, the display <NUM> may include a panel for receiving a user input using the digital pen <NUM>. The user input may include a contactless input such as a hovering input as well as a touch input.

According to an embodiment, the processor <NUM> may control the overall operation of the electronic device <NUM>. According to an embodiment, when the instructions stored in the memory <NUM> are executed, the processor <NUM> may perform the operations described below.

According to an embodiment, the processor <NUM> may execute an application stored in memory <NUM>. The processor <NUM> may display the GUI of the executed application on the display <NUM>. According to an embodiment, the processor <NUM> may execute a plurality of apps. The processor <NUM> may execute one of the plurality of executed apps in the foreground and execute the remaining apps in the background. The processor <NUM> may display the GUI of the app executing in the foreground on the display <NUM>.

According to an embodiment, the processor <NUM> may receive a user input using the digital pen <NUM>. According to an embodiment, the processor <NUM> may recognize the position of the digital pen <NUM> to receive a user input. For example, the processor <NUM> may recognize the position at which the digital pen <NUM> contacts (or touches) through an inductive panel (not shown), or a position at which the digital pen <NUM> is present (or hovering) within a specified distance, thereby receiving the user input. For example, the input using the position of the digital pen <NUM> may be a digitizer input.

According to an embodiment, the processor <NUM> may receive a user input from the digital pen <NUM> through the communication module <NUM>. For example, the processor <NUM> may receive a user input through the side button <NUM> of the digital pen <NUM>. The processor <NUM> may receive a signal corresponding to a user input from the digital pen <NUM> through the communication module <NUM>. For example, the user input through the side button <NUM> may be a wireless communication input. According to an embodiment, the processor <NUM> may receive mutually different user inputs through the side button <NUM> of the digital pen <NUM>. For example, the processor <NUM> may receive user inputs that differ in the number of times or a time the side button <NUM> of the digital pen <NUM> is pressed. In other words, the processor <NUM> may distinguish mutually different user inputs based on the number of times or the times the side button <NUM> of the digital pen <NUM> is pressed. For example, the mutually different user inputs may include inputs of pressing the side button <NUM> once, twice, for a long time, and the like. According to an embodiment, the processor <NUM> may receive a signal corresponding to each of the different user inputs from the digital pen <NUM>.

According to an embodiment, the processor <NUM> may distinguish first and second inputs based on the number of times the side button <NUM> of the digital pen <NUM> is pressed on the basis of a specified time. The processor <NUM> may receive a signal (or an event signal) corresponding to an event of first pressing the side button <NUM> from the digital pen <NUM> and recognize the first or second input depending on whether an event (or an event signal) of pressing the side button <NUM> secondly within a specified time is received. For example, when the processor <NUM> receives the event of first pressing the side button <NUM> from the digital pen <NUM>, the processor <NUM> may suspend the determination as to whether the event is the first input (e.g., an input of pressing the side button <NUM> of the digital pen <NUM> once), and may determine (or recognize) the event as the first input when an event of pressing the side button <NUM> once more within a specified time is not received. In addition, when the processor <NUM> receives a signal corresponding to an input that presses the side button <NUM> secondly within a specified time after receiving the event of first pressing the side button <NUM> from the digital pen <NUM>, the processor <NUM> may determine the signal as the second input (e.g., an input that presses the side button <NUM> of the digital pen <NUM> twice) different from the first input.

According to an embodiment, the digital pen <NUM> transmits a signal corresponding to an event at the time of releasing or pressing the side button <NUM>, thereby allowing the processor <NUM> to recognize the second input. For example, the digital pen <NUM> may transmit a signal at a time point t1 when the side button <NUM> is first pressed and a signal at a time point t3 when the side button <NUM> is pressed secondly within a specified time. As another example, the digital pen <NUM> may transmit a signal at a time t2 when the side button <NUM> is released, and may transmit a signal at a time t3 when the side button <NUM> is pressed secondly within a specified time. As still another example, the digital pen <NUM> may transmit a signal at a time t1 when the side button <NUM> is first pressed and transmit a signal at a time t4 when the side button <NUM> is released secondly within a specified time. As still another example, the digital pen <NUM> may transmit a signal at a time t2 when the side button <NUM> is first released and transmit a signal at a time t4 when the side button <NUM> is released secondly within a specified time. Thus, as compared with transmitting signals at both time points when the side button <NUM> of the digital pen <NUM> is pressed and released, the consumption of the battery of the digital pen <NUM> (e.g., the battery <NUM> of <FIG>) may be reduced, and by minimizing the interval between the time points when the presses of the button are determined many times, it is possible to respond immediately to the user input (e.g., perform the specified operation). In addition, when any actions for the second input (e.g., the input of pressing the side button <NUM> of the digital pen <NUM> twice) are not set to the executed app and the processor <NUM> first receives a signal corresponding to the first input from the digital pen <NUM>, the processor <NUM> may determine the signal as the first input (e.g., the input of pressing the side button <NUM> of the digital pen <NUM> once) without suspending the determination as to whether the signal is the first input. Thus, by shortening the waiting time (or the determination suspend time) for determining the pressing of the side button <NUM> many times, it is possible to provide an immediate response to a user input.

According to an embodiment, the processor <NUM> may perform at least one operation corresponding to each user input received from the digital pen <NUM>. According to an embodiment, the processor <NUM> may perform an action corresponding to a user input (e.g., a touch input or a hovering input) based on the location of the digital pen <NUM>. For example, the processor <NUM> may receive an input for selecting an object (e.g., an icon or a virtual button) displayed at a designated position of the GUI by using the digital pen <NUM>, and any perform an operation corresponding to the selected object. According to an embodiment, the processor <NUM> may perform an operation corresponding to a user input through the side button <NUM> of the digital pen <NUM>. For example, the processor <NUM> may receive a signal corresponding to a user input received through the side button <NUM> of the digital pen <NUM>, and may perform an operation of an app corresponding to the received signal. For example, the operation of the app may be set to be executed when the user input through the side button <NUM> of the digital pen <NUM> is received. The app may be an app being executed in the electronic device <NUM>.

According to an embodiment, the processor <NUM> may display, on the display <NUM>, a function that can be performed using the digital pen <NUM>, and guide information about a controllable operation. For example, the processor <NUM> may display the guide information on the display <NUM> when detecting the detachment (or separation) of the digital pen <NUM>. According to an embodiment, the processor <NUM> may display, on the display <NUM>, the guide information (e.g., first information) about a function that can be performed through a user input (e.g., a touch input, a hovering input) based on the position of the digital pen <NUM>. For example, the first information may include information about a function related to a user input based on the position of the digital pen <NUM>, such as a handwriting memo function, a drawing function, and the like. According to an embodiment, the processor <NUM> may display, on the display <NUM>, guide information (e.g., second information) about an action controllable through a user input through the side button <NUM> of the digital pen <NUM>. For example, the second information may include information about the action of the app being executed.

According to an embodiment, the processor <NUM> may display an object corresponding to the digital pen <NUM> on the display <NUM>. For example, the processor <NUM> may display the object on the display device <NUM> when the digital pen <NUM> is detached from the electronic device <NUM>. For example, the object may be a floating icon. A function that can be performed using the digital pen <NUM> while being adjacent to the floating icon, and guide information on a controllable action may be displayed. Thus, the processor <NUM> may indicate the separated state of the digital pen <NUM> through the object.

According to an embodiment, the processor <NUM> may indicate, through the object, a state in which the digital pen <NUM> is connected to the communication module <NUM>. For example, the processor <NUM> may change the color of the edge of the object to indicate whether it is in a state capable of through the side button <NUM> of the digital pen <NUM>.

According to an embodiment, the processor <NUM> may display, on the display <NUM>, an indicator that indicates the state of the digital pen <NUM>. For example, the processor <NUM> may display, on the display <NUM>, an indicator that indicates wireless connection state information of the digital pen <NUM>, information about a battery (e.g., the battery <NUM> of <FIG>), and the like.

<FIG> is a view illustrating a framework and a plurality of apps stored in a memory of an electronic device according to various embodiments of the disclosure.

Referring to <FIG>, the memory <NUM> of the electronic device <NUM> may store a plurality of apps <NUM> and frameworks <NUM> to <NUM> for managing the plurality of apps <NUM>.

According to an embodiment, actions (e.g., remote actions) may be defined in extensible markup language (XML) content 611a and 613a. The actions may be executed based on a user input through the side button <NUM> of a digital pen (e.g., the digital pen <NUM> of <FIG>).

According to an embodiment, a user input through the button of the digital pen may be set to correspond to a function performed in at least one app. For example, the user input through the button of the digital pen may be set to correspond to a camera control function performed in at least one app (e.g., B612, Snapchat, Snow). In addition, the user input through the button of the digital pen may be set to correspond to a media control function performed in at least one app (e.g., Google play music, Samsung music, Melon, and the like). For example, the app in which the media control function is performed may be remotely controlled by using the side button <NUM> of the digital pen <NUM> even when being executed in the background as well as in the foreground. Thus, the function set to correspond to the user input through the button of the digital pen may be applied to a plurality of apps for performing the function.

For example, each of a plurality of apps <NUM> may declare an intent filter as following "REMOTE_ACTION' inactivity to process a user input received through the user input received through the side button <NUM> of the digital pen <NUM> of the app manifest file (AndroidManifest. xml) included in each app <NUM> in order to define at least one action corresponding to at least one user input through the side button <NUM> of the digital pen <NUM>.

In addition, the XML (or an XML resource) defining at least one action corresponding to at least one user input through the side button <NUM> of the digital pen <NUM> may be designated under the activity tag declared as the 'REMOTE_ACTION' as following meta data.

In addition, at least one action corresponding to at least one user input received through the side button <NUM> of the digital pen <NUM> may be defined in XML. In other words, the XML may include action information (or key information) corresponding to at least one user input received through the side button <NUM> of the digital pen <NUM>.

For example, the actions of following Table <NUM> may include at least one action that may be executed corresponding to at least one user input (e.g., first to third functions) through the side button <NUM> of the digital pen <NUM> for each app <NUM>.

For example, the apps <NUM> installed in the electronic device <NUM> may define actions corresponding to the types of user inputs that may be received through the side button <NUM> of the digital pen <NUM> as following table <NUM>.

According to an embodiment, the memory <NUM> may store, as a framework for controlling the actions of the executed apps <NUM> and <NUM>, a use state management unit <NUM>, an input management unit <NUM>, a package management unit <NUM>, and a remote action management unit <NUM>. According to an embodiment, the use state management unit <NUM>, the input management unit <NUM>, the package management unit <NUM>, and the remote action management unit <NUM> may be executed by the processor <NUM> of the electronic device <NUM> and perform following actions.

According to an embodiment, the use state management unit <NUM> may manage the execution states of the apps <NUM> and <NUM>. For example, the use state management unit <NUM> may determine the priority for processing information of the apps <NUM> and <NUM> based on a user input and state information of the apps <NUM> and <NUM> executed in the foreground and background. According to an embodiment, the use state management unit <NUM> may provide information about the execution state of the apps <NUM> and <NUM>. For example, the use state management unit <NUM> may provide information related to the apps <NUM> and <NUM> executed in the foreground and background.

According to an embodiment, the input management unit <NUM> may transmit a message for performing a specified action to the apps <NUM> and <NUM> based on the occurrence of an event. For example, the input management unit <NUM> may transmit a message (or a key information message) for performing a selected action (or a remote action) based on a user input received via the side button <NUM> of the digital pen <NUM> to the apps <NUM> and <NUM>.

According to an embodiment, the package management unit <NUM> may manage information about the installed apps <NUM> and <NUM>. According to an embodiment, the package management unit <NUM> may manage key information about at least one of the apps <NUM>. When the apps <NUM> and <NUM> are installed or at least one action corresponding to at least one user input through the side button <NUM> of the digital pen <NUM> is defined, the package management unit <NUM> may obtain key information about the at least one action defined. The package management unit <NUM> may store the obtained key information in the memory <NUM> together with the information about the apps <NUM> and <NUM>.

According to an embodiment, the remote action management unit <NUM> can obtain state information of the apps <NUM> and <NUM> that are being executed, through the use state management unit <NUM>. For example, the remote action management unit <NUM> may receive the information about the apps <NUM> and <NUM> being executed in the foreground and background and the switching information of the apps <NUM> and <NUM> being executing in the foreground, through the use state management unit <NUM>.

According to an embodiment, the remote action management unit <NUM> may obtain action information defined in the apps <NUM> and <NUM> by which an action (or a remote action) selected based on a user input received through the package management unit <NUM> is performed. According to an embodiment, the remote action management unit <NUM> may provide the obtained at least one action information to the user. For example, the remote action management unit <NUM> may display the obtained action information adjacent to an object (e.g., a floating icon) displayed on the display <NUM>. In addition, the remote action management unit <NUM> may display, on the object, information about whether a user input is received through the side button <NUM> of the digital pen <NUM>.

According to an embodiment, the remote action management unit <NUM> can transmit a key information message to the apps <NUM> and <NUM> that are to perform a remote action through the input management unit <NUM>.

For example, the electronic device <NUM> receives a user input (or first input) (e.g., an input that pushes the side button <NUM> once) through the side button <NUM> of the digital pen <NUM> (first operation) while the first app <NUM> is being executed (first operation). The electronic device <NUM> may receive a first input through the side button <NUM> from the digital pen <NUM> through the communication module <NUM>. The electronic device <NUM> may perform a first action of the first app <NUM> being executed in response to the received first input. Thus, the electronic device <NUM> may display a GUI associated with the first action of the first app <NUM> on the display <NUM>.

In addition, the electronic device <NUM> may receive a user input through the side button <NUM> of the digital pen <NUM> while the second app <NUM> is executed (a second operation). The electronic device <NUM> may receive a user input through the side button <NUM> from the digital pen <NUM> through the communication module <NUM>. The received user input may be the same kind of input as the first input received in the first operation. The electronic device <NUM> may perform a first action of the second app <NUM> being executed in response to the received user input. For example, the first action of the second app <NUM> may be an action different from the first action of the first app <NUM>. In other words, even though the electronic device <NUM> receives the same user input through the side button <NUM> of the digital pen <NUM> when the second app <NUM> is executed, the electronic device <NUM> may perform an action different from that while the first app <NUM> is executed. Thus, the electronic device <NUM> may display, on the display <NUM>, the GUI related to the first action of the second app <NUM>.

As another example, the electronic device <NUM> may receive a user input (or the second input) (e.g., an input that presses a button twice) through the side button <NUM> of the digital pen <NUM> while the first app <NUM> is executed (the first operation). The electronic device <NUM> may receive the second input through the button from the digital pen <NUM> through the communication module <NUM>. The electronic device <NUM> may perform the second action of the first app <NUM> being executed, in response to the received second input. Thus, the electronic device <NUM> may display the GUI related to the second action of the first app <NUM> on the display <NUM>.

In addition, the electronic device <NUM> may receive a user input through the side button <NUM> of the digital pen <NUM> while the second app <NUM> is executed (the second operation). The electronic device <NUM> may receive the user input through the side button <NUM> from the digital pen <NUM> through the communication module <NUM>. The received user input may be the same kind of input as that of the second input received in the first operation. In response to the received user input, the electronic device <NUM> may perform a second action of the second app <NUM> being executed. For example, even though the electronic device <NUM> receives the same user input through the side button <NUM> of the digital pen <NUM> when the second app <NUM> is executed, the electronic device <NUM> may perform an action different from that while the first app <NUM> is executed. Thus, the electronic device <NUM> may display, on the display <NUM>, the GUI related to the second action of the second app <NUM>.

<FIG> is a view illustrating an action when an electronic device receives a user input through a button of a digital pen according to an embodiment of the disclosure.

Referring to <FIG>, the electronic device <NUM> may perform an action corresponding to a user input received through the side button <NUM> of the digital pen <NUM> based on an app <NUM> being executed.

According to an embodiment, in operation <NUM>, the remote action management unit <NUM> of the electronic device <NUM> may receive at least one user input through the side button <NUM> of the digital pen <NUM> through the communication module <NUM>. For example, the remote action management unit <NUM> may receive a signal corresponding to at least one user input from the digital pen <NUM>.

According to an embodiment, in operation <NUM>, when the remote action management unit <NUM> receives the at least one user input, the remote action management unit <NUM> may obtain the execution state information of the app <NUM> through the use state management unit <NUM>. In other words, the remote action management unit <NUM> may identify the app being executed through the use state management unit <NUM>. According to an embodiment, the remote action management unit <NUM> may detect the detachment of the digital pen <NUM> and receive the execution state information of the app <NUM> in response to the detection. According to an embodiment, the remote action management unit <NUM> may provide, to the user, guide information for at least one selected action of the app <NUM> corresponding to at least one user input received through the side button <NUM> of the digital pen <NUM>.

According to an embodiment, in operation <NUM>, the remote action management unit <NUM> may transmit a key information message to the input management unit for performing a specified action. In response, in operation <NUM>, the input management unit <NUM> transmits information to the app <NUM> to perform the specified action based on the execution state information of the app <NUM> obtained through the input management unit <NUM>. Thus, the electronic device <NUM> may perform the specified action of the app <NUM>.

<FIG> is a view illustrating a screen for providing a list of apps to which an action corresponding to a user input through a button of a digital pen of an electronic device according to an embodiment of the disclosure is set.

Referring to <FIG>, the electronic device <NUM> may provide, to a user, a list <NUM> of settable apps (e.g., the plurality of apps <NUM> of <FIG>) by at least one action which corresponds to at least one user input through the side button <NUM> of the digital pen <NUM>.

According to an embodiment, in a screen <NUM>, the electronic device <NUM> may provide state information <NUM> of a digital pen (e.g., the digital pen <NUM> of <FIG>). For example, the electronic device <NUM> may provide information 711a about the battery <NUM> of the digital pen <NUM>. The electronic device <NUM> may receive a user input for connecting with the digital pen through an activation button 711b and turning on or off a control function by the side button <NUM> of the digital pen <NUM>.

According to an embodiment, in the screen <NUM>, the electronic device <NUM> may provide a list <NUM> of at least settable app <NUM> corresponding to at least one user input received through a button (e.g., the side button <NUM> in <FIG>) of the digital pen <NUM>. For example, the list <NUM> of the apps <NUM> may include information 713a about at least one action corresponding to each of at least one user input for each installed app <NUM>. According to an embodiment, the electronic device <NUM> may update the list <NUM> of the apps <NUM>. For example, the electronic device <NUM> may add a newly installed app to the list <NUM> of the apps <NUM>. According to an embodiment, the electronic device <NUM> may activate or deactivate an action set through the items 713_1, 713_2, 713_3, and 713_4 included in the list <NUM> of the app <NUM>.

According to an embodiment, the electronic device <NUM> may deactivate the action set to each app based on an average use time of each app included in the list <NUM> of the apps <NUM>. For example, the electronic device <NUM> may deactivate an action set to an app whose the average use time is longer than the operable time (or a threshold time) of the digital pen <NUM> based on the remaining amount of the battery <NUM> of the digital pen <NUM>. According to an embodiment, when deactivating the action set to the app, the electronic device <NUM> may provide a pop-up window to receive a user input for deactivating the set action. The electronic device <NUM> may deactivate the set action when receiving the user input. According to an embodiment, the electronic device <NUM> may provide a guide for inducing the user to charge the battery <NUM> of the digital pen <NUM>. According to an embodiment, the electronic device <NUM> may check the use time of the app to be executed every time when the app is executed, and calculate the average use time of the executed app based on the confirmed use time.

According to an embodiment, in the screen <NUM>, the electronic device <NUM> may receive a selection input 710a that selects an app for setting at least one action corresponding to at least one user input. According to an embodiment, in a screen <NUM>, the electronic device <NUM> may provide information about the setting of at least one action corresponding to at least one user input received through the side button <NUM> of the digital pen <NUM> of the selected app. For example, the electronic device <NUM> may provide action information <NUM> set to the first input (e.g., an input of pushing the side button <NUM> once) and the second input (e.g., an input of pushing the side button <NUM> twice). According to an embodiment, the electronic device <NUM> may receive an input 720a of selecting a user input for setting a specified action. According to an embodiment, in a screen <NUM>, the electronic device <NUM> may provide settable action information <NUM> for the selected user input.

According to an embodiment, in the screen <NUM>, the electronic device <NUM> may receive a drag input 710b that scrolls up or down the list <NUM> of the apps <NUM>. According to an embodiment, in a screen <NUM>, based on the drag input 710b, the electronic device <NUM> may display sub-items <NUM> on the display instead of at least a portion of state information <NUM> of the digital pen <NUM>. The electronic device <NUM> may again receive a scroll-up drag input 740a. The area capable of receiving the scroll-up drag input 740a may be an entire area 740b of a screen. According to an embodiment, in a screen <NUM>, the electronic device <NUM> may provide a list <NUM> of apps through the entire area 740b of the screen. The electronic device <NUM> may again receive a scroll-up drag input 750a. The area capable of receiving the scroll-up drag input 750a may be an area 750b where items are displayed on the screen. According to an embodiment, in a screen <NUM>, the electronic device <NUM> may provide the lowest item <NUM>. When the electronic device <NUM> again receives a scroll-up drag input 760a, the electronic device <NUM> may display, on the display <NUM>, an indicator <NUM> of informing that there is no lower item.

<FIG> is a flowchart illustrating a method of displaying on a display an object corresponding to a digital pen of an electronic device according to an embodiment of the disclosure.

Referring to <FIG>, in operation <NUM>, the electronic device <NUM> may detect the detachment of a digital pen (e.g., the digital pen <NUM> of <FIG>). For example, the electronic device <NUM> may detect that the digital pen <NUM> is detached from a containing space (e.g., the containing space <NUM> of <FIG>) of the electronic device <NUM>.

According to an embodiment, in operation <NUM>, the electronic device <NUM> may display on the display <NUM> an object corresponding to the digital pen <NUM> in response to the detection. The object may be, for example, a floating icon.

<FIG> is a view illustrating an object displayed on a display of an electronic device according to an embodiment of the disclosure.

Referring to <FIG>, an electronic device (e.g., the electronic device <NUM> of <FIG>) may represent, in an object (e.g., a floating icon) <NUM>, <NUM> and <NUM> displayed on the display <NUM>, whether at least one operation corresponding to the at least one user input can be performed. In addition, the electronic device <NUM> may provide guidance information for at least one action corresponding to at least one user input received through the side button <NUM> of the digital pen <NUM> adjacent to the object.

Referring to (a) of <FIG>, the electronic device <NUM> may display, on the display <NUM>, a floating icon <NUM> corresponding to the digital pen <NUM> connected through the communication module <NUM>. The electronic device <NUM> may perform at least one action corresponding to at least one user input received through the side button <NUM> of the digital pen <NUM>.

According to an embodiment, the electronic device <NUM> may display a floating icon <NUM> on the display <NUM> when the digital pen <NUM> is detached. For example, the electronic device <NUM> may display the floating icon <NUM> in a first state. The first state may be a grayed state. According to an embodiment, the electronic device <NUM> may change the edge of the floating icon <NUM> to a different color (e.g., blue) to represent whether at least one action corresponding to at least one user input received through the side button <NUM> of the digital pen <NUM> connected through the wireless communication circuit can be performed.

According to an embodiment, the electronic device <NUM> may change the state of a floating icon <NUM> when the digital pen <NUM> receives an input (e.g., a hovering input) for selecting the floating icon. For example, the electronic device <NUM> may display the floating icon <NUM> in the second state. The second state may be a blacked state. According to an embodiment, the electronic device <NUM> may provide, to a user, guide information 913a for at least one action corresponding to at least one user input received through the side button <NUM> of the digital pen <NUM> of the app (e.g., the app <NUM> of <FIG>) being executed. For example, the electronic device <NUM> may display the guide information 913a adjacent to the floating icon <NUM>. The edge of the guide information 913a may be displayed in a color similar to the edge of the floating icon <NUM>. Thus, the connection state of the digital pen <NUM> may be more clearly provided to the user.

According to an embodiment, when an app to which an action corresponding to a user input through the side button <NUM> of the digital pen <NUM> is set is executed (or first entered), the electronic device <NUM> may show guide information 913a related to the set action. The electronic device <NUM> may not display the guide information 913a at the time of a later execution of the action.

According to an embodiment, when the electronic device <NUM> receives at least one user input through the side button <NUM> of the digital pen <NUM>, the electronic device <NUM> may maintain the displaying state of a floating icon <NUM>. For example, the floating icon <NUM> may be maintained in the second state and the changed state of the edge.

Referring to (b) of <FIG>, the electronic device <NUM> may display, on the display <NUM>, a floating icon <NUM> corresponding to the digital pen <NUM> connected through the communication module <NUM>. The electronic device <NUM> may not be able to perform at least one action corresponding to at least one user input received through the side button <NUM> of the digital pen <NUM>.

According to an embodiment, although the electronic device <NUM> is wirelessly connected to the digital pen <NUM>, when the user sets the remote action to off, or does not set the remote action to the app being executed in the foreground and background, the electronic device <NUM> may display, on the display <NUM>, the floating icon <NUM> whose the edge is not changed in color.

According to an embodiment, when the digital pen <NUM> is detached, the electronic device <NUM> may display a floating icon <NUM> on the display <NUM>. For example, the electronic device <NUM> may display the floating icon <NUM> in the first state.

According to an embodiment, when the digital pen <NUM> receives an input (e.g., a hovering input) for selecting a floating icon, the electronic device <NUM> may change the state of a floating icon <NUM> into the second state. According to an embodiment, the electronic device <NUM> may provide name (e.g., air command) information 923a of the floating icon to the user.

According to an embodiment, when the electronic device <NUM> receives at least one user input through the side button <NUM> of the digital pen <NUM>, the displaying state of a floating icon <NUM> except for the name information 923a may be maintained.

Referring to (c) of <FIG>, the electronic device <NUM> may display, on the display <NUM>, a floating icon <NUM> corresponding to the digital pen <NUM> which is not connected through the communication module <NUM>.

According to an embodiment, similarly to the action of (b), the electronic device <NUM> may display a first floating icon <NUM> in the first state when the digital pen <NUM> is detached. According to an embodiment, similarly to the action of (b), when the digital pen <NUM> receives an input (e.g., a hovering input) for selecting a floating icon, the electronic device <NUM> may change the state of a floating icon <NUM> to the second state. According to an embodiment, when receiving the input for selecting the floating icon, the electronic device <NUM> may provide name (e.g., air command) information 933a of the floating icon to the user.

According to an embodiment, when receiving at least one user input through the side button <NUM> of the digital pen <NUM>, the electronic device <NUM> may not display the floating icon (N/A).

<FIG> is a flowchart illustrating a method of providing guidance information by an electronic device according to an embodiment of the disclosure.

Referring to <FIG>, an electronic device (e.g., the electronic device <NUM> of <FIG>) may provide information about an remote action set to an app that performs an action corresponding to at least one user input through the side button <NUM> of the digital pen <NUM>.

According to an embodiment, in operation <NUM>, the electronic device <NUM> may determine whether the digital pen (e.g., the digital pen <NUM> of <FIG>) is connected to the communication module <NUM>. For example, the electronic device <NUM> may determine whether the digital pen <NUM> is connected to the electronic device <NUM> through wireless communication.

According to an embodiment, in operation <NUM>, the electronic device may determine an app for performing an action (or a remote action) corresponding to a user input, among the apps being executed. For example, the user input may be a user input received through the side button <NUM> of the digital pen <NUM>. The operation <NUM> of determining the app for performing an action corresponding to a user input will be described later in detail with reference to <FIG>.

According to an embodiment, in operation <NUM>, the electronic device <NUM> may provide guidance information related to a remote action set to the determined app.

<FIG> is a flowchart illustrating a method of determining an app for performing a remote action by an electronic device according to an embodiment of the disclosure.

Referring to <FIG>, operation <NUM> will be illustrated in further detail. Initially, in operation <NUM>, the electronic device <NUM> may identify information about an app executed in the foreground. The GUI of the app being executed in the foreground may be displayed on the display <NUM>.

According to an embodiment, in operation <NUM>, the electronic device <NUM> may determine whether a remote action is set to the app being executed in the foreground. In addition, the electronic device <NUM> may determine whether the remote action set to the app being executed in the foreground is executable. According to an embodiment, in operation <NUM>, when the remote action is set to the app being executed in the foreground (Yes), the electronic device <NUM> may displays guide information related to the remote action set to the app being executed in the foreground on the display <NUM>. In addition, when the remote action set to the app being executed in the foreground is executable, the electronic device <NUM> may display the guide information on the display <NUM>.

According to an embodiment, in operation <NUM>, when the remote operation is not set to the app being executed in the foreground, the electronic device <NUM> may identify the information about the app being executed in the background. In addition, when the action set in the app being executed in the foreground is inexecutable, the electronic device <NUM> may identify the information about the app being executed in the background.

According to an embodiment, in operation <NUM>, the electronic device <NUM> may determine whether the remote action is set to the app being executed in the background. According to an embodiment, when the remote action is not set to the app being executed in the background (No), the electronic device <NUM> may terminate the process for providing guide information. In addition, when the remote action set to the app being executed in the background is not executable, the electronic device <NUM> may terminate the process for providing the guide information.

According to an embodiment, the electronic device <NUM> may provide guide information related to a remote action set in an app being executed in the background when the remote action is set to the app being executed in the background. Alternatively, when the remote action set to the app being executed in the background is executable, the electronic device <NUM> may provide the guide information.

For example, the electronic device <NUM> may receive at least one user input through the side button <NUM> of the digital pen <NUM> in a state in which a gallery app is executed in the foreground and a media app is executed in the background. In a state in which at least one remote action set to the gallery app is executable, the electronic device <NUM> may execute at least one action (e.g., displaying of a next image, or displaying of a previous image) of the gallery app corresponding to the received at least one user input (e.g., an input of pushing the side button <NUM> once, an input of pushing the side button <NUM> twice, and the like). In a state in which at least one remote action set to the gallery app is inexecutable, the electronic device <NUM> may execute at least one remote action (e.g., media playback and pause) set to the media app. For example, at least one remote operation set to the media app may be implemented to be performed by definition to correspond to a media control function.

<FIG> is a view illustrating a screen for providing guide information by an electronic device according to an embodiment of the disclosure.

Referring to <FIG>, the electronic device <NUM> may be connected to the digital pen <NUM> through the communication module <NUM>.

According to an embodiment, in screen <NUM>, when the digital pen <NUM> is detached from the electronic device <NUM>, the electronic device <NUM> may display an object (e.g., a floating icon) <NUM> on the display <NUM>. According to an embodiment, the electronic device <NUM> may display an executable state for the remote action set to the app corresponding to at least one user input through the side button <NUM> of the digital pen <NUM> through the object <NUM>.

According to an embodiment, the electronic device <NUM> may display, on the display <NUM>, guide information <NUM> for a plurality of functions (e.g., creating of a note, viewing of all notes, and the like) that can be performed by using the digital pen <NUM>. The electronic device <NUM> may display the guide information <NUM> for the functions on the display <NUM> adjacent to the object <NUM>. For example, the electronic device <NUM> may display the guide information <NUM> for the functions at a specified distance around the object <NUM>.

According to an embodiment, the electronic device <NUM> may display, on the display <NUM>, the guide information <NUM> about the at least one action (or a remote action) that can be controlled by at least one user input through the side button <NUM> of the digital pen <NUM>. The electronic device <NUM> may display the guide information <NUM> on a specified portion (e.g., a lower end) 1110a of a screen of the display <NUM>. The guide information <NUM> displayed on the display <NUM> may be information about the remote action set to the app being executed in the foreground.

<FIG> is a view illustrating a screen for providing guide information for a remote action set to an app by an electronic device according to an embodiment of the disclosure.

Referring to <FIG>, the electronic device <NUM> may display, on the display <NUM>, the guide information related to the remote action set to the executed app.

According to an embodiment, in a screen <NUM>, when the digital pen <NUM> is detached from the electronic device <NUM>, the electronic device <NUM> may display an object (e.g., a floating icon) <NUM> on the display <NUM>. For example, the digital pen <NUM> may be connected to the electronic device <NUM> through wireless communication. According to an embodiment, the electronic device <NUM> may display, on the display, guide information <NUM> related to the remote action set to the app being executed in the background. For example, the electronic device <NUM> may display, on the display <NUM>, the guide information <NUM> related to at least one remote action (e.g., playback and pause actions, and skip action) set to the media app being executed in the background.

According to an embodiment, in a screen <NUM>, the electronic device <NUM> may execute an app in the foreground. For example, the electronic device <NUM> may execute a media app that was executed in the background in the foreground. Thus, the electronic device <NUM> may display, on the display <NUM>, a GUI 1220a of the media app being executed in the foreground. According to an embodiment, the electronic device <NUM> may display, on the display <NUM>, guide information <NUM> for the remote operation set on the app newly executed in the foreground. For example, the electronic device <NUM> may display, on the display <NUM>, the guide information <NUM> related to the remote action set to the media app that is newly executed in the foreground. The guide information <NUM> may be displayed adjacent to an object <NUM> corresponding to the digital pen <NUM>.

According to an embodiment, in a screen <NUM>, the electronic device <NUM> may switch an app being executed in the foreground. For example, the electronic device <NUM> may newly execute a camera app. The camera app may be executed in the foreground. The electronic device <NUM> may display a GUI 1230a of the camera app on the display <NUM>. According to an embodiment, the electronic device <NUM> may display, on the display <NUM>, guide information <NUM> related to at least one remote operation <NUM> set to the switched app. For example, the electronic device <NUM> may display, on the display <NUM>, the guide information <NUM> related to at least one remote action (e.g., an image photographing action and a camera switching action) set to the switched camera app. For example, at least one remote action set to the camera app may be implemented to be performed by definition to correspond to a camera control function.

Accordingly, the electronic device <NUM> may display, on the display <NUM>, the guide information <NUM> or <NUM> related to at least one remote action set to the app being executed in the foreground, when the app is newly executed or terminated, or when the app is switched.

<FIG> is a view illustrating an indicator indicating state information of a digital pen in an electronic device according to an embodiment of the disclosure.

Referring to <FIG>, the electronic device <NUM> may display an indicator indicative of specified information (e.g., display <NUM> of <FIG>) on an upper end <NUM> of a screen of the display (e.g., display <NUM> of <FIG>).

According to an embodiment, the electronic device <NUM> may display indicators 1311a, 1313a and 1315a indicative of the state of the electronic device (e.g., a battery communication connection state, a charging state, time information, or the like), an activated function (e.g., Bluetooth, alarm, or the like) and notification information in a first area <NUM>, a second area <NUM>, and a third area <NUM> of the upper end <NUM> of the display screen, respectively.

According to an embodiment, the electronic device <NUM> may display an indicator indicative of state information of the digital pen <NUM> in the second and third areas <NUM> and <NUM>.

According to an embodiment, the electronic device <NUM> may display, on the display <NUM>, an indicator 1311a. 1313a, and 1315a indicating the state information of the digital pen <NUM>. According to an embodiment, when the digital pen <NUM> is detached from the electronic device <NUM>, the electronic device <NUM> may display an indicator indicating state information of the digital pen <NUM> in one of the second and third areas <NUM> and <NUM>. In addition, the electronic device <NUM> may not display any indicator when the digital pen <NUM> is inserted into the electronic device <NUM>.

<FIG> is a view illustrating an indicator indicating a charging state and a wireless communication connection state of a digital pen by an electronic device according to an embodiment of the disclosure.

Referring to <FIG>, an electronic device (e.g., the electronic device <NUM> of <FIG>) may indicate a wireless communication connection state 1410a and a charging state 1410b of a digital pen (e.g., the digital pen <NUM> of <FIG>) through an indicator <NUM>. According to an embodiment, the electronic device may display the indicator <NUM> in a second area (e.g., the second area <NUM> of <FIG>).

According to an embodiment, the electronic device <NUM> may display an indicator <NUM> in a first state 1410_1, a second state 1410_2, a third state 1410_3, a fourth states 1410_4, and a fifth state 1410_5 according to the charging state 1410b of the digital pen <NUM>. For example, the electronic device <NUM> may indicate the charging state 1410b of the digital pen <NUM> with the same gap (e.g., <NUM>%, <NUM>%, <NUM>%, <NUM>%, or <NUM>%). As another example, the gap may be specified by more enlarging the gap in a state in which the charging of the battery (e.g., the battery <NUM> of <FIG>) is high, or by displaying a remaining amount of the battery of the digital pen <NUM> of a certain level or more capable of controlling only the minimum action related to disconnection from the electronic device <NUM> as the lowest charge state. When the gap is made larger in a state where the battery is charged at a high level, the battery state may be displayed at the same level even if the maximum charging amount is reduced due to deterioration of the battery. According to an embodiment, the electronic device may receive information about the charging state of the battery <NUM> from the digital pen. The digital pen <NUM> may transmit a signal including information on the charging state at a specified time period.

<FIG> is a view illustrating an indicator indicating a state of setting connection with a digital pen by an electronic device according to an embodiment of the disclosure.

Referring to <FIG>, an electronic device (e.g., the electronic device <NUM> of <FIG>) may display, through an indicator <NUM>, the state of setting a connection using wireless communication. According to an embodiment, the electronic device may display the indicator <NUM> in a second area (e.g., the second area <NUM> of <FIG>).

According to an embodiment, the electronic device may indicate a wireless communication connection (e.g., connecting or pairing) by alternately displaying an indicator 1510_1 in the first state and an indicator 1510_3 in the second state.

<FIG> is a view illustrating an indicator indicating a warning about a state of a digital pen of an electronic device according to an embodiment of the disclosure.

Referring to <FIG>, an electronic device (e.g., the electronic device <NUM> of <FIG>) may indicate, through an indicator <NUM>, a warning about the connection state of the wireless communication and the battery charge state. According to an embodiment, the electronic device may display the indicator <NUM> in a third area (e.g., the third area <NUM> of <FIG>).

According to an embodiment, the electronic device may display an indicator 1610_1 in a normal action state. For example, the normal action state may be a state in which a wireless communication connection with the electronic device is set and the remaining charging amount of the battery <NUM> of the digital pen <NUM> is sufficient to perform a specified action.

According to an embodiment, the electronic device <NUM> may display an indicator 1610_3 in a disconnected state with the digital pen <NUM>. According to an embodiment, the electronic device <NUM> may display an indicator 1610_5 in a state in which the remaining charge amount of the battery <NUM> is low. For example, the low state of a charge amount of the battery <NUM> may be a state in which it is insufficient to perform a specified action.

<FIG> is a view illustrating a screen showing a panel showing a state of a digital pen of an electronic device according to an embodiment of the disclosure.

Referring to <FIG>, the electronic device <NUM> may display state information 1711a of the digital pen <NUM> on a portion 'a' of a panel <NUM> which is scrolled down at an upper end of a display screen <NUM>.

According to an embodiment, when the digital pen <NUM> is not connected to the communication module <NUM>, the electronic device <NUM> may display information 1711a_1 about a state in which the digital pen <NUM> is not connected to the panel <NUM>. According to an embodiment, when the charge amount of the battery <NUM> of the digital pen <NUM> is not sufficient, the electronic device <NUM> may display charge amount information 1711a_3 of the battery <NUM> of the digital pen <NUM> on the panel <NUM>. When an input 1710b for selecting the charge amount information 1711a_3 of the battery <NUM> of the digital pen <NUM> is received, the electronic device <NUM> my provide information for charging the battery <NUM> of the digital pen <NUM> to the user.

According to an embodiment, when the connection attempt with the digital pen <NUM> fails, the electronic device <NUM> may display connection failure information 1711a_5 of the digital pen <NUM> on the panel <NUM>. When receiving an input 1710c for selecting the connection failure information 1711a_5 of the digital pen <NUM>, the electronic device <NUM> may retry the action of wirelessly connecting with the digital pen <NUM>.

<FIG> is a view illustrating a method of setting a remote action to an app being executed by an electronic device according to an embodiment of the disclosure.

Referring to <FIG>, a remote action may not be set to an app (e.g., a media app) being executed in the electronic device <NUM>. The electronic device <NUM> may display the GUI of the app being executed on a display screen <NUM>.

According to an embodiment, when the digital pen <NUM> is detached from the electronic device <NUM>, the electronic device <NUM> may display an object (e.g., a floating icon) <NUM> on the display <NUM>. According to an embodiment, the electronic device <NUM> may set at least one remote action corresponding to at least one user input received through the side button <NUM> of the digital pen <NUM> through the object <NUM>. For example, the electronic device <NUM> may set a remote action (e.g., music playback and pause) corresponding to an input <NUM> of pressing the side button <NUM> of the digital pen <NUM> once. In addition, the electronic device <NUM> may set a remote action (e.g., a skip action) corresponding to an input <NUM> of pressing the side button <NUM> of the digital pen <NUM> once. According to an embodiment, the electronic device <NUM> may provide the user with information 1811a and 1811b related to at least one settable remote action. The information 1811a and 1811b related to at least one settable remote action may be displayed adjacent to the object <NUM>.

<FIG> is a view illustrating a method of setting a remote action to an app installed in an electronic device according to an embodiment of the disclosure.

Referring to <FIG>, the electronic device <NUM> may set a remote action corresponding to an input of pressing and holding the side button <NUM> of the digital pen <NUM>.

According to an embodiment, the electronic device <NUM> may display, on a display screen <NUM>, a list <NUM> of apps for setting remote actions corresponding to inputs of pressing and holding the side button <NUM> of the digital pen <NUM>. The list <NUM> of apps may include an object (e.g., an icon) corresponding to the installed app. According to an embodiment, the electronic device <NUM> may separately display a recommended app or the like.

According to an embodiment, the electronic device <NUM> may receive an input 1910a for selecting one object from the list <NUM> of apps and set a remote action corresponding to the long-pressing input. According to an embodiment, the electronic device <NUM> may display, on a display screen <NUM>, information about a remote action executed by a long-pressing input in the selected app.

According to an embodiment, the electronic device <NUM> may turn on or off a function of remotely controlling a remote action corresponding to an input of pressing and holding the side button <NUM> of the digital pen <NUM> through an activation button <NUM>. For example, when the remote control function is turned on through the activation button <NUM>, the electronic device <NUM> may display the list <NUM> of apps on the display <NUM>. In addition, when the remote control function is turned off through the activation button <NUM>, the electronic device <NUM> may hide the list <NUM> of apps from the display screen <NUM>. The electronic device <NUM> according to the various embodiments of the disclosure described with reference to in <FIG> may perform an action corresponding to a user input received through the side button <NUM> of the digital pen <NUM> based on the state of the app being executed, so that the user may remotely control the electronic device <NUM> by using the side button <NUM> of the digital pen <NUM> without changing the setting for the input.

In addition, when the digital pen <NUM> is detached from the electronic device <NUM>, the electronic device <NUM> may allow the user to clearly recognize the execution contents of an action by the user input by providing the guide information for the action corresponding to the user input received through the side button <NUM> of the digital pen <NUM>.

The electronic device according to various embodiments disclosed in the disclosure may be various types of devices. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a mobile medical appliance, a camera, a wearable device, or a home appliance. The electronic device according to an embodiment of the disclosure should not be limited to the above-mentioned devices.

It should be understood that various embodiments of the disclosure and terms used in the embodiments do not intend to limit technical features disclosed in the disclosure to the particular embodiment disclosed herein; rather, the disclosure should be construed to cover various modifications, equivalents, or alternatives of embodiments of the disclosure. With regard to description of drawings, similar or related components may be assigned with similar reference numerals. As used herein, singular forms of noun corresponding to an item may include one or more items unless the context clearly indicates otherwise. In the disclosure disclosed herein, each of the expressions "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "one or more of A, B, and C", or "one or more of A, B, or C", and the like used herein may include any and all combinations of one or more of the associated listed items. The expressions, such as "a first", "a second", "the first", or "the second", may be used merely for the purpose of distinguishing a component from the other components, but do not limit the corresponding components in other aspect (e.g., the importance or the order). When an (e.g., first) element is referred to as being "(operatively or communicatively) coupled with/to" or "connected to" another (e.g., second) element with or without term "functionally" or "communicationally", it may be directly coupled with/to or connected to the other element or an intervening element (e.g., a third element) may be present.

The term "module" used in the disclosure may include a unit implemented in hardware, software, or firmware and may be interchangeably used with the terms "logic", "logical block", "part" and "circuit". The "module" may be a minimum unit of an integrated part or may be a part thereof. The "module" may be a minimum unit for performing one or more functions or a part thereof. For example, according to an embodiment, the "module" may include an application-specific integrated circuit (ASIC).

Various embodiments of the disclosure may be implemented by software (e.g., the program <NUM>) including an instruction stored in a machine-readable storage medium (e.g., an internal memory <NUM> or an external memory <NUM>) readable by a machine (e.g., the electronic device <NUM> and <NUM>). For example, the processor of a machine (e.g., the electronic device <NUM> and <NUM>) may call the instruction from the machine-readable storage medium and execute the instructions thus called. This means that the machine may perform at least one function based on the called at least one instruction. The one or more instructions may include a code generated or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of non-transitory storage medium. Here, the term "non-transitory", as used herein, means that the storage medium is tangible, but does not include a signal (e.g., an electromagnetic wave). The term "non-transitory" does not differentiate a case where the data is permanently stored in the storage medium from a case where the data is temporally stored in the storage medium.

The method according to various embodiments disclosed in the disclosure may be provided as a part of a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of machine-readable storage medium (e.g., a compact disc read only memory (CD-ROM)) or may be directly distributed (e.g., download or upload) online through an application store (e.g., a Play Store™) or between two user devices (e.g., the smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or generated in a machine-readable storage medium such as a memory of a manufacturer's server, an application store's server, or a relay server.

According to various embodiments, each component (e.g., the module or the program) of the above-described components may include one or plural entities. According to various embodiments, at least one or more components of the above components or operations may be omitted, or one or more components or operations may be added. Alternatively or additionally, some components (e.g., the module or the program) may be integrated in one component. In this case, the integrated component may perform the same or similar functions performed by each corresponding components prior to the integration. According to various embodiments, operations performed by a module, a programming, or other components may be executed sequentially, in parallel, repeatedly, or in a heuristic method, or at least some operations may be executed in different sequences, omitted, or other operations may be added.

According to the embodiments, the electronic device may perform an action corresponding to a user input received through the button of the digital pen based on the state of the app being executed, so that the user may remotely control the electronic device by using the button of the digital pen <NUM> without changing the setting for the input.

In addition, when the digital pen is detached from the electronic device, the electronic device may allow the user to clearly recognize the execution contents of an action by the user input by providing the guide information for the action corresponding to the user input received through the button of the digital pen.

In addition, various effects that are directly or indirectly understood through the disclosure may be provided.

Claim 1:
An electronic device (<NUM>, <NUM>) comprising:
a digital pen (<NUM>);
a housing (<NUM>), wherein the digital pen is inserted into the housing;
a wireless communication circuit (<NUM>, <NUM>);
a display (<NUM>, <NUM>);
a processor (<NUM>, <NUM>); and
a memory (<NUM>, <NUM>) operatively connected to the processor,
wherein the memory stores instructions which, when executed, cause the processor to:
in response to receiving a first input through the wireless communication circuit from the digital pen, identify an application among a plurality of applications being executed for performing an operation corresponding to the first input, the plurality of applications being executed including a first application and a second application different from the first application,
wherein the first input is received through a button included in the digital pen and corresponds to a first operation of the first application and to a second operation of the second application,
when the identified application corresponds to the first application, perform the first operation of the first application corresponding to the first input, and
when the identified application corresponds to the second application perform the second operation of the second application corresponding to the first input,
wherein the first operation is different from the second operation;
characterised in that,
to identify the application for performing an operation, the instructions further cause the processor to:
identify whether the first application is executed in a foreground from the plurality of applications being executed in the electronic device,
identify whether the first operation is capable of being performed by the first application;
when the first operation is capable of being performed by the first application, display, on the display, an interface that provides information about the first operation,
when the first operation is incapable of being performed by the first application, identify the second application being executed in a background among the plurality of applications being executed,
identify whether the second operation is capable of being performed by the second application, and
when the second operation is capable of being performed by the second application, display, on the display, an interface that provides information about the second operation..