Patent Description:
With diversification of functions, a smart phone and the like electronic apparatus have already become necessities for modern people. As one of the functions, there is a function of using the electronic apparatus to control another electronic apparatus (hereinafter, referred to as an 'external apparatus'). By the function of controlling the external apparatus, a user makes an input to the electronic apparatus and controls the external apparatus through the electronic apparatus.

For example, when various functions the external apparatus has are displayed on a screen of the electronic apparatus in the form of a graphic user interface (GUI), a user makes a touch input or the like for selecting one of various functions displayed as the GUI while looking at the screen of the electronic apparatus. Then, the electronic apparatus recognizes a user's touch input received through the touch screen, and controls the external apparatus to implement a function corresponding to the GUI selected by the recognized touch input.

<CIT> describes a control method applied to an electronic device on which at least two sensing units are arranged. <CIT> describes methods and systems for controlling electronic devices through digital signal processor and handler logic. <CIT> describes a user terminal device, a method of controlling the user terminal device, and a system for providing content.

The user terminal device includes a communicator configured to communicate with a display device, a detector configured to detect a distance between the user terminal device and the display device, a plurality of sensors configured to receive input of a user command, and a controller. The plurality of sensors comprise a proximity sensor configured to receive a user command corresponding to a detection of a gesture of a user who is hovering with a finger above a finger hovering area of the proximity sensor. The controller is configured to activate the proximity sensor in response to a detection that the user terminal device is within a predetermined distance with respect to the display device, thereby allowing the user to control an operation of the display device based on a gesture detected by the proximity sensor of the user terminal device. To ensure easy manipulation of the user terminal device even in a dark indoor place, the display device is configured to sense the surrounding illuminance and send a brightness control signal to the user terminal device when the sensed illuminance is less than a preset threshold value. The user terminal device controls a backlight in a housing of the user terminal device when the brightness control signal is received.

<CIT> describes an electronic device including a biometric sensor that identifies an input received. <CIT> describes an electronic device having an infrared sensing assembly for detecting and interpreting gestures.

When a user input is made by touching a GUI displayed on a touch screen of an electronic apparatus, as it is sometimes the case in the prior art, a user has to look at the screen of the electronic apparatus in order to make an input in the middle of watching the screen of the external apparatus, and it is thus inconvenient for the user since his/her eyes are moved from the external apparatus to the electronic apparatus. To solve such a problem, if a free motion type of user input is made instead of the touch input using the GUI, the electronic apparatus additionally needs a motion recognition sensor to recognize a user motion and therefore causes another problem of increasing costs.

There is provided an electronic apparatus in accordance with claim <NUM> and a method of controlling an electronic apparatus in accordance with claim <NUM>. Other aspects of the invention are set forth in the dependent claims.

The above and/or the aspects will become apparent and more readily appreciated from the following description of exemplary embodiments, taken in conjunction with the accompanying drawings, in which:.

Below, embodiments of the disclosure will be described in detail with reference to accompanying drawings. In the description of the following embodiments, elements illustrated in the accompanying drawings will be referenced, and like numerals or symbols set forth in the drawings refer to like elements having substantially the same function. In the disclosure, at least one among a plurality of elements refers to not only all the plurality of elements but also both each one of the plurality of elements excluding the other elements and a combination thereof.

<FIG> illustrates an example that an electronic apparatus according to one embodiment of the disclosure controls an external apparatus. <FIG> is given on the assumption that a user controls an external apparatus <NUM> through an electronic apparatus <NUM> while looking an image <NUM> displayed on the external apparatus <NUM>. The external apparatus <NUM> may be a television (TV) or the like display apparatus, but may be actualized without limitation in the disclosure. For example, the external apparatus <NUM> may be actualized by various apparatus such as a computer, a multimedia player, an electronic frame, a digital billboard, a large format display (LFD), a digital signage, a set-top box, a refrigerator, etc..

The electronic apparatus <NUM> refers to a separate apparatus from the external apparatus <NUM>, and may for example be actualized by a smart phone, a smart pad, a wearable smart device, a mobile phone, a tablet personal computer (PC), an electronic book terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), etc. without limitation.

The electronic apparatus <NUM> may perform various function in accordance with its actualization forms. For example, the electronic apparatus <NUM> may perform not only controlling the external apparatus <NUM> but also various functions such as playing back content, making a voice call, making a video call, capturing a moving picture, capturing a still picture, sending a message, browsing Internet, etc..

To perform the foregoing functions, the electronic apparatus <NUM> uses a sensor value obtained by a sensor. For example, the electronic apparatus <NUM> may perform the function of capturing a still picture by executing a photographing application. The electronic apparatus <NUM> may control a focus or exposure of a camera on the basis of a proximity sensor value from a proximity sensor and an optical sensor value from an optical sensor. Like this, a mode where the electronic apparatus <NUM> employs the proximity sensor, the optical sensor and the like sensor and performs its own functions such as the function of capturing a still picture, etc. will be called a first mode. In the first mode according to an exemplary embodiment, various functions such as playing back content, making a voice call, making a video call, capturing a moving picture, capturing a still picture, sending a message, browsing Internet, etc. may be performed.

The electronic apparatus <NUM> according to an exemplary embodiment controls the functions of the external apparatus <NUM> in response to a user input. A mode where the electronic apparatus <NUM> controls the functions of the external apparatus <NUM> except the foregoing functions will be called a second mode. In the second mode, the electronic apparatus <NUM> may recognize a user input for controlling the functions of the external apparatus <NUM> through the plurality of sensors.

In the second mode, the user input includes a user motion. That is, the electronic apparatus <NUM> employs a plurality of sensors, which are used in carrying out its own functions in the first mode, to recognize the user motion for controlling the external apparatus <NUM> in the second mode. As an example of the plurality of sensors according to an exemplary embodiment, the proximity sensor and the optical sensor are used for the functions of the electronic apparatus <NUM> to set a photographing condition in the first mode, but used to recognize a user motion for controlling the external apparatus <NUM> in the second mode.

In more detail, it will be assumed that the user motion is for example activity of a hand among body parts, and a motion of moving a user's hand from left to right is a control command for turning up the volume of the external apparatus <NUM>.

When a user makes a motion of moving a hand from left to right with respect to the electronic apparatus <NUM> in the second mode, the electronic apparatus <NUM> recognizes that the motion is made from left to right on the basis of the sensor values of the proximity sensor and the optical sensor. In an example not covered by the claims, the electronic apparatus <NUM> transmits information about the recognized motion to the external apparatus <NUM>, and the external apparatus <NUM> determines the control command based on the received information about the motion, and performs an operation of turning the volume up in response to the control command. In accordance with the claimed invention, the electronic apparatus <NUM> transmits the control command, e.g., for turning up the volume of the external apparatus <NUM> to the external apparatus <NUM> on the basis of the information about the recognized motion, and the external apparatus <NUM> may perform the operation of turning up the volume in response to the received control command.

The motion in this embodiment is not limited to a hand motion, and the electronic apparatus <NUM> may sense a motion based on another body part such as a user's face, arm, body, leg, etc. Further, the electronic apparatus <NUM> may be designed to recognize directions of various motions, and different control commands may be respectively assigned to the directions of the motions. For example, the electronic apparatus <NUM> may recognize left/right directions, up/down directions, a diagonal direction, a perpendicularly approaching/receding direction, etc. with respect to the electronic apparatus <NUM>. Corresponding to the directions of the motions of the electronic apparatus <NUM>, various control commands for power on/off, channel up/down, volume up/down, content selection, content downloading, content play, content stop, content deletion, etc. in the external apparatus <NUM>.

Thus, the electronic apparatus <NUM> according to an exemplary embodiment is more convenient for a user because the user can input a control command based on a motion without looking at the screen of the electronic apparatus <NUM> for a touch input and thus control the external apparatus <NUM> while watching the external apparatus <NUM>. Further, the plurality of sensors used to carry out the functions of the electronic apparatus <NUM> are employed in recognizing the user motion for controlling the external apparatus <NUM>, and thus costs are reduced because there are no needs of separate motion recognition sensor for recognizing the user motion.

<FIG> is a block diagram of the electronic apparatus in <FIG>. The electronic apparatus <NUM> in <FIG> includes a display <NUM>, a communicator <NUM>, a processor <NUM>, and a sensor unit <NUM>, and may include a user input unit <NUM>, a sound receiver <NUM>, a sound output unit <NUM>, a storage <NUM>, and an image capturer <NUM>. However, the electronic apparatus <NUM> may be designed to exclude at least one optional element from the elements shown in <FIG> or add another element not shown in <FIG>.

The sensor unit <NUM> includes a plurality of sensors for obtaining various sensor values related to the electronic apparatus <NUM>. For example, the sensor unit <NUM> may include an image sensor <NUM>, a proximity sensor <NUM>, an optical sensor <NUM>, a sound sensor, a temperature sensor, a humidity sensor, a pressure sensor, an impact sensor, a depth sensor, a global positioning system (GPS) sensor, a gyro sensor, etc..

The image sensor <NUM> obtains a capture image of a front direction thereof. The image sensor <NUM> may be actualized by a camera of a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD). There may be two or more image sensors <NUM>.

The proximity sensor <NUM> is configured to sense a distance from a surrounding object. For example, when the proximity sensor <NUM> employs a time-of-flight method for measuring an infrared signal, the electronic apparatus <NUM> measures a distance from a moving object by measuring a delay time until an output infrared signal is reflected and returns from the object. When the proximity sensor <NUM> employs a magnetic measurement method of an object, a distance from the moving object is measured based on a magnetic field, magnetic intensity, a magnetic field direction, change in magnetic force, etc. However, there are no limits to the kind of the proximity sensor <NUM>, and the proximity sensor <NUM> may be actualized by a magnetic saturation type, a high-frequency oscillation type, a differential coil, electrostatic capacitance type, etc..

The optical sensor <NUM> measures the intensity and amount of light around the electronic apparatus <NUM>. The optical sensor <NUM> may for example be actualized by lead sulfide (PbS) or cadmium sulfide (CdS) photoconductive cells. According to an exemplary embodiment, the image sensor <NUM> may serve as the optical sensor <NUM>.

The display <NUM> displays an image based on an image signal. The display <NUM> may be actualized by a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a <NUM>-dimensional (3D) display, etc. The display <NUM> may be also actualized by a touch screen together with the user input unit <NUM> for receiving a touch input.

The user input unit <NUM> receives a user input for controlling the operations of the electronic apparatus <NUM>. The user input unit <NUM> may include a touch screen, a key pad, a dome switch, a touch pad (static pressure/electrostatic), a jog wheel, a jog switch, a finger mouse, etc..

The sound receiver <NUM> receives and processes a sound signal into an electric audio signal. The sound receiver <NUM> may use various noise removal algorithms to remove noise generated while receiving a sound signal. The sound receiver <NUM> may be actualized by a microphone, and the microphone may be used as the sound sensor. The sound output unit <NUM> may output a processed audio signal to the outside. The sound output unit <NUM> may be actualized by a loudspeaker.

The communicator <NUM> may perform wireless communication with the external apparatus <NUM> through various communication standards. For example, the communicator <NUM> may perform wireless communication based on Wi-Fi, Bluetooth, ZigBee, Wi-Fi Direct (WFD), ultra-wideband (UWB), infrared Data Association (IrDA), Bluetooth low energy (BLE), near field communication (NFC), etc..

The communicator <NUM> may further include a mobile communication mobile <NUM> for transmitting and receiving a wireless signal to and from at least one of a base station, an external terminal and a server on a mobile communication network, a wireless Internet module <NUM> for wireless connection with the Internet, a near field communication module for near field communication, a GPS module, etc..

The storage <NUM> may be configured to store data, a program, or an application for various functions to be implemented by the electronic apparatus <NUM>. The storage <NUM> may include a storage medium of at least one type among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g. a secure digital (SD) or extreme digital (XD) memory, etc.), a random access memory (RAM), and a read only memory (ROM). The storage <NUM> may be actualized by a web storage that operates on the Internet.

The processor <NUM> may generally control the elements of the electronic apparatus <NUM>. The processor <NUM> performs the functions of the electronic apparatus <NUM> based on the sensor value sensed by the sensor unit <NUM> in the first mode. Further, the processor <NUM> performs a control function to recognize a user motion, and outputs a control command indicating the direction of the recognized motion to the external apparatus <NUM>, based on the sensor value sensed by the sensor unit <NUM> in the second mode different from the first mode.

The processor <NUM> executes a control program (or an instruction) for generally controlling the elements as described above. The electronic apparatus <NUM> may include a nonvolatile memory in which the control program is installed, and a volatile memory into which at least a part of the installed control program is loaded. Further, such a control program may be also stored in another electronic apparatus besides the electronic apparatus <NUM>.

The control program may include a program(s) actualized in the form of at least one among a basis input/output system (BIOS), a device driver, an operating system, a firmware, a platform, and an application. According to an exemplary embodiment, the application may be previously installed or stored when the electronic apparatus <NUM> is manufactured, or may be installed based on application data received from the outside when it is used in the future. The application data may for example be downloaded from an application market and the like external server. Such an external server is an example of a computer program product, but not limited to this example.

The electronic apparatus <NUM> may further include a signal processor and a power supply. The signal processor applies various processes to a received video or audio signal so that the display <NUM> or the sound output unit <NUM> can output an image or a sound. The signal processor may include a hardware processor actualized by a chipset, a circuit, a buffer, etc. mounted on to a printed circuit board, and may also be designed as a system on chip (SoC). Further, the power supply may receive external power or internal power and supply power necessary for the elements of the electronic apparatus <NUM> under control of the processor <NUM>.

<FIG> is a flowchart of a control method of the electronic apparatus in <FIG>. The control method of the electronic apparatus shown in <FIG> may be performed as the processor <NUM> of the electronic apparatus <NUM> shown in <FIG> executes at least one control program. In the following description, unless otherwise mentioned, the operation of the processor <NUM> is an operation performed by executing the control program.

First, the processor <NUM> performs an operation of the electronic apparatus <NUM> based on the sensor value sensed though the sensor unit <NUM> including the plurality of sensors in the first mode (S310). Further, the processor <NUM> identifies a direction of a user motion based on a sensor value sensed through the sensor unit <NUM>, and outputs a control command corresponding to the direction of the determined motion to the external apparatus <NUM>, in the second mode (S320).

In the operations S310 and S320, the processor <NUM> may identify whether a current mode is the first mode or the second mode in accordance with whether a preset event occurs or not. For example, the processor <NUM> identifies that the current mode is the first mode when the preset event has not occurred yet. When the present event occurs in the first mode, the processor <NUM> identifies that the current mode is the second mode. That is, the electronic apparatus <NUM> may switch over to the second mode and operate in the second mode in response to the occurrence of the preset event while operating in the first mode.

<FIG> illustrates an example that the electronic apparatus in <FIG> operates by switching over from a first mode to a second mode. In <FIG>, it will be assumed that the external apparatus <NUM> is displaying the image <NUM>, and a user <NUM> wants to control the display of the image <NUM> in the external apparatus <NUM>. Further, it will be also assumed that the electronic apparatus <NUM> carries out an operation in the first mode.

The operation in the first mode may for example include not only an operation for providing a function to be positively usable by the user <NUM>, such as playing back content, making a call, capturing an image, sending a message, browsing the Internet, etc., but also an operation for providing a function not to be positively usable by the user <NUM>, such as a standby mode of a smart phone, etc..

The processor <NUM> determines whether the preset event occurs while operating in the first mode. The present event in this embodiment may for example include that the user <NUM> of the external apparatus <NUM> moves close to the electronic apparatus <NUM> within a predetermined distance from the electronic apparatus <NUM>.

For example, when the user <NUM> moves toward the external apparatus <NUM> while taking the electronic apparatus <NUM> so as to watch the image <NUM> of the external apparatus <NUM>, the processor <NUM> identifies that the external apparatus <NUM> approaches the electronic apparatus <NUM> within a predetermined distance from the electronic apparatus <NUM> on the basis of information obtained through the communicator <NUM> or the like.

Alternatively, the preset event may include that the electronic apparatus <NUM> is connected for communication with the external apparatus <NUM>. A communication method between the electronic apparatus <NUM> and the external apparatus <NUM> may for example be Wi-Fi, Bluetooth, etc. For example, when the user <NUM> moves close to the external apparatus <NUM> while taking the electronic apparatus <NUM>, the processor <NUM> controls the communicator <NUM> to connect with the external apparatus <NUM> by a preset communication method such as Bluetooth, etc., and identifies that the preset event occurs based on the completion of such connection.

Alternatively, the preset event may include that the user input unit <NUM> receives a command of entering the second mode from the user <NUM>. For example, the processor <NUM> may identify whether a received touch input, voice command, etc. corresponds to a command to enter the second mode in response to the touch input, the voice command, etc. received on the touch screen during the operation in the first mode. In this case, the processor <NUM> displays a UI including a menu item, which is to receive a command of the user <NUM> to enter the second mode, on the display <NUM>, and identifies the preset event based on the touch input received through the menu item of the Ul.

Alternatively, the preset event may include that a request for entering the second mode is received from the external apparatus <NUM> or an external server <NUM>. For example, while the electronic apparatus <NUM> performs the operation of the first mode as connected for communication with the external apparatus <NUM> or the external server <NUM>, the processor <NUM> may identify whether a signal received from the external apparatus <NUM> or the external server <NUM> corresponds to a request for entering the second mode, based on the signal received from the external apparatus <NUM> or the external server <NUM> through the communicator <NUM>. The external apparatus <NUM> or the external server <NUM> may recognize a user's intention through the sensor or the like or receive a user's input, thereby transmitting a request signal for entering the second mode to the electronic apparatus <NUM>.

Alternatively, the preset event may include that the external apparatus <NUM> displays a preset image <NUM>. The preset image <NUM> displayed by the external apparatus <NUM> may for example be an image <NUM> of preset content or content from a preset category. For example, the processor <NUM> may identify that the external apparatus <NUM> displays the preset image <NUM> based on information included in a signal received from the external apparatus <NUM> through the communicator <NUM>. In this case, the external apparatus <NUM> may transmit a signal, which includes information about currently displayed content, to the electronic apparatus <NUM>.

Alternatively, the processor <NUM> may identify whether the external apparatus <NUM> displays the preset image <NUM> based on the information obtained by the sensor unit <NUM> such as the image sensor <NUM>, etc..

The preset event according to an exemplary embodiment is not limited to the foregoing examples, and may include various events besides the foregoing examples.

According to one embodiment, the processor <NUM> may identify that the preset event occurs based on an apparatus use history of a user. The apparatus use history of a user may for example be a use time, a use place, etc. of the electronic apparatus <NUM> or the external apparatus <NUM>. The preset event based on the apparatus use history may for example include that a user watches a TV in a certain place at a certain time. Further, the preset event according to an exemplary embodiment may be based on a user's daily life, such as a wake-up time, a bed time, etc. of a user.

According to an exemplary embodiment, the apparatus use history of the user or the information about a user's daily life may be stored in the electronic apparatus <NUM> or may be stored in the external apparatus <NUM> or the external server <NUM>. When the information is stored in the external apparatus <NUM> or the external server <NUM>, the processor <NUM> may receive information from the external apparatus <NUM>, the external server <NUM> or may receive a request for entering the second mode from the external apparatus <NUM> or external server <NUM> storing the information.

When it is identified that the preset event occurs during the operation of the first mode, the processor <NUM> may switch over to the second mode and then operate. Specifically, the processor <NUM> may execute an application dedicated to control of the external apparatus in response to the occurrence of the preset event. The dedicated application according to an exemplary embodiment may be one of at least one control program as described above.

According to an exemplary embodiment, the processor <NUM> may display a UI <NUM>, which guides a control mode of the external apparatus <NUM> to a user, on the display <NUM> when the preset event occurs. The UI <NUM> may include items for allowing a user to select whether to execute the control mode of the external apparatus <NUM>. When a user's touch input to execute the control mode is received through the UI <NUM>, the processor <NUM> may execute the dedicated application and enter the second mode.

<FIG> illustrates an example in accordance with the claimed invention that the electronic apparatus in <FIG> displays a control image Referring to <FIG>, the processor <NUM> controls the display <NUM> to display an image (hereinafter, referred to as a "control image") for controlling the external apparatus <NUM> in response to entering the second mode.

According to an example not covered by the claims, the brightness or color of a control image <NUM> may be previously determined. According to the claimed invention, the brightness or color of the control image <NUM> varies depending on surrounding environments. The control image <NUM> serves to let a user intuitively know the position of the electronic apparatus <NUM> without moving his/her eyes toward the electronic apparatus <NUM>, when the user makes a motion while watching the screen <NUM> of the external apparatus <NUM> in order to control the external apparatus <NUM>. Therefore, the control image <NUM> according to the claimed invention is configured to have the brightness or corresponding color so that a user can easily estimate the position of the electronic apparatus <NUM> while watching the screen <NUM> of the external apparatus <NUM>.

For example, the processor <NUM> may control the brightness of the control image <NUM> to be higher than intensity of surrounding illumination. The processor <NUM> may identify the surrounding illumination based on information obtained by the image sensor <NUM>, the optical sensor <NUM>, etc. Thus, a user can easily estimate the position of the electronic apparatus <NUM> based on the control image <NUM> brighter than the surrounding illumination.

As an additional or alternative example, the processor <NUM> may control the color of the control image <NUM> to be distinguishable from the color of ambient light. The processor <NUM> may identify the color of the ambient light based on information obtained by the image sensor <NUM>, the optical sensor <NUM>, etc. For example, the processor <NUM> may set a color complementary to the color of the ambient light as the color of the control image <NUM>. Therefore, a user can more easily estimate the position of the electronic apparatus <NUM> based on the control image <NUM> showing a contrast to the color of the ambient light.

In accordance with the claimed invention, the processor <NUM> identifies the surrounding illumination or the color of the ambient light based on the brightness or color <NUM> of the image <NUM> displayed on the external apparatus <NUM>. The processor <NUM> receives information about the image <NUM> displayed on the external apparatus <NUM> from the external apparatus <NUM> through the communicator <NUM>, and identifies the surrounding illumination or the color of the ambient light based on the received information. As an optional example, the processor <NUM> may change the brightness or color of the control image <NUM> in accordance with change in the surrounding illumination or the color of the ambient light. As an optional example not covered by the claims, the processor <NUM> may set the brightness or color of the control image <NUM> based on settings set up by a user. In this case, the processor <NUM> may control the display <NUM> to display a UI including menu items for setting the brightness or color of the control image <NUM>, and set the brightness or color of the control image <NUM> based on a user input received through the Ul.

The processor <NUM> recognizes a user motion for controlling the external apparatus <NUM> based on a sensor value obtained by the sensor unit <NUM> in the second mode, and performs control to transmit information about the recognized user motion to the external apparatus <NUM>. Below, an example that the electronic apparatus <NUM> according to an exemplary embodiment recognizes the user motion based on the sensor value obtained by the sensor unit <NUM> will be described in detail.

<FIG> and <FIG> illustrate examples that the electronic apparatus in <FIG> recognizes a user motion in accordance with a change in a sensor value. The electronic apparatus <NUM> includes a case <NUM> forming an outer appearance. The case <NUM> is provided as the front and rear of the electronic apparatus <NUM>, forming a space to accommodate the elements of the electronic apparatus <NUM>.

On the case <NUM>, there is provided the display <NUM>, and there may be provided the proximity sensor <NUM>, the optical sensor <NUM>, the user input unit <NUM>, the sound receiver <NUM>, the sound output unit <NUM>, and the image capturer <NUM>. Alternatively, the electronic apparatus <NUM> may be designed to exclude at least one optional element from or include an additional element to the elements provided on the case <NUM>.

The proximity sensor <NUM> and the optical sensor <NUM> of the electronic apparatus <NUM> may be arranged leaving a predetermined distance D on the case <NUM>. The proximity sensor <NUM> of the electronic apparatus <NUM> may be arranged at the left of the optical sensor <NUM> and spaced apart at a predetermined distance from the optical sensor <NUM> in a widthwise direction. However, there are no limits to the arrangement or direction. Alternatively, the two sensors <NUM> and <NUM> may be arranged in a lengthwise or diagonal direction of the electronic apparatus <NUM>, and the optical sensor <NUM> may be also arranged at the left of the proximity sensor <NUM> as spaced apart at a predetermined distance D from the proximity sensor <NUM>.

According to an exemplary embodiment, it will be assumed that the proximity sensor <NUM> outputs a sensor value which decreases as the object approaches the proximity sensor <NUM> and thus a distance between the proximity sensor <NUM> and an object becomes shorter, and the optical sensor <NUM> outputs a sensor value which increases as the object blocks out light incident to the optical sensor <NUM> and thus the intensity of the light decreases. However, such sensing features of the proximity sensor <NUM> and the optical sensor <NUM> are merely an example, and may be designed to conversely obtain sensor values.

<FIG> shows a user's general operation of moving a hand <NUM> from the left toward the right of the electronic apparatus <NUM> as divided into (a) and (b). (a) in <FIG> illustrates that the hand <NUM> of the user is positioned at the left of the electronic apparatus <NUM>, and (b) in <FIG> illustrates that the hand <NUM> is positioned at the right of the electronic apparatus <NUM> as moved from the left. Further, the graph of <FIG> shows relative changes of the proximity sensor value and the optical sensor value as the hand <NUM> moves from the left toward the right of the electronic apparatus <NUM>.

Referring to the graph of <FIG>, a proximity sensor value gradually decreases as the hand <NUM> moves from the left toward the right of the electronic apparatus <NUM>, reaches a minimum value Ma at a first point in time a1, maintains the minimum value Ma, and increases at a second point in time a2. Here, the minimum value Ma refers to a proximity sensor value corresponding to the shortest distance between the hand <NUM> and the proximity sensor <NUM>.

The optical sensor value gradually increases, reaches a maximum value Mb at a third point in time b1 when the hand <NUM> covers the optical sensor <NUM>, maintains the maximum value Mb, and decreases at a fourth point in time b2. Here, the maximum value Mb refers to an optical sensor value corresponding to the lowest intensity of light blocked by the hand <NUM>.

Due to difference in position between the proximity sensor <NUM> and the optical sensor <NUM>, the first point in time a1 at which the proximity sensor value reaches the minimum value Ma precedes the third point in time b1 at which the optical sensor value reaches the maximum value Mb. That is, when the first point in time a1 precedes the third point in time b1, the electronic apparatus <NUM> determines that a motion is made in a direction from the left toward the right of the electronic apparatus <NUM>.

To more accurately determine the direction of the user motion, the electronic apparatus <NUM> may also determine whether the second point in time a2 at which the proximity sensor value increases from the minimum value Ma precedes the fourth point in time b2 at which the optical sensor value decreases from the maximum value Mb. That is, when the second point in time a2 precedes the fourth point in time b2, the electronic apparatus <NUM> determines that a motion is made in a direction from the left toward the right of the electronic apparatus <NUM>.

<FIG> shows a user's general operation of moving the hand <NUM> from the right to the left of the electronic apparatus <NUM> as divided into (a) and (b). (a) in <FIG> illustrates that the hand <NUM> of the user is positioned at the right of the electronic apparatus <NUM>, and (b) in <FIG> illustrates that the hand <NUM> is positioned at the left of the electronic apparatus <NUM> as moved from the right. Further, the graph of <FIG> shows relative changes of the proximity sensor value and the optical sensor value as the hand <NUM> moves from the right to the left of the electronic apparatus <NUM>.

Referring to the graph of <FIG>, due to difference in position between the proximity sensor <NUM> and the optical sensor <NUM>, the first point in time a1 at which the proximity sensor value reaches the minimum value Ma follows the third point in time b1 at which the optical sensor value reaches the maximum value Mb. That is, when the first point in time a1 lags behind the third point in time b1, the electronic apparatus <NUM> determines that a motion is made in a direction from the right to the left of the electronic apparatus <NUM>.

To more accurately determine the direction of the user motion, the electronic apparatus <NUM> may also determine whether the second point in time a2 at which the proximity sensor value increases from the minimum value Ma follows the fourth point in time b2 at which the optical sensor value decreases from the maximum value Mb. That is, when the second point in time a2 lags behind the fourth point in time b2, the electronic apparatus <NUM> determines that a motion is made in a direction from the right to the left of the electronic apparatus <NUM>.

Like this, the electronic apparatus <NUM> determines the direction of the user motion based on the change in the proximity sensor value and the optical sensor value. Thus, the electronic apparatus <NUM> is convenient for a user because the electronic apparatus <NUM> determines the direction of the user motion and the user does not have to move his/her eyes, which have been focused on the external apparatus <NUM> to be remotely controlled, to the electronic apparatus <NUM>.

<FIG> illustrates positions where sensors provided in the electronic apparatus in <FIG> are arranged. In <FIG> and <FIG>, the proximity sensor <NUM> and the optical sensor <NUM> are arranged leaving a predetermined distance D in the widthwise direction, but there are no limits to the distance D and the arrangement direction. Alternatively, various distances D and arrangement directions are possible.

For example, the proximity sensor <NUM> and the optical sensor <NUM> are arranged in the widthwise direction of the electronic apparatus <NUM> as shown in (a) of <FIG>, but the proximity sensor <NUM> and the optical sensor <NUM> may be respectively arranged at the left and the right of the sound output unit <NUM>. According to an alternative embodiment, the proximity sensor <NUM> may be arranged at the right and the optical sensor <NUM> may be arranged at the left.

With this arrangement, a predetermined distance D between the proximity sensor <NUM> and the optical sensor <NUM> is larger than those of <FIG> and <FIG>. As the distance D between the proximity sensor <NUM> and the optical sensor <NUM> increases, it is easier to determine whether the first point in time a1 precedes or follows the third point in time b1 and whether the second point in time a2 precedes or follows the fourth point in time b2.

Further, as shown in (b) of <FIG>, the proximity sensor <NUM> and the optical sensor <NUM> may be arranged in the lengthwise direction of the electronic apparatus <NUM>, and the proximity sensor <NUM> and the optical sensor <NUM> may be respectively the left and the right of the display <NUM>. As an alternative embodiment, it may be designed to arrange the proximity sensor <NUM> at the right and arrange the optical sensor <NUM> at the left.

With this arrangement, a predetermined distance D between the proximity sensor <NUM> and the optical sensor <NUM> is larger than that in (a) of <FIG>, and therefore it is easier to determine whether the first point in time a1 precedes or follows the third point in time b1 and whether the second point in time a2 precedes or follows the fourth point in time b2.

<FIG> illustrates an example of guiding an orientation direction of the electronic apparatus in <FIG>. When the electronic apparatus <NUM> is switched over to the second mode, a preferred orientation direction of the electronic apparatus <NUM> is guided in accordance with the arrangement of the proximity sensor <NUM> and the optical sensor <NUM>. The guiding method may include a method of displaying a UI through the display <NUM> or a method of outputting a sound through the sound output unit <NUM>.

According to an exemplary embodiment, as shown in (a) of <FIG>, when the proximity sensor <NUM> and the optical sensor <NUM> are arranged along the widthwise direction of the electronic apparatus <NUM>, a Ul <NUM> where the upper and lower sides of the electronic apparatus <NUM> are oriented toward the external apparatus <NUM> may be displayed on the display <NUM>. The Ul <NUM> may guide a user to correctly orient the electronic apparatus <NUM>. As necessary, a UI <NUM> indicating a motion direction recognizable by the electronic apparatus <NUM> may be further displayed to guide a user to correctly make the motion.

Further, as shown in (b) of <FIG>, the proximity sensor <NUM> and the optical sensor <NUM> are arranged along the lengthwise direction of the electronic apparatus <NUM>, a UI <NUM> where the left and right sides of the electronic apparatus <NUM> are oriented toward the external apparatus <NUM> may be displayed on the display <NUM>. As necessary, a UI <NUM> indicating the motion direction recognizable by the electronic apparatus <NUM> may be further displayed.

<FIG> and <FIG> illustrate examples that the electronic apparatus in <FIG> recognizes a user motion in accordance with patterns of a sensor value in the second mode. In the electronic apparatus <NUM> of <FIG> and <FIG>, it will be assumed that the proximity sensor <NUM> and the optical sensor <NUM> of are arranged along the widthwise direction of the electronic apparatus <NUM>, but not limited thereto.

<FIG> shows a user's general operation of moving the hand <NUM> from the bottom toward the top of the electronic apparatus <NUM> as divided into (a) and (b). (a) of <FIG> illustrates that the hand <NUM> of the user is positioned at the bottom of the electronic apparatus <NUM>, and (b) of <FIG> illustrates that the hand <NUM> is positioned at the top of the electronic apparatus <NUM> as moved from the bottom. Further, the graph of <FIG> shows relative changes of the proximity sensor value and the optical sensor value as the hand <NUM> moves from the bottom toward the top of the electronic apparatus <NUM>.

Referring to the graph of <FIG>, a proximity sensor value gradually decreases as the hand <NUM> moves from the bottom toward the top of the electronic apparatus <NUM>, and maintains the minimum value Ma from the first point in time a1. The optical sensor value gradually increases, and maintains the maximum value Mb from the third point in time b12 when the hand <NUM> covers the optical sensor <NUM>.

Because the proximity sensor <NUM> and the optical sensor <NUM> are arranged along the widthwise direction of the electronic apparatus <NUM>, the first point in time a1 may be equal to the third point in time b12. In other words, the electronic apparatus <NUM> may determines that the motion direction is oriented from the bottom toward the top of the electronic apparatus <NUM> when the first point in time a1 is equal to the third point in time b12.

However, the profile of the hand <NUM> is so irregular that the first point in time a1 and the third point in time b12 may be changed. For example, even when the hand <NUM> moves from the bottom toward the top of the electronic apparatus <NUM>, the third point in time b11 preceding the first point in time a1 or the third point in time b13 following the first point in time a1 may be detected.

In this case, the motion direction may be determined based on the patterns of the sensor values. Referring back to the graph of <FIG>, in which the hand <NUM> moves from the left toward the right, in order to describe the case of identifying the motion direction based on the pattern of the optical sensor value, the optical sensor value in the graph of <FIG> includes a first section w1 where the optical sensor value increases before the third point in time b1 at which the maximum value Mb starts, a second section w2 where the maximum value Mb is maintained, and a third section w3 where the optical sensor value decreases after the fourth point in time b2 at which the maximum value Mb ends.

On the other hand, referring to the graph of <FIG> in which the hand <NUM> moves from the bottom toward the top, the optical sensor value includes the first section w1 where the optical sensor value increases before the third point in time b12 at which the maximum value Mb starts, and a second section w2 where the maximum value Mb is maintained. In other words, when the hand <NUM> moves from the bottom toward the top, the optical sensor value does not include the third section w3 where the optical sensor value decreases after the fourth point in time b2 at which the maximum value Mb ends.

That is, the electronic apparatus <NUM> may determine that the hand <NUM> moves from the bottom toward the top based on the pattern of the optical sensor value including the first section w1 and the second section w2, even though it is unclear whether the first point in time a1 precedes or follows the third point in time b12 as they are changed. On the other hand, the electronic apparatus <NUM> may determine that the hand <NUM> moves from the left toward the right based on the pattern of the optical sensor value that includes the third section w3 as well as the first section w1 and the second section w2.

In the same manner, the electronic apparatus <NUM> can determine the motion direction based on the pattern of the proximity sensor value. That is, the electronic apparatus <NUM> determines that the hand <NUM> moves from the bottom toward the top when the pattern of the proximity sensor value does not include the section, in which the proximity sensor value increases from the minimum value Ma after the second point in time a2, as shown in the graph of <FIG>.

<FIG> shows a user's general operation of moving the hand <NUM> from the top toward the bottom of the electronic apparatus <NUM> as divided into (a), (b) and (c). (a) of <FIG> illustrates that the hand <NUM> of the user is positioned at the bottom of the electronic apparatus <NUM>, (b) of <FIG> illustrates that the hand <NUM> is positioned at the top as moved from the bottom. Here, (a) and (b) of <FIG> illustrate that the general operation of moving the hand <NUM> from the bottom toward the top, which will be a precondition for moving the hand <NUM> from the top toward the bottom. (a) and (b) of <FIG> involve vertical movement of the hand <NUM> having a height difference H greater than or equal to a predetermined threshold while the hand <NUM> moves from the bottom toward the top, the hand <NUM>, whereas (a) and (b) of <FIG> illustrate horizontal movement of the hand <NUM> having a height difference H less than or equal to the predetermined threshold.

Meanwhile, (c) of <FIG> illustrates that the hand <NUM> is positioned at the bottom as moved from the top. Further, the graph of <FIG> shows relative changes of the proximity sensor value and the optical sensor value as the hand <NUM> moves from the bottom toward the top of the electronic apparatus <NUM>.

Referring to the graph of <FIG>, the proximity sensor value gradually decreases as the hand <NUM> moves from the bottom toward the top of the electronic apparatus <NUM>, reaches the minimum value Ma at the first point in time a1, maintains the minimum value Ma, and starts increasing at the second point in time a2. The optical sensor value gradually increases, reaches the maximum value Mb at the third point in time b1 when the hand <NUM> covers the optical sensor <NUM>, maintains the maximum value Mb, and starts decreasing at the fourth point in time b2.

Because the proximity sensor <NUM> and the optical sensor <NUM> are arranged in the widthwise direction of the electronic apparatus <NUM>, the first point in time a1 may be equal to the third point in time b1, or the second point in time a2 may be equal to the fourth point in time b2. In this case, the electronic apparatus <NUM> may determine that the motion direction is oriented from the bottom toward the top of the electronic apparatus <NUM>.

However, the profile of the hand <NUM> is so irregular that the third point in time b1 and the fourth point in time b2 may be respectively changed with regard to the first point in time a1 and the second point in time a2. In this case, the electronic apparatus <NUM> may determine the motion direction based on the patterns of the sensor values.

Referring back to the graph of <FIG> and <FIG>, in which the hand <NUM> moves from the left toward the right, in order to describe the case of identifying the motion direction based on the pattern of the optical sensor value, the optical sensor value in the graph of <FIG> increases by a first change rate c1 in the first section w1, and the optical sensor value in the graph of <FIG> increases by a second change rate c2 in the first section w1.

However, the hand <NUM> in <FIG> and <FIG> shows the horizontal movement of which the height difference H is less than or equal to the predetermined threshold, but the hand <NUM> in <FIG> involves vertical movement of which the height difference H is greater than or equal to the predetermined threshold. Therefore, the optical sensor value in the graph of <FIG> increases by a third change rate c3 lower than the first change rate c1 or the second change rate c2.

That is, the electronic apparatus <NUM> may determine that the hand <NUM> moves from the top toward the bottom based on the pattern of the optical sensor value showing a gentle gradient in the first section w1. Likewise, the electronic apparatus <NUM> may determine that the hand <NUM> moves from the top to the bottom based on the proximity sensor value different in the pattern from the proximity sensor value of <FIG> and <FIG>.

<FIG> illustrates an example that the electronic apparatus in <FIG> recognizes a user motion based on a change in a sensor value of a control image. As described above with reference to <FIG>, the electronic apparatus <NUM> displays the control image <NUM> on the display <NUM> when entering the second mode in response to various events. In an example not covered by the claims, the control image <NUM> may have a predetermined brightness or color.

For example, the processor <NUM> may display a control image <NUM> with a predetermined color on the display <NUM>, and sense a change or a change pattern in a sensor value of a light reflected as the hand <NUM> moves from (a) to (b). The control image <NUM> may be displayed on any area of the display <NUM>, but it is preferable that the control image <NUM> is displayed on an area near to the optical sensor <NUM>.

A process, in which the electronic apparatus <NUM> determines the moving direction of the hand <NUM> based on the change or changing pattern in the proximity sensor value and the optical sensor value of the reflected light is equivalent to those described with reference to <FIG>, <FIG>, <FIG>, and <FIG>, and thus repetitive descriptions will be avoided. Like this, when the control image <NUM> having a predetermined color is displayed on the display <NUM>, it is easy to remotely control the external apparatus <NUM> without being affected by surrounding environments.

<FIG> illustrates an embodiment as an alternative to the control image of <FIG>. The electronic apparatus <NUM> in <FIG> displays the control image <NUM> having one color on a certain area of the display <NUM>, but the electronic apparatus <NUM> of <FIG> displays at least two images <NUM> and <NUM> different in color on a certain area of the display <NUM>.

<FIG> illustrates that a first control image <NUM> having a first color is displayed at the left of a second control image <NUM> having a second color along the widthwise direction of the electronic apparatus <NUM>. However, there are no limits to the display position or arranging order between the first control image <NUM> and the second control image <NUM>. For example, the first control image <NUM> and the second control image <NUM> may be displayed on an area adjacent to the optical sensor <NUM> along the arrangement direction of the proximity sensor <NUM> and the optical sensor <NUM>.

The optical sensor <NUM> may sense a first color value based on the first control image <NUM> and a second color value based on the second control image <NUM>. The optical sensor <NUM> may be provided as a single sensor capable of sensing two or more colors, or may be provided as two or more optical sensors for sensing different colors.

The graph of <FIG> shows relative changes in a first color value and a second color value as the hand <NUM> moves from the left toward the right of the electronic apparatus <NUM>, for example, from (a) to (b). Referring to the graph of <FIG>, the first color value gradually increases as the hand <NUM> moves from the left toward the right of the electronic apparatus <NUM>, reaches the maximum value Ma at the first point in time a1, maintains the maximum value Ma, and starts decreasing at the second point in time a2. Here, the maximum value Ma may refer to the first color value corresponding to the maximum intensity of the first color reflected from the hand <NUM>. The second color value also gradually increases, reaches the maximum value Mb at the third point in time b1, maintains the maximum value Mb and starts decreasing at the fourth point in time b2. Here, the maximum value Mb may refer to the second color value corresponding to the maximum intensity of the second color reflected from the hand <NUM>.

Due to difference in position between the first control image <NUM> and the second control image <NUM>, the first point in time a1 precedes the third point in time b1. That is, when the first point in time a1 precedes the third point in time b1, the electronic apparatus <NUM> may determine that the motion direction is oriented from the left toward the right of the electronic apparatus <NUM>. To more correctly determine the motion direction, the electronic apparatus <NUM> may also determine whether the second point in time a2 precedes the fourth point in time b2.

Meanwhile, a process, in which the electronic apparatus <NUM> determines the moving direction of the hand <NUM> based on the change or changing pattern in the first color value and the second color value, is equivalent to those described with reference to <FIG>, <FIG>, <FIG> and <FIG>, and therefore repetitive descriptions will be avoided.

<FIG> illustrates an example that the electronic apparatus in <FIG> recognizes a user motion in accordance with a change in the second sensor value of a sound sensor and an optical sensor. The electronic apparatus <NUM> of <FIG> employs a sound sensor <NUM> instead of the proximity sensor <NUM> used by the electronic apparatus <NUM> of <FIG>, <FIG>, <FIG>, and <FIG>. The sound sensor <NUM> senses whether a sound output through the sound output unit <NUM> is changed in strength by movement of the hand <NUM>.

Referring to the graph of <FIG>, a sound sensor value gradually decreases as the hand <NUM> moves from the left toward the right of the electronic apparatus <NUM>, for example from (a) to (b), reaches the minimum value Ma at the first point in time a1, maintains the minimum value Ma, and starts increasing at the second point in time a2. Here, the minimum value Ma refers to a sound sensor value corresponding to the maximum sound strength between the hand <NUM> and sound sensor <NUM>. The optical sensor value gradually increases, reaches the maximum value Mb at the third point in time b1 when the hand <NUM> covers the optical sensor <NUM>, maintains the maximum value Mb, and starts decreasing at the fourth point in time b2.

Due to difference in position between the sound sensor <NUM> and the optical sensor <NUM>, the first point in time a1 at which the sound sensor value reaches the minimum value Ma precedes the third point in time b1 at which the optical sensor value reaches the maximum value Mb, and the second point in time a2 at which the sound sensor value starts increasing from the minimum value Ma precedes the fourth point in time b2 at which the optical sensor value starts decreasing from the maximum value Mb. In this case, the electronic apparatus <NUM> may determine that the motion direction is oriented from the left toward the right of the electronic apparatus <NUM>. A process, in which the electronic apparatus <NUM> determines the moving direction of the hand <NUM> based on the change or changing pattern in the sound sensor value and the optical sensor value, is equivalent to those described with reference to <FIG>, <FIG>, <FIG>, and <FIG>, and thus repetitive descriptions thereof will be avoided.

The methods according to the foregoing exemplary embodiments may be achieved in the form of a program command that can be implemented in various computers, and recorded in a computer readable medium. Such a computer readable medium may include a program command, a data file, a data structure or the like, or combination thereof. For example, the computer readable medium may be stored in a volatile or nonvolatile storage such as a ROM or the like, regardless of whether it is deletable or rewritable, for example, a RAM, a memory chip, a device or integrated circuit (IC) like memory, or an optically or magnetically recordable or machine (e.g., a computer)-readable storage medium, for example, a compact disk (CD), a digital versatile disk (DVD), a magnetic disk, a magnetic tape or the like. It will be appreciated that a memory, which can be included in a mobile terminal, is an example of the machine-readable storage medium suitable for storing a program having instructions for realizing the exemplary embodiments. The program command recorded in this storage medium may be specially designed and configured according to the exemplary embodiments, or may be publicly known and available to those skilled in the art of computer software.

Accordingly, the electronic apparatus according to the disclosure recognizes the direction of the user motion for controlling the external apparatus without using a separate motion recognition sensor and performs a function of controlling the external apparatus, and it is more convenient for a user because the user can control the external apparatus while looking at the external apparatus.

Claim 1:
An electronic apparatus (<NUM>) comprising:
a communicator (<NUM>) configured to communicate with an external apparatus (<NUM>);
a plurality of sensors (<NUM>, <NUM>) arranged in a predetermined direction;
a display (<NUM>); and
a processor (<NUM>) configured to:
in a first mode, perform an operation of the electronic apparatus (<NUM>) based on a first sensor value sensed through the plurality of sensors;
in a second mode, based on an information about a content image (<NUM>) displayed by the external apparatus (<NUM>) being received from the external apparatus through the communicator (<NUM>):
identify the surrounding illumination and control the display (<NUM>) to display an image (<NUM>) with a brightness for indicating a position of the electronic apparatus (<NUM>), wherein the brightness of the image (<NUM>) is higher than an intensity of the surrounding illumination; or
identify the color of ambient light and control the display (<NUM>) to display an image (<NUM>) with a color for indicating a position of the electronic apparatus (<NUM>), wherein the color of the image (<NUM>) is distinguishable from the color of the ambient light,
identify, in the second mode, a direction of a user motion based on a change in second sensor values sensed through the plurality of sensors (<NUM>, <NUM>), the user motion occurring above and in proximity to the electronic apparatus (<NUM>) in the direction of arranging the plurality of sensors, and control the communicator (<NUM>) to output a control command to the external apparatus (<NUM>) based on the identified direction of the user motion.