Patent Description:
With the rapid development of functions provided in a smartphone and large distribution of smartphones, people have become highly dependent on smartphones. However, people who concentrate in an electronic apparatus such as a smartphone, a tablet personal computer (PC), etc., may not notice their surrounding environment and can be unexpectedly exposed to unexpected dangerous situation. In this regard, the term "smombie" has been generated to reflect this phenomenon.

In displaying a position of an external object, in the related art, icons corresponding to each of the electronic apparatus and external objects are merely displayed on a graphic map on which phases of cardinal points north, south, east, and west are displayed, and it is difficult for the user to intuitively grasp the direction of the external object on the basis of the gaze of the user, because the user needs to recognize the direction in which the user is looking in before determining the direction of the external object. In the related art, in order to identify a direction or a location of the external object through the audio signal output from the external object, it was necessary to use three or more microphones spaced apart by a relatively long distance from each other. For example, the <CIT> discloses an electronic device including a user interface in which a coordinate system with respect to a reference plane is shown having icons for acoustic sources. Further, the <CIT> discloses a system for localizing acoustic sources in which a screen is used for visualize the positions of sound sources. The <CIT> discloses an object display device for displaying an object in response to an input sound based on the direction in which the external object is located. The <CIT> teaches a mobile device display management which reduces the amount of battery consumption by efficient use of the display.

However, in general, a portable device such as a smartphone mostly includes two microphones, and due to a limitation of a size of a portable device, the distance between the microphones of the portable device is very small, and thus there is a limitation in using the reception difference of the audio signal using the distance of the microphone.

Provided is an electronic apparatus for informing a dangerous element (an external object) by intuitively displaying a direction of an external device based on a gaze of a user viewing a display in consideration of the direction of the display according to the posture of the electronic apparatus and the direction of the external object based on the electronic apparatus.

Also provided is an electronic apparatus that provides information on a direction of an external object through an audio signal output from the external object, while using two microphones that have a comparatively short distance between each other.

Also provided is an electronic apparatus, such as a smartphone, a tablet PC, etc., which performs an operation according to a user command to secure the safety of a so-called 'smombie' which is defenselessly exposed to an unexpected situation or a dangerous element by not recognizing a surrounding situation with eyes and ears, while concentrating on an electronic apparatus such as a smartphone, a tablet PC, etc., and informs a user of the direction and presence of an external object based on the sound of the upcoming external object.

In accordance with an aspect of the invention, there is provided an electronic apparatus according to claim <NUM>.

The processor is further configured to determine an area of the display that corresponds to the direction in which the external object is located, based on the posture of the electronic apparatus, and control the display to display the graphical object in the determined area of the display.

The processor is further configured to control the display to display the graphical object in a determined shape in the determined area of the display, based on the posture of the electronic apparatus and the direction in which the external object is located.

The processor is further configured to, based on the posture of the electronic apparatus being within a first predetermined range from a first posture in which the display faces up, control the display to display the graphical object corresponding to a top view of the external object, and based on the posture of the electronic apparatus being within a second predetermined range from a second posture that is perpendicular to a direction of the display from the first posture, control the display to display the graphical object corresponding to a side view of the external object.

The processor is further configured to, based on the posture of the electronic apparatus being within the first predetermined range from the first posture, control the display to display the graphical object corresponding to the top view at an angle corresponding to the determined area.

The processor may be further configured to, based on the posture of the electronic apparatus being within the second predetermined range from the second posture, control the display to display the graphical object corresponding to the side view at a viewpoint corresponding to the determined area.

The processor may be further configured to identify a speed at which the external object approaches the electronic apparatus by comparing the audio signal obtained by the audio receiver with a prestored audio signal, and provide a notification based on the identified speed.

The processor may be further configured to identify the external object based on the audio signal obtained by the audio receiver, and control the display to display the graphical object corresponding to the identified external object.

The audio receiver may include a first microphone configured to obtain an audio signal of the sound output by the external object; and a second microphone configured to obtain an audio signal of the sound output by the external object, and the processor may be further configured to determine the direction in which the external object is located based on a first audio signal obtained by the first microphone and a second audio signal obtained by the second microphone when the electronic apparatus is in a first position, and at least one of a third audio signal obtained by the first microphone and a fourth audio signal obtained by the second microphone when the electronic apparatus is in a second position.

In accordance with an aspect of the invention, there is provided a method of controlling an electronic apparatus according to claim <NUM>.

The displaying the graphical object includes determining a region in the display that corresponds to the direction in which the external object is located, based on the posture of the electronic apparatus, and displaying the graphical object in the determined area of the display.

The displaying the graphical object includes displaying the graphical object having a determined shape in the determined area of the display, based on the posture of the electronic apparatus and the direction in which the external object is located.

The displaying the graphical object includes, based on the posture of the electronic apparatus being within a first predetermined range from a first posture in which the display faces up, displaying the graphical object corresponding to a top view of the external object, and based on the posture of the electronic apparatus being within a second predetermined range from a second posture that is perpendicular to a direction of the display from the first posture, displaying the graphical object corresponding to a side view of the external object.

The displaying the graphical object may include, based on the posture of the electronic apparatus being within the first predetermined range from the first posture, displaying the graphical object corresponding to the top view at an angle corresponding to the determined area.

The displaying the graphical object may include, based on the posture of the electronic apparatus being within the second predetermined range from the second posture, displaying the graphical object corresponding to the side view at a viewpoint corresponding to the determined area.

The method may further include identifying a speed at which the external object approaches the electronic apparatus by comparing the audio signal obtained through the audio receiver with a prestored audio signal; and providing a notification for a risky situation based on the identified speed.

The method may further include identifying the external object based on the audio signal obtained through the audio receiver, and the displaying the graphical object may include displaying the graphical object corresponding to the identified external object.

The determining the direction in which the external object is located may include determining the direction in which the external object is located based a first audio signal of the sound output by the external object obtained through a first microphone and a second audio signal of the sound output by the external object obtained through a second microphone when the electronic apparatus is in a first position, and at least one of a third audio signal of the sound output by the external object obtained through the first microphone and a fourth audio signal of the sound output by the external object obtained through the second microphone when the electronic apparatus is in a second position.

In accordance with an aspect of the disclosure, there is provided a non-transitory computer readable medium having stored therein a computer instruction which is executed by a processor of an electronic apparatus to perform the method including: based on obtaining through an audio receiver of the electronic apparatus an audio signal of sound output from an external object, determining a direction in which the external object is located with respect to the electronic apparatus based on the audio signal; and displaying on a display of the electronic apparatus a graphical object that corresponds to the external object based on a posture of the electronic apparatus obtained through a sensor of the electronic apparatus and the direction in which the external object is located.

The electronic apparatus may notify a user with the presence and direction of the risk factor (external object) by intuitively displaying a direction of the external object based on the gaze of the user viewing the display.

The electronic apparatus may obtain information on the external object through an audio signal output from the external object while using only two microphones.

The terms used in the present specification and the claims are general terms identified in consideration of the functions of the various embodiments of the disclosure. However, these terms may vary depending on intention, legal or technical interpretation, emergence of new technologies, and the like of those skilled in the related art. Also, there may be some terms arbitrarily identified by an applicant. Unless there is a specific definition of a term, the term may be construed based on the overall contents and technological common sense of those skilled in the related art.

Further, like reference numerals indicate like components that perform substantially the same functions throughout the specification. For convenience of descriptions and understanding, the same reference numerals or symbols are used and described in different exemplary embodiments. In other words, although elements having the same reference numerals are all illustrated in a plurality of drawings, the plurality of drawings do not mean one exemplary embodiment.

The terms such as "first," "second," and so on may be used to describe a variety of elements, but the elements should not be limited by these terms. The terms are used only for the purpose of distinguishing one element from another. For example, the elements associated with the ordinal numbers should not be limited in order or order of use by the numbers. If necessary, the ordinal numbers may be replaced with each other.

A singular expression includes a plural expression, unless otherwise specified. It is to be understood that the terms such as "comprise," "include," or "consist of" are used herein to designate a presence of a characteristic, number, step, operation, element, component, or a combination thereof, and not to preclude a presence or a possibility of adding one or more of other characteristics, numbers, steps, operations, elements, components or a combination thereof.

The term such as "module," "unit," "part", and so on is used to refer to an element that performs at least one function or operation, and such element may be implemented as hardware or software, or a combination of hardware and software. Further, except for when each of a plurality of "modules", "units", "parts", and the like needs to be realized in an individual hardware, the components may be integrated in at least one module or chip and be realized in at least one processor.

Also, when any part is connected to another part, this includes a direct connection and an indirect connection through another medium. Further, when a certain portion includes a certain element, unless specified to the contrary, this means that another element may be additionally included, rather than precluding another element.

In an embodiment, at least one of configuration <NUM>, configuration <NUM> or configuration <NUM> may include "configuration <NUM> only," "configuration <NUM> only," "configuration <NUM> only," "configuration <NUM> and configuration <NUM>," "configuration <NUM> and configuration <NUM>," "configuration <NUM> and configuration <NUM>," and "configuration <NUM>, configuration <NUM>, and configuration <NUM>.

<FIG> is an example of an electronic apparatus according to an embodiment.

Referring to <FIG>, a user <NUM> who is wearing a headphone <NUM> and concentrating on an electronic apparatus <NUM>, that is a smartphone, without hearing ambient sound may not hear the sound of a motorcycle <NUM> which is approaching from a left direction with reference to the gaze of the user <NUM>.

At this time, the electronic apparatus <NUM> of <FIG> can identify the direction of the motorcycle <NUM> based on an audio signal such as the engine sound output from the motorcycle <NUM>. Additionally, a graphical object <NUM>' corresponding to the motorcycle <NUM> can be displayed in the left area of the display <NUM> in consideration of the direction of the display according to the posture or orientation of the electronic apparatus <NUM>. In this case, an alarm sound may be output through the headphone <NUM> connected by wire or wirelessly to the electronic apparatus <NUM>, the display <NUM> may be blinking in a red light, or a red warning line may be displayed in an edge portion of the display.

As a result, the user <NUM> may recognize that the motorcycle <NUM> is located at the left side or is approaching and prepare for danger.

<FIG> is a block diagram of the electronic apparatus <NUM> according to an embodiment.

The electronic apparatus <NUM> may be implemented as a portable electronic apparatus such as a mobile phone, a smartphone, a tablet PC, a notebook PC, or the like. The electronic apparatus <NUM> may be implemented as a driving aid device such as a navigation system provided inside a vehicle, or the like.

The electronic apparatus <NUM> may be implemented as a wearable device such as a smart watch or a smart glasses capable of displaying at least one image.

Referring to <FIG>, the electronic apparatus <NUM> includes an audio receiver <NUM>, a sensor <NUM>, a display <NUM>, and a processor <NUM>.

The audio receiver <NUM> is configured to receive an external audio signal and may be implemented as a circuitry including at least one microphone. In particular, the audio receiver <NUM> may include two microphones.

The sensor <NUM> is a configuration for the electronic apparatus <NUM> to identify a posture and a position of the electronic apparatus <NUM>. For this, the sensor <NUM> may be implemented as one or more sensors including a gyro sensor and an acceleration sensor. The sensor <NUM> may further include at least one of a global positioning system (GPS) sensor and a geomagnetic sensor.

The sensor <NUM> may include various types of sensors, and the electronic apparatus <NUM> may obtain information on an external environment or information on a user command through the sensor <NUM>. Various sensors will be described through <FIG> below.

The electronic apparatus <NUM> identifies a posture or orientation of the electronic apparatus <NUM> through one or more gyro sensor included in the sensor <NUM>. The posture of the electronic apparatus <NUM> may include information on a rotational angle of the electronic apparatus <NUM> with reference to at least one of roll, pitch, and yaw.

The display <NUM> is configured to display one or more images according to the control of the processor <NUM>. For this purpose, the display <NUM> may be implemented as a liquid crystal display (LCD), a plasma display panel (PDP), an organic light emitting diode (OLED), transparent OLED (TOLED), or the like. In the case of the LCD, the display <NUM> may include a driving circuit and a backlight unit, which may be implemented in the form of an a-si TFT, a low temperature poly silicon (LTPS) TFT, an organic TFT (OTFT), or the like.

The display <NUM> may be implemented as a flexible display or a foldable display. The display <NUM> may be implemented with a material having a flexible characteristic such as a plastic substrate (for example, polymer film) thin glass, or metal foil that can be deformed by an external pressure.

The display <NUM> may be implemented with a touch screen type including a touch panel capable of sensing a user's touch manipulation.

The processor <NUM> may control overall operations of the electronic apparatus <NUM>. The processor <NUM> may include random access memory (RAM), read only memory (ROM), central processing unit (CPU), graphic processing unit (GPU), and system bus, or the like, and may perform operation or data processing related to control of one or more elements included in the electronic apparatus <NUM>.

The processor <NUM> may execute one or more commands stored in a storage or memory for controlling one or more elements included in the electronic apparatus <NUM>, controlling one or more elements as a hardware circuitry or chip, or controlling one or more elements by the combination of software and hardware.

The processor <NUM> may be electrically connected to various elements in the electronic apparatus <NUM> including the audio receiver <NUM>, the sensor <NUM>, the display <NUM>, and a storage and may connect the elements.

The audio receiver <NUM> may receive sound that is output from an external object and converts the sound into an audio signal that is output to the processor <NUM>. Based on the audio signal output by the audio receiver <NUM>, the processor <NUM> may determine a direction in which an external object is located relative to the electronic apparatus <NUM>.

Specifically, the processor <NUM> may determine the direction in which the external object is located relative to the electronic apparatus <NUM> using the audio signals of the external object output by the first and second microphones included in the audio receiver <NUM> when the electronic apparatus is in the first position and the audio signal of the external object obtained through at least one of the first and second microphones when the electronic apparatus is in the second position.

Only when the volume of the audio signal output by each of the first microphone and the second microphone is greater than or equal to a predetermined value and the audio signals output by the first microphone and the second microphone match with each other by a predetermined ratio or more, the processor <NUM> may determine a direction in which the external object is located from the electronic apparatus <NUM> using the audio signals.

<FIG>, and <FIG> are views to describe an example of identifying a direction of an external object using a first microphone <NUM> and a second microphone <NUM> included in the audio receiver.

Referring to <FIG>, when a sound arrives at a first microphone <NUM> and a second microphone <NUM> from the external object <NUM> corresponding to the source when the electronic apparatus is in the first position, the processor <NUM> may obtain a value d (<NUM>) by using the time difference and the sound velocity of the audio signal obtained by the first microphone <NUM> and the audio signal obtained by the second microphone <NUM>, respectively.

Through the equation below, the angle Θ(<NUM>) can be calculated, where r is the distance between the first microphone <NUM> and the second microphone <NUM>.

Referring to <FIG>, under the premise that the external object is on the surface of the ground, an area where the external object exists may be one of two direction ranges <NUM>-<NUM> and <NUM>-<NUM> with reference to the electronic apparatus <NUM>.

Referring to <FIG>, as the electronic apparatus <NUM> moves and/or the posture of the electronic apparatus <NUM> is changed, the processor <NUM> may identify a posture and a position of the electronic apparatus <NUM> after the movement using at least one of a gyro sensor, an acceleration sensor, a GPS sensor, and a geomagnetic sensor included in the sensor <NUM>. The processor <NUM> can obtain information about the position of the first microphone <NUM> and the second microphone <NUM> which moved (from <NUM>-<NUM> to <NUM>-<NUM>, and from <NUM>-<NUM> to <NUM>-<NUM>) by identifying the position and posture after the movement.

The processor <NUM> may identify a direction in which the external object is located with reference to the electronic apparatus <NUM> before movement, between direction range <NUM>-<NUM> and direction range <NUM>-<NUM>, using the result of comparing the audio signal of the external object output by at least one of the first microphone <NUM> at the position <NUM>-<NUM> and the second microphone <NUM> at the position <NUM>-<NUM> with the audio signal output by the first microphone <NUM> at the position <NUM>-<NUM> and the audio signal output by the second microphone <NUM> at the position <NUM>-<NUM>.

The processor <NUM> can determine the direction in which the external object is located with reference to the electronic apparatus <NUM> after the movement, by applying the direction in which the external object is located with reference to the electronic apparatus <NUM> before the movement to the posture and position of the electronic apparatus <NUM> after the movement. At this time, the time interval before and after the movement (the time interval at which the audio signal is received) can be very short of less than or equal to the millisecond unit, and therefore, the position change of the electronic apparatus <NUM> before and after the movement may not be almost considered.

The audio signals of the sound output from the external object, that is, the audio signals of the sound obtained through the first microphone <NUM> and the second microphone <NUM>, may be audio signals having a certain period in amplitude or other characteristics. In general, various sounds, such as a warning sound, an engine operation sound, a klaxon sound, or the like, have a periodicity at least for a short time. Using this periodicity, the processor <NUM> may identify the time of arrival difference between the sounds received via each microphone before and after the movement.

With reference to <FIG>, and <FIG>, it has been described that the audio signal of the sound obtained through the first microphone <NUM>, at position <NUM>-<NUM> before movement, the audio signal of the sound obtained through the second microphone <NUM> at position <NUM>-<NUM> before movement, and the audio signal of the sound obtained through at least one of the first microphone <NUM> at position <NUM>-<NUM> and the second microphone <NUM> at position <NUM>-<NUM> after movement, but it is possible to use the audio signal of the sound obtained through the first microphone <NUM> at position310-<NUM> , the audio signal of the sound obtained through the second microphone <NUM> at position <NUM>-<NUM> after movement, and the audio signal of the sound obtained through at least one of the first microphone <NUM> at position <NUM>-<NUM> and the second microphone <NUM> at position <NUM>-<NUM> before movement.

It is possible to identify a positon of the external object by applying an audio signal (with periodicity) of the sound obtained through at least three microphones among the first microphone <NUM> at position <NUM>-<NUM> before movement, the second microphone <NUM> at position <NUM>-<NUM> before movement, the first microphone <NUM> at position <NUM>-<NUM> after movement, and the second microphone <NUM> at position <NUM>-<NUM> after movement to the conventional Triangulation.

The processor <NUM> controls the display to display a graphical object indicating an external object based on a direction where the external object is located with reference to the posture of the electronic apparatus <NUM> obtained through the sensor <NUM> and the electronic apparatus <NUM>.

The processor <NUM> determines an area corresponding to a direction where the external object is positioned in the display <NUM> with reference to the posture of the electronic apparatus <NUM>, and control the display <NUM> to display a graphical object in the determined area.

Specifically, the processor <NUM> may identifies a direction of the display <NUM> using the posture of the electronic apparatus <NUM>, compare the identified direction with a direction in which the external object is located, and determine a corresponding area.

The processor <NUM> may identify an external object based on an audio signal of the sound of the external object obtained via the audio receiver <NUM>, and control the display to display a graphical object corresponding to the identified external object. For example, if the external object is an automobile, the graphical object is an image for the vehicle, and if the external object is a motorcycle, the graphical object may be an image for the motorcycle.

For this purpose, the processor <NUM> may compare the audio signal of the received sound to a pre-stored plurality of audio signals. The plurality of audio signals may be matched to various sound sources. When the audio signal is input, the processor <NUM> may use a network-learned artificial intelligence (Al) model to determine a type of the sound source of the input audio signal. In this case, the Al model may be stored in a storage of the electronic apparatus <NUM>, or may be stored in an external device capable of receiving or transmitting data by wire or wirelessly.

In addition to determining an area of the display <NUM> in which the graphical object is displayed, a shape of the graphical object is determined according to the posture of the electronic apparatus <NUM> and a direction in which the external object is located.

That is, the processor <NUM> controls the display <NUM> to display a graphical object of a shape determined based on the posture of the electronic apparatus <NUM> and the direction in which the external object is located.

Specifically, when the posture of the electronic apparatus <NUM> is within a predetermined range from a first posture in which the display <NUM> faces up, the processor <NUM> controls the display <NUM> to display a graphical object corresponding to a top view of the external object. Here, the predetermined range may mean that the posture of the electronic apparatus <NUM> is rotated within a predetermined angle range from the first posture based on the pitch, the roll, and the yaw axis.

The first posture may include a case where the display <NUM> faces down as well as a case where the display <NUM> faces up.

When the posture of the electronic apparatus <NUM> is within a predetermined range from a second posture perpendicular to the direction in which the display <NUM> faces in the first posture described above, the processor <NUM> controls the display <NUM> to display a graphical object corresponding to a side view of the external object. In other words, the second posture of the electronic apparatus <NUM> may be within a predetermined range from a posture in which the display <NUM> is facing a direction parallel to the ground surface. Here, the predetermined range may mean that the electronic apparatus <NUM> is rotated within a predetermined angle range from the first posture with reference to the pitch, the roll, and the yaw axis.

Here, the term side view means that all views seen from a direction of <NUM> degrees including front, back, left, and right directions of the specific object are included.

<FIG> is a view to describe an example of distinguishing the posture of the electronic apparatus <NUM> with respect to the graphical object.

Referring to <FIG>, an electronic apparatus <NUM> is in a first posture with a display <NUM> facing up (+z direction). Accordingly, since the electronic apparatus <NUM> corresponds to the first posture, it is more likely that the user's gaze direction looking at the display <NUM> is in the downward direction (-z direction). When the electronic apparatus <NUM> is in the first posture, it is natural to a user that the graphical object displayed on the display <NUM> of the electronic apparatus <NUM> corresponds to the top view of the external object, that is, the user viewing the external object.

Additionally, when an electronic apparatus <NUM>, which has been rotated by a predetermined angle <NUM> from the first posture but within a predetermined range from the first posture, it is also more likely that the user's gaze direction is in the downward direction (-z direction), since the display <NUM> is still facing the relatively upward direction. Thus, the graphical object corresponding to the top view of the external object may be displayed on the display <NUM>.

Referring to <FIG>, an electronic apparatus <NUM> is in the second posture where the display is perpendicular to the display direction, that is, when compared to the direction of the display when the display is in -y direction (the first posture). In this case, since it is highly likely that the gaze direction of the user looking at the display is to be the +y direction <NUM>, the graphical object corresponding to the side view of the external object can be displayed on the display of the electronic apparatus <NUM>.

Further, in an electronic apparatus <NUM> rotated by a predetermined angle <NUM> from the second posture but within a predetermined range from the second posture, the display is facing a direction comparatively similar to +y direction, it is likely that the user's gaze direction looking at the display is in the +y direction <NUM>. Therefore, a graphical object corresponding to the side view of the external object may be displayed on the display of the electronic apparatus <NUM>.

When the posture of the electronic apparatus <NUM> is within a predetermined range from the second posture, the processor <NUM> controls the display to display a graphical object corresponding to the side view at a viewpoint corresponding to the determined area. At this time, the processor <NUM> can control the display to display a graphical object corresponding to the orientation of the display <NUM> in the area determined in the display <NUM> or the orientation of the external object moving in a direction toward the display <NUM>.

<FIG>, <FIG> are views to describe embodiments to display various graphic objects corresponding to the side view of the external object according to the instruction of the electronic apparatus.

Referring to <FIG>, while the user <NUM> is holding the electronic apparatus <NUM> in the second posture or within a predetermined range from the second posture, a first case where an external object <NUM> is approaching from or is located in the opposite direction of the direction in which the display <NUM> of the electronic apparatus <NUM> is facing (front side with reference to the user's gaze), a second case where an external object <NUM> is approaching from or is located in a right direction (left side with reference to the user's gaze) of the direction which the display <NUM> faces, and a third case where an external object <NUM> is approaching from or is located in the left direction (right side with reference to the user's gaze) in the direction toward the display <NUM>, are respectively shown.

The first case of the external object <NUM> corresponds to <FIG>, the second case of the external object <NUM> corresponds to <FIG>, and the third case of the external object <NUM> corresponds to <FIG>.

Referring to <FIG>, the processor <NUM> may control the display <NUM> to display a graphical object <NUM>' corresponding to a front view among side views of the external object <NUM> on a middle area <NUM> of the display <NUM>, based on the posture of the electronic apparatus <NUM> being within a preset range from the second posture and the direction in which the external object <NUM> is located being the rear direction of the electronic apparatus <NUM>. Specifically, the processor <NUM> may control the display <NUM> to display a graphical object <NUM>' corresponding to the front image of the type of the external object <NUM> such as a car or a motorcycle.

Referring to <FIG>, the processor <NUM> may control the display <NUM> to display a graphical object <NUM>' corresponding to a view of the external object <NUM> from the right side, among the side views of the external object <NUM> , on a left area <NUM> of the display <NUM>, based on the posture of the electronic apparatus <NUM> being within a predetermined range from the second posture, and the direction in which the external object <NUM> is positioned being a right direction from the direction in which the display <NUM> is oriented. To be specific, the processor <NUM> may control the display <NUM> to display a graphical object <NUM>' corresponding to the right side image of the type of the external object <NUM> such as a car or a motorcycle.

As shown in <FIG>, the graphical object <NUM>' may not correspond to an entire view of the external object <NUM> from the right side, but may be a portion thereof. The graphical object <NUM>' may be a front portion of the type of the external object <NUM>.

Referring to <FIG>, the processor <NUM> controls the display <NUM> to display a graphical object <NUM>' corresponding to the view of the external object <NUM> from the left side among the side views on the right area <NUM> of the display <NUM>, based on the posture of the electronic apparatus <NUM> being within a predetermined range from the second posture and the direction in which the external object <NUM> is located being the left direction from the direction toward the display <NUM> of the electronic apparatus <NUM>. Specifically, the processor <NUM> controls the display <NUM> to display a graphical object <NUM>' corresponding to a left side image of the type of the external object <NUM> such as an automobile, a motorcycle, or the like.

As shown in <FIG>, the graphical object <NUM>' may not be an entire view of the external object <NUM> from the left side, but be a portion thereof. In particular, the graphical object <NUM>' may correspond to a front portion of the object corresponding to the type of the external object <NUM>.

Further, when the electronic apparatus <NUM> is within a predetermined range from the second posture, if the external object is located in a direction toward the display <NUM> of the electronic apparatus <NUM> (in the opposite direction of <NUM> with respect to the user <NUM>), a graphical object <NUM>' that is the same as <FIG> is displayed and a mark "rear direction" may be separately displayed on the display <NUM>, or visual / audible notification that an external object is approaching in a rear direction may be provided through the electronic apparatus <NUM>.

The processor <NUM>, when the posture of the electronic apparatus <NUM> is within a predetermined range from the first posture, may control the display to display the graphical object corresponding to the top view at an angle corresponding to the determined area. Specifically, the processor <NUM> may control the display <NUM> to display a graphical object corresponding to a shape that faces toward the center of the display <NUM> in a determined area of the display <NUM>.

<FIG>, <FIG>, and <FIG> are views to describe examples of displaying various graphical objects corresponding to a top view of an external object according to a posture of an electronic apparatus.

Referring to <FIG>, while the user <NUM> is holding the electronic apparatus <NUM> in a first posture or a posture within a predetermined range from the first posture, respectively, a first case where the external object <NUM> that is a vehicle is approaching from or is located in an upper direction (i.e., a front side with reference to the user) of the electronic apparatus <NUM>, a second case where the external object <NUM> is approaching from or is located in a left direction of the electronic apparatus <NUM> (i.e., a left side with reference to the user), a third case where the external object <NUM> is approaching from or is located in the right direction (i.e., a right side with reference to the user), and a fourth case where the external object <NUM> is approaching from or is located in a lower direction (i.e., a rear side with reference to the user) of the electronic apparatus <NUM>, are respectively shown.

The first case of the external object <NUM> corresponds to <FIG>, the second case of the external object <NUM> corresponds to <FIG>, the third case of the external object <NUM> corresponds to <FIG>, and the fourth case of the external object <NUM> corresponds to <FIG>.

Referring to <FIG>, the processor <NUM> may control the display <NUM> to display the graphical object <NUM>' corresponding to the top view of the object in an upper area <NUM> of the display <NUM> based on the posture of the electronic apparatus <NUM> being within a predetermined range from the first posture and the direction in which the external object is located being the upward direction of the electronic apparatus <NUM>. In this case, the graphical object <NUM>' corresponds to a shape of the display <NUM> facing toward the center of the display <NUM> in the upper area <NUM> of the display <NUM>.

Referring to <FIG>, the processor <NUM> controls the display <NUM> to display the graphical object <NUM>' corresponding to the top view of the object in a left area <NUM> of the display <NUM> based on the posture of the electronic apparatus <NUM> being within a predetermined range from the first posture and the direction in which the external object is located being the left direction of the electronic apparatus <NUM>. In this case, the graphical object <NUM>' may correspond to a shape of the display <NUM> facing toward the center of the display <NUM> in the left area <NUM> of the display <NUM>.

Referring to <FIG>, the processor <NUM>, based on the posture of the electronic apparatus <NUM> being within a preset range from the first posture and the direction in which the external object is located being the right direction of the electronic apparatus <NUM>, controls the display <NUM> to display a graphical object <NUM>' corresponding to the top view of the object in the right area <NUM> of the display <NUM>-<NUM>. In this case, the graphical object <NUM>" may correspond to a shape that is facing toward the center of the display <NUM> in the right area <NUM> of the display <NUM>.

Referring to <FIG>, the processor <NUM> controls the display <NUM> to display a graphical object <NUM>' corresponding to the top view of the object in a lower area <NUM> of the display <NUM> based on the posture of the electronic apparatus <NUM> being within a predetermined range from the first posture and the direction in which the external object is located being the downward direction of the electronic apparatus <NUM>. In this case, the graphical object <NUM>' may correspond to a shape of the display <NUM> facing toward the center of the display <NUM> in the lower area <NUM> of the display <NUM>.

Referring to <FIG>, each of the graphical objects <NUM>', <NUM>", <NUM>", and <NUM>" does not correspond to the top view of the type of the object, and may correspond to a portion of the external object. For example, the graphical object may correspond a front portion of a type of an external object, but it is not limited thereto.

The processor <NUM> may determine whether the external object is approaching the electronic apparatus <NUM> using an audio signal of the sound obtained through the audio receiver <NUM>.

When the volume of the audio signal of the sound obtained through the audio receiver <NUM> increases, the processor <NUM> may identify that the external object that outputs an audio signal is approaching the electronic apparatus <NUM>.

The processor <NUM> may compare the audio signal of the sound of the external object obtained through the audio receiver <NUM> with a prestored audio signal and identify the speed at which the external object approaches the electronic apparatus <NUM>, to determine whether the external object is approaching the electronic apparatus <NUM>.

The processor <NUM> may identify a type of an external object such as a vehicle, a motorcycle, or the like, using an audio signal of the sound of the external vehicle obtained through the audio receiver <NUM>, compare a plurality of audio signals corresponding to the identified type with the audio signal, and determine the speed at which the external object is approaching the electronic apparatus <NUM>.

The speed at which the external object is approaching the electronic apparatus <NUM> may be determined based on the fact that, the closer or farther the external object with respect to the electronic apparatus <NUM>, the audio signal received by the electronic apparatus <NUM> changes according to a Doppler effect.

For example, the processor <NUM> may identify that the external object is a vehicle based on a result of comparing the received audio signal with a plurality of pre-stored audio signals. And, the processor <NUM> may compare the received audio signal with an audio signal corresponding to a case in which a vehicle approaches at a speed of <NUM>/s among a plurality of pre-stored audio signals, an audio signal corresponding to a case in which a vehicle approaches at a speed of <NUM>/s, a case in which a vehicle approaches at a speed of <NUM>/s, and a case in which a vehicle moves away at a speed of <NUM>/s, etc. As a result, the processor <NUM> may determine a speed at which the external object, which is a vehicle, is approaching or moving away from the electronic apparatus <NUM>. In this case, when an audio signal is input, a trained network model may be used to determine a speed at which an object outputting the input audio signal is approaching.

The processor <NUM> may identify whether the external object is approaching or moving away from the electronic apparatus <NUM>, and display a graphical object in a different manner considering the above.

<FIG> are views to describe embodiments of displaying a graphic object differently as an external object approaches an electronic apparatus.

<FIG> and <FIG> correspond to the case of the external object <NUM> of <FIG> approaching (getting closer to) and the electronic apparatus <NUM>.

In the case of <FIG>, the processor <NUM> displays a graphical object <NUM> that is not relatively large in a determined area <NUM>, but as the external object <NUM> becomes closer to the electronic apparatus <NUM>, a larger graphical object <NUM> is displayed as <FIG>. In this case, if the electronic apparatus <NUM> and the external object <NUM> are even closer in the situation of <FIG>, the processor <NUM> may increase the size of the displayed graphical object and, if the distance gets farther, may decrease the size of the graphical object or may not display the graphical object any more.

<FIG> and <FIG> correspond to the case of the external object <NUM> of <FIG> approaching (getting closer to) the electronic apparatus <NUM>.

In the case of <FIG>, the processor <NUM> displays the graphical object <NUM> corresponding to a portion that is less than half of the side view of the type of the external object in the determined area <NUM>, but if it is determined that as the external object and the electronic apparatus <NUM> are closer, a graphical object <NUM> corresponding to a larger portion of the side view is displayed as shown in <FIG>. In this case, if the electronic apparatus <NUM> and the external object becomes closer in the situation of <FIG>, the processor <NUM> may display a graphical object that corresponds to a larger portion of the side view, and if the distance between the electronic apparatus <NUM> and the external object gets farther, the processor <NUM> may display a graphical object that corresponds to a smaller portion of the side view or may not display a graphical object. Unlike <FIG> and <FIG>, the graphical object <NUM> may be an image for a larger vehicle than the graphical object <NUM>.

<FIG> and <FIG> correspond to the case of the external objection <NUM> of <FIG> approaching (getting closer to) the electronic apparatus <NUM>.

Referring to <FIG>, as the external object gets closer to the electronic apparatus <NUM> than the point of view of <FIG>, the processor <NUM> may display a graphical object <NUM> increase the portion of the top view of the type of the external object as compared to the graphical object <NUM> of <FIG>. In this case, if the electronic apparatus <NUM> and the external object are closer together in the situation of <FIG>, the processor <NUM> may display a graphical object <NUM> corresponding to a larger portion of the top view, and the graphical object may further move to a middle area of the display <NUM>. Conversely, if the distance between the electronic apparatus <NUM> and the external object becomes farther, the processor <NUM> may display a graphical object that includes a smaller portion of the top view and the graphical object moves away from the central area of the display <NUM>, or the graphical object may not be displayed any more. The graphical object <NUM> of <FIG> may be an image for a larger vehicle than the graphical object <NUM> of <FIG>.

In <FIG> described above, it has been described that even if the electronic apparatus <NUM> is closer to the external object, a determined area <NUM>, <NUM>, and <NUM> is constant, but unlike <FIG>, the area where the graphical object is displayed may change like the graphical object. The closer or farther the electronic apparatus is with respect the external object, the shape and/or size of the determined area may change as well. For example, the size and/shape may change to include all the displayed graphical object.

The processor <NUM> may identify the speed at which the external object approaches the electronic apparatus <NUM>, and then provide a notification for a dangerous situation based on the speed. That is, the faster the speed, the stronger the notification. For example, if the speed at which the external object approaches the electronic apparatus <NUM> is greater than or equal to a predetermined first value or less than a second value, the processor <NUM> may control the electronic apparatus <NUM> to display a warning light to the display <NUM>, and when the speed is greater than or equal to the second value, the processor <NUM> may control the electronic apparatus <NUM> to audibly provide a warning sound and display the warning light. As the speed increases, the warning light may be stronger or the warning sound may be louder.

The processor <NUM> may modify the color or shape of the graphical object according to the speed at which the external object is approaching the electronic apparatus <NUM>. For example, if the speed at which the external object approaches the electronic apparatus <NUM> is greater than or equal to a predetermined first value, the processor <NUM> may display a modified graphical object in which the edge or some portions of the graphical object is modified as a red color, or the like, and if the speed is greater than or equal to a second value greater than the first value, the graphical element of the warning light shape may be further displayed, in addition to the modified graphical object.

When the speed at which the external object approaches the electronic apparatus <NUM> is less than a preset speed, the processor <NUM> may not display the graphical object, and only in a case where the speed is greater than or equal to a predetermined speed, the processor <NUM> may display a graphical object in the determined area.

<FIG> is a block diagram of an electronic apparatus <NUM> according to various embodiments.

Referring to <FIG>, the electronic apparatus <NUM> may further include at least one of an audio outputter <NUM>, a storage <NUM>, a communicator <NUM>, and an input and output port <NUM>, in addition to the audio receiver <NUM>, the sensor <NUM>, the display <NUM>, and the processor <NUM>.

Referring to <FIG>, the audio receiver <NUM> may include a first microphone <NUM>-<NUM> and a second microphone <NUM>-<NUM>. As described above, the processor <NUM> may identify a direction of the external object that outputs the audio signal, with reference to the electronic apparatus <NUM> using the audio signal received through the first microphone <NUM>-<NUM> and the second microphone <NUM>-<NUM>.

The sensor <NUM> may include various sensors.

Referring to <FIG>, the sensor <NUM> may include a gyro sensor <NUM>-<NUM>, a GPS sensor <NUM>-<NUM>, an acceleration sensor <NUM>-<NUM>, a camera sensor <NUM>-<NUM>, a touch sensor <NUM>-<NUM>, a tension sensor <NUM>-<NUM>, a proximity sensor <NUM>-<NUM>, a geomagnetic sensor <NUM>-<NUM>, or the like. However, the configuration of the sensor <NUM> is not limited to a case where the sensors shown in <FIG> are included all or only the sensors shown in <FIG> are included.

The gyro sensor <NUM>-<NUM> is configured to measure angular velocity or an angle with which the electronic apparatus <NUM> rotates with respect to at least one axis among roll, pitch, and yaw. The sensor <NUM> may include one or more gyro sensors <NUM>-<NUM>.

The GPS sensor is configured to receive a signal sent from a satellite, and the processor <NUM> may obtain a position of the electronic apparatus <NUM> based on the signal received through the GPS sensor.

The acceleration sensor <NUM>-<NUM> is configured to obtain an acceleration at which the electronic apparatus <NUM> moves, and the processor <NUM> may obtain the information on the acceleration in at least one axis direction through the acceleration sensor <NUM>-<NUM>.

The processor <NUM> may obtain information on a moving direction and position of the electronic apparatus <NUM> using the gyro sensor <NUM>-<NUM> and the acceleration sensor <NUM>-<NUM>.

The camera sensor <NUM>-<NUM> is configured to obtain at least one image and may be implemented as a digital camera inside the electronic apparatus <NUM>.

The processor <NUM> may use the image for the user obtained through the camera sensor <NUM>-<NUM> and determine whether the user is viewing the display of the electronic apparatus <NUM>. If it is determined that the user views the display of the electronic apparatus <NUM>, a graphical object for the external object may be displayed on the display <NUM>.

The touch sensor <NUM>-<NUM> is configured to receive a user touch for at least one area of the electronic apparatus <NUM>. The processor <NUM> may obtain a user command or other information using a user touch that is input through the touch sensor <NUM>-<NUM>.

The touch sensor <NUM>-<NUM> may be formed on the display <NUM> and may be implemented as a touch panel for sensing a touch input of a user for the display <NUM>.

A tension sensor <NUM>-<NUM> is configured to obtain information about bending or curbing of at least a portion of the electronic apparatus <NUM>. If the electronic apparatus <NUM> is a flexible device or a foldable device, the electronic apparatus <NUM> may include the tension sensor <NUM>-<NUM>. When the shape of the electronic apparatus <NUM> is modified, the processor <NUM> may obtain information on the degree of deformation through the tension sensor <NUM>-<NUM>, and identify information about the location of each of the first microphone <NUM>-<NUM> and the second microphone <NUM>-<NUM> using the obtained information. The processor <NUM> may identify information about the distance between the first microphone <NUM>-<NUM> and the second microphone <NUM>-<NUM> using the obtained information.

When the shape of the electronic apparatus <NUM> is modified, the value "d" in <FIG> may be different and thus, in determining by the processor <NUM> a direction in which the external object is positioned using an audio signal received through the audio receiver <NUM>, information obtained through the tension sensor <NUM> may be necessary.

A proximity sensor <NUM>-<NUM> is a sensor for detecting at least one object approaching the electronic apparatus <NUM> and a motion of the corresponding object. The proximity sensor <NUM>-<NUM> may be implemented as various types of sensors such as a high frequency oscillation type to form high frequency magnetic field and detect a current induced by a magnetic field characteristic that changes when an object is approached, a magnetic type to use a magnet, and an electrostatic capacitance type that senses the electrostatic capacity that changes due to the approach of an object.

The processor <NUM> may use the proximity sensor <NUM>-<NUM> to identify whether the user is in the vicinity of the electronic apparatus <NUM> or if the user is viewing at the display <NUM> of the electronic apparatus <NUM>. The processor <NUM> may display the graphical object for the external object on the display <NUM> only when the user is in the vicinity of the electronic apparatus <NUM> or when the user is viewing the display <NUM> of the electronic apparatus <NUM>.

The geomagnetic sensor <NUM>-<NUM> is a sensor for sensing the electronic apparatus <NUM> based on the magnetic direction of the Earth. The processor <NUM> may obtain information on the posture of the electronic apparatus <NUM> using the geomagnetic sensor <NUM>-<NUM>.

The processor <NUM> may use at least one of the geomagnetic sensor <NUM>-<NUM> and the GPS sensor <NUM>-<NUM> in setting a criterion of at least one angular velocity measured by the gyro sensor <NUM>-<NUM> or at least one acceleration that is measured by the acceleration sensor <NUM>-<NUM>.

The audio outputter <NUM> is configured to output a specific sound according to control of the processor <NUM>. The audio outputter <NUM> may be implemented as a speaker and/or a headphone and an earphone output terminal.

The processor <NUM> may display the graphical object and simultaneously provide a warning sound or guide sound in an audible manner to notify the presence of the external object corresponding to the graphical object or approach of the external object using the audio outputter <NUM>.

The storage <NUM> is configured to store a command or data related to the elements of the electronic apparatus <NUM> and the operating system (OS) for controlling overall operations of the elements of the electronic apparatus <NUM>.

The storage <NUM> may be implemented as a non-volatile memory (e.g., a hard disc, a solid state drive (SSD), or a flash memory), a volatile memory, or the like.

In the storage <NUM>, audio signals corresponding to sounds output by various types objects may be stored.

In addition, a plurality of images corresponding to the top view, the side view, or the like, viewed at various angles may be stored in the storage <NUM>. The processor <NUM> may display, according to the identified external object, a graphical object corresponding to at least one portion or all of the stored plurality of images on the display <NUM>.

In the storage <NUM>, information for an application to determine a direction in which the external object is positioned and perform operations such as displaying a graphical object, by using the received audio signal, may be stored.

The processor <NUM> may identify a degree of risk using the identified type of external object and the speed at which the external object is approaching, or the like. If the identified risk exceeds a predetermined degree, at least one of the information on the time zone in which the audio signal of the corresponding external object is received and information about a place where the electronic apparatus <NUM> is in the time zone can be stored in the storage <NUM>.

When the electronic apparatus <NUM> is located at a corresponding place and/or in the corresponding time zone, the processor <NUM> may set sensitivity of at least one of the first microphone <NUM>-<NUM> and the second microphone <NUM>-<NUM> included in the audio receiver <NUM> to be higher than usual.

When the electronic apparatus <NUM> is located at the corresponding place and/or in the corresponding time zone, the processor <NUM> may identify the external object using the audio signals of the sound received through the first microphone <NUM>-<NUM> and the second microphone <NUM> and determine a direction in which the external object is located.

For example, in a state in which the electronic apparatus <NUM> is in the first place, if there is a history that an external object corresponding to a vehicle or a motorcycle approached the electronic apparatus <NUM> at a speed of <NUM>/s or more, the processor <NUM> may determine that the risk of the corresponding situation exceeds a predetermined degree and store at least one of the information about the "<NUM> pm" and "the first place" in the storage <NUM>.

When the electronic apparatus <NUM> is located within a predetermined distance from "the first place" at a predetermined range of time from "<NUM> pm," the processor <NUM> may set the sensitivity of the first microphone <NUM>-<NUM> and the second microphone <NUM>-<NUM> to be higher than usual.

Alternatively, the processor <NUM> may identify the external object using an audio signal of sound received through the audio receiver <NUM> and may perform operations for determining a direction in which the external object is located only when the electronic apparatus <NUM> is within a predetermined distance from "the first place" in a predetermined range of time zone from "<NUM> pm. " To be specific, the processor <NUM> may execute an application for performing the operations only when the electronic/ apparatus <NUM> is located within a predetermined distance from "the first place" at a time zone in a predetermined range from "<NUM> pm.

The communicator <NUM> is configured to perform data communication with an external device by wire or wirelessly. The processor <NUM> may perform communication with various external devices using the communicator <NUM>.

When performing data communication with an external device in a wireless communication method, the communicator <NUM> may include at least one of a WiFi direct communication module, a Bluetooth module, an infrared data association (IrDA module, a near field communication (NFC) module, a Zigbee module, a cellular communication module, a 3rd generation (<NUM>) mobile communication module, a fourth generation (4D) mobile communication module, a fourth Long Term Evolution (LTE) communication module.

When performing data communication with an external device by wired communication method, the communicator <NUM> may be connected to a coaxial cable, an optical cable, or the like, and transceive various data.

Through the input and output port <NUM>, the electronic apparatus <NUM> may receive a signal / data for an image from the outside or transmit date / signal for an image to the outside.

The input and output port <NUM> may be implemented as a wired port such as a high-definition multimedia interface (HDMI) port, a display port, a red-green-blue (RGB) port, a digital visual interface (DVI) port, Thunderbolt and component ports.

The input and output port <NUM> may be implemented with the HDMI port or the Thunderbolt, or the like, and may be implemented to transmit an image and a voice signal, but the first port for transmitting an image signal and the second port for transmitting a voice signal may be separately implemented.

The input and output port <NUM> may include an interface module, such as a USB, and may be physically connected to an external terminal device such as a PC through the interface module, and transceive voice or image data or transceive firmware data for performing firmware upgrade.

In the above embodiments, the audio receiver <NUM>, the sensor <NUM>, the display <NUM>, the processor <NUM>, or the like, are all included in one electronic apparatus <NUM>, but the above elements may be included in two or more electronic apparatuses capable of wireless and wired communication.

<FIG> is a view to describe an example of providing a direction of an external object using a plurality of electronic apparatuses.

Referring to <FIG>, the audio receiver including the first microphone <NUM>-<NUM>' and the second microphone <NUM>-<NUM>" is included in a headset <NUM>-<NUM>, and the display <NUM> is included in the mobile phone <NUM>-<NUM>. <FIG> illustrates a situation in which the headset <NUM>-<NUM> and the mobile phone <NUM>-<NUM> are connected to each other through a wired or wireless connection.

The distance between the first microphone <NUM>-<NUM>' and the second microphone <NUM>-<NUM>' may be different according to a wearing state of the headset <NUM>-<NUM> or a user wearing the headset <NUM>-<NUM> and thus, the headset <NUM>-<NUM> may further include a tension sensor for identifying the distance between microphones.

The mobile phone <NUM>-<NUM> may receive information on the audio signal of the sound received through the microphones <NUM>-<NUM>' and <NUM>-<NUM>' included in the headset <NUM>-<NUM>, determine the type of the object and the direction in which the object is located, and then display the graphical object corresponding to the object on the display <NUM>".

Hereinafter, a controlling method of the electronic apparatus will be described.

<FIG> is a flowchart of a controlling method of an electronic apparatus according to an embodiment.

Referring to <FIG>, the controlling method includes, based on an audio signal output from the external object being received from the audio receiver, determining a direction at which the external object is located from the electronic apparatus based on the received audio signal in operation S1210.

A direction in which the external object is located may be determined by using an audio signals of the sound of the external object received through the first and second microphones included in the audio receiver and the audio signal(s) of the sound of the external object received through at least one of the first and second microphones when the electronic apparatus is in the second position.

The graphical object representing the external object is displayed based on the posture of the electronic apparatus obtained through the sensor and the direction in which the external object is located in operation S1220. At this time, the external object can be identified based on an audio signal of an external object received through the audio receiver, and a graphical object corresponding to the identified external object can be displayed. For example, if the received audio signal is identified as being an engine sound of a "vehicle", a graphical object corresponding to the "vehicle" can be displayed.

The area corresponding to the direction in which the external object is located is determined on the basis of the posture of the electronic apparatus, and the graphical object can be displayed in the determined area. In addition, a graphical object of a shape determined based on the posture of the electronic apparatus and the direction in which the external object is located is displayed in the determined area.

As an example, if the posture of the electronic apparatus is within a predetermined range from a first posture in which the display is facing up, a graphical object corresponding to a top view of an external object is displayed. Specifically, the graphical object corresponding to the top view may be displayed at an angle corresponding to the determined area. In this case, the first posture may also include a posture in which the display faces up, as well as a posture in which the display faces down.

When the posture of the electronic apparatus is within a predetermined range from a second posture perpendicular to the direction in which the display is facing in the first posture, a graphical object corresponding to a side view of the external object is displayed. Specifically, a graphical object corresponding to the side view viewed from a viewpoint corresponding to the determined area.

The controlling method can compare an audio signal of an external object received through an audio receiver with a pre-stored audio signal to identify a speed in which an external object approaches the electronic apparatus. Based on the identified speed, a notification of the presence or proximity of the external object may be provided.

<FIG> is a flowchart of a controlling method of an electronic apparatus to determine an area of a display corresponding to a direction of an external object based on an audio signal received through two microphones according to an embodiment.

Referring to <FIG>, after the audio signal of the sound is received through two microphones in operation S1310, when the electronic apparatus is moved in operation S1320'-Y, the audio signal of the sound may be received through at least one of the two microphones in operation S1330. At this time, the movement of the electronic apparatus can be determined through at least one of an acceleration sensor and a GPS sensor.

Based on the time difference of arrival of each received audio signal, information about a direction in which an external device is located can be obtained on the basis of the electronic apparatus in operation S1340. In addition, the posture of the electronic apparatus can be determined by using at least one of the gyro sensor and the geomagnetic sensor in operation S1350.

Thereafter, based on the posture of the electronic apparatus, the area corresponding to the direction in which the external object is located is determined in operation S1360, and the graphical object representing the external object can be displayed in the corresponding area in operation S1370. The shape of the graphical object may vary depending on the posture of the electronic apparatus and the direction in which the external object is located.

<FIG> is a flowchart of displaying a graphical object corresponding to an external object in a method of controlling an electronic apparatus according to an embodiment.

Referring to <FIG>, after receiving the audio signal in operation S1405, it is possible to identify whether the volume of the received audio signal is greater than or equal to a predetermined value in operation S1410. If the volume is equal to or greater than a predetermined value in operation S1410-Y, the posture of the electronic apparatus and the direction in which the external object is located may be identified in operation S1415. At this time, the posture of the electronic apparatus can be identified by using the gyro sensor, and the direction in which the external object is located can be identified based on the previously received audio signal.

The area corresponding to the direction in which the external object is located on the display is determined based on the posture of the electronic apparatus in operation S1420. In this case, if the posture of the electronic apparatus is within a preset range from the first posture in which the display faces up or down in operation S1425-Y, the graphical object corresponding to the top view of the external object may be displayed in operation S1430. The graphical object including at least a part of the top view of the object corresponding to the type of the external object may be displayed.

In this case, by using the volume change of the received audio signal or the result of comparing the received audio signal with the pre-stored plurality of audio signals, whether the external object approaches the electronic apparatus may be identified in operation S1435.

If the external object approaches in operation S1435-Y, the graphical object corresponding to the top view of the external object is still displayed in operation S1440, but the size and/or shape of the graphical object may be changed. Specifically, rather than step S1430, a larger portion of the top view of the object corresponding to the type of external object may be included in the graphical object.

If the external object is not approaching in operation S1435-N, the graphical object can stop being displayed in operation S1445.

If the posture of the electronic apparatus is not in a predetermined range from the first posture in operation S1425-N, the graphical object corresponding to the side view of the external object can be displayed in operation S1450. Specifically, a graphical object including at least a portion of the images viewed from <NUM>-degree angle including front, back, left, and right of the object corresponding to the type of the external object may be displayed.

By using the volume change of the received audio signal and a result of comparing the received audio signal with a plurality of prestored audio signals, whether the external object approaches the electronic apparatus may be identified in operation S1455.

If the external object approaches in operation S1455-Y, the graphical object corresponding to the side view of the external object is still (continuously) displayed in operation S1460, and the size and/or shape of the graphical object may be changed. Specifically, rather than step S1450, a graphical object may be displayed such that the graphical object gets larger or a graphical object that includes a larger portion of the side view of the object corresponding to the type of external object may be displayed.

If the external object does not approach in operation S1435-N, the graphical object may stop being displayed in operation S1445.

The controlling method of the electronic apparatus described with reference to <FIG> and <FIG> can be implemented through the electronic apparatus <NUM> shown and described with reference to <FIG> and <FIG>. Alternatively, it may be implemented via a system that includes a plurality of electronic apparatus, and one example is <FIG>.

The various embodiments described above may be implemented in a recordable medium which is readable by computer or a device similar to computer using software, hardware, or the combination of software and hardware.

By hardware implementation, the embodiments of the disclosure may be implemented using at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, or electric units for performing other functions.

In some cases, embodiments described herein may be implemented by the processor <NUM> itself. According to a software implementation, embodiments such as the procedures and functions described herein may be implemented with separate software modules. Each of the above-described software modules may perform one or more of the functions and operations described herein.

The computer instructions for performing the processing operations in the electronic apparatus <NUM> according to the various embodiments described above may be stored in a non-transitory computer-readable medium. The computer instructions stored in this non-transitory computer-readable medium cause the above-described specific device to perform the processing operations in the electronic apparatus <NUM> according to the above-described various embodiments when executed by the processor of the specific device.

Claim 1:
An electronic apparatus comprising:
an audio receiver (<NUM>) configured to obtain an audio signal of sound output by an external object;
a sensor (<NUM>) configured to sense a posture of the electronic apparatus;
a display (<NUM>); and
a processor (<NUM>) configured to:
based on the audio signal that is obtained by the audio receiver (<NUM>), determine a direction in which the external object is located with respect to the electronic apparatus, determine a posture of the electronic apparatus using the sensor (<NUM>),
determine an area (<NUM>, <NUM>, <NUM>; <NUM>, <NUM>, <NUM>) of the display (<NUM>) that corresponds to the direction in which the external object is located, based on the posture of the electronic apparatus,
characterized in,
determine a shape of a graphical object that corresponds to the external object according to the posture and the direction in which the external object is located,
control the display to display the graphical object in the determined shape and in the determined area of the display (<NUM>),
based on the posture of the electronic apparatus being within a first predetermined range from a first posture in which the display (<NUM>) faces up, control the display to display (<NUM>) the graphical object (<NUM>', <NUM>', <NUM>') corresponding to a top view of the external object, and
based on the posture of the electronic apparatus being within a second predetermined range from a second posture that is perpendicular to a direction of the display (<NUM>) from the first posture, control the display to display the graphical object (<NUM>', <NUM>', <NUM>') corresponding to a side view of the external object.