Patent Description:
As social network services are generally used in people's everyday lives, use of selfie photography and interest in it are on an increasing trend.

Following the increased use of social network services, people's need for taking best selfie photographs is also increasing. Accordingly, use of applications which recommend angles of selfie photographs, and applications which provide additional functions for editing selfie photographs such as filters, face image correction and stickers for selfie photographs is increasing.

Document <CIT> describes a mobile device for optimizing a user's position with respect to the display, comprising a CPU that configured to displaying the abstracted positioning imagery of the user on a display facing the user. The abstracted positioning imagery provides visual feedback for guiding the user to move to a desired position with respect to the camera. It is indicated that visual feedback can be elicit movements of the user towards or away from the display and/or camera (i.e., the z-direction) of a mobile device or system to which a camera and a display are connected as separate units. Suitable visual feedback may include altering the size of the displayed abstracted imagery or inviting the user to adjust their distance from the camera until their outline just fits within the outline, when the size of the displayed abstracted imagery is varied. Both methods may be used in differing proportions to elicit the desired z-direction movements.

<CIT> discloses focus control in a camera photographing a person wearing augmented reality (AR) glasses.

<CIT> discloses a user interface for instructing a person to move with respect to a smartphone camera.

<CIT> discloses using audio or vibrations to instruct a user how to rotate a smartphone when taking a selfie using the rear facing camera.

Previously, such functions had a problem, which is that they could not analyze selfie photographs correctly, and thus they could neither recommend angles of selfie photographs to suit the user's intention nor edit selfie photographs precisely according to the user's intention.

One or more example embodiments provide an electronic apparatus that analyzes a three-dimensional (3D) coordinate corresponding to a user image included in a photographed image and provides a guide for adjusting the photographing position, or provides editing features such as stickers to be added to a user image, and a control method thereof.

The invention provides AR glasses as set out in appended claim <NUM> and a method of controlling AR glasses as set out in appended claim <NUM>.

According to the various embodiments of the disclosure, an electronic apparatus provides a guide that adjusts photographing positions based on 3D coordinate information, and thus photography may be performed in a composition intended by a user. Also, an electronic apparatus may provide precise editing functions such as adjusting a sticker to be added to a user image precisely based on 3D coordinate information.

The matters defined in the description, such as detailed construction and elements, are provided to assist in a comprehensive understanding of the exemplary embodiments. However, it is apparent that the exemplary embodiments can be practiced without those specifically defined matters.

As terms used in the embodiments of the disclosure, general terms that are currently used widely were selected as far as possible, in consideration of the functions described in the disclosure. However, the terms may vary depending on the intention of those skilled in the art, previous court decisions or emergence of new technologies. Also, in particular cases, there may be terms that were arbitrarily designated by the applicant, and in such cases, the meaning of the terms will be described in detail in the relevant descriptions in the disclosure. Thus, the terms used in the disclosure should be defined based on the meaning of the terms and the overall content of the disclosure, but not just based on the names of the terms.

Further, various modifications may be made to the embodiments of the disclosure, and there may be various types of embodiments. Accordingly, specific embodiments will be illustrated in drawings, and the embodiments will be described in detail in the detailed description. However, it should be noted that the various embodiments are not for limiting the scope of the disclosure to a specific embodiment.

In addition, the expressions "first," "second" and the like used in the disclosure may be used to describe various elements, but the expressions are not intended to limit the elements. Such expressions are used only to distinguish one element from another element.

Also, singular expressions may be interpreted to include plural expressions, unless defined obviously differently in the context. In this specification, terms such as 'include' and 'consist of' should be construed as designating that there are such characteristics, numbers, steps, operations, elements, components or a combination thereof in the specification, but not as excluding in advance the existence or possibility of adding one or more of other characteristics, numbers, steps, operations, elements, components or a combination thereof.

Also, "a module" or "a part" in the disclosure perform at least one function or operation, and these elements may be implemented as hardware or software, or as a combination of hardware and software. Further, a plurality of "modules" or "parts" may be integrated into at least one module and implemented as at least one processor, except "modules" or "parts" that need to be implemented as specific hardware.

Hereinafter, the embodiments of the disclosure will be described in detail with reference to the accompanying drawings, such that those having ordinary skill in the art to which the disclosure belongs can easily carry out the disclosure. However, it should be noted that the disclosure may be implemented in various different forms, and is not limited to the embodiments described herein. Also, in the drawings, parts that are not related to explanation were omitted, for explaining the disclosure clearly, and throughout the specification, similar components were designated by similar reference numerals.

<FIG> is a diagram for illustrating an operation of performing selfie photography, i.e., an operation of taking a selfie or an operation of capturing a selfie, by using an electronic apparatus according to an example embodiment.

As illustrated in <FIG>, an electronic apparatus <NUM> may be implemented as a mobile phone such as a smartphone, but the present embodiment is not limited thereto, and the electronic apparatus may be implemented as various types of apparatuses including a camera function and a display function, such as a tablet personal computer (PC), a mobile phone, an e-book, a desktop personal computer (PC), a laptop personal computer (PC), a netbook computer, a personal digital assistant (PDA), a portable multimedia player (PMP), an MP3 player, a mobile medical instrument, a camera, a camcorder, an electronic photo frame or a wearable device (e.g., a head-mounted-device (HMD) smart watch, electronic clothing, an electronic bracelet, an electronic necklace and the like), a near eye display (NED), a large format display (LFD), digital signage, a Digital Information Display (DID) and a video wall. Also, the display apparatus <NUM> has a touch screen embedded therein, and thus the apparatus may be implemented such that a program can be executed by using a finger or a pen (e.g., a stylus pen).

According to an embodiment of the disclosure, in case in which a user takes a photograph (i.e., takes an image, captures an image, etc.) including himself/herself using the electronic apparatus <NUM>, the electronic apparatus <NUM> may provide guide information so that the user may obtain a desirable photograph. In the present disclosure, the term "selfie" may refer to an image that includes oneself (with another person or a part of a group) and is taken by oneself using a camera. For example, the electronic apparatus <NUM> may provide guide information based on a three-dimensional (3D) coordinate corresponding to the face region of a user in a real-time photographed image.

<FIG> is a block diagram for illustrating an electronic apparatus according to an example embodiment.

According to <FIG>, the electronic apparatus <NUM> includes a camera <NUM>, a storage <NUM>, a display <NUM> and a processor <NUM>.

The camera <NUM> may obtain an image by photographing an object according to a user instruction. Here, a user instruction may be in various forms such as a touch input, a button input, a voice input and a motion input.

When an object is photographed through the camera <NUM>, the object is converted into an electronic image signal through a semiconductor optical element, such as a charge coupled device (CCD), and the electronic image signal is amplified and converted into a digital signal by an analog signal processor. Then, the digital signal is processed by a digital signal processor, and the processor <NUM> may control the display <NUM> to adjust color and brightness of an image that is represented by the digital signal and to display the image with the adjusted color and brightness.

Here, the displayed image may be a still image or a moving image. The camera <NUM> may photograph a still image at a specific time point, or may continuously photograph still images. Also, the camera <NUM> may provide an image obtained under the control of the processor <NUM> to the display <NUM>.

The camera <NUM> may be implemented in a plural number, such as a front camera provided on the front surface of the electronic apparatus <NUM> and a rear camera provided on the rear surface of the electronic apparatus <NUM>. For example, selfie photography of which subject is a user may be performed through a front camera.

Meanwhile, the storage <NUM> may store various types of data, programs or applications for driving/controlling the electronic apparatus <NUM>. The storage <NUM> may store a control program for controlling the electronic apparatus <NUM> and the processor <NUM>, applications that were initially provided by the manufacturer or downloaded from the outside, databases or relevant data.

In particular, the storage <NUM> may store at least one of reference 3D coordinate information or reference image information. Here, reference 3D coordinate information is information on a 3D coordinate which was obtained from a reference image, and which corresponds to the face region included in the reference image. The reference image may be at least one of an image selected by a user or an image provided by the electronic apparatus <NUM>, or it may be a target image aimed by an image photographed by a user.

Further, the storage <NUM> may store information on the actual size of the electronic apparatus <NUM>. To be specific, the storage <NUM> may store information on sizes such as the horizontal length, longitudinal length and thickness of the electronic apparatus <NUM>.

Meanwhile, the storage <NUM> may be implemented as internal memory such as read-only memory (ROM) and random-access memory (RAM) included in the processor <NUM>, or it may be implemented as separate memory from the processor <NUM>. In this case, the storage <NUM> may be implemented in the form of memory embedded in the electronic apparatus <NUM>, or memory that can be attached to or detached from the electronic apparatus <NUM>, depending on the use of the stored data. For example, in the case of data for driving the electronic apparatus <NUM>, the data may be stored in memory embedded in the electronic apparatus <NUM>, and in the case of data for the extending function of the electronic apparatus <NUM>, the data may be stored in memory that can be attached to or detached from the electronic apparatus <NUM>. Meanwhile, in the case of memory embedded in the electronic apparatus <NUM>, the memory may be implemented in the form of nonvolatile memory, volatile memory, a hard disc drive (HDD) or a solid state drive (SSD).

According to another example embodiment, it is possible that at least one of reference 3D coordinate information or reference image information is received from a server. Here, a server may be implemented as a cloud server, but is not limited thereto. A server may also be implemented as an external server or an embedded server provided in the electronic apparatus <NUM>, depending on its form of physical implementation.

The display <NUM> displays various contents such as a still image, a moving image, texts and music, an application execution screen including various contents, a graphic user interface (GUI) screen and the like.

Also, the display <NUM> may display an image (e.g., a preview image or a photographed image) that is being photographed through the camera <NUM>. In addition, the display <NUM> may provide a guide GUI for adjusting the photographing position, i.e., the image capture position, of the camera <NUM> by the control of the processor <NUM> and a guide GUI (or a guide graphic item) for moving the face region of a user. Further, the display <NUM> may provide a filtering effect such as a sticker to be added to a user image included in a photographed image by the control of the processor <NUM>.

Meanwhile, the display <NUM> may be implemented in various forms such as a liquid crystal display (LCD), an organic light-emitting diode (OLED), liquid crystal on silicon (LCoS), digital light processing (DLP), a quantum dot (QD) and a micro light-emitting diode (LED) display. In particular, the display <NUM> may be implemented in the form of a touch screen that forms an interlayer structure with a touch pad. In this case, the display <NUM> may be used as a user interface as described above, as well as an output device. Here, a touch screen may be constituted such that it can detect the pressure of a touch input as well as the location and area of a touch input.

The processor <NUM> controls the overall operation of the electronic apparatus <NUM>.

According to an example embodiment, the processor <NUM> may be implemented as a digital signal processor (DSP) processing digital signals, a microprocessor and a time controller (TCON). However, the disclosure is not limited thereto, and the processor <NUM> may include one or more of a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP) or a communication processor (CP) and an ARM processor, or may be defined by the terms. Also, the processor <NUM> may be implemented as a system on chip (SoC) or a large scale integration (LSI) having a processing algorithm embedded therein, or as a field programmable gate array (FPGA).

Further, the processor <NUM> may obtain a 3D coordinate corresponding to the face region of a user included in an image photographed through the camera <NUM>. Here, the photographed image may be an image in a preview state where the face region of a user can be checked through the display <NUM>. Meanwhile, the 3D coordinate is a coordinate of which starting point is the camera <NUM>, and it may be implemented as coordinate systems in various forms, such as a rectangular coordinate system, a polar coordinate system and a spherical coordinate system.

Meanwhile, the processor <NUM> may obtain a 3D coordinate corresponding to the face region of a user by using various methods.

According to an example embodiment, the processor <NUM> may obtain a 3D coordinate corresponding to the face region of a user from a photographed two-dimensional (2D) image. For example, the processor <NUM> may calculate information on the distance among feature points of the face region of a user based on the number of pixels, and thereby obtain a 3D coordinate.

According to another example embodiment, the processor <NUM> may obtain a 3D coordinate by receiving position information from an external apparatus (e.g., an accessory attached to a user) provided with a sensor, which is included in a photographed image. For example, the processor <NUM> may obtain a 3D coordinate based on the distance between the electronic apparatus <NUM> and an external apparatus, which was calculated based on the position information received from the external apparatus.

According to still another example embodiment, the processor <NUM> may obtain a 3D coordinate based on an image received from an external electronic apparatus such as an augmented reality (AR) glass worn by a user. For example, the processor <NUM> may calculate the distance between the electronic apparatus <NUM> and the face region of a user by comparing information on the size of the electronic apparatus <NUM> calculated from a received image with information on the actual size of the electronic apparatus <NUM>, and thereby obtain a 3D coordinate.

Further, the processor <NUM> may obtain position adjustment information for adjusting the photographing position of the camera <NUM> based on a difference between the obtained 3D coordinate and the reference 3D coordinate information stored in the storage <NUM>. Here, the reference 3D coordinate information is information on a 3D coordinate which was obtained from a reference image, and which corresponds to the face region included in the reference image. The reference image may be at least one of an image selected by a user or an image provided by the electronic apparatus <NUM>, or it may be a target image aimed by an image photographed by a user.

Meanwhile, position adjustment information may include the moving distance of the camera <NUM> by which the camera is to be moved, the moving direction of the camera, the rotating direction of the camera and the degree of rotation of the camera, according to the difference between the obtained 3D coordinate and the reference 3D coordinate information. Here, the position adjustment information may be 3D coordinate information corresponding to the difference between the obtained 3D coordinate and the reference 3D coordinate information.

The processor <NUM> may control the display <NUM> to provide a guide GUI for adjusting the photographing position of the camera <NUM> based on the obtained position adjustment information.

Here, the photographing position of the camera <NUM> may include at least one of the photographing distance between the face region of the user and the camera <NUM>, the photographing direction or the photographing angle.

Meanwhile, a guide GUI may be a sign including at least one of the moving distance of the camera <NUM>, the moving direction of the camera <NUM>, the rotating direction of the camera <NUM> or the degree of rotation of the camera <NUM>. For example, a guide GUI may be provided as a sign in the shape of an arrow that indicates directions on the display <NUM> for displaying the moving direction and the rotating direction. Also, a guide GUI may be provided in the form of indicating a clockwise direction or a counterclockwise direction through an arrow shape.

The processor <NUM> may display XYZ axes in an area of the display <NUM>, and display a guide GUI on the XYZ axes. Accordingly, the guide GUI may be displayed more stereoscopically.

In addition, the processor <NUM> may indicate the moving distance and degree of rotation of the camera <NUM> by adjusting the length of an arrow. That is, the processor <NUM> may display the length of an arrow to be proportional to the moving distance and degree of rotation of the camera <NUM>. For example, in a case in which the camera <NUM> needs to be moved to the right side by <NUM> based on the obtained position adjustment information, the processor <NUM> may display an arrow having a length of <NUM> on the right side of the display <NUM>. Afterwards, when the camera <NUM> is moved to the right side by <NUM>, the processor <NUM> may adjust the length of the arrow to <NUM>.

According to another example embodiment, the processor <NUM> may obtain information for moving the face region of a user based on a difference between a 3D coordinate corresponding to the face region of a user included in a photographed image and the reference 3D coordinate information stored in the storage <NUM>, and control the display <NUM> to provide a guide GUI for moving the face region of the user based on the obtained information. Here, the information for moving the face region of the user may include information on the moving distance by which the face region of the user is to be moved, the moving direction, the rotating direction and the degree of rotation according to the difference between the obtained 3D coordinate and the reference 3D coordinate information. Further, the information for moving the face region of the user may include a distance between the center of the photographed image and the center of the face region. The information for moving the face region of the user may be 3D coordinate information corresponding to the difference between the obtained 3D coordinate and the reference 3D coordinate information.

That is, the processor <NUM> may provide the display <NUM> with a guide GUI for guiding movement of the face region of a user, while maintaining the photographing position of the camera <NUM>.

To be specific, the processor <NUM> may provide the display <NUM> with a guide GUI corresponding to the face of a user in a target position based on the information for moving the face region of the user. For example, a guide GUI may be implemented in the form of a circle or an oval, and may guide such that the face region of a user is positioned on the guide GUI. Alternatively, the guide GUI may guide such that the face region of a user is positioned on the guide GUI by displaying the face image of the user in a target position. However, the disclosure is not limited thereto, and a guide GUI can obviously be implemented in various forms as long as it performs the role of guiding movement of a user's body.

According to still another example embodiment, the processor <NUM> may provide a guide for adjusting the photographing position of the camera <NUM> or moving the face region of a user as voice. For example, the processor <NUM> may provide voice such as "Move the camera to the right side by <NUM>" or "Tilt your face to the left side by <NUM> degrees" through a speaker, based on the obtained position adjustment information or information for moving the face region of a user.

For example, when reference 3D coordinate information is obtained from a reference image in which the user is not gazing at the camera <NUM>, the processor <NUM> may provide guidance on posture that makes the user not gaze at the display <NUM>. In a case in which a user cannot gaze at the camera <NUM> according to the guidance, the user cannot recognize a guide GUI provided on the display <NUM> since the user cannot look at the screen of the display <NUM>. In such a case, a voice guide may be used instead of a visual guide.

Meanwhile, the processor <NUM> may additionally provide a voice guide, as well as providing the display <NUM> with a guide GUI for adjusting the photographing position of the camera <NUM> or a guide GUI for moving the face region of a user.

Hereinafter, various embodiments of the disclosure wherein a 3D coordinate is obtained from a photographed image will be described.

According to an example embodiment, the processor <NUM> may identify a plurality of feature points from the face region of a user, calculate information on the distance among the feature points based on the number of pixels, and obtain a 3D coordinate based on the calculated distance information. Here, the feature points may be extracted from at least one of the eye, nose or mouth of the user in the photographed image. As an example, the processor <NUM> may group the entire pixels in a photographed image into a plurality of blocks consisting of n*m pixels, identify whether there is a feature point in each block, and thereby identify feature points in the face region of the user.

The processor <NUM> may calculate distance information such as the size of the eye, the distance between the eyes, the length of the nose and the length of the lip based on the number of pixels. Further, the processor <NUM> may obtain a 3D coordinate corresponding to the face region of a user by correcting the distance information to be more precise by comparing the calculated distance information with information on average faces stored in the storage <NUM> or the stored information on the face of the user. For example, the processor <NUM> may calculate distance information for the size of the eye based on the number of pixels of a photographed image, and obtain information on the size of the eye by comparing the distance information with the stored distance information for an average eye size.

Also, the processor <NUM> may calculate information on angles based on comparison of the sizes of the eyes, the angle of the line connecting the eyes, the angle of the nose line and the angle of the lip line.

For example, the processor <NUM> may identify facial feature points based on a rectangular 3D coordinate system in which the starting point is assumed as the location of the camera <NUM>. In particular, the processor <NUM> may identify the eye, nose and mouth in the face region of a user as feature points, and obtain a rectangular coordinate value of which starting point is the camera <NUM>, based on the number of pixels. As an example, the processor <NUM> may obtain a 3D coordinate for the eye, nose and mouth such as {Xeye = <NUM>, Yeye = <NUM>, Zeye = <NUM>}, {Xnose = <NUM>, Ynose = <NUM>, Znose = <NUM>} and {Xlip = <NUM>, Ylip = <NUM>, Zlip = <NUM>}.

The processor <NUM> may obtain position adjustment information for adjusting the photographing position of the camera <NUM> based on a difference between the obtained 3D coordinate and the reference 3D coordinate information stored in the storage <NUM>.

For example, the processor <NUM> may obtain {Xeye = <NUM>, Yeye = -<NUM>, Zeye = <NUM>}, {Xnose = <NUM>, Ynose = <NUM>, Znose = <NUM>} and {Xlip = <NUM>, Ylip = <NUM>, Zlip = <NUM>} corresponding to the difference value between the 3D coordinate obtained for the eye, nose and mouth and the reference 3D coordinate information as position adjustment information.

The electronic apparatus <NUM> may calculate the target position of the camera <NUM> guided by a guide GUI, by using a rotation transformation matrix as follows. <MAT> <MAT>.

Here, θ(X axis), Φ(Y axis) and Ψ(Z axis) denote the angle of the camera <NUM> which is to be changed based on the position adjustment information, and Xcam, Ycam and Zcam denote the distance of the camera <NUM> by which the camera is to be moved. Meanwhile, X, Y and Z denote a 3D coordinate corresponding to the position adjustment information, and X', Y' and Z' denote a 3D coordinate corresponding to the target position guided by a guide GUI.

The electronic apparatus <NUM> may calculate the target position of the camera <NUM> based on Formula <NUM> described above.

In a case in which the position of the camera <NUM> is moved to the target position according to the guide GUI, the 3D coordinate obtained in the target position and the reference 3D coordinate information may be identical.

The processor <NUM> may provide the display <NUM> with a guide GUI for adjusting the position of the camera <NUM> based on the obtained position adjustment information.

<FIG> is a diagram for illustrating the process of obtaining a 3D coordinate based on the position information received from an external apparatus according to another example embodiment.

According to <FIG>, the processor <NUM> may receive the position information of the external apparatus <NUM> via a communication interface. The external apparatus <NUM> may be present in an image photographed by the electronic apparatus <NUM>. The processor <NUM> may obtain first distance information between the electronic apparatus <NUM> and the external apparatus <NUM> based on the received position information. Also, the processor <NUM> may obtain second distance information between the external apparatus <NUM> included in the photographed image and at least one feature point included in the face region of the user. Further, the processor <NUM> may obtain a 3D coordinate based on the first distance information and the second distance information.

Here, the external apparatus <NUM> is an apparatus provided with a sensor that can calculate distance from the electronic apparatus <NUM>. For example, the external apparatus <NUM> may be implemented as accessories such as earrings <NUM>-<NUM> and necklaces or earphones <NUM>-<NUM>, and the like. However, the disclosure is not limited thereto, and the external apparatus <NUM> can be implemented in various forms, if it is an apparatus provided with a sensor that can calculate distance from the electronic apparatus <NUM>, and is attachable to a user.

Meanwhile, position information may include at least one of the distance and the angle between the electronic apparatus <NUM> and the external apparatus <NUM>.

The type of the sensor in the external apparatus <NUM> may include an infrared ray (IR) sensor, an accelerometer, an angular velocity sensor, a gyro sensor, a hall sensor, an ultrasonic sensor and the like.

An IR sensor is a sensor that emits infrared ray and measures distance according to the amount of the infrared ray reflected on a reflecting object. Based on a value measured by an IR sensor, the distance between the electronic apparatus <NUM> and the external apparatus <NUM> can be measured.

An accelerometer can detect the orientation of gravity. Also, an accelerometer can detect tilt in a motionless state. In addition, an accelerometer can detect the amount of change of the velocity with respect to a time unit. An accelerometer may be implemented as a three-axis accelerometer. In case an accelerometer is implemented as a three-axis accelerometer, the accelerometer includes accelerometers of x, y and z axes that are disposed in different directions from one another, and are orthogonal to one another. The accelerometer converts the output values of each of the accelerometers of x, y and z axes into digital values, and provide the values to a preprocessor. Here, the preprocessor may include a chopping circuit, an amplifying circuit, a filter, an analog-to-digital (A/D) converter and the like. Accordingly, an electronic signal output from the three-axis accelerometer is chopped, amplified and filtered, and is then converted into a digital voltage value. Then, the angle between the electronic apparatus <NUM> and the external apparatus <NUM> can be measured based on the value measured by the accelerometer.

A gyro sensor is a sensor that detects the amount of change of the preset direction of a user terminal device <NUM> during a time unit, and thereby detects the angular velocity. As a gyro sensor, a gyroscope having three axes may be used. Through the definite integral value of the angular velocity detected by the gyro sensor, the angle between the electronic apparatus <NUM> and the external apparatus <NUM> can be measured.

A hall sensor is a sensor that measures the magnetic field. Based on the size of the magnetic field measured at the hall sensor, the distance between the electronic apparatus <NUM> and the external apparatus <NUM> can be measured.

Meanwhile, the processor <NUM> may obtain second distance information which is the distance information between the external apparatus <NUM> and at least one feature point included in the face region of the user based on the number of pixels. Also, the processor <NUM> may correct the second distance information to be more precise by comparing the calculated second distance information with the information on average bodies stored in the storage <NUM> or the stored information on the user's body. For example, when the external apparatus <NUM> is an earphone <NUM>-<NUM> including a sensor, the processor <NUM> may calculate the distance between the ear to which the earphone <NUM>-<NUM> is attached and the nose based on the number of pixels. Afterwards, the processor <NUM> may correct the second distance information to be more precise by comparing the distance calculated based on the number of pixels with the distance between the ear and the nose stored in the storage <NUM>. Specifically, the processor <NUM> may calculate a difference between the calculated distance and the distance stored in the storage <NUM>, and may scale down the calculated down by applying a weight less than <NUM> to the calculated distance when the difference is greater than a predetermined threshold. For example, the weight may have a value greater than <NUM> and less than <NUM>, and the value of the weight may decrease in proportion to the amount of the difference exceeding the predetermined threshold.

Also, the processor <NUM> may obtain a 3D coordinate based on the first distance information and the second distance information. Afterwards, the processor <NUM> may obtain position adjustment information for adjusting the photographing position of the camera <NUM> based on a difference between the obtained 3D coordinate and the reference 3D coordinate information stored in the storage <NUM>, and provide a guide GUI for adjusting the photographing position of the camera <NUM> based on the obtained position adjustment information.

In an embodiment in which a 3D coordinate is obtained based on information received from the external apparatus <NUM>, the distance information between the electronic apparatus <NUM> and the external apparatus <NUM> is relatively correct. Accordingly, a precise 3D coordinate corresponding to the face region of the user can be obtained.

<FIG> is a diagram for illustrating the process of obtaining a 3D coordinate based on an image received from an external electronic apparatus according to the invention.

According to <FIG>, the processor <NUM> may receive, from the external electronic apparatus <NUM>, an image <NUM> including the electronic apparatus <NUM> through the communication interface. The processor <NUM> may compare the size of the electronic apparatus <NUM> obtained from the received image <NUM> with the actual size of the electronic apparatus <NUM> stored in the storage <NUM>, and thereby obtain a 3D coordinate.

Here, the external electronic apparatus <NUM> may be implemented as augmented reality (AR) glass including the camera <NUM>. Also, the size may be information including the horizontal length, longitudinal length and thickness of the electronic apparatus <NUM>.

The processor <NUM> may calculate the distance between the electronic apparatus <NUM> and the face region of the user by comparing the size of the electronic apparatus <NUM> calculated from the image <NUM> received from the external electronic apparatus <NUM> with the actual size of the electronic apparatus <NUM>. Also, the processor <NUM> may calculate the size of the electronic apparatus <NUM> from the received image <NUM> based on the number of pixels.

For convenience of explanation, it is assumed that the storage <NUM> stores a length of <NUM> and a width of <NUM> as the actual dimension of the electronic apparatus <NUM>. Here, the terms "length" may be also referred to as "longitudinal length" or "horizontal length", and the term "width" may be also referred to as "vertical length". If the horizontal length of the electronic apparatus <NUM> included in the received image <NUM> is <NUM>, and the longitudinal length is <NUM>, the processor <NUM> may calculate the number of pixels in the horizontal length and in the longitudinal length of the electronic apparatus <NUM>. Further, the processor <NUM> may calculate the relation between the number of pixels and the actual size, and calculate the distance between the electronic apparatus <NUM> and the face region of the user based on the calculated information.

Also, the processor <NUM> may obtain a 3D coordinate based on the calculated distance between the electronic apparatus <NUM> and the face region of the user. Afterwards, the processor <NUM> may obtain position adjustment information for adjusting the photographing position of the camera <NUM> based on the difference between the obtained 3D coordinate and the reference 3D coordinate information stored in the storage <NUM>, and provide a guide GUI for adjusting the photographing position of the camera <NUM> based on the obtained position adjustment information.

As described above, the distance between the electronic apparatus <NUM> and the face region of the user can be calculated relatively precisely by comparing the size of the electronic apparatus <NUM> calculated from the image <NUM> received from the external electronic apparatus <NUM> with the actual size of the electronic apparatus <NUM>. Thus, the processor <NUM> can obtain a precise 3D coordinate corresponding to the face region of the user.

Here, the external electronic apparatus <NUM> may be implemented in various forms other than AR glass, if it is an apparatus that can photograph the electronic apparatus <NUM> and transmit the photographed image to the electronic apparatus <NUM>.

Returning to <FIG>, when the photographing position of the camera <NUM> is adjusted to a target position, the processor <NUM> may perform photographing automatically. Also, the processor <NUM> may obtain position adjustment information based on a difference between a 3D coordinate corresponding to the face region of the user included in a photographed image and reference 3D coordinate information in real time. Further, the processor <NUM> may perform photographing an image automatically, based on identifying that the 3D coordinate and the reference 3D coordinate information are identical based on the position adjustment information that is measured in real time (i.e., if the photographing position is adjusted to a target position or the face region of the user is moved to a target position).

When the photographing position of the camera <NUM> or the face region of the user is adjusted to a target position, photographing is performed without a user's manipulation. Thus, photographing can be performed in a state wherein the user is not ready for photographing. Accordingly, the processor <NUM> may provide a feedback informing that the photographing position of the camera <NUM> or the face region of the user is adjusted to a target position, or perform photographing automatically after a preset time has lapsed after providing a feedback. Here, a feedback may be in the form of, for example, vibrating the electronic apparatus <NUM>, or outputting specific sound or voice. Alternatively, a feedback may be in the form of an LED light source emitting light.

The processor <NUM> may control the display <NUM> to adjust at least one of the size, direction or form of a sticker to be added to a user image included in a photographed image based on a 3D coordinate corresponding to the face region of the user included in the photographed image, and display the image. As various embodiments of obtaining a 3D coordinate corresponding to the face region of a user were described earlier, detailed description therefor will be omitted.

Here, a sticker may be all objects added to a user image to decorate the user image included in a photographed image. For example, a sticker may be implemented in a form wherein a shape in the form of a rabbit's ears is added to the face region of a user image, or in a form wherein a cosmetic effect is applied to a user image. However, the disclosure is not limited thereto, and a sticker can obviously be implemented in various forms if it is an object that can decorate a user image.

The processor <NUM> may adjust at least one of the size, direction or form of a sticker to be added to a user image based on the obtained 3D coordinate.

For example, a sticker in the shape of rabbit ears may be added to the head portion of the face region of a user. In that case, the processor <NUM> may add a sticker based on the state of the user image based on a 3D coordinate. For example, if a user looks at the left, the processor <NUM> may express a cubic effect by adjusting the sizes of the rabbit's left and right ears to match the user's line of sight. Afterwards, if the looks at the right, the processor <NUM> may readjust the sizes of the rabbit's left and right ears.

As described above, according to an example embodiment, a sticker is applied based on a 3D coordinate. Accordingly, a sticker can be applied to a user image while being adjusted relatively precisely.

Meanwhile, in a case in which the electronic apparatus <NUM> is implemented as AR glass, the AR glass may obtain position adjustment information for adjusting the photographing position of a smart phone based on a difference between a 3D coordinate corresponding to the face region of a user included in a photographed image and the stored reference 3D coordinate information. Then, based on the obtained position adjustment information, the AR glass may provide a guide GUI for adjusting the position of the smartphone on the glass as an AR object. That is, the AR glass may provide the target position of the smartphone based on the position adjustment information on the glass, and guide the position. Here, an AR object is a virtual object related to the actual object included in an image, and in the disclosure, an AR object may be used to represent a guide GUI for adjusting the position of a smart phone.

According to an example embodiment, an AR object may be provided to an image photographed by a camera. For example, an AR object may be provided in various forms, such as being displayed while being rendered on a photographed image, being displayed in the form of an on-screen display (OSD) on a photographed image, or being displayed on a layer that is different from the layer wherein a photographed image is displayed.

According to another example embodiment, an AR object may be provided based on the position of an object that is shown to a user through glass implemented as a transparent display. For example, an AR object may be provided on glass based on a method such as digital light processing (DLP) and a laser scanning projector (LSP).

So far, descriptions have been made by assuming a case of selfie photography wherein an image is photographed by a front camera. However, the example embodiments may be implemented using an image photographed by a rear camera as well as a front camera.

<FIG> is a block diagram for illustrating the electronic apparatus of <FIG> in greater detail.

According to <FIG>, the electronic apparatus <NUM> includes a camera <NUM>, a storage <NUM>, a display <NUM>, a processor <NUM>, a communication interface <NUM> and a speaker <NUM>. Regarding the components illustrated in <FIG> that overlap with the components illustrated in <FIG>, detailed description will be omitted.

The processor <NUM> may include a CPU <NUM>, a ROM (or nonvolatile memory) storing a control program for controlling the electronic apparatus <NUM> and a RAM (or volatile memory) which stores data that is input from the outside of the electronic apparatus <NUM> or is used as a storing area corresponding to the various kinds of work performed at the electronic apparatus <NUM>.

Further, when a preset event occurs, the processor <NUM> may execute the operating system (OS), programs and various applications stored in the storage <NUM>. Also, the processor <NUM> may include a single core, a dual core, a triple core, a quad core and a core of multiple numbers thereof.

The CPU <NUM> accesses the storage <NUM>, and performs booting by using the O/S stored in the storage <NUM>. Also, the CPU performs diverse operations by using various kinds of programs, contents, data and the like stored in the storage <NUM>.

The communication interface <NUM> performs communication with an external apparatus (e.g., an accessory including a sensor) or an external electronic apparatus (e.g., AR glass). Here, the communication interface <NUM> may perform wireless communication with the user terminal device <NUM> by a communication method such as Bluetooth (BT), wireless fidelity (Wi-Fi), zigbee and infrared (IR). Also, the processor <NUM> may perform communication with an external apparatus or an external electronic apparatus through various communication methods such as a serial interface, a universal serial bus (USB) and near field communication (NFC).

For example, when a preset event occurs, the communication interface <NUM> may become an interlocked state by performing communication according to a predefined communication method with an external apparatus or an external electronic apparatus. Here, interlocking may mean all states wherein communication is possible, such as an operation of initializing communication between the electronic apparatus <NUM> and an external apparatus or an external electronic apparatus, an operation of forming a network and an operation of performing paring of apparatuses. For example, apparatus identification information of an external apparatus or an external electronic apparatus may be provided to the electronic apparatus <NUM>, and a process of paring between the two apparatuses may be performed accordingly. As an example, when a preset event occurs in the electronic apparatus <NUM> or an external apparatus or an external electronic apparatus, nearby apparatuses may be searched through a digital living network alliance (DLNA) technology, and paring with the searched apparatuses may be performed, and as a result, the communication interface <NUM> may become an interlocked state.

In particular, position information may be received from an external apparatus through the communication interface <NUM>, and an image including the electronic apparatus <NUM> may be received from an external electronic apparatus.

The speaker <NUM> is a component for outputting various kinds of sound. In particular, the speaker <NUM> may output a voice guide for adjusting the photographing position of the camera <NUM> or a voice guide for moving the face region of a user according to the control by the processor <NUM>. Further, when the photographing position of the camera <NUM> or the face region of a user is adjusted to a target position, the speaker <NUM> may output a voice feedback informing this according to the control by the processor <NUM>.

The microphone <NUM> is a component for receiving voice uttered by a user. The microphone <NUM> may receive voice related to photographing, and transmit the voice to the processor <NUM>.

<FIG> is a diagram for illustrating the process of obtaining position adjustment information according to an example embodiment.

The electronic apparatus <NUM> may obtain a 3D coordinate corresponding to the face region of a user included in an image <NUM> that is photographed through the camera <NUM>.

According to an example embodiment, the electronic apparatus <NUM> may identify a plurality of feature points <NUM> in the face region of a user. Here, the features points may be feature points included in at least one of the eye, nose or mouth of a user. For identifying features points in the face region of a user, technologies such as face detection and face recognition may be used. The electronic apparatus <NUM> may calculate information on the distance among the feature points based on the number of pixels, and obtain a 3D coordinate based on the calculated distance information.

In addition, the electronic apparatus <NUM> may calculate distance information such as the size of the eye, the distance between the eyes, the length of the nose and the length of the lip based on the number of pixels. Further, the electronic apparatus <NUM> may obtain a 3D coordinate corresponding to the face region of a user by correcting the distance information to be more precise by comparing the calculated distance information with the stored information on average faces or the stored information on the face of the user. For example, the electronic apparatus <NUM> may calculate the distance between the face of the user and the camera <NUM> based on the information on the distance between the eyes calculated from a photographed image.

In addition, the electronic apparatus <NUM> may calculate information on angles of facial features based on comparison of the sizes of the eyes, the angle of the line connecting the eyes, the angle of the nose line and the angle of the lip line.

For example, a 3D coordinate is represented by a rectangular coordinate system in which a starting point is the camera <NUM>. According to the rectangular coordinate system, the electronic apparatus <NUM> may identify the eye, nose and mouth in the face region of a user as feature points, and obtain a rectangular coordinate value of which starting point is the camera <NUM>, based on the number of pixels. As an example, the electronic apparatus <NUM> may obtain 3D coordinates <NUM> for the eye, nose and mouth such as {Xeye = <NUM>, Yeye = <NUM>, Zeye = <NUM>}, {Xnose = <NUM>, Ynose = <NUM>, Znose = <NUM>} and {Xlip = <NUM>, Ylip = <NUM>, Zlip = <NUM>}.

Further, the electronic apparatus <NUM> may obtain, from a reference image <NUM>, a 3D coordinate corresponding to the face region of the user included in a photographed image, and may compare the 3D coordinate with reference 3D coordinate information <NUM>. The electronic apparatus <NUM> may store the reference image <NUM>, or store only the reference 3D coordinate information <NUM>.

Then, the electronic apparatus <NUM> may obtain position adjustment information for adjusting the photographing position of the camera <NUM> by calculating a difference between a 3D coordinate corresponding to the face region of the user included in a photographed image and the reference 3D coordinate information <NUM>.

For example, the electronic apparatus <NUM> may obtain {Xeye = <NUM>, Yeye = -<NUM>, Zeye = <NUM>}, {Xnose = <NUM>, Ynose = <NUM>, Znose = <NUM>} and {Xlip = <NUM>, Ylip = <NUM>, Zlip = <NUM>} corresponding to the difference value between the 3D coordinate obtained for the eye, nose and mouth and the reference 3D coordinate information as position adjustment information.

Meanwhile, in a case in which the position of the camera <NUM> is moved to a target position according to a guide GUI, the 3D coordinate obtained in the target position and the reference 3D coordinate information may be identical.

The electronic apparatus <NUM> may provide the display <NUM> with a guide GUI for adjusting the position of the camera <NUM> based on the obtained position adjustment information.

<FIG> are diagrams for illustrating the types of guide GUIs according to an example embodiment.

According to <FIG>, the electronic apparatus <NUM> may provide guide GUIs <NUM>, <NUM>, and <NUM> based on the position adjustment information for adjusting the photographing position of the camera <NUM>.

Here, the position adjustment information may include information on the moving distance of the camera <NUM> by which the camera <NUM> is to be moved, the moving direction of the camera <NUM>, the rotating direction of the camera <NUM> and the degree of rotation of the camera <NUM>, according to the difference between the obtained 3D coordinate and the reference 3D coordinate information. Further, the position adjustment information may be 3D coordinate information corresponding to the difference between the obtained 3D coordinate and the reference 3D coordinate information.

Meanwhile, the guide GUIs <NUM>, <NUM>, and <NUM> may indicate at least one of the moving distance of the camera <NUM>, the moving direction of the camera <NUM>, the rotating direction of the camera <NUM> or the degree of rotation of the camera <NUM>. For example, the guide GUIs <NUM>, <NUM>, and/or <NUM> may have the shape of an arrow that indicates directions for displaying the moving direction and the rotating direction. Also, the guide GUIs <NUM>, <NUM>, and/or <NUM> may be provided as signs in the shape of an arrow that indicates a clockwise direction or a counterclockwise direction for displaying the rotating direction.

For example, the electronic apparatus <NUM> may provide a guide GUI that includes a first graphic item <NUM>, a second graphic item <NUM>, and a third graphic item <NUM>. The first graphic item <NUM> may guide the user to move the lower end of the electronic apparatus <NUM> to the upper left side, the second graphic item <NUM> may guide the user to move the electronic apparatus <NUM> to the right side, and the third graphic item <NUM> may guide the user to rotate the electronic apparatus <NUM> in the direction of an arrow. Here, the guide GUI is described as including the three graphic items <NUM>, <NUM>, and <NUM>, but each of the graphic items <NUM>, <NUM>, and <NUM> may be referred to as a guide GUI. For example, the third graphic item <NUM> may be referred to as a rotation guide GUI.

According to <FIG>, the electronic apparatus <NUM> may display the first graphic item <NUM>, the second graphic item <NUM>, and the third graphic item <NUM> on XYZ axes, respectively.

For example, it is assumed that the first graphic item <NUM>, the second graphic item <NUM>, and the third graphic item <NUM> are displayed on XYZ axes. As an example, for displaying movement on XZ axes, the electronic apparatus <NUM> may display a dotted line on the XZ axes and display an arrow, and thereby provide the first graphic item <NUM> that guides movement of the electronic apparatus <NUM> to the upper left side. Also, the electronic apparatus <NUM> may display an arrow guiding movement on an X axis, and thereby provide the second graphic item <NUM> that guides the electronic apparatus <NUM> to the right side. In addition, the electronic apparatus <NUM> may display an arrow in a clockwise direction that guides rotational movement, and thereby provide the third graphic item <NUM>. Further, the electronic apparatus <NUM> may display a separate rotating axis, and thereby display rotating directions clearly. As an example, the electronic apparatus <NUM> may display a rotating axis corresponding to the Y axis, and use an arrow in a clockwise direction, and thereby provide the third graphic item <NUM>. Further, the electronic apparatus <NUM> may provide the third graphic item <NUM> on the XYZ axes without a separate rotating axis.

Meanwhile, the electronic apparatus <NUM> may display an angle by which the electronic apparatus <NUM> is to be rotated in the direction indicated by the third graphic item <NUM>.

As guide GUIs are displayed on XYZ axes and are thus provided more stereoscopically, a user can move the electronic apparatus <NUM> to a target position according to position adjustment information easily.

According to <FIG>, the electronic apparatus <NUM> may obtain information for moving the face region of a user based on a difference between a 3D coordinate corresponding to the face region of the user included in a photographed image and the stored reference 3D coordinate information, and provide a guide GUI <NUM> for moving the face region of the user based on the obtained information.

Here, the information for moving the face region of the user may include information on the moving distance by which the face region of the user is to be moved, the moving direction, the rotating direction and the degree of rotation, according to the difference between the obtained 3D coordinate and the reference 3D coordinate information. Also, the information for moving the face region of the user may be 3D coordinate information corresponding to the difference between the obtained 3D coordinate and the reference 3D coordinate information.

The electronic apparatus <NUM> may provide the display <NUM> with the guide GUI <NUM> that guides movement of the face region of a user while maintaining the photographing position of the camera <NUM>.

Here, the guide GUI <NUM> that corresponds to the face of a user may be in the form wherein the face region of the user photographed is virtually moved and displayed. As an example, as illustrated in <FIG>, the electronic apparatus <NUM> may provide a guide GUI in the form wherein the face region of a user photographed is displayed in a target position with dotted lines. Also, the guide GUI <NUM> that corresponds to the face of a user may be displayed in the form of a circle or an oval. However, the disclosure is not limited thereto, and the guide GUI <NUM> can be implemented in various forms if it is a component that performs the role of guiding movement of a user's body.

<FIG> is a diagram for illustrating guide GUIs in case the electronic apparatus <NUM> is implemented as AR glass.

In a case in which the electronic apparatus <NUM> is implemented as AR glass, the AR glass may obtain position adjustment information for adjusting the photographing position of a smart phone based on a difference between a 3D coordinate corresponding to the face region of a user included in a photographed image and the stored reference 3D coordinate information. Further, the AR glass may provide a guide GUI <NUM> for adjusting the position of the smart phone based on the obtained position adjustment information to the glass (display).

That is, the AR glass may provide the target position of a smart phone based on position adjustment information to the glass, and thereby guide the position.

Also, the AR glass may provide a guide GUI <NUM> in the shape of an arrow and thereby guide the direction and the distance by which the smart phone is to be moved.

<FIG> is a diagram for illustrating a guide GUI which is in proportion to the degree of movement of a camera according to an example embodiment.

<FIG> is a diagram wherein an arrow is provided as a guide GUI, and the electronic apparatus <NUM> may display the moving distance of the camera <NUM> by adjusting the length of the arrow. That is, the electronic apparatus <NUM> may display the length of the arrow to be in proportion to the moving distance of the camera <NUM>. For example, when the electronic apparatus <NUM> determines that the camera <NUM> needs to be moved to the right side by <NUM> based on the obtained position adjustment information, the electronic apparatus <NUM> may display a rightwards arrow having the length of <NUM> on the display <NUM>. Afterwards, when the camera <NUM> is moved to the right side by <NUM>, the electronic apparatus <NUM> may adjust the length of the rightwards arrow to <NUM>. The length of the rightwards arrow may gradually decrease as the camera <NUM> moves closer to a target position, and may gradually increase as the camera <NUM> moves away from the target position. In a case where the target position is located on the left side of the electronic apparatus <NUM>, the electronic apparatus <NUM> may display a leftwards arrow instead of the rightwards arrow.

Also, the electronic apparatus <NUM> may adjust the length of the arrow to display the degree of rotation of the camera <NUM> as well as the moving distance.

Meanwhile, the numerical values with respect to the moving distance of the camera <NUM> or the moving distance of the arrow are merely examples, and the disclosure is obviously not limited thereto.

<FIG> is a diagram for illustrating adjustment of a sticker that is added to a user image, according to an example embodiment.

A sticker may be all objects added to a user image to decorate the user image included in a photographed image.

According to <FIG>, a sticker may be implemented in a form wherein a shape in the form of a rabbit's ears <NUM> is added to the face region of a user.

The electronic apparatus <NUM> may adjust at least one of the size, direction or form of a sticker to be added to a user image based on a 3D coordinate corresponding to the face region of the user included in a photographed image. Here, as various embodiments of obtaining a 3D coordinate corresponding to the face region of a user were described earlier, detailed description therefor will be omitted.

As illustrated in image (a) of <FIG>, when the user looks at the left, the electronic apparatus <NUM> may obtain a 3D coordinate corresponding thereto. Then, based on the obtained 3D coordinate, the electronic apparatus <NUM> may adjust the sizes of the rabbit ears <NUM> on the left and right sides. As an example, the electronic apparatus <NUM> may express a cubic effect <NUM> by adjusting the size of the rabbit ear <NUM> on the right side to be bigger than the size of the rabbit ear <NUM> on the left side.

Afterwards, if the user looks at the right by changing his or her posture, the user may not be able to obtain a precise 3D coordinate according to his or her movement, even though the user's posture has been changed, and therefore in related art, the size and direction of the sticker may not be adjusted to correspond to the changed posture, as illustrated in image (b) of <FIG>.

However, according to an example embodiment, a 3D coordinate corresponding to the face region of a user included in a photographed image can be obtained in real time. Thus, as illustrated in image (c) of <FIG>, the electronic apparatus <NUM> may adjust the size and direction of the sticker to correspond to the user image. For example, as illustrated in image (c) of <FIG>, when the user looks at the left, the electronic apparatus <NUM> may express a cubic effect <NUM> by adjusting the size of the rabbit ear <NUM> on the left side to be bigger than the size of the rabbit ear <NUM> on the right side.

As described above, according to an example embodiment, a sticker is applied based on a 3D coordinate. Thus, a sticker can be applied to a user image while being adjusted relatively precisely.

<FIG> illustrates an embodiment wherein a shape in the form of a rabbit's ears <NUM> is added to the face region of a user image. However, various stickers may be applied based on a 3D coordinate, for example, a form wherein a cosmetic effect is applied to a user image.

<FIG> is a sequence diagram for illustrating an operation of guiding the movement of a camera by comparing 3D coordinates according to an example embodiment.

The electronic apparatus <NUM> may store a reference image in operation S1010. Here, the reference image may be at least one of an image selected by a user or an image provided by the electronic apparatus <NUM>, or it may be a target image aimed by an image photographed by a user.

Also, the electronic apparatus <NUM> may obtain a 3D coordinate from a reference image, and store the information on the 3D coordinate in operation S1020. Here, reference 3D coordinate information is information on a 3D coordinate corresponding to the face region included in the reference image. Meanwhile, the electronic apparatus <NUM> may not store a reference image separately, but store only the reference 3D coordinate information obtained from the reference image.

The electronic apparatus <NUM> may photograph an image including the face region of a user in operation S1030. Further, the electronic apparatus <NUM> may obtain a 3D coordinate corresponding to the face region of the user from the photographed image in operation S1040. Here, the photographed image may be an image in a preview state where the face region of a user can be checked. As various embodiments of obtaining a 3D coordinate corresponding to the face region of a user were described earlier, detailed description therefor will be omitted.

The electronic apparatus <NUM> may identify a difference between two 3D coordinates in operation S1050. The electronic apparatus <NUM> may obtain position adjustment information for adjusting the photographing position of the camera <NUM> based on a difference between a 3D coordinate corresponding to the face region of the user included in a photographed image and the stored reference 3D coordinate information, in operation S1060.

Here, position adjustment information may include information on the moving distance of the camera <NUM> by which the camera <NUM> is to be moved, the moving direction of the camera <NUM>, the rotating direction of the camera <NUM> and the degree of rotation of the camera <NUM>, according to the difference between the obtained 3D coordinate and the reference 3D coordinate information. Further, the position adjustment information may be 3D coordinate information corresponding to the difference between the obtained 3D coordinate and the reference 3D coordinate information.

Meanwhile, the electronic apparatus <NUM> may provide a guide GUI for adjusting the photographing position of the camera <NUM> based on the obtained position adjustment information in operation S1070. The electronic apparatus <NUM> may determine a movement direction of the electronic apparatus <NUM> in which the obtained 3D coordinate and the reference 3D coordinate information decreases, and may generate the guide GUI to indicate the movement direction. Here, the guide GUI may be a sign including at least one of the moving distance of the camera <NUM>, the moving direction of the camera <NUM>, the rotating direction of the camera <NUM> or the degree of rotation of the camera <NUM>. For example, the guide GUI may be provided as a sign in the shape of an arrow that indicates directions for displaying the moving direction and the rotating direction. Also, the guide GUI may be provided as a sign in the shape of an arrow that indicates a clockwise direction or a counterclockwise direction for displaying the rotating direction. The guide GUI may also be displayed on XYZ axes.

The photographing position of the camera <NUM> may be adjusted based on a user's manipulation in operation S1080. As the photographing position of the camera <NUM> is changed as described above, a photographed image in a preview state may be changed.

When the photographing position of the camera <NUM> is adjusted to a target position displayed by the guide GUI, the electronic apparatus <NUM> may perform photographing automatically.

<FIG> is a flow chart for illustrating a method of controlling an electronic apparatus according to an example embodiment.

The electronic apparatus <NUM> may obtain a 3D coordinate corresponding to the face region of a user included in an image photographed through the camera <NUM>, in operation S1110.

According to an example embodiment, the electronic apparatus <NUM> may identify a plurality of feature points from the face region of a user, calculate information on the distance among the feature points based on the number of pixels, and obtain a 3D coordinate based on the calculated distance information. Here, the feature points may be at least one of the eye, nose or mouth of the user.

According to another example embodiment, the electronic apparatus <NUM> may, based on receiving from an external apparatus included in a photographed image the position information of the external apparatus, obtain first distance information between the electronic apparatus and the external apparatus based on the received position information, obtain second distance information between the external apparatus included in the photographed image and at least one feature point included in the face region of the user, and obtain a 3D coordinate based on the first distance information and the second distance information. Here, the external apparatus may be an apparatus provided with a sensor that can calculate distance from the electronic apparatus <NUM>. Meanwhile, position information may be information including at least one of the distance between the electronic apparatus <NUM> and the external apparatus or the angle. Also, a sensor may be implemented as an IR sensor, an accelerometer, an angular velocity sensor, a gyro sensor, a hall sensor, an ultrasonic sensor and the like.

According to still another example embodiment, the electronic apparatus <NUM> may, based on receiving from an external electronic apparatus an image including the electronic apparatus <NUM>, compare the size of the electronic apparatus <NUM> obtained from the received image with the actual size of the pre-stored electronic apparatus <NUM>, and thereby obtain a 3D coordinate. Here, the external electronic apparatus may be implemented as AR glass including a camera. Also, the sizes may be information including the horizontal length, longitudinal length and thickness of the electronic apparatus <NUM>.

The electronic apparatus <NUM> may calculate the distance between the electronic apparatus <NUM> and the face region of the user by comparing the size of the electronic apparatus <NUM> calculated from an image received from an external electronic apparatus with the actual size of the electronic apparatus <NUM>. Also, the electronic apparatus <NUM> may calculate the size of the electronic apparatus <NUM> from the received image based on the number of pixels.

Further, the electronic apparatus <NUM> may obtain position adjustment information for adjusting the photographing position of the camera <NUM> based on a difference between the obtained 3D coordinate and reference 3D coordinate information in operation S1120. Here, the photographing position of the camera <NUM> may include at least one of the photographing distance between the face region of the user and the camera <NUM>, the photographing direction or the photographing angle.

Also, the electronic apparatus <NUM> may provide a guide GUI for adjusting the photographing position of the camera <NUM> based on the obtained position adjustment information in operation S1130. Here, the guide GUI may include at least one of the moving distance of the camera <NUM>, the moving direction of the camera <NUM>, the rotating direction of the camera <NUM> or the degree of rotation of the camera <NUM>.

According to another example embodiment, the electronic apparatus <NUM> may obtain information for moving the face region of a user based on a difference between a 3D coordinate corresponding to the face region of a user included in a photographed image and reference 3D coordinate information, and provide a guide GUI for moving the face region of the user based on the obtained information.

According to still another example embodiment, the electronic apparatus <NUM> may provide a guide for adjusting the photographing position of the camera <NUM> or moving the face region of a user as voice. For example, the electronic apparatus <NUM> may provide voice such as "Move the camera to the right side by <NUM>" or "Tilt your face to the left side by <NUM> degrees" through a speaker, based on the obtained position adjustment information or information for moving the face region of a user.

As detailed operations in each step were described earlier, detailed description therefor will be omitted.

Meanwhile, at least some of the components used in the methods according to the various embodiments of the disclosure described above may be installed on conventional electronic apparatuses, or may be implemented in the form of an application which is software that a user will directly use in an operating system (OS). An application may be provided in the form of an icon interface on the screen of the electronic apparatus <NUM>.

Also, at least some of the components in the various embodiments of the disclosure described above may be implemented through an embedded server provided to an electronic apparatus, or through an external server of an electronic apparatus.

Meanwhile, the various embodiments of the disclosure described above may be implemented in a recording medium that is readable by a computer or a device similar thereto, by using software, hardware or a combination thereof. In some cases, the embodiments described in this specification may be implemented as a processor itself. According to implementation by software, the embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described in this specification.

While not restricted thereto, an exemplary embodiment can be embodied as computer-readable code on a computer-readable recording medium. The computer-readable recording medium is any data storage device that can store data that can be thereafter read by a computer system. Examples of the computer-readable recording medium include read-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. The computer-readable recording medium can also be distributed over network-coupled computer systems so that the computer-readable code is stored and executed in a distributed fashion. Also, an exemplary embodiment may be written as a computer program transmitted over a computer-readable transmission medium, such as a carrier wave, and received and implemented in general-use or special-purpose digital computers that execute the programs. Moreover, it is understood that in exemplary embodiments, one or more units of the above-described apparatuses and devices can include circuitry, a processor, a microprocessor, etc., and may execute a computer program stored in a computer-readable medium.

In case instructions are executed by the processor of an electronic apparatus, a non-transitory computer-readable medium may store computer instructions that make the electronic apparatus perform an operation of obtaining a 3D coordinate corresponding to the face region of a user included in an image photographed by a camera, an operation of obtaining position adjustment information for adjusting the photographing position of the camera based on a difference between the obtained 3D coordinate and reference 3D coordinate information, and an operation of providing a guide GUI for adjusting the photographing position of the camera based on the obtained position adjustment information.

Claim 1:
Augmented reality, AR, glasses, the AR glasses comprising:
a display;
a communication interface for communicating with an electronic apparatus;
a storage; and
a processor configured to execute instructions stored in the storage at least to:
obtain an image, from the electronic apparatus via the communication interface, including the AR glasses worn by a user, wherein the image is obtained by a camera of the electronic apparatus;
characterized in that the processor is configured to:
obtain a three dimensional, 3D, coordinate of a face region of the user based on the obtained image;
obtain position adjustment information for adjusting a photographing position of the electronic apparatus based on a difference between the 3D coordinate of the face region and a reference 3D coordinate stored in the storage; and
control the display to display a guide graphic user interface, GUI, for adjusting the photographing position of the electronic apparatus based on the position adjustment information,
wherein the guide GUI is represented by a virtual object and includes at least one of a rotating direction of the electronic apparatus and a degree of rotation of the electronic apparatus, and
wherein the guide GUI includes a sign in a shape of an arrow that indicates a clockwise direction or a counterclockwise direction for displaying the rotating direction.