Patent Description:
An electronic device may provide an augmented reality (AR) service by overlaying information related to a preview image on the preview image, while displaying the preview image obtained through a camera. Such an electronic device may provide the AR service by overlaying coupon information, product information, object information, place information, food information associated with the preview image, or a combination thereof on the preview image.

<CIT> discloses a camera capable of displaying reference image candidates comprising the same or a similar content as a preview image as thumbnail overlays on the preview image.

<CIT> discloses a camera displaying a preview image on a display and providing thumbnail images as overlays, the thumbnail images showing the preview images with different photographing information being applied.

<CIT> discloses a camera that guides the user to obtain an image similar to a proposed sample image of the same theme as a preview image.

<CIT> pertains to a camera device that allows the user to cut out a region of interest from a first image and combine the cutout with a second image.

<CIT> describes a mobile terminal where, when a photo is taken using the terminal, a phototaker can be included in the taken photo in a manner of being synthesized with the corresponding photo.

When an electronic device provides an AR service within a dedicated application (for example, Bixby vision) or a specific application, the electronic device may display information according to the AR service on a screen of a display. When there are large amounts of information associated with a preview image, the electronic device may display information that is not desired by a user on the display. Accordingly, the user may feel inconvenience in using the AR service of the electronic device. Therefore, there may be a need for a solution to select information desired by a user and to overlay the information on a preview image in an electronic device.

The technical objects to be achieved by the disclosure are not limited to those mentioned above, and other technical objects that are not mentioned above may be clearly understood to those skilled in the art based on the description provided below.

An electronic device according to various embodiments may include a camera, a memory, a display, and a processor, and the processor may be configured as specified in claim <NUM>.

An operating method of an electronic device according to various embodiments may include: the steps specified in claim <NUM>.

According to an embodiment, the antenna module <NUM> may include an antenna including a radiating element composed of a conductive material or a conductive pattern formed in or on a substrate (e.g., printed circuit board (PCB)).

The electronic devices may include, for example, and without limitation, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, a home appliance, or the like.

It is to be understood that if an element (e.g., a first element) is referred to, with or without the term "operatively" or "communicatively", as "coupled with," "coupled to," "connected with," or "connected to" another element (e.g., a second element), the element may be coupled with the other element directly (e.g., wiredly), wirelessly, or via a third element.

Wherein, the "non-transitory" storage medium is a tangible device, and may not include a signal (e.g., an electromagnetic wave), but this term does not differentiate between where data is semi-permanently stored in the storage medium and where the data is temporarily stored in the storage medium.

<FIG> is a block diagram <NUM> illustrating the program <NUM> according to various embodiments. According to an embodiment, the program <NUM> may include an operating system (OS) <NUM> to control one or more resources of the electronic device <NUM>, middleware <NUM>, or an application <NUM> executable in the OS <NUM>. The OS <NUM> may include, for example, Android™, iOS™, Windows™, Symbian™, Tizen™, or Bada™. At least part of the program <NUM>, for example, may be pre-loaded on the electronic device <NUM> during manufacture, or may be downloaded from or updated by an external electronic device (e.g., the electronic device <NUM> or <NUM>, or the server <NUM>) during use by a user.

The OS <NUM> may control management (e.g., allocating or deallocation) of one or more system resources (e.g., process, memory, or power source) of the electronic device <NUM>. The OS <NUM>, additionally or alternatively, may include one or more driver programs to drive other hardware devices of the electronic device <NUM>, for example, the input device <NUM>, the sound output device <NUM>, the display device <NUM>, the audio module <NUM>, the sensor module <NUM>, the interface <NUM>, the haptic module <NUM>, the camera module <NUM>, the power management module <NUM>, the battery <NUM>, the communication module <NUM>, the subscriber identification module <NUM>, or the antenna module <NUM>.

The middleware <NUM> may provide various functions to the application <NUM> such that a function or information provided from one or more resources of the electronic device <NUM> may be used by the application <NUM>. The middleware <NUM> may include, for example, an application manager <NUM>, a window manager <NUM>, a multimedia manager <NUM>, a resource manager <NUM>, a power manager <NUM>, a database manager <NUM>, a package manager <NUM>, a connectivity manager <NUM>, a notification manager <NUM>, a location manager <NUM>, a graphic manager <NUM>, a security manager <NUM>, a telephony manager <NUM>, or a voice recognition manager <NUM>.

The application manager <NUM>, for example, may manage the life cycle of the application <NUM>. The window manager <NUM>, for example, may manage one or more graphical user interface (GUI) resources that are used on a screen. The multimedia manager <NUM>, for example, may identify one or more formats to be used to play media files, and may encode or decode a corresponding one of the media files using a codec appropriate for a corresponding format selected from the one or more formats. The resource manager <NUM>, for example, may manage the source code of the application <NUM> or a memory space of the memory <NUM>. The power manager <NUM>, for example, may manage the capacity, temperature, or power of the battery <NUM>, and determine or provide related information to be used for the operation of the electronic device <NUM> based at least in part on corresponding information of the capacity, temperature, or power of the battery <NUM>. According to an embodiment, the power manager <NUM> may interwork with a basic input/output system (BIOS) of the electronic device <NUM>.

The database manager <NUM>, for example, may generate, search, or change a database to be used by the application <NUM>. The package manager <NUM>, for example, may manage installation or update of an application that is distributed in the form of a package file. The connectivity manager <NUM>, for example, may manage a wireless connection or a direct connection between the electronic device <NUM> and the external electronic device. The notification manager <NUM>, for example, may provide a function to notify a user of an occurrence of a specified event (e.g., an incoming call, message, or alert). The location manager <NUM>, for example, may manage locational information on the electronic device <NUM>. The graphic manager <NUM>, for example, may manage one or more graphic effects to be offered to a user or a user interface related to the one or more graphic effects.

The security manager <NUM>, for example, may provide system security or user authentication. The telephony manager <NUM>, for example, may manage a voice call function or a video call function provided by the electronic device <NUM>. The voice recognition manager <NUM>, for example, may transmit a user's voice data to the server <NUM>, and receive, from the server <NUM>, a command corresponding to a function to be executed on the electronic device <NUM> based at least in part on the voice data, or text data converted based at least in part on the voice data. According to an embodiment, the middleware <NUM> may dynamically delete some existing components or add new components. According to an embodiment, at least part of the middleware <NUM> may be included as part of the OS <NUM> or may be implemented as another software separate from the OS <NUM>.

The application <NUM> may include, for example, a home <NUM>, dialer <NUM>, short message service (SMS)/multimedia messaging service (MMS) <NUM>, instant message (IM) <NUM>, browser <NUM>, camera <NUM>, alarm <NUM>, contact <NUM>, voice recognition <NUM>, email <NUM>, calendar <NUM>, media player <NUM>, album <NUM>, watch <NUM>, health <NUM> (e.g., for measuring the degree of workout or biometric information, such as blood sugar), or environmental information <NUM> (e.g., for measuring air pressure, humidity, or temperature information) application. According to an embodiment, the application <NUM> may further include an information exchanging application that is capable of supporting information exchange between the electronic device <NUM> and the external electronic device. The information exchange application, for example, may include a notification relay application adapted to transfer designated information (e.g., a call, message, or alert) to the external electronic device or a device management application adapted to manage the external electronic device. The notification relay application may transfer notification information corresponding to an occurrence of a specified event (e.g., receipt of an email) at another application (e.g., the email application <NUM>) of the electronic device <NUM> to the external electronic device. Additionally or alternatively, the notification relay application may receive notification information from the external electronic device and provide the notification information to a user of the electronic device <NUM>.

The device management application may control the power (e.g., turn-on or turn-off) or the function (e.g., adjustment of brightness, resolution, or focus) of the external electronic device or some component thereof (e.g., a display device or a camera module of the external electronic device). The device management application, additionally or alternatively, may support installation, delete, or update of an application running on the external electronic device.

<FIG> is a view illustrating an example of a functional configuration <NUM> of an electronic device (for example, the electronic device <NUM> of <FIG>) according to various embodiments. The functional configuration of <FIG> will be described with reference to the configuration of the electronic device <NUM> of <FIG>.

Referring to <FIG>, in various embodiments, a program <NUM> may include a plurality of applications <NUM>, <NUM> or <NUM>, an image analysis module <NUM>, an interaction management module <NUM>, an image information management module <NUM>, a user preference generation module <NUM>, an image recommendation module <NUM>, a database <NUM>, or a combination thereof. In various embodiments, the database <NUM> may include an image information database <NUM>, an interaction database <NUM>, a user preference database <NUM>, or a combination thereof. In various embodiments, the program <NUM> may correspond to the program <NUM> of <FIG> or <FIG>. In various embodiments, the program <NUM> may be used by the processor <NUM>.

In various embodiments, the plurality of applications <NUM>, <NUM>, or <NUM> may include an image shooting application, an image viewing application, an image search application, or a combination thereof. However, this is merely an example and types of the plurality of applications <NUM>, <NUM>, or <NUM> are not limited thereto.

In various embodiments, the plurality of applications <NUM>, <NUM>, or <NUM> may drive the camera module <NUM>. In various embodiments, the plurality of applications <NUM>, <NUM>, or <NUM> may display a preview image obtained through the camera module <NUM> on a display device (for example, the display device <NUM> of <FIG>). In various embodiments, the plurality of applications <NUM>, <NUM>, or <NUM> may display an object for shooting an image on the display device <NUM> along with the preview image. In various embodiments, the plurality of applications <NUM>, <NUM>, or <NUM> may store the currently displayed preview image in a memory <NUM>, a server <NUM>, or a combination thereof as a new image, based on an input on the object. In various embodiments, when the new image is stored, the plurality of applications <NUM>, <NUM>, or <NUM> may provide information regarding the newly stored image to the image information management module <NUM>. In various embodiments, the information regarding the newly stored image may include an identifier (for example, a name), a creation date, a shooting place, a storage path of the newly stored image, or a combination thereof.

In various embodiments, the plurality of applications <NUM>, <NUM>, or <NUM> may display an image stored in the memory <NUM>, the server <NUM>, or a combination thereof on the display device <NUM>. In various embodiments, the plurality of applications <NUM>, <NUM>, or <NUM> may display a thumbnail regarding the image on the display device <NUM> in the form of a list. In various embodiments, a user may view the image through the plurality of applications <NUM>, <NUM>, or <NUM>. In various embodiments, the plurality of applications <NUM>, <NUM>, or <NUM> may tag an object included in the image. In various embodiments, the plurality of applications <NUM>, <NUM>, or <NUM> may tag the object included in the image based on a user input. In various embodiments, the plurality of applications <NUM>, <NUM>, or <NUM> may share (or transmit) the image with (or to) another electronic device (for example, the electronic device <NUM> of <FIG>).

In various embodiments, when viewing, tagging, sharing of the image, or a combination thereof is identified, the plurality of applications <NUM>, <NUM>, or <NUM> may provide information generated regarding the image to the image information management module <NUM>. In various embodiments, the information generated regarding the image may include the number of times of viewing the image, tag information, the number of times of sharing the image, sharer information, or a combination thereof.

In various embodiments, the image analysis module <NUM> may include a face recognition engine <NUM>, a thing recognition engine <NUM>, a location tag generation engine <NUM>, a story creation engine <NUM>, or a combination thereof.

In various embodiments, the image analysis module <NUM> may analyze an image stored in the memory <NUM>, the server <NUM>, or a combination thereof. In various embodiments, the image analysis module <NUM> may extract data by analyzing the image. In various embodiments, the image analysis module <NUM> may provide the data extracted by analyzing the image to the image information management module <NUM>. In various embodiments, the data extracted through the image analysis module <NUM> may be included in image information. In various embodiments, the data extracted through the image analysis module <NUM> may include object recognition information, location information, story information, or a combination thereof. In various embodiments, the object recognition information may include person recognition information, thing recognition information, or a combination thereof. In various embodiments, the story information may be information indicating a category regarding an image. In various embodiments, the story information of an image may be set based on a similar image of the image (for example, an image shot at the same (or similar) date, an image shot in the same (or similar) location, an image including the same (or similar) person, or a combination thereof). In various embodiments, the story information of the image may be set to one piece of story information from among pre-set pieces of story information. In various embodiments, the story information of the image may be set to story information designated by the user.

In various embodiments, respective images shot at the same (or similar) date may have different pieces of time-based story information according to a shooting pattern of the user according to a shooting time interval between the images. In various embodiments, the images shot at the same date may have the same story information, based on a user's shooting pattern (for example, shooting in the same (or similar) location, shooting the same (or similar) person, or a combination thereof). In various embodiments, the images shot at the same date may have different story information based on a user's shooting pattern. In various embodiments, images shot at different dates may have the same story information (for example, "travel"), based on a user's shooting pattern (for example, shooting in the same (or similar) location, shooting the same (or similar) person, or a combination thereof).

In various embodiments, the face recognition engine <NUM> of the image analysis module <NUM> may extract person recognition information from the image. In various embodiments, the face recognition engine <NUM> may recognize an object indicating a person (for example, a person's face shape) from among objects included in the image. In various embodiments, the face recognition engine <NUM> may identify information of the person indicated by the recognized object. In various embodiments, the face recognition engine <NUM> may identify information of the person indicated by the recognized object, based on face tag information regarding images that is stored in the memory <NUM>, the server <NUM>, or a combination thereof. In various embodiments, when the information of the person indicated by the recognized object is not identified, the face recognition engine <NUM> may determine that the recognized object identifies information of an unknown person. In various embodiments, the face recognition engine <NUM> may provide the person recognition information to the image information management module <NUM>.

In various embodiments, the thing recognition engine <NUM> of the image analysis module <NUM> may extract thing recognition information from the image. In various embodiments, the thing recognition engine <NUM> may recognize an object indicating a thing (for example, a building, an article, a text, or a combination thereof) from among objects included in the image. In various embodiments, the thing recognition engine <NUM> may identify information of the thing indicated by the recognized object. In various embodiments, the thing recognition engine <NUM> may identify information of the thing indicated by the recognized object through an image search application. In various embodiments, the thing recognition engine <NUM> may provide the thing recognition information to the image information management module <NUM>.

In various embodiments, the location tag generation engine <NUM> of the image analysis module <NUM> may identify location information regarding a place (for example, an address, a geographical location, a point of interest (POI), or a combination thereof) where the electronic device (for example, the electronic device <NUM> of <FIG>) is located while the image is being obtained, based on a shooting location (for example, GPS) of the image. In various embodiments, the location tag generation engine <NUM> may provide the location information to the image information management module <NUM>.

In various embodiments, the story creation engine <NUM> of the image analysis module <NUM> may create story information of the image. In various embodiments, the story creation engine <NUM> may create the story information based on the person recognition information, the thing recognition information, the location information of the image, or a combination thereof. In various embodiments, the story information may include a story type, a story character, a story place, or a combination thereof.

In various embodiments, the story creation engine <NUM> may set a story type of the image to one story type from among a plurality of set story types. In various embodiments, the story creation engine <NUM> may set the story type of the image based on the corresponding image, and person recognition information, thing recognition information, location information of an image associated with the corresponding image, or a combination thereof. In various embodiments, the story creation engine <NUM> may set the story character of the image based on the person recognition information of the image. In various embodiments, the story creation engine <NUM> may set the story place of the image based on the location information of the image.

In various embodiments, the interaction management module <NUM> may collect information according to a user input. In various embodiments, the interaction management module <NUM> may collect information according to a user input on the plurality of applications <NUM>, <NUM>, or <NUM>. In various embodiments, when the user inputs a user input for viewing an image in the image viewing application, the interaction management module <NUM> may collect information indicating that the corresponding image is viewed. In various embodiments, when the user inputs a user input for tagging an unknown object of the viewed image in the image viewing application, the interaction management module <NUM> may collect information indicating that the unknown object of the corresponding image is tagged. In various embodiments, when the user inputs a user input for sharing the viewed image in the image viewing application, the interaction management module <NUM> may collect information indicating that the corresponding image is shared. In various embodiments, when the user inputs a user input for registering (or unregistering) a specific location (or place) of a map in a map application, the interaction management module <NUM> may collect information indicating that the corresponding location (or place) is registered (or unregistered). In various embodiments, the unknown object may be an object to which no tag is attached.

In various embodiments, the interaction management module <NUM> may transmit the collected information to the user preference generation module <NUM>. In various embodiments, the interaction management module <NUM> may provide the collected information to the image information management module <NUM>. In various embodiments, the interaction management module <NUM> may provide the information indicating that the corresponding image is viewed, the information indicating that the unknown object of the corresponding image is tagged, the information indicating that the corresponding image is shared, or a combination thereof to the image information management module <NUM>.

In various embodiments, the interaction management module <NUM> may refine the interaction database <NUM> of the database <NUM>, based on the collected information. In various embodiments, the interaction management module <NUM> may refine a place wish list of the interaction database <NUM>, based on the information indicating that the corresponding location (or place) is registered (unregistered).

In various embodiments, the image information management module <NUM> may refine the image information database <NUM> of the database <NUM>, based on information (or data) received from the plurality of applications <NUM>, <NUM>, or <NUM> or at least one of other modules. In various embodiments, the image information management module <NUM> may manage information of the image in the image information database <NUM>. In various embodiments, the information of the image in the image information database <NUM> may include media information, face tag information, place information, story information, or a combination thereof. In various embodiments, the media information may include a date, a viewing time, the number of times of viewing, the number of times of sharing, or a combination thereof. In various embodiments, the face tag information may include face recognition, a person tag, a user tag, a contact profile, or a combination thereof. In various embodiments, the place information may include a place tag. In various embodiments, the story information may include a story type, a story character, a story place, or a combination thereof.

In various embodiments, the image information management module <NUM> may extract data stored in the image information database <NUM>, in response to a request of the plurality of applications <NUM>, <NUM>, or <NUM> or at least one of the other modules (for example, a database query), and may transmit the extracted data to the application or module which has requested the data.

In various embodiments, the user preference generation module <NUM> may estimate a user's preference for the image stored in the memory <NUM>, the server <NUM>, or a combination thereof. In various embodiments, the user preference generation module <NUM> may calculate a preference value for the image, and may estimate a user's preference based on the calculated preference value.

In various embodiments, the user preference generation module <NUM> may generate a graph indicating a relationship of data (for example, person information, thing information, location information, or a combination thereof), based on information which is collected by the interaction management module <NUM> according to a user input. In various embodiments, the user preference generation module <NUM> may refine the graph indicating the relationship of the data, based on information which is newly collected by the interaction management module <NUM> according to a user input.

In various embodiments, the graph may include a node and a branch. In various embodiments, the node may indicate an identifier of information included in the data (for example, an identifier of person information). In various embodiments, the branch may indicate a relationship between two nodes connected by the branch. In various embodiments, a value of the branch may indicate a strength of the relationship between the two nodes connected by the branch. In various embodiments, the value of the branch may correspond to the number of images in which identifiers of two nodes connected by the branch are simultaneously identified, from among the images stored in the memory <NUM>, the server <NUM>, or a combination thereof.

In various embodiments, the user preference generation module <NUM> may generate a graph indicating a relationship of person information, based on a face tag tagged to the image stored in the memory <NUM>, the server <NUM>, or a combination thereof. In various embodiments, the node of the graph indicating the relationship of the person information may indicate an identifier of the face tag. In various embodiments, a value of the branch of the graph indicating the relationship of the person information may correspond to the number of images in which face tags of two nodes connected by the branch are simultaneously identified, from among the images stored in the memory <NUM>, the server <NUM>, or a combination thereof.

In various embodiments, the user preference generation module <NUM> may estimate a user's preference for a certain image based on the generated graph. In various embodiments, the user preference generation module <NUM> may calculate a preference value for the certain image based on the generated graph, and may estimate a user's preference based on the calculated preference value.

In various embodiments, the user preference generation module <NUM> may identify a center node based on the generated graph. In various embodiments, the center node may be a node that has the highest sum of values of branches connected to the node from among the nodes of the graph. In various embodiments, the center node may be a node regarding an identifier indicating that a face tag indicates herself/himself.

In various embodiments, the user preference generation module <NUM> may estimate the user's preference for the certain image, based on path lengths from the center node to nodes regarding the certain image. In various embodiments, the user preference generation module <NUM> may estimate the user's preference for the certain image to be higher as an average (or sum) of the path lengths of the nodes of the certain image is smaller. In various embodiments, the user preference generation module <NUM> may calculate a preference value corresponding to the average (or sum) of the path lengths of the nodes of the certain image.

In various embodiments, the user preference generation module <NUM> may estimate the user's preference for the certain image, based on the information of the image managed by the image information management module <NUM> in the image information database <NUM>. In various embodiments, the user preference generation module <NUM> may estimate the user's preference for the certain image, based on a viewing time, the number of times of viewing, the number of times of sharing, or a combination thereof. In various embodiments, the user preference generation module <NUM> may estimate the user's preference for the image to be higher as a value according to the viewing time, the number of times of viewing, the number of times of sharing, or a combination thereof is higher. In various embodiments, the user preference generation module <NUM> may calculate a preference value corresponding to the value according to the viewing time of the certain image, the number of times of viewing, the number of times of sharing, or a combination thereof.

In various embodiments, the user preference generation module <NUM> may estimate the user's preference for the certain image, based on place information of the image managed in the image information database <NUM> and the place wish list of the interaction database <NUM>. In various embodiments, the user preference generation module <NUM> may estimate the user's preference according to whether place information of the certain image is included in the place wish list. In various embodiments, the user preference generation module <NUM> may estimate that a user's preference for an image the place information of which is included in the place wish list is higher than that for an image the place information of which is not included in the place wish list. In various embodiments, the user preference generation module <NUM> may calculate a preference value of the image the place information of which is included in the place wish list to be higher than a preference value of the image the place information of which is not included in the place wish list.

In various embodiments, the user preference generation module <NUM> may estimate the user's preference for the certain image, based on story information of the image managed in the image information database <NUM>. In various embodiments, the user preference generation module <NUM> may identify the number of images set for each of the plurality of story types. In various embodiments, the user preference generation module <NUM> may estimate the user's preference for the certain image, based on a story type set for the certain image. In various embodiments, the user preference generation module <NUM> may calculate a preference value regarding the certain image, according to the number of images set to have the same story type as the story type set for the certain image.

In various embodiments, the user preference generation module <NUM> may store (or refine) the estimated user's preference in the user preference database <NUM> of the database <NUM>.

In various embodiments, the image recommendation module <NUM> may select at least one image from among the images stored in the memory <NUM>, the server <NUM>, or a combination thereof, based on the user's preference. In various embodiments, the image recommendation module <NUM> may select at least one image in order of the user's preference from among the images stored in the memory <NUM>, the server <NUM>, or a combination thereof.

In various embodiments, when there are two or more user preferences, the image recommendation module <NUM> may select at least one image based on a weighted average of the two or more preferences. In various embodiments, a sum of weights applied to the two or more preferences may be greater than or equal to <NUM>.

In various embodiments, the image recommendation module <NUM> may extract an image corresponding to a current location of the electronic device <NUM> from among the images stored in the memory <NUM>, the server <NUM>, or a combination thereof. In various embodiments, the image recommendation module <NUM> may extract an image corresponding to a location indicated by a preview image obtained by the camera module <NUM> in a current location of the electronic device <NUM>, from among the images stored in the memory <NUM>, the server <NUM>, or a combination thereof. In various embodiments, the image recommendation module <NUM> may extract an image corresponding to a thing object included in the preview image from among the images stored in the memory <NUM>, the server <NUM>, or a combination thereof. In various embodiments, the image recommendation module <NUM> may select at least one image from among the extracted images, based on user's preferences for the extracted images.

<FIG> illustrates a flowchart <NUM> for overlaying an image selected by an electronic device (for example, the electronic device <NUM> of <FIG>) according to various embodiments on a preview image. <FIG> will be described with reference to the functional configuration of <FIG> or the configuration of the electronic device <NUM> of <FIG>.

Referring to <FIG>, in various embodiments, in operation <NUM>, a processor (for example, the processor <NUM> of <FIG>) may identify a location of an electronic device (for example, the electronic device <NUM> of <FIG>) when a pre-set event is identified. In various embodiments, the pre-set event may be an event in which an input for performing a pre-set function of an application is received. In various embodiments, the pre-set function may include a function for obtaining a preview image through a camera module (for example, the camera module <NUM> of <FIG>), a function for providing an AR mode, or a combination thereof. However, this is merely an example and the pre-set function is not limited thereto. In various embodiments, the pre-set event may be an event in which an input for displaying an image stored in the memory <NUM>, the server <NUM>, or a combination thereof on the display device <NUM> is received.

In various embodiments, the processor <NUM> may identify a current location of the electronic device <NUM> through a GPS signal received through a communication module (for example, the communication module <NUM> of <FIG>).

In various embodiments, in operation <NUM>, the processor <NUM> may extract an image based on the identified location. In various embodiments, the processor <NUM> may extract an image a shooting location of which is within a set distance from the identified location, from among the images stored in the memory <NUM>, the server <NUM>, or a combination thereof. In various embodiments, the processor <NUM> may extract images corresponding to the current location of the electronic device <NUM> as shown in table <NUM> presented below:.

Table <NUM> illustrates examples of images corresponding to the current location of the electronic device <NUM> that are extracted from among the images stored in the memory <NUM>, the server <NUM>, or a combination thereof. Table <NUM> illustrates examples of a list of extracted images and image information of the extracted images.

In various embodiments, in operation <NUM>, the processor <NUM> may select a preference image based on preferences for the extracted images. In various embodiments, the processor <NUM> may estimate user's preferences for the extracted images. In various embodiments, the user's preferences for the extracted images may be data having values.

In various embodiments, the processor <NUM> may estimate user's preferences for the extracted images, based on information collected according to user inputs. In various embodiments, the processor <NUM> may estimate user's preferences for the extracted images, based on face tags tagged to the images. In various embodiments, referring to table <NUM>, in the extracted images, there are four images in which the face tag "Eden" is set, three images in which the face tag "Mary" is set, one image in which the face tag "Wife" is set, three images in which the face tag "Lee Sin" is set, and three images in which the face tag "me" is set. Therefore, the processor <NUM> may estimate that the user's preference for the images in which the face tag "Eden" is set is highest from among the extracted images. In various embodiments, referring to table <NUM>, the processor <NUM> may estimate that the user's preference is high in order of the images in which the face tag "Eden" is set, the images in which the face tag "Mary" is set, the images in which the face tag "Lee Sin" is set, the images in which the face tag "me" is set, and the images in which the face tag "Wife" is set. In various embodiments, referring to table <NUM>, the processor <NUM> may estimate that the user's preference for the images in which the face tag "Mary" is set, the user's preference for the images in which the face tag "Lee Sin" is set, and the user's preference for the images in which the face tag "me" is set are the same.

In various embodiments, the processor <NUM> may estimate the user's preferences for the extracted images, based on image information of the extracted images.

In various embodiments, the processor <NUM> may estimate the user's preferences for the extracted images, based on the number of times of viewing of the extracted images. In various embodiments, referring to table <NUM>, the processor <NUM> may estimate that the image (AA2. jpg) which is viewed a large number of times has the highest user preference. In various embodiments, referring to table <NUM>, the processor <NUM> may estimate that the images (CC1. jpg) which are viewed a small number of times have the lowest user preference.

In various embodiments, the processor <NUM> may estimate the user's preferences for the extracted images, based on the number of times of sharing of the extracted images. In various embodiments, referring to table <NUM>, the processor <NUM> may estimate that the images (AA1. jpg) which are shared a large number of times have the highest user preference. In various embodiments, referring to table <NUM>, the processor <NUM> may estimate that the images (BB1. jpg) which are shared a small number of times have the lowest user preference.

In various embodiments, the processor <NUM> may estimate the user's preferences for the extracted images, based on place information of the images and the place wish list. In various embodiments, the processor <NUM> may estimate the user's preference for an image having place information included in the place wish list, from among the extracted images, to be high. In various embodiments, the processor <NUM> may estimate the user's preference for an image having place information that is not included in the place wish list, from among the extracted images, to be low. In various embodiments, when "A mall" and "C restaurant" are included in the place wish list, the processor <NUM> may estimate the user's preferences for images (AA1. jpg, AA3 jpg, AA4. jpg) having place information included in the place wish list, from among the extracted images as shown in table <NUM>, to be high. In various embodiments, when "A mall" and "C restaurant" are included in the place wish list, the processor <NUM> may estimate the user's preferences for images (AA5. jpg) having place information that is not included in the place wish list, from among the extracted images as shown in table <NUM>, to be low.

In various embodiments, the processor <NUM> may estimate the user's preferences for the extracted images, based on story information of the images.

In various embodiments, referring to table <NUM>, in the extracted images, there are five images in which the story type is set to "Baby," two images in which the story type is set to "Location," three images in which the story type is set to "Manual," and one image in which the story type is not set. Therefore, the processor <NUM> may estimate that the user's preference for the images in which the story type is set to "Baby" is highest from among the extracted images. In various embodiments, referring to table <NUM>, the processor <NUM> may estimate that the user's preference is high in order of the images in which the story type is set to "Baby," the images in which the story type is set to "Manual," the images in which the story type is set to "Location," and the images in which the story type is not set.

In various embodiments, the processor <NUM> may select at least one image from among the extracted images in order of the estimated user's preference. In various embodiments, when there are two or more user preferences, the processor <NUM> may select at least one image based on a weighted average of the two or more preferences. In various embodiments, the processor <NUM> may select at least one image (for example, images AA1. jpg to AA5. jpg) from among the extracted images.

In various embodiments, the processor <NUM> may set a selection suspension period for the selected image. In various embodiments, the processor <NUM> may select at least one image in order of the user's estimated preference from a higher one from among the extracted images, but may not select an image corresponding to the selection suspension period even if the image has a high preference. In various embodiments, the selection suspension period may be a period from a time at which a set time is elapsed after a certain image is selected until a pre-set selection suspension cancel time. In various embodiments, images may be selected after the selection suspension cancel time.

In various embodiments, the processor <NUM> may select a pre-set number of images from among the extracted images, based on the estimated user's preference. In various embodiments, when images selected based on the estimated user's preference exceed the pre-set number, the processor <NUM> may select the pre-set number of images based on context information. In various embodiments, when the selected images exceed the pre-set number, the processor <NUM> may select the pre-set number of images in order of the most recent date. In various embodiments, when the selected images exceed the pre-set number, the processor <NUM> may select the pre-set number of images by selecting an image shot on an anniversary in preference to the other images. In various embodiments, the anniversary may include a public holiday such as Christmas, Chuseok (Korean Thanksgiving Day), Korean New Year's Day, and a personal anniversary such as a birthday, a wedding anniversary set in a user's planner. In various embodiments, the most recent date, the anniversary, or a combination thereof may be included in the context information.

In various embodiments, in operation <NUM>, the processor <NUM> may display an image corresponding to the selected preference image by overlaying the same on the preview image. In various embodiments, the image corresponding to the selected preference image may be a thumbnail. In various embodiments, the image corresponding to the selected preference image may be an image indicating an object included in the preference image.

<FIG> is a flowchart <NUM> illustrating a process according to an input on an image overlaid on a preview image of an electronic device (for example, the electronic device <NUM> of <FIG>) according to various embodiments. <FIG> is a view illustrating an example <NUM> of a process according to an input on an image overlaid on a preview image of an electronic device (for example, the electronic device <NUM> of <FIG>). <FIG> and <FIG> will be described with reference to the functional configuration of <FIG> or the configuration of the electronic device <NUM> of <FIG>.

Referring to <FIG>, in various embodiments, a processor (for example, the processor <NUM> of <FIG>) may identify whether there is a user input on an image overlaid on a preview image in operation <NUM>. In various embodiments, the user input may be a touch input on the overlaid image. In various embodiments, the overlaid image may be an image (for example, a thumbnail) corresponding to a selected preference image. In various embodiments, the selected preference image may be the image selected in operation <NUM> of <FIG>.

Referring to <FIG>, in various embodiments, the processor <NUM> may obtain a preview image <NUM> of <FIG> by executing an application for driving a camera module (for example, the camera module <NUM> of <FIG>). In various embodiments, the processor <NUM> may display the preview image <NUM> which is obtained through a user interface provided by the application on the display device <NUM>. In various embodiments, the processor <NUM> may generate an image <NUM> to be overlaid on the preview image <NUM> of <FIG>, and an object <NUM> (for example, an image object, a text object) to be overlaid on the preview image <NUM>, in response to a user input on an executable object <NUM> of the user interface provided by the application. In various embodiments, the processor <NUM> may generate a screen <NUM> in which the image <NUM> and the object <NUM> are overlaid on the preview image <NUM>, and may display the screen <NUM> on a display device (for example, the display device <NUM> of <FIG>). In various embodiments, the processor <NUM> may identify a user input of selecting the image <NUM> overlaid on the preview image <NUM>. In various embodiments, the overlaid image <NUM> may include images (for example, thumbnails) corresponding to selected preference images.

In various embodiments, when the application for driving the camera module (for example, the camera module <NUM> of <FIG>) is executed, the processor <NUM> may perform the operations shown in <FIG>. In various embodiments, the processor <NUM> may perform the operations shown in <FIG> in response to a user input on the executable object <NUM> of the user interface provided by the application after the application for driving the camera module (for example, the camera module <NUM> of <FIG>) is executed.

In various embodiments, when a user input on the image overlaid on the preview image is identified ("Yes"), the processor <NUM> may perform operation <NUM>. In various embodiments, when the user input on the image overlaid on the preview image is not identified ("No"), the processor <NUM> may finish the process.

In various embodiments, in operation <NUM>, the processor <NUM> may convert into a screen displaying the selected preference image. In various embodiments, the selected preference image may be the image which is selected in operation <NUM> of <FIG>.

Referring to <FIG>, when the user input on the image <NUM> overlaid on the preview image <NUM> is identified, the processor <NUM> may convert from the screen <NUM> into a screen <NUM> according to a user interface which provides a function of viewing the selected preference images <NUM> to <NUM> of the application.

In various embodiments, the processor <NUM> may identify at least one other image shot out of a region of a set range from a location of the electronic device <NUM> from among the plurality of images, based on location information of the plurality of images, and may display the identified at least one other image on the display device <NUM> by overlaying the same on the preview image along with the thumbnail image corresponding to the preference image.

<FIG> is a flowchart <NUM> illustrating a process according to an input on an image overlaid on a preview image of an electronic device (for example, the electronic device <NUM> of <FIG>) according to various embodiments. <FIG> is a view illustrating an example <NUM> of a process according to an input on an image overlaid on a preview image of an electronic device (for example, the electronic device <NUM> of <FIG>) according to various embodiments. <FIG> and <FIG> will be described with reference to the functional configuration of <FIG>, or the configuration of the electronic device <NUM> of <FIG>.

Referring to <FIG>, in various embodiments, in operation <NUM>, a processor (for example, the processor <NUM> of <FIG>) may identify whether there is a user input on an image overlaid on a preview image. In various embodiments, the user input may be a touch input on the overlaid image. In various embodiments, the overlaid image may be an image (for example, a thumbnail) corresponding to a selected preference image. In various embodiments, the selected preference image may be the image which is selected in operation <NUM> of <FIG>.

Referring to <FIG>, in various embodiments, the processor <NUM> may obtain a preview image <NUM> of <FIG> by executing an application for driving a camera module (for example, the camera module <NUM> of <FIG>). In various embodiments, the processor <NUM> may display the preview image <NUM> obtained through a user interface provided by the application on the display device <NUM>. In various embodiments, the processor <NUM> may generate images <NUM> to <NUM> to be overlaid on the preview image <NUM> of <FIG>, in response to a user input on an executable object <NUM> of the user interface provided by the application. In various embodiments, the processor <NUM> may generate a screen <NUM> in which the images <NUM> to <NUM> to be overlaid are overlaid on a preview image <NUM>, and may display the screen <NUM> on the display device (for example, the display device <NUM> of <FIG>). In various embodiments, the processor <NUM> may identify a user input of selecting at least one image (for example, the image <NUM>) from among the images <NUM> to <NUM> overlaid on the preview image <NUM>. In various embodiments, the images <NUM> to <NUM> to be overlaid may be images (for example, thumbnails) corresponding to the selected preference images.

In various embodiments, in operation <NUM>, the processor <NUM> may extract an object included in a selected image corresponding to the selected image. In various embodiments, in response to a user input of selecting the image <NUM> from among the images <NUM> to <NUM> overlaid on the preview image <NUM>, the processor <NUM> may extract an object included in a selected preference image corresponding to the image <NUM> (for example, the preference image <NUM> corresponding to the image <NUM> from among the preference images <NUM> to <NUM> of <FIG>).

In various embodiments, in operation <NUM>, the processor <NUM> may display the extracted object by overlaying the object on the preview image.

In various embodiments, the processor <NUM> may extract an object <NUM> included in the selected preference image <NUM> corresponding to the user input. In various embodiments, the processor <NUM> may generate a screen <NUM> in which the extracted object <NUM> is overlaid on a preview image <NUM>, and may display the screen <NUM> on the display device <NUM>. In various embodiments, the processor <NUM> may obtain an image in which the extracted object <NUM> is overlaid on the preview image <NUM> based on a user input on an executable object <NUM> provided by the user interface of the application.

In various embodiments, the processor <NUM> may display the object included in the preference image on the display device <NUM> by overlaying the object to be adjacent to an object having the same type from among objects included in the preview image which is obtained through the camera module <NUM>, according to the type of the object (for example, at least one of a person, a building, or a thing) included in the preference image. In various embodiments, the processor <NUM> may determine whether the object included in the preference image <NUM> indicates a person. In various embodiments, when the object included in the preference image <NUM> indicates a person, the processor <NUM> may identify an object indicating a person from among objects included in the preview image obtained through the camera module <NUM>. In various embodiments, the processor <NUM> may display the person object included in the preference image <NUM> on the display device <NUM> by overlaying the person object to be adjacent to the object indicating the person, which is included in the preview image, while displaying the preview image obtained through the camera module <NUM> on the display device <NUM>.

<FIG> is a flowchart for refining image information of an image in an electronic device (for example, the electronic device <NUM> of <FIG>) according to various embodiments. <FIG> will be described with reference to the functional configuration of <FIG> or the configuration of the electronic device <NUM> of <FIG>.

Referring to <FIG>, in various embodiments, a processor (for example, the processor <NUM> of <FIG>) may identify a user input on an imagen in operation <NUM>. In various embodiments, the user input on the image may include an input for viewing (or displaying) an image through an application, an input for sharing an image with another electronic device (for example, the electronic device <NUM>), an input for tagging certain information to an objected included in the image, or a combination thereof.

In various embodiments, in operation <NUM>, the processor <NUM> may refine image information of the image based on a user input. In various embodiments, the processor <NUM> may refine a viewing time of the image, the number of times of viewing, the number of times of sharing, tag information, or a combination thereof, based on the user input.

The electronic device and the method thereof according to various embodiments may select information desired by a user and may overlay the information on a preview image. The electronic device and the method thereof according to various embodiments may select information desired by a user and overlay the information on a preview image, such that availability of an AR service of the user can be enhanced.

An electronic device according to various embodiments may include a camera, a memory, a display, and a processor, and the processor may be configured to: when an input for executing a camera application is received, identify first images which are shot within a region of a set range from a location of the electronic device from among a plurality of images, based on location information of the plurality of images stored in the memory or an external electronic device; select at least one second image from the first images, based on a priority of each of the first images; and, while displaying a preview image obtained through the camera on the display, display a visual object including a thumbnail image corresponding to the at least one second image on the display by overlaying the visual object on the preview image.

In various embodiments, the processor may be configured to: identify the priority of each of the first images, based on image information of each of the first images; and select the at least one second image, based on the priority of each of the first images.

In various embodiments, the processor may be configured to identify that a priority of at least one image in which place information included in the image information is included in a place wish list of a user of the electronic device from among the first images is higher than priorities of other images in which the place information included in the image information is not included in the place wish list of the user.

In various embodiments, the processor may be configured to: identify face tag information included in the image information of each of the first images; identify the number of face tags for each person set for the first images, based on the identified face tag information; and identify that a priority of an image including a face tag of a person who is most identified from among the persons is higher than priorities of images including face tags regarding other persons.

In various embodiments, the processor may be configured to: identify story type information included in the image information of each of the first images; identify a number of story types set for the first images, based on the identified story type information; and identify that a priority of an image set to have a story type that is most identified from among the story types is higher than priorities of images set to have other story types.

In various embodiments, the processor may be configured to: identify a number of times of viewing of the image included in the image information of each of the first images; and identify that a priority of an image which is viewed a large number of times is higher than a priority of an image which is viewed a small number of times.

In various embodiments, the processor may be configured to: receive a user input on the visual object; extract an object included in the at least one second image in response to the user input; and while displaying the preview image obtained through the camera on the display, display the object on the display by overlaying the object on the preview image.

In various embodiments, the processor may be configured to: when the object included in the at least one second object indicates a person, identify an object indicating a person from among objects included in the preview image obtained through the camera; and, while displaying the preview image obtained through the camera on the display, display the object included in the at least one second image on the display by overlaying the object to be adjacent to the object indicating the person included in the preview image.

In various embodiments, the processor may be configured to: receive a user input on the visual object; in response to the user input, generate a screen for displaying the at least one second image; and display the generated screen on the display.

In various embodiments, the processor may be configured to identify at least one other image shot out of a region of a set range from a location of the electronic device from among the plurality of images, based on location information of the plurality of images stored in the memory or the external electronic device, and, while displaying the preview image obtained through the camera on the display, to display another visual object including the thumbnail image corresponding to the at least one second image and other thumbnail images corresponding to the at least one other image on the display by overlaying another visual object on the preview image.

In various embodiments, the processor may be configured to: identify face tag information included in the image information of each of the first images; identify a graph indicating a relationship of persons based on the identified face tag information; identify the number of face tags for each person set for the first images, based on face tag information regarding a person indicated by a center node of the graph; and identify that a priority of an image including a face tag of a person who is most identified from among the persons is higher than priorities of images including face tags regarding other persons.

In various embodiments, the processor may be configured to: when the number of the at least one second image exceeds a pre-set number, select a number of third images smaller than or equal to the pre-set number from among the at least one second image, based on context information; and, while displaying the preview image obtained through the camera on the display, display a visual object including a thumbnail image corresponding the number of third images smaller than or equal to the pre-set number on the display by overlaying the visual object on the preview image.

An operating method of an electronic device according to various embodiments may include: when an input for executing a camera application is received, identifying first images which are shot within a region of a set range from a location of the electronic device from among a plurality of images, based on location information of the plurality of images stored in a memory of the electronic device or an external electronic device; selecting at least one second image from the first images, based on a priority of each of the first images; and, while displaying a preview image obtained through a camera of the electronic device on a display of the electronic device, displaying a visual object including a thumbnail image corresponding to the at least one second image on the display by overlaying the visual object on the preview image.

In various embodiments, selecting the at least one second image may further include: identifying the priority of each of the first images, based on image information of each of the first images; and selecting the at least one second image, based on the priority of each of the first images.

In various embodiments, identifying the priority may include identifying that a priority of at least one image in which place information included in the image information is included in a place wish list of a user of the electronic device from among the first images is higher than priorities of other images in which the place information included in the image information is not included in the place wish list of the user.

In various embodiments, identifying the priority may include: identifying face tag information included in the image information of each of the first images; identifying the number of face tags for each person set for the first images, based on the identified face tag information; and identifying that a priority of an image including a face tag of a person who is most identified from among the persons is higher than priorities of images including face tags regarding other persons.

In various embodiments, identifying the priority may include: identifying story type information included in the image information of each of the first images; identifying a number of story types set for the first images, based on the identified story type information; and identifying that a priority of an image set to have a story type that is most identified from among the story types is higher than priorities of images set to have other story types.

In various embodiments, identifying the priority may include: identifying a number of times of viewing of the image included in the image information of each of the first images; and identifying that a priority of an image which is viewed a large number of times is higher than a priority of an image which is viewed a small number of times.

In various embodiments, the method may further include: receiving a user input on the visual object; extracting an object included in the at least one second image in response to the user input; and, while displaying the preview image obtained through the camera on the display, displaying the object on the display by overlaying the object on the preview image.

In various embodiments, the method may further include: when the object included in the at least one second object indicates a person, identifying an object indicating a person from among objects included in the preview image obtained through the camera; and, while displaying the preview image obtained through the camera on the display, displaying the object included in the at least one second image on the display by overlaying the object to be adjacent to the object indicating the person included in the preview image.

In various embodiments, the method may further include: receiving a user input on the visual object; in response to the user input, generating a screen for displaying the at least one second image; and displaying the generated screen on the display.

In various embodiments, the method may further include: identifying at least one other image shot out of a region of a set range from a location of the electronic device from among the plurality of images, based on location information of the plurality of images stored in the memory or the external electronic device; and, while displaying the preview image obtained through the camera on the display, displaying another visual object including the thumbnail image corresponding to the at least one second image and other thumbnail images corresponding to the at least one other image on the display by overlaying another visual object on the preview image.

In various embodiments, identifying the priority may include: identifying face tag information included in the image information of each of the first images; identifying a graph indicating a relationship of persons based on the identified face tag information; and identifying that a priority of an image including face tag information regarding a person indicated by a center node of the graph is higher than priorities of images including face tags regarding other persons.

In various embodiments, the method may further include: when the number of the at least one second image exceeds a pre-set number, selecting a number of third images smaller than or equal to the pre-set number from among the at least one second image, based on context information; and, while displaying the preview image obtained through the camera on the display, displaying a visual object including a thumbnail image corresponding the number of third images smaller than or equal to the pre-set number on the display by overlaying the visual object on the preview image.

The effects achieved by the disclosure are not limited to those mentioned above, and other effects that are not mentioned above may be clearly understood to those skilled in the art based on the description provided above.

Methods according to various example embodiments of the present disclosure may be implemented in hardware, software, or a combination of hardware and software.

The one or more program may include instructions that cause the electronic device to perform the methods according to various embodiments of the present disclosure as defined by the appended claims and/or disclosed herein.

Any combination of some or all of them may form a memory in which the program is stored.

In addition, the programs may be stored in an attachable storage device which is accessible through communication networks such as the Internet, Intranet, local area network (LAN), wide area network (WAN), and storage area network (SAN), or a combination thereof.

In the above-described example embodiments of the present disclosure, a component included in the present disclosure is expressed in the singular or the plural according to a presented example embodiment. However, the singular form or plural form is selected for convenience of description suitable for the presented situation, and various example embodiments of the present disclosure are not limited to a single element or multiple elements thereof. Further, either multiple elements expressed in the description may be configured into a single element or a single element in the description may be configured into multiple elements.

Claim 1:
An electronic device (<NUM>) comprising:
a camera (<NUM>);
a memory (<NUM>);
a display (<NUM>); and
a processor (<NUM>),
wherein the processor (<NUM>) is configured to:
based on an input for executing a camera application being received, while displaying a preview image obtained through the camera (<NUM>) on the display, identify a location of the electronic device corresponding to the preview image,
identify first images shot within a region of a set range from the location of the electronic device (<NUM>) from among a plurality of images based on location information of the plurality of images stored in the memory (<NUM>) or an external electronic device (<NUM>);
calculate a preference value respectively corresponding to the first images;
estimate, based on the preference value, a user's preferences for the first images, respectively, the preference value being based on face tags tagged to the images;
select at least one second image from the first images based on a priority of each of the first images determined by the user's preferences, wherein the at least one second image is shot within the region of the set range from the location of the electronic device (<NUM>); and
while displaying the preview image obtained through the camera (<NUM>) on the display (<NUM>), display a visual object comprising a thumbnail image corresponding to the at least one second image on the display (<NUM>) by overlaying the visual object on the preview image.