Electronic device having plurality of lenses where the device changes a visual object corresponding to a recommended lens

Disclosed is an electronic device comprising: a display which outputs a screen; a camera module including a plurality of lenses including a first lens; a processor; and a memory. The electronic device may output, through the display, a screen including a lens selection user interface, which includes a plurality of visual objects respectively corresponding to the plurality of lenses, and a first image obtained using the first lens, and may output a lens recommendation object representing a recommendation lens among the plurality of lenses. Various other embodiments that are derived from the specification are also possible.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a U.S. National Stage application under 35 U.S.C. § 371 of an International application number PCT/KR2020/000721, filed on Jan. 15, 2020, which is based on and claims priority of a Korean patent application number 10-2019-0010874, filed on Jan. 29, 2019, in the Korean Intellectual Property Office, the disclosure of which is incorporated by reference herein in its entirety.

TECHNICAL FIELD

Embodiments of the disclosure relate to an electronic device including a plurality of lenses for the purpose of photographing an image and a technology for controlling the electronic device for the purpose of photographing an image.

BACKGROUND ART

Various electronic devices capable of photographing an image with a camera module are being supplied. The camera module may generate a signal for constructing image data by using an image sensor from a light collected through a lens. Accordingly, there is a need to use various lenses for the purpose of photographing an image under various conditions.

To make it possible to photograph an image by using various lenses, the electronic device may include a plurality of various lenses. However, unless a user is an expert in photography, it is difficult to know the best lens for photographing a subject.

DISCLOSURE

Technical Problem

Embodiments of the disclosure may provide an electronic device and a method allowing a user to easily photograph an image under an optimum condition by using at least a part of a plurality of lenses included in the electronic device.

Technical Solution

According to an embodiment of the disclosure, an electronic device may include a display that outputs a screen, a camera module that includes a plurality of lenses including a first lens, a processor, and a memory. The memory may store instructions that, when executed, cause the processor to output, through the display, a screen including a lens selection user interface including a plurality of visual objects respectively corresponding to the plurality of lenses and a first image obtained by using the first lens and to output a lens recommendation object indicating a recommendation lens of the plurality of lenses.

According to an embodiment of the disclosure, a method of controlling an electronic device including a plurality of lenses may include outputting a screen including a lens selection user interface including a plurality of visual objects respectively corresponding to the plurality of lenses and a first image obtained through a first lens of the plurality of lenses, obtaining a second image through a second lens of the plurality of lenses, and outputting, on the screen, a lens recommendation object indicating a recommendation lens determined based on the first image and the second image, in a state where the second image is not output.

According to an embodiment of the disclosure, a storage medium may store a computer program that causes an electronic device to output a screen including a lens selection user interface including a plurality of visual objects respectively corresponding to a plurality of lenses and a first image obtained through a first lens of the plurality of lenses, to obtain a second image through a second lens of the plurality of lenses, and to output, on the screen, a lens recommendation object indicating a recommendation lens determined based on the first image and the second image, in a state where the second image is not output.

Advantageous Effects

According to embodiments of the disclosure, a method and an electronic device for allowing a user to easily perceive a lens appropriate to photograph a subject may be provided.

Besides, a variety of effects directly or indirectly understood through this disclosure may be provided.

With regard to description of drawings, the same or similar components will be marked by the same or similar reference signs.

MODE FOR INVENTION

Hereinafter, various embodiments of the disclosure may be described with reference to accompanying drawings. However, those of ordinary skill in the art will recognize that modification, equivalent, and/or alternative on various embodiments described herein can be variously made without departing from the scope and spirit of the disclosure.

The input device150may receive a command or data to be used by another component (e.g., the processor120) of the electronic device101, from the outside (e.g., a user) of the electronic device101. The input device150may include, for example, a microphone, a mouse, a keyboard, or a digital pen (e.g., a stylus pen).

FIG.2illustrates a configuration of an electronic device101, according to an embodiment.

According to an embodiment, the electronic device101may include the processor120, the memory130, the display device160, and the camera module180. The processor120may execute instructions stored in the memory130to control components of the electronic device101and to process data.

According to an embodiment, the camera module130may include a plurality of lenses281,282. . .28n. Under control of the processor120, the camera module130may generate a signal for sensing a light collected by using the plurality of lenses281,282. . .28nand generating an image.

The processor120may output the image, which is generated by using at least one of the plurality of lenses281,282. . .28n, through the display device160. For example, in the case where a camera application for image photographing is executed, the processor120may output an image, which is generated by using the first lens281, as a preview image for image photographing. According to an embodiment, the display device160may include a display for the purpose of outputting a screen including an image. In the case of receiving a user input of commanding image photographing, the processor120may store image data constituting the image output on the screen in the memory130. Here, the image data that are stored in the memory130may include at least one of a still image or a video image.

According to an embodiment, the processor120may further output a lens selection user interface, which includes visual objects corresponding to the plurality of lenses281,282. . .28n, through the display device160. For example, a visual object may mean an object, which the user is capable of visually perceiving, such as an icon. The user may perceive a visual object output through the display device160, and may input, to the electronic device101, a user command selecting at least one of the plurality of lenses281,282. . .28nby using the perceived visual object.

Also, the processor120may output a lens selection user interface, which further includes a lens recommendation object recommending at least one of the plurality of lenses281,282. . .28n, through the display device160. According to an embodiment, the lens recommendation object means an output of the electronic device101, which allows the user to identify and perceive a visual object corresponding to a recommended lens from among the visual objects corresponding to the plurality of lenses281,282. . .28n. For example, the lens recommendation object may be an image displayed around a visual object corresponding to a recommended lens. For another example, to output the lens recommendation object may mean to output a color of a visual object corresponding to a recommended lens so as to be distinguished from a color of any other visual object. For another example, an electronic device may output a voice capable of identifying a recommended object through a sound output device (e.g.,155ofFIG.1).

According to an embodiment, the processor120may obtain respective images through the plurality of lenses281,282. . .28nand may perform image recognition on each of the obtained images for the purpose of determining a recommendation lens. Here, the processor120may perform image recognition on an image (e.g., a second image obtained through the second lens282) obtained through any other lens, as well as an image (e.g., a first image obtained through the first lens281) output as a preview image through the display device160.

The processor120according to an embodiment may output, through the display device160, a screen including a lens recommendation object indicating a recommendation lens determined based on a result of the image recognition. For example, in the case where a person's face is recognized from an image, the processor120may determine a lens appropriate for portrait photography as a recommendation lens; in the case where an image is recognized as an image obtained by photographing a landscape, the processor120may determine a lens appropriate for landscape photography as a recommendation lens.

According to another embodiment, the processor120may automatically change settings for image photography of the electronic device101so as to photograph an image by using the determined recommendation lens; however, to allow the user to photograph an image as intended, the electronic device101according to an embodiment may display a lens recommendation object and may not automatically change settings for image photography.

An embodiment in which a plurality of preview images generated by using the plurality of lenses281,282. . .28nare output together is possible. However, because the electronic device101that the user carries to photograph an image is limited in size, the user may be inconvenient to perceive a plurality of images at the same time. For this reason, the electronic device101according to an embodiment may display the lens selection user interface that briefly displays information for selecting a lens together with preview images generated by using some lenses.

FIG.3is a view briefly illustrating an example of an appearance and an output screen of the electronic device101, according to an embodiment. Also,FIG.3illustrates the appearances of a front surface302and a rear surface304of the electronic device101.

Referring toFIG.3, the electronic device101may include the first lens281and the second lens282. The electronic device101may select a lens, which will photograph a preview image310, depending on a user command or stored settings for photography. In the case wherein the user inputs a photographing command through the preview image310output through a display device (e.g.,160ofFIG.2), the user may in advance check an image to be stored.

According to an embodiment, the electronic device101may further output a lens selection user interface320including two visual objects corresponding to the first lens281and the second lens282included therein. According to an embodiment, the lens selection user interface320may display information about at least one of a list of selectable lenses, a lens selected to obtain the preview image310, and a recommendation lens.

FIG.4is a flowchart400illustrating a process of controlling the electronic device101, according to an embodiment.

First, the electronic device101may perform operation410of obtaining a first image through a first lens of a plurality of lenses included in the electronic device101. Here, the first lens may be determined based on a user command or settings stored in the electronic device101.

Afterwards, the electronic device101may perform operation420of outputting a screen including a preview image generated based on the first image and visual objects corresponding to the plurality of lenses included in the electronic device101.

Also, the electronic device101may perform operation430of obtaining an image (e.g., a second image) through another lens (e.g., a second lens) other than the first lens among the plurality of lenses included in the electronic device101. For convenience of description, operation430is illustrated inFIG.4as being performed after operation420, but operation430may be performed in parallel with operation410and operation420depending on an embodiment.

Afterwards, the electronic device101may perform operation440of outputting a lens recommendation object indicating a recommendation lens determined based on the obtained images (e.g., the first image and the second image). Afterwards, when a user command that directs a change of a lens to be used to photograph an image is input from the user, the electronic device101may change a lens to be used to photograph an image from the first lens to another lens (e.g., the second lens) depending on the user command.

FIG.5illustrates an example for describing a method in which the electronic device101selects a recommendation lens, according to an embodiment.

FIG.5shows an example of a first image510obtained by using a first lens with a relatively narrow horizontal view angle and a second image520obtained by using a second lens with a relatively wide horizontal view angle.

Referring toFIG.5, according to an embodiment, the electronic device101may extract edges from the first image510and the second image520, respectively. Also, comparing a first trend line512and a second trend line522that are determined from the extracted edges, because the first image510fails to sufficiently include a subject compared to the second image520, the electronic device101may determine the second lens as a recommendation lens.

FIG.6illustrates an example for describing a method in which an electronic device selects a recommendation lens, according to an embodiment.

FIG.6shows an example of a first image610obtained by using a first lens with a relatively narrow horizontal view angle and a second image620obtained by using a second lens with a relatively wide horizontal view angle.

According to an embodiment, when one or more face objects612and622having human face features are recognized from photographed images, the electronic device101may determine a recommendation lens from lenses through which the images including the one or more objects612and622are photographed.

Alternatively, when both the first image610and the second image620include a face object, the electronic device101may determine the first lens photographing the first image610, in which an unnecessary white space (or margin) is less included, as a recommendation lens.

However, according to some embodiments,FIGS.5and6are for describing a method of using image recognition to determine a recommendation lens; according to an embodiment, a method of determining a recommendation lens may be replaced with any other method.

FIG.7illustrates an example of changing a lens selected to photograph an image, according to an embodiment.

Referring toFIG.7, the electronic device101may display a preview image510including an image determined by using a first lens, together with the lens selection user interface320. According to the example illustrated inFIG.7, the lens selection user interface320may include a first visual object322corresponding to the first lens, and a second visual object324corresponding to a second lens.

Also, in the case where the electronic device101determines the second lens as a recommendation lens, the electronic device101may further display a lens recommendation object330indicating a recommendation lens around the second visual object324.

Afterwards, the user may input a user command selecting the second lens to the electronic device101. For example, in the case where the electronic device101displays the lens selection user interface320through a touchscreen display, the electronic device101may receive a touch input to a region where the second visual object324is displayed. For another example, the electronic device101may receive a voice input such as “change a lens for photographing to a recommendation lens.”

In the case where the user command selecting the second lens is input, as a response to the user input, the electronic device101may display a preview image520including an image photographed by using the second lens, instead of the preview image510including the image photographed by using the first lens.

FIG.8is a flowchart illustrating a process of controlling an electronic device, according to an embodiment.

First, the electronic device101may perform operation810of obtaining a first image through a first lens of a plurality of lenses included in the electronic device101. Here, the first lens may be determined based on a user command or settings stored in the electronic device101.

Afterwards, the electronic device101may perform operation820of outputting a screen including a preview image generated based on the first image and visual objects corresponding to the plurality of lenses included in the electronic device101.

Also, the electronic device101may perform operation830of obtaining an image (e.g., a second image) through another lens (e.g., a second lens) other than the first lens among the plurality of lenses included in the electronic device101. For convenience of description, operation830is illustrated inFIG.4as being performed after operation820, but operation830may be performed in parallel with operation810and operation820depending on an embodiment.

Afterwards, the electronic device101may perform operation840of outputting a lens recommendation object indicating a recommendation lens determined based on the obtained images (e.g., the first image and the second image). According to an embodiment, the electronic device101may determine a recommendation lens by using a recommendation composition. The electronic device101according to an embodiment may determine a recommendation composition from one or more recommendation compositions stored in a memory (e.g., the memory130ofFIG.1) depending on a kind of an object recognized from the obtained images. Here, a recommendation composition may be data including feature information about an image. The electronic device101may determine the degrees by which features extracted from the obtained images correspond to the determined recommendation composition. The electronic device101may determine, as a recommendation lens, a lens used to photograph an image in which the degree corresponding to the recommendation composition is high.

Afterwards, when a user command that directs a change of a lens to be used to photograph an image is input from the user, the electronic device101may perform operation850of changing a lens to be used to photograph an image from the first lens to another lens (e.g., the second lens) depending on the user command and changing a preview image based on the changed lens.

Afterwards, the electronic device101may perform operation860of guiding the user to photograph an image depending on the recommendation composition by using the changed lens. For example, there may be output a photographing guidance object for guiding the user to move or rotate the electronic device101. Here, the photographing guidance object may be implemented in various forms capable of being output by the electronic device101. For example, the photographing guidance object may include an augmented reality object to be displayed on a preview image or a sound to be output through a speaker.

Afterwards, when a photographing command directing a save of image data is input, the electronic device101may perform operation870of storing an image obtained through the selected lens in a memory (e.g., the memory130ofFIG.1) as image data. According to an embodiment, the photographing command may be a user input of selecting an image photographing button. Alternatively, according to another embodiment, in the case where the image obtained through the selected lens satisfies a condition corresponding to the recommendation composition, the electronic device101may automatically determine that the photographing command is input.

FIG.9illustrates an example for describing a method in which the electronic device101selects a recommendation lens by using a recommendation composition, according to another embodiment.

Referring toFIG.9, the electronic device101may extract feature information910and920from images (e.g., the first image510and the second image520) obtained by using a plurality of lenses, respectively.

Afterwards, the electronic device101may compare each of the extracted feature information910and920with feature information included in a recommendation composition900. Depending on a result of the comparison, the electronic device101may calculate scores indicating the degrees by which the first image510and the second image520correspond to the recommendation composition900. According to an embodiment, the electronic device101may determine a recommendation lens based on magnitudes of the calculated scores.

FIG.10illustrates an example for describing a method in which the electronic device101guides to photograph an image depending on a recommendation composition, according to an embodiment.

An electronic device according to an embodiment may perform operation (e.g.,860ofFIG.8) of guiding the user to photograph an image depending on a recommendation composition. According to an embodiment, to guide the user to photograph an image depending on a recommendation composition, the electronic device101may overlap, on a preview image, a reference image indicating a reference location and a feature point indication image indicating a tracked feature location of a photographed subject.

According to an embodiment, the reference location may include at least one of coordinates and a direction (e.g., a horizontal direction) in an image frame, which are associated with the feature point of the subject to be located depending on the recommendation composition. For example, referring toFIG.10, the reference image indicating the reference location may include a location indication object1042to be displayed at a fixed location in a screen1010and a direction indication object1044indicating a direction.

Also, the feature location of the subject may mean a location that is determined by the photographed subject and has a feature corresponding to a feature of the reference location in the recommendation composition. For example, referring toFIG.10, a feature point image may include a location indication object1052indicating a location of the feature point of the subject and a direction indication object1054indicating a direction. In the case where an image of a subject in a frame moves or rotates along a movement or a rotation of the electronic device101or a lens, at least one of the location indication object1052and the direction indication object1054may move or rotate along the subject.

According to an embodiment, in the case where the user moves a lens and thus a feature point of a subject moves to a reference location, the electronic device101may display the object1052indicating that a location of the subject on a screen1020coincides with a location of a recommendation composition and the object1054indicating that a direction of the subject coincides with a direction of the recommendation composition.

FIG.11illustrates examples of lens selection user interfaces in which visual objects are arranged, according to various embodiments.

According to an embodiment, the electronic device101may construct a lens selection user interface depending on a configuration of lenses included in the electronic device101. For example, the electronic device101may display a lens selection user interface including visual objects, the number of which corresponds to the number of lenses available for photographing.

For example, referring to (a) ofFIG.11, in the case where the electronic device101includes three rear lenses1112disposed to photograph substantially the same direction, the electronic device101may display a lens selection user interface1114including three visual objects respectively corresponding to the lenses. In contrast, referring to (b) ofFIG.11, in the case where the electronic device101includes four rear lenses1122disposed to photograph substantially the same direction, the electronic device101may display a lens selection user interface1124including four visual objects respectively corresponding to the lenses.

Also, for another example, the electronic device101may display a lens selection user interface including visual objects disposed to correspond to a shape in which a plurality of lenses are arranged. As illustrated in (a) and (b) ofFIG.11, in the case where the lenses1112and1122are arranged in a row, the electronic device101may display the lens selection user interfaces1114and1124in which visual objects are arranged in a row. In contrast, in the case where the lenses1132are arranged in a plurality of columns, the electronic device101may display a lens selection user interface1134in which visual objects are arranged in a plurality of columns.

The electronic device101may allow the user to select a lens more intuitively by constructing a lens selection user interface depending on a configuration of lenses.

Also, to obtain information about a configuration of lenses included in the electronic device101, the electronic device101may execute an application program interface (API) installed on the electronic device101. The electronic device101may obtain information about the specifications of a plurality of lenses installed in the electronic device101by using the executed API. Based on the information about the specifications of the lenses thus obtained, the electronic device101may determine a recommendation lens or may construct a lens selection user interface.

FIG.12illustrates an example for describing a change of a lens selection user interface1200displayed in the process in which the electronic device101recommends a lens and changes a lens selected to photograph an image, according to an embodiment.

First, in the case where the electronic device101including a first lens and a second lens executes an application for photographing an image, the electronic device101may display the lens selection user interface1200including a first visual object1212corresponding to the first lens and a second visual object1222corresponding to the second lens. Also, according to an embodiment, in the case where a lens to be used to obtain a first image to be displayed in a display is the first lens, the electronic device101may display the first visual object1212corresponding to the selected first lenses so as to be distinguished from the second visual object1222.

Also, the electronic device101may obtain a second image through the second lens and may perform image recognition on the second image. While the second image is obtained through the second lens or an image recognition process is performed on the second image, the second visual object1222may be expressed as an image1224indicating that the image obtained through the second lens is under image analysis.

In the case where the second lens is determined as a recommendation lens as a result of performing the image recognition process, the electronic device101may display a lens recommendation object1226at a location of the second visual object1222.

Afterwards, in the case where the electronic device101receives a user input of selecting the lens recommendation object1226, the electronic device101may express the second visual object1222as an image1228indicating that a lens is selected. Also, the electronic device101may express the first visual object1212as an image1214indicating that a lens is not selected.

FIG.13illustrates examples of lens selection user interfaces, according to various embodiments.

A lens selection user interface may be implemented in various forms depending on embodiments.

For example, referring to (a) ofFIG.13, the electronic device101may display a lens selection user interface1310including an icon corresponding to a lens and a visual object in which a text expressing information about a lens is combined. The information about the lens may include, for example, a focal length of a lens corresponding to a visual object. According to an embodiment, the electronic device101may obtain information about a lens stored in the memory130by using an API installed thereon. Alternatively, the electronic device101may obtain information capable of identifying the electronic device101or a lens and may search an external server for the information about the lens by using the obtained information.

For another example, referring to (b) ofFIG.13, a lens selection user interface may be implemented in the form of a slide bar1320. When the user selects a desired setting value by using the slide bar1320, the electronic device101may select a lens having the specifications corresponding to the setting value.

FIG.14illustrates an example of a lens selection user interface, according to another embodiment.

According to another embodiment, the electronic device101may recommend a lens depending on a zoom in/zoom out operation associated with image photographing. In the case of receiving a zoom in command or a zoom out command, the electronic device101adjusts a magnification associated with image photographing. Here, the electronic device101may recommend an appropriate lens depending on the magnification associated with image photographing.

For example, in the case where a second lens is determined as a lens more appropriate for the zoom in or zoom out operation in a state where an image obtained through a first lens is being output as a preview image, the electronic device101may display a lens recommendation object recommending the second lens.

Alternatively, the electronic device101may recommend a lens by using a slide bar1400that slides depending on a magnification changed by the zoom in or zoom out operation. Referring toFIG.14, a visual object1402indicating each lens included in the electronic device101may be disposed on the slide bar1400depending on a magnification order (e.g., an ultra wide lens, a wide lens, and a tele lens) so as to move along a movement of the slide bar1400. The visual object1402may be displayed on a screen of the electronic device101or may disappear, while moving along the slide bar1400. When the visual object1402is displayed on the screen, the user may perceive that the corresponding lens is a lens recommended in a current magnification state and may input a user input of selecting the visual object1402to the electronic device101. The electronic device101that receives the user command selecting the visual object1402may change a lens corresponding to the visual object1402to a lens for photographing an image.