Patent Description:
In general, with scene understanding capabilities, user's naked eye cannot see or imagine the possible outputs that an intelligent camera along with an ultra-widen angles lens Field of View (FOV) would be able to provide. Currently, the camera has FOV clubbed with artificial intelligence (Al) provides plethora of opportunities to assist the user capture images that would not be possible otherwise.

The posture of holding the camera has a lot of impact on the taken picture. Each scene has its own set of rules which an ordinary camera user may not understand. For example, just taking a few steps towards a subject can give a better photo and the user may not understand this. Doing wide scene analysis and providing an indication/suggesting an action can help the user in getting a great capture.

Conventional systems identify a scene and understanding few filter/color/ correction options is known. Further, the systems use intelligence and machine learning to suggest best possible composition of the current view is of existence. Further, the conventional systems perform low aperture captures and pick objects from physical world and converting them to virtual objects.

<NPL>; <NPL> discloses an automatic zooming method for a camera capturing moving object in a FOV with a wide-angle mode being the starting point of the method, wherein the wide-angle mode is maintained as long as no object is detected in a FOV.

<CIT> discloses a method for automatically zooming in on a ROI selected by a user. <CIT> discloses a method and system for determining a region of interest on a screen based on a user's gaze towards the region of interest on the screen with respect to one or more images. Lastly, a publication of <NPL>, discloses a smartphone having <NUM> different smartphone-sized imaging modules.

Thus, it is desired to address the above mentioned shortcomings or at least provide a useful alternative.

In general, with scene understanding capabilities, user's naked eye cannot see or imagine the possible outputs that an intelligent camera along with an ultra-widen angles lens Field of View (FOV) would be able to provide.

An electronic device for recommending an image capture mode is defined as per the appended claims.

Further a method for recommending an image capture mode by an electronic device is defined as per the appended claims.

According to the embodiments herein, it is possible to identify at least one region of interest (ROI) displayed in a camera preview for capturing an image in a non-ultra-wide image capture mode.

This method and system is illustrated in the accompanying drawings, throughout which like reference letters indicate corresponding parts in the various figures. The embodiments herein will be better understood from the following description with reference to the drawings, in which:.

As is traditional in the field, embodiments may be described and illustrated in terms of blocks which carry out a described function or functions. These blocks, which may be referred to herein as managers, units, modules, hardware components or the like, are physically implemented by analog and/or digital circuits such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hardwired circuits and the like, and may optionally be driven by firmware and software. The circuits may, for example, be embodied in one or more semiconductor chips, or on substrate supports such as printed circuit boards and the like. The circuits constituting a block may be implemented by dedicated hardware, or by a processor (e.g., one or more programmed microprocessors and associated circuitry), or by a combination of dedicated hardware to perform some functions of the block and a processor to perform other functions of the block. Each block of the embodiments may be physically separated into two or more interacting and discrete blocks without departing from the scope of the disclosure. Likewise, the blocks of the embodiments may be physically combined into more complex blocks without departing from the scope of the disclosure.

Accordingly the embodiments herein provides a method for recommending an image capture mode by an electronic device. The method includes identifying, by the electronic device, at least one region of interest (ROI) displayed in a camera preview of the electronic device for capturing an image in a non-ultra-wide image capture mode. Further, the method includes determining, by the electronic device, that the at least one ROI is suitable to capture in an ultra-wide image capture mode. Further, the method includes providing, by the electronic device, at least one recommendation to switch to the ultra-wide image capture mode from the non-ultra-wide image capture mode for capturing the image.

The proposed method can be used to sense, analyze, understand a wider field of view and recreate a natural or artificial light from the ambience (physical world) into a current live experience for capturing a better image. The electronic device includes the ultra-wide lens for better scene understanding for sensing immense motion in the background to trigger a light painting nature of capture. The method can be used to understand the scene to provide intelligent capture suggestions.

The method can be used to provide best compositions for capturing photos by using the ultra-wide angle lens along with telephoto lens, wide lens clubbed with AI. The system provide effortless and auto triggered capturing suggestions for achieving signature aesthetics in photography using the ultra-wide lens and intelligence in a phone (best compositions).

The electronic device enable the user to capture a best possible shot even at the lack of resources like light (subject being dark) by borrowing and recreating actual ambient lighting for a natural looking output. The system uses the ultra-wide angle lens's field of view to ensure best composition due to more data availability. This improves the user's experience.

In general, each scene has its own set of rules which an ordinary camera user may not understand. In an example, just taking a few steps away from the subject can give a better photo, the user may not understand this. Based on the proposed methods, doing wide scene analysis and providing an indication / suggesting an action can help a user in getting a great capture based on photography rules.

<FIG> illustrates various blocks of an electronic device (<NUM>) for recommending an image capture mode, according to an embodiment as disclosed herein. The electronic device (<NUM>) can be, for example, but not limited to a cellular phone, a smart phone, a Personal Digital Assistant (PDA), a tablet computer, a laptop computer, a virtual reality device, a mixed reality device, an augmented reality device, a multi camera system, a smart watch, or the like.

In an embodiment, the electronic device (<NUM>) includes a processor (<NUM>), a communicator (<NUM>), a memory (<NUM>), a display (<NUM>), an ultra-wide lens (<NUM>), a telephoto lens (<NUM>), a sensor (<NUM>), an image capturing application (<NUM>), and a wide lens (<NUM>). The processor (<NUM>) is coupled with the communicator (<NUM>), the memory (<NUM>), the display (<NUM>), the ultra-wide lens (<NUM>), the telephoto lens (<NUM>), the sensor (<NUM>), the image capturing application (<NUM>) and the wide lens (<NUM>).

In an embodiment, the processor (<NUM>) is configured to identify at least one ROI displayed in a camera preview for capturing an image in a non-ultra-wide image capture mode. In an embodiment, the camera preview is shown on the image capturing application (<NUM>). In an embodiment, the least one ROI is automatically identified based on at least one of a presence of a user in the camera preview and a direction of a gaze of the user towards at least one portion in the camera preview. In an embodiment, the presence of the user in the camera preview and the direction of the gaze of the user towards at least one portion in the camera preview is identified using the sensor (<NUM>). After identifying the at least one ROI displayed in the camera preview for capturing the image in the non-ultra-wide image capture mode, the processor (<NUM>) is configured to determine that the at least one ROI is suitable to capture in an ultra-wide image capture mode.

In an embodiment, the processor (<NUM>) is configured to determine at least one object in the at least one ROI and determine whether the at least one object in the at least one ROI matches with at least one predefined object. The predefined object can be, for example, but not limited to a mountain, a sky, a snow, a sunset, a sunrise, a waterfall, a beach, a city, a waterside scenery, a stage, people or the like. Further, the processor (<NUM>) is configured to determine that the at least one ROI is suitable to capture in the ultra-wide image capture mode in response to determining that the at least one object in the at least one ROI matches with the at least one predefined object.

Based on the determination, the processor (<NUM>) is configured to provide at least one recommendation to switch to the ultra-wide image capture mode from the non-ultra-wide image capture mode for capturing the image. In an example, the electronic device (<NUM>) switches to the ultra-wide lens (<NUM>) from the telephoto lens (<NUM>) for capturing the image. In an embodiment, the at least one recommendation is provided in the camera preview. The at least one recommendation is provided when the wide lens (<NUM>) and the telephoto lens (<NUM>) are activated. In an embodiment, the suggestion to switch to the ultra-wide lens (<NUM>) can be provided upon analysis of wide frame data and upon user selection based on suggestion, the ultra-wide lens (<NUM>) is activated.

The ultra-wide lens (<NUM>) manipulates the view for better aesthetics and gaining perspective. The telephoto lens (<NUM>) is a type of camera lens designed for taking photographs of the user or object at moderate to far distances or the any distance. The wide lens (<NUM>) refers to a lens whose focal length is substantially smaller than a focal length of a normal lens for a given film plane. The ultra-wide lens (<NUM>) is a lens whose focal length is shorter than the short side of the sensor (<NUM>).

In another embodiment, the processor (<NUM>) is configured to identify the at least one ROI displayed in the camera preview. Further, the processor (<NUM>) is configured to determine at least one image capture mode from the plurality of image capture modes based on the at least one of ROI. Further, the processor is configured to provide at least one recommendation to activate the at least one determined image capture mode from the plurality of image capture modes to capture the image. In an example, based on the at least one of ROI, the electronic device (<NUM>) activates the ultra-wide lens (<NUM>) among the ultra-wide lens (<NUM>), the telephoto lens (<NUM>) and the wide lens (<NUM>) for capturing the image.

Further, the processor (<NUM>) is configured to provide a feedback to capture the image of the at least one ROI in the ultra-wide image capture mode. Further, the processor (<NUM>) is configured to activate the ultra-wide image capture mode. Further, the processor (<NUM>) is configured to generate the image of the at least one ROI in the ultra-wide image capture mode based on the feedback.

In an embodiment, the processor (<NUM>) is configured to capture scene data of the at least one ROI in the ultra-wide image capture mode. Further, the processor (<NUM>) is configured to automatically modify or correct at least one portion of the scene data corresponding to the at least one ROI based on the feedback. Further, the processor (<NUM>) is configured to generate the image based on the modified portions of the scene data corresponding to the at least one ROI or the correction portions of the scene data corresponding to the at least one ROI.

Further, the communicator (<NUM>) is configured for communicating internally between internal hardware components and with external devices via one or more networks. The memory (<NUM>) stores instructions to be executed by the processor (<NUM>). The memory (<NUM>) may include non-volatile storage elements. Examples of such non-volatile storage elements may include magnetic hard discs, optical discs, floppy discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories. In addition, the memory (<NUM>) may, in some examples, be considered a non-transitory storage medium. The term "non-transitory" may indicate that the storage medium is not embodied in a carrier wave or a propagated signal. However, the term "non-transitory" should not be interpreted that the memory (<NUM>) is non-movable. In some examples, the memory (<NUM>) can be configured to store larger amounts of information than the memory. In certain examples, a non-transitory storage medium may store data that can, over time, change (e.g., in Random Access Memory (RAM) or cache).

Various examples of the electronic device (<NUM>) configured to recommending the image capture mode is explained in the <FIG>.

Although the <FIG> shows various hardware components of the electronic device (<NUM>) but it is to be understood that other embodiments are not limited thereon. In other embodiments, the electronic device (<NUM>) may include less or more number of components. Further, the labels or names of the components are used only for illustrative purpose and does not limit the scope of the invention. One or more components can be combined together to perform same or substantially similar function to recommend the image capture mode in the electronic device (<NUM>).

<FIG> illustrates various blocks of the processor (<NUM>), according to an embodiment as disclosed herein. In an embodiment, the processor (<NUM>) includes an object detector (110a), a ROI detector (110b), a scene classifier (110c), a scene analyzer (110d), a view corrector (110e), a feedback provider (110f) and an ultra-wide image capture mode activator (<NUM>).

In an embodiment, the ROI detector (110b) is configured to identify the at least one ROI displayed in the camera preview for capturing the image in the non-ultra-wide image capture mode. After identifying the at least one ROI displayed in the camera preview for capturing the image in the non-ultra-wide image capture mode, the scene classifier (110c) and the scene analyzer (110d) are configured to determine that the at least one ROI is suitable to capture in the ultra-wide image capture mode.

In an embodiment, the object detector (110a) is configured to determine the at least one object in the at least one ROI and the view corrector (110e) determines whether the at least one object in the at least one ROI matches with at least one predefined object. Further, the scene classifier (110c) and the scene analyzer (110d) are configured to determine that the at least one ROI is suitable to capture in the ultra-wide image capture mode in response to determining that the at least one object in the at least one ROI matches with the at least one predefined object.

Based on the determination, the ultra-wide image capture mode activator (<NUM>) is configured to provide at least one recommendation to switch to the ultra-wide image capture mode from the non-ultra-wide image capture mode for capturing the image.

Further, the feedback provider (110f) is configured to provide the feedback to capture the image of the at least one ROI in the ultra-wide image capture mode. Further, the ultra-wide image capture mode activator (<NUM>) is configured to activate the ultra-wide image capture mode. Further, the ultra-wide image capture mode activator (<NUM>) is configured to generate the image of the at least one ROI in the ultra-wide image capture mode based on the feedback.

Although the <FIG> shows various hardware components of the processor (<NUM>) but it is to be understood that other embodiments are not limited thereon. In other embodiments, the processor (<NUM>) may include less or more number of components. Further, the labels or names of the components are used only for illustrative purpose and does not limit the scope of the invention. One or more components can be combined together to perform same or substantially similar function to recommend the image capture mode in the electronic device (<NUM>). <FIG> is a flow chart (S200) illustrating a method for recommending the image capture mode, according to an embodiment as disclosed herein. The operations (S202-S206) are performed by the processor (<NUM>).

At S202, the method includes identifying the at least one ROI displayed in the camera preview of the electronic device (<NUM>) for capturing the image in the non-ultra-wide image capture mode. At S204, the method includes determining that the at least one ROI is suitable to capture in the ultra-wide image capture mode. At S206, the method includes providing at least one recommendation to switch to the ultra-wide image capture mode from the non-ultra-wide image capture mode for capturing the image.

<FIG> is a flow chart (S204) illustrating various operations for determining that the at least one ROI is suitable to capture in the ultra-wide image capture mode, according to an embodiment as disclosed herein. The operations (S204a - S204c) are performed by the processor (<NUM>).

At S204a, the method includes determining the at least one object in the at least one ROI. At S204b, the method includes determining whether the at least one object in the at least one ROI matches with the at least one predefined object. At S204c, the method includes determining that the at least one ROI is suitable to capture in the ultra-wide image capture mode in response to determining that the at least one object in the at least one ROI matches with the at least one predefined object.

<FIG> is another flow chart (S300) illustrating a method for recommending the image capture mode, according to an embodiment as disclosed herein. The operations (S302-S306) are performed by the processor (<NUM>). At S302, the method includes identifying the at least one ROI displayed in the camera preview. At S302, the method includes determining the at least one image capture mode from the plurality of image capture modes based on the at least one of ROI. At S306, the method includes providing at least one recommendation to activate the at least one determined image capture mode from the plurality of image capture modes to capture an image.

In an embodiment, determining the at least one image capture mode from the plurality of image capture modes based on the at least one of ROI comprises determining at least one object in the at least one ROI, determining that the at least one object in the at least one ROI matches with at least one predefined object, and determining the at least one image capture mode from the plurality of image capture modes based on determination.

The various actions, acts, blocks, steps, or the like in the flow charts (S200, S204, and S300) may be performed in the order presented, in a different order or simultaneously. Further, in some embodiments, some of the actions, acts, blocks, steps, or the like may be omitted, added, modified, skipped, or the like without departing from the scope of the invention.

<FIG> are example scenarios in which the electronic device (<NUM>) recommends the image capture mode, according to an embodiment as disclosed herein.

As shown in the <FIG>, the electronic device (<NUM>) identifies the mountain present in the camera preview in the current image capture mode. As shown in the <FIG>, the electronic device (<NUM>) compares the identified mountain in the current image capture mode with that of a repository of pre-defined mountain objects, where the predefined objects are suitable for capturing in the ultra-wide image capture mode. As shown in the <FIG>, based on the comparison, the electronic device (<NUM>) provides a recommending in the preview, to switch to the ultra-wide image capture mode if the mountain in the current image capture mode match with that of the pre-defined mountain objects in the repository.

As shown in the <FIG>, the electronic device (<NUM>) identifies the ROI displayed in the camera preview for capturing the image in the non-ultra-wide image capture mode. The electronic device (<NUM>) identifies the user eye gaze direction and suggest the ultra-wide lens (<NUM>) (i.e., switch to the ultra-wide image capture mode from the non-ultra-wide image capture mode for capturing the image) to leave space in front of the user. The ultra-wide lens (<NUM>) manipulates the view for better aesthetics.

As shown in the <FIG>, the electronic device (<NUM>) identifies the at least one ROI displayed in the camera preview for capturing the mountain image in the non-ultra-wide image capture mode. Further, the electronic device (<NUM>) determines that the at least one ROI is suitable to capture in the ultra-wide image capture mode. Further, the electronic device (<NUM>) provides the recommendation (i.e., select ultra-wide lens (<NUM>) for interesting landscape) to switch to the ultra-wide image capture mode from the non-ultra-wide image capture mode for capturing the mountain image.

As shown in the <FIG>, the electronic device (<NUM>) identifies the at least one ROI displayed in the camera preview for capturing the child in the non-ultra-wide image capture mode. Further, the electronic device (<NUM>) determines that the at least one ROI is suitable to capture in the ultra-wide image capture mode using the eye direction of the child. Further, the electronic device (<NUM>) provides the recommendation (i.e., select the ultra-wide lens (<NUM>) and apply cropping to ensure eye position of user) to switch to the ultra-wide image capture mode from the non-ultra-wide image capture mode for capturing the child image.

As shown in the <FIG>, the electronic device (<NUM>) identifies at least one ROI displayed in the camera preview for capturing the subject in the non-ultra-wide image capture mode. Further, the electronic device (<NUM>) determines that the at least one ROI is suitable to capture in the ultra-wide image capture mode. Further, the electronic device (<NUM>) provides the recommendation (i.e., identify subject movement direction, select the ultra-wide lens (<NUM>) and appropriate crop to leave space for movement) to switch to the ultra-wide image capture mode from the non-ultra-wide image capture mode for capturing the subject.

As shown in the <FIG>, the electronic device (<NUM>) identifies the at least one ROI displayed in the camera preview for capturing the subject in the non-ultra-wide image capture mode. Further, the electronic device (<NUM>) determines that the at least one ROI is suitable to capture in the ultra-wide image capture mode based on the subject's head direction. Further, the electronic device (<NUM>) provides the recommendation (i.e., select ultra-wide lens (<NUM>) and appropriate crop to leave enough space above the subject's head) to switch to the ultra-wide image capture mode from the non-ultra-wide image capture mode for capturing the subject.

Claim 1:
An electronic device (<NUM>) for recommending an image capture mode, comprising:
a memory (<NUM>); and
a processor (<NUM>), coupled with the memory (<NUM>), configured to:
identify (S202) at least one region of interest, ROI, displayed in a camera preview of the electronic device (<NUM>) for capturing an image in a non-ultra-wide image capture mode associated with a telephoto lens (<NUM>) and a wide lens (<NUM>);
determine (S204) that the at least one ROI is suitable to be captured in an ultra-wide image capture mode associated with an ultra-wide lens (<NUM>) configured to capture a wider image than the telephoto lens (<NUM>) and the wide lens (<NUM>); and
provide (S206) at least one recommendation in the camera preview to switch to the ultra-wide image capture mode from the non-ultra-wide image capture mode for capturing the image and cropping the ultra-wide image such that after cropping a positioning of the at least one ROI in the captured image fulfils the photography rule of thirds;
wherein the determining that the at least one ROI is suitable to capture in the ultra-wide image capture mode comprises:
determining (S204a) at least one object in the at least one ROI;
determining (S204b) whether the at least one object in the at least one ROI matches with at least one predefined object; and
determining (S204c) that the at least one ROI is suitable to capture in the ultra-wide image capture mode in response to determining that the at least one object in the at least one ROI matches with the at least one predefined object.