Patent Description:
Modern electronic devices, such as smartphones, comprise two or more cameras. The cameras may have different hardware capabilities (e.g., different resolutions, different lens speeds, etc.) and/or different software settings (e.g., different shutter speeds, different ISO settings, etc.) suitable for different scenarios. The user of such a device therefore has a choice of different cameras, and can specify which camera to use when capturing an image of a scene.

<CIT> discloses storing a first image of a scene captured by a first camera, storing a second image of the scene captured by a second camera, and storing a first user-composed image.

The invention is defined in the independent claims, while advantageous embodiments are set out in the dependent claims.

According to a first aspect disclosed herein, there is provided a method carried out by an electronic device which comprises a plurality of cameras, the method comprising:.

The images captured by the different cameras may be substantially similar, but differ due to, for example, one or more different hardware or software settings of the cameras at the time the images are captured. The user may not know at the time which camera is best suited to capture the image. To address this, even though the user selected only one camera, examples described herein take at least one second image with one or more of the other cameras and store a "backup" of the other images, for at least a period of time, so that the user can access them should it turn out that they are required. By saving the at least one second image to the external storage, rather than the internal storage, storage space on the electronic device is saved.

The selected one of the plurality of cameras may be specified explicitly by the user command, i.e. the user command may include an indication of which camera to use. Alternatively, the selected one of the plurality of cameras may be specified implicitly, e.g. the selected camera may be a default camera, or a most recently-used camera.

In an example, the method may comprise causing metadata associating the first image with the at least one second image to be stored. The metadata may be stored at the electronic device, e.g. in the internal storage. An advantage of this is that the electronic device itself can locally determine that there is at least one second image associated with the first image in the internal storage. However, it is not excluded that the metadata is stored elsewhere, e.g. the external storage or another storage.

Alternatively or additionally, image recognition may be used to identify that a second image corresponds to the first image, in which case no metadata may be required.

In an example, the method comprises: receiving user input relating to the first image; and in response to receiving said user input relating to the first image, outputting an indication to the user that the at least one second image is available.

The user input may comprise one or more of: a zoom-in action on the first image; a crop action on the first image; a zoom-out action on the first image; an instruction to adjust a value of a setting of the first image; and an instruction to use the first image in an application. In other words, if the user performs any action on or with the first image, the electronic device may inform the user that there are other versions of the first image available which the user may wish to consider.

In an example, the method comprises: receiving user input relating to the first image; identifying an image property based on the user input relating to the first image; determining a second image of the at least one second images stored in the external storage having an improved value for the identified image property relative to the first image; and outputting a notification to the user that the determined second image is available. In other words, rather than simply informing the user that there are other versions of the first image available, the electronic device may actively suggest one to the user.

In an example, said user input comprises one or more of a zoom-in action on the first image and a crop action on the first image, said identified image property being one or more of an image resolution and an image sharpness; and wherein the second image is determined based on the second image having one or more of: a higher resolution than the first image; and a greater sharpness than the first image.

In an example, the method comprises identifying a portion of the first image based on the zoom-in or crop action, said identified image property being one or more of an image resolution within said portion and an image sharpness within said portion; and the second image is determined based on the second image having one or more of: a higher resolution in a portion of the second image corresponding to the identified portion of the first image and a greater sharpness in the portion of the second image corresponding to the identified portion of the first image.

In an example, said user input comprises a zoom-out action on the first image, said identified image property being a field of view; and wherein the second image is determined based on the second image having a wider field of view than the first image.

In an example, said user input comprises an instruction to use the first image in an application having a predefined image requirement, said identified image property being the predefined image requirement; and wherein the second image is determined based on the second image meeting said predefined image requirement.

In an example, the method comprises: obtaining one or more of said second images from the external storage; receiving user input selecting one of said second images; and storing the selected second image in the internal storage.

In an example, the method comprises deleting the first image from the internal storage. This may be done e.g. in response to receiving the user selection, or in response to storing the second image.

In an example, the method comprises automatically deleting the at least one second image from the external storage a predetermined amount of time after the at least one second image was stored to the external storage. The predetermined amount of time may be configurable by the user and may be, e.g. an hour, a day, a week, a month, etc..

The predetermined period of time may be configurable by the user and may be, e.g. an hour, a day, a week, a month, etc. Generally, the predetermined period of time may be shorter than the predetermined amount of time.

In an example, the user command specifies at least one setting for the selected camera for use in capturing the first image; and wherein the method comprises:.

For example, the user could specify to capture a black and white image with the selected camera, and the electronic device will also capture black and white images with the other cameras.

According to a second aspect disclosed herein, there is provided an electronic device comprising:.

The cameras may have a substantially corresponding field of view. That is, at any given moment in time, the respective images captured by the cameras may be similar. This may be the case, for example, if the cameras are mounted within the electronic device e.g. next to each other, pointing in same direction.

In an example, the controller is configured to store metadata associating the first image with the at least one second image.

In an example, the plurality of cameras each have at least one different hardware property selected from: sensor resolution, maximum aperture, and minimum focus distance.

In an example, the plurality of cameras have at least one different setting at the time of receipt of the user command selected from: shutter speed, f-stop, ISO, focal length, image format, saturation, contrast, gamma, supported colour space, picture enhancement capabilities (e.g. HDR), exposure time, and flash availability (i.e. whether or not flash was used when capturing the image).

In accordance with a third aspect disclosed herein, there is provided a computer program comprising instructions configured so as when executed by one or more processing units to perform the method of the first aspect or any example thereof.

Electronic devices comprising a plurality of cameras provide the user with a greater choice when it comes to capturing an image of a scene. However, it can be difficult for a non-expert user to know, at the time of image capture, which is the best choice of camera for that particular scene. Even an expert user may be in a hurry and not have time to select the ideal camera. Moreover, it may turn out at a later point in time that it would have been preferable to use a different one of the cameras than the one which with which the image was captured. Examples described herein address these and other problems.

<FIG> shows schematically an example of an electronic device <NUM> in accordance with examples described herein. Also shown in <FIG> is a network <NUM> and a data storage <NUM> external from the electronic device <NUM>, referred to herein as "external storage". The external storage <NUM> represents one or more storage locations which are accessible by the electronic device <NUM> via the network <NUM>. The network <NUM> may be a local or wide-area network such as, for example, the Internet.

The electronic device <NUM> may be, for example, a smartphone, a laptop or other portable computer, etc. The electronic device <NUM> comprises a plurality of cameras <NUM>, a data storage <NUM> (referred to herein as "internal storage"), a user interface <NUM>, a communications interface <NUM>, and a controller <NUM>. The controller <NUM> is operatively coupled to the plurality of cameras <NUM>, the internal storage <NUM>, the user interface <NUM>, and the communications interface <NUM>. The controller <NUM> may be implemented using one or more processors.

The user interface <NUM> may comprise, for example, one or more of a display, a microphone, a computer mouse, a touchscreen, an API, etc. In operation, the user interface <NUM> allows a user to provide user input to the controller <NUM> of the electronic device <NUM>. The user interface <NUM> may also allow the user to receive information from the electronic device <NUM>. In other examples, separate user interfaces may be provided for input and output of information from the electronic device <NUM>.

The communications interface <NUM> may comprise one or more of a wired communications interface (e.g. an Ethernet interface) or wireless communications interface (e.g. a WiFi interface). In operation, the communications interface <NUM> allows the controller <NUM> of the electronic device <NUM> to send data to the external storage <NUM>. The communications interface <NUM> may also allow the controller <NUM> to receive data from the external storage <NUM>. In other examples, separate communications interfaces may be provided for sending and receiving data from the external storage <NUM>.

The cameras <NUM> are generally mounted by or within a housing of the electronic device <NUM> such that they each have a similar field of view (e.g., the cameras <NUM> may point in the same direction). However, the actual images which would be captured by the cameras <NUM> at any given time may differ slightly. This can be due to a variety of factors, including one or more different hardware properties of the cameras <NUM>, one or more different hardware settings of the cameras <NUM>, and/or one or more different software settings of the cameras <NUM>, and the fact that the cameras <NUM> are at different locations on the electronic device <NUM> and so will have a slightly different view of any scene.

Put differently, even though each camera <NUM> captures an image of substantially the same scene (as the cameras <NUM> all view substantially the same thing), the cameras <NUM> may be of different types and/or have different settings at the time of image capture and have a different view, and this leads to the images being slightly different from one another.

Examples of different hardware properties (hardware capability) include: sensor resolution, maximum aperture (aka "lens speed"), and minimum focus distance. Examples of different hardware and software settings include: shutter speed, f-stop, ISO, focal length, image format, saturation, contrast, gamma, supported colour space, picture enhancement capabilities (e.g. HDR), exposure time, and flash availability (i.e. whether or not flash was used when capturing the image).

In the example of <FIG>, the electronic device <NUM> comprises four cameras 101a-d. However, it is appreciated that in other examples the electronic device <NUM> may comprises two cameras, three cameras, or five or more cameras. For ease of explanation, it is assumed that the first camera 101a is a 12MP ultra-wide camera, the second camera 101b is a 108MP wide camera, the third camera 101c is a 10x 10MP telephoto camera, and the fourth camera 101d is a 3x 10MP telephoto camera.

When capturing an image of a scene, the most appropriate or "best" of these cameras <NUM> depends on many factors, including, e.g., the distance to an object of interest within the scene, the size of the object of interest, the current lighting characteristics, motion within the captured image, etc. For example, the third camera 101c (the 10x 10MP telephoto camera) is better suited for capturing an image of a distant object than, say, the first camera 101a (the 12MP ultra-wide camera). As another example, the fourth camera 101d (the 3x 10MP telephoto camera) may have a higher shutter speed setting at the time of image capture than the third camera 101c (the 10x 10MP telephoto camera), and therefore the fourth camera 101d is better suited for capturing an image of a fast-moving object within the scene than the third camera 101c.

The user is able to specify which camera <NUM> to use to capture an image. A problem arises, however, because the user may not know which camera <NUM> is most appropriate. Even if the user does know which camera <NUM> he or she wants to use, the user may not have time to change which camera is selected. Moreover, the user may only realise at a later point in time (after the image has been captured) that a different camera <NUM> would have been preferable than the one that they did select. Examples described herein address these and other problems.

<FIG> shows a flow chart of an example of a first method performed by the controller <NUM> of the electronic device <NUM>.

At S210, the controller <NUM> receives, via the user interface <NUM>, a user command to capture an image using a selected one of the plurality of cameras 101a.

The selected camera 101a may be specified explicitly by the user command, i.e. the user command may include an indication of which camera to use. Alternatively, the selected camera 101a may be specified implicitly by the user command, e.g. the selected camera 101a may be a default camera, or a most recently-used camera.

In response to the user command, the controller <NUM> does two things: at S221 the controller <NUM> causes the selected camera 101a to capture a first image; and at S222 the controller <NUM> causes at least one other, non-selected camera 101b-d to capture a respective second image. In other words, not only does the selected camera 101a capture an image (as requested by the user), one or more other cameras 101b-d also capture images.

The decision of which one or more of the other, non-selected cameras 101b-d to use to capture second image(s) at S222 may be made based on a variety of factors. In a simple case, all of the other, non-selected cameras 101b-d may be used by default. Other examples are given later below.

Once the first image has been captured, at S231 the controller <NUM> causes the first image to be stored to the internal storage <NUM>. From the internal storage <NUM>, the first image is viewable by the user in a photo-viewing application on the electronic device <NUM> (commonly referred to as a "gallery").

Once the at least one second image has been captured, at S232, the controller <NUM> sends the second image(s) to the external storage <NUM> via the communications interface <NUM>, to cause the at least one second image to by stored in the external storage <NUM>.

The capture S221 and storage S231 of the first image represents the "normal" behaviour of the electronic device <NUM> which the user expects in response to their command. The capture S222 and storage S232 of the second image(s) represents additional actions performed "in the background". The fact that these one or more other images have been captured and stored (remotely) means that they are accessible by the user at a later point in time. As such, the problem of a user having specified the "wrong" or not best or most optimal camera at the time of image capture is solved. Moreover, by storing the other images in the external storage <NUM>, rather than the internal storage <NUM>, consumption of the internal storage <NUM> of the electronic device <NUM> is reduced.

At a later point in time, the electronic device <NUM> may inform the user that the at least one second image is available. Specifically, the controller <NUM> may cause a notification to be presented to the user via the user interface <NUM> informing the user of the existence of the at least one second image. The notification may be presented to the user in response to the controller <NUM> receiving user input relating to the first image (e.g. in response to the user navigating to the first image in a gallery application on the electronic device <NUM>). In other words, if the user interacts with the first image, then the controller <NUM> may inform the user of the availability of one or more alternative versions of that image (the at least one second image).

In order to identify the second image(s) from the first image, the controller <NUM> may store metadata associating the first image with the at least one second image. The metadata may be stored in the external storage <NUM>, in the internal storage <NUM>, or elsewhere. An advantage of storing the metadata in the internal storage <NUM> is that the controller <NUM> itself can locally determine that at least one second image associated with the first image exists in the external storage <NUM>. However, it is not excluded that the metadata is stored elsewhere, e.g. the external storage <NUM> or another storage. Alternatively or additionally, image recognition may be used to identify that a second image corresponds to the first image, in which case no metadata may be required.

In some examples, the notification is a simple (e.g. text) notification. In other examples, the controller <NUM> may obtain the available second image(s) from the external storage <NUM> and the images themselves may comprise part of the notification (i.e. the user may be presented with the available second image(s) when they interact with the first image).

The notification may request user input allowing the user to select one or more of the second images. In response, the controller <NUM> may store the selected one or more of the second images to the internal storage <NUM>. Optionally, at this point, the first image may be automatically deleted from the internal storage <NUM>.

In more sophisticated examples, the controller <NUM> may make a determination as to whether to inform the user of the existence of the at least one second image and, if there is more than one available, which one. As will now be described, this involves analysing the user input to infer a desirable image property, and determining whether any of the available second images have that desirable image property.

<FIG> shows a flow chart of an example of a second method performed by the controller <NUM> of the electronic device <NUM>. The second method is performed after the first method described above has been performed. That is, the second method is performed once the first image has been stored to the internal storage <NUM> and the one or more second images stored to the external storage <NUM>.

At S310, receives user input relating to the first image. The user input comprises one or more of: a zoom-in action on the first image; a crop action on the first image; a zoom-out action on the first image; an instruction to use the first image in an application, etc. Examples of such applications include social media applications and printer applications.

In response, at S320, the controller <NUM> identifies an image property based on the user input relating to the first image. Put generally, the controller <NUM> may infer what the user is trying to achieve based on the user input, and therefore identify a particular image property which is desirable or important to the user. For example, if the user input comprises a zoom-in action on the first image, then resolution may be identified as the image property of interest. As another example, if the user input comprises a crop action on the first image, then resolution may be identified as the image property of interest. As yet another example, if the user input comprises a zoom-out action on the first image, then field of view may be identified as the image property of interest. In further examples, if the user input comprises an instruction to use the first image in an application having a predefined image requirement, then that predefined image requirement may be identified as the image property of interest. Examples of such applications include social media applications and printer applications, as both of these can specify various image requirements.

In example, a threshold may be applied to the user input received at S310 such that the controller <NUM> only acts on user input exceeding the threshold (e.g. the user zooming-in beyond a threshold amount, cropping the first image to a portion smaller than a threshold portion, zooming-out beyond a threshold amount, etc.).

At S330, the controller <NUM> determines one or more of the second images from the second images stored in the external storage <NUM>. This is done based on the identified image property. Specifically, given the identified image property, the controller <NUM> determines whether any of the second images have an improved value for that image property (e.g. if the image property is resolution, an improved value would be a higher resolution; if the image property is field of view, an improved value would be a larger field of view; if the image property is a predefined image requirement, an improved value would be meeting that predefined image requirement).

In response to determining the one or more second images, at S340, the controller <NUM> presents a notification via the user interface <NUM> informing the user that the determined one or more of the second images are available. Unlike the earlier, simpler, example, the notification specifically indicates the one or more second images which have been determined at S330 (rather than simply informing the user that there are alternative images).

Note that if two or more second images have an improved value for the identified image property, then the controller <NUM> may notify the user of the existence of all of these second images. In other examples, may only notify the user of the existence of the second image having the most improved value for the identified image property. Alternatively or additionally, the controller <NUM> may apply a threshold to the image property and only inform the user of any second images having a value for the image property above the threshold.

In examples, the controller <NUM> may apply an image size threshold to the second images corresponding to an available amount of storage space in the internal storage <NUM>. The controller <NUM> may then only suggest second image(s) to the user have a size less than the threshold image size.

In any event, the controller <NUM> may obtain the determined second image(s) from the external storage <NUM> for viewing by the user on the user interface <NUM>, the determined second images themselves comprising part of the notification. The user may then select one or more of the second images to store to the internal storage <NUM>. At this point, the first image may optionally be automatically deleted from the internal storage <NUM>.

In examples, the controller <NUM> may identify whether user input relating to the first image has been received during a predetermined period of time (e.g. a number of hours, days, weeks, etc.) since the at least one second image was stored to the external storage <NUM>. If no user input relating to the first image has been received during said predetermined period of time, the controller <NUM> may cause the at least one second image to be deleted from the external storage <NUM>. In examples, the controller <NUM> may only cause the second image(s) to be deleted if no substantive user input relating to the first image (e.g. editing the first image) has been received during the predetermined period of time. That is, the controller <NUM> ignores minor user input such as the user merely navigating to the first image and/or viewing it in the gallery.

As mentioned earlier above, the decision of which other, non-selected cameras 101b-d to use to capture the second image(s) at S222 can be based on a variety of factors. For example, the user may specify a particular one or more of the cameras <NUM> to use to capture the second image(s). Alternatively or additionally, the controller <NUM> may automatically reduce the number of other, non-selected cameras 101b-d which are used based on an amount of available storage space in the external storage <NUM>. That is, if the user's free space availability in external storage <NUM> becomes less than a threshold level, then the controller <NUM> might not capture a second image using one or more of the other cameras 101b-d. The "deselection" of the other cameras 101b-d may be performed in order of decreasing image size (e.g. an ultra-wide camera option might be ignored first).

In examples, the controller <NUM> may automatically apply the same or similar settings (both hardware and software) to the other cameras 101b-d as are applied to the selected camera 101a for capturing the images. For example, if the user command is to capture a black and white image using the first camera 101a, then the controller <NUM> may automatically cause the second images to also be captured in black and white.

In examples, the controller <NUM> may identify the user's most-frequently used settings (both hardware and software) when capturing an image. In such cases, the controller <NUM> may automatically apply these settings to the other cameras 101b-d when capturing the second images.

The following are example use cases to demonstrate the benefits of some examples of the present disclosure.

In a first use case example, the user visits the first image in a gallery application and tries to perform a zoom-in action on the first image (e.g. by double-tapping or using a two-finger gesture on the screen). In response, the controller <NUM> checks if the zoom-in exceeds a threshold level (which could be set by the manufacturer depending on camera's quality and some experimental results in research and development testing stages of mobile device) and determines that the first photo, taken by the fourth camera 101d, is less detailed (lower quality or resolution) compared with the photo taken with the third camera 101c. Hence, the second photo taken by the third camera 101c provides a better image for the user in the sense that more detail is present when zoomed-in. Accordingly, the user interface <NUM> will show a dialog and ask for the user's consent to download the second photo.

In a second use case example, the user visits the first image in a gallery application and tries to perform a zoom-out action on the first image (e. g by using a two-finger gesture on the screen). In response, the electronic device <NUM> asks her consent to download e.g. a photo taken by the first camera 101a, as this photo provides has a wider angle scene when compared to the first image taken by the third camera 101c.

In a third use case example, the user visits the first image in a gallery application and tries to perform a cutting or cropping action on the first image. Here again a threshold level may be set to determine if the cropped first photo or one of the second photos in the external storage <NUM> provides the better image (higher resolution). Accordingly, the electronic device <NUM> will propose downloading the second image stored or not.

In another example, the user attempting to use the first image in a particular software application may trigger the controller <NUM> identifying and notifying the user of one or more of the second image(s). The following will be described with reference to a social media application, but it is appreciate that similar considerations apply in relation to other applications such as printer software application etc..

Some social media applications ("platforms") use special photo sizes which means that the user needs to crop their photo (the first image) before uploading if it does not already meet the social media application's requirements. However, cropping applied directly to the first image might not achieve satisfactory results, as will now be described with reference to <FIG>. In this example, the application specifies a requirement that the image is a square image (i.e. has a <NUM>:<NUM> aspect ratio).

<FIG> shows the user attempting to crop the first image <NUM> on the user interface <NUM>. There are two objects of interest 430a, 430b in the first image <NUM> which the user wants to include in the cropped area <NUM>. However, the application specifies that the crop area must be square. This means that it is not possible in this example to fit both the first object of interest 430a and second object of interest 430b into the crop area at the same time. As shown in <FIG>, the second object of interest 430b is included in the crop area, but the first object of interest 430a is not. It is not possible to increase the size of the crop area to also include the first object of interest 430a because the crop area already covers the entire horizontal width of the first image.

In this particular scenario, the controller <NUM> would suggest a second image which is a wide-angle version of the first image. <FIG> shows the user attempting to crop the second image <NUM>. The second image <NUM> corresponds to the first image <NUM> at least in that the two objects of interest 430a, 430b are also present in the second image <NUM> location in at least substantially the same or corresponding locations within the second image <NUM>. In other words, the second image <NUM> is a wide-angle version of the first image <NUM>. As shown in <FIG>, unlike the first image <NUM>, it is possible to crop the second image <NUM> in a <NUM>:<NUM> aspect ratio in such a manner that both objects of interest 430a, 430b are included in the cropped area.

It will be understood that the processor or processing system or circuitry referred to herein may in practice be provided by a single chip or integrated circuit or plural chips or integrated circuits, optionally provided as a chipset, an application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), digital signal processor (DSP), graphics processing units (GPUs), etc. The chip or chips may comprise circuitry (as well as possibly firmware) for embodying at least one or more of a data processor or processors, a digital signal processor or processors, baseband circuitry and radio frequency circuitry, which are configurable so as to operate in accordance with the exemplary embodiments. In this regard, the exemplary embodiments may be implemented at least in part by computer software stored in (non-transitory) memory and executable by the processor, or by hardware, or by a combination of tangibly stored software and hardware (and tangibly stored firmware).

Reference is made herein to data storage for storing data. This may be provided by a single device or by plural devices. Suitable devices include for example a hard disk and non-volatile semiconductor memory (including for example a solid-state drive or SSD).

Although at least some aspects of the embodiments described herein with reference to the drawings comprise computer processes performed in processing systems or processors, the invention also extends to computer programs, particularly computer programs on or in a carrier, adapted for putting the invention into practice. The program may be in the form of non-transitory source code, object code, a code intermediate source and object code such as in partially compiled form, or in any other non-transitory form suitable for use in the implementation of processes according to the invention. The carrier may be any entity or device capable of carrying the program. For example, the carrier may comprise a storage medium, such as a solid-state drive (SSD) or other semiconductor-based RAM; a ROM, for example a CD ROM or a semiconductor ROM; a magnetic recording medium, for example a floppy disk or hard disk; optical memory devices in general; etc..

Claim 1:
A method carried out by an electronic device (<NUM>) which comprises a plurality of cameras (<NUM>), the method comprising:
receiving (S210), at the electronic device (<NUM>), a user command to capture an image using a selected one of the plurality of cameras (101a);
in response to receiving said user command, capturing (S221) a first image using the selected camera (101a) and capturing (S222) at least one second image using at least one other, non-selected, camera (101b, 101c, 101d);
storing (S231) the first image to a data storage (<NUM>) internal to the electronic device (<NUM>);
sending (S232) the at least one second image to a data storage (<NUM>) external of the electronic device (<NUM>) for storage at said external storage (<NUM>);
receiving (S310) user input relating to the first image, the user input being selected from: a zoom-in action on the first image; a crop action on the first image; a zoom-out action on the first image; and an instruction to use the first image in an application;
in response to receiving said user input relating to the first image, outputting (S340) a notification to the user that the at least one second image is available;
obtaining one or more of said second images from the external storage (<NUM>);
receiving user input selecting one of said second images; and
storing the selected second image in the internal storage (<NUM>).