Patent Description:
In <CIT> regions of interest may be illuminated upon image acquisition.

In one example, an apparatus comprises an illumination unit configured to simultaneously illuminate a first portion of a scene with unstructured light and a second portion of the scene with structured light; and a digital image capture unit configured to capture at least one image frame of the illuminated scene.

In other examples, a system and a method have been discussed along with the features of the apparatus.

The detailed description provided below in connection with the appended drawings is intended as a description of the present examples and is not intended to represent the only forms in which the present example may be constructed or utilized. The description sets forth the functions of the example and the sequence of steps for constructing and operating the example. However, the same or equivalent functions and sequences may be accomplished by different examples.

Although some of the present examples may be described and illustrated herein as being implemented in a smartphone or a tablet computer, these are only examples of an apparatus and not a limitation. As those skilled in the art will appreciate the present examples are suitable for application in a variety of different types of apparatuses incorporating a digital image capture unit or a digital imaging system, for example, a stand-alone digital camera device, e.g. a compact camera, a SLR (Single-Lens Reflex) camera, or a mirrorless interchangeable-lens camera.

<FIG> shows a block diagram of one example of an apparatus <NUM> which may be implemented as any form of a computing device and/or electronic device that incorporates a digital image capture unit or a digital imaging system. For example, the apparatus <NUM> may be implemented as a stand-alone digital camera device, e.g. a compact camera, a SLR camera, or a mirrorless interchangeable-lens camera, or the apparatus <NUM> may be implemented e.g. as a smartphone, a tablet computer, a wearable camera or a web camera.

The apparatus <NUM> comprises an illumination unit <NUM>. The illumination unit <NUM> is configured to simultaneously illuminate a first portion of a scene with unstructured light and a second portion of the scene with structured light. The second portion of the scene may overlap the first portion of the scene partially, completely, or not at all. The unstructured light and/or the structured light may comprise light invisible to human eye, such as infrared light or ultraviolet light. The illumination unit <NUM> may be implemented e.g. as light-emitting diode (LED).

The illumination unit <NUM> may comprise a diffractive optical element (DOE) <NUM> that is configured to provide the structured light. The diffractive optical element <NUM> may be switchable. The diffractive optical element <NUM> may be implemented e.g. as a lens that may be installed e.g. in front of the illumination unit <NUM> so that the light emitting from the illumination unit <NUM> passes through the lens. The diffractive optical element <NUM> may comprise a first part configured to allow the light emitting from the illumination unit <NUM> pass through unaltered, thereby providing the unstructured light. The diffractive optical element <NUM> may further comprise a second part configured to cause predetermined patterns in the light emitting from the illumination unit <NUM>, thereby providing the structured light.

The apparatus <NUM> further comprises a digital image capture unit <NUM>. The digital image capture unit <NUM> is configured to capture at least one image frame of the illuminated scene. The digital image capture unit <NUM> may comprise at least an optical system including a lens arrangement and an image sensor, such as a charge-coupled device (CCD) sensor or a complementary metal-oxide-semiconductor (CMOS) sensor. The digital image capture unit <NUM> further comprises a rolling shutter. The scene may comprise a region of interest, and the illumination unit <NUM> may be further configured to illuminate the region of interest with higher power than the rest of the scene.

<FIG> shows a block diagram of one example of a system <NUM>. The system <NUM> comprises an apparatus <NUM> which may be implemented as any form of a computing device and/or electronic device that incorporates a digital image capture unit or a digital imaging system. For example, the apparatus <NUM> may be implemented as a stand-alone digital camera device, e.g. a compact camera, a SLR camera, or a mirrorless interchangeable-lens camera, or the apparatus <NUM> may be implemented e.g. as a smartphone, a tablet computer, a wearable camera or a web camera.

The system <NUM> comprises an illumination unit <NUM>. The illumination unit <NUM> is configured to simultaneously illuminate a first portion of a scene with unstructured light and a second portion of the scene with structured light. The second portion of the scene may overlap the first portion of the scene partially, completely, or not at all. The unstructured light and/or the structured light may comprise light invisible to human eye, such as infrared light or ultraviolet light. The illumination unit <NUM> may be implemented e.g. as light-emitting diode (LED). The illumination unit <NUM> is re-attachable to the apparatus <NUM>.

The illumination unit <NUM> may comprise a diffractive optical element (DOE) <NUM> that is configured to provide the structured light. The diffractive optical element <NUM> may be switchable. The diffractive optical element <NUM> may be implemented e.g. as a lens that may be installed e.g. in front of the illumination unit <NUM> so that the light emitting from the illumination unit <NUM> passes through the lens. The diffractive optical element <NUM> may comprise a first part configured to allow the light emitting from the illumination unit <NUM> pass through unaltered, thereby providing the unstructured light. The diffractive optical element <NUM> may further comprise a second part configured to cause predetermined pattern(s) in the light emitting from the illumination unit <NUM>, thereby providing the structured light.

The apparatuses <NUM>, <NUM> may comprise one or more processors <NUM>, <NUM> which may be microprocessors, controllers or any other suitable type of processors for processing computer executable instructions to control the operation of the apparatuses <NUM>, <NUM>. Platform software comprising an operating system <NUM>, <NUM> or any other suitable platform software may be provided at the apparatuses <NUM>, <NUM> to enable application software <NUM>, <NUM> to be executed on the device.

Computer executable instructions may be provided using any computer-readable media that is accessible by the apparatuses <NUM>, <NUM>. Computer-readable media may include, for example, computer storage media such as memory <NUM> and communications media. Computer storage media, such as memory <NUM>, <NUM>, includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device. In contrast, communication media may embody computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transport mechanism. As defined herein, computer storage media does not include communication media. Therefore, a computer storage medium should not be interpreted to be a propagating signal per se. Propagated signals may be present in a computer storage media, but propagated signals per se are not examples of computer storage media. Although the computer storage media (memory <NUM>, <NUM>) is shown within the apparatuses <NUM>, <NUM> it will be appreciated that the storage may be distributed or located remotely and accessed via a network or other communication link.

<FIG> shows a method which can be used to simultaneously illuminate a scene with both unstructured light and structured light. At step <NUM>, a first portion of a scene is illuminated with unstructured light using a single illumination unit, and simultaneously a second portion of the scene is illuminated with structured light using the same illumination unit. At step <NUM> at least one image frame of the illuminated scene is captured using a digital image capture unit.

As discussed above, the second portion of the scene may overlap the first portion of the scene partially, completely, or not at all. The unstructured light and/or the structured light may comprise light invisible to human eye, such as infrared light or ultraviolet light.

<FIG> shows a method which can be used to simultaneously illuminate a scene with both unstructured light and structured light. At step <NUM>, a first portion of a scene is illuminated with unstructured light using a single illumination unit, and simultaneously a second portion of the scene is illuminated with structured light using the same illumination unit.

In the example of <FIG>, the scene comprises a region of interest, and the illuminating step <NUM> comprises illuminating the region of interest with higher power than the rest of the scene. At step <NUM> at least one image frame of the illuminated scene is captured using a digital image capture unit.

In the example of <FIG>, the unstructured light is utilized in iris recognition and the structured light is utilized in depth calculation. At step <NUM> at least one image frame of the illuminated scene is captured using a digital image capture unit.

<FIG> illustrates a scene <NUM> that comprises a human face <NUM>. The first portion <NUM> illuminated with the unstructured light is an area around the eyes and the second portion <NUM> illuminated with the structured light is an area around the first portion. The second portion <NUM> of the scene <NUM> may overlap the first portion <NUM><NUM> of the scene <NUM> partially, completely, or not at all. Here, the unstructured light may be utilized e.g. in iris recognition and the structured light may be utilized e.g. in depth calculation.

The scene may comprise more than two portions, at least some of which may be illuminated with similar structured light as the second portion or with structured light having a different structure than that of the structured light illuminating the second portion. For example, a dense structure may be used on a portion requiring more accuracy and a sparse structure may be used on a portion requiring less accuracy.

<FIG> illustrates illumination of a region of interest with higher power than the rest of the scene. The scene <NUM> comprises a human face <NUM>. In addition to the scene having the first portion illuminated with the unstructured light and the second portion illuminated with the structured light, as discussed above, the scene <NUM> further comprises a region of interest <NUM> (ROI). In <FIG> related to iris recognition, the region of interest <NUM> includes eye area or iris area. The region of interest <NUM> may overlap the first portion of the scene and/or the second portion of the scene.

Curve <NUM> represents power or current used to control the illumination unit or LED. The example of <FIG> relates to a digital image capture unit comprising a rolling shutter. In other words, exposure and frame read-out starts from a top row, and bottom row exposure may start e.g. tens of milliseconds later than the top row one. Accordingly, lines <NUM> represent row exposure start, and lines <NUM> represent row exposure stop. Points <NUM> represent the moments of time when the illumination unit is turned on or to high power (i.e. when the rolling shutter exposure reaches the top of the ROI indicated by the upper dash line), and points <NUM> represent the moments of time when the illumination unit is turned off or to low power (i.e. when the rolling shutter exposure reaches the bottom of the ROI indicated by the lower dash line).

In <FIG>, illumination unit or LED may be turned on and off synchronized to the frame read-out of the digital image capture unit so that only e.g. iris area (ROI) <NUM> is illuminated with high power. Alternatively, instead of turning the IR light on and off, it may be turned to high power and low power. This may be more convenient for the eyes, as illumination source flickering is not as visible if there is some light emitting all the time. For example, substantially <NUM>% power may be used outside the ROI, whereas substantially <NUM>% power may be used within the ROI.

Iris recognition typically utilizes an infrared illumination unit and digital image capture unit matching the IR wavelengths. Typically, near-infrared (R) is used. However, a human eye can usually see also some part of the NIR radiation, so a NIR illumination unit may be visible to users. Especially, if NIR wavelength is close to red color (i.e. close to <NUM>), the NIR illumination unit may actually look like a normal red LED. Furthermore, IR radiation may be harmful for the eye, if power is high and/or exposure time is long.

The example of <FIG> allows reducing the average power emitting from the illumination unit. Reducing average power level may facilitate making e.g. an IR illumination unit less visible/irritating and less harmful for the eyes. Furthermore, if required IR energy for a proper exposure at distance x is <NUM>%, then due to the inverse square law, required IR energy at 2x distance is <NUM>%. Accordingly, being able to reduce the average power emitting from the illumination unit is beneficial.

At least some of the examples disclosed in <FIG> are able to provide simultaneous illumination of a scene with both unstructured light and structured light using a single illumination unit. As discussed above, the unstructured light may be utilized e.g. in iris recognition and the structured light may be utilized e.g. in depth calculation.

This may provide more secure authentication, since on parallel with iris recognition it could be verified that the visible object has a three-dimensional (3D) shape of a face (i.e. it is not e.g. paper or display). Also, utilizing facial 3D information may provide more secure authentication. Furthermore, depth information may be utilized for optimization of image capture parameters, such as to guide auto-focus, auto-exposure and/or illumination unit control. Furthermore, depth information may be utilized for safety decisions, such as for turning off the illumination unit when a face comes closer to the apparatus than a predetermined threshold. Furthermore, depth information may be utilized for power optimizations, such as for detecting when there is nothing in front of the apparatus and in response turning the illumination unit power off/lower. Furthermore, depth information may be utilized for e.g. removing/blurring a background in a video call. Furthermore, at least some of the examples disclosed in <FIG> may be used underwater and/or in darkness, since they do not rely on external light sources, such as sunlight or room lighting.

In an embodiment, alternatively or in addition, the apparatus comprises a mobile communication device.

An embodiment of a computer-readable storage medium not according to the claimed invention comprises executable instructions for causing at least one processor of an apparatus to perform operations comprising: illuminating, with an illumination unit, simultaneously a first portion of a scene with unstructured light and a second portion of the scene with structured light; and capturing, with a digital image capture unit, at least one image frame of the illuminated scene.

The term 'computer' or 'computing-based device' is used herein to refer to any device with processing capability such that it can execute instructions. Those skilled in the art will realize that such processing capabilities are incorporated into many different devices and therefore the terms 'computer' and 'computing-based device' each include mobile telephones (including smart phones), tablet computers and many other devices.

The methods described herein may be performed by software in machine readable form on a tangible storage medium e.g. in the form of a computer program comprising computer program code means adapted to perform all the steps of any of the methods described herein when the program is run on a computer and where the computer program may be embodied on a computer readable medium. Examples of tangible storage media include computer storage devices comprising computer-readable media such as disks, thumb drives, memory etc. and do not include propagated signals. Propagated signals may be present in a tangible storage media, but propagated signals per se are not examples of tangible storage media. The software can be suitable for execution on a parallel processor or a serial processor such that the method steps may be carried out in any suitable order, or simultaneously.

This acknowledges that software can be a valuable, separately tradable commodity.

Those skilled in the art will realize that storage devices utilized to store program instructions can be distributed across a network. For example, a remote computer may store an example of the process described as software. A local or terminal computer may access the remote computer and download a part or all of the software to run the program. Those skilled in the art will also realize that by utilizing conventional techniques known to those skilled in the art that all, or a portion of the software instructions may be carried out by a dedicated circuit, such as a DSP, programmable logic array, or the like.

Any range or device value given herein may be extended or altered without losing the effect sought, as will be apparent to the skilled person, provided that they fall under the scope of the appended claims.

The steps of the methods described herein may be carried out in any suitable order, or simultaneously where appropriate. Additionally, individual blocks may be deleted from any of the methods without departing from the spirit and scope of the subject matter described herein.

Claim 1:
A method of capturing a face image, comprising:
illuminating, with an illumination unit, simultaneously a first portion of a scene with unstructured light and a second portion of the scene with structured light;
capturing, with a digital image capture unit that includes a rolling shutter, at least one image frame of the illuminated scene; and
via the illumination unit, selectively adjusting illumination of the scene based on a position of the rolling shutter to illuminate a region of interest of the scene, wherein the region of interest includes eye or iris area, with higher power than the rest of the scene by operating the illumination unit at a first, higher power setting responsive to an exposure of the rolling shutter being within the region of interest and by operating the illumination unit at a second, reduced power setting responsive to an exposure of the rolling shutter being outside the region of interest.