Patent Description:
Digital image capture devices, such as cameras in cell phones and smart devices, use various signal processing techniques in an attempt to render high quality images. For example, these image capture devices automatically focus their lens for image sharpness, automatically set the exposure time based on light levels, and automatically adjust the white balance to accommodate for the color temperature of a light source. In some examples, image capture devices include facial detection technology. Facial detection technology allows the image capture device to identify faces in a field of view of an image capture device's lens. The image capture device may then apply the various signal processing techniques based on the facial identifications.

US patent application <CIT> discloses parsing a voice command from a user of an image capture device to determine an image capture goal of the user, extracting image features from captured image data of a scene including a target subject and generating an image capture settings adjustment advice based on the image capture goal.

US patent application <CIT> teaches processing viewfinder image data to detect faces, comparing the detected faces to reference faces ranked and stored in memory of an image capture device, setting a region of interest in the viewfinder image to include a highest-ranked detected face, and using the region of interest to set a configuration parameter, such as focus or exposure.

US patent application <CIT> discloses a method for enabling the autofocus feature of a camera to be directed towards priority targets within a frame.

US patent application <CIT> discloses a method of setting the focus of a digital image based on social relationship. An identity of an individual whose face was detected in an imaged scene is recognized by determining that the detected face is the face of an individual having a social relationship with the user of the electronic device.

The present disclosure provides a method for controlling an image capture device according to claim <NUM>, an image capture device according to claim <NUM>, and a non-transitory computer-readable storage medium according to claim <NUM>. Specific embodiments are subject of the dependent claims.

While the present disclosure is susceptible to various modifications and alternative forms, specific embodiments are shown by way of example in the drawings and will be described in detail herein. The objectives and advantages of the claimed subject matter will become more apparent from the following detailed description of these exemplary embodiments in connection with the accompanying drawings. It should be understood, however, that the present disclosure is not intended to be limited to the particular forms disclosed.

Many cameras are equipped to identify faces in the field of view (FOV) of the camera, and select a lens position that provides the best focus value for a region of interest (ROI) containing all of the identified faces. Many times, however, the selected lens position does not result in an optimal captured image for a particular face or subset of the faces in an image containing several faces.

This disclosure provides an image capture device having one or more of priority subject-based automatic focus (AF), automatic exposure (AE), or automatic white balance (AWB) control, and corresponding methods. The image capture device can identify subjects within its field of view (FOV), and determine whether any one or more of the identified subjects is a priority subject. A priority subject can be, for example, an owner or other primary user of the image capture device, or another person or animal designated as a priority subject, in a manner described below. If any one of the identified subjects is a priority subject, the image capture device can determine a region of interest (ROI) that includes the priority subjects. For example, the region of interest can include the priority subjects and exclude all subjects other than priority subjects. The image capture device can then adjust one or more of AF, AE, or AWB of the image capture device based on image data within the ROI, which includes the priority subject(s). In this description, unless expressly stated otherwise, the ROI refers to the region of interest that is used during AF, AE, and/or AWB.

In this description, the terms "priority subject" and "prioritized person" are used interchangeably to refer to person or animal (e.g., a pet), identified by a user of the camera as being prioritized, and for whom the image capture device contains one or more images to support face recognition. A prioritized person is an example of a priority subject, and in the examples described herein, the image capture device is equally capable of applying the same methods to animals, or statues or paintings including faces.

Among other advantages, the image capture device is capable of providing automated image capture enhancements in consideration of lighting conditions affecting the priority subjects. For example, the image capture device can automatically optimize one or more of AF, AE, or AWB based on image data within its field of view that represents priority subjects. For example, a primary user of the image capture device can be a predetermined priority subject. When the primary user of the image capture device takes a picture of themselves (e.g., a selfie), the AF, AE, and/or AWB settings are optimized for the ROI containing the primary user of the image capture device, regardless of others who may appear in the picture. Instead of selecting a compromise lens position that minimizes total focusing error of all the faces in the FOV, the lens position is selected to minimize the focusing error of the prioritized face(s). Thus, priority subjects receive optimal camera settings when the image capture device takes their picture. For example, when using the front camera for a selfie, the image capture device can ensure that the owner of the camera is in focus, even if other faces in the image are out of focus. Similarly, exposure and/or white balance may also be optimized for the priority subjects instead of averaged or otherwise weighted across other subjects.

<FIG> is a block diagram of an exemplary image capture device <NUM>. The functions of image capture device can be implemented in one or more processors, one or more field-programmable gate arrays (FPGAs), one or more application-specific integrated circuits (ASICs), one or more state machines, digital circuitry, any other suitable circuitry, or any suitable hardware. In this example, image capture device <NUM> includes at least one processor <NUM> that is operatively coupled to (e.g., in communication with) camera optics and sensor <NUM> for capturing images. Processor <NUM> is also operatively coupled to instruction memory <NUM>, working memory <NUM>, input device <NUM>, and storage medium <NUM>. Input device <NUM> can be, for example, a keyboard, a touchpad, a stylus, a touchscreen, or any other suitable input device. In some examples, processor <NUM> is also operatively coupled to one or more of Global Positioning System (GPS) unit <NUM>, transceiver <NUM>, and display <NUM>.

The image capture device <NUM> can be implemented in a computer with image capture capability, a special-purpose camera, a multi-purpose device capable of performing imaging and non-imaging applications, or any other suitable device. For example, image capture device <NUM> can be a portable personal computing device such as a mobile phone, digital camera, tablet computer, laptop computer, personal digital assistant, or any other suitable device.

Although this description refers to processor <NUM>, in some examples processor <NUM> can include one or more processors. For example, processor <NUM> can include one or more central processing units (CPUs), one or more graphics processing units (GPUs), one or more digital signal processors (DSPs), one or more image signal processors (ISPs), one or more device processors, and/or one or more of any other suitable processors. In some examples, processor <NUM> can include different types of processors. For example, processor <NUM> can include an image signal processor <NUM> and a device processor <NUM>. In this example, image signal processor <NUM> can perform various image capture operations on received image data to execute AF, AE, and/or AWB. Device processor <NUM> can perform various management tasks such as controlling display <NUM> to display captured images, or writing to or reading data from working memory <NUM> or storage medium <NUM>. Device processor <NUM> can also configure image capture parameters that can be used by image signal processor <NUM> to capture images, such as AF, AE, and/or AWB parameters. Although in <FIG> processor <NUM> is located within image capture device <NUM>, in some examples, processor <NUM> can include one or more cloud-distributed processors. For example, one or more of the functions described below with respect to processor <NUM> can be carried out (e.g., performed) by a remote processor, such as a cloud processor <NUM> within a cloud-based server, where the cloud processor <NUM> can be connected to the processor <NUM> via a network <NUM>. The cloud processor <NUM> can be coupled to non-transitory cloud storage media <NUM>, which may be collocated with, or remote from, the cloud processor <NUM>. The network <NUM> can be any personal area network (PAN), local area network (LAN), wide area network (WAN) or the Internet.

Camera optics and sensor <NUM> can include one or more image sensors and one or more lenses to capture images. Processor <NUM> can control camera optics and sensor <NUM> to capture images. For example, processor <NUM> can instruct camera optics and sensor <NUM> to initiate an image capture (e.g., take a picture), and can receive the captured image from camera optics and sensor <NUM>. Camera optics and sensor <NUM>, storage <NUM>, and the processor <NUM> provide a means for capturing second image data from the front facing camera based on the at least one of automatic focus, automatic exposure, or automatic white balance using the selected region of interest.

Instruction memory <NUM> can store instructions that can be accessed (e.g., read) and executed by processor <NUM>. For example, instruction memory <NUM> can include read-only memory (ROM) such as electrically erasable programmable read-only memory (EEPROM), flash memory, a removable disk, CD-ROM, any non-volatile memory, or any other suitable memory.

Processor <NUM> can store data to, and read data from, working memory <NUM>. For example, processor <NUM> can store a working set of instructions to working memory <NUM>, such as instructions loaded from instruction memory <NUM>. Processor <NUM> can also use working memory <NUM> to store dynamic data created during the operation of image capture device <NUM>. Working memory <NUM> can be a random access memory (RAM) such as a static random access memory (SRAM) or dynamic random access memory (DRAM), or any other suitable memory.

In this example, instruction memory <NUM> stores capture control instructions <NUM>, AF instructions <NUM>, AWB instructions <NUM>, AE instructions <NUM>, image processing instructions <NUM>, subject detector instructions <NUM>, priority subject selector instructions <NUM>, ROI adjuster instructions <NUM>, and operating system instructions <NUM>. Instruction memory <NUM> can also include additional instructions that configure processor <NUM> to perform various image processing and device management tasks.

AF instructions <NUM> can include instructions that, when executed by processor <NUM>, cause a lens of camera optics and sensor <NUM> to adjust its lens position. For example, processor <NUM> can cause a lens of camera optics and sensor <NUM> to adjust so that light from a region of interest within a field of view (FOV) of the imaging sensor is focused in a plane of the sensor. The selected ROI can correspond to one or more focus points of the AF system. AF instructions <NUM> can include instructions for executing autofocus functions, such as finding the optimal lens position for bringing light from a region of interest into focus in the plane of a sensor. Autofocus can include, for example, phase detection autofocus (PDAF), contrast autofocus, or laser autofocus. PDAF divides incoming light into pairs of images and captures the divided light rays coming from the opposite sides of the lens, creating a rangefinder. The two images are then analyzed to find a separation (phase) error and determine whether the region of interest of the sensor is in focus in the sensor's plane. Contrast AF moves a lens through its position range, stopping at a point where maximal contrast is detected between adjacent pixels at an edge in the FOV. Laser AF emits a light from a laser or light emitting diode (LED) on the subject and calculates a distance to the subject based on how long it takes the light to reach the subject and return.

AWB instructions <NUM> can include instructions that, when executed by processor <NUM>, cause processor <NUM> to determine a color correction to be applied to an image. For example, the AWB instructions <NUM>, when executed by processor <NUM>, can cause processor <NUM> to determine an average color temperature of the illuminating light source under which an image was captured, and to scale color components (e.g., R, G, and B) of the image so they conform to the light in which the image is to be displayed or printed. In some examples, the AWB instructions <NUM>, when executed by processor <NUM>, can cause processor <NUM> to determine the illuminating light source in a region of interest of the image. The processor <NUM> can then apply a color correction to the image based on the determined color temperature of the illuminating light source in the region of interest of the image.

AE instructions <NUM> can include instructions that, when executed by processor <NUM>, cause processor <NUM> to determine the length of time that sensing elements, such as an imaging sensor of camera optics and sensor <NUM>, integrate light before an image is captured. For example, processor <NUM> can meter ambient light, and select an exposure time for a lens based on the metering of the ambient light. As the ambient light level increases, the selected exposure time becomes shorter. As the ambient light level decreases, the selected exposure time becomes longer. In the case of a digital single-lens reflex (DSLR) camera, for example, AE instructions <NUM>, when executed, can determine the exposure speed. In some examples, processor <NUM> can meter the ambient light in a region of interest of the field of view of a sensor of camera optics and sensor <NUM>.

A 3A engine <NUM> provides a means for performing at least one of automatic focus, automatic exposure, or automatic white balance of the image capture device based on using the selected region of interest. In some examples of image capture devices <NUM>, the three sets of instructions, AF instructions <NUM>, AWB instructions <NUM> and AE instructions <NUM>, are included in "3A" engine <NUM>. The 3A engine <NUM> can include instructions that cause processor <NUM> to operate on raw image sensor data measured by an image sensor of camera optics and sensor <NUM> prior to capturing an image.

Capture control instructions <NUM> can include instructions that, when executed by processor <NUM>, configure processor <NUM> to adjust a lens position, set an exposure time, set a sensor gain, and/or configure a white balance filter of the image capture device <NUM>. Capture control instructions <NUM> can further include instructions that, when executed by processor <NUM>, control the overall image capture functions of image capture device <NUM>. For example, capture control instructions <NUM>, when executed by processor <NUM>, can cause processor <NUM> to execute AF instructions <NUM> to calculate a lens or sensor movement to achieve a desired autofocus position and output a lens control signal to control a lens of camera optics and sensor <NUM>.

Image data of priority subjects can be stored in a means for storing at least one priority subject image-indicative data corresponding to one or more priority subjects. The means for storing can be a non-transitory, machine-readable storage medium <NUM> or cloud storage <NUM>. A priority subject can include, for example, an owner or other user of the image capture device <NUM>, a person, an animal, or an object. Processor <NUM> can determine whether detected subjects are predetermined priority subjects based on previously stored image data corresponding to predetermined prioritized persons (e.g., the primary user, the owner, the owner's child or, or the owner's spouse) or animals (e.g., the owner's pet). Image data corresponding to a predetermined prioritized person can be stored, for example, in storage medium <NUM>. In some examples, a user of image capture device <NUM> can provide image data associated with predetermined priority subjects to image capture device <NUM> by capturing one or more images of the subject with the image capture device <NUM> and designating the subject of the images as a priority subject, or by downloading images of the priority subject from another device or network.

The means for storing can be a local storage medium <NUM>, such as a hard drive, a solid-state memory, or a FLASH memory, for example. The means for storing can also be a remote storage medium, such as cloud storage <NUM>, which can be embodied in a cloud-based server memory, a memory device on another image capture device <NUM>, a networked computer, or any other suitable remote storage device. Storage medium <NUM> can include priority subject data <NUM> comprising image data for one or more predetermined prioritized persons. For example, priority subject data <NUM> can include one or more images for each prioritized person. In some examples, priority subject data <NUM> includes multiple images corresponding to multiple views for each prioritized person, for example, a plurality of images from a plurality of different angles. For example, the multiple images can include a front image of the predetermined prioritized person, a right side image of the predetermined prioritized person, and a left side image of the predetermined prioritized person.

In the examples described below, the priority subject data <NUM> includes image data. In other embodiments, the priority subject data <NUM> can include other types of priority subject image-indicative data, which are indicative of image data, where the priority subject image-indicative data are based on (e.g., extracted from, derived from or representative of) image data of a priority subject. For example, the priority subject data <NUM> can include a plurality of extracted facial features and/or fiducial-point-based face graph data for the priority subject. In other embodiments the priority subject image-indicative data in priority subject data <NUM> can include holistic data for the priority subject, such as an array of intensity values or dimensionality-reduced eigenface picture data in a principle coordinate (latent variable) system. In other embodiments, the priority subject image-indicative data of priority subject data <NUM> can include coefficient data for a neural network trained to identify members of a predetermined set of priority subjects, such as faces. These are only examples of priority subject image-indicative data used in exemplary pattern recognition (e.g., face recognition) techniques, and are not exclusive. For brevity, the term, "image-indicative data" is used below to refer to image data and/or other forms of data based on, extracted from, or representative of image data. The image-indicative data include, but are not limited to, data to which a pattern recognition technique can be applied, as well as RAW, intermediate or partially-processed image sensor data.

For example, if the priority subject image-indicative data comprise extracted facial features and/or fiducial-point-based face graph data, then the pattern recognition (e.g., facial recognition) includes extracting facial features and/or fiducial point based graph data from the image data of the subject(s) in the FOV of the image capture device for comparison with the priority subject image-indicative data in the priority subject data.

If the priority subject image-indicative data comprise an array of intensity values, then the facial recognition includes determining an array of intensity values based on the image data from the subject(s) in the FOV of the image capture device for comparison with the priority subject image-indicative data in the priority subject data.

If the priority subject image-indicative data comprise dimensionality-reduced eigenface picture data in a principle coordinate (latent variable) system, then the facial recognition includes transforming image data from the subject(s) in the FOV of the image capture device to the principal coordinates (latent variables) for comparison with the priority subject image-indicative data in the priority subject data.

If the priority subject image-indicative data include coefficient data for a neural network trained to identify members of a predetermined set of priority subjects' faces then the facial recognition includes inputting the image data from the subject(s) in the FOV of the image capture device to the neural network and receiving from the neural network a value indicating a probability that the subject is one of the priority subjects.

In the example of <FIG>, the priority subject data <NUM> is stored locally in the image capture device <NUM>. In other embodiments, the priority subject data <NUM> is stored remotely. For example, the priority subject data can be stored in cloud storage <NUM> coupled to cloud processor <NUM>, and the processor <NUM> can access the priority subject data using a communications protocol, such as hypertext transport protocol (HTTP). In other embodiments, the priority subject data <NUM> is initially stored locally in image capture device <NUM>, and is copied to cloud storage <NUM>. If the user has multiple image capture devices, the user can access the priority subject data in from cloud storage. In some embodiments, face detection and/or face recognition are initiated by processor <NUM> and performed remotely by cloud processor <NUM>, using the copy of priority subject data <NUM> in cloud storage <NUM>.

Subject detector block <NUM> provides means for initiating facial detection on first image data representing one or more subjects within a field of view of the image capture device to detect one or more faces. Subject detector instructions <NUM> can include instructions that, when executed by processor <NUM>, cause processor <NUM> to detect one or more subjects in a field of view of a lens of camera optics and sensor <NUM> as being faces. For example, processor <NUM> can obtain raw image sensor data of an image in a field of view of a lens of camera optics and sensor <NUM>. In some embodiments, processor <NUM> can initiate face detection and determine if one or more subjects are in the field of view by, for example, performing the facial detection locally within processor <NUM>. In other embodiments, processor <NUM> can initiate remote performance of face detection by transmitting a request to a cloud processor <NUM> or other remote server. The request causes the cloud processor <NUM> or other remote server to perform the computations to determine if the field of view of image capture device <NUM> contains one or more faces, and respond to the processor <NUM> with identification of region(s) in the FOV containing a face.

Priority subject selector block <NUM> provides means for performing facial recognition on the first image data in response to obtaining the first image data from the camera (e.g., front facing camera) and detection of a face within the first image data. Priority subject selector <NUM> determines whether a recognition criterion is met between the image-indicative data (e.g., image data) of a detected person in the FOV and previously stored priority subject image-indicative data (e.g., image data) of a predetermined priority subject. For example, the recognition criterion may specify a threshold probability that the person in the FOV is a priority subject for whom corresponding priority subject image-indicative data (e.g., image data) are stored in the priority subject data <NUM>. Priority subject selector <NUM> can identify one or more priority subjects. For example, priority subject selector <NUM> can identify one of at least two detected persons as a priority subject. As another example, priority subject selector <NUM> can identify two of at least two detected subjects as priority subjects. More generally, priority subject selector <NUM> can select any subset of a plurality of detected persons or animals as priority subjects.

Priority subject selector <NUM> initiates a facial recognition process to determine whether any face detected by the subject detector <NUM> is a priority subject. In some embodiments, priority subject selector <NUM> can include initiation instructions that, when executed by processor <NUM>, cause processor <NUM> to locally determine (e.g., identify) whether any of the subjects detected is a priority subject for whom priority subject image-indicative data (e.g., images) have been previously stored. In other embodiments, priority subject selector <NUM> can include initiation instructions that, when executed by processor <NUM>, cause processor <NUM> to transmit a request to the remote cloud processor <NUM>. The request causes the cloud processor <NUM> to determine whether any of the detected subjects is a priority subject for whom priority subject image-indicative data have been previously stored and to respond to processor <NUM> with identification(s) of any recognized priority subject.

For example, processor <NUM> can determine if any detected subject in the region of interest is a previously-identified priority subject identified in priority subject data <NUM> stored in the image capture device <NUM>. Processor <NUM> can use priority subject image-indicative data (e.g., image data) corresponding to predetermined priority subjects to make the determination. For example, processor <NUM> can compare image data of the subjects detected to image data of a predetermined priority subject using, for example, facial recognition techniques. For example, processor <NUM> can compare image data of the detected subjects (or data extracted or derived therefrom) to image data of primary user (e.g., owner) of the image capture device <NUM>.

ROI adjuster instructions <NUM> are executed in processor <NUM> to provide a means for selecting a region of interest of the image data corresponding to the one or more subjects determined to be priority subjects. Processor <NUM> can determine a region of interest of the field of view of the lens of camera optics and sensor <NUM> that includes any areas of the image determined to include detected subjects. ROI adjuster instructions <NUM> can include instructions that, when executed by processor <NUM>, cause processor <NUM> to adjust a region of interest (e.g., such as one determined by processor <NUM> executing subject detector instructions <NUM>) to include only one or more detected subjects determined to be priority subjects from the priority subject data <NUM>. For example, ROI adjuster instructions <NUM> can adjust a region of interest (for performing AF, AE and/or AWB) to include only detected subjects that were also identified priority subjects in the priority subject data <NUM>. In other words, the region of interest (for performing AF, AE and/or AWB) can exclude all subjects other than priority subjects. This allows the prioritized persons/subjects to have the best focus, exposure and white balance, even if there are other subjects in the FOV closer to the image capture device than the prioritized persons/subjects.

For example, in an image containing a large group of people in multiple rows, with a priority subject near an end of the back row, the image capture device will autofocus on the priority subject. In another example, in a dark room (or a brightly lit space) containing a priority subject, the image capture device will set the exposure speed so the priority subject has mid-tone exposure. In another example, the image capture device identifies the neutral tones (the whites, grays, and blacks) in the ROI containing a priority subject and then calibrates the rest of the image to the temperature of the neutral colors in the ROI containing the priority subject.

If the FOV contains one or more priority subjects, the priority subject detector <NUM> adjusts the ROI for AF, AE, and/or AWB to include the priority subject(s) and exclude other detected faces from consideration during AF, AE and/or AWB. (The adjustment of the ROI for AF, AE, and/or AWB can affect the lens position, exposure speed and/or white balance, but does not exclude any area within the FOV from the image captured.

Processor <NUM> can use the adjusted region of interest to perform one or more of AF, AWB, or AE. For example, ROI adjuster instructions <NUM> can cause processor <NUM> to use the adjusted region of interest as the region of interest when executing AF instructions <NUM>. Similarly, ROI adjuster instructions <NUM> can cause processor <NUM> to use the adjusted region of interest as the region of interest when executing AWB instructions <NUM> or AE instructions <NUM>.

In some examples, subject detector instructions <NUM>, priority subject selector instructions <NUM>, and ROI adjuster instructions <NUM> are included in a subject based ROI engine <NUM>. Subject based ROI engine <NUM> can be executed by, for example, image signal processor <NUM>, or a dedicated digital signal processor (DSP) (not shown). Subject based ROI engine <NUM>, when executed by one or more processors, can cause the one or more processors to identify an ROI such that the one or more processors, when executing 3A engine <NUM>, perform at least one of automatic focus, automatic exposure, or automatic white balance of image capture device <NUM> based on the adjusted ROI.

In some examples, subject based ROI engine <NUM> is activated based on activation of a camera or image capture device <NUM>. For example, processor <NUM> can detect when a user activates camera optics and sensor <NUM>. Upon detecting activation, processor <NUM> can execute subject based ROI engine <NUM>. In some examples, detecting activation of a camera or image capture device <NUM> includes detecting activation of a front camera of the image capture device <NUM> located on the same side of the device as the display (e.g., a front camera of a mobile phone). Activation of the front camera is an indication that the user intends to take a selfie, and can be used as a criterion for activating the subject based ROI engine <NUM>.

In some examples, subject based ROI engine <NUM> is activated based on fingerprint detection of the primary user of image capture device <NUM>. For example, fingerprint detection device <NUM> can indicate to processor <NUM> when a user has successfully authenticated themselves via a fingerprint authentication. Upon detecting successful authentication, processor <NUM> executes subject based ROI engine <NUM>. Other forms of biometric authentication are also contemplated, such as retina scanning or voice recognition systems and methods, and other systems and methods as well. For example, subject based ROI engine <NUM> can be automatically activated when the camera is active and the user performs a fingerprint authentication or speaks a predetermined "priority subject" command for activating subject based ROI engine <NUM>.

Image processing instructions <NUM> can include instructions that cause processor <NUM> to execute one or more image processing functions such as demosaicing, noise reduction, cross-talk reduction, color processing, gamma adjustment, image filtering (e.g., spatial image filtering), lens artifact or defect correction, image sharpening, or other image processing functions.

Operating system instructions <NUM> can include instructions that, when executed by processor <NUM>, cause processor <NUM> to implement an operating system. The operating system can act as an intermediary between programs, such as user applications, and the processor <NUM>. Operating system instructions <NUM> can include device drivers to manage hardware resources such as the camera optics and sensor <NUM>, display <NUM>, GPS unit <NUM>, or transceiver <NUM>. Instructions contained in image processing instructions <NUM> discussed above can interact with hardware resources indirectly, through standard subroutines or application program interfaces (APIs), that can be included in operating system instructions <NUM>. Operating system instructions <NUM> can then interact directly with these hardware components. Operating system block <NUM> can further configure the image signal processor <NUM> to share information with device processor <NUM>.

<FIG> is an illustrative flow chart of an exemplary method <NUM> for selecting a region of interest in a field of view of an image capture device, such as the image capture device <NUM> of <FIG>. The method can be carried out by, for example, the ROI engine <NUM> of <FIG>.

At block <NUM>, one or more subjects among the image sensor data <NUM> in a field of view of an image capture device are detected to provide a region of interest (ROI). In some embodiments, block <NUM> determines whether the FOV contains a face. Block <NUM> can be a face detection module.

At block <NUM>, the detected face(s) (or data extracted or derived therefrom) is (are) compared to the priority subject image-indicative data (e.g., images of faces) in the priority subject data <NUM>, and a determination is made as to whether any of the regions of interest includes any priority subject(s). For example, facial recognition may be applied by comparing extracted features, fiducial measurements, or the like. If the region of interest does not include any priority subjects, execution proceeds to block <NUM>. If the region of interest includes any priority subjects, the method proceeds to block <NUM>.

At block <NUM>, the regions of interest can be set to include all faces detected by the face detection block <NUM>. This causes the AF to select the optimum lens position for all of the detected faces. For example, the total focus error for all the faces in the image is minimized (but the individual focus error for each individual image is not necessarily minimized).

At block <NUM>, the region of interest is adjusted based on the priority subjects. For example, the region of interest for AF, AE, and/or AWB is adjusted to include subjects determined to be prioritized persons and to exclude other faces within the FOV. This will cause the camera to select a lens position, exposure speed, and/or color temperature optimized for the prioritized faces. The adjusted region of interest is provided as the output region of interest.

<FIG>, and <FIG> illustrate an exemplary image preview <NUM> within a field of view of the exemplary image capturing device <NUM> of <FIG>. The FOV may contain a single subject, or two or more subjects. The image preview <NUM> includes faces of a first person <NUM>, second person <NUM>, and third person <NUM>. Image capture device <NUM> can perform facial detection and recognition on image data associated with image <NUM> to identify detected subjects.

For example, <FIG> illustrates a region of interest <NUM> that includes image data for all identified subjects (faces) including faces of first person <NUM>, second person <NUM>, and third person <NUM>. If used for AF, AE, and AWB, the ROI <NUM> as shown in <FIG> is optimized for all three persons <NUM>, <NUM> and <NUM>. The lens position is selected to minimize the total focus error among all three faces, but does not guarantee the best possible focus for any individual one of the three faces. Similarly, the exposure is selected to minimize the total exposure error among all three faces, but does not guarantee the best possible exposure for any individual one of the three faces. And the white balance is selected to minimize the total color temperature error among all three faces, but does not guarantee the best possible color temperature for any individual one of the three faces.

For the example of <FIG>, first person <NUM> is a predetermined prioritized person, for whom the priority subject data <NUM> (prioritized person list) includes one or more priority subject image-indicative data (e.g., images). The remaining subjects <NUM> and <NUM> are not prioritized persons. For example, image capturing device <NUM> can compare features of the face of person <NUM> to images stored in the prioritized person list and identify image data corresponding to first person <NUM> as a predetermined prioritized person. Thus, an ROI <NUM> as shown in <FIG> can result in an image quality for the prioritized person <NUM> that is not the highest possible image quality that the image capture device <NUM> is capable of capturing.

<FIG> identifies an adjusted region of interest <NUM> of the image <NUM> that includes predetermined prioritized person from the priority subject data <NUM>. For example, adjusted region of interest <NUM> includes image data corresponding to first person <NUM>, but not second person <NUM> nor third person <NUM>. Image capture device <NUM> can adjust at least one of automatic focus, automatic exposure, or automatic white balance based on the adjusted region of interest <NUM>. Using the ROI <NUM>, the image capture device <NUM> can capture an image with the best image quality for the face of the first person <NUM>, and permit a possible slight reduction in image quality of the other subjects <NUM>, <NUM>.

<FIG>, <FIG> illustrate an exemplary image preview <NUM> in a field of view of the exemplary image capturing device <NUM> of <FIG>. The image <NUM> includes images of a first person <NUM>, second person <NUM>, and third person <NUM>. Image capture device <NUM> can perform facial detection and recognition on image data associated with image <NUM> to identify detected subjects.

For example, <FIG> illustrates a region of interest <NUM> that includes image data for all identified subjects including first person <NUM>, second person <NUM>, and third person <NUM>. For this example, assume that first person <NUM> and third person <NUM> are each predetermined prioritized persons, but the second person <NUM> is not included in the priority subject data <NUM>. For example, image capture device <NUM> can include image data or other priority subject image-indicative data corresponding to first person <NUM> and third person <NUM> in the priority subject data <NUM>. If used for AF, AE, and AWB, the ROI <NUM> as shown in <FIG> is optimized for all three persons <NUM>, <NUM> and <NUM>. The lens position is selected to minimize the total focus error among all three faces <NUM>, <NUM> and <NUM>, but does not guarantee the best possible focus, exposure or white balance for persons <NUM> and <NUM>. Thus, the ROI <NUM> could increase the image quality of the subject <NUM> at the expense of the priority subjects <NUM> and <NUM>.

<FIG> identifies an adjusted region of interest <NUM> of the image <NUM> that includes predetermined prioritized persons <NUM> and <NUM>. For example, adjusted region of interest <NUM> includes image data corresponding to the faces of first person <NUM> and third person <NUM>, but not second person <NUM>. Image capture device <NUM> can adjust at least one of automatic focus, automatic exposure, or automatic white balance based on the adjusted region of interest <NUM>. Adjusted region of interest <NUM> can provide the optimum focus, exposure and white balance with minimum total error for the faces of the prioritized persons <NUM>, <NUM>.

In some embodiments, the processor <NUM> can train a neural network with several images of the primary user, and use the neural network to determine whether the face in the FOV of the image capture device <NUM> belongs to the primary user. For example, the processor <NUM> can use unsupervised learning to train a variational autoencoder with images of the priority subject, and use outlier detection to determine if a subject in the FOV of the camera belongs to the class of the training subject (corresponding to the priority subject). The outlier detection can use a random sampling consensus (RANSAC) or random forest technique.

In some examples, processor <NUM> compares image data (or data extracted or derived therefrom) for each detected subject in the FOV to previously stored priority subject image-indicative data (e.g., image data) of priority subjects (e.g., persons or animals) in a priority subject data <NUM>. (As discussed herein, the priority subject data <NUM> identifies one or more subjects for whom the storage medium <NUM> stores priority subject image-indicative data (e.g., one or more images or images plus text), and who are to be given priority when selecting the ROI for an AF, AE or AWB operation. ) The priority subject data <NUM> can include priority subject image-indicative data (e.g., image data) for one or more predetermined prioritized persons, and can be stored as priority subject data <NUM> in the non-transitory, machine readable storage medium <NUM>, for example. The priority subject data <NUM> can include a plurality of training images (or training data corresponding to other priority subject image-indicative data) for each predetermined priority subject. The priority subject data <NUM> may also include text, such as a name and/or bibliographic data associated with one or more of the prioritized subjects.

In some embodiments, the priority subject data <NUM> has a single priority level; faces in the priority subject data <NUM> are given priority over faces that are not included in the priority subject data <NUM>. In other embodiments, the priority subject data <NUM> has multiple priority levels. If the FOV contains persons included in priority subject data <NUM> with two or more different priority levels and/or persons who are not included in the priority subject data <NUM>, then the ROI for AF, AE and AWB can be selected to only include the face of the person with the highest priority level. The persons with lower priority levels and persons not included in the priority subject data <NUM> can be excluded from the ROI.

<FIG> is a flow chart of an illustrative method <NUM> useful for the better understanding of the present invention that can be carried out by, for example, image capture device <NUM>. In the example of <FIG>, the image capture device <NUM> is configured to detect and recognize when the camera's primary user (e.g., owner) is within the FOV, and to use an ROI narrowly enclosing the face of the camera primary user for AF, AE and AWB, regardless of whether the FOV contains additional people. This will cause the primary user to have the best image quality possible, even if there are other faces closer to the camera than the primary user.

At block <NUM>, an priority subject image-indicative data (e.g., an image) of the primary user's face is stored in a non-transitory, machine-readable storage medium <NUM> (<FIG>) in the image capture device <NUM>. For example, the image can be captured via a selfie and stored in storage medium <NUM>. Alternatively, the image can be received from an external camera (not shown) and saved in the photographs folder (not shown) of the image capture device <NUM>. In some embodiments, the image capture device <NUM> has a folder containing priority subject data <NUM> (<FIG>) for storing priority training images of the primary user.

At block <NUM>, an image capture device <NUM>, detects activation of a front camera on a front face of the image capture device, where the front face has a display. For example, if the image capture device <NUM> is a mobile phone, the front camera is the camera on the face of the phone having the display. In response to activation of the camera located on the front face of the image capture device, block <NUM> initiates a "selfie mode", in which the image capture device <NUM> searches for priority subjects in the FOV of the camera.

At block <NUM>, the subject detector <NUM> (<FIG>) applies face detection and facial recognition to search for at least one face within the first image data within the FOV (i.e., preview image data). While operating in the selfie mode, the priority subject selector <NUM> attempts to match the detected face in the first image data within the FOV of the image capture device against the face of the primary user. In other illustrative examples, the priority subject selector <NUM> attempts to match the detected face in the first image data within the FOV of the image capture device against faces of all priority users.

At block <NUM>, a determination is made as to whether the face of the primary user was recognized. For example, the priority subject selector <NUM> can compare a face within the first image data within the FOV against one or more stored images of the primary user's face stored in the storage medium <NUM>. If the primary user is not recognized, execution proceeds to block <NUM>. If the primary user is detected, the method proceeds to block <NUM>.

At block <NUM>, the ROI used for at least one of automatic focus, automatic exposure, or automatic white balance is adjusted based on identification of one or more faces within the field of view of the device that includes the primary user's face. For example, one or more of automatic focus, automatic exposure, or automatic white balance parameters can be adjusted. Then AF, AE and/or AWB is performed using the adjusted parameters.

At block <NUM>, one or more preview settings are adjusted in accordance with the new parameters computed by AF, AE and/or AWB based on the adjusted ROI.

At block <NUM>, the image capture device <NUM> captures a second image data within the FOV of the image capture device <NUM>, based on the automatic focus, automatic exposure, or automatic white balance using the selected region of interest. For example, if the primary user is recognized by the priority subject selector <NUM> (<FIG>), the image of the primary user within the FOV is captured using the one or more adjusted automatic focus, automatic exposure, or automatic white balance.

<FIG> is a flowchart <NUM> of an illustrative method not claimed in the appended claims that can be carried out by, for example, the image capture device <NUM> of <FIG>. In the example of <FIG>, the image capture device <NUM> has a priority subject data <NUM> containing a set of images or other priority subject image-indicative data showing any desired number of prioritized persons (e.g., members of the primary user's family, the family pet). In some embodiments, the priority subject data <NUM> contains a plurality of images or other priority subject image-indicative data for each priority subject. In other embodiments, the priority subject data <NUM> also includes bibliographic information associated with each of the priority subjects. The ROI adjuster <NUM> (<FIG>) adjusts the ROI to include each person in the priority subject data <NUM> within the FOV, and to exclude any persons who are not in the priority subject data <NUM>.

At block <NUM>, a region of interest is determined based on all subjects in a field of view of a camera. For example, if subject detector <NUM> (<FIG>) determines that there are a plurality of faces (e.g., faces <NUM>, <NUM>, <NUM>, <FIG>) in the FOV of the image capture device <NUM>, the ROI is set to ROI <NUM>, so as to include all of the identified faces in the FOV.

At block <NUM>, subject detector <NUM> (<FIG>) determines whether any subjects in the field of view of the camera have priority. For example, subject detector <NUM> compares image data of the subjects in the region of interest (or data extracted or derived therefrom) to priority subject image-indicative data (e.g., image data) of predetermined prioritized persons in a priority subject data <NUM>. If no subjects have priority, the method proceeds to block <NUM>. Otherwise, if any subjects have priority, the method proceeds to block <NUM>.

At block <NUM>, ROI adjuster <NUM> (<FIG>) adjusts the region of interest based on the determination that one or more of the subjects in the FOV have priority. For example, the region of interest can be adjusted to include subjects having priority, but exclude subjects not having priority. The method then proceeds to block <NUM>.

At block <NUM>, the 3A engine <NUM> performs at least one of AF, AE, or AWB based on the adjusted region of interest. If any subjects were determined to have priority in block <NUM>, then the region of adjusted interest (e.g., <NUM>, <FIG>) includes the regions of interest <NUM> (<FIG>) determined in block <NUM>. Otherwise, the region of interest <NUM> (<FIG>) includes all the regions of interests <NUM>, <NUM> (<FIG>) determined in block <NUM>.

At block <NUM>, the camera optics and sensor <NUM> (<FIG>) capture an image. For example, the image can be captured using the AF, AE, and AWB optimized for the adjusted ROI <NUM> (<FIG>).

In other embodiments, the priority subject data <NUM> can include two or more priority levels. The priority subject data <NUM> can assign one or more predetermined prioritized persons to respective priority levels. For example, a first predetermined prioritized person (e.g., the primary user's child) can be assigned a highest priority, a second predetermined prioritized person (e.g., the primary user) can be associated with a next highest priority, and so on. If the FOV of the image capture device <NUM> includes a higher priority subject and a lower priority subject, the ROI adjuster <NUM> includes the higher priority subject in the ROI (for AF, AE and/or AWB) and excludes the lower priority subject. If the FOV of the image capture device <NUM> includes a lower priority subject and an unknown subject, the ROI adjuster <NUM> includes the lower priority subject in the ROI and excludes the unknown subject. In general, if the FOV contains subjects with two or more priority levels, the ROI adjuster <NUM> includes the subject having the highest priority level in the ROI and excludes any other persons within the FOV from the ROI.

In some examples, some prioritized persons can be predetermined or selected in advance by a user. Other prioritized persons are not pre-selected, but may be automatically added to the priority subject data <NUM> in response to determining that the priority subject selector <NUM> determining that the same subject has been captured at least a threshold number of times within a predetermined time interval.

In some examples, processor <NUM> can determine whether any of the detected subjects in the image have different priority levels among persons of the priority subject data <NUM>. For example, processor <NUM> can identify two priority subjects in the image, where one subject is associated with the highest priority level in the priority subject data <NUM>, and another subject is associated with the next highest priority level in the priority subject data <NUM>. In some examples, processor <NUM> determines if a configurable number of detected subjects are prioritized persons. For example, the configurable number of detected subjects can be set by a user of image capture device <NUM> via input device <NUM>.

In some examples, the priority subject data <NUM> includes predetermined prioritized persons associated with a priority level, and dynamically selected prioritized persons automatically added by virtue of being captured a threshold number of times within a predetermined period. In other embodiments, the priority subject data <NUM> contains two separate lists or folders: static priority subject data <NUM> containing the priority subjects previously selected (by the primary user) and a temporary list or folder containing automatically added priority subjects. Processor <NUM> can determine whether any of the detected subjects is a previously selected priority subject with a previously selected priority level. If no detected subjects within the FOV are identified as being prioritized persons, processor <NUM> can dynamically determine whether any detected subjects have been captured a threshold number of times within a predetermined period. Dynamically selected subjects can be added to a list or folder of temporary priority subject data <NUM> for a predetermined period of time.

In some examples, processor <NUM> can determine if up to a configurable number of subjects detected are priority subjects. A user of image capture device <NUM> can set the configurable number of priority subjects via input device <NUM>.

In some examples, priority subject selector instructions <NUM> can include instructions that, when executed by processor <NUM>, cause processor <NUM> to identify (e.g., select) at least two detected subjects as predetermined prioritized persons. For example, if two out of three or more persons within the FOV are priority subjects in the priority subject data <NUM>, the ROI for performing AF, AE and AWB includes the faces of the two priority subjects, and any other persons within the FOV are excluded from the ROI for AF, AE, and AWB.

In some examples, priority subject selector <NUM> can include instructions that, when executed by processor <NUM>, cause processor <NUM> to add a dynamically selected priority subject to the temporary priority subject data, based on a location of the image capture device <NUM> at the time of image capture. This capability can allow the user to select a location (e.g., the user's home or a vacation spot) at which every person captured is added to the temporary priority subject data <NUM>. The selected location can be, for example, the person's home address, home state, favorite place, or any other location. For example, processor <NUM> can obtain a location of image capture device <NUM> by receiving location data from GPS unit <NUM>, and compare the current location to the selected location. Location data can include, for example, latitude and longitude information. In other embodiments, the processor <NUM> can determine the current location of the image capture device <NUM> based on the location of a WiFi access point, triangulation from beacons, signal from a "smart watch", or the like. If processor <NUM> determines that the location of image capture device <NUM> is at or near the selected location, the priority subject selector <NUM> can add the person to the priority subject data <NUM>. For example, any person within the FOV of the image capture device <NUM> while the image capture device is located at the user's home will be added to the temporary priority subject data <NUM>.

In some embodiments, the image capture device maintains separate priority subject data <NUM> for persons associated with each of a plurality of selected locations. In some examples, a person can be associated with more than one location. In some examples, location information associated with priority subjects can be included in the priority subject data <NUM> stored in storage medium <NUM>. For example, a user can provide location information to be associated with priority subjects to image capture device <NUM> via input device <NUM>.

In some examples, priority subject selector instructions <NUM> can include instructions that, when executed by processor <NUM>, cause processor <NUM> to recognize at least one priority subject based on image data corresponding to the priority subject obtained over a network. For example, transceiver <NUM> is operable to transmit data to, and receive data from, a network. The network can be a WiFi® network, a cellular network such as a 3GPP® network, a Bluetooth® network, or any other suitable network. The network can provide access to, for example, the Internet. Processor <NUM> is in communication with transceiver <NUM>, and can request and receive image data corresponding to one or more predetermined prioritized persons from the network. For example, processor <NUM> can search social media, a search engine, or an online encyclopedia over the network for images of a nearby person based on the location of the image capture device, obtain one or more images of the nearby person from social media, search engine, or online encyclopedia via the network, and add the nearby person to the priority subject data <NUM>. Processor <NUM> can add the one or more obtained images (or other image-indicative data based on the obtained images) of the nearby person to a folder storing images of persons identified in the priority subject data <NUM>, for example.

In other embodiments, the image capture device <NUM> allows the user to add the name of an individual to the priority subject data <NUM>, and the image capture device automatically retrieves training images associated with the named individual from a search engine, social media and/or the user's photo library in the image capture device. The image capture device <NUM> uses machine learning to learn the person's face, and recognizes the person's face as being associated with a prioritized person when the face is within the FOV of the image capture device <NUM>.

In some examples, priority subject selector instructions <NUM> can include instructions that, when executed by processor <NUM>, cause processor <NUM> to add at least one predetermined prioritized person to the priority subject data <NUM>, based on how often the predetermined prioritized person has been detected as a subject in the FOV of the image capture device <NUM>. For example, processor <NUM>, when executing subject detector instructions <NUM>, can associate a time of capture with each detected subject. Processor <NUM> can store image data associated with each detected subject, along with a time of capture, as part of the priority subject data <NUM> stored in storage medium <NUM>. If the same subject is detected, processor <NUM> can update the most recent time of capture associated with that subject. Processor <NUM> can also maintain a count of how many times the subject has been detected. For example, the count can be based on how often the subject was detected over a period of time (e.g., over the last month, week, day, etc.). In some examples, the count can be configured by a user via input device <NUM>. The count can also be stored as part of priority subject data <NUM> in storage medium <NUM>.

Processor <NUM>, when executing priority subject selector instructions <NUM>, can read priority subject data <NUM> from storage <NUM> to determine how often the subject in the FOV has been detected over a recent period of time. If the subject has been detected more than a threshold number of times during that period, processor <NUM> can add the person to the priority subject data <NUM>. If the image capture device <NUM> does not associate the person with a name, the image capture device can generate a temporary name for the subject. If the subject has not been detected more than the threshold number of times during the period, the subject is not added to the priority subject data <NUM>. For example, processor <NUM> can determine that at least one of one or more subjects in the field of view of the image capture device <NUM> has previously been photographed using the image capture device (and previously detected as a subject). Processor <NUM> can store image data (or other image-indicative data based on the image data) corresponding to the previously captured subject to the priority subject data <NUM> in storage medium <NUM>.

Processor <NUM> can be configured to add images or other image-indicative data of a person to the priority subject data <NUM> (or a temporary priority subject data <NUM>) if a threshold number of images of the same person are determined to be captured within a predetermined period of time. For example, processor <NUM> can add images or other image-indicative data of a person to the priority subject data <NUM> (or a temporary priority subject data <NUM>) if the same person is captured in ten images within one minute. In some embodiments, the temporary priority subject data <NUM> can be purged upon passage of a predetermined period (e.g., one week) after being added to the temporary priority subject data <NUM>. In all other respects the temporary priority subject data <NUM> can be structured and used the same way as the static priority subject data <NUM> described herein.

<FIG> is a flowchart <NUM> of another illustrative method not claimed in the appended claims that can be carried out by, for example, the image capture device <NUM> of <FIG>. In the example of <FIG>, instead of using images captured by the user to specify predetermined prioritized persons, the priority subject selector <NUM> (<FIG>) allows the user to input the name of a subject (e.g., a well-known public figure) whom the user wishes to photograph. The priority subject selector <NUM> can obtain images of the predetermined person from the Internet using a search engine (e.g., Google). The image capture device can save the images or other priority subject image-indicative data into a folder within priority subject data <NUM> for "temporary prioritized persons". The saved images can be used for adjusting the ROI for AF, AE and AWB, in the manner described above for the primary user of the image capture device <NUM>. When the image capture device is used, the subject in the FOV can be compared to the temporary prioritized person; if the FOV contains the temporary prioritized person, the priority subject selector <NUM> selects an ROI containing the face of the temporary predetermined person.

At block <NUM>, priority subject selector <NUM> retrieves the name of a priority subject from the priority subject data <NUM>. In some embodiments, priority subject selector <NUM> determines that the number of images or views (or other priority subject image-indicative data) of the subject is insufficient for accurate recognition. In other embodiments, the priority subject data <NUM> includes a separate folder or storage area for identifying names of priority subjects without accompanying images or other priority subject image-indicative data.

At block <NUM>, subject detector <NUM> tries again to obtain images of the named priority subject using a search engine (e.g., Google or Yahoo!) or using social media (e.g., Facebook).

At block <NUM>, subject detector <NUM> performs facial detection on image data within a field of view of an image capture device <NUM> to detect one or more subjects.

At block <NUM>, priority subject selector <NUM> determines whether any of the subjects in the field of view of the image capture device <NUM> is a predetermined person.

At block <NUM>, a region of interest of image data is identified that contains one or more of the subjects determined to be a predetermined person, and excludes one or more subjects determined not to be pre-determined persons.

At block <NUM>, at least one of AF, AE, or AWB of the image capture device is performed based on the identified region of interest. For example, the AF, AE, and AWB can all be performed based on the identified region of interest.

<FIG> is a flow chart of a method <NUM> according to the present invention that can be carried out by image capture device <NUM>. In the example of <FIG>, the priority subject detection mode is activated when the primary user's fingerprint is detected while the front camera of the image capture device <NUM> is activated.

At block <NUM>, an image of the device primary user's face (or other priority subject image-indicative data derived therefrom) is stored in a non-transitory, machine-readable storage medium <NUM> (<FIG>) in the image capture device <NUM>. For example, the image can be captured via a selfie and stored in storage medium <NUM> or, the image can be received from an external camera (not shown) and saved in the priority subject data <NUM> of image capture device <NUM>.

At block <NUM>, a front camera of image capture device <NUM> is activated, where the front face is the face having a display.

At block <NUM>, biometric information (fingerprint, iris, or voice) of the user is detected. For example, if the image capture device <NUM> is a cell phone, a fingerprint detection device <NUM> can be located on the home button of the phone. In response to the fingerprint detection device <NUM> (<FIG>) of image capture device <NUM> detecting the primary user's fingerprint while the front camera is active, execution automatically passes to block <NUM>, activating facial recognition. Similarly, in other embodiments, execution passes to block <NUM> in response to the phone detecting the primary user's iris or voice.

At block <NUM>, a determination is made as to whether a face of the primary user is in the FOV. For example, the priority subject selector <NUM> can compare a face within the FOV against one or more stored images of the primary user's face stored in the storage medium <NUM>. If the primary user is not detected, execution proceeds to block <NUM>. If the primary user is detected, the method proceeds to block <NUM>.

At block <NUM>, the ROI used for at least one of automatic focus, automatic exposure, or automatic white balance is adjusted based on identification of the primary user's face.

At block <NUM>, an image is captured. For example, if the primary user is recognized by the priority subject selector <NUM> (<FIG>), the image is captured using the one or more adjusted automatic focus, automatic exposure, or automatic white balance.

<FIG> shows additional blocks that can be added to the method of <FIG>, in some examples not claimed in the appended claims. The additional blocks in <FIG> designate a person in a field of view of the image capture device as a priority subject in response to determining that the image capture device is in a predetermined location. Blocks <NUM> and <NUM> of <FIG> are executed prior to execution of block <NUM> (<FIG>).

At block <NUM>, a determination is made whether the image capture device <NUM> is located in a predetermined location entered into the priority subject data <NUM>. For example, the predetermined location can be identified by name, address or by GPS coordinates. If the image capture device <NUM> is located in a predetermined location, block <NUM> is executed. If the image capture device <NUM> is not located in a predetermined location, execution passes to step <NUM> of <FIG>.

At block <NUM>, image data (or other image-indicative data extracted or derived therefrom) for each face in the FOV are added to the priority subject data <NUM> in the image capture device <NUM>. After block <NUM>, execution passes to block <NUM>, <FIG>.

Using the method of <FIG> and <FIG>, the user can instruct the image capture device to make every face detected in a predetermined location (e.g., the user's home, a vacation spot, a place of employment, etc.) a priority subject. Subsequently, when these priority subjects are within the FOV of the image capture device, they will be included in the ROI for AE, AF, and/or AWB.

Claim 1:
A method for controlling an image capture device (<NUM>), comprising:
obtaining first image data from a front facing camera, where the front face of the image capture device includes a display, the first image data representing one or more subjects within a field of view of the image capture device;
detecting (<NUM>) biometric authentication of a primary user of the image capture device, while the front facing camera of the image capture device is operating;
upon successful biometric authentication of the primary user, performing priority-subject based region of interest selection, comprising:
performing (<NUM>, <NUM>; <NUM>) facial recognition on the first image data, to determine whether any of the one or more subjects is the primary user, wherein the determination whether any of the one or more subjects is the primary user is made based on priority subject image-indicative data (<NUM>) for the primary user accessible by the image capture device and wherein the facial recognition to determine whether any of the one or more subjects is the primary user is activated automatically upon successful biometric authentication of the primary user; and
upon determination that the primary user is included in the one or more subjects, selecting (<NUM>, <NUM>; <NUM>) a region of interest (<NUM>; <NUM>) of the first image data including a face of the primary user;
performing (<NUM>) at least one of automatic focus, automatic exposure, or automatic white balance using the selected region of interest; and
capturing (<NUM>) second image data from the front facing camera based on the automatic focus, automatic exposure, or automatic white balance using the selected region of interest.