Method and apparatus for face tracking utilizing integral gradient projections

A method, apparatus and computer program product are provided for estimating and verifying translation motion and/or a scaling factor between face regions in respective frames during face tracking. A method determines translation motion between face regions in two successive frames based upon integral gradient projections of a face region in a first frame and a corresponding window in a second frame. The method also verifies the translation motion between the first and second frames utilizing integral gradient projections. A method also determines a transfer function relating integral projection curves of a face region in a first frame that has a predefined position and a predetermined size and a co-located window of same size in a second frame, determines a scaling factor based upon the transfer function and then verifies the scaling factor utilizing integral gradient projections.

TECHNOLOGICAL FIELD

An example embodiment of the present invention relates generally to face tracking and, more particularly, to face tracking utilizing integral gradient projections for estimating translation motion and/or a scaling factor.

BACKGROUND

In various image processing applications, it may be desirable to track an object, such as a face, between successive frames in a video. In order to track a face from one frame to the next, the translation motion of the face and the scaling of the face from one frame to the next may need to be determined. However, the determination of the translation motion and the scaling of a face from one image to the next may be a computationally intensive process which may be a challenge, at least for those devices with limited computational resources, to perform in an efficient and timely manner.

Face tracking may be computationally intensive for various reasons. For example, some face tracking techniques analyze an entire frame or at least a relatively large portion of the frame, as opposed to focusing upon the face region. Additionally, or alternatively, some face tracking techniques utilize multi-dimensional searches which further add to the computational requirements. As such, it would be desirable to provide an improved technique for face tracking between frames, such as frames of a video, that provides accurate results with reduced computational requirements.

BRIEF SUMMARY

A method, apparatus and computer program product are provided in accordance with an example embodiment of the present invention in order to provide an improved technique for face tracking. In this regard, the method, apparatus and computer program product of one embodiment may provide for face tracking in a manner that reduces the associated computational requirements while continuing to provide reliable and robust face tracking. Indeed, the method, apparatus and computer program product of one example embodiment may provide for face tracking including the estimation of translation motion and/or a scaling factor between face regions of successive frames utilizing integral projections in a computationally efficient manner. The verification of translation motion and/or a scaling factor between face regions of successive frames is done utilizing the integral gradient projections.

In one embodiment, a method is provided that includes determining a translation motion between face regions of first and second frames based upon respective integral projections of a face region in the first frame that has a predefined position (x0,y0) and a predetermined size and a co-located window of same size in the second frame. The method also includes verifying the translation motion between the first and second face regions belonging to the respective frames. In this regard, the verification includes determining integral gradient projections for the first face region and the window in the second frame following movement of the window by the translation motion between the first and second face regions. The verification also includes determining a distance between the integral gradient projections of the window, following movement by the translation motion, and the face region. Further, the verification includes determining whether the translation motion between the faces in two frames is verified based on a relationship between the distance and a predefined threshold.

A method of one embodiment may also include determining the integral projections of the face region in the first frame in first and second mutually orthogonal directions and determining the integral projections for the window in the second frame in first and second mutually orthogonal directions. In one embodiment, the determination of the translation motion includes shifting respective integral projection curves of the first and second face regions and determining an amount of shift that creates a minimum distance between the respective integral projection curves of the first and second face regions.

The determination of the integral gradient projections for the window in the second frame following movement of the window by the translation motion may include determining integral gradient projections for the window in the second frame in first and second mutually orthogonal directions. Additionally, the determination of the integral gradient projections for the face region in the first frame may include determining integral gradient projections for the face region in the first frame in the first and second mutually orthogonal directions. The method of one embodiment may also include redefining the predefined position for other frames based upon the translation motion in an instance in which the translation motion is verified. In a further embodiment, the method may include evaluating a scaling factor between the face region of the first frame and the co-located window of same size in the second frame in an instance in which the translation motion is not verified in that the distance fails to satisfy predefined threshold.

In another embodiment, an apparatus is provided that includes at least one processor and at least one memory including computer program code configured to, with the at least one processor, cause the apparatus to determine a translation motion between first and second face regions based upon respective integral projections of a face region in the first frame that has a predefined position and a predetermined size and a co-located window of same size in the second frame. The at least one memory including the computer program code is also configured to, with the at least one processor, cause the apparatus to verify the translation motion between the first and second face regions. In this regard, the verification includes determining integral gradient projections for the face region in the first frame and the window in the second frame following movement of the window by the translation motion between the first and second face regions. The verification also includes determining integral gradient projections for a face region in the first frame and determining a distance between the integral gradient projections of the window, following movement by the translation motion, and the face region. Further, the verification includes determining whether the translation motion between face regions is verified based on a relationship between the distance and a predefined threshold.

The at least one memory including the computer program code of one embodiment is also configured to, with the at least one processor, cause the apparatus to determine the integral projections of the face region in the first frame in first and second mutually orthogonal directions and to determine the integral projections for the window in the second frame in first and second mutually orthogonal directions. In one embodiment, the determination of the translation motion between face regions includes shifting respective integral projection curves of the first and second frames and determining an amount of shift that creates a minimum distance between the respective integral projection curves of the first and second frames.

The determination of the integral projections for the window in the second frame following movement of the window by the translation motion may include determining integral projections for the window in the second frame in first and second mutually orthogonal directions. Additionally, the determination of the integral projections for the face region in the first frame may include determining integral projections for the face region in the first frame in the first and second mutually orthogonal directions. The at least one memory including the computer program code may also be configured to, with the at least one processor, cause the apparatus to redefine the predefined position for other frames based upon the translation motion in an instance in which the translation motion is verified. In a further embodiment, the at least one memory including the computer program code may also be configured to, with the at least one processor, cause the apparatus to evaluate a scaling factor between the face region of the first frame and the window of the second frame in an instance in which the translation motion is not verified in that the distance fails to satisfy predefined threshold.

In a further embodiment, a computer program product is provided that includes at least one non-transitory computer-readable storage medium having computer-readable program instructions stored therein with the computer-readable program instructions including program instructions configured to determine a translation motion between first and second face regions based upon respective integral projections of a face region in the first frame that has a predefined position and a predetermined size and a co-located window of same size in the second frame The computer-readable program instructions also include program instructions configured to verify the translation motion between the first and second face regions in respective frames. In this regard, the verification includes determining integral gradient projections for the window in the second frame following movement of the window by the translation motion between the first and second face regions. The verification also includes determining integral gradient projections for a face region in the first frame and determining a distance between the integral gradient projections of the window, following movement by the translation motion, and the face region. Further, the verification includes determining whether the translation motion is verified based on a relationship between the distance and a predefined threshold.

In yet another embodiment, an apparatus is provided that includes means for determining a translation motion between first and second face regions based upon respective integral projections of a face region in the first frame that has a predefined position and a predetermined size and a co-located window of same size in the second frame. The apparatus also includes means for verifying the translation motion between the first and second face regions in respective frames. In this regard, the verification includes determining integral gradient projections for the window in the second frame following movement of the window by the translation motion between the first and second face regions. The verification also includes determining integral gradient projections for a face region in the first frame and determining a distance between the integral gradient projections of the window, following movement by the translation motion, and the face region. Further, the verification includes determining whether the translation motion is verified based on a relationship between the distance and a predefined threshold.

In yet a further embodiment, a computer program is provided that, upon execution, is configured to determine a translation motion between first and second face regions based upon respective integral projections of a face region in the first frame that has a predefined position and a predetermined size and a co-located window of same size in the second frame. The computer program is also configured, upon execution to verify the translation motion between the first and second face regions in respective frames. In this regard, the verification includes determining integral gradient projections for the window in the second frame following movement of the window by the translation motion between the first and second face regions. The verification also includes determining integral gradient projections for a face region in the first frame and determining a distance between the integral gradient projections of the window, following movement by the translation motion, and the face region. Further, the verification includes determining whether the translation motion is verified based on a relationship between the distance and a predefined threshold.

In one embodiment, a method is provided that includes determining a transfer function relating integral projection curves of a face region in a first frame that has a predefined position and a predetermined size and a co-located window of the same size in a second frame. The method of this embodiment also determines a scaling factor by using integral projections of the first face regions and second face region based upon the transfer function. To compute the distance between two face regions of different scale, initially the scale difference is compensated by applying the transfer function to co-ordinates of the integral projections of the face region of the first frame. The method may also determine a distance between the integral projections of the face region in the first frame that are transformed using the transfer function and the window in the second frame and may determine whether the scaling factor is verified based on a relationship between the distance and a predefined threshold.

The determination of the transfer function may include determining a transfer function line that provides a minimum point-to-point distance between the integral projection curves. The determination of the scaling factor may include determining the scaling factor based upon scaling factors in first and second mutually orthogonal directions. The method of one embodiment may also include performing a face search using a pattern recognition based face classifier in an instance in which the scaling factor is not verified in that the distance fails to satisfy the predefined threshold. In addition, the method of one embodiment may also include redefining the predetermined size for other frames based upon the scaling factor in an instance in which the scaling factor is verified.

In another embodiment, an apparatus is provided that includes at least one processor and at least one memory including computer program code configured to, with the at least one processor, cause the apparatus to determine a transfer function relating integral projection curves of a face region in a first frame that has a predefined position and a predetermined size and a co-located window in a second frame with the same size. The at least one memory including the computer program code may also be configured in this embodiment to, with the at least one processor, cause the apparatus to determine a scaling factor based upon the transfer function and integral projections for the windows in the first and second frames. The at least one memory including the computer program code may also be configured to, with the at least one processor, cause the apparatus to determine a distance between the integral projections of the face region in the first frame and the window in the second frame that has a size that is based upon the scaling factor and to determine whether the scaling factor is verified based on a relationship between the distance and a predefined threshold.

The determination of the transfer function may include determining a transfer function line that provides a minimum point-to-point distance between the integral projection curves. The point to point distance between two face projection curves at a particular scale factor is computed by first compensating the integral projection curves for the face region of the first frame for scale and using this to compute the point to point distance with the integral projection curves of the face region for the second frame. The determination of the scaling factor may include determining the scaling factor based upon scaling factors in first and second mutually orthogonal directions. The at least one memory including the computer program code may also be configured in one embodiment to, with the at least one processor, cause the apparatus to perform a face search using a pattern recognition based face classifier in an instance in which the scaling factor is not verified in that the distance fails to satisfy the predefined threshold. In addition, the at least one memory including the computer program code may also be configured to, with the at least one processor, cause the apparatus to redefine the predetermined size for other frames based upon the scaling factor in an instance in which the scaling factor is verified.

In a further embodiment, a computer program product is provided that includes at least one non-transitory computer-readable storage medium having computer-readable program instructions stored therein with the computer-readable program instructions including program instructions configured to determine a transfer function relating integral projection curves of a face region in a first frame that has a predefined position and a predetermined size and a co-located window of same size in a second frame. The computer-readable program instructions also include program instructions configured to determine a scaling factor based upon the transfer function and using integral projections for the face region in first frame and of the window in the second frame. The computer-readable program instructions may also include program instructions configured to determine a distance between the integral projections of the face region in the first frame and the window in the second frame that has a size that is based upon the scaling factor and program instructions configured to determine whether the scaling factor is verified based on a relationship between the distance and a predefined threshold.

In yet another embodiment, an apparatus is provided that includes means for determining a transfer function relating integral projection curves of a face region in a first frame that has a predefined position and a predetermined size and a co-located window of same size in a second frame. The apparatus of this embodiment also includes means for determining a scaling factor based upon the transfer function using integral projections of the face region in the first frame and for the window in the second frame with the window having a size that is based upon the scaling factor. The apparatus may also include means for determining a distance between the integral projections of the face region in the first frame and the window in the second frame that has a size that is based upon the scaling factor and means for determining whether the scaling factor is verified based on a relationship between the distance and a predefined threshold.

In yet a further embodiment, a computer program is provided that, upon execution, is configured to determine a transfer function relating integral projection curves of a face region in a first frame that has a predefined position and a predetermined size and a co-located window of same size in a second frame. The computer program is also configured, upon execution, to determine a scaling factor based upon the transfer function and using integral projections for the face region in first frame and of the window in the second frame. The computer program is further configured to determine a distance between the integral projections of the face region in the first frame and the window in the second frame that has a size that is based upon the scaling factor and to determine whether the scaling factor is verified based on a relationship between the distance and a predefined threshold.

DETAILED DESCRIPTION

As used herein, the terms “data,” “content,” “information” and similar terms may be used interchangeably to refer to data capable of being transmitted, received, displayed and/or stored in accordance with various example embodiments. Thus, use of any such terms should not be taken to limit the spirit and scope of the disclosure.

The term “computer-readable medium” as used herein refers to any medium configured to participate in providing information to a processor, including instructions for execution. Such a medium may take many forms, including, but not limited to a non-transitory computer-readable storage medium (for example, non-volatile media, volatile media), and transmission media. Transmission media include, for example, coaxial cables, copper wire, fiber optic cables, and carrier waves that travel through space without wires or cables, such as acoustic waves and electromagnetic waves, including radio, optical and infrared waves. Examples of non-transitory computer-readable media include a floppy disk, hard disk, magnetic tape, any other non-transitory magnetic medium, a compact disc read only memory (CD-ROM), compact disc compact disc-rewritable (CD-RW), digital versatile disc (DVD), Blu-Ray, any other non-transitory optical medium, a random access memory (RAM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), a FLASH-EPROM, any other memory chip or cartridge, or any other non-transitory medium from which a computer can read. The term computer-readable storage medium is used herein to refer to any computer-readable medium except transmission media. However, it will be appreciated that where embodiments are described to use a computer-readable storage medium, other types of computer-readable mediums may be substituted for or used in addition to the computer-readable storage medium in alternative embodiments.

An apparatus10for performing face tracking in accordance with an example embodiment of the present invention is shown inFIG. 1. The apparatus may be embodied in a wide variety of computing devices, such as mobile terminals, e.g., mobile telephones, mobile computers, personal digital assistants (PDAs), pagers, laptop computers, desktop computers, gaming devices, televisions and other types of mobile electronic systems, or various fixed computing devices, such as workstations, personal computers or the like. It should also be noted that whileFIG. 1illustrates one example of a configuration of an apparatus for performing face tracking, numerous other configurations may also be used to implement embodiments of the present invention. As such, in some embodiments, although devices or elements are shown as being in communication with each other, hereinafter such devices or elements should be considered to be capable of being embodied within a same device or element and thus, devices or elements shown in communication should be understood to alternatively be portions of the same device or element.

The user interface18may be in communication with the processor12to receive an indication of a user input at the user interface and/or to cause provision of an audible, visual, mechanical or other output to the user. As such, the user interface18may include, for example, a keyboard, a mouse, a joystick, a display, a touch screen(s), touch areas, soft keys, a microphone, a speaker, or other input/output mechanisms. Alternatively or additionally, the processor may comprise user interface circuitry configured to control at least some functions of one or more elements of the user interface, such as, for example, a speaker, ringer, microphone, display, and/or the like. The processor and/or user interface circuitry comprising the processor may be configured to control one or more functions of one or more elements of the user interface through computer program instructions (e.g., software and/or firmware) stored on a memory accessible to the processor (e.g., memory device14, and/or the like).

As shown inFIG. 2, a method of determining the translation motion, if any, between two frames in which a face was tracked is shown. Prior to determining the translation motion between face regions in successive frames, a face is detected in a first frame. The face may be detected in the first frame by any general object detection method, such as the Viola-Jones method. Once a face has been detected in the first frame, the position of the face, such as the center of the face denoted (x0, y0) and the size of the face denoted M×M are determined. Thereafter, the translation motion of the face region in between the first frame and a subsequent, second frame, may be determined. In this regard, the apparatus10may include means, such as the processor12or the like, for determining the translation motion between the first and second frames based upon respective integral projection of the face region in the first frame that has a predefined position (x0, y0) and a predetermined size M×M and a window in the second frame around the predefined position (x0, y0) that has the predetermined size M×M. See operation20ofFIG. 2.

Further details regarding the determination of the translation motion between the first and second frame are provided by the flowchart ofFIG. 3such that reference is now made toFIG. 3and, in particular, to operation30ofFIG. 3. In this regard, the apparatus10may include means, such as the processor12or the like, for determining integral projections of the face region in the first frame in first and second mutually orthogonal directions, such as x and y directions. For example, the horizontal and vertical integral projection curves h(y) and v(x) for the face region of size M×M in the first frame I(x,y) may be determined as follows:h(y)=Σxε{−M/2,M/2}|I (x−x0, y−y0)| where y ε {−M/2,M/2} for the face centered at [x0, y0]v(x)=Σyε{−M/2,M/2E}|I(x−x0, y−y0)| where x ε {−M/2,M/2} for the face centered at [x0, y0]

As shown in operation32ofFIG. 3, the apparatus10may also include means, such as processor12or the like, for determining the integral projection curves for the corresponding window in the second frame in the first and second mutually orthogonal directions, such as the x and y directions. The integral projection curves of the window in the second frame may be determined in various manners including as follows:h′(y)=Σx⊂{−M/2,M/2}|I′(x−x0, y−y0)| where y ε {−M/2,M/2} for the face centered at [x0, y0]v′(x)=Σyε{−M/2,M/2}|I′(x−x0, y−y0)| where x ε {−M/2,M/2} for the face centered at [x0, y0]
where h′( ) represents the horizontal integral projection curves and v′( ) represents the vertical integral projection curves in the second frame I′(x,y).

Based upon these integral projections, the translation motion of the face region between the first frame I(x,y) and the second frame I′(x,y) may then be determined. As shown in operation34ofFIG. 3, the apparatus10may include means, such as the processor12or the like, for shifting the integral projection curves of the first and second frames. The apparatus10of this embodiment may also include means, such as the processor12or the like, for determining the amount of shift that creates a minimum distance between the respective integral projection curves of the first and second frames as shown in operation36ofFIG. 3. By way of example, the translation motion in the x direction may be obtained by shifting the vertical integral projection curves for the first and second frames, e.g., v(x) and v′(x) as described above, and determining the amount of shift which results in the minimum distance dxbetween the two vertical integral projection curves. In one embodiment, for example, the minimum distance dxbetween the vertical integral projection curves for the first and second frames may be determined as follows:

Similarly, the motion in the y direction may be obtained by shifting the horizontal integral projection curves for the first and second frames, e.g., h(y) and h′(y) as described above, and determining the amount of shift which results in the minimum distance dybetween the two horizontal integral projection curves. In one embodiment, for example, the minimum distance dybetween the horizontal integral projection curves for the first and second frames may be determined as follows:

Once the translation motion has been determined, the method and apparatus10of one example embodiment may verify the translation motion between the first and second frames. As shown in operation22ofFIG. 2and operation38ofFIG. 3, the apparatus10may include means, such as a processor12or the like, for determining integral gradient projections for the window in the second frame following movement of the window by the translation motion that was determined between the first and second frames. In accordance with the foregoing example, the window in the second frame I′(x,y) may be moved by the translation motion that was determined to be (dx, dy). Following the movement of the window in the second frame I′(x,y), the integral gradient projection of the window in the second frame in the x and y direction may be determined as follows with h″(x) representing the horizontal projection of the vertical gradient and v″(y) representing the vertical projection of the horizontal gradient:h″(x)=Σyε{−M/2,M/2}|I′(x+dx−x0, y+dy−y0)−I(x+dx−x0, y+dy+1−y0)| where x ε {−M/2,M/2}v″(y)=Σxε{−M/2,M/2}|I′(x+dx−x0, y+dy−y0)−I(x+dx+1−x0, y+dy−y0)| where y ε {−M/2,M/2}

As shown in operation24ofFIG. 2and operation40ofFIG. 3, the apparatus10may also include means, such as a processor12or the like, for determining the integral gradient projections for a face region in the first frame. For example, the horizontal projection hg(x) of the vertical gradient and the vertical projection vg(y) of the horizontal gradient in the first frame I(x,y) may be determined as follows:hg(x)=Σyε{−M/2,M/2}|I(x−x0, y−y0)−I(x−x0, y+1−y0)| where x ε {−M/2,M/2} for the face centered at [x0, y0]Vg(y)=Σxε{−M/2,M/2}|I(x−x0, y−y0)−I(x+1−x0, y−y0)| where y ε {−M/2,M/2} for the face centered at [x0, y0]
By performing the verification of the translation motion utilizing the integral gradient projections, the method and apparatus10of example embodiments of the present invention may avoid the disadvantageous effects of illumination so as to more accurately estimate the translation motion.

As shown in operation26ofFIG. 2and operation42ofFIG. 3, the apparatus10may also include means, such as processor12or the like, for determining a distance between the integral gradient projections of the window, following movement by the translation motion, and the face region. In this regard, the distances in the x and y directions, fdx, fdy, may be determined between the horizontal integral gradient projection curves of the first and second frames and the vertical integral gradient projection curves of the first and second frames as follows:

fdx=∑x=[1,M]⁢{h″⁡(x)-hg⁡(x)}fdy=∑y=[1,M]⁢{v″⁡(y)-vg⁡(y)}
The apparatus10may also include means, such as a processor12or the like, for determining whether the translation motion is verified based on a relationship between the distance between the integral gradient projections and a predefined threshold thr1, as shown in operation28ofFIG. 2and operation44ofFIG. 3. In one example, the apparatus, such as a processor, may verify the translation motion of the face between the first and second frames in an instance in which the distances are less than the predefined threshold, e.g., fdx<thr1and fdy<tthr1. While the same predefined threshold thr1is utilized in conjunction with both the distance between integral gradient projections in the x direction and the distance between the integral gradient projections in the y direction, different predefined thresholds may be utilized in the x and y directions in other embodiments. In an instance in which the translation motion of the face is between the first and second frames is verified, the apparatus may include means, such as a processor12or the like, for redefining the predefined position of other frames such as subsequent frames, based upon the translation motion. See operation46ofFIG. 3. In other words, the predefined position of the face region about which the face region is centered may be updated based upon the translation motion that was identified as described above. However, if the apparatus, such as the processor, fails to verify the translation motion, such as in an instance in which the distance between the integral gradient projections fails to satisfy the predefined threshold, e.g., fdx>thr1or fdy>thr1, the method, apparatus and computer program product of one embodiment may then evaluate a scaling factor between the face region of the first frame and the window of the second frame so as to determine possible scaling therebetween, as shown in operation48ofFIG. 3and as described below in conjunction withFIG. 4.

The method, apparatus and computer program product of the example embodiment of the present invention as described above in conjunction withFIGS. 2 and 3, for example, may provide a substantial computational advantage with increased computational efficiency by determining integral gradient projections for only the face region in the first frame and a corresponding window in the second frame, as opposed to larger portions or, indeed, the entirety of the frame and by estimating the translation motion utilizing a one-dimensional search.

In an instance in which the translation motion is not verified, such as in an instance in which the distance between the integral gradient projections fails to satisfy a predefined threshold, the method, apparatus and computer program product of one example embodiment may then analyze the integral projection curve for the presence of scaling between the face region of the first frame and the corresponding window of the second frame. While scaling may be introduced for a variety of reasons, scaling may generally occur in an instance in which either the face or the image capturing device, e.g., camera, moves relative to the other, such as closer or further relative to the other.

In accordance with this embodiment, the apparatus10may include means, such as the processor12or the like, for determining a transfer function relating the integral gradient projections curves of the face region of the first frame and the corresponding window in the second frame. See operation50ofFIG. 4. By way of example, reference is now made toFIG. 5in which h1(x) and h2(2) are the horizontal integral projection curves for the face region of the first frame and the corresponding window of the second frame, respectively. In order to determine the transfer function, the apparatus10of one example embodiment may include means, such as a processor12or the like, for determining a transfer function line that provides a minimum point-to-point distance between the integral projection curves. In regards to the embodiment ofFIG. 5, for example, the transfer function line may be defined as i=m*j+c. In this regard, j is the indices of h2(x) and i is the indices of h1(x). Moreover, the distance between the integral projection curves of this example embodiment is defined to be the sum of the absolute difference between the two curves.

Of the plurality of potential mappings between the two curves, the search of one example embodiment may be constrained to a search for the slope m in which m is defined to between 0.75 and 1.25 since the scale change of the face region will generally not change by more than 1.25 times within two frames. The slope for the transfer function line between the horizontal integral projection curves may be determined in this example embodiment as follows:mx=arg minm{|h2(j)−h1(m*j+ck)|} where as m ε (0.75,01.25)
In this regard, ckmay be determined by assuming that the integral transfer function line crosses a diagonal line that extends through the origin and is shown as a dashed line inFIG. 5at (k*M/4, k*M/4) with M being the length of the line and k being an element of [0,3]. Although described above in conjunction with the determination of the slope mxof the transfer function line between the horizontal integral projection curves, the apparatus10may also include means, such as a processor12or the like, for determining the slope myof the transfer function line that relates the vertical integral projection curves of the face region of the first frame and the corresponding window of the second frame.

The determination of the optimal mapping between the integral projection curves by the transfer function, such as the transfer function line, may also be viewed graphically by fixing a pivot at some point on the diagonal line represented by dashed lines inFIG. 5and then rotating the transfer function line to different angles θ corresponding to different slopes m, with the optimal mapping being defined by the transfer function line that provides the minimum distance as defined by the foregoing equations for mxand my. In this analysis, it is noted that the pivot may be fixed at four different places, that is, at (k*M/4, k*M/4) with k being an element [0,3].

The apparatus10of one example embodiment may also include means, such as a processor12or the like, for determining the scaling factor based upon the scaling factors in the x and y directions. See operation52ofFIG. 4. In this regard, the scaling factor mfmay be determined to be the average of mxand my, that is, mf=(mx+my)/2. Based upon the scaling factor, the size of the window in the second frame may be modified based upon the scaling factor, such as being increased or decreased in size based on the scaling factor. As shown in operation54ofFIG. 4, the apparatus10may also include means, such as a processor12or the like, for determining the integral gradient projection curve for the window in the second frame following resizing of the window based upon the scaling factor. In this regard, the horizontal and vertical integral gradient projection curves h″(x) and v″(y) may be determined as described above, albeit with a resized or scaled window. The apparatus10of one example embodiment may also include means, such as processor12or the like, for redefining the lengths of the horizontal and vertical integral gradient projection curves h″(x) and v″(y) to be the same as the projection length for the first frame, that is, M.

The apparatus10of one embodiment may also include means, such as the processor12or the like, for determining a distance between the integral gradient projection curves of the face region in the first frame and the resized window in the second frame, such as in the manner described above in conjunction with the equations for fdxand fdy. See operation56ofFIG. 4. Thereafter, the apparatus10may include means, such as a processor12or the like, for determining whether the scaling factor is verified based on a relationship between the distance between the integral gradient projections and a predefined threshold. See operation58. For example, the scaling factor may be verified by comparing the distances fdxand fdyto a predefined threshold thr1as described above in conjunction with operation28ofFIG. 2and operation44ofFIG. 3. While the same threshold is utilized as that described above, a different threshold may be utilized and, indeed, a different threshold may be used for the x and y directions in one embodiment. In an instance in which the scaling factor is verified, the apparatus10may include means such as the processor12or the like, for redefining the predetermined size of the face region and the corresponding window for other frames, such as subsequent frames, based upon the scaling factor. See operation60ofFIG. 4. Alternatively, in an instance in which the scaling factor is not verified, such as by one or more of the distances failing to satisfy the predefined threshold, the method, apparatus and computer program product of one example embodiment may perform a face search utilizing a pattern recognition based face classifier as shown in operation62ofFIG. 4.

In an instance in which neither the translation motion nor the scaling factor is verified, there may have been pose changes and substantial blurs between the first and second frames. As such, the apparatus10may include means, such as the processor12or the like, for performing a constrained face search using a pattern recognition based face classifier. This face search may be performed for a predetermined number of face sizes, such as x, 1.25x and 0.75x, with x being the face size for the face that was previously found. For each face size, the apparatus, such as a processor, may perform the face search in a region of interest surrounding the location at which the face was previously located in a prior frame. By performing a face search using a pattern recognition based face classifier for the predetermined number of face sizes, a face may still be found in the first and second frames in the event of pose changes and massive blurs in some instances.

The use of integral gradient projections may be advantageous in order to identify translation motion and/or scaling in a reliable fashion, particularly with respect to video in which translation and zoom or scaling occurs regularly. By determining translation motion and/or scaling in a manner that is successfully verified in a meaningful percentage of instances, the face tracking may be successful in a large number of instances, thereby increasing the efficiency of the face tracking process.

Accordingly, blocks of the flowcharts ofFIGS. 2-4support combinations of means for performing the specified functions and combinations of operations for performing the specified functions. It will also be understood that one or more blocks of the flowcharts, and combinations of blocks in the flowcharts, can be implemented by special purpose hardware-based computer systems which perform the specified functions, or combinations of special purpose hardware and computer instructions.