AN EYE LOCALIZATION FILTER

Abstract: A system for fast eye localization based on a filter utilizes the relatively high horizontal-contrast density of the eye region to determine eye positions in a greyscale image of a human face. The system comprises a camera for scanning an individual and a processor for performing the required filtering. The filtering includes a horizontal-contrast computation filter, a horizontal-contrast density determination filter, facial geometry reasoning and eye position determination and works with various eye shapes, face orientations and other factors such as eye glasses or even when the eyes are closed.

Description: AN EYE LOCALIZATION FILTER Background of the Invention Field of the Invention The present invention relates to determining eye positions and more particularly to utilizing the relatively high horizontal-contrast density of the eye region in contrast with the greyscale image of a face. Description of the Prior Art For many visual monitoring and surveillance applications, it is important to determine human eye positions from an image sequence containing a human face. Once the human eye positions are determined, all of the other important facial features, such as positions of the nose and mouth, can easily be determined. The basic facial geometric information, such as the distance between two eyes, nose and mouth size, etc., can further be extracted. This geometric information can then be used for a variety of tasks, such as for recognizing a face from a given face database. The eye localization system can also be directly used for detecting the sleepy behavior of a car driver. Some techniques exist for eye localization based on the Hough transform, geometry and symmetry checks and deformable models. Most of these techniques are not sufficiently robust against shape changes. These systems also require an extensive amount of computer processing time. Furthermore, none of these existing systems can locate eyes when the eyes are closed. Summary of the Invention The present invention is a system for fast eye localization which is based on filters that utilize the relatively high horizontal-contrast density of the eye region to determine eye positions in a greyscale image of a human face. The system comprises a camera that scans an individual and is attached to a processor which performs required filtering. The filtering comprises a horizontal-contrast computation filter, a horizontal- contrast density determination filter, facial geometry reasoning and eye position determination. Brief Description of the Drawings Figure one illustrates one embodiment of the present invention. Figure two illustrates a signal flow diagram of the filtering of the present invention. Figure three illustrates the horizontal-contrast filter utilized in the present invention. Figure four illustrates the horizontal-contrast density determination. Figure five illustrates the results of the horizontal-contrast filter and the horizontal-contrast density determination. Figure six illustrates facial geometry reasoning. Figure seven illustrates another embodiment of facial geometry reasoning. Figure eight illustrates eye position determination. Figure nine illustrates eye localization for representative faces. Figure ten illustrates three typical representative frames from a video sequence. Figure eleven illustrates examples showing the performance of the system with and without glasses. Detailed Description of the Invention The present invention utilizes the relatively high horizontal contrast of the eye regions to locate eye positions. The basic system as shown in Figure 1 comprises a camera 11 that scans an individual 12 and is connected to a processor 13 which performs required filtering of the scanned image. The filtering includes a horizontal-contrast computation, a horizontal-contrast density determination, facial geometry reasoning and eye position determination. The signal flow diagram of the filtering of the present invention is shown in Figure 2. From Figure 2, the greyscale image of the face is an input to the horizontal-contrast filter. The output of the horizontal-contrast filter, the filtered image, is then sent to the horizontal-contrast density filter for further filtering. The output of the horizontal-contrast density filter flows to the facial geometry reasoning section of the system. The output from the facial geometry reasoning section is sent to the eye position determination section of the system. The output from the eye position determination section, the output of the present invention, is the left and right eye positions. The operation of the horizontal-contrast filter, horizontal-contrast density filter, facial geometry reasoning and eye position determination are described below. The signal flow diagram of the horizontal-contrast filter is shown in Figure 3. The horizontal-contrast filter operates as follows. In a small local window of the size m pixels by n pixels in the image, a summation in the horizontal direction over m pixels is performed at first for smoothing out the vertical structures within the filter window. Then, the maximum difference among the m pixels summation values is calculated. If this maximum difference is larger than a given threshold, the pixel is classified as a pixel with high horizontal- contrast. If the horizontal-contrast is high and if the values si, ..., sn are in decreasing order, the output of the filter is "1" which is representative of the "white" pixels in an image. Otherwise, the output of the filter is "0" which corresponds to the "black" pixels in an image. As is known in the art, a window of size 3x3 pixels or 5x5 pixels is sufficient for an input image of size 256 by 256 pixels. A typical input greyscale image of the face and the corresponding output image, the binary mask image, of the horizontal-contrast filter are shown in figures 5a and 5b respectively. It is important to note that the horizontal-contrast filter described above is only one of many possible embodiments. Most existing horizontal-edge detection techniques can also be used with some minor modifications. There are two observations that can be made from the binary mask image that is output from the horizontal- contrast filter. First, the output of the horizontal- contrast filter is a "1" near the eyes and the hair, as well as near the nose and the lips. Second, the filter gives some spurious responses in the regions that do not correlate with facial features. In order to clean up the binary mask image and to generate a more suitable image for eye localization, horizontal-contrast density determination is required. Horizontal-contrast density determination is shown in Figure 4. The binary mask image output from the horizontal-contrast filter is sent to the horizontal- contrast density filter. A search of "white" pixels in the binary mask image is performed. A relatively large window, such as 30 by 15 pixels is used to count and threshold the number of "white" pixels within this window for each "white" pixel in the binary mask image shown in Figure 5(b). In other words, for each "white" pixel, the number of "white" pixels in its neighborhood within the window are counted. Since the number of "white" pixels within the local window can be seen as the density of the pixels with high horizontal-contrast, this number is referred to as the horizontal contrast density. A threshold is then applied for removing the output pixels with contrast density below a threshold for cleaning up the effects of noise and irrelevant features. Figure 5(c) shows the greyscale mask image depicting the output of the horizontal-contrast density filter. Figure 6 illustrates facial geometry reasoning where a-priori information about the geometry of facial features is used to detect and verify the eye positions. Since the eyes usually have a very high (and most likely the maximum) horizontal-contrast density, we search for the maximum intensity in a given area of the greyscale mask image received from the horizontal-contrast density filter for the first estimation. For most images, it can be assumed that the eyes are not located in the upper quarter of the image. Hence, the top one fourth of the mask image can be skipped for searching the maximum pixel value. Likewise, the bottom one fourth of the mask image can also be skipped for searching the eye locations. Eliminating these regions lowers the computational cost of the present invention. After the maximum pixel in the mask image is localized, verification of whether this position really corresponds to one of the two eye positions occurs. The fact that the two eyes should be located within a horizontal strip of width 2k+l is utilized (allowing for a small tile of the head) . The column-wise sum (projection) of the pixels in this strip are then computed. This results in a one dimensional (ID) curve Cl which has two significant peaks corresponding to the eye regions. If two significant peaks are not found, the search area is changed and the procedure is performed again. Figure 7 illustrates a second embodiment of facial geometry reasoning. This embodiment utilizes more information on facial geometry to refine the verification procedure for eye localization. One possible approach is to use the additional information of the mouth to make the verification more robust. As shown in figure 5(c), the horizontal-contrast density filter usually has a strong response near the eyes as well as near the mouth. After detecting the peaks in Cl, the system looks for a strong response for the mouth below the eyes. Since the distance between the two peaks in the Cl curve indicates the approximate distance between the two eyes, an approximate region for the mouth can be estimated. A one dimensional (ID) curve C2 for this region can then be generated. A strong peak in C2 verifies the position of the mouth, which in turn, verifies the position of the eyes. Figure 8 illustrates the eye position determination which refines the eye positions provided by the facial geometry reasoning of Figures 6 or 7. The original greyscale image of the face and the approximate eye positions provide the required inputs. A low pass filter is applied to the original greyscsale image within small windows around the approximate eye positions. A search then occurs for the minimum within small windows around the approximate eye locations and the positions of minima, the output, are the iris positions. Testing of the present invention has been performed on video sequences of different people. The test results have been recorded under different indoor illumination conditions with minimum background clutter. All of the images were subsampled to a resolution of 256x256 pixels. The system needed about 200 msec, on a SUN SPARC 10 workstation for locating both eyes for a 256x256 image. Figure 9 illustrates facial images of different people with a crosshair indicating the eye positions determined by the present invention. Figures 10a, 10b and 10c illustrate three typical representative frames from a video sequence with eye closure and variation of head size and orientation. Figure 10a represents the case when both eyes are closed. Figure 10b shows a change of head size and a slight change of head orientation. Figure 10c represents a change of head orientation. Figure 11 illustrates the performance of the system with and without eye glasses. The present invention is very simple, fast and robust against different eye shapes, face orientations and other factors such as eye glasses. Another distinctive and important feature of the present invention is that the system can detect eye regions even when both eyes are closed. The system can operate very quickly on a general purpose computer. As an example, for a facial image with 256x256 pixels, the system utilizes only 200 msec, on a SUN SPARC 10 workstation. The present invention can be implemented with specialized hardware for real time performance. It is not intended that the present invention be limited to the hardware or software arrangement, or operational procedures shown disclosed. This invention includes all of the alterations and variations thereto as encompassed within the scope of the claims as follows.

Claims: WE CLAIM : 1. An eye localization filter comprising: imaging means for scanning an individual; and, processor means connected to said imaging means for locating positions of two eyes of said individual. 2. An eye localization filter as claimed in claim 1 wherein said processor means comprises: horizontal-contrast computation filter means; horizontal-contrast density determination filter means connected to said horizontal-contrast computation filter means; facial geometry reasoning means connected to said horizontal-contrast density determination filter means; and, eye position determination means connected to said facial geometry reasoning means. 3. An eye localization filter as claimed in claim 2 wherein said horizontal-contrast computation filter means comprises: summation means for receiving a greyscale image of a face and for smoothing out vertical structures within a filter window; and, calculation means for calculating maximum difference among summation values, for analyzing said maximum difference and for providing a binary mask image. 4. An eye localization filter as claimed in claim 3 wherein said horizontal-contrast density determination filter means comprises: pixel search means for searching white pixels in said binary mask image; count means for counting number of white pixels within a local window for each white pixel; and, threshold means for removing output pixels with contrast density below a threshold and for providing a greyscale mask image. 5. An eye localization filter as claimed in claim 4 wherein said facial geometry reasoning means comprises: determination means for establishing a row having maximum pixel value in a selected search area in said greyscale mask image; compute means for computing column wise sum of pixels in a strip; and, analysis means for establishing if said strip has two peaks and for providing approximate positions of eyes. 6. An eye localization filter as claimed in claim 4 wherein said facial geometry reasoning means comprises: determination means for establishing a row having maximum pixel value in a selected search area in said greyscale mask image; first compute means for computing column wise sum of pixels in a first strip; first analysis means for establishing if said strip has two peaks; second compute means for computing column wise sum of pixels in a second strip below said first strip; and, second analysis means for establishing if said second strip has one peak and for providing approximate positions of eyes. 7. An eye localization filter as claimed in claim 5 wherein said eye position determination means comprises: low pass filter means for filtering said greyscale image within small windows around said approximate positions of eyes; and, search means for searching minimum within small windows around said approximate positions of eyes and for outputting positions of two eyes. 8. An eye localization filter as claimed in claim 6 wherein said eye position determination means comprises: low pass filter means for filtering said greyscale image within small windows around said approximate positions of eyes; and, search means for searching minimum within small windows around said approximate positions of eyes and for outputting positions of two eyes. 9. An eye localization filter comprising: imaging means for scanning an individual; and, processor means connected to said imaging means wherein said processor means comprises; horizontal-contrast computation filter means for receiving a greyscale image of a face and for providing a binary mask image; horizontal-contrast density determination filter means for receiving said binary mask image and for providing a greyscale mask image; facial geometry reasoning means for receiving said greyscale mask image and for providing approximate positions of two eyes; and, eye position determination means for receiving said greyscale image of a face and said approximate positions of two eyes and for providing positions of two eyes. 10. An eye localization filter as claimed in claim 9 wherein said horizontal-contrast computation filter means comprises: summation means for receiving said greyscale image of a face and for smoothing out vertical structures within a filter window; and, calculation means for calculating maximum difference among summation values, for analyzing said maximum difference and for providing said binary mask image. 11. An eye localization filter as claimed in claim 10 wherein said horizontal-contrast density determination filter means comprises: pixel search means for searching white pixels in said binary mask image; count means for counting number of white pixels within a local window for each white pixel; and, threshold means for removing output pixels with contrast density below a threshold and for providing said greyscale mask image. 12. An eye localization filter as claimed in claim 11 wherein said facial geometry reasoning means comprises: determination means for establishing a row having maximum pixel value in a selected search area in said greyscale mask image; compute means for computing column wise sum of pixels in a strip; and, analysis means for establishing if said strip has two peaks and for providing said approximate positions of two eyes. 13. An eye localization filter as claimed in claim 11 wherein said facial geometry reasoning means comprises: determination means for establishing a row having maximum pixel value in a selected search area in said greyscale mask image; first compute means for computing column wise sum of pixels in a first strip; first analysis means for establishing if said strip has two peaks; second compute means for computing column wise sum of pixels in a second strip below said first strip; and, second analysis means for establishing if said second strip has one peak and for providing said approximate positions of two eyes. 14. An eye localization filter as claimed in claim 12 wherein said eye position determination means comprises: low pass filter means for filtering said greyscale image within small windows around said approximate positions of eyes; and, search means for searching minimum within small windows around said approximate positions of two eyes and for outputting said positions of two eyes. 15. An eye localization filter as claimed in claim 13 wherein said eye position determination means comprises: low pass filter means for filtering said greyscale image within small windows around said approximate positions of two eyes; and, search means for searching minimum within small windows around said approximate positions of two eyes and for outputting said positions of two eyes. 16. A method of locating eyes comprising the steps of: scanning an individual with a camera; providing a scanned image; processing said scanned image, wherein said step of processing comprises: horizontal-contrast computation filtering of a greyscale image of a face for providing a binary mask image; horizontal-contrast density determination filtering of said binary mask image for providing a greyscale mask image; performing facial geometry reasoning on said greyscale mask image for providing approximate positions of two eyes; and. performing eye position determination of said greyscale image of a face and said approximate positions of two eyes for providing positions of two eyes. 17. A method of locating eyes as claimed in claim 16 wherein horizontal-contrast computation filtering comprises the steps of: performing a summation on a greyscale image of a face; smoothing out vertical structures within a filter window; calculating maximum difference among summation values; analyzing said maximum difference; and, providing a binary mask image. 18. A method of locating eyes as claimed in claim 17 wherein horizontal-contrast density determination filtering comprises the steps of: searching white pixels in said binary mask image; counting number of white pixels within a local window for each white pixel; removing output pixels with contrast density below a threshold; and, providing a greyscale mask image. 19. A method of locating eyes as claimed in claim 18 wherein performing facial geometry reasoning comprises the steps of: establishing a row having maximum pixel value in a selected search area in said greyscale mask image; computing column wise sum of pixels in a strip; analyzing if said strip has two peaks; and, providing approximate positions of eyes. 20. A method of locating eyes as claimed in claim 19 wherein performing eye position determination comprises the steps of: filtering said greyscale image within small windows around said approximate positions of eyes; searching minimum within small windows around said approximate positions of eyes; and, outputting positions of two eyes.

Inventors: CHIU MING-YEE, FANG MING, SINGH AJIT, CHIU, MING-YEE, FANG, MING, SINGH, AJIT
Applicants: SIEMENS CORP RES INC, SIEMENS CORPORATE RESEARCH, INC.
Application year: 1996
Main classification: G06K9
IPC codes: G06K9, G06T1, G06T7