IS3-17

第16回画像センシングシンポジウム，横浜，2010年6月

Local Independent Components Analysis-Based Facial Features Detection Method
M. Hassaballah1,2, Tomonori KANAZAWA3, Shinobu IDO3, Shun IDO1
1

Department of Computer Science, Ehime University, 790-8577, Japan2

Department of Mathematics, Faculty of Science, South Valley University, Qena, Egypt
3

eCompute Corporation, 2821-4, Matsuyama, Ehime 791-8042, Japan
E-mail: m.hassaballah@ic.cs.ehime-u.ac.jp, ido@cs.ehime-u.ac.jp

Abstract

image patch contains an eye within itself. An ensemble

目や鼻や口といった顔特徴の検出は、その後の顔画像の解析において重要なステップである。本稿では、グ
レースケールの画像から目と鼻を検出する方法を紹介する。独立成分分析法（ICA）を顔特徴の外観や形状の学習に用いることで、ICA 基底ベクトル に対して高応
答の領域を顔特徴として検出した。一層の性能改善のために ICA に加えて目と鼻の局所特徴を適用し、異なるデータベースによる実験において有望な結果を得

network consisting of a multitude of independent MLPs
was used to enhance the general performance of a single
MLP. It was tested on 180 images without glasses from
ORL database and its performance was 91.7% and 85.5%
for left and right eye respectively. The advantage is thatit does not need a large training set by taking advantageof eigenfeatures and sliding window. Kroon et al. [3]address the problem of eye detection for the purpose offace matching in low and standard definition image andた。

video content. They present a probabilistic eye detection
method based on well-known multi-scale local binary

1patterns (LBPs). The detailed survey of the most recent

Introductioneyedetection

methods[4]

concluded

that

the

Facial feature detection plays an important role indevelopment of a general eye detection techniquemany face-related applications including face recognition,

involves addressing many challenges, requires further

face validation, and facial expression recognition [1].

theoretical developments, and is consequently of interest

These applications need to detect the facial features

to many other domains problems in computer vision androbustly and efficiently. However, this is not an easy

beyond.

machine-vision task at all. The difficulty comes fromhighinter-personalvariation(e.g.

gender,On the other hand, there are few methods that addressrace),

nose detection problem, even though it is not lessintra-personal changes (e.g. pose, expression), and fromimportant than eye or other facial features. It does notacquisition conditions (e.g. lighting, image resolution).

affect so much by facial expressions and in several cases

In spite of these difficulties, several methods haveis the only facial feature which is clearly visible during

been proposed to detect certain facial features, especiallythe head motion. Most of the existing approaches detect

eye as it is the most salient and stable feature of thenose depending basically on the prior detection of eye

human face. Ryu and Oh [2] introduced a method based

center and considering nose can be located within certain

on eigenfeatures derived from the eigenvalues andpixels below the line connecting the centers of two eyes

eigenvectors of the binary edge data set and neural[5]. Other approaches detect nose depending on the

networks to detect eye region. The eigenfeaturesreflection of light on the nose tip or using projection

extracted from the positive and negative training samples

methods [6]. However, all existing projection methodsof eye are used to train a multilayer perceptron (MLP)

do not consider complex conditions such as illumination

whose output indicates the degree to which a particular

IS3-17-1

第16回画像センシングシンポジウム，横浜，2010年6月

and pose variation, thus they will fail under these
imaging conditions. In [7] a method for facial landmarks‘linear mixtures’ in its rows X = [ X 1 , X 2 ,..., X m ] T , A

detection based on extracting oriented edges andis a nonsingular mixing matrix, and S denotes the source

constructing edge maps at two resolution levels is

matrix containing statistically independent source vectors

proposed. It can achieve average nose detection of 78%
on 330 images from Pictures of Facial Affect database.

in

its

rows

S = [ S 1 , S 2 ,..., S m ]

T .

Under

the

The method was not fully automatic and required manualassumptions that the sources are statistically independentclassification of the located edge regions.

and non-Gaussian (at most one of them can have

In spite of considerable amount of previous work on

Gaussian distribution), an un-mixing matrix W can be

the subject, detection of facial features will remains afound by maximizing some measure of independence. In

challenging problem because the shape and texture ofother words, the estimated separation matrix, W underfacial features vary widely under changing expression,

ideal conditions, is the inverse of the mixing matrix A.

head pose and illumination. Therefore, new robust
Y =W X , W = A

methods are required. In this paper, we present a method
to detect facial features (e.g. eye and nose). The method

−1

,

and

Y ≈S

(2)

Generally, ICA aims to estimate the un-mixing matrix

depends on higher-order statistics and second-orderW

andmoments of the training data decorrelated by ICA basis

using S

vectors as well as local characteristics of facial features.

thus

to

recover, where W
k = Wk X
keachhiddensourceis the kth row of W

The novelty of the method is to apply ICA on parts ofwithout any prior knowledge. There are a number offace containing facial features rather than whole facealgorithms to estimate the un-mixing matrix W in theimage as done in face recognition field. This new use of

ICA model (2). The FastICA algorithm [8] is used in thisICA might capture the advantages of spatially localized

work to estimate the ICA basis vectors.

of its basis vectors which will help in highlighting salient

The FastICA is basically based on a fixed-point

regions in the data and provide a better probabilistic

iteration scheme for finding a maximum of themodel of the training data. It was evaluated on different
databases under various imaging conditions.

nongaussianity of WT X , where the nongaussianity is

The remainder of this paper is organized as follows. A

measured by the maximization of the so-called

brief background about ICA is introduced in Section 2.

negentropy. According to [8] and after a series of

The outline of the proposed method for detection ofmathematical calculations, the ICs can be estimated byfacial features is presented in Section 3. ExperimentalTTwi ← E{ z g ( wi z )} − E{ g ′( wi z ) } wi

results are reported in Section 4, the conclusions are
given in Section 5.

2

(3)

T
where wi is a row vector in W, z is the preprocessing

Independent component analysis

whitening data and the nonlinearity g can be any smooth

Independent component analysis can be seen as anunsupervised learning method based on higher orderfunction that gives robust approximations of negentropy
such as,g ( y ) = tanh(α y ) ,statistics for extracting independent sources from an
observed mixture, where neither the mixing matrix nor
the distribution of the sources are known. Formally, the1≤α ≤ 2

(4)

The estimation process in (3) is repeated for a chosenn (number of basis vectors) with orthogonalize theclassical model of ICA is given as
X = AS(1)

where X denotes the observation matrix containing thematrix W = ( w1 , w2 ,...wn ) T using W ← (WW T ) −1/ 2 W ,where

IS3-17-2

第16回画像センシングシンポジウム，横浜，2010年6月(WW

T −1 / 2−1 / 2−1 / 2T= E dig ( d 1),.... d n
) E

(5)

The convergence will occur when the absolute valueof the dot-product of old and new W is (almost) equal 1,
i.e. < W t , W t +1 > = 1 . More information and details
about ICA and FastICA can be found in [8].

3Fig. 2: Eye subspace; the highest 20 ICA basis images

Method Overview

3.1 Computing ICA features subspaces
A set of 200 images for eye and nose is used in the
training step. In order to minimize the impact of noise orvariations in illumination each image is preprocessedwith norm normalization. The width w and height h of
the training images are determined experimentally withrespect to the face width WF. Examples of eye and nosetraining images are shown in Fig. (1). Then the FastICAalgorithm is implemented to estimate the ICA basis
vectors which will form the feature subspace. The

Fig. 3: Nose subspace; the highest 30 ICA basis images

number of basis vectors for each subspace is optimized
experimentally. Hence, 20 vectors are found sufficient

3.2 Detection of facial features

for eye subspace, while 30 vectors are sufficient for nose

Searching for facial features in the image can be done

subspace. The eye and nose subspaces are shown in Fig.

either directly in the entire image, or rely on the output(2) and (3) respectively. It is clear that the computedof a face detector indicating that these features arebasis vectors of each subspace are spatially localized topresent in the image. Unfortunately, searching in thethe salient regions of eye and nose such as iris, eyelids,

whole image is not a suitable method for real time

nostrils, textures, and edges. This localization might

implementations and is more prone to errors. Therefore,robust the performance to local noise and distortion.

a face detector [9] is applied first to locate the face, thensearching for regions that contain eye and nose is done in
the located face. The located face is normalized to a
fixed size of WF x WF pixels. The normalized face isscanned to find the region which might contain eye/nose,
the regions data are treated as w x h dimensional vectors.

(a) Eye training images 60 x 30 pixels

Each region (vector) is locally preprocessed using norm
normalization as in the training step. The subspaces
corresponding to the feature point vector in the w x hdimensional facial feature subspace can be expressed aslinear subspaces spanned by multiple ICA vectors. The
projection angle θ of an input vector projected onto the(b) Nose training images 70 x 50 pixels

ICA subspace represents the extent to which the input

Fig. 1: Facial Features training images

vector is analogous to the feature point vector. For
verification the value of θ; specifically Cos (θ) between

IS3-17-3

第16回画像センシングシンポジウム，横浜，2010年6月

the input vector and each feature point’s subspace (ICA

(N=30 in this work). To detect the region that mostbasis vectors) is obtained. This angle measure is

probably contains eye using the proposed eye varianceinvariant under linear changes in the contrast of thefilter, variance vector of each candidate region of theimage and furthermore the cosine similarity measure wasface is calculated in the same manner as in the eyepreviously found to be effective for face processing. Theeye or nose region is the input vector that falls into its

variance filter Fe . Then the correlation between thesubspace with the smallest θ or highest similarity. The

variance vector of each subblock of the region and thesimilarity Sθ ( R j , ICAVectors) between the region Rj,
jeye variance filter Fe (7) is calculated using this formula

j =1,2, ...M (number of regions) and ICA basis vectors is
calculated using Cos (θ) of the projection componentn2∑ V , BaseVector i2i =1
S θ = Cos θ j =2jV

R( Iσ , Fe ) =(6)where n is the number of basis vectors that form thefeature subspace andV , BaseVectori

E[(ξ Iσi− E (ξ Iσi)) (ξ F − E(ξ F )]
e
e

(8)

D(ξ I

) D(ξ F )eσiwhere ξ I and ξ F are the concatenated vectors ofe
σivariance of subblock I σi of the eye candidate regionis the innerproduct between input vector (representing j-th candidate

and Fe respectively.

region) and i-th base vector of the ICA subspace. The

All the regions which have high response to the eyeregion Rk with the highest similarity is declared as thevariance filter are selected to generate a small list oftarget facial feature region.

possible eye region pairs. If the response of the candidate
region to Fe is greater than expected threshold (e.g. =4),

3.3 Local characteristics facial features
The performance of the method described in section
3.2 can be improved significantly if local characteristics
of eye and nose are used besides ICA method. For eye, avariance filter is used. To construct this eye variancefilter a set of 30 eye images of left and right eye of same

this means that the candidate region might contain eye.
After that, all the selected regions by applying eye
variance filter are then verified using ICA method
mentioned in section 3.2 to select only two regions
which represent the left and right eye.

size (i.e. 60 x 30 pixels) as the eye training images isselected randomly of different persons. Each eye image
is divided into 3 x 3 non-overlapped subblocks of size
20 x 10 pixels thus, each subblock has its unique features
as shown in Fig. (4). The variance of each subblock is
calculated, and then the eye variance filter Fe isFig. 4: Variance filter modelOn the other hand, for nose three different local

constructed by calculating average of the variance in thecharacteristics are used as follows:

subblock images over all 30 eye images(i) Similarity of both sides: The left and right sides of

Fe =

1
N

N
j
∑ Iσ
j =1

nose are similar in a front-view face as shown in Fig.
(7)jwhere Iσ is the j-th variance filter of the σ subblockeye image and N is the number of used eye images(5a), this property of similarity can be measured using
Euclidean distance between both sides.

(ii) Dark-White-Dark (DWD) property: Also, thelower part of nose region is characterized by two dark

IS3-17-4

第16回画像センシングシンポジウム，横浜，2010年6月

nostrils and a light subregion due to the reflection ofBecause of noise in the images due to lighting condition

light on the nose as shown in Fig. (5b). This property canor facial expression, the detected region is not accurate

be identified by the average of gray intensity in eachand thus the whole successful detection rate (i.e. images

subregion, where the average in the two nostrils regionswith correctly detected eye/nose region relative to theis less than the average of middle lighter subregion

whole set of facial database) is not so high, especially on

containing nose tip.

BioID and JAFEE databases. The best results have been(iii) The variation in lower/upper parts property:
When the face is rotated some degrees these two
properties are despaired and the only clear property is thevariation between lower part and upper part as shown in
Fig. (5c). This variation can be measured by the variancein each part. Based on this analysis, we search for acertain region among the ten highest regions detected by
ICA method which satisfies the properties (i)-(iii). Notethat in the case of eye detection ICA method has been
applied before local characteristics while in nose

obtained on the XM2VTS database because the image
quality is very high, the illumination is always uniform,
and the image scale is fixed.
The performance of ICA method is improved
significantly when the local characteristics of eye andnose are used besides ICA as shown in Table (2). In
some databases such as XM2VTS an improvement rate
more than 5% has been achieved while in BioID
database the improvement is not significant due to thesevere conditions in this database.
Compared with other methods [2] achieved detection

detection the opposite has been done.

rate of 91.7% and 85.5% for left and right eye on 180
images without glasses from ORL database, the proposed
method achieves an average eye detection rate of 94.3 %on about 4000 images while for nose it can achieve an
average detection rate of 96 % outperforming [7] which

4Fig. 5: The local characteristics of nose region

achieved average detection rate of 78% on 330 images

Experimental Resultsfrom Pictures of Facial Affect database. Example ofsuccessful results is given in Fig. (7).

In order to find optimum number of ICA basis vectors

The main advantages of the proposed method are; thefor both feature spaces the proposed method was

method is very simple. The average execution time of theevaluated on XM2VTS database as shown in Fig. (6).

method on a PC with Core(TM)2 Duo CPU, 2.4 GHz,

The highest successful detection rate of 93.2 % for eye isand 4GB RAM is less than 80 msec. Moreover, detection

achieved when the number of ICA components is 20

of nose does not depend on eye detection as in thewhile for nose 95.8% successful detection rate achieved

existing methods. Future research will focus on detecting

at 30 vectors, and after that the marginal benefit slows.

fiducial points of eye and nose such as eye corners, iris,Also, there is trade-off between running time and ICA

nostrils or nose tip within the detected region as well as

dimension. Using fewer basis vectors requires lessmouth corners.

storage and results in faster matching hence 20 and 30
vectors are sufficient. Therefore, the number of ICA
basis vectors is set to the first highest 20 and 30 basis
vectors for eye and nose subspaces respectively.
The proposed ICA method is also evaluated on otherthree databases namely; 1500 images of FERET, BioID,
and JAFFE. In this context, the facial features aredetected based on the highest response to feature space.
The results of this experiment are reported in Table (1).

Fig. 6: Selection the optimum number of basis vectors

IS3-17-5

第16回画像センシングシンポジウム，横浜，2010年6月

Table 1: Performance of ICA only on different databases

Reference

FeatureEye

XM2VTS
93.2%

FEERT
91.3%

BioID
87%

JAFFE
85.3%

[1] Asteriadis S., Nikolaidis N., Pitas, I., Facial

Nose

95.8%

94.3%

91.8%

91.5%

feature detection using distance vector fields,
Pattern Recognition, 42, 1388-1398, 2009.

Table 2:Performance of ICA besides local characteristics
FeatureEye

XM2VTS
98.4%

FEERT
97.1%

BioID
91.6%

JAFFE
90.2%

Nose

99%

96%

92.3%

96.2%

[2] Ryu Y.-S., Oh S.-Y., Automatic extraction of eye
and mouth fields from a face image using
eigenfeatures and ensemble networks, AppliedIntelligence, 17, pp. 171-185, 2002.
[3] Kroon B., Maas S., Boughorbel S., Hanjalic A.,
Eye localization in low and standard definition
content with application to face matching,Computer Vision and Image Understanding, 113(8), 921-933, 2009.
[4] Dan W. H., Qiang J., In the eye of the beholder:
A survey of models for eyes and gaze, IEEE
Trans.

on

Pattern

Analysis

andMachineIntelligence, 32(3), pp. 478-500, 2010.
[5] Sankaran, P., Gundimada, S., Tompkins, R.C.,
Asari, V.K., Pose angle determination by faces,

Fig. 7: Example of successful results

eyes and nose localization, IEEE CVPR’05,
161-167, 2005.

Conclusions[6] Campadelli, P., Lanzarotti, R., Fiducial point
In this paper, an automatic method to detect eye andnose location from facial images based on the responseof face regions to feature subspaces is presented. Eye andnose spaces are estimated using independent components
analysis basis vectors. In order to further improve theperformance of the method, we proposed a subregionbased framework that depends on local characteristics of
facial features regions. The efficiency of the method is
evaluated in different databases stressing variety ofimaging conditions. It has been shown by experimental
results that the proposed method can accurately detect
the eye/nose with high detection rate comparable withexisting methods.

localization in color images of face foregrounds,
Image and Vision Computing, 22, 863-872, 2004.
[7] Gizatdinova

Y.,

SurakkaV.,detection of facial landmarks from neutral andexpressive facial images, IEEE Trans. on PatternAnalysis

andMachineIntelligence,

The authors would like to acknowledge the financial
support of the Egyptian government for this work underGrant No.1/13/101-2006-2007. The authors would also
like to thank the owners of used databases XM2VTS,FERET, BioID, and JAFFE for experiments of this work.

IS3-17-628(1),

135-139, 2006.
[8] Hyvarinen A., Oja E., Independent component
analysis: algorithms and applications, Neural
Networks, 13, pp. 411-430, 2000.
[9] Viola P., Jones J. M., Robust real-time facedetection, International Journal of Computer
Vision, 57(2), pp. 137-154, 2004.

Acknowledgments

Feature-based