Hairstyle Suggestion Using Statistical Learning
Wei Yang, Masahiro Toyoura, and Xiaoyang Mao
University of Yamanashi
4-4-11 Takeda, Kofu, Yamanashi, Japan 400-8511{g10mk039,mtoyoura,mao}@yamanashi.ac.jp

Abstract. Hairstyle is one of the most important features people use tocharacterize one’s appearance. Whether a hairstyle is suitable or not is said to
be closely related to one’s facial shape. This paper proposes a new techniquefor automatically retrieving a suitable hairstyle from a collection of hairstyleexamples through learning the relationship between facial shapes and suitablehairstyles. A method of hair-face image composition utilizing modern matting
technique was also developed to synthesize realistic hairstyle images. The
effectiveness of the proposed technique was validated through evaluation
experiments.
Keywords: Hairstyle retrieval, example-based, statistical learning, nonparametric sampling, hairstyle image synthesis.

1

Introduction

With the advance of media acquisition and processing technologies, multimedia is
now playing a very important role in enriching our daily life. This paper presents a
new technology for suggesting a user with his/her suitable hairstyles by combing
statistical learning and image processing techniques. Hairstyle is one of the mostimportant features people use to determine their appearance and mood. People canlook completely different simply by changing their hairstyles. Everyone would like tohave a suitable hairstyle to make them look attractive, but it is usually difficult to find
one as we cannot easily try out various styles with our real hair. Several commercials
or free software have been developed allowing users to simulate how they look withdifferent hairstyles by manually selecting hairstyle samples and superimposing themover their facial images. Although these systems do provide some general guidelines
on choosing hairstyles, they do not provide any hints on what a suitable hairstyle isfor a particular face. Therefore, users usually need to go through a very tedious
process of trying out many different hairstyles before the one they like can be found.
On the other hand, several papers have been published related to hairstyles [1,2,3,4] inthe field of computer graphics. However, to the best of our knowledge, all these havefocused on how to model and render hairstyles with computer graphics and havemainly been applied to create virtual characters and animations. In this paper, we
propose a new technique for automatically retrieving a suitable hairstyle for a given
face from a collection of successful hairstyle examples. What is a suitable hairstyle
K. Schoeffmann et al. (Eds.): MMM 2012, LNCS 7131, pp. 277–287, 2012.
© Springer-Verlag Berlin Heidelberg 2012

278

W. Yang, M. Toyoura, and X. Maofor someone? Suitability is a perceptual attribute and it is difficult to model itcomputationally. A hairstyle that appears to be attractive to one person may not lookthat acceptable to another. Personal aesthetics may be affected by many factors, suchas one’s cultural background and living environment. There are also many otherfactors that may affect how one looks in a particular hairstyle. Despite this, there is
still some common aesthetics sense on hairstyles, and a skilled hair stylist can usually
successfully create styles that conform to such common aesthetics sense. We started
our project by interviewing several hair stylists. An important fact we observed is that
although there are no stylists who can tell any explicit rules about their designs, they
all viewed the shape of the face as the most important attribute in designing ahairstyle. This inspired us to adopt an example-based framework, which is anapproach that has been successfully used for texture synthesis [5, 6] and styletransferring [7,8,9,10] in recent years. Our new technique finds suitable hairstyles fora given face by learning the relationship between facial shapes and successful
hairstyles.
The four major contributions of this paper can be summarized as follows:
1.

2.
3.
4.

2

A new framework for retrieving suitable hairstyles through learning therelationship between facial shapes and hairstyles from successful hairstyleexamples.
The design of a compact feature vector space enabling fast non-parametricsampling in statistical learning.
A method of hair-face image composition utilizing modern matting techniques
for synthesizing realistic hairstyle images automatically.
An evaluation experiment demonstrating the validity of the feature vector andthe effectiveness of the example-based approach.

Example-Based Framework

Given a face image, we want to create another image
matching the face best. We achieve this in two steps:

with hairstyle S

Statistical Learning: Find the most suitable hairstyle S, through learning therelationship between facial shapes and suitable hairstyles.
Composition: Superimpose hairstyle S over face imageof the face in a suitable hairstyle.
image

2.1

to obtain realistic

Statistical Learning

The statistical learning step can be described within the Bayesian inference
framework. Based on Bayes theorem, posterior probability ( |) , i.e., theprobability for face image
to have its best hairstyle S, can be represented as:
|

=

|

( )

.

(1)

Hairstyle Suggestion Using Statistical Learning

279( ) is the prior probability of S, and P(I|S) is the probability of observed facegiven hairstyle S. Consequently, the aim of finding S can be turned into
image). Since evidence () can bean optimization problem maximizing ( |treated as a constant, S is actually the one maximizing the product of likelihood
and prior (S),=| ) ( ).

(

(2)

We obtain S by using non-parametric sampling, which has been proved to be an easy
yet very efficient method in style transferring and texture synthesis applications[5,6,8,9]. Professionally designed hairstyle examples are used as the training data to| ).
learn prior ( ) and likelihood (To compute S, we first construct an approximation to conditional probability
distribution ( |) and sample from this. Assuming ( ,) is the distance
between two facial images under some metric, if we define set
Ω

=

,

,

=0 .

(3)

containing all occurrences of I in training data set T, then the conditional probability
distribution of ( ) can be estimated with a histogram of all the hairstyles for facesin Ω (). In other words, we can obtain a distribution of possible hairstyles. However, since we only have a finite number of examples from the trainingfordata set, we may fail to find any matching facial image
with,= 0. To), let us specify small distance allowance e, andobtain an approximation to Ω (
) =
|
,
,
. The distance function d will beobtain Ω (discussed in Section 3. Allowance e is determined by multiplying the minimum of,with a given constant, in our current implementation.
There are several possible ways of computing S from its conditional probability
distribution:
1. Integrate the distribution and obtain an expected hairstyle image by compositing
all the possible hairstyles.
2. Take the hairstyle with the highest probability. If there is no maximum value inthe distribution, randomly choose a hairstyle.
.
3. Take the hairstyles of the k-nearest-neighbor of
Method 1 may produce some unrealistic hairstyle images due to the discontinuity inthe hairstyles from the distribution. Since the training dataset usually consists of no
more than one hairstyle for one single facial image, the result from Method 2 is
Ω ().
usually a randomly chosen hairstyle from the set of hairstyles
Method 3 is used in our current implementation and the hairstyles of the k-nearestneighbor of
are recommended as candidates for the best suitable hairstyles.
The k can be adjusted interactively.

280

2.2

W. Yang, M. Toyoura, and X. Mao

System Overview( =As shown in Fig 1, our system assumes n successful hairstyle images
1,2, … , ) are available. The three operations below are executed in the trainingis the featurephase to build a training data set T( , )(i = 1, . . . n) where
vector characterizing the shape of each faceandis the α-matte indicating theprobability of a hair area in the image:Fig. 1. System framework

1.
2.
3.

Apply robust matting technique [11] to ( = 1,2, … , ) to create ( =1,2, … , ).
A trained Active Shape Models (ASM) [12] model is used to detect facial featurepoints on ( = 1,2, … , ).
Construct feature vector( = 1,2, … , ) from the ASM model feature points.

While the first operation requires example strokes to be manually specified to create
the tri-map for estimating the α-matte, the other two operations are performed fully
automatically.
Given face image
in the runtime phase, the system performs six operations.
to compute a set of suitable hairstyles for
1.
2.
3.

Apply a trained ASM to detect the facial feature points of.
Construct feature vector
characterizing the shape of the face in
Search through all images in T in the feature vector space.

.

Hairstyle Suggestion Using Statistical Learning

4.
5.
6.

If
,
add
to Ω () .
) by,.
Sort images in Ω (( = 1,2, … , ) from Ω (Take top k images
using ( = 1,2, … , ).
with

281

) and composite them

The design of feature vector V is crucial to obtain good results, as well as to quickly
searches through the training examples. We will discuss this in detail in the next section.

3

Feature Vector Design

Active Shape Models (ASM) [5] is one of the most popular techniques for detecting
the geometric features of faces from images. Instead of using ASM directly, we wantto have a more compact feature vector space, which can successfully model the
relationship between facial shapes and hairstyles. As seen in Fig 2(a), it is known thathuman faces can be roughly classified into four categories by shape: oval, round,
triangular and home base.

Oval

Round

TriangularHome base(a)Shapes of face(b) Hairstyles for round faces

Fig. 2. Facial shapes and their relationships to hairstyles

A hairstyle giving an impression of an oval shaped face is likely to be suitable [13].
For example, as we can see from Fig 2(b), for the round face, the styles with longbangs flowing smoothly toward both sides or the back make the face look longer andhence are suitable, while the one with bangs cut straight across and a thick volume on
top of the head is not suitable because it further emphasizes the impression ofroundness. To find these kinds of relationships between facial shapes and hairstyles,
we first compute six line segments (Fig 3(b)):
h
w1
w2
w3
ht

:
:
:
:
:

hb

:

the center vertical line segment
the horizontal line segment at the height of the eyebrowsthe horizontal line segment at the widest position of the facial area
the horizontal line segment at the height of the mouth
the vertical line segment from the top of the face to the crosssection of h and w1
the vertical line segment from the bottom of the face to the crosssection of h and w3

282

W. Yang, M. Toyourra, and X. Mao

From the six line segments above, we define a six-dimensional feature vecctor
V(v1,v2,v3,v4,v5,v6) with vj(j=1,2,..6)(
being the ratio of two line segments, normaliized
to be in (0,1] (Fig 3(c)). Too automatically compute the feature vector, we trained theASM model with 81 pointss so that it includes the end points of the six line segmeentsas the feature vector (Fig 3((a)).
The distance function forr non-parametric sampling is defined asd IT, I

=∑

.

(4)

Coefficient( = 1,2, … ,6) controls the weight of each dimension in determinningthe hairstyle. For example, if we use a large k1 and a small k2, then the system woould
treat the aspect ratio of the face as being more important and the shape of the cheekk as
being less important in deetermining a suitable hairstyle. Two faces with differrent
shapes for cheeks but a sim
milar aspect ratio may be suggested with similar hairstyyles.
We set all six coefficients to be equal to treat all dimensions homogenously in ourcurrent implementation. We
W plan to explore the best coefficients through machhinelearning in future work. A. Kagian et al used neural network to model theattractiveness of human faace based on a 3240(81C2) dimension feature vector [14].
Each dimension of the featture vector is the length of a line segment connecting ttwo
feature points of ASM moddel and the 3240 dimension corresponds to all the possiible
combination among 81 feeature points. They trained the neural network throuughsubjective an experiment. As
A the future work, we plan to adopt a similar approoach
explore the best feature vectorv
to model the relationship between the geomeetricfeatures of faces and hairsty
yles(a)Result by
ASM(b)Feature lines

v =

h
w

v =

h
w

v =

w
w

v =

w
w

h
h
v =
h
h
(c)Components of feature vector

Fig. 3. Feature vectorv =Hairstyle Suggestion Using Statistical Learning

4283

Hairstyle Image Synthesis

Before we can superimpose obtained hairstyle image S over input face image,
position alignment and size adjustment between the two images are required. This is/ in width,/in height, followed byachieved by scaling H with
with that of. As shown in Fig
translation aligning the upper end point of4(a), due to fitting error with the ASM model, we may fail to obtain centeredor, and this may result in an unnatural composition like the one in Fig 4(b). To
correct error, we translate h horizontally by displacement D (Fig 4 (c)):)= · · ((

(

)

(

).

Fig 4(d) shows an improved result obtained by using the new position offor position alignment.

(a) Error of h(b)Result with displaced h(a)With binary mask(c)Correction of h(d)Result with corrected h(b) With α-matteandFig. 5. Compare composition results
using binary mask and α-matteFig. 4. Position alignment

Finally, the α-matte of S is used to composite S and
= (1

(5)

)

to obtain output image.

(6)

Here, p denotes a pixel, and , , and , correspond to the pixel values for the
output hairstyle image, input image, and suitable hairstyles suggested by the system,
respectively. Fig 5 compares the results of binary mask based composition with our αmatting based method. We can see our method produces more realistic images,
especially at regions near the boundary of hair.

284

W. Yang, M. Toyoura, and X. Mao

5Implementation and Evaluation

5.1

Implementation and Result

We built a training data set consisting of 84 hairstyle images collected through thecourtesy of hair stylists from three hair salons in our current implementation. Througha preliminary user study, we found that in many cases users wanted to specify thelength of their hair before searching for the best hairstyles, and hence it would be
helpful if we could advise them of the best candidates for different lengths. Currently,
both the hairstyle examples and the input face need to be frontal photos and the facial
area in the input image should not be covered by strands of hair. In addition to the six
dimensions representing the shape of a face, three additional dimensions, representing
the length, hardness and volume of hair, are also added and a nine-dimensionalfeature vector is computed for each example. Each hair can take one of three values inthe three newly added dimensions:
LengthVolumeHardness: long, medium, short.
: large, medium, small.
: hard, medium, soft.

The necessity to consider the properties (hardness and volume) of hair arose from thefact that the hairstyle one can actually have is constrained by the properties of his/her
hair, even if one can find the best hairstyle making her look virtually attractive.
Therefore, with the additional dimensions characterizing the properties of hair, we canconstrain sampling to only hairstyles with similar hair properties. The hair propertiesof the input face are specified by the user.
Fig. 8 shows two examples of results. For the input face images at the left, thenearest face in the feature vector space for each of the long, medium and short hairstyletraining sets are shown in the upper row and below it are the resulting hairstyles.
5.2

Evaluation

We conducted two experiments to validate the effectiveness of our approach. The firstwas aimed at investigating how the hairstyles recommended by our system to a person
would look for other people, while the second experiment was aimed at how
satisfactory the result would be for the person herself.
Ten female college students participated in the first experiment. We prepared ninesets of hairstyle images, each consisting of ten hairstyles with two of themrecommended by the system as suitable hairstyles. At each trial, a subject waspresented with one set of images and asked to mark the top three most suitablehairstyles out of the 10 hairstyles. Fig 6 shows an example of the hairstyle image set
used in the experiment. Since hair color can largely affect the impression of hairstyle,we used the monochrome picture in the experiment to exclude the effect of color.
There were a total number of 90 trials (9 sets×10 people) and we evaluated theprobability of the occurrence of the following three cases.

Hairstyle Suggestion Using Statistical Learning

Case 1:Case 2:

Case 3:285

The hairstyles recommended by the system were marked as the top
suitable hairstyle.
At least one hairstyle recommended by the system was included in the
top two suitable hairstyles.
At least one hairstyle recommended by the system was included in the
top three suitable hairstyles.

We used a binomial test and our null hypothesis was that all 10 hairstyles in an image
set would be marked with the same probability. Table 1 summarizes the experimental
results. Out of the 90 trials, there were 20 trials for Case 1, 58 trials for Case 2 and 80
trials for Case 3. The probability for the number of occurrences above those observed
ones upon the null hypothesis is listed at the rightmost column of Table 1. The null
hypothesis was rejected at a significance level lower than 0.05 for the latter two cases.
In other words, the experimental results suggested that the hairstyles recommended byour system at least could be viewed as the second best hairstyle even though it might
not be the best.
The second experiment had the same setting as that for the first except that eachsubject was presented with hairstyle images of herself. The subjects were ten female
college students and 3 image sets were prepared for each of them. Therefore, thesewere a total of 30 trials (3 sets×10 people). The number of trials for the three cases
and the corresponding probability upon null hypothesis are summarized in Table 2.
We can see the null hypothesis was rejected with a very low level of significance forthe latter two cases, which is the same as the results for the first experiment.
Table 1. Result for Experiment 1: Viewed by
Others (binomial test)

Case 1Case 2
Case 3

Number ofoccurrence(Out of 90
trials)
22%
64%
89%

Table 2. Results from Experiment 2: Viewed
by self (binomial test)

Probability
upon null
hypothesis
0.25
0.00
0.00

Case 1Case 2
Case 3

Number ofoccurrence(Out of 30
trials)
17%
53%
83%

Probability
upon null
hypothesis
0.57
0.02
0.00

We can conclude from the experiment results that even though it may not be thebest one, our system can advise users of good candidates for hairstyles viewed to besuitable both by themselves and others.

Fig. 6. Example of image set used for experimentFig. 7. Example of mobile application

286

W. Yang, M. Toyourra, and X. Mao

Fig. 8. Example of results

6

Concluding Rem
marks

We presented a new exam
mple-based framework for creating suitable hairstyles foor agiven face image. Suitabilitty is a perceptual attribute and our evaluation experimeentsdemonstrated the effectiveeness of addressing these kinds of problems with an
example-based approach. Since the proposed technique is fully automatic, a promissing

Hairstyle Suggestion Using Statistical Learning

287

application is to implement it on mobile terminals with camera. For example, shows in
Fig. 7, a user can retrieve a suitable hairstyle before going to hair salon simply bytaking a photo of herself with the camera on her cell phone and then show it to the hairstylist after arrives the hair salon. In future research work, we want to apply the same
framework to other fashion simulation problems, such as advising people of suitableattire by learning the relationships between body shape and successful choice in dress.

References
1. Paris, S., Chang, W., Kozhushnyan, O.I., Jorosz, W., Matusik, W., Zwicker, M., Durand,F.: Hair Photobooth: Geometric and Photometric Acquisition of Real Hairstyles. ACM
Transactions on Graphics 27(3), article 30 (2008)
2. Wang, L., Yu, Y., Zhou, K., Guo, B.: Example based hair geometry synthesis. ACM
Transactions on Graphics 28(3), Article 56 (2009)
3. Xu, Z., Dong, X., Yang, D.: V-hair studio: An interactive tool for hair design. IEEE
Computer Graphics and Applications 21(3), 36–43 (2001)
4. Yu, Y.: Modeling realistic virtual hair styles. In: IEEE Pacific Conference on Computer
Graphics and Applications(PG), p. 295. IEEE Computer Society, Washington (2001)
5. Efros, A., Leung, T.K.: Texture synthesis by nonparametric sampling. In: IEEE
International Conference on Computer Vision (ICCV), vol. 2, pp. 1033–1038 (1999)
6. Hertzmann, A., Hacobs, C.E., Oliver, N., Curless, B., Salesin, D.H.: Image analogies.
ACM Transactions on Graphics 25(4), 1360–1379 (2006)
7. Chang, C., Peng, Y., Chen, Y., Wang, S.: Artistic Painting Style Transformation Using
Example-based Sampling Method. Journal of Information Science and Engineering 26(4),1443–1458 (2010)
8. Chen, H., Xu, Y.Q., Shum, H., Zhu, S.C., Zheng, N.N.: Example-based facial sketch
generation with nonparametric sampling. In: IEEE International Conference on Computer
Vision (ICCV), pp. 433–438 (2001)
9. Chen, H., Liang, L., Xu, Y.Q., Shum, H.Y., Zheng, N.N.: Example-based automatic
portraiture. In: IEEE Asian Conference on Computer Vision, pp. 23–25 (2002)
10. Suo, J., Min, F., Zhu, S., Shan, S., Chen, X.: A multi-resolution dynamic model for face
aging simulation. In: IEEE Conference on Computer Vision and Pattern Recognition,
Minnesota, USA (June 2007)
11. Wang, J., Cohen, M.F.: Optimized Color Sampling for Robust Matting. In: IEEE
International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8(2007)
12. Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models - their training
and application. Computer Vision and Image Understanding 61, 38–59 (1995)
13. Best Hairstyle by Face Shape. Edited by Shufunotomo Co., Ltd (2007) (in Japanese)
14. Kagian, A., Dror, G., Leyvand, T., Cohen-Or, D., Ruppin, E.: A humanlike predictor of
facial attractiveness. In: Schölkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural
Information Processing Systems 19, MIT Press (2007)