芸術科学会論文誌 Vol. 4 No. 4 pp. 135 – 144

3 次元顔モデルの美男美女化技法
高木 佐恵子 † 岡田 陽介 † 岩崎 慶 † 吉本 富士市 †和歌山大学システム工学部
魅力的な外見は一般に好まれる傾向にあるため，通常，人々は自分をよりよく見せようとする傾向がある．我々は，顔の個人的特徴をできるだけ残しつつ，その美しさを向上させるために，顔データを変形する手法を提案する．顔の魅力に関する既存研究を調査した結果，小さい，かつ／または，アンバランスな目と，突出した口を変形対象とする．変形は，3 次元の顔
データに対して適用される．用いる顔データには，顔の部分（目や唇など）に分類された特徴点が含まれている．変形は，特徴点の移動により実現し，その移動は特徴点を囲む計測サンプ
ル点の座標にも反映される．提案手法を適用して変形した 8 人分の顔データを用いて，アンケート調査を行った．その結果，提案手法が個人的特徴を残しつつ美男美女化することに有効であることが確認された．
キーワード: 顔の美しさ，美男美女化，3 次元変形

Modification Technique of 3D Facial Models for Improving Beauty
Saeko Takagi† Yosuke Okada † Kei Iwasaki† Fujiichi Yoshimoto †Faculty of Systems Engineering, Wakayama University
People usually try to show themselves attractively, since attractive looks are generally preferred. We
propose a new method for modifying facial data in order to improve facial beauty while retaining theoriginal features as much as possible. Based on investigations into related work of facial attractiveness, the targets for modification in our method are small and/or unbalanced eyes, and a projecting
mouth. We apply these modifications to 3D facial data sets. The data sets have the characteristic
points that are classified into facial parts. The coordinates of the characteristic points are moved, themodification affects the raw data points surrounding the characteristic points, and a modified faceis represented. We evaluated the modified faces of eight people with a questionnaire survey. As aresult, we confirmed our method was useful for increasing the beauty of facial data while retaining
the original features as much as possible.
Keywords: facial beauty, facial beautification, 3D modification

1 Introduction
People have been doing facial research for thousands of years. For more than 2,000 years, people
have tried to establish the principles that underlie all
judgments of beauty [1, 2, 15, 22]. Although we do
not yet have any established principle, anthropological research makes it clear that all over the world,
people seem to agree on what faces are considered
beautiful [1, 4, 10, 21, 22]. One facial research
topic in computer graphics is the modification of facial data for a specific purpose. There are systems
proposed that can make materials for psychologi†

saeko, s051012, iwasaki, fuji@sys.wakayama-u.ac.jp.

cal experiments [7, 8, 11, 14]. Although new facial images/models with different features are generated from facial data inputted to the systems, thesystems do not deal with the modifications of facial
images/models while retaining the original features.
Other outcomes are cosmetic surgery simulations,which are widely used at cosmetic surgery clinics.
In such a simulation, there is no restriction for retaining personal natural characteristics as much as
possible.
As a typical work in generating portraits,
Koshimizu et al. have proposed a technique formodifying facial data [12]. In their method, thecharacteristics of each person are defined as the dif-

– 135 –

芸術科学会論文誌 Vol. 4 No. 4 pp. 135 – 144

(a)

(b)

(c)

(d)

(e)

(f)

Figure 1: Original and modified facial models: (a) the original face in full face; (b) a face with modified eyesin full face; (c) the original face in profile; (d) a face with a modified mouth in profile; (e) the original face in
half-right profile; and (f) a face with modified eyes and a mouth in half-right profile.
ferences between an average face and each person’sface. The modifications are performed by emphasizing the differences. They applied this techniqueto both of 2D and 3D facial data. Although themethod can generate attractive representations offacial data, the result does not always satisfy themodel himself/herself. It is because he/she does notalways consider the emphasized differences as positive. Gooch et al. proposed a modification methodthat was not dependent on any average data [6].
In their work, although the result illustrations were
useful for recognizing a person, facial beauty was
not considered.
The face plays an important role in human communication. Attractive looks are generally preferred. Therefore, we tend to try to show ourselves
at our best. If there was a method of modificationthat could improve our appearance and still retain
our features, it could be used for portraits, avatars
in virtual environments, etc.
We propose a new modification technique for improving facial beauty that is able to retain original features as much as possible. We first reported
the results of the proposed technique in a previous paper [16]. According to investigations into
facial attractiveness, the targets of modification in
our method are small and/or unbalanced eyes, anda projecting mouth. The modification process isbased on the 3D facial data sets that are defined in

the system of 3D facial caricature [20]. In the data
sets, the characteristic points are classified into facial parts (e.g. eye, mouth, nose, etc.). The conceptof our method is to make beautiful faces by making only small modifications. This is because small
modifications retain the original features. Characteristic points in relation to the eyes and mouth are
moved at a certain rate, the modification affects raw
data points surrounding the characteristic points,
and then a modified face is represented, as shownin Figure 1. This kind of modification was applied
to the facial data of fifteen people. As a result, wewere able to confirm the validity of the proposed
method.

2What are the Factors of Facial Beauty?

It is generally thought that evaluating beauty isdifficult, because each mind perceives beauty differently. Cross-cultural studies, however, have confirmed that all over the world, people seem to agree
on which faces are considered beautiful, whether
they are judging men or women, and people fromtheir own or different racial backgrounds. Somestudies have confirmed that women are generally regarded as beautiful if they have smooth skin, largeeyes, and plump lips [1, 4, 21, 22]. These features
are also valued in judging babies’ beauty. Concerning men, most people tend to be attracted to malefaces with large eyes [1, 4, 10, 21, 22].
The line tangent from the tip of the nose to the– 136 –

芸術科学会論文誌 Vol. 4 No. 4 pp. 135 – 144(a)Figure 2: Ricketts’ aesthetic line (E-line).

3D Digitizer
facial shape data
facial color data

Figure 4: Definitions in the 3D facial data set [20]:(a) arrangement of the characteristic points; and (b)example of the shape vectors.

Generating Facial Modelspecify characteristic points
regularize data
generate facial parts
generate textures

Digital Camera(b)obtain color data
make a hair model

facial image

Beautification

beautified
facial model
Figure 3: Outline of preliminary processing for proposed beautification methods.

end of the chin was termed aesthetic line/plane (Eline/E-plane) by Robert Ricketts [17]. The E-line
is illustrated in Figure 2. In 1954, he developed abeauty standard based on the E-line principle thatthe tips of lips did not protrude from the E-line. The
details of this standard are different for each race.
For the Japanese, it is said that the tip of the lower
lip should be around the E-line [15]. In addition
to the E-line, there are several linear profile measurements: the S-line by Steiner [18]; the H-line
by Holdaway [9]; the Profile line and Z angle by
Merrifield [13]; the B-line by Burstone [3]; and theS’-line by Sushner [19]. The reason we chose theE-line is that extracting the E-line is easier than theother indexes. Moreover, the E-line is well-known
in Japan and the ideal range of the distance between
the mouth and the E-line for the Japanese has been
researched.
From a biological view point, it has been confirmed that symmetrical looks are more sexually attractive [1, 22]. Because symmetry suggests that
there is little harmful mutation, animals have an
acknowledgment system that makes them feel sexually attracted to symmetry. However, a very attractive human face is not completely symmetrical[1, 22].
Age is also an important element of beauty. Although there are many standards of beauty, in thefirst step of our research we target large, balancedeyes and an unprojecting mouth. We currently
use the same standard of beauty for both men andwomen. This is because large eyes and symmetrical looks are attractive in both men and women. In
addition, there are no different standards for the position of lower lip between the sexes.

3

Target Modification Data

We use the 3D facial data set defined in the system by Takemoto et al. [20] as the target data ofmodification. Figure 3 shows the outline of the data
set generation. The data sets consist of raw datameasured with a non-contact 3D digitizer, and image data taken with a digital camera. Each data

– 137 –

芸術科学会論文誌 Vol. 4 No. 4 pp. 135 – 144

set is composed of the following: raw data measured with a non-contact 3D digitizer; characteristic
points (196 points as shown in Figure 4(a)); classifying information of the characteristic points; vectors from the center of each facial part to all thecharacteristic points belonging to the part (shapevectors as shown in Figure 4(b)); vectors from thecenter of the face to the center of each facial part
(position vectors); the color values of the skin andthe lips; the texture of eyebrows and eyes; and the
texture of the hair. The characteristic points aredefined in order to generate a 3D caricature. The
position of each characteristic point is hand specified using a pointing device. As mentioned below,
parts of these characteristic points are used in beautification. Since these points are basically easy to
specify, no error reductions are currently applied tothe specified points. During the beautification process, the coordinates of the characteristic points are
modified. After the modification of all characteristic points, the raw data points are modified in orderto make the modification valid not only for the characteristic points but also the related raw data points.
Finally a modified face is represented.

4 Facial Beautification
4.1 Enlarging the eyes
When enlarging the eyes, we use the characteristic points and the shape vectors of the eyes. An
eye’s characteristic point Pk is moved to the point
Pk using the shape vector Vk by the following equation:(1)
Pk Pk  Vk  rate  rate xy¼¼

where rate is the enlarging rate that is inputted by auser and rate xy is the weight depending on the direction of the width or height. The reason we use rate xy
is that the total balance of the face becomes strange
if the eyes are enlarged and the aspect ratio is stillretained. We experimentally set the rate xy as the ratio of width (x) to height (y) eleven thirteenth. The
coordinates of the depth axis do not change. Thisis because changing the depth information is unimportant in relation to the size of eyes in full face andthere is no standard for sharply-chiseled features.
Figure 5 shows an example of enlarged eyes. In Fig-

(a)

(b)

(c)

(d)

Figure 5: Facial model with enlarged eyes: (a) and(b) the original eyes; and (c) and (d) enlarged eyes.
ure 5(b), the width of eyes is 1.1 times wider thanoriginal width, and the height is 1.3 times higher
than the original height. The weight rate xy seems
appropriate.
4.2 Balancing the eyes
By reducing differences in the shape and positionof each eye, the eyes become balanced. In order toretain the original features as much as possible, botheyes are modified to an average shape and position
instead of reversing and exchanging one eye for theoriginal one. After this, the coordinates of the depth
axis are not changed again. Therefore, the eyes do
not become exactly identical.
A new shape vector Vk is calculated using theright eye’s shape vector Vkr and the left eye’s shapevector Vkl with the following equation:
¼

¼

Vk

Vkr  Vkl
2(2)

where the coordinates of the depth axis are notchanged again. The unchanged depth value produces eyes that are not totally even. This means– 138 –

芸術科学会論文誌 Vol. 4 No. 4 pp. 135 – 144

(a)

(b)
Figure 7: E-plane for reducing mouth projection.

(c)

(d)

Figure 6: Facial model with balanced eyes: (a) theoriginal eyes; (b) balanced eyes; (c) enlarged eyes;and (d) enlarged and balanced eyes.
that the original characteristics are retained somewhat in spite of balancing. The characteristic points
corresponding to the modified shape vectors are updated. Figure 6 shows some examples of balanced
eyes. The effect is obvious in the model with enlarged and balanced eyes.
4.3 Reducing mouth projection
To modify the characteristic points in and aroundmouth, first, the E-plane, which is defined by the
point of the nose tip and the line which goes throughthe nose tip and one of chin points, is calculated.
The E-plane is parallel to the horizontal axis, shownas Z f y in Figure 7.
To reduce mouth projection, we can use several
methods. The simplest way is to pull the mouthback without making any change to the lip shape.
In this case, the facial features in full face are notdrastically changed. Sliding the mouth vertically or
scaling down the mouth also makes the mouth projectless or not at all. This method causes facial im-

(a)

(b)

(c)

(d)

Figure 8: Facial models with an unprojecting
mouth: (a) and (c) the original mouth; and (b) and(d) an unprojecting mouth.
pression changes in the whole face and we could not
find any standards among them. As we are trying toretain the original features as much as possible, we
simply pull the mouth back.
When the depth coordinates z M of the lower lip
are farther than the E-plane, the characteristic pointof the tip of the lower lip is pulled back to the Eplane using the following equation:
¼

zM

zN
yN

zC
yM
yC

yN  zC

(3)

where PM xM yM zM  is the coordinate of the
tip of the lower lip, PM xM yM zM  is the coordinate of the moved tip of the lower lip, PN
xN yN zN  is the coordinate of the nose tip, andPC xC yC zC  is the coordinate of the chin tip.
All characteristic points of the lip are also moved
in the same direction and at the same distance. In
order to avoid distortion, the surrounding characteristic points are also moved with weight according– 139 –

¼

¼

¼

¼

芸術科学会論文誌 Vol. 4 No. 4 pp. 135 – 144

(a)

(b)

Figure 9: Examples of facial models using the original photograph: (a) the original face; and (b) a beautified face.

(a)

(b)

Figure 10: Examples of facial models using a illustration style image: (a) the original face; and (b) abeautified face.
to the distance from the mouth. Figure 8 shows examples of nonprojecting mouths. In the modified
models, the mouth and surrounding areas are drawn
in, but the original characteristics are still retained.

5 Experimental Results
We experimented with our method on the facial
data of fifteen people. In addition to Figures 1, 5,
6, and 8, the beautified facial models are shown in
Figures 9, 10, and 11.
The differences between the mapped images areshown in Figures 9 and 10. Figure 9 shows examples of an original and beautified facial model using
original color data taken with non-contact 3D digitizer. Figure 10 shows examples using the color
values of the skin and the lips, and texture imagesof the eyebrows and eyes, similar to the previous

figures. Although a uniform skin color makes theartificial wrinkles around the corners of the eyes
prominent, we used these illustration style images
for the questionnaire since it is easy to recognize
the differences between the original and beautified
models.
In our experiment, there were twenty two subjects; 11 men and 11 women. Their ages ranged
from fifteen to forty years old. The proposed
method is designed according to the Japanese criterion of facial beauty. Therefore, we carried out aquestionnaire survey of Japanese people, who rated
the beautified models. The subjects had never met
the facial models.
We used eight Japanese people in their twenties(four men and four women) as facial models for thequestionnaire. Each model has some of the following features: model No. 1 (m) has narrow eyes. His
mouth projects a little; model No. 2 (m) has slightly
narrow eyes. His mouth projects; model No. 3 (m)has slightly small eyes. His mouth projects a little;
model No. 4 (m) has small eyes. His mandibular isslightly augmentative; model No. 5 (w) has slightly
small and unbalanced eyes; model No. 6 (w)’smouth projects a little; model No. 7 (w) has largeeyes. Her mouth projects a little; and model No. 8(w) has slightly large eyes. Her mouth projects a little. For each model, we generated five patterns: (A)the original model; (B) a model with enlarged eyes;(C) a model with balanced eyes; (D) a model withan unprojecting mouth; and (E) a beautified model(that has enlarged and balanced eyes and an unprojecting mouth). In enlarging the eyes, the widthof eyes is 1.1 times as wide as the original width,
and the height is 1.3 times as high as the original
height in models Nos. 1-6, and 8. In model No.
7, the width of eyes is 1.05 times as wide and theheight is 1.15 times as high. We printed out fullcolor pictures of the models. The full face, the facein half-right profile, and the face in full profile wereshown in one line as shown in Figure 11. The sizeof printed faces was almost 4 cm in height.
In the first question, we showed four sheets on
which the images (A) and (E) of each model wereprinted. Two model’s faces were printed on one– 140 –

芸術科学会論文誌 Vol. 4 No. 4 pp. 135 – 144(a) the original facial model A.

(b) the original facial model B.

(c) a beautified facial model A.

(d) a beautified facial model B.

(e) the original facial model C.

(f) the original facial model D.

(g) a beautified facial model C.

(h) a beautified facial model D.

Figure 11: Examples of original and beautified facial models used for the questionnaire survey.

– 141 –

芸術科学会論文誌 Vol. 4 No. 4 pp. 135 – 144

7

90
80

6

70

point

model No.

100

women
men
all subjects

8

5
4

60
50

model 1model 2model 3model 4model 5model 6model 7model 8

40

3

30

2

20
10

1

0
original

0

20

40

60

80

100

enlarged eyesbalanced eyes unprojecting mouth

beautified

types of processing

chosen rates of beautified models (%)

Figure 12: Results of the first question. The chosen
rates of beautified models and not original models.
Model Nos. 1- 4 are men and the rest are women.

sheet and the type-order was random. From thetwo examples of each model, the subjects chose themore beautiful one. Figure 12 shows the results.
The rates of beautified models chosen were higher
than the rates of the original models chosen. The
rates of beautified models chosen by women werehigher than those chosen by men. There were twomodels whose original model was chosen at rates
similar to the beautified models. Those two models had large eyes originally. This indicates that theeyes of those two models were already suitable for
their faces.
In the second question, we showed eight sheets
where all images (A)-(E) of each model wereprinted on each sheet. The order of images was random. The subjects chose the three highest ranking
images. We assigned five points to the first ranking, three points to the second, and one point to thethird. Figure 13 shows the total points. Models withenlarged eyes and the beautified models were chosen first at the almost same rates. In the rest, models
with an unprojecting mouth were chosen more thanothers. There was a model that got high points in
the image showing an unprojecting mouth that was
the same as the model with enlarged eyes. Originally that model had a projecting mouth. The original mouths of the other models projected a little
and the effect of making the mouth to not project
was not prominent. The result indicates that the unprojecting modification worked well on the model
when the original model had a conspicuous project-

Figure 13: Results of the second question. Total
ranking points. The correspondence of a model No.
with a person is same as the first question (Figure
12).
original 7%
beautified
38%

unprojecting mouth
12%

enlarged eyes
35%balanced eyes
8%

Figure 14: Overall ratio of the answers to the second question. Ratio of types in the chosen models.
ing mouth.
Figure 14 shows the overall ratio of answers to
the second question. The beautification process isfirst, the enlarging process is second, and the unprojecting process is third. The Japanese have been
focusing on the beauty of the full face for longtime [15]. In full face, the size of eyes stands out
from a unprojecting mouth. Our purpose is to modify facial data in order to improve facial beauty
while maintaining the original features as much as
possible. From this view point, the rate of enlarging
the eyes was somewhat unsuitable. Since eyes areimportant for recognizing a person’s face [5], thebest rate for unprominent beauty should be investigated. Concerning the balanced eyes and an unprojecting mouth, we have to collect facial data andconduct our investigation again since we had only
one model who had unbalanced eyes and one model
who had an obviously projecting mouth. In addi-– 142 –

芸術科学会論文誌 Vol. 4 No. 4 pp. 135 – 144

female physical attractiveness. Journal of Personality & Social Psychology, 68(2):61–279,
1995.

tion, we did not modify the mandibular augmentation, because the difference between the upper andlower jaws had not been considered in our method
yet. Modifications to reduce mouth projection, including the factor of mandibular augmentation, may
be similar to cosmetic surgery simulations. For better results, this should be considered.

[5] I. Fraser, G. Craig, and D. Parker. Reaction
time measures of feature saliency in schematic
faces. Perception, 19(5):661–673, 1990.
[6] B. Gooch, E. Reinhard, and A. Gooch. Human
facial illustrations: Creation and psychophysical evaluation. ACM Trans. Gr., 23(1):27–44,
January 2004.

6 Conclusions
We proposed a method to beautify facial 3D models. The targets of modification in our method
were small and/or unbalanced eyes, and a projecting mouth. We investigated the effectiveness of our
method and confirmed that the proposed method
was useful for faces that did not originally possess
specifically beautiful factors.
We decided that the modification had to retain theoriginal feature as much as possible in order to assure the recognizability of a person’s face. Our approach can achieve this. There is a trade-off between recognizability and beautification. As a future work, we are going to find the best range ofeach parameter value that can maintain the balance
between recognizability and beautification. We
would like to integrate the other elements of facial
beauty into the current technique, for example, thesmoothness and color of the skin, especially for 3D
facial models that use the original photograph formapping. These generated models will be usefulfor non-anonymous avatars in virtual environments
and for materials for psychological experiments.

References[1] B. Bates and J. Cleese. The Human Face. DK
Publishing Inc., 2001.
[2] V. Bruce. Recognizing Faces. Lawrence Erlbaum associates Ltd., 1988.

[7] H. Harashima. An Invitation to Study of theFace. Iwanami Shoten, 1998 (in Japanese).
[8] H. Harashima. The concealing face, the nameless face. NatureInterface, 4:16–21, 2001
(its online document translated into English:.
www.natureinterface.com/e/ni04/P016-021/).
[9] R. Holdaway. A soft-tissue cephalometricanalysis and its use in orthodontic treatment
planning. Part I. American journal of orthodontics, 84(1):1–28, July 1983.
[10] H. Ishi and J. Gyoba. Analyses of psychological aspects of attractiveness in feminized andjuvenilized japanese faces. Tohoku Psychologica Folia, 60:29–34, 2001.
[11] K. Isono, M. Oda, and S. Akamatsu. A facial
image generating system based on a genetic
algorithm – generating subjective impression–. IEICE Trans. Inf. & Syst., Pt. 2, J82-DII(3):483–493, 1999 (in Japanese).
[12] Koshimizu Lab. PICASSO. www.koshi-lab.
sccs.chukyo-u.ac.jp/research/picasso/.

[3] C. Burstone. Lip posture and its significancein treatment planning. American journal of orthodontics, 53(4):262–284, April 1967.

[13] L. Merrifield. The profile line as an aid in
critically evaluating facial esthetics. American
journal of orthodontics, 52:804–822, November 1966.

[4] M. Cunningham, A. Roberts, A. Barbee, D. P.,
and C. Wu. Their ideas of beauty are, on
the whole, the same as ours: Consistency andvariability in the cross cultural perception of[14] S. Mukaida, M. Kamachi, M. Oda, T. Kato,
S. Yoshikawa, S. Akamatsu, and K. Chihara.
Facial image symthesis system : Futon – evaluation as tools for cognitive research on face

– 143 –

芸術科学会論文誌 Vol. 4 No. 4 pp. 135 – 144

processing –. IEICE Trans. Fundamentals,
J85-A(10):1126–1137, 2002 (in Japanese).
[15] H. Murasawa. Fashions in Face and Beauty inJapan. Tokyo-shoseki, 1987 (in Japanese).
[16] Y. Okada, S. Takagi, K. Iwasaki, and F. Yoshimoto. Modification technique for improving
facial beauty. In Proceedings of NICOGRAPH
International 2005, pages 23–128, 2005.
[17] R. Ricketts. A foundation for cephalometric communication. American journal of orthodontics, 46(5):330–357, May 1960.
[18] C. Steiner. The use of cephalometrics as
an aid to planning and assessing orthodontic
treatment. American journal of orthodontics,
46(10):721–735, October 1960.
[19] N. Sushner. A photographic study of the softtissue profile of the negro population. American journal of orthodontics, 72(4):373–385,
October 1977.
[20] S. Takemoto, S. Takagi, and F. Yoshimoto. A
generation system of 3D facial caricature withcolor emphasis. In Proceedings of 20th NICOGRAPH, pages 59–64, 2004 (in Japanese).
[21] The Shiseido Institute of Beauty Sciences, editor. The Psychology of Cosmetic Behavior.
Fragrance Journal Ltd., 1993 (in Japanese).
[22] S. Yoshikawa, M. Masutani, and M. Nakamura. Face and heart: An introduction to face
psychology. Saiensu-sha, 1993 (in Japanese).

– 144 –