# Basic Research on How to Apply Foundation Makeup Evenly on Your Own Face

Miho Kajita, Satoshi Nakamura

<figure><img src="image_1.png" /></figure>

## To cite this version:

Miho Kajita, Satoshi Nakamura. Basic Research on How to Apply Foundation Makeup Evenly on Your Own Face. 20th International Conference on Entertainment Computing (ICEC), Nov 2021, Coimbra, Portugal. pp.402-410, 10.1007/978-3-030-89394-1_32. hal-04144355

HAL Id: hal-04144355

https://inria.hal.science/hal-04144355

Submitted on 28 Jun 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Basic Research on How to Apply Foundation Makeup
Evenly on Your Own Face

Miho Kajita¹ and Satoshi Nakamura¹

¹ Meiji University, 4-21-1 Nakano, Nakano-ku, Tokyo, Japan
kajitafms5@gmail.com

**Abstract.** Makeup is an entertainment that allows people to present themselves in a way that is close to their ideal. People apply foundation makeup evenly to cover skin imperfections, but it is not easy to distinguish between areas where foundation is applied or not. Therefore, problems like applying too much foundation or forgetting to apply foundation on some areas are likely to occur. Our project aims to realize a makeup support system that can visualize the state of foundation application in real-time to eliminate uneven application. As an initial step, we conducted a large-scale crowdsourcing survey on makeup, precisely foundation, and investigated a method that uses machine learning to classify between images of skin with foundation applied and of bare skin. As a result, we found that we could distinguish between both with high accuracy of 82.3%.

**Keywords:** Foundation, Makeup, Machine Learning.

## 1 Introduction

The face is a part of the body that quickly displays personal characteristics such as age and gender as well as emotions [1]. Therefore, people apply makeup to bring their face's impression gives closer to their ideal [2]. According to an Internet survey conducted by POLA in 2019 [3], about 80% of women aged 15-64 wear makeup. In addition, the demand for cosmetics has been increasing not only among women but also among men. Then, the number of brands selling cosmetics for men has been growing. On the other hand, make-up videos of people pretending to be celebrities or characters have become popular on SNS, and make-up is enjoyed as a form of entertainment.

One of the difficulties of wearing makeup is to apply foundation makeup evenly. Since it is recommended to choose a foundation close to one's skin tone, foundation easily blends in with one's bare skin, so it isn't easy to distinguish between areas where foundation is applied and areas where it is not. As a result, people tend to apply too much foundation or to forget to apply only a particular area. If foundation is not appropriately applied, people cannot hide blemishes, pores, and other skin problems. In addition, many foundations contain sunscreen ingredients, and uneven application may cause some parts of the skin to become sunburned. However, the most important reason
why it is necessary to apply the appropriate amount of foundation is because makeup tends to fall off from areas where foundation has been applied repeatedly.

To solve those problems, we came up with a system that indicates where and how much foundation has been applied and correct uneven application easily and in real-time. Nishino et al. [4], who focused on the fact that foundation skin absorbs light more efficiently than bare skin [5], developed a measurement system for quantifying and distributing the amount of foundation by using an optical filter that emphasizes the difference in the wavelength characteristics of reflection between bare skin and foundation skin. However, this system requires an optical filter that spectral transmission characteristics have been optimized through experiments, making it difficult for general users.

Our goal is to realize a system that enables users to check for unevenness in applied foundation in real-time easily, not only at home but also when they are away from home. An image of the system is shown in Figure 1. In this study, as an initial step, we investigated a method for classifying images of the skin with foundation applied (hereafter referred to as “foundation skin”) and bare skin captured by a smartphone camera using machine learning with color features as feature values. In addition, we conducted a questionnaire survey on makeup and foundation to clarify the problems in makeup and foundation.

<figure><img src="image_4.png" /><figcaption>Fig. 1. Our system</figcaption></figure>

## 2 Survey on makeup and foundation

In this research, we first conducted a questionnaire survey of 1,000 women who have worn makeup through Yahoo! Crowdsourcing [6] (conducted from September 4 to 5, 2020). Then, we excluded 16 responses from 1,000 because they were unreliable, such as ignoring the instructions. The following sections describe the results of the analysis of the 984 valid responses.

Figure 2 shows the results of makeup frequency before/amid the COVID-19. From these results, about 90% of respondents wore makeup more than 2 or 3 times a week before the COVID-19. Similarly, 75% of respondents wore makeup at least two or three

<figure><img src="image_5.png" /><figcaption>Fig. 2. Makeup frequency before/amid the COVID-19 pandemic</figcaption></figure>
times a week amid the COVID-19 pandemic, when the number of makeup wearers was thought to have decreased due to the effects of the voluntary curfew.

Figure 3 shows the responses about the difficulty with makeup. From this result, 24.5% of the respondents answered that they felt that it is “quite difficult” to apply makeup, and 48.9% responded that they felt “a little difficulty.”

<figure><img src="image_6.png" /><figcaption>Fig. 3. Difficulty with makeup</figcaption></figure>

In addition, more than half of the respondents mentioned that the process of applying foundation was the most time-consuming part of base makeup (see Figure 4). Followed by 33.5% of respondents said they spend more time on primer, 9.8% on concealer, and almost none on highlighting or shading.

<figure><img src="image_7.png" /><figcaption>Fig. 4. Processes that take the most time for base makeup</figcaption></figure>

Figure 5 also shows that people tend to purchase foundations with high “coverage of blemishes, pores, and other skin difficulties.” In addition, when buying foundations, many people (50.6% and 40.4%, respectively) place importance on “durability” and “ease” of use. These results suggest that many people want to hide their concerns quickly when using base makeup.

<figure><img src="image_8.png" /><figcaption>Fig. 5. What to look for when buying foundation</figcaption></figure>

Figure 6 shows the result of how to learn base makeup. From this result, about 80% of the responders learn how to wear base makeup by themselves. 37.2% of those who answered that they learn base makeup themselves have experiences difficulties in applying foundation, such as “uneven color and uneven application” and “applying too much foundation and making it thick.” This result suggests that these difficulties need to be resolved to make base makeup easier to use.

<figure><img src="image_9.png" /><figcaption>Fig. 6. How they learned base makeup</figcaption></figure>
So, a system for applying foundation evenly, which is the objective of this research, will be helpful for users who have the above problems because it enables them to hide blemishes and pores quickly, easily, and beautifully.

## 3 Constructing the data set

This study aims to develop a makeup support system that can visualize the application of foundation in real-time to make it easier to check where and how much foundation has been applied and to correct uneven application. This system is intended to be easy for users to use with their smartphones and other devices. We first examined whether it is possible to estimate foundation skin and bare skin mechanically. This section described the construction of datasets used for discriminating the state of foundation applied to the skin in detail.

There are various types of foundations, and users use the appropriate one depending on the texture and finish of the skin they want to achieve. Since the type of foundation drastically affects the appearance of the foundation skin, in this construction, we prepared 12 types of foundation (Table 1), with two types of finish, glowing or matte, for three types of foundation: powder, liquid, and cream. Then, we photographed skin applying foundations. Glowing skin is characterized by a moisturized and shiny appearance of bare skin, while matte skin is smooth and covers imperfect skin. In this construction, it wasn't easy to prepare the appropriate foundation for each participant's skin tone, so the color was selected on the basis of the author's skin tone. According to an oral questionnaire used in the construction, no participant felt uncomfortable with the difference in color between the foundation and their bare skin when the 12 types of foundation were applied.

Here, we collected bare skin and foundation skin images to construct the dataset. Sixteen undergraduate and graduate students (20-26 years old) (5 males and 11 females) participated. We asked them to apply four different types of foundation per day for a total of three days. We used Android Xperia XZ3 (SONY) for the photography. The camera has approximately 19.2 megapixels, an f/1.9 aperture, and an ISO sensitivity of up to 12,800.

<table><caption>**Table 1. List of foundations used**</caption><thead><tr><th>Type</th><th>Skin type</th><th>ID</th><th>Product name</th></tr></thead><tbody><tr><td rowspan="6">liquid</td><td rowspan="2">Glowing</td><td>L1</td><td>Shu Uemura [unlimited glow fluid 574]</td></tr><tr><td>L2</td><td>Amplitude [long lasting liquid foundation 20]</td></tr><tr><td rowspan="2">Matte</td><td>L3</td><td>Estee Lauder [Double Wear Stay-in-Place Makeup 12]</td></tr><tr><td>L4</td><td>Lancome [Teint Idole Ultra Wear Liquid BO-01]</td></tr><tr><td rowspan="4">powder</td><td rowspan="2">Glowing</td><td>P1</td><td>ONLY MINERALS [Foundation 5 light ocher]</td></tr><tr><td>P2</td><td>Chanel [Le Blanc Whitening Compact Foundation B10]</td></tr><tr><td rowspan="2">Matte</td><td>P3</td><td>Maquillage [Dramatic Powdery UV ocher10]</td></tr><tr><td>P4</td><td>Dior [Dior Skin Forever Compact Extreme Control 020]</td></tr><tr><td rowspan="4">cream</td><td rowspan="2">Glowing</td><td>C1</td><td>RMK [Creamy Foundation EX #101]</td></tr><tr><td>C2</td><td>SUQQU [The Crème Foundation 110]</td></tr><tr><td rowspan="2">Matte</td><td>C3</td><td>Alblanc [Moist White Creme Foundation Ocher 03]</td></tr><tr><td>C4</td><td>Laura Mercier [Silk Creme Oil Free Photo Edition Foundation 03]</td></tr></tbody></table>
In the dataset construction, we asked the participants to wash their faces to remove sebum and makeup before taking photos. Then, we photographed their bare skin after wiping off all moisture. We took five patterns of photographs of the forehead and cheeks: frontal, right oblique, left oblique, with the chin in front of the camera, and with the chin pulled back (see Figure 7). The distance between the camera and the participant was about 15cm, and the camera was illuminated with an LED light. The camera was focused on the center of the area used for photographing.

<figure><img src="image_10.png" /><figcaption>**Fig. 7.** Five patterns: top row left to right, frontal, right oblique, left oblique, bottom row from left to right, with the chin in front of the camera and with the chin pulled back</figcaption></figure>

Next, we photographed the foundation skin. We asked participants to apply a primer to their forehead (from the hairline to the top of the eyebrows) and their cheeks (from the side of the nose to the start of the ear). Then, we asked them to apply foundation using a sponge puff. Afterward, we took pictures from the same angle as when photographing bare skin. Again, we instructed the participants to apply foundation carefully to avoid unevenness.

Then, we collected 10 images of the bare skin without foundation (2 areas × 5 directions) and 120 images of the face with foundation applied on the forehead and cheeks (12 types of foundation × 2 areas × 5 directions) for each participant. Thus, a total of 2,080 images were collected. From these images, one image was cut into 500px squares to show only the skin. The image was further divided into 25 100px squares to create datasets consisting of 4,000 images of bare skin (10 images × 25 segments × 16 persons) and 4,000 images of foundation skin (10 images × 25 segments × 16 persons) in each foundation.

Thus, we created datasets consisting of 4,000 bare skin images and 48,000 foundation skin images.

# 4 Discrimination of Bare Skin and Foundation Skin by Machine Learning

From the images collected in Section 3, we generated features to be used for training.

Skin color is determined by the melanin and hemoglobin content in the blood in the capillaries [7]. Depending on the balance of these two factors, the skin becomes reddish
or yellowish. Since foundation suppresses such redness and yellowing of the skin [8], it is expected that the color of skin to which foundation is applied will be less reddish and yellowish than that of bare skin. Therefore, we thought it would be possible to discriminate between bare and foundation skin by setting up features representing the color tendency of the images.

We classified foundation colors along two axes: “hue,” which indicates reddish or yellowish, and “lightness,” which shows the range of lightness or darkness [9]. In addition, we divided foundations into two types of finish textures: “glowing,” which is saturated, clear, and has good coloration, and “matte,” which is less saturated and has a slightly dull coloration (see Figure 8). For these reasons, we believe that it was appropriate to use the HSV color space instead of the RGB one. Therefore, after converting the image from RGB to HSV, we obtained and calculated the mean and standard deviation of hue, saturation, and lightness, which generated a 2 × 3 = 6-dimensional feature value.

<figure><img src="image_11.png" /><figcaption>**Fig. 8.** Examples of skin applying foundations whose types are glowing (left) and matte (right)</figcaption></figure>

We considered three ways of dividing the data for training (see Table 2). First, the dataset was trained as is (partitioning method 1). Second, we divided the dataset into two types according to whether the image was of the cheek or the forehead. Third, we trained a dataset consisting of 2,000 bare skin images and 2,000 foundation skin images (partitioning method 2). Finally, we prepared a dataset composed of 2,000 images of bare skin and 2,000 images of foundation skin for each of the 16 participants (partitioning method 3). In the following subsection, we describe the results of training and classification for each of the three training data sets.

We used the random forest as the learning algorithm for the binary classification of foundation skin and bare skin by scikit-learn. We used 75% of the datasets as training data and 25% as test data, with positive values for foundation skin and negative values for bare skin.

<table><caption>**Table 2.** Datasets</caption><thead><tr><td>Partitioning method</td><td>Division number</td><td>Datasets size</td></tr></thead><tbody><tr><td>Method 1</td><td>1</td><td>(4,000 bare skin images and 4,000 foundation skin images) ×12 types of foundation</td></tr><tr><td>Method 2</td><td>2</td><td>(2,000 bare skin images and 2,000 foundation skin images) ×12 types of foundation</td></tr><tr><td>Method 3</td><td>16</td><td>(250 bare skin images and 250 foundation skin images) ×12 types of foundation</td></tr></tbody></table>
Table 3 shows the results of partitioning methods 1 and 2, showing that they could discriminate between foundation skin and bare skin with an average accuracy of 82.3%. As a result of learning for each region, the average correct-answer rate was 89.6% for the forehead and 81.0% for the cheeks.

As described above, the features were the mean and standard deviation of hue, saturation, and lightness. To investigate the color tendency of the images that could be discriminated, we trained and classified the images using only the mean and standard deviation of the hue, saturation, and lightness feature values for partitioning methods 1 and 2. The results showed almost no difference in the average percentage of correct answers due to changes in the feature values. However, when we compared each type of foundation, the powder foundation had the best accuracy with the mean and standard deviation of hue and saturation were used as features. The cream foundation had the best accuracy when the mean and standard deviation of lightness as features.

To examine individual differences, we divided the dataset into 16 equal parts for each participant (partitioning method 3) and trained. As a result, the average percentage of correct answers for each individual exceeded the average rate of 82.3% for all 16 participants for partitioning method 1 (see Table 3), with a mean of 92.4%.

<table><caption>Table 3. Results</caption><thead><tr><th rowspan="3"></th><th rowspan="3"></th><th colspan="2">partitioning method 1</th><th colspan="4">partitioning method 2</th></tr><tr><th rowspan="2">accuracy rate</th><th rowspan="2">F-Score</th><th colspan="2">accuracy rate</th><th colspan="2">F-Score</th></tr><tr><th>forehead</th><th>cheek</th><th>forehead</th><th>cheek</th></tr></thead><tbody><tr><td rowspan="2">Glowing</td><td>L1</td><td>81.9%</td><td>0.822</td><td>85.9%</td><td>83.5%</td><td>0.861</td><td>0.837</td></tr><tr><td>L2</td><td>77.6%</td><td>0.777</td><td>85.5%</td><td>76.4%</td><td>0.858</td><td>0.772</td></tr><tr><td rowspan="2">Matte</td><td>L3</td><td>78.4%</td><td>0.785</td><td>87.5%</td><td>76.3%</td><td>0.875</td><td>0.766</td></tr><tr><td>L4</td><td>81.3%</td><td>0.814</td><td>87.7%</td><td>78.5%</td><td>0.881</td><td>0.787</td></tr><tr><td rowspan="2">Glowing</td><td>P1</td><td>85.1%</td><td>0.853</td><td>93.6%</td><td>85.0%</td><td>0.938</td><td>0.852</td></tr><tr><td>P2</td><td>83.4%</td><td>0.837</td><td>91.3%</td><td>83.5%</td><td>0.915</td><td>0.829</td></tr><tr><td rowspan="2">Matte</td><td>P3</td><td>82.0%</td><td>0.820</td><td>89.7%</td><td>79.5%</td><td>0.896</td><td>0.799</td></tr><tr><td>P4</td><td>84.0%</td><td>0.838</td><td>91.7%</td><td>83.0%</td><td>0.918</td><td>0.829</td></tr><tr><td rowspan="2">Glowing</td><td>C1</td><td>82.4%</td><td>0.822</td><td>89.1%</td><td>80.2%</td><td>0.891</td><td>0.801</td></tr><tr><td>C2</td><td>83.6%</td><td>0.836</td><td>91.3%</td><td>82.0%</td><td>0.912</td><td>0.824</td></tr><tr><td rowspan="2">Matte</td><td>C3</td><td>83.3%</td><td>0.831</td><td>89.5%</td><td>82.2%</td><td>0.893</td><td>0.822</td></tr><tr><td>C4</td><td>84.0%</td><td>0.838</td><td>92.1%</td><td>81.7%</td><td>0.920</td><td>0.818</td></tr><tr><td colspan="2">Average</td><td>82.3%</td><td>0.823</td><td>89.6%</td><td>81.0%</td><td>0.897</td><td>0.811</td></tr></tbody></table>

# 5 Conclusion

In this paper, we first surveyed makeup. It showed that many people want to hide their concerns quickly when using base makeup. However, they have difficulty applying foundation evenly and have little means of solving the problem because most of them learn how to wear base makeup by themselves.

We also investigated whether it is possible to mechanically classify images of foundation-applied skin and images of bare skin. We asked 16 participants for foundation application, and created the data set. In addition, the mean and standard deviation of hue, saturation, and lightness were used as features to learn from the
features of bare skin and foundation. The results showed that the system could discriminate between images of both skin types with a high rate of correct answers. In addition, images of the forehead, which has more wrinkles and pores than the cheek, could be used to discriminate the state of foundation application with a higher accuracy rate.

There are two types of uneven foundation. One is when foundation is not applied to a part of the skin, i.e., where the skin is bare, or the applied foundation is mixed. The other is when foundation is not applied at a certain thickness, i.e., foundation is applied thickly or thinly. This paper showed that we could discriminate whether foundation is applied or not. In addition, this paper showed the possibility that we could also determine whether it is applied thickly or lightly by using the same feature. However, to classify the amount of foundation applied, we must set an appropriate threshold value, which we plan to study in the future.

In addition, we trained on 100 px square images. However, to detect unevenness, it is necessary to determine whether foundation is applied or not on smaller images. For this reason, we plan to create an image dataset with a smaller resolution for training and estimation.

This research aims to realize a makeup support system that can visualize the application of foundation in real-time. As a prospect, we plan to investigate whether the application state of foundation can be discriminated in the video in the same way.

## References

1. Leslie, Z.: Reading Faces: Window To The Soul?. Westview Press (1997).

2. Graham, J. A., Jouhar, A. J.: The effects of cosmetics on person perception. International Journal of Cosmetic Science, vol. 3, no. 5, pp. 199-210 (1981).

3. POLA research institute of beauty & culture, https://www.cosmetic-culture.poholdings.co.jp/report/pdf/191212kitai.pdf, last accessed 2021/08/10.

4. Nishino, K., Nakamura, M., Miyashita, K.: Development and Application of Quantitative/Spatial Measurement System of Cosmetics Foundation with Functional Spectral Filter. Journal of the Color Science Association of Japan, vol. 37, no. 3, pp. 202-203 (2013).

5. Igarashi, T.: Optical properties of foundation. Journal of Japan Society of Colour Material. vol. 85, no. 4, pp. 156-163 (2012).

6. Yahoo!, https://crowdsourcing.yahoo.co.jp/, last accessed 2020/08/10.

7. Anderson, R. R., Parrish, J. A.: The optics of human skin. Journal of Investigative Dermatology, vol. 77, no. 1, pp. 13-19 (1981).

8. Igarashi, T.: Research and development of cosmetics that control the texture of the skin. Optical Society of Japan, vol. 43, no. 7, pp. 318-324 (2014).

9. Siseido, https://www.shiseido.co.jp/foundation100/answer/q07.html, last accessed 2020/12/09.