  Makeup transfer is the task of applying on a source face the makeup style
from a reference image. Real-life makeups are diverse and wild, which cover not
only color-changing but also patterns, such as stickers, blushes, and
jewelries. However, existing works overlooked the latter components and
confined makeup transfer to color manipulation, focusing only on light makeup
styles. In this work, we propose a holistic makeup transfer framework that can
handle all the mentioned makeup components. It consists of an improved color
transfer branch and a novel pattern transfer branch to learn all makeup
properties, including color, shape, texture, and location. To train and
evaluate such a system, we also introduce new makeup datasets for real and
synthetic extreme makeup. Experimental results show that our framework achieves
the state of the art performance on both light and extreme makeup styles. Code
is available at https://github.com/VinAIResearch/CPM.
