  In this paper, we present a method of clothes retargeting; generating the
potential poses and deformations of a given 3D clothing template model to fit
onto a person in a single RGB image. The problem is fundamentally ill-posed as
attaining the ground truth data is impossible, i.e., images of people wearing
the different 3D clothing template model at exact same pose. We address this
challenge by utilizing large-scale synthetic data generated from physical
simulation, allowing us to map 2D dense body pose to 3D clothing deformation.
With the simulated data, we propose a semi-supervised learning framework that
validates the physical plausibility of the 3D deformation by matching with the
prescribed body-to-cloth contact points and clothing silhouette to fit onto the
unlabeled real images. A new neural clothes retargeting network (CRNet) is
designed to integrate the semi-supervised retargeting task in an end-to-end
fashion. In our evaluation, we show that our method can predict the realistic
3D pose and deformation field needed for retargeting clothes models in
real-world examples.
