Do object part localization methods produce bilaterally symmetric results on mirror images? Surprisingly not, even though state of the art methods augment the training set with mirrored images. In this paper we take a closer look into this issue. We first introduce the concept of mirrorabil ity as the ability of a model to produce symmetric results in mirrored images and introduce a corresponding measure, namely the mirror error that is defined as the dif f erence be tween the detection result on an image and the mirror of the detection result on its mirror image. We evaluate the mir rorability of several state of the art algorithms in two of the most intensively studied problems, namely human pose es timation and face alignment. Our experiments lead to sev eral interesting findings: l) Most of state of the art methods struggle to preserve the mirror symmetry, despite the fact that they do have very similar overall peiformance on the original and mirror images; 2) the low mirrorability is not caused by training or testing sample bias -all algorithms are trained on both the original images and their mirrored versions; 3) the mirror error is strongly correlated to the localization/alignment error (with correlation coefficients around 0.7). Since the mirror error is calculated without knowledge of the ground truth, we show two interesting ap plications -in the first it is used to guide the selection of difficult samples and in the second to give feedback in a popular Cascaded Pose Regression method for face align ment.
The "mirrorability" constraint is used in REF to force the estimated 2D landmarks update be consistent between the image and its mirror image.
Mirror, mirror on the wall, tell me, is the error small?
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Computer Science