The top image of the stereo pair is the original view, while the bottom image of the stereo pair is the desired, gaze-corrected view. These were collected in a separate training phase. The main justification is that the mapping between face parameters in the original view to face parameters in the gaze-corrected view is expected to be relatively invariant across population. Face Synthesis . Given the gaze-corrected face parameters, a synthetic image can be generated and superimposed on the original image frame.
Top images show original views with registered face models.
Bottom images show the corresponding gaze-corrected view. A Framework for Representing and Matching Object Classes, International Journal of Computer Vision , Kluwer, August 1998. Tom M. Mitchell, Machine Learning , McGraw-Hill, 1997.