Subtract the long term average from the eye gaze directions.  Best solution I've found so far for aligning the gaze.