In the face detector, we record an image of the actual face pattern wherever the detector records a hit. For optimal classification, we want the scale, alignment, and view of detected faces to be comparable. We resize the pattern to normalize for size, and discard images which are not in canonical pose or expression, which is determined by normalized correlation with an average canonical face. In general, we compute statistics of these attributes while users are being tracked over the short-term, and compare against stored statistics of all previous tracked users. When we observe a new person, we see if there is a previously tracked individual which could have generated the current observations.
U j are the saved statistics for person j .