Patent ID: 8824801

Claim:
A method for processing video showing at least one deforming object comprising: (i) capturing a sequence of video frames showing the at least one deforming object; (ii) identifying a plurality of features of the at least one deforming object within the frames, each feature having a corresponding pixel; (iii) tracking the movement of the plurality of identified features between frames, the movement of each feature of the plurality of identified features being represented by a respective track of a corresponding plurality of tracks; (iv) analyzing the plurality of tracks representing the movement of the plurality of identified features to determine, for each pair of the tracks, a difference in position along the respective tracks; (v) associating, with each track, a 2D coordinate such that a difference between a pair of the 2D coordinates associated with a corresponding pair of the tracks is similar to a maximally separated distance in video frames of the sequence of video frames for the corresponding pair of the tracks; (vi) building a map using the tracks and the associated 2D coordinates of each track; (vii) accounting for camera roll and zoom using a spatial regularizer comprising: a minimizing function comprising a determinant of a discrete mapping Jacobian and an overall 2D affine transform for each frame; and a temporal coherence term comprising a mapping term for each frame; and (viii) determining an energy for a mosaic created using the map and, if the mosaic does not match predetermined standards, performing energy minimization iteratively until said standards are reached, the predetermined standards being that a change in the energy from a previously calculated energy is less than a threshold value.