Patent ID: 8531504

Claim:
A method, comprising: receiving output from an orientation sensor of a video camera; formatting the output from the orientation sensor as an orientation time series; applying a low pass filter to the orientation time series to produce filtered orientation information; receiving a video sequence from the video camera; estimating interframe three-dimensional (3D) camera rotation for the video sequence; aligning successive images of the video sequence using the estimated camera rotation; formatting the aligned images as an aligned image time series; applying a high pass filter to the aligned image time series, to produce filtered aligned image information; and combining the filtered aligned image information with the filtered orientation information to produce a stabilized video sequence, wherein the above are performed using a suitably programmed processor, and wherein said estimating comprises: computing a Gaussian multi-resolution representation (MRR) of a first of the successive images; for each level of the MRR, proceeding from coarse to fine levels, performing an iteration of the following sequence: for each of a plurality of pixels at location x, y in the first image, defining a constraint ω x [ - I x ⁢ xy f 2 - I y ( f 1 + y 2 f 2 ) + Δ ⁢ ⁢ I ⁢ y f 2 ] + ω y ⁡ [ I x ( f 1 + x 2 f 2 ) + I y ⁢ xy f 2 - Δ ⁢ ⁢ I ⁢ x f 2 ] + ω z ⁡ [ f ⁢ ⁢ 1 f ⁢ ⁢ 2 ⁢ ( - I x ⁢ y + I y ⁢ x ) ] + ( f ⁢ ⁢ 1 f ⁢ ⁢ 2 - 1 ) ⁢ ( I x ⁢ x + I y ⁢ y ) + Δ ⁢ ⁢ I = 0 where I x and I y respectively represent changes in pixel intensity I in x and y directions, thereby forming an over-determined linear system; solving the system to estimate the vector ω=(ω x , ω y , ω z ); creating a rotational matrix R, R ≈ ( 1 - ω z ω y ω z 1 - ω x - ω y ω x 1 ) ; using R, calculating a projection transform P, P = ( f 1 0 0 0 f 1 0 0 0 1 ) ⁢ R ⁡ ( 1 / f 2 0 0 0 1 / f 2 0 0 0 1 ) where f 1 and f 2 are respective focal lengths; and combining P with a projection transform from the previous iteration, to create a projection transform of the current iteration; and using a final projection transform P to calculate a displacement d(x).