Statistical Analysis of Global Motion Chains
Jenny Yuen1? and Yasuyuki Matsushita2
1

2

CSAIL, Massachusetts Institute of Technology, Cambridge MA 02139
Visual Computing Group, Microsoft Research Asia, Beijing 100080, China
jenny@csail.mit.edu, yasumat@microsoft.com

Abstract. Multiple elements such as lighting, colors, dialogue, and camera motion contribute to the style of a movie. Among them, camera
motion is commonly overlooked yet a crucial point. For instance, documentaries tend to use long smooth pans whereas action movies usually
have short and dynamic movements. This information, also referred toas global motion, could be leveraged by various applications in video
clustering, stabilization, and editing. We perform analyses to study the
in-class characteristics of these motions as well as their relationship withmotions of other movie types. In particular, we model global motion
as a multi-scale distribution of transformation matrices from frame toframe. Secondly, we quantify the difference between pairs of videos using the KL-divergence of these distributions. Finally, we demonstrate
an application modeling and clustering commercial and amateur videos.
Experiments performed show advantage compared to the usage of somelocal motion-based approaches.

1

Introduction

Global motion plays an important role in defining and characterizing the style ofa movie. Think about the style of Alfred Hitchcock’s movies, where his characteristic talent manipulating the audience’s fears and desires was known greatly for
his technique in moving and placing his camera [28, 2]. Another example is thehighly dynamic motion in The Blair Witch Project, where critics have described
how the camera motion can be used to reveal a story through its startling images
and lucid characterizations [19]. Moreover, global motion not only varies depending on the director style, but also its content. For example, the characteristicsmooth and continuous camera motions in National Geographic documentaries
are different from the fast and diverse mix of camera movements in an action
movie like Spiderman.
Videos of different genres clearly contain different global motion chains, which
is a sequence of global motions. In this work, we study the statistics of global
motion chains in the spirit of analogous works analyzing image and local flow
statistics [23, 11]. Unlike previous works that use global motions for some particular applications such as video classification, video stabilization and etc., thiswork focuses on the development of a framework to analyze global motion chainsin two aspects (Figure 1):?

Part of this work was done while the author was visiting Microsoft Research Asia.

2

Jenny Yuen and Yasuyuki Matsushita
Input

Preprocessing

Class #1

In-class analysis

Multi-scale sampling…

Class #2…

Dimensionality

…

…

…

…

…

…

…
Space #1 Space #2 Space #N

Typical motion

Inter-class analysis…

Global image-motion chain… Ti 1 Ti

Class #NTi 1 …

Class similarity…
Video frames of different classes…

…

Space of all categories

Differentiating
factorFig. 1. Overview of the analysis of motion chains.

1. In-class Analysis. We want to learn what the motion in a class of videos islike by finding its characteristic camera movements and learning about theirintrinsic dimensionality.
2. Inter-class Analysis. We want to learn what makes camera motions of a classdifferent from motions in another class and, if particular motions are betterat distinguishing a class from another one.
Furthermore, we apply the learnt statistics to compare and cluster videos
using KL-divergence as an application of the analyses. We show the results forthree types of professional (action movies, basketball games, and documentaries)
as well as unprofessional videos taken by numerous people. Finally, we compare
the effectiveness of our feature with other motion-based ones when used forclustering and will demonstrate the differences and advantages of our feature in
the area of video clustering.
1.1

Related work

There has been a lot of work using global image motion but to the extent to ourknowledge, none studying the statistics of global motion chains. The notion of
global motion chains, or sequence of transformations between consecutive video
frames in a video, has been widely used in the context of video stabilization [13,20, 16]. These tend to be parameter-based with the objective of smoothing sequences for motion compensation. Motion compensated frames are obtained bywarping the original video frames by a cascade of the original and smoothedtransformation chains.
While not using the global motion chains, there are several previous works analyzing actions in video clips. These are classified in model-based approaches [21,18, 1, 14, 24] and nonparametric approaches [5, 6, 30]. The model-based approaches
are designed for detecting predefined activities occurring in videos. These areshown to be useful for restricted conditions where the assumed models are valid.
More recent trends in action analysis lies in non-parametric approaches.

Statistical Analysis of Global Motion Chains

3

Chomat and Crowley [5] use a set of spatio-temporal filters and evaluate thejoint statistics of filter responses for the purpose of probabilistic action recognition. Efros et al . [6] use dense optical flow fields as a feature to perform a
nonparametric action recognition. Zelnik-Manor and Irani [30] propose a multiscale spatio-temporal feature which captures the similarity between video clips
based on the behavior contained in the videos. Their feature is based on a local
intensity gradient in space-time domain, and it successfully captures local motion
occurring in videos. Wang et al . [29] developed various methods for detecting
and classifying events in surveillance videos using hierarchical linear discriminant analysis (LDA) to cluster local motion attributes. Stauffer et al . [27] [9]use tracking for recognition and segmentation of objects in the video . Thesemethods are efficient and robust in many applications such as surveillance or
activity recognition and have been extensively tested in still camera settings.
We further extend the problem domain and work with videos in a large numberof settings and containing significant camera motion.
For the purpose of video classification and other applications, the global
motion has been used by several researchers. Roach et al . [22] use a backgroundimage motion represented by (x, y) translation for the purpose of video genre
classification. Affine camera motion is used as one of video features by Smith andKanade [26] for video skimming. Kobla et al . [12] use the camera motion featurefor identifying sports videos. Bouthemy and Ganansia [4] propose a method forfinding shot changes in a video using global motion. Closest to our work, Fablet etal . [7] use global motion for video classification and retrieval. The global motion isrepresented by a temporal Gibbs random field to perform the temporal analysis.
All these works show interesting applications for camera dynamics. We present
this paper with a different focus: we want to learn and understand the statistics
of motion chains of camera motions across different types of video.
Our work differs from these previous works because we focus on a multi-scale
analysis of global motion chains. In particular, we are interested in observing
the degree of information contained in the global motion chains across videos ofdifferent categories and, as an example, to evaluate to what extent classification
can be done in a setting of varied short clips and a moving camera.
While, to the best of our knowledge, there has been no work studying thestatistics of global motion chains, Roth and Black [23] propose a model to learn
the spatial statistics of optical flow fields using 3D range data. They model theflow field as a Field-of-Experts and show numerous statistics of the velocity andorientation of this information. Furthermore, they apply these statistics as aprior for a more robust estimation of optical flow. This paper is analogous toRoth and Black’s work, with the objective of learning the form, distribution,and characteristics of global motion chains, which has not yet been done.
1.2

Background

Let V = {I1 , ..., In } be an image sequence composed of n frames where Ii is its
i-th frame. Assume for now that only the camera moves while the environment
captured in the video is static and planar. We refer to the term global motion

4

Jenny Yuen and Yasuyuki Matsushita

between two consecutive frames as a 3×3 projective transformation matrix T. By
representing 2D pixel positions (x, y) in homogeneous coordinates x = (x, y, 1)T ,T I 0 (x0 , y 0 ) can be written as x0 ∼ Tx,
the geometric transformation I(x, y)=⇒where ∼ denotes equality up to scale. The condition for the input video is relaxed
for the case of an image sequence with moving objects in the scene by extracting
the transformation matrix assuming that the majority of the objects in thecaptured scene remain static in the real world.
For our global motion estimation, we will assume the affine transformation
model. Our reasoning for this assumption is justified by the fact that often the
majority of image pixels will be background pixels and they will be far enoughfrom the camera that could be approximated to being on a plane. The affine
transformation can be written as:a11 a12 a13(1)x0 = Ax =  a21 a22 a23  x,
0 0 1
where x is the original position, x0 is the new position, A is an affine transformation matrix. This will reduce the number of variables to six per matrix. We
also define a vector representation of T(= A) in scan-line order as t:t = [a11 , a12 , a13 , a21 , a22 , a23 ]T .

(2)

Global Motion Chains In our analysis, we use ordered sequences of global motions
which we will refer to as global motion chains. We represent the global motion
chain of an image sequence centered at the i-th frame as an ordered list oftransformation matrices of consecutive frames as:T
Cj = [ti−r ]T , [ti−(r+1) ]T , . . . , [ti+r ]T ,(3)
where r is a positive constant that denotes the radius of the temporal window of
the transformation chain Ci , ti is the vector representation of the transformation
matrix Ti from frame i to frame i + 1.
We further extend this notation to a multi-scale scheme where the samplingof frames is controlled by a frame skip rate s. The global motion chain with theframe skip rate s can be written asT
Csj = [tsi−rs ]T , [tsi−(r+1)s ]T , . . . , [tsi+rs ]T ,(4)
where, analogously, Csi is a global motion chain centered at the i-th frame andtsi is the vector representation of the transformation matrix Tsi from frame i toframe i+s. The frame skip rate allows us to perform a multi-scale representationof the motion information used later in our study.

2Analysis of Global Motion Chains

In this section we will describe a framework for comparing global motions. We
are interested in learning how global motions vary per class and what theirStatistical Analysis of Global Motion Chains

5

T1x1

x1

T1x 2

x2

T2 x 2

T2 x1
X

T1X

T2 X

Fig. 2. Illustration of the similarity of global motions. Left: points are distributed ina two dimensional coordinate system. These points are transformed by T1 and T2 in
the middle and right figures. The similarity of global motions T1 and T2 is defined by
the sum of distances between corresponding points X.

characterizing elements are. We model a class of videos as a distribution of
motion chains. In general, there will be a motion chain over a temporal window
r for each frame in each video of a class (excluding some frames at the temporaledges of each video). We will further extend this to a multi-scale approach where
we will subsample frames at different rates but the general idea of our model isa distribution of motion chains.
2.1

Similarity of global motion chains

We will first define a similarity measure of global motion for general linear transformations in 2-D, e.g., rigid body, similarity, affine and projective. For this case,the similarity of two transformations has the nice and simple property that it
can be expressed as the L2 norm distance between the two transformations whendefining the similarity as the amount of point displacement in a 2D plan. The
derivation is as follows.
Given some disk in a 2D plane and distributed points on the disk (Figure 2left). We define the similarity of two linear transformations by the sum of distances of the warped points. Given two linear transformations T1 and T2 , thesimilarity S(T1 , T2 ) of these two transformations can be described asZ
def 1
S(T1 , T2 ) =p(x, y)||T1 (x y)T − T2 (x y)T ||22 dxdy
|D| D
Z
1
p(x, y)||(T1 − T2 )(x y)T ||22 dxdy,
(5)
=
|D| D
where D represents a set of points distributed on the disk, |D| is the number ofpoints, || · ||2 denotes the L2 norm, and p(x, y) is the probability density functionof the point at (x, y). Assuming that p(x, y) has a uniform distribution, as itis the case in a regular 2D images if we consider pixels as points, the above
equation can be simplified toS(T1 , T2 ) = Z||t1 − t2 ||22 ,

(6)

6

Jenny Yuen and Yasuyuki Matsushita

where the constant Z is written asZ1Z=p(x, y)||(x y)T ||22 dxdy.
|D| D

(7)

From this result, the similarity of two linear transformations can be computedby the L2 norm. Consequently, the similarity SC of a pair of global motion chains
C and C 0 can be written as
X(C)(C 0 )SC (C, C 0 ) =S(Ti , Ti ),
(8)
i
(C)

where Ti represents the i-th transformation matrix in C.
The similarity metric can be augmented by modifying the probability density
function p(x, y). For example, if we assume non-uniform importance of pixels,
i.e., regions of interest, p(x, y) becomes non-uniform. While it is possible to use
this generalized form of the similarity metric, we assume the uniform density ofp(x, y) for simplicity.
2.2

In-class Analysis

In-class typical motions The purpose of this section is to study what typicalmotion chains are like. Intuitively, we can think about how shaky camera motion
could characterize handheld-captured videos, slow pans might be common in
documentaries about nature, or highly dynamic movements in action movies.
To identify the typical transformation chains, we fit a mixture of Gaussians
for each class distribution of transformation chains. The means of the Gaussian
centers describe typical transformation chains in the class. We choose the numberof Gaussians m according to the Bayesian information criterion (BIC) [25]:m = argmin{− log P (data|m, θ̂m ) +mkm
log n},
2(9)

where km is the number of parameters given m Gaussians, θ̂m is the maximum
likelihood parameters for the distribution with m Gaussians, and n is the numberof points in the distribution.
Space of global motion chains We will also study the compactness in the distribution of each class of videos. For example, consider the extreme case forsurveillance videos where the camera is static. In this case, the camera motionof this video can be compactly characterized by this static state (or viewed in
our terms as an identity matrix since there is no global motion from frame toframe). On the other hand, if we have completely random motion, the information required to characterize the space will be larger than in the previous
example.
To analyze the compactness of the in-class distribution of transformation
chains, we project each distribution of points to their principal components andanalyze the cumulative variance of the distribution as a function of the numberof principal components selected.

Statistical Analysis of Global Motion Chains
Frame skip
factorAction Movie

Frame skip
factor

200

200

100

100

50

50

1

1

Significance
Frame skip
factorDocumentary

Basketball Game

Significance
Frame skip
factor

200

200

100

100

50

50

1

1

Significance

7

Unprofessional

SignificanceFig. 3. Characteristic motions in each category of videos across different frame skip
rates. The motions are ordered by descending significance (see Section 2.3) from left toright. Each frame in the motion chain is described as a rectangle undergoing an affine
transformation from frame to frame. The first frame is denoted by the rectangle withthe * in each corner.

2.3

Inter-class Analysis

This section focuses on the development of a comparison metric between camera
motion distributions. Based on the modeling of a group of movies of a particularclass as a distribution of motion chains, we develop a metric to compare between
two videos (or groups of videos). Moreover, exploiting the form of a Gaussian
Mixture and the representation of a typical motion as a Gaussian component,
we propose a metric for ranking characteristic motions in order of significance,
which intuitively means we order typical motions with respect to how useful they
are distinguishing the class they belong to from others.
Video Comparison The divergence of two distributions is useful for comparing
pairs of video classes, which in turn can be used for clustering of individual
videos. In section 4 we show how we apply this metric for video clustering.

8

Jenny Yuen and Yasuyuki Matsushita

We compare pairs of distributions (each representing a class) and analyze howmuch one diverges from the other using an approximation of the KL-divergenceof two mixtures of Gaussians proposed by Goldgerber et al . [8]:KL(f ||g) ≈

n
X



αi
,
αi min KL(fi ||gj ) + log
j
βj
i=1

(10)

where αi and βj are the mixing coefficients for fi and gj respectively andKL(fi ||gj ) is the KL divergence approximation for two Gaussian mixture components fi and gj :|Σgj |1
T −1log+ Tr(Σg−1
Σ
)
+
(µ
−
µ
)
Σ
(µ
−
µ
)
. (11)KL(fi ||gj ) ≈
f
f
g
f
g
i
i
j
i
j
fi
j
2
|Σfi |
Significance of a Typical Motion Identifying the canonical motions that differentiate a class from others can be helpful in creating stronger classifiers. In thissection we present a ranking method for typical motions to identify what are themotions that characterize each class and at the same time differentiate it fromthe rest.
As a consequence of equation (10), Goldberger et al . [8] propose a method formatching some Gaussian component from a mixture of Gaussians (MoG) f toanother component π(i) of another MoG g via the following matching function:|Σ2,j |1−1log
+ Tr(Σ2,jΣ1,i ) +(12)
π(i, g) = argmin2|Σ1,i |j−1(µ1,i − µ2,j )T Σ2,j(µ1,i − µ2,j ) − log βj .
Based on this matching-based approximation, we propose a way of ranking
canonical motions of each distribution in a set of MoG G as|Σ2,jg |1−1
Σ1,i ) +(13)
p(i) = minlog+ Tr(Σ2,jg|Σ1,i |g∈{G−f } 2−1(µ1,i − µ2,j )T Σ2,j(µ1,i − µ2,jg ) − log βj g .
gThis ranking is based on the matching-based approximation from equation 13
where, the larger a value is, the more divergent the typical motion (Gaussiancomponent) is from other typical motions in the set of distributions (other videoclasses).

3

Experiments and Results

3.1

Dataset and Data Processing

We perform our analyses over two video datasets:

Statistical Analysis of Global Motion Chains

Contribution

frame skip factor: 1frame skip factor: 100

frame skip factor: 200

1

1

1

0.9

0.9

0.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

0.6

0.5

0.5

0.5

0.4

0.4

0.4

0.3

0.3

0.3

0.2

0.2

0.2

0.1

0.1

0.1

0

0

1

10

19

28

9

ActionDocumentary
Basketball
Handheld

0

1

10

19

28

1

10

19

28

number of principal components

Fig. 4. Plots of cumulative variance as a function of principal components of motion
chains. These plots show the size of space accounted by some number of principal
components.

1. Dataset 1. Consists of commercially produced video and unprofessional video(home videos captured by various amateurs) divided into the following categories: action movie (180 minutes) , documentary (120 minutes), basketballgame (120 minutes), and unprofessional (360 minutes).
2. Dataset 2. Consists of 176 video clips downloaded from YouTube: 130 professional and 46 unprofessional
We extracted the similarity transformation matrices between each pair forconsecutive video frames in the sequences sampled at the particular skip rates toconstruct transformation chains of a temporal window of radius r = 7 using thehierarchical Lucas-Kanade algorithm [15, 3]. Using the similarity transformation
we have 6 variables, therefore each transformation chain is a feature point in 42(= 6 × 7) dimensions in our case.
3.2

In-class Analysis

Figure 4 shows the cumulative variances of the unprofessional videos and thethree professional classes of videos as a function of number of principal components used (ordered by eigenvalue). Each graph shows this relation acrossdifferent levels (skip rates).
In these plots, it is observed that the cumulative variance of the unprofessional dataset (handheld) is consistently high among the four datasets. This suggests that the space of unprofessional motion-chain is more compact comparedwith others in these levels. This result is counter-intuitive because the unwanted
high-frequency motions in unprofessionally-created movies are expected to create a larger variability. One interpretation for this result is that professionallycreated movies have wider variety of global motions that are well-designed, while
the unprofessional ones do not. This differentiates the variety of global motions,and it appears as a difference in size of spaces.
Figure 3 shows the motion means from each of the unprofessional videos aswell as the three professional subclasses. Each row shows characteristic motions

10

Jenny Yuen and Yasuyuki Matsushita

at a particular skip rate and in each row, the motions are ordered in order ofsignificance.
3.3

Inter-class Analysis

Figure 3 shows typical motions in each class ordered by significance (see equation 14). As expected, in all classes, the static camera motion chain was ranked
the lowest in terms of significance. In other words, a non-moving camera shot
exists in videos of all types and contributes little in characterizing the video or its
class. In contrast, more complex motions serve to characterize and discriminate
within classes:– Action Movies. Characterized by a variety of horizontal and vertical motions. While some translations are carefully-generated continuous paths,
there are some drastic changes in location and zooms. At a low level there
are simple vertical and horizontal motions while at a higher skip rate, there
are more zooms with highly dynamic motions. For instance, the camera canbe focusing on a character and moving quickly when the focus changes toanother character.
– Basketball Games. Contain very carefully guided curved camera paths
with continuous zoom. Low level motions are general motions in horizontal
and vertical directions as well as the lack of movement (static camera). Highlevel motions are mostly careful zooms with little translations. There are
more horizontal motions at higher skip rates compared to other classes. Thisis probably due to the rapid and varied motions of players in a sport ofthis nature. Moreover, these effects are possible thanks to equipment notavailable in other classes like the unprofessional one.
– Documentaries. Motions are very simple and there are fewer characteristictranslations with respect to the other classes studied. At a low level there arecontinuous motions vertically and horizontally while more complex motions
happen at higher skip rates including zooms and carefully controlled paths.
At very high skip rates, there are mostly zooms with non or very little
translation. This is characteristic of documentaries when filming packs ofanimals in aerial views.
– Unprofessional. This class has several characteristic motions in all directions. Due to the limited translation capability of the cameraman in at
a higher level, there is little translation at higher skip rates. In general
there are less continuous zooms and unstable translations compared to theprofessionally-generated videos.

4

Application to video classification

We are interested in seeing to what extent clustering can be achieved only using
global motion as a feature. We represent each video clip as a distribution of
global motion chains and use the symmetric version of the KL-divergence approximation previously described in section 2 to compare pairs of global motion

Statistical Analysis of Global Motion Chains
BG

Unprofessional

AC

0.022
0.02

DOC

0.018

0.022
0.02
0.018
0.016

0.014

0.014

0.012

0.012

0.008
0.006

BG

Professional

0.016

0.01

(a) Within professional videos

Unprofessional

DOC

Professional

AC

11

0.01
0.008
0.006

0.004

0.004

0.002

0.002

(b) Professional and unprofessional videos

Fig. 5. Similarity matrices: (a) Similarity matrix for professional videos: action movies(AC), documentaries (DOC), and basketball games (BG), and (b) Similarity matrix
for professional and unprofessional YouTube videos.

chain distributions to create an affinity matrix used to perform a normalized-cuts
spectral clustering [17] . The similarity between two video clips with distributions of motion chains fs and gs at scale s respectively is therefore defined asfollows:
1
def
S(vf , vg ) = PM.
(14)
s=1 {KL(fs ||gs ) + KL(gs ||fs )}Figures 5 (a) and (b) show the affinity matrix for the professional videos indataset 1 and for all of dataset 2 respectively. The brightness of the region is
directly proportional to the affinity of the videos in that section. In dataset 1,
basketball game clips are more similar to each (and easier to cluster) other thanaction movie and documentary clips. In the case of dataset 2, unprofessional
clips are more similar to each other than professional videos.
Comparison to other Motion Features We evaluate the effectiveness of our global
motion chain feature representation in the dataset described in section 3. The
13 hours of video are cut into 3 minute long non-overlapping clips totalling 253
video clips.
We compare our motion chains feature with three other motion-based features: motion images described by [10], motion magnitude histograms, and motion direction histograms. Motion Images are 2D grids accumulating motion information at the corresponding parts of the canvas where the motion happened.
The motion magnitude and direction histograms summarize the magnitude anddirection respectively of optical flow vectors throughout the video clip.
We implement a version of motion images and the magnitude and direction
histograms using optical flow over pairs of consecutive frames using the blockmatching implementation in OpenCV. The flow vector direction and magnitude

Jenny Yuen and Yasuyuki Matsushita

100

90

100

100

87.5

83.3

95.8

Motion chain
Motion images
Magnitude
Direction

82.4

Clustering accuracy

80
70

61.2

60
50

52.9
41.7
35.3

40

47.4
42.1

39.7
36.2

26.3

30
20
10

96.5

100
90

Clustering accuracy

12

80

89.5
84.2

84.0

73.7

71.4

70
60
50
40

Motion chain
Motion images
Magnitude
Direction

61.8
58.8
53.1

50.0
33.3

30
20

16.7

10

0

0

0
ActionmoviesDocumentary Basketball
game

Average(a) Dataset 1

Professional

Unprofessional

Average(b) Dataset 2

Fig. 6. Clustering accuracy over datasets 1 and 2.

information are summarized in normalized histograms and the values of each bin
are used as feature values.
To compute the accuracy of the clustering results within each class, we calculate the ratio between the number of instances that were grouped together (ifthere is more than one grouping, we choose the largest grouping) and the total
number of instances in the ground-truth of that class. Each of the 4 types offeatures were used to construct similarity matrices and spectral clustering wasapplied to them. Figure 6 (a) shows the classification rates amongst professional
videos using the different features for dataset 1. Global motion performed far
better than our competition in all the three classes with an average of advantage
of at least 34.57%. The most challenging class was the documentaries one (DOC)
where we achieved an 87% accuracy, still significant in comparison to the othermethods tested.
Figure 6 (b) shows the performance of our feature when used to discriminate
between professional and unprofessional videos in our dataset of YouTube videos.
We are able to correctly group unprofessional videos with 96.48% accuracy andprofessional ones with 71.43% accuracy.

5

Discussion and Conclusion

We perform an analysis of the global motion distribution within various videoclasses and show how the space of unprofessional motions is more compact withrespect to professionally-captured ones at a high level. At a low level, action
movies are the least compact followed by basketball games and documentaryfilms in that order. In addition, we learned that unprofessional movies have
similar wide range of motion chains with respect to professional films while at ahigher level, the camera motion of professional films is more controlled and lessvaried in comparison with unprofessional movies.
We also develop a way to compare camera motions which, as a result, weapply in a video clustering application. Our method is more efficient at clusteringStatistical Analysis of Global Motion Chains

13

videos when compared to other motion-based features in the literature. Global
motion features perform better in our dataset where there is a large variety ofcamera motions characterizing the genre of the video while other features suchas global motion and other low-level ones based on local motion perform wellin certain scenarios where the clip is short, the camera is static, and contains
simple actions.
In conclusion, have performed a camera motion study and show that camera
motion is indeed different across video classes and describe characteristics acrossclasses and what differentiates them. We also show experimentally that global
motion carries significant information about the movie type by comparing our
feature to other features based on local motion, where the clustering results wereconsistently better using global motion as a feature. Other potential applicationsof this work include camera motion transfer between sequences to change styles,
recognition of directors in videos, or simple characteristic motion extractors forlearning about different genres or direction styles.

References
1. M. J. Black and Y. Yacoob. Recognizing facial expressions in image sequences using
local parameterized models of image motion. International Journal of Computer
Vision, 25(1):23–48, 1997.
2. S. Blanford. The Film Studies Dictionary. Oxford University Press, 2001.
3. J. Y. Bouguet. Pyramidal implementation of the lucas kanade feature tracker :description of the algorithm. In OpenCV Document, Intel, Microprocessor ResearchLabs, 2000.
4. P. Bouthemy and F. Ganansia. Video partitioning and camera motion characterization forcontent-based video indexing. In Proceedings of International Conference
on Image Processing, volume 1, pages 905–908, 1996.
5. O. Chomat and J. L. Crowley. Probabilistic recognition of activity using local
appearance. In Proceedings of Computer Vision and Pattern Recognition, pages
104–109, 1999.
6. A. Efros, A. Berg, G. Mori, and J. Malik. Recognizing action at a distance. In
Proceedings of International Conference on Computer Vision, volume 2, pages 726–
733, 2003.
7. R. Fablet, P. Bouthemy, and P. Pèrez. Statistical motion-based video indexing
and retrieval. In Proceedings of RIAO Conference on Content-Based Multimedia
Information Access, 2000.
8. J. Goldberger, S. Gordon, and H. Greenspan. An efficient image similarity measure based on approximations of kl-divergence between two gaussian mixtures. In
Proceedings of International Conference on Computer Vision, 2007.
9. E. Grimson, C. Stauffer, R. Romano, and L. Lee. Using adaptive tracking toclassify and monitor activities in a site. In Proceedings of Computer Vision and
Pattern Recognition, volume 2, pages 22–29, 1998.
10. A. Haubold and M. Naphade. Classification of video events using 4-dimensional
time-compressed motion features. In Computer Vision and Image Understanding,pages 178–185, 2007.
11. J. Huang. Statistics of natural images and models. In Ph.D. Thesis, 2000.

14

Jenny Yuen and Yasuyuki Matsushita

12. V. Kobla, D. DeMenthon, and D. Doermann. Identifying sports videos using replay,text, and camera motion features. In Proceedings of the SPIE conference on Storageand Retrieval for Media Databases, pages 332–343, 2000.
13. A. Litvin, J. Konrad, and W. C. Karl. Probabilistic video stabilization using
kalman fitering and mosaicking. In Proceedings of IS&T/SPIE Symposium on
Electronic Imaging, Image and Video Communications, pages 663–674, 2003.
14. F. Liu and R. W. Picard. Finding periodicity in space and time. In Proceedings ofInternational Conference on Computer Vision, pages 376–383, 1998.
15. B. D. Lucas and T. Kanade. An iterative image registration technique with anapplication to stereo vision. In Proceedings of International Joint Conference on
Artificial Intelligence, pages 674–679, 1981.
16. Y. Matsushita, E. Ofek, X. Tang, and H.-Y. Shum. Full-frame video stabilization.
In Proceedings of Computer Vision and Pattern Recognition, volume 1, pages 50–
57, 2005.
17. M. Meila and J. Shi. A random walks view of spectral segmentation. In Artificial
Intelligence and Statistics, 2001.
18. S. A. Niyogi and E. H. Adelson. Analyzing and recognizing walking figures in xyt.
In CVPR, pages 469–474, 1994.
19. T. A. Ohanian and M. E. Phillips. Digital Filmmaking: The Changing Art andCraft of Making Motion Pictures. Focal Press, 2000.
20. M. Pilu. Video stabilization as a variational problem and numerical solution withthe viterbi method. In Proceedings of Computer Vision and Pattern Recognition,
volume 1, pages 625–630, 2004.
21. R. Polana and R. C. Nelson. Detecting activities. In Proceedings of Computer
Vision and Pattern Recognition, pages 2–7, 1993.
22. M. J. Roach, J. D. Mason, and M. Pawlewski. Video genre classification using
dynamics. In ICASSP, pages 1557–1560, 2001.
23. S. Roth and M. J. Black. On the spatial statistics of optical flow. International
Journal of Computer Vision, 74(1):33–50, 2007.
24. P. Saisan, G. Doretto, S. Soatto, and Y. N. Wu. Dynamic texture recognition. In
Proceedings of Computer Vision and Pattern Recognition, volume 2, pages 58–63,
2001.
25. G. Schwartz. Estimating the dimension of a model. The Annals of Statistics,
5(2):461–464, 1978.
26. M. Smith and T. Kanade. Video skimming and characterization through the combination of image and language understanding techniques. In Proceedings of Computer Vision and Pattern Recognition, pages 775–781, 1997.
27. C. Stauffer and E. Grimson. Learning patterns of activity using real-time tracking.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8):747–757,
2000.
28. M. Walker. Hitchcocks’s Motifs. Amsterdam University Press, 2005.
29. X. Wang, X. Ma, and E. Grimson. Unsupervised activity perception by hierarchicalbayesian models. In Proceedings of Computer Vision and Pattern Recognition,
volume 2, pages 1–8, 2007.
30. L. Zelnik-Manor and M. Irani. Statistical analysis of dynamic actions. IEEE
Transactions on Pattern Analysis and Machine Intelligence, 28(9):1530–1535, 2006.