Akitsugu Noguchi and Keiji Yanai
The University of Electro-Communications, Tokyo, JapanBackground◆Number of videos is increasing rapidlyExperiments◆Human action classification

・ Web video like Youtube

・ Classify human action by SVM
・ Dataset
KTH dataset which contains 6 motions
Each motion contains 100 videos
Multi-class classification with 5-fold-cross-validation
・ Evaluate the following four combination of the feature
visual+motion+rotation(VMR)
visual+motion(VM)visual only(V)motion only(M)◆Video search is very difficult problem
・ Text based search is utilized to search web for videos.
・ It is difficult to search video which user want to watch only using
text based search.

◆Content based video analysis will be needed・ Recently spatio-temporal feature has been proposed・ To investigate vast amount of video data, speed up technique
is very important.

Past work
Temporal t

Extract features from cuboids◆Existing method
・ 3D Harris corner detector(Laptev et al. 2008)
・ 2D-Gaussian filter and 1D-Gabor filter(Dollar et al. 2005)◆Issue11
VMR
86% 0.90.8
VM
0.7
83.3%0.6V
64.4% 0.5
M 0.469.9% 0.3
0.2
0.1
00.9
0.8
0.70.6・ Extracting features from a whole cuboid costs much in termof computation.
・ Is it necessary to extract features from whole cuboid?

◆In our work
・ We want to extract features more fast and efficiently.
・ We describe feature with a point and local track.
・ Taking advance its high-speed performance, we apply to large scale
web video search.

Proposed method◆System overview

0.5
0.4
0.3
0.2
0.1
0
walking running jogging boxing waving clapping average

Result by four types of combination of featureOur86% method
Dollar
82.3%
91.5%
91.8%

Comparison with other result by the state-of-art◆Web video clustering
・Collect 100 soccer videos from Youtube
・ Divide Each video into shots
・ Extract features from each shot・ Classify web video shot by k-means clustering

Result of web video shot clustering per single video

・Extract features based on SURF detector.
(figure shows extracted points)

2. Detection of tracking points
・Points without motion are not suitable
for spatio-temporal feature.
・In-motion points are selected by optical flow analysis.

Extraction of motion features N frames(processing unit)

・Extract following
Detected optical flow
N-frames as theprocessing unit.
・Divide N-frames intosome interval, andoptical flows arecalculated from
Optical flows are calculated fromeach interval.
each interval
・To make features more robust about rotation, rotate optical flow
along the dominant direction of visual feature.

3.4. Build vector

・Concatenate visual and motion features into one vector withweight w.
・Build bag-of-video-words

Laptevwalking running jogging boxing waving clapping average① Extract visual feature point using SURF detector.
② Detection of tracking point and extract motion feature.
③ Build vectors which combine motion and visual features.
④ Build bag-of-video-words.

1. Extract visual features

Alireza

Result of all web video shot clustering

Future work
◆To improve this feature
Detection of camera motion◆Vast amount of web video shot clustering
Collect more than 1000 video
Clustering not only soccer video but many kinds of videos