（様式 20）成果報告書

TSUBAME 共同利用 平成 24 年度 学術利用 成果報告書

利用課題名 マルチメディア内容解析に関する研究
英文：A Study on Multimedia Content Analysis
利用課題責任者 佐藤 真一
First name Surname Shin’ichi Satoh所属 国立情報学研究所
Affiliation National Institute of Informatics
URL http://www.nii.ac.jp
邦文抄録（300 字程度）
本研究では、映像等のマルチメディアコンテンツの主として視覚情報を解析し、その意味内容情報を自動抽出する手法について検討する。特に、映像・画像からの特徴量抽出、学習データを用いた意味内容識別器の学習、並びに評価用データの識別・評価という、一連のマルチメディア意味解析処理の TSUBAMEにおける効率の良い実現を目指す。
英文抄録（100 words 程度）
We study multimedia content analysis, namely, analyzing visual information and delineating
semantic content information. Especially, we try to investigate efficient implementation ofmultimedia semantic content analysis pipeline on TSUBAME, including extraction of features fromimages and videos, training semantic content classifiers using training data, classification andevaluation using test data.

Keywords: ５つ程度
マルチメディア内容解析、特徴抽出、識別器
Multimedia Content Analysis, Feature Extraction, Classifiers
1. Background and Purpose
We have developed NII-KAORI-SECODE [1], amoments,

general framework for semantic concept detection,

histogram, and local binary patterns. The localand use it to participate several benchmarks suchfeature is based on the BOW model in which theasand

SIFT descriptor is extracted at interest points

TRECVID [3]. The purpose is to evaluate

detected by affine invariant and multi-scale dense

performance of various visual representations forsampling detectors.

IMAGE-CLEF[2],

PASCAL-VOC

color

histogram,edge

orientation

concept detection-like task. In this framework,
first features are extracted from keyframes, thenFor

such

concept detectors using these features are learned

PASCAL-VOC,

by using SVM with chi-square RBF kernel. The

detection performance, it requires to use as many

probability output scores of the learned concept

featuresdetectors are used for ranking.

computational resource is needed. For example,

asbenchmarks
TRECVID,possible.

as

IMAGE-CLEF,

toachieve

Therefore,ahighhuge

BBNVISER team (USA) used a shared 309 node,
We evaluate both global features and local

Dual Quad Core, Intel(R) Xeon(R), 32-48GBfeatures.

RAM/node cluster to run for more than 1,000The

globalfeatures

includecolor

（様式 20）成果報告書
hours to detect 10 events in 4,000 hours of video.

3. Results and Discussions
We report here experiments for ImageCLEFThepurposeinvestigateoftheusing

TSUBAME

implementation

KAORI-SECODE

forsuchoflarge

istoPhotoThefeature

theconfigurations are shown in Table

1. Thescale

Annotation

Task.

detection performance is shown in Table 2, which

experiments. Specically, we are studying how tois consistent with our submission last year.

efficiently handle large I/O requests when

It took about 2 hours to extract one type of featureprocessing a large number of images and their(SIFT-variations) for 1,000 images. Quantization

associated features; large number of jobs (e.g.

and soft-assignment take about 3.5 hours. It

training and testing classifiers) needed to run

takes 1 to 4 hours to train a classifier (dependingin parallel so that the experiments can beon the number of training samples), and 2 hours

finished in a short time;

for testing the trained classifier to test images.

2. Technical Details
A general framework for building semantic
content detectors is shown in Figure 1 (Courtesy

Referenceof IBM-Columbia Team in TRECVID 2011).

[1]

We use VLFEAT library [4] to extract local

ComprehensiveStudyfeatures such as DSIFT and PHOW using dense

Representations

forsampling at multi-scales. For each image frame,Detection. ICSC 2011: 235-238.

the number of local features is approx. 10,000.

[2] Duy-Dinh Le, Shin'ichi Satoh: NII, Japan at

The total number of image frames is approx.

ImageCLEF 2011 Photo Annotation Task.

1,000,000. These local features are quantized into

CLEF (Notebook Papers/Labs/Workshop) 2011.

visual words using 1,000-word codebook. After[3] Duy-Dinh Le, Sébastien Poullot, Xiaomengquantization, soft assignement is used to form aWu,

feature vector for each frame.

National Institute of Informatics, Japan atWe use these features to train and test 100

TRECVID 2010. TRECVID 2010.

classifiers for ImageCLEF – Photo Annotation[4] VLFEAT: http://www.vlfeat.org/Task, 346 classifiers for TRECVID – SemanticIndexing Task, and 20 classifiers for TRECVID –Multimedia Event Detection Task.
The huge number of features, images, andclassifier models require a huge disk space tostoreandI/O

requeststoprocess.

Whenmulti-jobs (usually hundreds to thousands of jobs)
access the data for I/O at the same time, it causes
deadlock at disk storages, network bandwidths.
By using TSUBAME, we would like to study how
these problems can be solved.

Duy-Dinh

BertrandLe,

Shin'ichi

Satoh:

of

SemanticNouvel,

Shin'ichi

AFeature
Concept

Satoh:

（様式 20）成果報告書

Figure 1: A general framework for buiding
semanticcontentdetectors(Courtesy

IBM-CU team at TRECVID-MED 2011).

of