Mizuki Akiyama, Yoshiyuki Kawano, Keiji Yanai
Department of Informatics
The University of Electroo-Communication,
Tokyo, JAPAN Object Recognition

 Generic object recognition - object, scene, face, …

TIGER
 Specific object recognition - DB search, name search Object Recognition

 Generic object recognition - object, scene, face, …

Generic Object
Recognition
TIGER
 Specific object recognition - DB search, name searchSpecific Object Recognition

 Specific Object Recognition Large-scale Image database + local feature matching High precision for recognition of specific-shapedobjects
 Generic Object Recognition Small-scale images database + machine learning Low precision due to inter-class ambiguity
 Specific object recognition applies togeneric object recognition.

 Generic object recognition Local feature matching based specific objectrecognition Large-scale training and categorization
 150,000 training images
The chair

A chair Object recognition using a large amount of images

 Specific object recognition method to landmark
database creation[Zheng et al, ICCV2009].
 20 million geo-tagged images on the Web Search a very similar image

 SIFT-matching based image search Generic object recognition using a large amount ofimage data 80 million images [A. Torralba et al., PAMI (2008)]
 Image categorization by k-NN with 32x32 tiny images
 Comparable performance to the state-of-the-art method
 K-NN method with a very large amount of image data
was one of promising approaches for generic object
recognition.

 Our work is inspired by the “80 million images”
 80 million images

 Use the sum of squared differences (SAD)
between 32x32 tiny images
 Our work: Apply SIFT-based local feature matching and voting Collect a large number of images from the Web
 Use them for experiments on image categorization
without excluding noise images

Imagecollection+Feature
extraction

create adatabasefeaturematching

・Collect
150 ,000 fromthe Web・Extract localfeatures

・Local featuresof training
images stored
in thedatabase・Featurematching for test
images・Vote the k mostsimilar samples
stored in thedatabaserecognition

・Majority
based
categorization

 Image collection

 25 categories, total 150,000
 Source: Google,Yahoo!,Flickr noise images included Feature extraction

 SIFT 128 dimension, invariant to rotation, scale-change andillumination change PCA-SIFT
 36 dimension, extension of SIFT

 Bag-of-Features (BoF) Many local patches, and vector-quantizing Explore the best setting of the codebook size k Database kd-tree
 SIFT, PCA-SIFT

 Inverted index
 BoF

 Feature matching Simple NN search is very costly Approximate Nearest Neighbor（ANN）
 Kd-tree based approximate nearest neighbor search method
 Search the top n nearest points for each query local feature point Vote on the image from which the nearest local features areextracted Recognition

 Sort the image having votes in the descendingorder of the number of votes
 Decide one of the given categories by applying
k-Nearest Neighbor classification

 k-Nearest Neighbor Majority of the categories of the top k samples 5 and 25 class categorization

 Feature representation

 SIFT,PCA-SIFT,BoF
 parameters

 ANN top n and k-NN top k Codebook size 5 and 25 categories Dataset

 Examples of images

 The number of training images

 32GB linux machine# of training
images# of images
per class# of local features

memorySIFT

26,250

1,050

15million

20GB

PCA-SIFT

73,5002,940

53.50million

25GB

BoF

145,000

5,800-5GB Evaluation

 Recall, precision, classification rate Baseline: Bag-of-Features + SVM

5 class
25 class
Classification rate(%) Classification rate(%)

SIFT(n=5,k=7,000)
Proposed

60.1

32.5

PCA(n=5,k=7,000)
Proposed
BoF(size=200,000,k=20,000)
Proposed
BoF+SVM(linear kernel)
Baseline
BoF+SVM(χ^2 kernel)
Baseline

57.2

29.8

54.9

30.7

51.7

17.1

66.936.2

5 class classification (SIFT, PCA-SIFT)

Classification rate

ANN: nkNN (top-k)

25 class classification (SIFT, PCA-SIFT)

Classification rate

ANN: nkNN (top-k)

Classification rate

5 class classification (BoF)[thousand]kNN (top-k)

:

100

:
:
:

50
20
5

Classification rate

25 class classification (BoF)[thousand]kNN (top-k)

:

100

:
:
:

50
20
5

1

2

3

45 Recall(%)

1: Animal

155

1437

37

7

2: Car

10228

34

5

3: Flower

35

15150

41

9

4: Food
5: Musicalinstrument

43

24

40

135

862
91
60
54

12128

10148634

Precision(%)

615663587560.3

Higher ranked images
Higher rank images

queryQuery images

 Generic object recognition by feature matchingbased specific object recognition
 It performed wellwith a large number ofsample images
 Almost equivalent to the result by genericrecognition
 5class - 60.3%, 25class - 32.5%

 More large-scale experiments

 Using parallel computing
 informative feature

 Use only discriminative features for image
categorization.

25 class classification (SIFT, ANN n=5)Classification rate

# of training
imageskNN (topk)