Image Classiﬁcation by a Probabilistic Model
Learned from Imperfect Training Data on the Web
Keiji Yanai
Department of Computer Science,
The University of Electro-Communications
1-5-1 Chofugaoka, Chofu-shi,
Tokyo, 182-8585 JAPANyanai@cs.uec.ac.jp
ABSTRACT
Current approaches to image classiﬁcation require training
images prepared by hand. In this paper, we describe experiments on image classiﬁcation using images gathered fromthe Web automatically as training images. To gather images
from the Web, we use the probabilistic method we proposed
before. In the method, we build a generative model which
is based on the Gaussian mixture model (GMM) from imperfect training images gathered from the Web in order todistinguish relevant images from irrelevant images. In thispaper, we propose applying the model built during Web image gathering process to generic image classiﬁcation task. In
the experiments, we classiﬁed Corel images with the probabilistic model learned from Web images automatically.

Categories and Subject Descriptors
I.4 [Image Processing and Computer Vision]: Miscellaneous

General Terms
Algorithms, Experimentation, Measurement

Keywords
image annotation, probabilistic image selection, Web image
mining

1. INTRODUCTION
Recently there has been much work related to semantic
image classiﬁcation [8, 13, 4] and annotation of words toimages [1, 9]. In these studies, image sets gathered by hand

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for proﬁt or commercial advantage and that copies
bear this notice and the full citation on the ﬁrst page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior speciﬁc
permission and/or a fee.
MDM/KDD’06, August 20, 2006, Philadelphia, Pennsylvania, USA.
Copyright 2006 ACM 1-59593-446-4 ...$5.00.

or commercial image collections such as Corel image librarywere used as training images, since training images withkeywords were required.
On the other hand, recently we are working on gathering
images from the World Wide Web and applying them togeneric object recognition tasks as visual knowledge insteadof hand-made or commercial image collections [13]. Learning of image concepts from the Web is being paid attention
as a new framework to avoid human labor for making training image sets [11, 5, 6]. Web images are as diverse as realworld scenes, since Web images are taken by a large numberof people for various kinds of purpose. It can be expectedthat diverse training images enable us to classify/recognize
diverse real world images. We believe that use of image
data on the Web, namely visual knowledge on the Web,is promising and important for resolving real world image
recognition.
In this paper, we describe experiments on image classiﬁcation using images gathered from the Web automatically as
training images. To gather images from the Web, we use theprobabilistic method [15] which we proposed recently. In themethod, we build a GMM-based generative model from imperfect Web image sets to distinguish relevant images from
irrelevant images. Then, in this paper we propose applying the model built during Web image gathering process togeneric image classiﬁcation task.
So far much work regarding Web image search has been
proposed as well as commercial services. However, most ofthem focused on only “search”. On the other hand, ourpurpose of use of images on the Web is “Web image mining”[14, 13], which means searching the Web for images andthen using them as visual knowledge for some applications.
Regarding text data, there are many studies about how
to gather data from the Web and use it as “knowledge” effectively. While such Web text mining is an active researcharea, mining image data on the Web poses additional challenges and has seen less research activity. The problem withmining images for knowledge is that it is not known how
to reliably automatically determine semantics from image
data. This has been refereed to as the semantic gap. To

solve it, it is indispensable to introduce sophisticated image
recognition methods into Web mining regarding images.
In [14, 13], we proposed gathering a training data set
for generic image recognition from the Web automatically,and have revealed that we could use Web images as “visual knowledge”. In this paper, we extend this idea with asophisticated probabilistic method.
The advantage of our method for Web image gathering[15] is that we can obtain many images relevant to a certain keyword “X” by just providing the keyword “X” without any supervision or any feedback. The combination ofHTML analysis and probabilistic image selection enable it.
We proposed starting with images evaluated as highly relevant ones by analyzing associated HTML texts as training
images. In our previous work [12], we revealed that images
whose ﬁle name, ALT tag or link word includes a certain
keyword “X” are relevant to the keyword “X” with around
75% precision on average. Although the images include 25%
irrelevant images, and many of the remaining 75% are not adesired canonical example, they provide an adequate starting point for our approach. We then build a model of a visualconcept associated to the keyword “X”. We use a generativemodel based on the Gaussian mixture model to represent“X” model, and estimate the model with the EM algorithm.
Next, we “recognize” images evaluated as highly relevant
by analyzing associated HTML texts with the model, andselect “X” images from them. By repeating this image selection and model estimation for several times, we can reﬁnethe “X” model and ﬁnally obtain “X” images with the highaccuracy.
In our previous work [15], we used a probabilistic model
only to select “X” images from Web images. In this paper,
we then apply the models built during Web image gathering process to generic image classiﬁcation task which is
not limited to Web images. To investigate these ideas, webuilt models regarding six keywords through the probabilistic Web image gathering processes, and applied them to image classiﬁcation of 6 kinds of Corel image sets.
The rest of the paper is as follows. In Section 2, we describe the method to build models and to apply them toimage classiﬁcation task. In Section 3, we explain the experimental results of 6-category image classiﬁcation. In Section
4, we conclude this paper.

2. METHOD
We apply the probabilistic method employed in our probabilistic Web image gathering system [15] to generic image
classiﬁcation tasks. Therefore, the method we propose inthis paper is an extension of the Web image selection method
we employed in our previous work.
In our Web image gathering scheme, we gather several
hundreds of Web images relevant to a given concept. At ﬁrst,
we provide keywords which represent the visual concept ofimages we like to obtain. For example, “mountain”, “beach”
and “sunset”. Using Web image/text search engines, wegathered “raw” images related to the given concept fromthe World Wide Web. The “raw” image always includes
many irrelevant images, the ratio of which is 50% or moreon average.
Next, we carry out HTML analysis and select “A-ranked”
images which are very likely to be relevant. Here, we regard
images whose ﬁle name, ALT tag or link word includes a certain keyword “X” as “A-ranked” ones [12]. A-ranked images
are relevant to the keyword “X” with around 75% precision
on average. Using them as training images, we employ aprobabilistic method to select only relevant images from all
the A-ranked images 25% of which are irrelevant. This is
why our method is unsupervised although we use a probabilistic learning method which requires training images.
In our probabilistic learning framework, we allow training data to include some irrelevant data and we can remove
them by repeating both estimation of a model and selection of relevant regions of images from all the regions of raw
images. We use a generative model based on the Gaussian
mixture model to represent models associated to keywords,
and estimate models with the EM algorithm. After estimating the model, we “recognize” relevant region out of all
regions in all the A-ranked Web images with the model. We
repeat this model estimation and region selection. Afterthe second iteration, we use regions selected in the previous
iteration as training data for estimating a model.
After obtaining a model, we apply the learned model toimage classiﬁcation. We classify a test image into one of thegiven class so that the probability of “X” given the image isthe largest.

2.1

Segmentation and Image Feature Extraction

To extract image features from each region, we carry out
the region segmentation in advance. In the experiments, weused JSEG [2]. After segmentation, we extract image features from each region whose size is larger than a certain
threshold. As image features, we prepare three kinds of features: color, texture and shape features, which include theaverage RGB value and its variance, the average response to
the diﬀerence of 4 diﬀerent combination of 2 Gaussian ﬁlters,
region size, location, the ﬁrst moment and the area divided
by the square of the outer boundary length. An image feature vector we use in this paper is totally 24-dimension. We
need to do such pre-processing for all the test images to beclassiﬁed as well as all of the Web images.

2.2 Training a Model from Imperfect Training
Data
As a method to select images, we adopt a probabilistic
method with a Gaussian mixture model. This approach isbased on the method for learning to label image regions fromimages with associated text without the correspondence between words and images regions [3, 1]. That method uses
a mixture of multi-modal components, each combining a

Web searchengines
query

URLs

keyword gather"X"
HTML files
(1)

World
Wide Web(7) repeating
img. files

A

gatherA-rank img.

B

gatherB-rank img.

HTML files

extract andevaluate URL
(2)

(3)

other

(4)
select"X" regions
randomly

loop

(5)

estimate"X" model

Test image sets
(6)

(8)
selectselect"X" regionsapply "X" images
with "X"
"X"
model
model

resultnot gatheredcollecting Web images (HTML analysis)

selecting regions & learning model

detecting"X" images

Figure 1: Processing ﬂow of learning from Web and classifying “X” images.

multinomial for words and a Gaussian over image features.
Here, we simplify things a bit, and build models of the distribution of image features for a given concept for regions
which are obtained by a region segmentation algorithm.
To get a model of regions associated to a certain concept, we need training images. As mentioned before, ourbasic policy is no human intervention, so that we use images which are highly evaluated by the HTML analysis as
training images. Most of such images are relevant ones, butthey always include outliers due to no supervision. Moreover, in general, images usually include backgrounds as well
as objects associated with the given concept. Therefore, weneed to eliminate outlier images and regions unrelated tothe concept such as backgrounds, and pick up only the regions strongly associated with the concept in order to makea model correctly. We use only the regions expected to behighly related to the concept to estimate a model. In our
method, we need negative training images corresponding to“non-X” images in addition to positive training images. We
prepare about one thousand images by fetching them fromthe Web randomly as negative training images in advance.
Our method to ﬁnd regions related to a certain concept
is an iterative algorithm similar to the expectation maximization (EM) algorithm applied to missing value problems.
Initially, we do not know which region is associated with aconcept “X”, since an image with an “X” label just means
the image contain “X” regions. In fact, with the images
gathered from the Web, even an image with an “X” label
sometimes contains no “X” regions at all. So at ﬁrst we haveto ﬁnd regions which are likely associated with “X”. To ﬁnd“X” regions, we also need a model for “X” regions. Here we
adopt a probabilistic generative model, namely a mixture ofGaussian, ﬁtted using the EM algorithm.
In short, we need to know a model for “X” regions andwhich regions are associated with “X” simultaneously. However, each one depends on each other, so we proceed iteratively. Once we know which regions corresponds to “X”,
we can regard images containing “X” regions as ”X” images, and therefore we can compute the probability of an“X” image for each image.

2.3 Detail of the Computation
To obtain P (X|ri ), which represents the probability of
how much the region is associated with the concept “X”,
and some parameters of the Gaussian mixture model, which
represents a generative model of “X” regions, at the sametime, we use an iterative algorithm.
At the initial iteration, we regard A-ranked images which
are expected to be highly relevant to the concept “X” by
HTML analysis as positive training images, and prepare negative training images by gathering images from the Web inadvance. To gather negative training images, we provided
Google Image Search with randomly selected 200 adjective
keywords which have no relation to noun concepts, and collected 4000 negative training images.
Next, we select n “X” regions randomly from A-ranked
images, and select n “non-X” regions randomly from regions
which come from negative training images, respectively. In
the experiment, we set n as 1000.
Taking positive and negative regions together, we apply
the EM algorithm, which is a kind of a probabilistic clustering algorithm, to 2n image feature vectors of the regions
selected from positive and negative initial training images,
and obtain the Gaussian mixture model.
To select positive components and negative components
from all components of the mixture model, we computeP (cj |X) which represents the ratio that the j-th component of the mixture model, cj , contributes to the concept“X” within the obtained GMM, according to the following
formula:P (cj |X)

=

n
X

=

i=1
n
X

P (cj |riX )P (riX )αP (riX |cj )P (cj )P (riX )

i=1

=

α

n
X

X
wj f (riX ; µj , Σj )Si /Σn
i=1 Sii=1(1)where riX is the i-th “X” region, nX is the number of positive
regions, α is a constant for the normalization, and wj is thePweight of j-th Gaussian on the condition of m
j=0 wj = 1.
Xf (ri ; µj , Σj ) is the Gaussian distribution where µj and Σjare the mean vector and the covariant matrix of the j-thcomponent. This function is represented by the following
equation:
1p(2π)N |Σj |f (riX ; µj , Σj ) =

1

X

T

−1

exp− 2 (ri −µj ) Σj(riX −µj )(2)where N is the dimension of the feature vectors. As thesame way, we also compute P (cj |nonX).
Next, we compute pX
j for all components j as follows:pXj=

P (cj |X)
P (cj |X) + P (cj |nonX)(3)

We select components where pX> th1 as positive comj
ponents and components where 1 − pX
j > th1 as negative
components. Positive components and negative components
means Gaussian components associated with the concept“X” and Gaussian components strictly not to associatedwith “X”, respectively. The key point in this component
selection process is that mixing positive samples and negative samples together before applying the EM, and throwing
away neutral components which belongs to neither positive
nor negative components, since neutral components are expected to be associated with image features included in both
positive and negative samples and to be useless for discrimination between “X” and “non-X”. This is diﬀerent fromother work (e.g.[10]) which estimates two GMMs separately
with EM to model positive and negative image concepts.
We regard the mixture of only positive components as an“X” model and the mixture of only negative components as a“non-X” model as. With these models of “X” and “non-X”,
we can compute P (X|ri ) and P (nonX|ri ) for all the regions
extracted from A-ranked images First, we compute P (ri |X)
which is the output of the model of “X’ and P (ri |nonX)
which is the output of the model of “non-X” for each regionri :P (ri |X) =mX
XX
wjX f (riX ; µX
j , Σj )

(4)

j=1
mnonX

P (ri |nonX) =X

wjnonX f (rinonX ; µnonX, ΣnonX
()5)
j
j

j=1

where mX is the number of positive components, wjX is
the weight of j-th positive Gaussian on the condition of
PmX X
X
X
X
j=0 wj = 1, and f (ri ; µj , Σj ) is the Gaussian distribuX
X
tion where µj and Σj are the mean vector and the covariant
matrix of j-th positive component.
Finally, we obtain P (X|ri ) and P (nonX|ri ) with the Bayesian
theorem as follows:
P (X|ri )=P (ri |X)P (X)P (ri |X)P (X) + P (ri |nonX)P (nonX)(6)

For the next iteration, we select the top n regions regarding P (X|ri ) as “X” regions and the top 23 n regions regarding
P (nonX|ri ) as “non-X” regions. In addition, we add 31 n regions randomly selected from negative images gathered fromthe Web in advance to the “non-X” regions. We repeatbuilding models and selecting images for several times. For
every iteration, we use newly selected n positive regions andn negative regions as training data.
The detail on the computation of probability describedabove is basically the same as the method we proposed in[15].

2.4 Applying Trained Models to Classify Test
ImagesTo detect images relevant to a certain concept, we apply the trained models in the same way as region selection
during the training stage, and estimate the probability of“X” (P (X|rj )) for all the regions extracted from all the test
images. Finally, we regard the mean of the probability of“X” of top T regions within each image as the probability of “X” (P (X|Ii )) for each image. This estimation of
P (X|Ii ) is based on the heuristic that an image having regions whose P (X|rj ) are high can be regarded as an “X”
image. Since images usually includes backgrounds as well
as target objects, background regions or unrelated regions
should be ignored for estimating P (X|Ii ). Therefore, we use
not all regions but only several important regions to compute the probability of “X” of images. In the experiment,
we set T as 2. We compute P (X|Ii ) for each test image regarding several kinds of “X” keywords. Finally, we classify
a test image into one of the given classes so that P (X|Ii ) is
the largest.

2.5 Algorithm
To summarize our method we described above, the algorithm is as follows:(1) Carry out region segmentation for all the images andextract image features from each region of each image.
(2) At the ﬁrst iteration, regard A-ranked images as positive
training images which are associated with the concept“X” and images gathered from the Web with non-noun
keywords in advance as negative training images.
(3) Select n “X” regions randomly from positive images,
and select n “non-X” regions randomly from negative
images, respectively (Figure 1 (4)).
(4) Applying the EM algorithm to the image features ofregions which are selected as both positive and negative
regions, compute the Gaussian mixture model for the
distribution of both “X” and “non-X” (Figure 1 (5)).
(5) Find the components of the Gaussian mixture which
contributes “X” regions or “non-X” regions greatly. They
are regarded as “X” components or “non-X” components, and the rest are ignored. The mixture of only
“X” regions is a model of “X” regions, and the mixture

of only “non-X” is a model of “non-X” regions.
(6) Based on the mixture of “X” components and the mixture of “non-X” components, compute P (X|rj ) andP (nonX|rj ) for all the regions which come from “X”
images, where rj is the j-th region.
(7) Select the top n regions in terms of P (X|rj ) as new positive regions and the top 23 n regions in terms of P (nonX|rj ) as new negative regions. Add 31 n regions randomly selected from the negative training images to newnegative regions.
(8) Repeat from (4) to (7) with newly selected positive andnegative regions (Figure 1 (7)).
(9) After repeating several times, apply the trained model
to test data sets. Calculate P (X|rj ) for all the regions
extracted from all test images, and then obtain P (X|Ii )for all the test images (Figure 1 (8)).

3. EXPERIMENTAL RESULTS
In the experiment, we used six keywords, “apple”, “beach”,“ﬂower”, “lion”, “sunset” and “waterfall”. These keywords
was imported from the experiments in [15]. We gathered
1204 A-ranked images from the Web for one keyword on
average.
We prepared 50 test images for each keyword by selecting
images from the Corel Image Gallery based on their attachedkeywords, and we also prepared 50 test images consisting of
Web images which were not included in training data sets.
We carried out four kinds of experiments. First, we used allof the A-ranked images as training images to build models.
Next we used only relevant images selected by hand from the
A-ranked images as training ones. Note that our ﬁnal goal is
to develop methods which can learn from raw Web images
appropriately, although we made the experiment with all
relevant training sets. We used Corel images as test data
sets in the ﬁrst two experiments. Table 3 s hows all the
results of the six-class image classiﬁcation.
The second column shows the precision of the raw Aranked Web images. They were from 67.1% to 87.8% andthe average precision was 76.2% This result followed ourobservation in [12].
The third to ﬁfth columns in Table 1 show the precision,
the recall and the F-measure of the result of image classiﬁcation in case of using the raw A-ranked images as training
images, where the F-measure is the harmonic mean of the
precision and the recall. The average F-measure was 36.7%.
This was not as good as we expected. There are several reasons. The biggest one is that some of Corel images we usedas test images were taken in the situation which was notcommon but too special to be classiﬁed. For example, mostof the Corel images on “ﬂower” (the right side of Figure 2)
are close-up, while most of the Web “ﬂower” images (theleft side of Figure 2) include many ﬂowers in ﬁelds in one
image. In case of “apple” shown in Figure 3, both the Web
images and the Corel images are too various to recognize.

Table 1: Results of the six-class image classiﬁcation. This table includes the precision of the raw Aranked Web images, the precision, the recall and theF-measure of the two kinds of results of image classiﬁcation experiments in case of using raw A-ranked
images and in case of using only relevant images.
class
apple
beach
ﬂower
lion
sunsetwaterfall
AVERAGE

training
image
precision
66.8
68.8
72.2
87.5
67.1
70.9
76.2

raw A-ranked
Corel image
pre. rec. F
36.4 5.7 9.9
29.8 25.8 27.6
39.5 18.1 24.8
55.1 27.3 36.5
34.1 79.4 47.7
42.2 49.5 45.6
39.5 34.3 36.7

only relevantCorel image
pre. rec.
F
27.1 18.6 22.0
60.9 72.2 66.0
38.3 19.1 25.5
100.0 67.7 80.7
42.3 84.5 56.4
36.2 21.2 26.7
50.8 47.2 48.9

Table 2: Results of the six-class image classiﬁcationin case of using other Web images as test data sets.
class
apple
beach
ﬂower
lion
sunsetwaterfall
AVERAGE

training
image
precision
66.8
68.8
72.2
87.5
67.1
70.9
76.2

raw A-ranked
Web imagepre. rec. F
50.0 2.0 3.8
33.3 60.8 43.1
31.9 29.4 30.6
61.8 41.2 49.4
51.3 76.5 61.4
39.2 39.2 39.2
44.6 41.5 43.0

only relevant
Web imagepre. rec.
F
31.7 37.3 34.2
41.5 52.9 46.5
35.0 41.2 37.8
100.0 72.5 84.1
54.7 56.9 55.8
50.0 27.5 35.4
52.2 48.0 50.0

Including too many types of “apple” images caused the lowclassiﬁcation rate. On the other hand, in case of “sunset”
and “lion” both Corel and Web images include many similar
images (Figure 4 and 5).
The sixth to eighth columns in Table 1 show the precision,
the recall and the F-measure of the result of image classiﬁcation in case of using the only relevant images selected outof the raw A-ranked images by hand. For most of the keywords, the F-measure increased compared to the case of the
raw images. This is a reasonable result, since the precisionof training images was 100% in this case.
As additional experiments, we made two experiments using hand-selected Web image sets which includes only relevant images as test data sets. The diﬀerence between two
experiments are whether training sets include irrelevant images. This is the same way as the ﬁrst two experiments.
The results are shown in Table 2. The results are improved
slightly, since the nature of Web test images are similar to
the nature of training data sets obtained from the Web.
As the third experiment, we added four classes (“baby”,“mountain”, “Chinese noodle” and “laptop PC”) which are
not included in the Corel library except “mountain”, andmade ten-class classiﬁcation experiments. These ten classes

exactly correspond to the ten keywords we used in the experiments of Web image gathering in [15]. Since the Corelimage library has no images corresponding to some of theadded classes, we made experiments with only Web images
as test data. In addition, we tried RANSAC (RANdom
SAmpling Cosensus)[7] during the training process to eliminate the eﬀect of irrelevant training images. According to
the RANSAC method, we split the training data into twogroups, one of them is used for building the model, and theother is used for evaluation the built model. We repeatedthis building and evaluating of the models with 30 times.
Finally, we selected the best model the evaluation of which
was the highest among the 30 models.
The experimental results are shown in Table 3. The results are slightly improved due to RANSAC. The diﬀerences
are 3.7% in case of raw images and 0.6% in case of all relevant images as training data. In the latter case, irrelevant
images are already removed, so the amount of improvement
due to RANSAC is very small.

Table 3: Results of the ten-class classiﬁcation withand without RANSAC.
methodraw A-ranked
only relevant
pre. rec.
F
pre. rec.
F
normal25.7 23.4 24.5 28.5 37.7 32.5
RANSAC 26.9 29.7 28.2 29.0 38.5 33.1

4. CONCLUSIONSIn this paper, we describe experiments on image classiﬁcation using images gathered from the Web automaticallyas training images. We proposed the method to learn modelfrom imperfect training image data by modifying the model
employed in the Web image gathering process and to apply it to generic image classiﬁcation task. Such methods
are very important for learning from the Web, since dataon the Web always includes noise. The experimental results
indicated that the method still needed to be improved, but
this framework is one of the promising directions to realize
generic image classiﬁcation/recognition, which is the ﬁnal
goal of our project.
Since this research project is still in the early stage, wehave a lot of what to do. For example, we plan to improve
the probabilistic method and image features. In this paper,
we applied the model of Web image gathering to genericimage classiﬁcation as it was. We are going to modify it for
multi-class classiﬁcation. Recently the use of small parts ofimages as image features is being paid much attention andenables relatively good performance on image detection andclassiﬁcation [4, 6]. We also plan to import this idea into
our method.

5. REFERENCES[1] K. Barnard, P. Duygulu, N. d. Freitas, D. Forsyth,D. Blei, and M. Jordan. Matching words and pictures.
Journal of Machine Learning Research, 3:1107–1135,
2003.
[2] Y. Deng and B. S. Manjunath. Unsupervised
segmentation of color-texture regions in images andvideo. IEEE Transactions on Pattern Analysis andMachine Intelligence, 23(8):800–810, 2001.
[3] P. Duygulu, K. Barnard, J. d. Freitas, and D. Forsyth.
Object recognition as machine translation: Learning alexicon for a ﬁxed image vocabulary. In The Seventh
European Conference on Computer Vision, pages
IV:97–112, 2002.
[4] L. Fei-Fei, R. Fergus, and P. Perona. Learning
generative visual models from few training examples:
an incremental bayesian approach tested on 101 object
categories. In Proc. of IEEE CVPR Workshop of
Generative Model Based Vision, 2004.
[5] R. Fergus, L. Fei-Fei, P. Perona, and A. Zisserman.
Learning object categories from google’s image search.
In Proc. of IEEE International Conference on
Computer Vision, pages 1816–1823, 2005.
[6] R. Fergus, P. Perona, and A. Zisserman. A visual
category ﬁlter for google images. In Proc. of European
Conference on Computer Vision, pages 242–255, 2004.
[7] M. Fischler and R. Bolles. Random sample consensus:
A paradigm for model ﬁtting with application toimage analysis and automated cartography.
Communications of the ACM, 24:381–395, 1981.
[8] J. Li and J. Z. Wang. Automatic linguistic indexing of
pictures by a statistical modeling approach. IEEE
Transactions on Pattern Analysis and MachineIntelligence, 25(9):1–14, 2003.
[9] Y. Mori, H. Takahashi, and R. Oka. Image-to-wordtransformation based on dividing and vector
quantizing images with words. In Proc. of FirstInternational Workshop on Multimedia Intelligent
Storage and Retrieval Management, 1999.
[10] M. R. Naphade, S. Basu, J. R. Smith, C. Y. Lin, and
B. Tseng. Modeling semantic concepts to support
query by keywords in video. In Proc. of IEEE Intl.
Conference on Image Processing, pages I:145–148,
2002.
[11] X.-J. Wang, L. Zhang, F. Jing, and W.-Y. Ma.
Annosearch: Image auto-annotation by search. In
Proc. of IEEE Computer Vision and Pattern
Recognition, pages 1483–1490, 2006.
[12] K. Yanai. Image collector: An image-gathering systemfrom the World-Wide Web employing keyword-based
search engines. In Proc. of IEEE International
Conference on Multimedia and Expo, pages 704–707,
2001.
[13] K. Yanai. Generic image classiﬁcation using visualknowledge on the web. In Proc. of ACM International

Conference Multimedia, pages 67–76, 2003.
[14] K. Yanai. Web image mining toward generic image
recognition. In Proc. of the Twelfth International
World Wide Web Conference, 2003.
[15] K. Yanai and K. Barnard. Probabilistic web image
gathering. In Proc. of ACM SIGMM International
Workshop on Multimedia Information Retrieval, pages
57–64, 2005.

Figure 2: ”Flower” Web images (left) and Corel images (right).

Figure 3: ”Apple” Web images (left) and Corel images (right).

Figure 4: ”Lion” images. Web images are on theleft, and Corel images are on the right.

Figure 5: ”Sunset” Web images (left) and Corel images (right).