ProbSim-Annotation : A novel image annotation algorithm using a probability-based similarity measure

ProbSim-Annotation is an image annotation algorithm for heterogeneous data driven by a probability-based similarity assessment. Image annotation consists of associating to an image a description in terms of labels (or words) from a dictionary. This association rests on the premise that similar images have similar annotations. Evaluation of an annotation algorithm is conveyed by the relevance of retrieval from an image data base when the query is described by labels of the dictionary. Previous studies have shown the probability-based similarity to be very useful in assessing similarity between heterogeneous data by mapping heterogeneous distances into their probability distributions. These can then be estimated from the training set. In this paper, for practical use, the empirical CDF of distances is approximated by polynomial series. Combining probability-based similarity across multiple attributes/dimensions leads to an overall similarity. This serves as an important cue to transfer annotations from the training set to a test set using a kNN algorithm. Experimental results performed on Corel5K benchmark dataset show that ProbSim-Annotation is a promising image annotation algorithm. Introduction One of the important problems in computer based image understanding is mapping low-level physical features of an image into an image description using a higher level language. One can say that the ultimate goal is for the higher level language to coincide with natural language. The goal of this mapping is to achieve an ”understanding” of the image similar to that of a subject looking at the image. Moreover, for practical use these description must be accurate enough to support meaningful retrieval results whenever they are used as queries from an image data set. Interest in image annotation has increased drastically in the wake of on-line information processing and retrieval. However, research in linguistic image understanding goes back several years. Indeed this was the objective of a series of studies on image understanding carried out at the Laboratory for International Fuzzy Engineering Research (LIFE) in Yokohama, Japan, during the years 1991-1995 (see for example (Ralescu 1995) and references therein). Most of the current state-of-the-art models are exploiting multiple image features such as color, texture, and hoping to find a good similarity measurement to combine these heterogeneous features. The probability-based similarity assessment (Le and Ho 2004), (Ralescu, Popovici, and Ralescu 2008), (Popovici 2008), (Fang et al. 2008), consists of two distinct steps: (1) first similarity along each image feature is computed as a probability of certain event in the domain corresponding to that feature; (2) similarities (in fact probabilities) obtained in step (1) are combined across image features. A training set is used to estimate the distribution underlying the domains of each feature. Multiple (often redundant) image features are usually employed to better measure the image similarity in annotation task (Duygulu et al. 2002) (Makadia, Pavlovic, and Kumar 2008). Similarity assessment is one of the most critical parts in image annotation. Its appeal for image annotation rests on the following assumptions: • Similar images are more likely to share the same keywords. • Multiple and heterogeneous image features are more likely to effectively capture meaningful similarity between two images. Standard distance combining technique L1, L2 distances are widely used as distance measures. However, for multi-dimensional data, the heterogeneity problem is more critical since we are willing to combine data channels that are from totally different sources, e.g. different image features. Two standard distance combining techniques are reported by (Makadia, Pavlovic, and Kumar 2008). Since these are used for comparison purposes with the proximity measure proposed here, a review of these distances follows. Joint Equal Contribution(JEC)(Duygulu et al. 2002) Each feature contributes equally towards the image distance, and JEC scales each feature distance by their empirically estimated upper and lower bounds into within [0, 1]. Combined distance between image Ii and Ij is denoted as d(i, j), original distance as d(i,j), and scaled distance as d̃ k (i,j). d̃(i,j) = d(i,j) max(d(i,j)) ∈ [0, 1], (1)