Content Based Retrieval of VRML Objects - An Iterative and Interactive Approach

We examine the problem of searching a database of three dimensional objects given in VRML for objects similar to a given ob ject We introduce an algorithm which is both iterative and interactive Rather than base the search solely on geometric feature similarity we propose letting the user in uence future search results by marking some of the results of the current search as relevant or irrelevant thus in dicating personal preferences A novel approach based on SVM is used for the adaptation of the distance measure consistently with these mark ings which brings the relevant objects closer and pushes the irrelevant objects farther We show that in practice very few iterations are needed for the system to converge well on what the user had in mind Introduction The problem of automatically comparing objects and detecting which ones are alike is a di cult problem After all similarity is in the eye of the beholder Objects similarity is a subjective matter dependent on the human viewer since objects have semantics and are not only geometric entities Does a parasaurolo phus Figure look more like a kangaroo or more like an allosaurus A Parasaurolophus A Kangaroo An Allosaurus Fig Which one is more alike Finding similarity between geometric objects has been a lively topic of re search in computational geometry Objects are considered similar if their geometric features are close given a metric measuring the distance A lot of work has been done matching sets of points in two dimensions under var ious transformation sets The Hausdor distance has been frequently used to measure the distance between point sets Polygonal shapes have also been considered Algorithms for computing the minimum Hausdor distance between polygons were proposed by Hausdor distance has some limitations as it is not robust for outliers Compar ing polygons as turning functions is proposed in In the Frechet distance is used to compare polygons In a re ection metric is de ned and computed for two unions of line segments Less work has been invested in the three dimensional case In particular extending methods of comparing polygonal curves in two dimensions to higher dimensions is non trivial both in theory and in practice However as VRML objects are becoming more popular on the World Wide Web this problem is expected to have many applications one prominent example being in e commerce Much of the research on similarity has been done in the context of images On one hand nding similarities between three dimensional models seems like it should be easier than nding similarities between their projections to images After all the whole object can be seen thus occlusions self occlusions lighting e ects and re ections are avoided On the other hand three dimensional models can be harder to handle since they do not have a simple parameterization and registration and feature correspondence are more di cult to nd In this paper we focus on nding similarities between three dimensional ge ometric objects Rather than dealing solely with the choice of features and the de nition of a distance measure on these features we want to give the user the added ability of in uencing the search as it is being conducted by means of rele vance feedback We suggest an iterative method where each user can specify how relevant the results are according to this user s preferences The scheme we propose is as follows Given an object the system searches a database for similar ones Once a set of results is obtained the user is presented with the best most similar objects and is given a chance to mark a subset of them as relevant or as irrelevant Using this feedback from the user the dis tance function is updated and another iteration of the search may be conducted A new set of results of the updated search is presented to the user to mark and re iterate the process if need be Thus with the same database and the same object each user may get di erent objects as the closest to the chosen one Figure illustrates the intent of the algorithm as well as some results ob tained by the system implementing it In Figure a an initial set of the top thirty similar objects is presented when a search for gures similar to man is conducted on a database of more than a VRML objects After the user marked a subset of the results as relevant or irrelevant and a couple of itera tions were run the nal set of results is produced and shown in Figure b The latter set of results can be contrasted with the initial set Not only more human gures were found among the top thirty but also their ranking among the top thirty has been improved Moreover gures which di er geometrically such as allien where the legs are open and man where one leg is folded and an arm is pointing forward were selected and re ect sematic similarity a Initial results b Final results Fig Searching the database for objects similar to man Three main issues are tackled in this paper The rst issue is the choice of features The goal is to have a representation which is compact su ces to uniquely identify each object and re ects similarities and dissimilarities be tween objects Such features can be considered as a signature associated with each three dimensional object just as a few keywords are associated with each document and are used when searching large databases or the web The second main issue is the choice of a distance measure which should not only re ect ob ject similarities dissimilarities well but should also be amenable to adaptations to enable re nement of searches The third issue is nding an adaptation rule on the distance measure We will show that the moments of the three dimensional objects surfaces up to some order are a relevant choice for features as only a handful of them are needed to represent the essence of the data We will also show that the weighted Euclidean distance leads to an e cient and e ective adaptation scheme based on the user s feedback Finally based on the user s input we nd the optimal weights for the weighted Euclidean metric in a way which re ects the user s desire and warps the space such that the relevant results are brought closer and the irrelevant results are pushed farther Using ideas based on Support Vector Machine learning algorithms SVM we formulate an optimization problem in which the weights are the unknowns and their optimal values correspond to the maximal margin between the distances The obtained optimization problem is convex QP thus ensuring a unique solution We view the contributions of this paper to be threefold First We propose a novel algorithm for retrieving three dimensional VRML objects an area which is relatively unexplored Second we propose to use moments as features rep resenting three dimensional objects and in particular we show how to use mo ments within a relevance feedback scheme While moments can have problems in pictures due to occlusions and self occlusions they can very well characterize three dimensional models which are given wholly Last but not least we show how to use relevance feedback in the context of SVM We tested the algorithm with a database containing over a objects given in VRML Simulations of the search mechanism exhibit very promising results With very few iterations usually the search process converges on what sample users had in mind for these searches The rest of this paper is organized as follows Section describes the object features we use Section describes the similarity measure and the actual it erative search process In Section we show some runs of the algorithm on a database of three dimensional VRML objects and explain the workings of the algorithm by way of demonstration Finally Section concludes the paper Object Features We assume an object is given in VRML i e it is a three dimensional object represented by a set of vertices and a set of polygonal faces embedded in three dimensions The features we choose to represent the objects are the moments computed for object surfaces assuming all objects are hollow For object D we denote the surface by D D s p q r th moment is the given by mpqr Z D xyzdxdydz The set of moments fmpqrg have a property of fundamental importance they uniquely determine and are uniquely determined by the object Thus the triple sequence of moments constitutes a full and complete object description and a partial object description can be obtained by using some subset of them Sampling to Approximate the Moments At the crux of our algorithm lies the computation of a subset of the p q r moments of each object which are used as the feature set Thus we perform a pre processing stage where the features are calculated for each database object A practical way to evaluate the integral de ning moments is to compute it analytically for each facet of the object and then sum over all the facets We use an alternative approach yielding an approximation of the moments The algorithm draws a sequence of points x y z distributed uniformly over the object s surface The number of points drawn from each of the object s facets is proportional to its relative surface area If we denote the list of points for a given object by fxi yi zig N i the p q r th moment is then approximated by mpqr N N X i x p i y q i z r i Normalizing the Objects We want the similarity measure to be invariant to spatial position scale and rotation of the di erent objects We therefore need to normalize the feature vectors of all objects The rst moments m m and m represent the object s center of mass Thus the normalization starts by estimating the rst moments for each object represented as a set of surface sample points and subtracting them from each of these points i N xi yi zi T xi m yi m zi m T This amounts to positioning all objects so that their center of mass is at coordi nates thus removing any dependence on translation or spatial position This also sets each of m m and m to for all objects and thus renders them useless for further computations The second moments m m m m m and m repre sent th