Eurographics Workshop on 3D Object Retrieval (2012), pp. 1–8
M. Spagnuolo, M. Bronstein, A. Bronstein, and A. Ferreira (Editors)

SHREC’12 Track: Generic 3D Shape Retrieval
B. Li1,6 , A. Godil1 , M. Aono2 , X. Bai3 , T. Furuya4 , L. Li3 , R. López-Sastre5 ,H. Johan6 , R. Ohbuchi7 , C. Redondo-Cabrera5 , A. Tatsuma2 , T. Yanagimachi7 , S. Zhang3
1 National Institute of Standards and Technology, Gaithersburg, USA 2 Toyohashi University of Technology, Japan
3 Northwestern Polytechnical University, Xi’an, China 4 Nisca Corp., Yamanashi, Japan
5 Department of Signal Theory and Communications, University of Alcalá, Spain
6 School of Computer Engineering, Nanyang Technological University, Singapore 7 University of Yamanashi, Yamanashi, Japan

Abstract
Generic 3D shape retrieval is a fundamental research field of content-based 3D model retrieval. The aim of thistrack is to measure and compare the performance of generic 3D shape retrieval methods implemented by different
participants over the world. The track is based on a new generic 3D shape benchmark, which contains 1200
triangle meshes that are equally classified into 60 categories. In this track, 16 runs have been submitted by 5groups and their retrieval accuracies were evaluated using 7 commonly used performance metrics.
Categories and Subject Descriptors (according to ACM CCS): H.3.3 [Computer Graphics]: Information Systems—
Information Search and Retrieval

1. Introduction

Figure 1: Examples of generic 3D models.

Generic 3D model retrieval is a fundamental research direction in the field of 3D model retrieval. Generic models
include the 3D objects that we often see in our common life.
Some examples are shown in Figure 1. Compared to professional models, generic models usually have more variations,for example, a chair can have diverse shapes and it may havea wheel or not; a table can be round or rectangular; an insect may be able to fly or does not have wings. Thus, it is
non-trivial to classify different models within one class intoa semantic group. On the other hand, generic models represent 3D objects that are utmost important to us and there area lot of needs to retrieve such models.
Several generic 3D shape benchmarks have been built,
such as Princeton Shape Benchmark (PSB) [SMKF04],
National Taiwan University database (NTU) [CTSO03],
Konstanze 3D Model Benchmark (CCCC) [Vra04], NIST
Generic Shape Benchmark (NSB) [FGLW08], SHREC’10
submitted to Eurographics Workshop on 3D Object Retrieval (2012)

Generic 3D Warehouse [PGD∗ 10], and SHREC’11 Generic
3D Benchmark [DGD∗ 11].
However, there are still some facets that can be further
improved. For example, for some databases different classes
have different number of models, which is regarded as abias related to retrieval; for other datasets some important
types of models are not included. In fact, there are apparent
overlapping in terms of classes among the aforementioned
datasets and we can utilize this to merge and select the models in the same class to build a class in a new generic 3D
model dataset. Hence, in order to create a more comprehensive and overcome some shortcomings of previous generic
datasets, we build a new dataset named SHREC’12 Generic
3D Benchmark based on the above mentioned benchmarks.
We also randomly assign the index number of each modeland develop evaluation code specially for the new benchmark. In this paper, we report the results of five 3D retrieval algorithms tested in the generic shape retrieval trackof SHREC 2012, held in conjunction with the fifth Eurographics Workshop on 3D Object Retrieval (EG3DOR’12).

2. Data Collection
The dataset comprises 1200 models, divided into 60 classes,with 20 models each. In order to build a more comprehen-

2B. Li et al. / SHREC’12 Track: Generic 3D Shape Retrieval

Table 1: 60 classes of the SHREC’12 Generic 3D ShapeBenchmark.
Bird
FlyingInsect
ApartmentHouse
BottleHandGunMug
Sword
Monitor
WheelChair
RoundTable
Tree
Monoplane
Motorcycle
BicycleDrum
TruckNonContainer
Truck
Knife
Chess
Door

Fish
Biped
Skyscraper
Cup
SubmachineGun
FloorLamp
Cellphone
Bed
Sofa
Bookshelf
BiplaneRocketCar
Bus
HumanHead
PianoBoard
Violin
Train
City
Face

NonFlyingInsect
Quadruped
SingleHouse
Glasses
GuitarDeskLamp
DeskPhone
NonWheelChair
RectangleTable
HomePlant
Helicopter
Ship
MilitaryVehicle
ClassicPiano
ComputerKeyboard
Spoon
BooksetPlier
ComputerHand

sive generic 3D benchmark with more diversity, we combine
and merge models based on several previous SHREC generic
3D benchmarks. In detail, we create our SHREC’12 Generic
3D Benchmark based on four datasets: SHREC’11 Generic
3D Benchmark [DGD∗ 11], SHREC’10 Generic 3D Warehouse [PGD∗ 10], Princeton Shape Benchmark [SMKF04]
and SHREC’07 Watertight Shape Benchmark [VtH07].
Firstly, we utilize the 1000 models of SHREC’11 Generic
3D Benchmark [DGD∗ 11], which contains the 800 modelsof the NIST Generic Shape Benchmark (NSB) [FGLW08]
and the selected 200 models from the SHREC’10 Generic
3D Warehouse [PGD∗ 10]. Secondly, we select three additional classes from the SHREC’10 Generic 3D Warehouse[PGD∗ 10], which are “Bookset”, “Knife” and “Train”.
Thirdly, we add six new classes from the Princeton ShapeBenchmark (PSB) [SMKF04], and they are “chess”, “city”,“computer”, “door”, “face” and “hand”. To make each classhas 20 models, we also select some models from the Konstanze 3D Model Benchmark (CCCC) [Vra04] and National Taiwan University database (NTU) [CTSO03]. Finally, we also include the “plier” class of the SHREC’07 Watertight Shape Benchmark (WSB) [VtH07] into our dataset.
The file format to represent the 3D models is the ASCII Object File Format (*.off). Table 1 lists all the classes in the
SHREC’12 Generic 3D Benchmark.
3. Evaluation
The participants submit a 1200×1200 distance matrix per
method/run. The matrix gives the pairwise dissimilarity values of all the possible model pairs in the dataset. Using

the dissimilarity matrices provided by the participants, we
perform our evaluations based on seven standard metrics
which are widely used by 3D model retrieval community:
Precision-Recall curve (PR), Nearest Neighbor (NN), FirstTier (FT ), Second-Tier (ST ), E-Measure (E), Discounted
Cumulative Gain (DCG) [SMKF04] and Average Precision(AP).
4. Participants
There are 5 groups who have successfully participated in the
SHREC’12 Generic 3D Shape Retrieval track. In total, they
have submitted 16 dissimilarity matrices. The details about
the participants and their runs are as follows.
• LSD-r02, LSD-r03, LSD-r08 and LSD-sum submitted by
Xiaoliang Bai, Liang Li and Shusheng Zhang from Northwestern Polytechnical University, China• ZFDR submitted by Bo Li and Henry Johan from
Nanyang Technological University, Singapore• 3DSP_L2_200_hik, 3DSP_L2_1000_hik, 3DSP_L3_200_hik, 3DSP_L2_200_chi2 and 3DSP_L2_1000_chi2 submitted by Carolina Redondo-Cabrera and Roberto Javier
López-Sastre from University of Alcalá, Spain• DVD, DVD+DB and DVD+DB+GMR submitted by Atsushi Tatsuma and Masaki Aono from Toyohashi University of Technology, Japan• DSIFT, DGSIFT and DG1SIFT submitted by Tomohiro
Yanagimachi, Takahiko Furuya and Ryutarou Ohbuchi
from University of Yamanashi and Nisca Corp, Japan
5. Methods
5.1. 3D Model Retrieval Using Local ShapeDistributions, by X. Bai, L. LI and S. Zhang
The idea of the proposed method is to perform 3D model
retrieval using the local shape features. The method is inspired by the Bag of Geodesic Histograms (BOGH) algorithm [LGB∗ 11]. However, the method follows a different strategy by suggesting the new Local Shape Distribution(LSD) descriptor as the shape representation for 3D objects.
5.1.1. Local Shape Distribution Descriptor
Let P denotes a surface point of a 3D object. Its rneighborhood is defined as the spherical region centered at
P with the radius r. The LSD descriptor associated to thisregion is a histogram-vector of the Euclidean-distances between P and other surface points within the region. Since all
the points in the r-neighborhood of p have their own contributions to the local shape of the 3D object in this region, andsuch contributions are decreased with the increase of distances between the points and the center of the region, i.e.
point P, each bin of the LSD histogram is Gaussian weighted(σ = 0.3), with an attempt to accurately indicate the shape
distribution in the region.
submitted to Eurographics Workshop on 3D Object Retrieval (2012)

B. Li et al. / SHREC’12 Track: Generic 3D Shape Retrieval

3

5.1.2. Feature Extraction
The proposed method starts feature extraction by randomly
sampling n points on the surface of a 3D object. It assumes
that the scale normalization on the model has been conducted beforehand. For each sample point, the LSD descriptor of its r-neighborhood is computed, which is composedof d bins (d = 32). After that, the k-means algorithm is employed to carry out clustering on the resulting n LSD descriptors. The aim of this step is to select those characteristic
descriptors, i.e. the centers of k clusters, in order to improve
the speed of similarity matching. By this way, the 3D modelis represented by a set of k LSD descriptors. In this track, n
and k are set to 3000 and 200 respectively. The feature extraction process is shown in Figure 2.

Figure 3: ZFDR feature extraction process [LJon].

model. The details about the feature extraction are describedas follows.

Figure 2: Feature extraction process.

5.1.3. Similarity Matching
The similarity matching step of the proposed method is analogous to that of BOGH. Let LQ and LC denote the LSD descriptor sets of a query and a target 3D model respectively.
The Hungarian algorithm [Kuh10] is employed to establish
the correspondence between LQ and LC . The dissimilarity
between two descriptors is measured by the χ2 distance.
5.1.4. Parameters SettingsThree configurations of the proposed method (r = 0.2, 0.3
and 0.8) were chosen to calculate the dissimilarity matrices(LSD-r02, LSD-r03 and LSD-r08). A version (LSD-sum)combining the above resulting matrices under the sum rule
was also presented. Software to compute LSD descriptor isfreely available at [BLZ12].
5.2. Hybrid Shape Descriptor ZFDR, by B. Li and H.
Johan [LJon]
The hybrid shape descriptor ZFDR [LJon], containing bothvisual and geometric information of a 3D model, is composed of four parts: Zernike moments feature, Fourier descriptor feature, Depth information feature and Ray-based
feature. The shape descriptor computation process consistsof two steps: 3D model normalization and feature extraction, as graphically shown in Figure 3. Continuous Principle Component Analysis (CPCA) [Vra04] alignment algorithm is utilized during the normalization step to align the 3D
submitted to Eurographics Workshop on 3D Object Retrieval (2012)

An image descriptor [ZL02], which comprises Zernike
moments and Fourier descriptors, is adopted to represent thefeatures of a silhouette view. These two features characterize the visual information of a 3D model. They are effective
in representing certain types of models (e.g. “sea animal"),while not as effective as depth buffer-based features for someother classes (like “car") [BKS∗ 04]. Therefore, we devise ahybrid shape descriptor by also integrating certain geometric information of a 3D model. Particularly, we integrate thedepth buffer-based feature and ray-based with spherical harmonic representation feature developed by Vranic [Vra04]
into our hybrid shape descriptor.
1) Cube-Based View Sampling: Considering the tradeoff
between the feature extraction time and retrieval efficiency,
the approach samples 13 silhouette views to represent a 3Dmodel by setting cameras at the following locations on a
cube: (1,0,0), (0,1,0), (0,0,1), (1,1,1), (-1,1,1), (-1,-1,1), (1,1,1), (1,0,-1), (0,1,-1), (1,1,0), (0,1,1), (1,0,1), (1,-1,0).
2) Zernike Moments Feature (Z): Zernike moments feature is utilized to extract the region-based features of a silhouette view. For each sample view, the algorithm computes
the Zernike moments [KH90] (up to the 10th order, totally 35 moments). Then, the Zernike moments features areconcatenated orderly according to the order of the view sequence, resulting a 13×35 matrix as the Zernike moments
feature of a 3D model.
3) Fourier Descriptor Feature (F): Centroid distancebased Fourier descriptor [ZL01] is adopted to extract thecontour feature of a silhouette view. The first 10 Fourier coefficients are used as the Fourier descriptor of a view. The
combination of the Fourier descriptors of 13 views forms
a 13×10 matrix as the Fourier descriptor feature of a 3Dmodel.
4) Depth Information Feature (D): This feature is to extract the Fourier features of the six depth buffer images ofa 3D model. First, the six depth views of a 3D model are
rendered, then 2D Fourier Transform is applied on the depth

4

B. Li et al. / SHREC’12 Track: Generic 3D Shape Retrieval

views, and finally 438 Fourier coefficients are employed as
the depth features of a 3D model.
5) Ray-Based Feature (R): First, a set of rays emanating from the center of the model are shoot and based on
the outmost intersections between the rays and model, theray-based feature vector in the spatial domain is extracted.
Then, Spherical Harmonics Transform [KFR03] is appliedon the obtained radial distance feature vector to transformit from the spatial domain to the spectral domain. Finally, a136-dimensional feature vector is obtained to depict the raybased features of a 3D model.
6) Hybrid Shape Descriptor ZFDR Distance Computation: The hybrid shape descriptor ZFDR of a model is acombination of Zernike moments feature Z, Fourier descriptor F, Depth information feature D and Ray-based descriptor
R. First, appropriate distance metrics are assigned to measure the component distances dZ , dF , dD and dR between
two models, then the four component distances are linearly
combined to form the hybrid descriptor distance dZFDR . For
more details about the shape descriptor computation, pleaserefer to [LJon].
5.3. 3D Shape Recognition via 3D Spatial Pyramids, by
C. Redondo-Cabrera and R. López-Sastre

the 3DSP method should be able to capture the spatial distribution of such labels at different scales and locations in aworking volume Ω(0) . Similar to [LSP06], but now in 3D,
it defines a pyramid structure by partitioning Ω(0) into fine
sub-cubes (see Figure 4). For each level l of the pyramid,
the volume of the previous level, Ω(l−1) , is decomposed into
eight sub-cubes, hence a pyramid P(L) of L levels contains
D = 8L sub-cubes.
Once the pyramid P(L) is composed, the method proposes
to characterize each 3D[ shape S by a weighted ensemble]of histograms H(S) = ω0 H 0 (S), ω1 H 1 (S), ..., ωL H L (S) ,where H l (S) is the histogram of the features in the level lof the pyramid. Each H l (S) is obtained by concatenating 8lhistograms computed in all of the 8l sub-cubes for level l.
In order to penalize the future matches (between 3D shapes)
1
found in larger volumes, it defines the weight ωl as 2L−l.
Given two 3D shapes SX and SY , the algorithm firstcomputes their corresponding 3DSP representations H(SX )and H(SY ). In order to compute the dissimilarity ofthese 3D shapes, it proposes two distance functions. First,
it proposes the HIK ∗ , the histogram intersection kernel including a small modification for normalization. The
distance measures the distance between two histograms
D(H(SX ), H(SY ))HIK ∗ as,
D(H(SX ), H(SY ))HIK ∗ =
1−

(1)
∑N
i=1 min(H(SX )i , H(SY )i ),NNmax(∑i=1 H(SX )i , ∑i=1 H(SY )i )where N is the number of components of histograms H(SX )and H(SY ).

Figure 4: 3D Spatial Pyramid Model.
The 3D Spatial Pyramid (3DSP) is a method for 3D shaperecognition inspired by the work [LSP06, KPW∗ 10]. The
approach is shown in Figure 4. It starts from a 3D shapeof the object of interest. Each shape is characterized by aset of 3D SURF local descriptors [KPW∗ 10]. In contrast to
a random or dense coverage of the shape with spin images[JH99], the 3D SURF is equipped with a 3D interest point
detector, which picks out a repeatable and salient set of interest points in the shapes. The local 3D SURF descriptors arecomputed in these points. Then, by following a traditional
Bag of Words approach, it quantizes these 3D descriptors,
into 3D visual words. Finally, each 3D shape can be characterized by a histogram of its 3D visual words.
In the approach, it proposes to adapt the Spatial Pyramid
Matching Kernel [LSP06] to work with 3D shapes. It models a 3D shape by an orderless set of 3D visual words. Thatis, if we define a visual codebook of size K, each 3D feature is associated to a codebook label {1, . . . , K}. However,Second, it proposes to use the χ2 distance. The distance measures the χ2 dissimilarity between two histograms
D(H(SX ), H(SY ))χ2 as,

D(H(SX ), H(SY ))χ2 =

1 N (H(SX )i − H(SY )i )2∑ H(SX )i + H(SY )i . (2)2 i=1For the track, it submitted 5 different runs of the 3DSP
methods, using the following parameters: visual vocabulary
size K = 200, 1000, 2000; pyramid levels L = 0, 1, 2, 3.
5.4. Dense Voxel Spectrum Descriptor and Globally
Enhanced Manifold Ranking, by A. Tatsuma and
M. AonoThe approach proposes a novel 3D shape feature called
Dense Voxel Spectrum Descriptor (DVD) that aims to capture 3D spatial information. 3D spatial information is theinformation that describes how each piece of a 3D geometric shape occupies which location of a 3D volumetric space.
It also proposes a novel Manifold Ranking algorithm thatgrasps both local and global structures in feature space.
submitted to Eurographics Workshop on 3D Object Retrieval (2012)

B. Li et al. / SHREC’12 Track: Generic 3D Shape Retrieval

55.4.2. Globally Enhanced Manifold Ranking

Figure 5: Dense Voxel spectrum Descriptor (DVD).

5.4.1. Dense Voxel Spectrum Descriptor
The overview of how the method defines the proposed DVDfeature is illustrated in Figure 5. Voxels are represented as
2D pixels for simplicity in the figure. The essential idea of
DVD is in that Fourier spectrum related to 3D voxels should
not be computed directly from the entire voxel. Instead it
cuts off voxels by a fixed block size, and applies Fouriertransform to each block in order to obtain DVD feature.
DVD is, however, sensitive to the position, size, and orientation of the 3D object. To circumvent this, the method has
employed a couple of pose normalizations developed by theauthors previously: PointSVD and NormalSVD [TA].
After pose normalization, it computes voxels. To do this,it generates random points on the surface and quantizes theminto voxel space of size 64 × 64 × 64. The non-empty voxel
has a distance from the center of voxel space.
The voxel produced as above is cut off by a fixed block
size with a fixed sliding (overlapping) window, and theFourier spectrum is computed for each fixed block. The
method adopts 32 as the size for a block, and 16 as thesize for a sliding window. It should be noted that the method
only uses lower frequencies of Fourier spectrum in the range
1 ≤ x, y, z ≤ 8, because higher frequencies tend to have morenoises. Finally, the net Fourier spectra are normalized with
L1 norm.

Recently, several methods, originated from Manifold Ranking (MR) [ZWG∗ 04], have been reported to achieve highsearch performance by considering local structures in feature space, and having them reflected to ranking scores[DGD∗ 11]. Considering only local structures in manifold
learning, however, might cause some problems [BM05].
Thus, a new method has been developed for computing ranking scores by taking both local and global structures into account, which is called Globally enhanced Manifold Ranking(GMR).
In MR, a neighborhood graph is first generated together
with all data including a given search query, then affinity matrix W is computed by using Gaussian kernel as
their weights. Subsequently, affinity matrix W is normalized by diagonal matrix Dii = ∑ j Wi j , to obtain a matrix

S = D−1/2W D−1/2 . Given a column vector q with 1s forquery element and 0s otherwise, the ranking function f in
MR is represented as follows:f = (I − αS)−1 q(3)Zhou et al. [ZBS11] also generalized MR by using agraph Laplacian. They derived a ranking function f by using
Green’s function of an iterated graph Laplacian L as below.
f = {(βI + L)−1 }m q

(4)

Ranking scores with MR have relied only on local structures represented by a neighborhood graph. In the research on robust Laplacian Eigenmaps, Roychowdhury et al.
[RG09] demonstrated that it was possible to capture global
structure by using Minimum Spanning Tree (MST). Their
proposed GMR also takes advantage of MST to capture
global structures. In GMR, they first generate a neighborhood graph as well as an MST from all data including a given
search query. They then compute affinity matrices WNN fora neighborhood graph and WMST for an MST, respectively.
Since a composite of graphs can be done by adding affinity
matrices, the graph Laplacian of the GMR can be expressedby a simple weighted addition.

DVD consists of concatenated Fourier spectra assembled from a block of voxels. This definition of DVD results in a high dimensional feature vector. Specifically, withthe above-mentioned adopted sizes, the total dimension of
DVD becomes ((64 − 32)/16 + 1)3 × 83 = 13, 824. The
approach reduces the dimension from 83 = 512 for lower
frequencies down to 20 by applying Principal Component
Analysis (PCA). The total reduced dimension amounts to((64 − 32)/16 + 1)3 × 20 = 540. Normalization with L1
norm is applied to this reduced dimensional feature vector.

LGMR = LNN + λLMST = (DNN −WNN ) + λ(DMST −WMST )(5)
From Equations (4) and (5), the ranking function of GMR isfinally reduced to the following equation.

The method employs a Manhattan distance for dissimilarity of DVD between two 3D objects. In addition to DVD, it
uses a Depth-buffer shape descriptor [TA] to make a composite feature vector.

The method is essentially the one described in [OF10] andan overview of the algorithm is shown in Figure 6. It involves multi-viewpoint rendering, Bag-of-Features (Bag-ofWords) integration of thousands of local visual features per

submitted to Eurographics Workshop on 3D Object Retrieval (2012)

f = {(βI + LGMR )−1 }m q(6)5.5. Sum of Visual Distances for Generic 3D Model
Retrieval, by T. Yanagimachi, T. Furuya, R.
Ohbuchi [OF10]

6

B. Li et al. / SHREC’12 Track: Generic 3D Shape Retrieval

Figure 6: Three distances computed using three visual features are summed, after normalization, to yield an overall
distance.

3D model, distance metric learning, and heterogeneous feature combination. A nice property of such an appearancebased approach is that 3D models in almost any shape representation can be compared.
The method first renders a 3D model into range images
from multiple (in this case 42) viewpoints. It then extracts
visual features from the images (Figure 7) for comparison.
The method extracts two sets of local features, Dense SIFT,
or DSIFT [FO09], and Grid SIFT, or GSIFT [OF10] per
range image. In addition, it extracts a global feature One
SIFT, or 1SIFT, per range image. The three variations of themethods are named after the features involved. For example,
the best performing DG1SIFT include all the three, while
DGSIFT includes DSIFT and GSIFT. A global visual feature is added since this 3D Generic track database appeared
to consist almost entirely of rigid models.

hundreds of SIFT features [Low04]. The DSIFT employs
random and dense sampling pattern with prior to concentratesamples on or near 3D model. The GSIFT employs a simple grid sampling pattern. Thousands to tens of thousands of
DSIFT (or GSIFT) features are integrated into a feature vector per 3D model by using BoF approach. DSIFT or GSIFT
combined with BoF integration assume invariance against
articulation and global deformation. Global feature 1SIFT
samples a range image at its center with a SIFT feature.
According to the experiments for rigid 3D objects [OF10],
1SIFT performed comparably to LFD [CTSO03].
For DSIFT and GSIFT, a distance between a pair of 3Dmodels is given simply by applying a distance metric, e.g.,
L1-norm. For 1SIFT, distance computation is a little bit more
involved. Assume that there is a pair of 3D models, eachhaving k 1SIFT features extracted from k viewpoints. Using
1SIFT, a distance among the two models is the minimum of
distances among all the (k-1)k/2 distances among k 1SIFT
features each. Comparing 3D models using 1SIFT is slower
than using DSIFT or GSIFT with BoF integration.
To gain extra retrieval accuracy, distance metric learning
is applied based on Manifold Ranking (MR) [ZBL∗ 03] toeach of three distances computed from the three visual features. The distance adjustment described in [OF10] is applied prior to MR so that MR is more effective for high dimensional (e.g., 30k dimension) feature per 3D model of
DSIFT and GSIFT. Affinities or rank values resulted fromapplying MR to distances of three features (in the case of
DG1SIFT) are normalized and then combined into an overall affinity value by summation with equal weight.
6. Results
In this section, we perform a comparative evaluation of theresults of the 16 runs submitted by the 5 groups. To have acomprehensive comparison, we measure the retrieval performance based on the 7 metrics mentioned in Section 3: PR,
NN, FT , ST , E, DCG and AP.

(a) Salient SIFT (original (b) Dense SIFT (DSIFT)
SIFT) [Low04])(c) Grid SIFT (GSIFT)(d) One SIFT (1SIFT)

Figure 7: The method combines multiple visual features perview.
Both DSIFT and GSIFT sample each range image with

Figure 8 shows the Precision-Recall performances of all
the 16 runs while Figure 9 compares the best runs of eachgroup. Table 2 lists the other 6 performance metrics of all the
16 runs. As can be seen from Figure 9 and Table 2, Yanagimachi’s DG1SIFT performs the best, followed by Tatsuma’s
DVD+DB+GMR and Li’s ZFDR, in terms of groups.
As can be seen from Figure 8, overall, Li’s ZFDR method
is comparable to Tatsuma’s DVD+DB approach. However,after applying an enhanced Manifold Ranking learning
method, which considers both the local and global structures in feature space, Tatsum et al. achieved an apparent
performance improvement which can be seen by the resulting DVD+DB+GMR method. Compared to DVD+DB,
DVD+DB+GMR has a 6.1% and 15.7% gain in DCG andAP, respectively. In fact, all the three runs proposed by
Yanagimachi et al. also have adopted Manifold Ranking
submitted to Eurographics Workshop on 3D Object Retrieval (2012)

B. Li et al. / SHREC’12 Track: Generic 3D Shape Retrieval

method to improve the retrieval performance. This indicatesthe advantage of employing machine learning approach inthe 3D model retrieval research field.
However, we can see that all the runs except two of
Yanagimachi’s (DGSIFT and DG1SIFT) have the precision of 1 when recall <= 0.05. Based on the submitted
results of the DG1SIFT method, we have found that for
18 classes, especially for “Door”, “SingleHouse”, “Train”,“Tree”, “Truck” and “Violin”, it cannot achieve a precisionof 1 at recall == 0.05. This should be because of the distance adaption and re-ranking during the Manifold Ranking
process in the DG1SIFT algorithm, which can be regardedon disadvantage of this technique. But overall, it apparently
improves the retrieval performance.
Bai(LSD_r02)
Bai(LSD_r03)
Bai(LSD_r08)
Bai(LSD_sum)
Li(ZFDR)
Redondo(3DSP_L2_200_hik)
Redondo(3DSP_L2_1000_hik)
Redondo(3DSP_L3_200_hik)
Redondo(3DSP_L2_200_chi2)
Redondo(3DSP_L2_1000_chi2)
Tatsuma(DVD)
Tatsuma(DVD_DB)
Tatsuma(DVD_DB_GMR)
Yanagimachi(DSIFT)
Yanagimachi(DGSIFT)
Yanagimachi(DG1SIFT)

1

0.9

0.8

0.7

Precision

0.6

0.5

7

(Bai, Redondo and Yanagimachi) have employed a local
shape descriptor, 1 group (Tatsuma) adopts a global shapedescriptor and the last one (Li) uses both global and local features. Similarly, the three groups (Bai, Redondo andYanagimachi) extracting local features have applied the Bagof-Words framework or K-means clustering on the local features, which shows the popularity of the Bag-of-Words technique in dealing with local features.
7. Conclusions
In this paper, we first present the motivation of the organization of this generic 3D shape retrieval track and then introduce the data collection process. Next, we briefly introduce our evaluation method, followed by the short description of the 5 methods (16 runs) submitted by the 5 groups.
Finally, a comprehensive evaluation has been conducted interms of 7 different performance metrics. Based on the comparative evaluation, Yanagimachi’s DG1SIFT method performs best, followed by Tatsuma’s DVD+DB+GMR methodand Li’s ZFDR approach in terms of groups. According to
the track, manifold ranking learning method and Bag-ofWords approach are two popular and promising techniquesin generic 3D shape retrieval, which shows current research
trend in the field of 3D model retrieval.

0.4

References

0.3[BKS∗ 04] B USTOS B., K EIM D. A., S AUPE D., S CHRECK T.,
V RANIC D. V.: Using entropy impurity for improved 3D object
similarity search. In ICME (2004), pp. 1303–1306. 3

0.2

0.1

0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

RecallFigure 8: Precision-Recall plot performance comparison of
all the 16 runs of the 5 groups.

Bai(LSD_sum)
Li(ZFDR)
Redondo(3DSP_L2_1000_hik)
Tatsuma(DVD_DB_GMR)
Yanagimachi(DG1SIFT)

0.8[DGD∗ 11] D UTAGACI H., G ODIL A., DARAS P., A XENOPOU LOS A., L ITOS G. C., M ANOLOPOULOU S., G OTO K.,YANAGIMACHI T., K URITA Y., K AWAMURA S., F URUYA T.,O HBUCHI R.: SHREC ’11 track: Generic shape retrieval. In
3DOR (2011), pp. 65–69. 1, 2, 5

0.7

Precision

0.6

0.5

[FGLW08] FANG R., G ODIL A., L I X., WAGAN A.: A newshape benchmark for 3D object retrieval. In ISVC (1) (2008),
pp. 381–392. 1, 2

0.4

0.3

[FO09] F URUYA T., O HBUCHI R.: Dense sampling and fast encoding for 3D model retrieval using bag-of-visual features. In
CIVR (2009). 6

0.2

0.1

0

[BM05] B ENGIO Y., M ONPERRUS M.: Non-local manifold tangent learning. In Advances in Neural Information Processing
Systems 17 (2005), pp. 129–136. 5[CTSO03] C HEN D.-Y., T IAN X.-P., S HEN Y.-T., O UHYOUNG
M.: On visual similarity based 3D model retrieval. Computer
Graphics Forum 22, 3 (2003), 223–232. 1, 2, 61

0.9

[BLZ12] BAI X., L I L., Z HANG S.:
Software for
3D model retrieval using local shape distributions.
http://code.google.com/p/shape-retrieval, Feb. 2012. 3

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

RecallFigure 9: Precision-Recall plot performance comparison of
the best runs of each group.
In conclusion, among the 5 methods submitted, 3 groups
submitted to Eurographics Workshop on 3D Object Retrieval (2012)[JH99] J OHNSON A., H EBERT M.: Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence 21, 5 (1999),
433–449. 4[KFR03] K AZHDAN M. M., F UNKHOUSER T. A.,
RUSINKIEWICZ S.: Rotation invariant spherical harmonic
representation of 3D shape descriptors. In Symposium on
Geometry Processing (2003), pp. 156–164. 4

8

B. Li et al. / SHREC’12 Track: Generic 3D Shape Retrieval

Table 2: Other Performance metrics for the performance comparison.
Participant

Method

NN

FT

ST

E

DCG

AP

Bai
Bai
Bai
Bai

LSD-r02
LSD-r03
LSD-r08
LSD-sum

0.431
0.474
0.489
0.517

0.179
0.196
0.212
0.232

0.257
0.277
0.301
0.327

0.175
0.190
0.207
0.224

0.510
0.528
0.545
0.565

0.326
0.341
0.359
0.381

Li

ZFDR

0.818

0.491

0.621

0.442

0.776

0.650

RedondoRedondoRedondoRedondoRedondo3DSP_L2_200_hik
3DSP_L2_1000_hik
3DSP_L3_200_hik
3DSP_L2_200_chi2
3DSP_L2_1000_chi2

0.671
0.685
0.708
0.667
0.662

0.349
0.376
0.361
0.341
0.367

0.470
0.502
0.481
0.462
0.496

0.329
0.351
0.335
0.322
0.346

0.669
0.685
0.679
0.662
0.678

0.501
0.526
0.462
0.442
0.468

Tatsuma
Tatsuma
TatsumaDVD
DVD+DB
DVD+DB+GMR

0.790
0.831
0.828

0.459
0.496
0.613

0.588
0.634
0.739

0.416
0.450
0.527

0.756
0.785
0.833

0.621
0.661
0.765

Yanagimachi
Yanagimachi
Yanagimachi

DSIFT
DGSIFT
DG1SIFT

0.863
0.860
0.879

0.639
0.654
0.661

0.787
0.794
0.799

0.562
0.572
0.576

0.857
0.866
0.871

0.792
0.803
0.811

[KH90] K HOTANZAD A., H ONG Y.: Invariant image recognition
by Zernike moments. IEEE Transactions on Pattern Analysis andMachine Intelligence 12, 5 (1990), 489–497. 3[KPW∗ 10] K NOPP J., P RASAD M., W ILLEMS G., T IMOFTE R.,
VAN G OOL L.: Hough transform and 3D SURF for robust three
dimensional classification. In Proceedings of the 11th European
Conference on Computer vision (2010). 4[Kuh10] K UHN H. W.: The Hungarian method for the assignment
problem. In 50 Years of Integer Programming 1958-2008. 2010,
pp. 29–47. 3[LGB∗ 11] L IAN Z., G ODIL A., B USTOS B., DAOUDI M., H ER MANS J., K AWAMURA S., K URITA Y., L AVOUÉ G., N GUYEN
H. V., O HBUCHI R., O HKITA Y., O HISHI Y., P ORIKLI F.,R EUTER M., S IPIRAN I., S MEETS D., S UETENS P., TABIA H.,
VANDERMEULEN D.: SHREC ’11 track: Shape retrieval on nonrigid 3D watertight meshes. In 3DOR (2011), pp. 79–88. 2[LJon] L I B., J OHAN H.: 3D model retrieval using hybrid features and class information. Multimedia Tools and Applications(2011, Online first version), 1–26. 3, 4
[Low04] L OWE D. G.: Distinctive image features from scaleinvariant keypoints. International Journal of Computer Vision60, 2 (2004), 91–110. 6[LSP06] L AZEBNIK S., S CHMID C., P ONCE J.: Beyond bags offeatures: Spatial pyramid matching for recognizing natural scene
categories. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006). 4[OF10] O HBUCHI R., F URUYA T.: Distance metric learning andfeature combination for shape-based 3D model retrieval. In Proceedings of the ACM workshop on 3D object retrieval (2010),
3DOR ’10, ACM, pp. 63–68. 5, 6
[PGD∗ 10] P ORETHI V. T., G ODIL A., D UTAGACI H., F URUYA
T., L IAN Z., O HBUCHI R.: SHREC’10 track: Generic 3D warehouse. In 3DOR (2010), pp. 93–100. 1, 2
[RG09]ROYCHOWDHURY S., G HOSH J.:

eigenmaps using global information. In Proc. of AAAI Fall Symp
on Manifold Learning and Its Applications (2009). 5[SMKF04] S HILANE P., M IN P., K AZHDAN M. M.,
F UNKHOUSER T. A.: The Princeton shape benchmark. In
SMI (2004), pp. 167–178. 1, 2
[TA] TATSUMA A., AONO M.: Multi-Fourier spectra descriptor
and augmentation with spectral clustering for 3D shape retrieval.
Vis. Comput. 25. 5[Vra04] V RANIC D.: 3D Model Retrieval. PhD thesis, Universityof Leipzig, 2004. 1, 2, 3
[VtH07] V ELTKAMP R. C., TER H AAR F. B.: SHREC 2007 3D
Retrieval Contest. Technical Report UU-CS-2007-015, Department of Information and Computing Sciences, Utrecht University, 2007. 2[ZBL∗ 03] Z HOU D., B OUSQUET O., L AL T. N., W ESTON J.,
S CHÖLKOPF B.: Learning with local and global consistency. In
NIPS (2003), Thrun S., Saul L. K., Schölkopf B., (Eds.), MIT
Press. 6[ZBS11] Z HOU X., B ELKIN M., S REBRO N.: An iterated graphlaplacian approach for ranking on manifolds. In Proc. of the 17th
ACM SIGKDD international conference on Knowledge discovery
and data mining (2011), ACM, pp. 877–885. 5[ZL01] Z HANG D., L UO G.: A comparative study on shape retrieval using Fourier Descriptors with different shape signatures.
In Proc. of International Conference on Intelligent Multimedia
and Distance Education (ICIMADE01) (2001), pp. 1–9. 3[ZL02] Z HANG D., L UO G.: An integrated approach to shapebased image retrieval. In Proc. of the 5th Asian Conference on
Computer Vision (ACCV 2002) (2002), pp. 652–657. 3[ZWG∗ 04] Z HOU D., W ESTON J., G RETTON A., B OUSQUETO., S CHÖLKOPF B.: Ranking on data manifolds. In Advances in
Neural Information Processing Systems 16 (2004), MIT Press. 5

Robust Laplacian
submitted to Eurographics Workshop on 3D Object Retrieval (2012)