Eurographics Workshop on 3D Object Retrieval (2012), pp. 1–9
M. Spagnuolo, M. Bronstein, A. Bronstein, and A. Ferreira (Editors)

SHREC’12 Track: Sketch-Based 3D Shape Retrieval
B. Li1,8 , A. Godil1 , T. Schreck2 , M. Alexa3 , T. Boubekeur4 , B. Bustos5 , J. Chen6 ,M. Eitz3 , T. Furuya7 , K. Hildebrand3 , S. Huang6 , H. Johan8 , A. Kuijper9 , R. Ohbuchi6 ,R. Richter3 , J. M. Saavedra5 , M. Sherer10 , T. Yanagimachi6 , G. J. Yoon11 , S. M. Yoon12
1 National Institute of Standards and Technology, Gaithersburg, USA 2 Computer and Information Science, University of Konstanz, Germany
3 TU Berlin, Germany 4 Telecom ParisTech/CNRS, France 5 Department of Computer Science, University of Chile, Chile
6 University of Yamanashi, Yamanashi, Japan 7 Nisca Corp., Yamanashi, Japan
8 School of Computer Engineering, Nanyang Technological University, Singapore 9 Fraunhofer IGD, Germany
10 GRIS, TU Darmstadt, Germany 11 National Institute for Mathematical Science, Korea
12 Yonsei Institute of Convergence Technology, Yonsei University, Korea

Abstract
Sketch-based 3D shape retrieval has become an important research topic in content-based 3D object retrieval. The
aim of this track is to measure and compare the performance of sketch-based 3D shape retrieval methods implemented by different participants over the world. The track is based on a new sketch-based 3D shape benchmark,which contains two types of sketch queries and two versions of target 3D models. In this track, 7 runs have been
submitted by 5 groups and their retrieval accuracies were evaluated using 7 commonly used retrieval performance
metrics. We hope that the benchmark, its corresponding evaluation code, and the comparative evaluation resultsof the state-of-the-art sketch-based 3D model retrieval algorithms will contribute to the progress of this research
direction for the 3D model retrieval community.
Categories and Subject Descriptors (according to ACM CCS): H.3.3 [Computer Graphics]: Information Systems—
Information Search and Retrieval

1. Introduction
Sketch-based 3D model retrieval is to retrieve 3D models using a 2D sketch as input. This scheme is intuitive and convenient for users to search for relevant 3D models and also important for several applications including sketch-based modeling and sketch-based shape recognition. However, mostexisting 3D model retrieval algorithms target the Queryby-Model framework, that is, using existing 3D models as
queries. Much less research work has been done regarding
the Query-by-Sketch framework. In addition, until now there
is no comprehensive evaluation or comparison for available
sketch-based retrieval algorithms. Considering of this, weorganize this track to foster this challenging research areaby providing a common sketch-based retrieval benchmark
and soliciting retrieval results from current state-of-the-art
retrieval methods for comparison. We will also provide corresponding evaluation code for computing a set of performance metrics similar to those used in the Query-by-Model
retrieval technique.
submitted to Eurographics Workshop on 3D Object Retrieval (2012)

The objective of this track is to evaluate the performanceof different sketch-based 3D model retrieval algorithms using both hand-drawn and standard line drawings sketch
queries on a watertight 3D model dataset. Every participant
will perform the queries and send us their retrieval results.
We will then do the performance assessment.
In this paper, we report the results of five 3D retrieval
algorithms tested in the Sketch-Based 3D Shape Retrieval
track of SHREC 2012, held in conjunction with the fifth Eurographics Workshop on 3D Object Retrieval.
2. Data Collection
2.1. 3D Target Dataset
Our 3D benchmark dataset is built based on the WatertightModel Benchmark (WMB) dataset [VtH07] which has 400
watertight models, divided into 20 classes, with 20 models
each. The 3D target dataset contains two versions: Basic andExtended. The Basic version comprises 13 selected classes

2

B. Li et al / SHREC’12 Track: Sketch-Based 3D Shape Retrieval

from the WMB dataset with each 20 models (in summary,
260 models). In the basic version, all 13 classes are considered relevant for the retrieval challenge. Figure 1 (c) shows
one typical example for each class of the basic benchmark.
The Extended version adds to the basic version all remaining 7 classes of the WMB dataset (each 20 models). These
additional classes, however, are not considered relevant for
the retrieval challenge but added to increase the retrieval difficulty of the basic version. Figure 1 (d) illustrates typicalexamples for these remaining 7 irrelevant classes. The Extended version is utilized to test the robustness performanceof a sketch-based retrieval algorithm.
2.2. 2D Query Set
The 2D query set comprises two subsets, falling into two
different types.
• Hand-drawn sketches We utilize the hand-drawn sketch
data compiled by TU Darmstadt and Fraunhofer IGD[YSSK10]. It contains 250 hand-drawn sketches, divided
into 13 classes. One typical example for each class isshown in Figure 1 (a).
• Standard line drawings We also select 12 relevant
sketches from the Snograss and Vanderwart’s standardline drawings dataset [SV]. Some examples are shownin Figure 1 (b).
In this track, the two subsets will be tested separately.
However, users can also form a query set by combining
these two to form a query set which contains diverse typesof sketches.

(a) Hand-drawn sketches(b) Standard line drawings

3. Evaluation
All the sketches and models are already categorized according to the classification of the WMB dataset, which contains
20 classes, as shown in Figure 1 (c) and (d). They are ant,
teddy, bird, chair, cup, fish, glasses, hand, human, octopus,
airplane, table, plier, spring, armadillo, bust, mechanic, bearing, vase and four legs, respectively.
To have a comprehensive evaluation of the retrieval algorithms, we employ seven commonly adopted performance
metrics in 3D model retrieval technique. They are PrecisionRecall plot (PR), Nearest Neighbor (NN), First Tier (FT ),Second Tier (ST ), E-Measures (E), Discounted Cumulated
Gain (DCG) [SMKF04] and Average Precision (AP). We
also have developed the code [SBR] to compute them.

4. Participants
Five groups have participated in SHREC’12 track on SketchBased 3D Shape Retrieval. Totally, seven rank list results(runs) for different methods have been submitted. The participants and their runs are listed as follows:• BOF-SBR submitted by Mathias Eitz, Ronald Richter,
Tamy Boubekeur, Kristian Hildebrand and Marc Alexa
from TU Berlin, Germany and Telecom ParisTech/CNRS,France• SBR-2D-3D submitted by Bo Li and Henry Johan from
Nanyang Technological University, Singapore• HKO-KASD submitted by Jose M. Saavedra, Benjamin
Bustos, Tobias Schreck and Sang Min Yoon from University of Chile, Chile; University of Konstanz, Germany;and Yonsei University, Korea• Orig_DG1SIFT and Dilated_DG1SIFT submitted by Tomohiro Yanagimachi, Jipeng Chen, Songhua Huang,
Takahiko Furuya and Ryutarou Ohbuchi from University
of Yamanashi and Nisca Corp., Japan• HOG-DTF and HOG-SC submitted by Sang Min Yoon,
Maximilian Scherer, Gang Joon Yoon, Tobias Shreck and
Arjan Kuijper from Yonsei University, Korea; National
Institute for Mathematical Science, Korea; GRIS, TU
Darmstadt, Germany; University of Konstanz, Germany;and Fraunhofer IGD, Germany

5. Methods(c) 13 relevant 3D watertight models classes(d) 7 irrelevant 3D watertight models classes

Figure 1: Typical 3D model and 2D sketch for each class ofYoon et al.’s [YSSK10] benchmark.

5.1. Bag-of-Features Sketch-Based 3D Shape Retrieval,by Mathias Eitz, Ronald Richter, Tamy Boubekeur,
Kristian Hildebrand and Marc Alexa [EHBA10]
The method employs a bag-of-features based approach forsketch-based shape retrieval [EHBA10] and uses nonphotorealistic rendering (NPR) algorithms to extract important feature lines from a mesh. Recent research on such feature lines indicates that people agree on similar lines when
asked to depict a certain model [CGL∗ 08]. Additionally, thesubmitted to Eurographics Workshop on 3D Object Retrieval (2012)

B. Li et al / SHREC’12 Track: Sketch-Based 3D Shape Retrieval

3

resulting in 64-dimensional local descriptor. Each local feature is quadratic and covers 15% of the area of the view image, resulting in a large overlap between all features.
3) Histogram of Visual Words: Each view is represented
using 1000 local features that do not carry any spatial information. A histogram of visual words is learned from asubset of those features using k-means clustering with 1000
clusters, and represent each view by its specific distributionof “visual words” using hard quantization.

Figure 2: System overview of BoF-SBR approach.

set of feature lines generated by recent NPR methods is often sufficient to convey the shape of an object [CSD∗ 09].
Building on those insights, an image-based approach is employed to 3D shape retrieval to exploit the similarity of human sketches and the results of current line drawing algorithms. The system takes a binary sketch of the desiredmodel drawn by a user as the input and compares this sketch
to a set of line drawings automatically generated for each of
the models in the collection, see Figure 2.
5.1.1. Approach
In the approach, the 3D retrieval problem is mapped to asimpler image retrieval problem such that matching a user
sketch to an NPR rendered view of a mesh is needed. However, this is still an extremely challenging problem due to
two reasons:• User sketches are typically extremely abstract with strong
local and global distortions with respect to the original• Retrieval should be fast as we can expect to see a growthin the size of future shape collection and it is roughly similar to the growth we have seen for image collections during the last decade
To account for those challenges, an engine is built upon abag-of-features model. Robustness against deformations aswell as translation invariance are achieved by using quantized local features. Retrieval is fast, as inverted indices are
employed during a query.
1) Generating Views: A set of 102 views per model are
generated by uniformly sampling from the bounding sphereof the model. Each point on the bounding sphere defines acamera position and is used as input for view-dependent line
drawing algorithms. Specifically, occluding and suggestive
contours [DFRS03] are extracted.
2) Local Descriptor: The method relies on a histogramof oriented gradients extracted from small local regions
as previously employed for sketch-based image retrieval[EHBA11]. It uses 4x4 spatial bins and 4 orientational bins,
submitted to Eurographics Workshop on 3D Object Retrieval (2012)

4) Retrieval: To query a collection, the approach computes the histogram of visual words of the query image andreturns those models with most similar views to the query.
This process is accelerated by using inverted indices. As aresult, only views that have at least one visual word in common with the query need to be compared and the query is
extremely quick.
5.2. Sketch-Based 3D Model Retrieval by Incorporating
2D-3D Alignment, by B. Li and H. Johan [LJ12]
The algorithm [LJ12] consists of two stages which are precomputation and retrieval. The retrieval stage is divided into
2D-3D alignment utilizing a 3D model feature named View
Context [LJ10] and 2D-3D matching based on relative shapecontext matching [BMP02]. The 2D-3D alignment step reduces the search space from many densely sampled views to
only a set of candidate views, thus avoiding a directly bruteforce matching between the sketch and many sample views.
Its main idea is as follows: a sample view is replaced withthe sketch and if its new View Context is very similar to theoriginal one, then it is regarded as a candidate view.
1) Feature Extraction: Silhouette and outline feature
views are generated for both 2D sketches and 3D models
to effectively and efficiently measure the differences among
them. Two examples are shown in Figure 3.

Figure 3: The feature views of a 3D teddy model and a 2D
ant standard line drawing sketch. For each row, from left toright: model/sketch, silhouette view; outline view.
2) Feature Distance: A computationally efficient integrated image descriptor named ZFEC is adopted for View
Context computation. It contains a region-based Zernike
moments feature Z for the silhouette view and a contourbased Fourier descriptor feature F for the outline view. Additionally, eccentricity feature E and circularity feature C are
also utilized to extract the geometric feature of the outline

4

B. Li et al / SHREC’12 Track: Sketch-Based 3D Shape Retrieval

view. To more accurately measure the difference between
the sketch and each candidate view, the relative shape context matching method [BMP02] is adopted.
3) Sketch’s View Context Feature Extraction: The integrated image descriptor distances between the sketch andall the base views of the target model are computed andthe resulting distance vector Dk =<d1 ,d2 ,. . . ,dm > is named
sketch’s View Context.
4) 2D-3D Alignment: To align the 2D sketch and a 3Dmodel, some candidate views are shortlisted by keeping acertain percentage (e.g. 20% or 16 sample views for the
track) of the sample views with top View Context similarities as the sketch, in terms of correlation similarity Si ,Si =Dsi · Dk.
Dsi Dk(1)

where, Dsi and Dk are the View Contexts of the ith sample
view Vis of the 3D model and the 2D sketch, respectively.
5) Sketch-Model Distance Computation: Comparing thesketch with every candidate outline view using the relative
shape context matching and regarding the minimum relative shape context distance obtained as the sketch-model distance.

5.3.2. Global Approach: Histogram of Keyshape
Orientations (HKO)
Unlike gradient-based global methods [DT05, SB10], theapproach takes into account the information given by
keyshapes. In this way, it computes a histogram of keyshapeorientations (HKO) made up with the orientation of lines detected previously. It quantizes θ(Li ) ∈ [0, π] (i = 1 . . . n) into
8 bins. In this way, each HKO bin b represents the number oflines with orientation quantized as b, b = 1, . . . , 8. The final
descriptor is the corresponding unitary version of the HKO
descriptor.
For each 3D model, the method chooses the suggestive
contour image with the smallest distance to the input interms of the HKO descriptor. It uses L1 metric (Manhattan
distance) as distance function.
5.3.3. Local Approach: Keyshape Angular Spatial
Descriptor (KASD)
Let LR be a referent keyshape, the approach defines a circular local region around LR . In addition, the local region isdivided in angular partitions (slices). An example of a local
region and its partitions is depicted in Figure 4 (a).

6) Ranking and Output: Sorting all the sketch-model distances between the sketch and the models in an ascendingorder and listing the retrieved models accordingly.
5.3. HKO-KASD: Histogram of Keyshape Orientations- Keyshape Angular Spatial Descriptor, by J. M.
Saavedra, B. Bustos, T. Shreck and S. Yoon[SBSY12]To compare a hand-drawn image with a set of 3D models, themethod transforms each 3D model into a set of projections
that are computed using 14 suggestive contour (SC) images
as specified by Yoon et al. [YSSK10].
The approach [SBSY12] comprises two stages. First, a
global descriptor is used to determine the most appropriate
SC for each 3D model having a query sketch as input. Next,
it uses a local descriptor exploiting both structural and locality information provided by sketches or suggestive contour
images.
5.3.1. Getting Keyshapes
First, let I be an edge map representation of a sketchor SC image. I is represented by a set of strokes I ={S1 , S2 , . . . , SNs }. Second, for getting keyshapes, the method
takes each stroke S to be approximated by a set of straightlines leading to define I = l1 , l2 , . . . , ln , where n is the total number of detected lines or keyshapes. Finally, keyshapes
are classified as horizontal line (H), vertical line (V ), diagonal line with slope 1 (D1 ), or diagonal line with slope -1(D2 ).

(a)(b)

Figure 4: (a) Local region around a referent keyshape. (b)Local descriptor and its 4-bin histogram for each slice.
It proceeds to compute a 4-bin histogram for each partition (see Figure 4 (b)). This histogram represents the
distribution of keyshape types around LR computed foreach edge pixel. Each bin corresponds to a keyshape type
(H,V, D1 , D2 ). The local descriptor is the unitary version of
the juxtaposition of the eight histograms.
For matching a sketch S and a suggestive contour image C,
the method solves an instance of the bipartite graph problem
using the well known Hungarian Method [Kuh10] between
sets of descriptors of the same class belonging to S and C.
The final cost is the average match cost normalized by the
number of matches. It uses as cost function the well known
Manhattan distance.
5.4. Visual Features on Silhouettes for Sketch-Based 3D
Model Retrieval, by T. Yanagimachi, J. Chen, S.
Huang, T. Furuya and R. Ohbuchi [OF09]
The algorithm turns both query sketch and 3D model into silhouette images for image-based comparison. Its processing
submitted to Eurographics Workshop on 3D Object Retrieval (2012)

B. Li et al / SHREC’12 Track: Sketch-Based 3D Shape Retrieval

5

distance values becomes the distance from the sketch to the
3D model. Three distances derived from three features are
combined, after normalization, by using linear combination
to become an overall distance from a sketch to a 3D model.
Note that, unlike the Generic 3D track, the method does notuse distance metric learning based on Manifold Ranking forthis track.
5.5. Sketch-Based 3D Model Retrieval Using Histogramof Oriented Gradient in Diffusion Tensor Fields, byS.M. Yoon, M. Sherer, T. Shreck and A. Kuijper[YSSK10]Figure 5: Overview of the sketch-based 3D model query algorithm.

pipeline (Figure 5) is quite similar to the one employed forthe partial view 3D model retrieval track in SHREC 2009
[OF09]. Most of the code are the same between the two.
However, scale weighting described in [OF09] was not employed for this track.
To make a filled, silhouette-like image from a sketch image with possible gaps in curcumference, the algorithm first
applies dilation. Then, after most of the gaps are closed, area
filling is done to turn majority of sketches into silhouettelike images. Inevitably, some of the sketches are left as
non-silhouette drawings, possibly impacting retrieval performance. Each 3D model in the database is rendered from multiple (i.e., 42) viewpoints into silhouette images.
After both query and database models are turned into a setof silhouette images, their similarities are compared by using
visual features. The method employs the same three features
as the Generic 3D track, that are, set of local features Dense
SIFT (DSIFT) and Grid SIFT (GSIFT), plus a global feature
One SIFT (1SIFT). (Please refer to [FO09] and [LGA∗ 12]
for details on these features and their distance computation
algorithms.) In sketch-based retrieval, a view of 3D model
is compared against a sketch. Thus, for GSIFT and DSIFT,
Bag-of-Features integration is performed per view, to produce feature vector per view. (For 3D-model-to-3D-modelcomparison of Generic 3D track, all the SIFT features from
42 views of a 3D model are integrated into a feature vector
for the 3D model.) To have a reasonably well-populated histogram per view for DSIFT and GSIFT, the number of SIFT
samples per view is increased from about 300 in [LGA∗ 12]
to about 1,200 for this track.
For each of the three features, 42 distances from a sketch
to 42 views of a 3D model are computed. Minimum of the 42
submitted to Eurographics Workshop on 3D Object Retrieval (2012)

A new approach is proposed for content based 3D model
retrieval by hand-drawn sketch images. This approach to retrieve visually similar mesh models from a large databaseconsists of three major steps: (1) suggestive contour renderings from different viewpoints to compare against theuser drawn sketches; (2) descriptor computation by analyzing diffusion tensor fields of suggestive contour images, orthe query sketch respectively; (3) similarity measurement toretrieve the models and the most probable view-point fromwhich a model was sketched.
This approach for 3D model retrieval using hand-drawn
sketch images evaluates the similarity by comparing thequery image to 14 projected views of the model by following
the approach of Yoon et al. [YSSK10]. For each such image,it extracts a histogram of orientation from the corresponding
diffusion tensor field.
1) Suggestive Contour Extraction from Different Viewpoints: To find the most similar features of the user-drawn
sketches, it extracts the Suggestive Contours [DFRS03] toconstruct descriptors from different viewpoints. It closely resembles the way most people sketch three dimensional objects. To be able to compare 3D models and user sketches,it renders the suggestive contour of each model from 14 different, equally spaced viewpoints.
2) Feature Analysis in Diffusion Tensor Fields: To extract a feature vector from each suggestive contour image
and the query image itself, the method analyzes its properties in the space of diffusion tensor fields [YG09].
3) Similarity Measure: The similarity between the query
image Ic and one projected view image of a 3D model Is ,S(Ic ,Is ) is then given by the following equation:

S(Ic , Is ) =Hc · Hs∥Hc ∥ ∥Hs ∥(2)

For user-drawn sketch based 3D model retrieval, themethod projected the 3D model into 14 different viewpoints.
The similarity measure between a query image and a 3Dmodel is determined by extracting the max |S(Ic ,Is )| from the
14 similarity values. In the track, this algorithm is denoted
as HOG-DTF.

6

B. Li et al / SHREC’12 Track: Sketch-Based 3D Shape Retrieval

5.6. Sketch-Based 3D Model Retrieval Using Sparse
Coding, by S.M. Yoon, G. J. Yoon and T. Shreck
The performance of any content-based 3D object retrieval
system crucially depends on the availability of effective descriptors and similarity measures for this kind of data. An
improved approach of Section 5.5 is presented for supporting 3D object retrieval by optimizing the appropriate gradient descriptor using a sparse coding approach, which is denoted as HOG-SC. It performs encompassing experiments,
the results of which show that the retrieval quality outperforms existing state-of-the-art methods and is therefore recommended for sketch-based 3D object retrieval.
1) HOG-DTF Feature Descriptor: The same approach as1) and 2) of Section 5.5.
2) Feature Optimization Using Sparse Coding: Used forfeature optimization, sparse coding, which is well known to
be powerful for retrieving similar 3D objects using a dramatically smaller trained dictionary, is regarded as a suitable
technique for optimally representing an input HOG-DTF interms of a linear combination of items in an overcomplete
trained dictionary of basis vectors, with sparse coefficientsthat are sufficient for preserving specific features. Sparse
coding is the method of finding the optimal representationof input data using a linear combination of an overcomplete
trained dictionary basis with sparse coefficients for extracting or preserving specific features [CDS99,DE]. Sparse coding has become considerably popular to save or retrieve observed data using a dramatically small quantity of the preassigned dictionary of basis vectors (which consists of featuredescriptors in some contexts). It uses the sparse coding algorithm proposed by Lee et al. [LBRN07].
6. Results
In this section, we perform a comparative evaluation of theresults of the 7 runs submitted by the 5 groups. To have acomprehensive comparison, we measure the retrieval performance based on the 7 metrics mentioned in Section 3: PR,
NN, FT , ST , E, DCG and AP.
As described in Section 2, there are two versions of target dataset: Basic and Extended. There are also two setsof sketch queries: hand-drawn sketches and standard line
drawings, which will be tested separately. Thus, there are
four combinations: (1) Hand-drawn sketch queries and Basicversion of target dataset; (2) Standard line drawing queries
and Basic version of target dataset; (3) Hand-drawn sketch
queries and Extended version of target dataset; (4) Standardline drawing queries and Extended version of target dataset.
Comparisons of the participating methods for the above fourcases are respectively shown in Figure 6 and Table 1; Figure 7 and Table 2; Figure 8 and Table 3; and Figure 9 andTable 4. For some cases, results are not provided for certain
group(s), while each group submit results for the first case,which is the most preliminary experiment.

First, we start with the overall performance evaluation. As
shown in the aforementioned figures and tables, Li’s SBR2D-3D apparently performs the best, followed by Eitz’s
BOF-SBR. While, the performances of other remained threegroups are comparable and the disparity among them is relatively small.
Second, we look into different types of queries. Compared to hand-drawn sketches queries, standard line drawing
queries usually achieve superior performance. One reason isthat the number of the standard line drawings for each classis limited, which is because of the essential property of thistype of queries: standardization. Thus it does not have the diversity issues of the hand-drawn sketch queries. On the other
hand, the standard line drawings also posses most importantsalient features of a typical class of object, which also explains its relatively higher retrieval accuracy.
Last but not least, we evaluate the robustness of different
methods based on the provided results. Table 5 lists the percentage of performance decrease based on the hand-drawn
sketch queries and the two versions of target dataset. Similarly, we can find that Li’s SBR-2D-3D is the most robust,
followed by Eitz’s BOF-SBR. Compared to these two approaches, other three methods show more decrease in the
retrieval performance when adding the irrelevant models to
the target dataset. The conclusion also applies on the standard line drawing sketch queries.
In addition, we classify the five participants into different types based on different standards. Three groups (Eitz,
Saavedra and Yoon) utilize suggestive contours to extract 3Dmodel features. Two groups (Etiz and Yanagimachi) adopt
the Bag-of-Words framework. Three groups (Eitz, Saavedra and Yanagimachi) employ local features while othertwo groups (Li and Yoon) perform global feature matching. Therefore, diverse approaches can be employed forthe sketch-based 3D shape retrieval task and shape illustration techniques, Bag-of-Words approach and local/global
shape matching methods are often utilized and promising inachieving better performance.
7. Conclusions
In this paper, we first present the motivation of the organization of this sketch-based 3D shape retrieval track. Then,
we introduce our approach to build the benchmark including both target 3D models and 2D sketch queries. Next,
we briefly introduce our evaluation method, followed by theshort descriptions of the 6 methods (7 runs) submitted by
5 groups. Finally, a comprehensively comparative evaluation has been conducted in terms of accuracy, robustness
and query types. Based on all the above comparisons, Li’s
SBR-2D-3D method performs the best, followed by Eitz’s
BOF-SBR approach while broadly speaking the other threegroups are comparable in terms of overall performance.
This sketch-based retrieval track is the first attempt to insubmitted to Eurographics Workshop on 3D Object Retrieval (2012)

B. Li et al / SHREC’12 Track: Sketch-Based 3D Shape Retrieval

7

Table 1: Other Performance metrics for the performance comparison on the Hand-drawn sketch queries and Basic version oftarget dataset
Participant

Method

NN

FT

ST

E

DCG

AP

Eitz

BOF-SBR

0.532

0.339

0.497

0.338

0.662

0.450

Li

SBR-2D-3D

0.688

0.415

0.581

0.411

0.731

0.556

Saavedra

HKO-KASD

0.248

0.150

0.258

0.166

0.503

0.254

Yanagimachi
Yanagimachi

Orig_DG1SIFT
Dilated_DG1SIFT

0.172
0.212

0.152
0.168

0.253
0.276

0.167
0.183

0.490
0.503

0.290
0.302

Yoon
Yoon

HOG-DTF
HOG-SC

0.220
0.312

0.167
0.215

0.286
0.335

0.182
0.225

0.513
0.554

0.292
0.331

Table 2: Other Performance metrics for the performance comparison on the Standard line drawing queries and Basic versionof target dataset
Participant

Method

NN

FT

ST

E

DCG

AP

Li

SBR-2D-3D

0.750

0.542

0.700

0.516

0.807

0.675

Saavedra

HKO-KASD

0.083

0.146

0.263

0.164

0.463

0.243

Yanagimachi
Yanagimachi

Orig_DG1SIFT
Dilated_DG1SIFT

0.250
0.333

0.192
0.225

0.321
0.363

0.205
0.240

0.518
0.567

0.328
0.373

Yoon
Yoon

HOG-DTF
HOG-SC

0.417
0.583

0.192
0.183

0.313
0.242

0.212
0.170

0.549
0.551

0.335
0.307

1

1

Eitz(BOF−SBR)
Li(SBR−2D−3D)
Saavedra(HKO−KASD)
Yanagimachi(Orig_DG1SIFT)
Yanagimachi(Dilated_DG1SIFT)
Yoon(HOG−DTF)
Yoon(HOG−SC)

0.9
0.8

0.8
0.7

0.6

0.6

Precision

Precision

0.7

0.5
0.4

0.5
0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

Li(SBR−2D−3D)
Saavedra(HKO−KASD)
Yanagimachi(Orig_DG1SIFT)
Yanagimachi(Dilated_DG1SIFT)
Yoon(HOG−DTF)
Yoon(HOG−SC)

0.9

0

0.2

0.4

0.6

0.8

1

Recall

0

0

0.2

0.4

0.6

0.8

1

RecallFigure 6: Precision-Recall plot performance comparison on
the Hand-drawn sketch queries and Basic version of target
dataset.

Figure 7: Precision-Recall plot performance comparison on
the Standard line drawing queries and Basic version of target dataset.

clude this topic in SHREC in order to foster this challenging
and interesting research direction. Even though it is the firsttime, we already have 5 groups who have successfully partic-

ipated into the newly proposed track. In fact, the most important is that we provide a common platform (the benchmark)to solicit current latest sketch-based 3D model retrieval ap-

submitted to Eurographics Workshop on 3D Object Retrieval (2012)

8

B. Li et al / SHREC’12 Track: Sketch-Based 3D Shape Retrieval

Table 3: Other Performance metrics for the performance comparison on the Hand-drawn sketch queries and Extended versionof target dataset
Participant

Method

NN

FT

ST

E

DCG

AP

Eitz

BOF-SBR

0.460

0.278

0.412

0.281

0.614

0.383

Li

SBR-2D-3D

0.628

0.371

0.520

0.364

0.692

0.498

Yanagimachi
Yanagimachi

Orig_DG1SIFT
Dilated_DG1SIFT

0.100
0.168

0.092
0.120

0.158
0.212

0.100
0.137

0.426
0.462

0.224
0.254

Table 4: Other Performance metrics for the performance comparison on the Standard line drawing queries and Extended
version of target dataset
Participant

Method

NN

FT

ST

E

DCG

AP

Li

SBR-2D-3D

0.750

0.454

0.625

0.442

0.750

0.574

Yanagimachi
Yanagimachi

Orig_DG1SIFT
Dilated_DG1SIFT

0.083
0.167

0.100
0.133

0.163
0.229

0.106
0.144

0.426
0.465

0.224
0.264

1

1
Eitz(BOF−SBR)
Li(SBR−2D−3D)
Yanagimachi(Orig_DG1SIFT)
Yanagimachi(Dilated_DG1SIFT)

0.9

0.8

0.7

0.7

0.6

0.6

Precision

Precision

0.8

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

0

0

0.2

0.4

0.6

0.8

Li(SBR−2D−3D)
Yanagimachi(Orig_DG1SIFT)
Yanagimachi(Dilated_DG1SIFT)

0.9

1

0

0.2

0.4

0.6

0.8

1

RecallRecallFigure 8: Precision-Recall plot performance comparison on
the Hand-drawn sketch queries and Extended version of target dataset.

Figure 9: Precision-Recall plot performance comparison on
the Standard line drawings queries and Extended version oftarget dataset.

proaches to help us to identify the current research progressand existing status that can be reached in terms of retrieval
performance. We also hope that the sketch retrieval benchmark together with the evaluation code will become a goodreference for researchers in this community.

jper (Fraunhofer IGD) who collected the TU Darmstadt and
Fraunhofer IGD sketch data.
We would also like to thank Daniela Giorgi who built theWatertight Shape Benchmark for SHREC 2007.
We would like to thank Snograss and Vanderwart who
built the standard line drawings dataset.

Acknowledgments
This work by Bo Li and Afzal Godil from NIST, were supported by the Shape Metrology IMS.
We would like to thank Sang Min Yoon (Yonsei University, Korea), Maximilian Scherer (TU Darmstadt, Germany),
Tobias Schreck (University of Konstanz) and Arjan Kui-

References[BMP02] B ELONGIE S., M ALIK J., P UZICHA J.: Shape matching and object recognition using shape contexts. IEEE Trans.
Pattern Anal. Mach. Intell. 24, 4 (2002), 509–522. 3, 4
[CDS99]C HEN S., D ONOHO D., S AUNDERS M.: Atomic desubmitted to Eurographics Workshop on 3D Object Retrieval (2012)

B. Li et al / SHREC’12 Track: Sketch-Based 3D Shape Retrieval

9Table 5: Robustness performance comparison in terms of performance decrease (%) on the Hand-drawn sketch queries.
Participant

Method

NN

FT

ST

E

DCG

AP

Eitz

BOF-SBR

13.5

18.0

17.1

16.9

7.3

14.9

Li

SBR-2D-3D

8.7

10.6

10.5

11.4

5.3

10.4

Saavedra

HKO-KASD

53.2

40.0

36.8

38.6

14.5

22.0

Yanagimachi
Yanagimachi

Orig_DG1SIFT
Dilated_DG1SIFT

41.9
20.8

39.5
28.6

37.5
23.2

40.1
25.1

13.1
8.2

22.8
15.9

composition by basis pursuit. SIAM journal on scientific computing 20, 1 (1999), 33–61. 6[CGL∗ 08] C OLE F., G OLOVINSKIY A., L IMPAECHER A.,
BARROS H. S., F INKELSTEIN A., F UNKHOUSER T. A.,
RUSINKIEWICZ S.: Where do people draw lines? ACM Trans.
Graph. 27, 3 (2008), 1–11. 2[CSD∗ 09] C OLE F., S ANIK K., D E C ARLO D., F INKELSTEIN
A., F UNKHOUSER T. A., RUSINKIEWICZ S., S INGH M.: How
well do line drawings depict shape? ACM Trans. Graph. 28, 3(2009), 1–9. 3[DE] D ONOHO D., E LAD M.: Optimally sparse representation in
general (nonorthogonal) dictionaries via ℓ1 minimization. Proc.
Nat. Aca. Sci. 6[DFRS03] D E C ARLO D., F INKELSTEIN A., RUSINKIEWICZ S.,S ANTELLA A.: Suggestive contours for conveying shape. ACM
Trans. Graph. 22, 3 (2003), 848–855. 3, 5
[DT05] DALAL N., T RIGGS B.: Histograms of oriented gradientsfor human detection. In CVPR (2005), pp. 886–893. 4[EHBA10] E ITZ M., H ILDEBRAND K., B OUBEKEUR T.,
A LEXA M.: Sketch-based 3D shape retrieval. In SIGGRAPH
Talks (2010). 2[EHBA11] E ITZ M., H ILDEBRAND K., B OUBEKEUR T.,
A LEXA M.: Sketch-based image retrieval: Benchmark and bagof-features descriptors. IEEE Trans. Vis. Comput. Graph. 17, 11(2011), 1624–1636. 3[FO09] F URUYA T., O HBUCHI R.: Dense sampling and fast encoding for 3D model retrieval using bag-of-visual features. In
CIVR (2009). 5[Kuh10] K UHN H. W.: The Hungarian method for the assignment
problem. In 50 Years of Integer Programming 1958-2008. 2010,
pp. 29–47. 4[LBRN07] L EE H., BATTLE A., R AINA R., N G A. Y.: Efficientsparse coding algorithms. In Advances in Neural Information
Processing Systems 19, Schölkopf B., Platt J., Hoffman T., (Eds.).
MIT Press, Cambridge, MA, 2007, pp. 801–808. 6[LGA∗ 12] L I B., G ODIL A., AONO M., BAI X., F URUYA T., L I
L., L ÓPEZ -S ASTRE R., J OHAN H., O HBUCHI R., R EDONDO C ABRERA C., TATSUMA A., YANAGIMACHI T., Z HANG S.:SHREC’12 track: Generic 3D shape retrieval. In 3DOR (2012),
pp. 1–8. 5[LJ10] L I B., J OHAN H.: View context: A 3D model feature for
retrieval. In: S. Boll et al. (eds.): MMM 2010, LNCS, Springer,
Heidelberg 5916 (2010), 185–195. 3[LJ12] L I B., J OHAN H.: Sketch-based 3D model retrieval byincorporating 2D-3D alignment. Multimedia Tools and Applications (2012), 1–23 (online first version). 3
submitted to Eurographics Workshop on 3D Object Retrieval (2012)[OF09] O HBUCHI R., F URUYA T.: Scale-weighted dense bag ofvisual features for 3D model retrieval from a partial view 3Dmodel. In IEEE ICCV 2009 workshop on Search in 3D and Video(S3DV) (2009), pp. 63 –70. 4, 5
[SB10] S AAVEDRA J., B USTOS B.: An improved histogram ofedge local orientations for sketch-based image retrieval. In Pattern Recognition, vol. 6376 of Lec. Notes in Computer Science.
2010, pp. 432–441. 4[SBR]2

http://www.itl.nist.gov/iad/vug/sharp/contest/2012/SBR/.

[SBSY12] S AAVEDRA J., B USTOS B., S HRECK T., YOON
S. M.: Sketch-based 3D model retrieval using keyshapes for
global and local representation. In Eurographics Wokshop on 3D
Object Retrieval (Submitted) (2012). 4[SMKF04] S HILANE P., M IN P., K AZHDAN M. M.,
F UNKHOUSER T. A.: The Princeton shape benchmark. In
SMI (2004), pp. 167–178. 2[SV] S NODGRASS J. G., VANDERWART M.: A standardized
set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental
Pyschology: Human Learning and Memory 6, 2, 174–215. 2[VtH07] V ELTKAMP R. C., TER H AAR F. B.: SHREC 2007 3D
Retrieval Contest. Technical Report UU-CS-2007-015, Department of Information and Computing Sciences, Utrecht University, 2007. 1[YG09] YOON S. M., G RAF H.: Automatic skeleton extraction
and splitting of target objects. In ICIP (2009), pp. 2421–2424. 5[YSSK10] YOON S. M., S CHERER M., S CHRECK T., K UIJPER
A.: Sketch-based 3D model retrieval using diffusion tensor fields
of suggestive contours. In ACM Multimedia (2010), pp. 193–200.
2, 4, 5