Eurographics Workshop on 3D Object Retrieval (2016)
A. Ferreira, A. Giachetti, and D. Giorgi (Editors)

SHREC’16 Track: Partial Shape Queries for 3D Object Retrieval
I. Pratikakis1,2† , M.A. Savelonas1,2† , F. Arnaoutoglou2† , G. Ioannakis2† , A. Koutsoudis2† , T. Theoharis3† , M.-T. Tran4 , V.-T. Nguyen5 ,V.-K. Pham4 , H.-D. Nguyen4 , H.-A. Le4 , B.-H. Tran4 , Q.H. To4 , M.-B. Truong4 , T.V. Phan4 , M.-D. Nguyen6 , T.-A. Than6 , K.-N.C. Mac7 ,M.N. Do7 , A-D. Duong8 , T. Furuya9 , R. Ohbuchi9 , M. Aono10 , S. Tashiro10 , D. Pickup11 , X. Sun11 , P.L. Rosin11 , R.R. Martin11
1 Democritus University of Thrace, Department of Electrical and Computer Engineering, Greece
2 ATHENA Research Center, Institute for Language and Speech Processing, Greece
3 Norwegian University of Science and Technology, Department of Computer and Information Science, Norway
4 University of Science, VNU-HCM, Vietnam
5 University of Science and University of Information Technology, VNU-HCM, Vietnam
6 University of Science and John von Neumann Institute, VNU-HCM, Vietnam
7 University of Illinois at Urbana-Champaign, USA
8 University of Information Technology, VNU-HCM, Vietnam
9 Graduate School of Medicine and Engineering, University of Yamanashi, Japan
10 Knowledge Data Engineering Lab, Toyohashi University of Technology, Japan
11 School of Computer Science and Informatics, Cardiff University, UK

Abstract
Despite numerous recent efforts, 3D object retrieval based on partial shape queries remains a challenging problem, far frombeing solved. The problem can be defined as: given a partial view of a shape as query, retrieve all partially similar 3D models
from a repository. The objective of this track is to evaluate the performance of partial 3D object retrieval methods, for partial
shape queries of various scan qualities and degrees of partiality. The retrieval problem is often found in cultural heritage
applications, for which partial scans of objects query a dataset of geometrically distinct classes.
Categories and Subject Descriptors (according to ACM CCS): H.3.2 [Information storage and retrieval]: Information search andretrieval—I.2.10 [Artificial Intelligence]: Vision and Scene Understanding—Shape

1. Introduction
Partial 3D object retrieval (P3DOR) concerns the search of 3Dmodels which are similar to a query, when the available information
for the query is not complete. For each partial query, a partial 3D
object retrieval method is required to return a ranked list of complete objects, which are retrieved from a database and are ranked
according to their similarity with the query. The similarity assessedis partial and can be distinguished from global similarity in that it
implies a matching of only a part of the complete object with thequery.
The interest for partial retrieval algorithms has been significantly
boosted by the wide availability of 3D scanners, as well as byprogress in 3D graphics technologies. This interest has been further amplified by the advent of several application domains, such as
face recognition and digital libraries of cultural heritage (CH) objects, which require partial 3D object retrieval capabilities. In thiscontext, two milestone challenges exist: (i) scanned queries can berough and noisy; (ii) it is not straightforward to effectively match† Track Organisersc 2016 The Author(s)Eurographics Proceedings c 2016 The Eurographics Association.

a partial query against a complete 3D model, since there is a gap
between their representations. This representation gap complicatesthe extraction of a signature that will enable a matching between acomplete 3D model and its partial counterpart.
Methodologies that appear in the literature addressing P3DOR can be roughly classified as: (i) viewbased[SMM∗ 10],
[AMS11],[SPK∗ 14],(ii)
partbased [CB07], [TVD09], [APP∗ 10] (iii) bag of visual words(BoVW)-based [Lav12], [LGJ14], [BBGO11], and (iv) hybrid
methods combining the three main paradigms or methods whichcannot be straightforwardly classified [FO09], [SPS16]. A detailed overview of the corresponding methodologies is presented
in [SPS15], wherein the interested reader may also find the full listof benchmark datasets that have been used in the context of partial
3D object retrieval.
In the SHREC series framework, previous efforts for benchmarking 3D object retrieval with partial queries comprise the SHREC’09
track: Querying with Partial Models [DGA∗ 09] and SHREC’10
track: Range Scan Retrieval [DGC∗ 10], both organised by Dutagaci and Godil as well as SHREC’13 track: Large-scale partial
shape retrieval using simulated range images organised by Sipiran

I. Pratikakis et al. / SHREC’16 Track: Partial Shape Queries for 3D Object Retrieval

et al. [SMB∗ 13]. The typical feature of the aforementioned tracks
is the use of artificial queries.
Unlike the previous benchmarking efforts, this track uses fortesting a partial query dataset which comprises not only artificial
but also real scans with a quality which ranges from low to highwith different degrees of partiality. Moreover, the retrieval problem
that the participants of the track are dealing with, is motivated byan intra-class problem, often found in CH applications, for which
partial scans of potteries query a dataset of geometrically distinct
classes (see Section 2).
The authors of submitted methods registered in the competitionand downloaded the target dataset along with representative samples of partial queries. At a next step, all registered participants
were required to submit the ranking list for each requested query.
After the evaluation of all participating methods, the 3D partial
query dataset is publicly available at the competition’s website:
http://vc.ee.duth.gr/shrec16/.
The remainder of the paper is organized as follows: Section 2
describes the data used for the benchmarking, Section 3 provide acomprehensive description of the participating methods, the produced results are given in Section 4 and finally conclusions are
drawn in Section 5.

Figure 1: Examples of artificial queries: for each example, theassociated complete object from the Hampson dataset (top), alongwith queries of 25% (middle) and 40% (bottom) partial coverage
are illustrated.

2. Dataset
The dataset used for evaluation is related to the CH domain andconsists of 3D pottery models originating from the Virtual Hampson Museum collection [VHM]. It consists of 383 models classified to 6 distinct geometrically defined classes. The partial shapequeries are provided in 3 different forms:(i) artificial queries created by slicing and cap filling of the complete 3D models. 21 artificial queries are provided in two different
degrees of partial coverage: 25% and 40%, respectively (resulting
in a total of 21 × 2 queries). Figure 1 illustrates three examples ofartificial queries. For each example, the associated complete object
from the Hampson dataset, along with the 25% and the 40% queries
are illustrated,(ii) real queries of high quality, obtained with the smartSCAN
Breuckmann scanner. In total, 25 high quality queries are provided,
each obtained from three different views (resulting in a total of
25 × 3 queries). Figure 2 illustrates three examples of high quality
queries. For each example, the associated complete object from theHampson dataset, along with the queries obtained from each of thethree different views, are illustrated,(iii) real queries of low quality, obtained with Microsoft Kinect
V2 sensor. In total, 25 low quality queries are provided, each obtained from three different views (resulting in a total of 25 × 3
queries). Figure 3 illustrates three examples of low quality queries.
For each example, the associated complete object from the Hampson dataset, along with the queries obtained from each of the threedifferent views, are illustrated,
In the case of (ii) and (iii), the queries were created by scanningderivative vessels that have been constructed by a professional potter using the Hampson models as a template (Fig. 4). All query sets

Figure 2: Examples of high quality queries. For each example,
the associated complete object from the Hampson dataset (top),
along with the high quality queries obtained from each of the threedifferent views.

have been created in the context of PRESIOUS-predictive digitization, restoration and degradation assessment of cultural heritage
objects EU-funded project [PRE].
3. Methods
Four (4) research groups have participated in the competition witheight (8) distinct algorithms (each of two participants submitted
three algorithms). Brief descriptions of the methods are given in
the following (the order of appearance is the chronological order ofthe expression of interest in the competition).
c 2016 The Author(s)Eurographics Proceedings c 2016 The Eurographics Association.

I. Pratikakis et al. / SHREC’16 Track: Partial Shape Queries for 3D Object Retrieval

code local features with VLAD, FV, and GOLD. Given a partial
shape query Q, we derive a unique viewpoint v by the following
equation:v=∑ p∈Q a p n p|| ∑ p∈Q a p n p ||(1)

where p is a polygon organizing Q, a p is an area of p, and n p is anormal on p. Next, we perform single-view rendering for the queryand generate a depth buffer image and a face orientation image. We
then extract local features and encode them with VLAD, FV, andGOLD. Power and L2 normalizations are applied to VLAD, FV,
and GOLD vectors. Finally, we compute dissimilarity d between aquery Q and a target 3D object M by the following equation:d = mini ||fQ − fi ||Figure 3: Examples of low quality queries. For each example, theassociated complete object from the Hampson dataset (top), alongwith the low quality queries obtained from each of the three different views.

(2)

where f Q is a feature vector of Q and f i is the i-th feature vector
obtained from M.
In this run, we chose image resolution as 300 × 300, the number of viewpoints as 92 based on geodesic sphere, the dimensionof a KAZE feature as 128, the number of centroids as k = 100,and the number of components of GMM as n = 50, amounting
to 128 × 100 × 2 = 25600 dimension of VLAD, 128 × 50 × 2 × 2= 25600 dimension of FV, and (128 + 128 × (128 + 1)/2) × 2 =16768 dimension of GOLD vectors.
3.2. View-based Feature Matching (D. Pickup, X. Sun, P.L.
Rosin, R.R. Martin)

Figure 4: Derivative vessels have been constructed by a professional potter using the Hampson models as a template.

3.1. View-based Partial 3D Object Retrieval with Feature
Encoding (M. Aono, S. Tashiro)
We have taken a view-based approach to partial shape queries for3D object retrieval, attempting to minimize the dissimilarity between a query and a target 3D model. To compare views, we haveadopted three different combinations of feature-based encodings.
The first one is to encode KAZE [ABD12] features with vector of
locally aggregated descriptor (VLAD) [JDSP10], the second is to
encode KAZE features with Fisher vector (FV) [PS10], whereas thethird is to encode KAZE features with Gaussian of local distribution (GOLD) [SGMC15]. To obtain VLAD, FV, and GOLD vectors
from target 3D models, first we perform multi-view rendering andgenerate multiple depth buffer and face orientation images. A faceorientation image is represented as a collection of inner products ofa view vector and a surface normal. Second, we smooth the images
with Gaussian filter and extract KAZE features as local featuresfrom the smoothed images. Third, we find k centroids and n components of Gaussian mixture model (GMM). In this track, since no
training data is provided, we estimate the centroids and parametersof GMM from the target dataset. After obtaining centroids, we enc 2016 The Author(s)Eurographics Proceedings c 2016 The Eurographics Association.

View-based methods have proved successful in the past at matchingrigid objects. We therefore use a variant of the view-based methodby Lian et al. [LGSX13] for partial matching. Our method assumes
that there is a view of the complete target object, and a view of thepartial query object where the same part is visible in both. To compare a target object to a partial query we render 66 depth imagesof each of them, where the rendered views are uniformly sampled
on a sphere surrounding the respective object (Figure 5(b)). We
then compute SIFT features from each depth image (Figure 5(c)),
and compare every image of the query to every image of the target by computing the mean Euclidean distance between matching
SIFT features [Low04]. The matching SIFT features are found using the vl_ubcmatch function of the VLFeat [VF08] computer vision library, which matches features using the method presented by
Lowe [Low04]. The distance between the most similar pairing ofimages between the query and the target is used as the final distance
between the two objects.
3.3. Partial 3D Object Retrieval with Bag-of-Visual-Words
and ICP (Tran et al.)We submit 7 runs which are shown in Table 1. The submission iscategorized into 3 main types: bag-of-visual words (BoW) only,
ICP only, and BoW-ICP fusion. In the BoW runs (1, 2, 3), we follow the visual object retrieval scheme proposed by J. Sivic [SZ03].
This framework inherits the idea of text retrieval model and has
been successfully applied on visual data such as images and videos.
A standard BoW model includes 6 steps (as illustrated in Figure 6):
feature extraction (detection and description), training codebook,

I. Pratikakis et al. / SHREC’16 Track: Partial Shape Queries for 3D Object Retrieval

Table 1: Run description(a) 3D object

Run
1

Type
BoW

2

BoW

3

BoW

4

ICP

5

BoW+ICP

6

BoW+ICP

7

BoW+ICP(b) Depth im- (c) SIFT feature extraction
ages

Figure 5: For each mesh (a), we compute 66 depth images (b), andextract SIFT features from each depth image (c). The distance between two objects is computed as the distance between the matched
SIFT features of their most similar images.

quantization, tf-idf weighting, building inverted file, and computing distance. There are 5 main parameters: feature detector, featuredescriptor, codebook size, assignment type in quantization, and distance metric. For example, in run 1 we use dense features with 10%
3D points sampled from vertices of each target model. Instead ofdense sampling which is time consuming, we use a sparser featuresuch as ISS [Zho09]. Figure 7 shows the result of feature detection
on a 3D model using ISS detector. To describe feature points formatching, we use an extension of ROPS descriptor [GSB∗ 13] basedon the idea of rootSIFT [AZ12]. We named the similarly transformed ROPS feature as rootROPS. All features extracted from target models are used to build a codebook using the approximate kmeans algorithm. Aiming to facilitate comparisons, we fix the number of codewords to 5, 000. In order to reduce quantization error,
we use soft assignment [PISZ08] with 3 nearest neighbors setting.
Tf-idf weighting and building inverted file are two more standard
methods to enhance both accuracy and efficiency of retrieval. In
this contest, we do not use any other options for these components.
Lastly, instead of using a symmetric distance, we use the asymmetric distance proposed by C. Zhu [ZJS13] to compare a partial querywith each complete target model.

Settings
Dense sampling, Descriptor:
ROPS, Codebook size: 5000,
Quantization:
SoftAssignment,
AsymmetricDistance: L1
Detector: ISS, Descriptor:
ROPS, Codebook size: 5000,
Quantization:
SoftAssignment,
AsymmetricDistance: L1
Detector: ISS, Descriptor: Root
ROPS, Codebook size: 5000,
Quantization:
SoftAssignment,
AsymmetricDistance: L1
Translation and Rotation at Initialization
Artificial queries: Combine
Run and Run 2;
Breuckmann queries: Run 1;
Kinect queries: Run 4
Artificial queries: Combine
Run 1 and Run 2;
Breuckmann queries: Run 4;
Kinect queries: Run 4
Artificial queries: Combine
Run 1 and Run 2;
Breuckmann queries: Run 3;
Kinect queries: Run 4

Figure 7: An example of sparse feature using ISS detector

model in the target dataset. The candidate point clouds are set fixed
while ICP iteratively translates and rotates the query model to best
match the reference. Algorithm convergence is handled with a typical difference-based criterion.
Figure 6: BOW framework for 3D object retrieval
In the Iterative Closest Point algorithm [CM91, BM92](ICP)run (4), we exploit the geometric information of 3D models, byoverlaying each pair of models onto each other and comparing theproximity distance between the underlying point sets. The idea of
ICP is to find a proper rigid transformation that associates eachpoint in the cloud of a query model to a point of each candidate

As the performance of ICP relies on the initial pose and relative
position of 2 structures [RL01], we propose a variant to improve thematching results. The first step is to shift a query model towards
a candidate target to match the centroids of the two models. The
query model is rotated by an arbitrary angle before being aligned by
ICP. We consider 8 possible angles to rotate a query model aroundeach axis; each angle is a multiple of 45◦ , hence 83 = 512 poses.
The pose of a query that yields the minimum distance with top
c 2016 The Author(s)Eurographics Proceedings c 2016 The Eurographics Association.

I. Pratikakis et al. / SHREC’16 Track: Partial Shape Queries for 3D Object Retrieval

matched targets is selected. Figure 8 shows two examples of 3D
point cloud registration using the fast ICP algorithm.

retrievalpre-processing

part-based
query

retrieval target
3D modelconvert tooriented point set

convert tooriented point set

oriented
point setoriented
point set
generate
sub-volumes

Figure 8: 3D point cloud registration using ICP algorithm
Run 5, 6, 7 are fusions of both BoW model and ICP algorithmwith different combinations. For artificial queries, we use a fusionof the results with dense samping BoW (Run1) and key-point sampling BoW (Run2). For Breuckmann queries, we select the resultsfrom either Run1, or Run2, or Run3. Because of the noise and low
level of details in Kinect queries, we choose the results from Run4
(with ICP).

extract and hasha feature(1001)2a binary codea set of sub-volumes
extract and hashfeatures(0001)2(1110)2
a set of binary codes(1011)2(0110)2

compute distancesin Hamming spacea distance between the query and the retrieval target model

3.4. Randomized Sub-Volume Partitioning for Part-based 3D
Model Retrieval (T. Furuya, R. Ohbuchi)
To efficiently and accurately compare a part-based query against
retrieval target 3D models in a dataset, we employed the randomized sub-volume partitioning (RSVP) algorithm [FKO15]. RSVP
randomly and iteratively partitions each of the target retrieval 3D
models into a set of sub-volumes by using 3D grids with random
intervals and orientations (Fig. 9). The part-based query is compared against the set of sub-volumes of all target 3D models byusing 3D geometric features extracted from the query and the subvolumes. Since the number of sub-volumes for the dataset is verylarge due to randomized partitioning, feature comparison between
the query and the sub-volumes must be very efficient. In addition,
each sub-volume feature must be compact so that all the features
for all the sub-volumes of all the target 3D models could be stored
on memory. To reduce the cost for retrieval, high-dimensional realvalued vectors extracted from sub-volumes are hashed into compact
binary codes. To generate a ranking result, a binary code for a partbased query is efficiently compared against the binary codes for the
sub-volumes of target 3D models by using Hamming distance.
3.4.1. Partitioning Retrieval Targets into Sub-Volumes
Prior to generating sub-volumes, each 3D model defined as a polygonal mesh is converted into an oriented point set by using the algorithm by Osada et al. [OFC02]. It randomly and uniformly samples
surfaces of the 3D polygonal model by oriented points. Each point
is associated with the normal vector of the triangle at which thepoint is sampled. We sample 4K oriented points on a query modeland 16K oriented points on a target 3D model. Given the oriented
point set of a target 3D model, the bounding box of the point set
is partitioned into Ng × Ng × Ng (we use Ng = 2) non-overlapping
cuboid sub-volumes by using a 3D grid having random intervals.
To generate sub-volumes having diverse position, scale, and orientation, this partitioning is iterated for Ni = 50 times and the pointc 2016 The Author(s)Eurographics Proceedings c 2016 The Eurographics Association.

Figure 1: Pipeline of the RSVP algorithm.

Figure 9: The pipeline of RSVP method submitted by Furuya andOhbuchiset is randomly rotated prior to each partitioning. As a result of
the partitioning, Nv = Ng × Ng × Ng × Ni = 400 sub-volumes are
generated per 3D model. For the SHREC track, the total numberof sub-volumes is 400 × 383 = 153, 200 since the dataset contains
383 retrieval target 3D pottery models.
3.4.2. Extracting Compact Features from Sub-Volumes
Each sub-volume generated by random grid partitioning is described by a 3D geometric feature called super vector of simplified point feature histograms (SV-SPFH). Oriented points within
the sub-volumes are described by a set of SPFH features [RBB09],
which has invariance against 3D rotation of oriented points. The
set of SPFH is encoded by super vector coding [ZYZH10] and isaggregated into a SV-SPFH feature per sub-volume. However, thecomputational cost for extracting SV-SPFH features from 153,200
sub-volumes in the dataset is quite high due to re-aggregation ofnumerous overlapping sub-volumes. To reduce the cost of persub-volume feature computation, we employ a ’late-binding’ approach [FKO15] which performs extraction and encoding of SPFH
features before sub-volume partitioning. Following sub-volume
partitioning, an SV-SPFH feature for a sub-volume is efficiently
computed by simply sum-pooling the encoded features that lie inside the sub-volume. The codebook for SV coding is learned by
performing Gaussian mixture model clustering on 250K SPFH features randomly selected from the dataset. Since we use 16 codewords for SV coding and each SPFH is a 125-dimensional vector,
the number of dimensions of SV-SPFH feature is (125 + 1) × 16 =

I. Pratikakis et al. / SHREC’16 Track: Partial Shape Queries for 3D Object Retrieval

2, 016. The SV-SPFH is power-normalized and then L2-normalized
as in [PS10].

datasets, whereas Tables 2- 9 present the corresponding retrieval
performance metrics.

Hashing sub-volume features: taking into account that the SVSPFH is a high-dimensional and real-valued vector, temporal costand spatial cost for feature comparison are both a problem. Aiming to reduce these costs, the set of SV-SPFH features is hashedinto a set of compact binary codes. The SV-SPFH features are
first projected onto a lower-dimensional (512 dimensions), realvalued subspace by using PCA. In the low-dimensional space, ahash function is learned by using the iterative quantization (ITQ)
algorithm [GS11] to generate compact (512 bit) binary codes. Eachof 153,200 sub-volumes in the dataset is described by a binary codewith 512 bits, resulting in a footprint of merely 9 MBytes, for all
the features in the dataset. All the features would easily fit in a main
memory of a modern PC for fast retrieval.

In the dataset of artificial queries, both for 25% and 40% partial
coverage, the BoW-based runs submitted by Tran et al. achieve thehighest retrieval performance, with the exception of run 4 which
is solely based on ICP (Fig. 10, 11 and Tables 2, 3). In particular, run 1 obtains the highest NN, FT and DCG. The SPFH-based
method submitted by Furuya and Ohbuchi approaches this retrieval
performance. It can also be noticed that the FV and VLAD-based
methods of Aono and Tashiro benefit from the increase in partial
coverage from 25% to 40%. In the latter case, these methods arecomparable to the methods mentioned above.

3.4.3. Ranking 3D Models
Ranking of 3D models in the dataset against a given query is performed efficiently by comparing their binarized SV-SPFH features.
The distance D between the query q and the target 3D model t iscomputed as:D(q, t) = argmin1≤i≤Nv d(bq , bti )(3)

where Nv is the number of sub-volumes for the model t (i.e.,
Nv = 400), bq is a binary code for the query, and bti is a binary
code for a sub-volume i of the model t. We use Hamming distanceas a distance function d between a pair of binary codes. Hamming
distance between a pair of binary codes can be computed very efficiently by using a combination of (SIMD) XOR instruction andbit count instruction. Finally, 3D models are ranked based on theirHamming distances to the part-based query. The RSVP is capablefor fast part-based 3D model retrieval. It took 1.3 seconds to extract
binarized SV-SPFH features from a query and took only 8 × 10−4
seconds to compute Hamming distances among the query and all
the 383 retrieval target 3D models. These processing times were
measured by using a single thread code run on a PC with two Intel
Xeon E5-2680 v2 CPUs.

In the dataset of high quality queries, the method of Furuya
and Ohbuchi obtains comparable retrieval performance with runs
1,3,5,7 submitted by Tran et al. (Fig. 12-14 and Tables 4-6). Overall, these methods outperform the rest of the methods submitted.
In the dataset of low quality queries, the runs 4-7 submitted byTran et al. achieve by far the highest retrieval performance. Interestingly the results are identical in these runs (Fig. 15-17 and Tables
7-9). The SFPH-based method of Furuya and Ohbuchi obtains significantly lower retrieval performance than the one it obtains in therest of the datasets. This can be attributed to the noise sensitivity of
SPFH, which is more pronounced in the case of low quality queries.
5. Conclusions
This track compared P3DOR methods submitted by four researchgroups, on datasets of 3D pottery models. All methods submitted
were designed for generic P3DOR. The partial shape queries havebeen provided in three different forms: (i) artificial queries, (ii) real
queries of high quality and (iii) real queries of low quality. Basedupon the performance of the participating methods, it can be derived that the two last forms are very challenging. It is evident that
P3DOR is still an open problem, for which the particular SHREC
track has set a meaningfull base for benchmarking to support future
research efforts for improvement.

4. Results

PickupAono-FV

Aono-GOLD

Aono-VLAD

Tran-1Tran-2Tran-3Tran-4Tran-5Tran-6Tran-7

0.9
0.8
0.7
0.6
PrecisionExperimental evaluation is based on precision-recall (P-R) curves
and four quantitative measures: nearest neighbor (NN), first tier
(FT), second tier (ST) and discounted cumulative gain (DCG). For
every query model that belongs to a class C, recall denotes the percentage of models of class C that are retrieved and precision denotes the proportion of retrieved models that belong to class C over
the total number of retrieved models. The maximum score is 100%for both quantities. Nearest neighbor (NN) indicates the percentageof queries where the closest match belongs to the query class. Firsttier (FT) and second tier (ST) statistics, measure the recall value forthe (D − 1) and 2(D − 1) closest matches respectively, where D isthe cardinality of the query class. The DCG statistic weights correct results near the front of the list more than correct results laterin the ranked list under the assumption that a user is less likely toconsider elements near the end of the list [SMKF04]. Figures 1017 present the P-R curves estimated for the 12 different runs in all

Furuya

0.5
0.4
0.3
0.2
0.1
0.1

0.2

0.3

0.4

0.5

0.6
Recall

0.7

0.8

0.9

1

Figure 10: P-R curves of the submitted methods on the artificial
query set with 25% partial coverage.

c 2016 The Author(s)Eurographics Proceedings c 2016 The Eurographics Association.

I. Pratikakis et al. / SHREC’16 Track: Partial Shape Queries for 3D Object Retrieval

FuruyaPickupAono-FV

Aono-GOLD

Aono-VLAD

Tran-1

FuruyaPickupAono-FV

Aono-GOLD

Aono-VLAD

Tran-1Tran-2Tran-3Tran-4Tran-5Tran-6Tran-7Tran-2Tran-3Tran-4Tran-5Tran-6Tran-7

0.9

0.5

0.8
0.7
0.4
Precision

Precision

0.6
0.5
0.4
0.3
0.3
0.2
0.1

0.2
0.1

0.2

0.3

0.4

0.5

0.6
Recall

0.7

0.8

0.9

1

Figure 11: P-R curves of the submitted methods on the artificial
query set with 40% partial coverage.

0.1

0.2

0.3

0.4

0.5

0.6
Recall

0.7

0.8

0.9

1

Figure 14: P-R curves of the submitted methods on the high quality
query set-view 3.

FuruyaPickupAono-FV

Aono-GOLD

Aono-VLAD

Tran-1

FuruyaPickupAono-FV

Aono-GOLD

Aono-VLAD

Tran-1Tran-2Tran-3Tran-4Tran-5Tran-6Tran-7Tran-2Tran-3Tran-4Tran-5Tran-6Tran-7

0.7

0.5

0.6

Precision

0.5

Precision

0.4

0.3

0.4

0.3

0.2

0.2

0.1
0.1

0.2

0.3

0.4

0.5

0.6
Recall

0.7

0.8

0.9

1

Figure 12: P-R curves of the submitted methods on the high quality
query set-view 1.

0.1

0.2

0.3

0.4

0.5

0.6
Recall

0.7

0.8

0.9

1

Figure 15: P-R curves of the submitted methods on the low quality
query set-view 1.

FuruyaPickupAono-FV

Aono-GOLD

Aono-VLAD

Tran-1

FuruyaPickupAono-FV

Aono-GOLD

Aono-VLAD

Tran-1Tran-2Tran-3Tran-4Tran-5Tran-6Tran-7Tran-2Tran-3Tran-4Tran-5Tran-6Tran-7

0.5

0.7

0.6

Precision

0.5

Precision

0.4

0.3

0.4

0.3

0.2

0.2

0.1
0.1

0.2

0.3

0.4

0.5

0.6
Recall

0.7

0.8

0.9

1

Figure 13: P-R curves of the submitted methods on the high quality
query set-view 2.

c 2016 The Author(s)Eurographics Proceedings c 2016 The Eurographics Association.

0.1

0.2

0.3

0.4

0.5

0.6
Recall

0.7

0.8

0.9

1

Figure 16: P-R curves of the submitted methods on the low quality
query set-view 2.

I. Pratikakis et al. / SHREC’16 Track: Partial Shape Queries for 3D Object Retrieval

FuruyaPickupAono-FV

Aono-GOLD

Aono-VLAD

Tran-1Tran-2Tran-3Tran-4Tran-5Tran-6Tran-70.7

0.6

Precision

0.5

0.4

0.3

0.2

0.1
0.1

0.2

0.3

0.4

0.5

0.6
Recall

0.7

0.8

0.9

1

Figure 17: P-R curves of the submitted methods on the low quality
query set-view 3.

Group

Run

NN

FT

ST

DCG

Aono et al.
Aono et al.
Aono et al.
Pickup et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Furuya et al.

FV
GOLD
VLAD
View-based
1
2
3
4
5
6
7
RSVP

0.38
0.38
0.48
0.10
1.00
1.00
1.00
0.62
1.00
1.00
1.00
0.86

0.41
0.40
0.42
0.18
0.49
0.47
0.47
0.49
0.47
0.47
0.47
0.47

0.59
0.63
0.60
0.37
0.69
0.64
0.65
0.64
0.64
0.64
0.64
0.70

0.72
0.71
0.72
0.58
0.81
0.81
0.80
0.79
0.81
0.81
0.81
0.81

Table 2: The results of the submitted methods on the artificial query
set with 25% partial coverage.

Group

Run

NN

FT

ST

DCG

Aono et al.
Aono et al.
Aono et al.
Pickup et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Furuya et al.

FV
GOLD
VLAD
View-based
1
2
3
4
5
6
7
RSVP

0.76
0.62
0.76
0.14
1.00
1.00
1.00
0.86
1.00
1.00
1.00
0.90

0.51
0.40
0.50
0.18
0.52
0.51
0.51
0.48
0.51
0.51
0.51
0.49

0.72
0.65
0.74
0.38
0.71
0.72
0.71
0.64
0.72
0.72
0.72
0.71

0.81
0.74
0.80
0.58
0.82
0.83
0.83
0.80
0.83
0.83
0.83
0.82

Table 3: The results of the submitted methods on the artificial query
set with 40% partial coverage.

Group

Run

NN

FT

ST

DCG

Aono et al.
Aono et al.
Aono et al.
Pickup et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Furuya et al.

FV
GOLD
VLAD
View-based
1
2
3
4
5
6
7
RSVP

0.24
0.20
0.24
0.24
0.52
0.36
0.32
0.60
0.52
0.60
0.32
0.56

0.28
0.27
0.29
0.26
0.31
0.22
0.29
0.23
0.31
0.23
0.23
0.32

0.57
0.54
0.60
0.48
0.53
0.45
0.56
0.48
0.53
0.48
0.48
0.53

0.67
0.66
0.68
0.65
0.68
0.64
0.67
0.68
0.68
0.68
0.68
0.69

Table 4: The results of the submitted methods on the high quality
query set-view 1.

Group

Run

NN

FT

ST

DCG

Aono et al.
Aono et al.
Aono et al.
Pickup et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Furuya et al.

FV
GOLD
VLAD
View-based
1
2
3
4
5
6
7
RSVP

0.32
0.48
0.24
0.20
0.56
0.36
0.28
0.52
0.56
0.52
0.28
0.36

0.25
0.26
0.29
0.25
0.32
0.23
0.32
0.30
0.32
0.30
0.32
0.33

0.57
0.52
0.58
0.48
0.52
0.47
0.58
0.50
0.52
0.50
0.58
0.55

0.67
0.66
0.69
0.63
0.69
0.65
0.68
0.67
0.69
0.67
0.68
0.68

Table 5: The results of the submitted methods on the high quality
query set-view 2.

Group

Run

NN

FT

ST

DCG

Aono et al.
Aono et al.
Aono et al.
Pickup et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Furuya et al.

FV
GOLD
VLAD
View-based
1
2
3
4
5
6
7
RSVP

0.24
0.32
0.16
0.20
0.60
0.32
0.48
0.28
0.60
0.28
0.48
0.32

0.19
0.23
0.22
0.27
0.32
0.25
0.31
0.28
0.32
0.27
0.31
0.30

0.47
0.51
0.50
0.47
0.53
0.48
0.55
0.49
0.53
0.49
0.55
0.48

0.63
0.64
0.64
0.64
0.69
0.65
0.69
0.64
0.69
0.64
0.69
0.67

Table 6: The results of the submitted methods on the high quality
query set-view 3.

c 2016 The Author(s)Eurographics Proceedings c 2016 The Eurographics Association.

I. Pratikakis et al. / SHREC’16 Track: Partial Shape Queries for 3D Object Retrieval

Group

Run

NN

FT

ST

DCG

References

Aono et al.
Aono et al.
Aono et al.
Pickup et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Furuya et al.

FV
GOLD
VLAD
View-based
1
2
3
4
5
6
7
RSVP

0.12
0.32
0.20
0.36
0.08
0.20
0.24
0.60
0.60
0.60
0.60
0.08

0.22
0.23
0.24
0.24
0.25
0.25
0.25
0.40
0.40
0.40
0.40
0.21

0.51
0.50
0.55
0.49
0.55
0.51
0.51
0.61
0.61
0.61
0.61
0.49

0.64
0.65
0.66
0.65
0.64
0.64
0.64
0.76
0.76
0.76
0.76
0.62

[ABD12] A LCANTARILLA P., BARTOLI A., DAVISON A. J.: Kaze features. In European Conference on Computer Vision (ECCV) (2012). 3

Table 7: The results of the submitted methods on the low quality
query set-view 1.

Group

Run

NN

FT

ST

DCG

Aono et al.
Aono et al.
Aono et al.
Pickup et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Furuya et al.

FV
GOLD
VLAD
View-based
1
2
3
4
5
6
7
RSVP

0.16
0.32
0.20
0.16
0.08
0.24
0.36
0.72
0.72
0.72
0.72
0.08

0.22
0.23
0.24
0.20
0.30
0.29
0.29
0.36
0.36
0.36
0.36
0.18

0.51
0.49
0.53
0.45
0.60
0.55
0.56
0.61
0.61
0.61
0.61
0.45

0.66
0.65
0.67
0.60
0.66
0.67
0.67
0.75
0.75
0.75
0.75
0.60

Table 8: The results of the submitted methods on the low quality
query set-view 2.

Group

Run

NN

FT

ST

DCG

Aono et al.
Aono et al.
Aono et al.
Pickup et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Tran et al.
Furuya et al.

FV
GOLD
VLAD
View-based
1
2
3
4
5
6
7
RSVP

0.12
0.48
0.16
0.12
0.08
0.24
0.32
0.80
0.80
0.80
0.80
0.08

0.21
0.26
0.23
0.22
0.30
0.27
0.28
0.36
0.36
0.36
0.36
0.20

0.51
0.52
0.34
0.45
0.59
0.51
0.52
0.58
0.58
0.58
0.58
0.47

0.63
0.65
0.64
0.61
0.65
0.67
0.67
0.75
0.75
0.75
0.75
0.60

Table 9: The results of the submitted methods on the low quality
query set-view 3.

[AMS11] A DAN A., M ERCHAN P., S ALAMANCA S.: 3D scene retrieval
and recognition with depth gradient images. Pattern Recognition Letters32, 9 (2011), 1337–1353. 1[APP∗ 10] AGATHOS A., P RATIKAKIS I., PAPADAKIS P., P ERANTONIS
S., A ZARIADIS P., S APIDIS S.: 3D articulated object retrieval using agraph-based representation. The Visual Computer 26, 10 (2010), 1301–
1319. 1
[AZ12] A RANDJELOVIC R., Z ISSERMAN A.: Three things everyone
should know to improve object retrieval. In Computer Vision and Pattern
Recognition (CVPR), 2012 IEEE Conference on (June 2012), pp. 2911–
2918. doi:10.1109/CVPR.2012.6248018. 4[BBGO11] B RONSTEIN A., B RONSTEIN M., G UIBAS L., OVSJANIKOV
M.: Shape google: geometric words and expressions for invariant shape
retrieval. ACM Transactions on Graphics 30, 1 (2011), 1–20. 1[BM92] B ESL P., M C K AY N.: A Method for Registration of 3-D Shapes.
In IEEE Transactions on Pattern Analysis and Machine Intelligence(1992), vol. 14, pp. 239–256. 4[CB07] C HEN H., B HANU B.: 3D free-form object recognition in range
images using local surface patches. Pattern Recognition Letters 28, 10(2007), 1252–1262. 1[CM91] C HEN Y., M EDIONI G.: Object modeling by registration of multiple range images. Proceedings. 1991 IEEE International Conferenceon Robotics and Automation (1991). 4[DGA∗ 09] D UTAGACI H., G ODIL A., A XENOPOULOS A., DARAS P.,
F URUYA T., O HBUCHI R.: SHREC’09 track: Querying with partial
models. In Proc. 3DOR (2009). 1[DGC∗ 10]D UTAGACI H., G ODIL A., C HEUNG C., F URUYA T., H IL LENBRAND U., O HBUCHI R.: SHREC’10 track: Range scan retrieval.

In Proc. 3DOR (2010). 1[FKO15] F URUYA T., K URABE S., O HBUCHI R.: Randomized subvolume partitioning for part-based 3D model retrieval. In EG 3DOR (2015),
pp. 15–22. 5[FO09] F URUYA T., O HBUCHI R.: Dense sampling and fast encoding
for 3D model retrieval using bag-of-visual features. In Proc ACM CIVR
(2009). 1[GS11] G ONG Y., S. L.: Iterative quantization: A procrustean approachto learning binary codes. In Computer Vision and Pattern Recognition(CVPR), 2011 IEEE Conference on (2011), pp. 817–824. 6[GSB∗ 13] G UO Y., S OHEL F. A., B ENNAMOUN M., L U M., WAN J.:
Rotational projection statistics for 3D local surface description and object recognition. CoRR abs/1304.3192 (2013). URL: http://arxiv.
org/abs/1304.3192. 4[JDSP10] J EGOU H., D OUZE M., S CHMID C., P EREZ P.: Aggregating
local descriptors into a compact image representation. In In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on (2010),
pp. 3304–3311. 3[Lav12] L AVOUÉ G.: Combination of bag-of-words descriptors for robust partial shape retrieval. The Visual Computer journal 28, 9 (2012),
931–942. 1[LGJ14] L I B., G ODIL A., J OHAN H.: Hybrid shape descriptor and meta
similarity generation for non-rigid and partial 3D model retrieval. Multimedia Tools and Applications 72, 2 (2014), 1531–1560. 1[LGSX13] L IAN Z., G ODIL A., S UN X., X IAO J.: CM-BOF: visual
similarity-based 3D shape retrieval using clock matching and bag-offeatures. Machine Vision and Applications (2013), 1–20. 3[Low04] L OWE D.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 2 (2004), 91–110.
3

c 2016 The Author(s)Eurographics Proceedings c 2016 The Eurographics Association.

I. Pratikakis et al. / SHREC’16 Track: Partial Shape Queries for 3D Object Retrieval[OFC02] O SADA R., F UNKHOUSER T., C HAZELLE B.: Shape distributions. ACM Trans. on Graphics 21, 4 (2002), 807–832. 5[PISZ08] P HILBIN J., I SARD M., S IVIC J., Z ISSERMAN A.: Lost inquantization: Improving particular object retrieval in large scale image
databases. In In CVPR (2008). 4[PRE]

PRESIOUS project. URL: presious.eu. 2[PS10] P ERONNIN F., S ANCHEZ J. AND M ENSINK T.: Improving thefisher kernel for large-scale image classification. In European Conference on Computer Vision (ECCV) (2010), pp. 143–156. 3, 6
[RBB09] RUSU R., B LODOW N., B EETZ M.: Fast point feature histograms for 3D registration. In IEEE International Conference on
Robotics and Automation (ICRA) (2009), pp. 3212–3217. 5[RL01] RUSINKIEWICZ S., L EVOY M.: Efficient variants of the ICP
algorithm. In Third International Conference on 3D Digital Imagingand Modeling (3DIM) (June 2001). 4[SGMC15] S ERRA G., G RANA C., M ANFREDI M., C UCCHIARA R.:
Gold: Gaussians of local descriptors for image representation. Computer
Vision and Image Understanding 134 (2015), 22–32. 3[SMB∗ 13] S IPIRAN I., M ERUANE R., B USTOS B., S CHRECK T., J O HAN H., L I B., L U Y.: SHREC’13 track: large-scale partial shape retrieval using simulated range images. In Proc. 3DOR (2013). 2[SMKF04] S HILANE P., M IN P., K AZHDAN M., F UNKHOUSER T.: The
princeton shape benchmark. In Proc. SMI (2004), pp. 167–178. doi:
10.1109/SMI.2004.63. 6[SMM∗ 10] S TAVROPOULOS G., M OSCHONAS P., M OUSTAKAS K.,
T ZOVARAS D., S TRINTZIS M.: 3D model search and retrieval fromrange images using salient features. IEEE Transactions on Multimedia
12, 7 (2010), 692–704. 1[SPK∗ 14] S FIKAS K., P RATIKAKIS I., KOUTSOUDIS A., S AVELONAS
M., T HEOHARIS T.: Partial matching of 3D cultural heritage objects
using panoramic views. Multimedia Tools and Applications (2014).
doi:10.1007/s11042-014-2069-0. 1[SPS15] S AVELONAS M. A., P RATIKAKIS I., S FIKAS K.: An overview
of partial 3D object retrieval methodologies. Multimedia Tools and Applications 74 (2015), 11783–11808. 1[SPS16] S AVELONAS M., P RATIKAKIS I., S FIKAS K.: Fisher encodingof differential fast point feature histograms for partial 3D object retrieval.
Pattern Recognition (2016). doi:10.1016/j.patcog.2016.02.
003. 1[SZ03] S IVIC J., Z ISSERMAN A.: Video Google: A text retrieval approach to object matching in videos. In Proceedings of the International Conference on Computer Vision (Oct. 2003), vol. 2, pp. 1470–
1477. URL: http://www.robots.ox.ac.uk/~vgg. 3[TVD09] T IERNY J., VANDEBORRE J., DAOUDI M.: Partial 3D shape
retrieval by reeb pattern unfolding. Computer Graphics Forum 28, 1(2009), 41 – 55. 1[VF08] V EDALDI A., F ULKERSON B.: VLFeat: An open and portablelibrary of computer vision algorithms. http://www.vlfeat.org/,
2008. 3
[VHM] The Virtual Hampson Museum. URL: http://hampson.
cast.uark.edu/. 2[Zho09] Z HONG Y.: Intrinsic shape signatures: A shape descriptor for 3d
object recognition. In Computer Vision Workshops (ICCV Workshops),
2009 IEEE 12th International Conference on (Sept 2009), pp. 689–696.
doi:10.1109/ICCVW.2009.5457637. 4[ZJS13] Z HU C.-Z., J EGOU H., S ATOH S.: Query-adaptive asymmetrical dissimilarities for visual object retrieval. In Computer Vision (ICCV),
2013 IEEE International Conference on (Dec 2013), pp. 1705–1712.
doi:10.1109/ICCV.2013.214. 4[ZYZH10] Z HOU X., Y U K., Z HANG T., H UANG T.: Image classification using super-vector coding of local image descriptors. In European
Conference on Computer Vision (ECCV) (2010), pp. 141–154. 5

c 2016 The Author(s)Eurographics Proceedings c 2016 The Eurographics Association.