Takahiko Furuya, Ryutarou Ohbuchi, Accurate Aggregation of Local Features by using K-sparse Autoencoder for 3D Model Retrieval, shortpaper, ACM International Conference on Multimedia Retrieval 2016 (ICMR 2016), June 6-9, New York, NY, USA. (2016)

Accurate Aggregation of Local Features by using
K-sparse Autoencoder for 3D Model Retrieval
Takahiko Furuya

Ryutarou Ohbuchi

University of Yamanashi
4-3-11 Takeda, Kofu-shi
Yamanashi-ken, 400-8511, Japan+81-55-220-8570

University of Yamanashi
4-3-11 Takeda, Kofu-shi
Yamanashi-ken, 400-8511, Japan+81-55-220-8570

takahikof AT yamanashi.ac.jp

ohbuchi AT yamanashi.ac.jp

ABSTRACT
Aggregating a set of local features has been used widely to realize
recognition or retrieval of multimedia data including 2D images
and 3D models. A number of feature aggregation algorithms (e.g.,
Bag-of-Features [5], Locality-constrained Linear coding [26], orFisher Vector coding [23]) have been proposed. They first learn acodebook, or a set of codewords, by clustering the local features
and then encode these local features by using the learned codebook.
Despite the great success of these feature aggregation algorithms,
we argue that they are not necessarily optimal in terms of accuracy
since their codebook learning and feature encoding are computed
separately. In this paper, we propose two novel feature aggregation
algorithms based on k-Sparse Autoencoder (kSA) [20] that realize
more accurate local feature aggregation. Our proposed algorithms,
called Database-adaptive kSA (DkSA) aggregation and Per-dataadaptive kSA (PkSA) aggregation, jointly optimize codebook
learning and feature encoding. In addition, the kSA-based feature
encoding enhances saliency of local features due to k-sparsenessconstraints and non-negativity constraints. Of the two proposed
algorithms, the PkSA aggregation exploits reconstruction error of a
local feature derived from the kSA for more accurate aggregatedfeature. Experimental evaluation using a shape-based 3D model
retrieval scenario showed that the retrieval accuracy of our
proposed algorithms are superior to the existing feature aggregation
algorithms we have compared against.

Keywords
Feature aggregation, feature encoding, autoencoder, sparse coding,
local feature, bag-of-features, content-based retrieval, 3D shape.

1. INTRODUCTION
Aggregation of local features is an important technologicalcomponent for effective and efficient comparison among
multimedia data objects, including 2D images, 3D models, etc.
Recently, for the task of classification or retrieval of 2D images,“end-to-end” deep learning has achieved higher accuracy than theconventional approach which combines hand-crafted local features
Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. Copyrights forcomponents of this work owned by others than ACM must be honored.
Abstracting with credit is permitted. To copy otherwise, or republish, topost on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from Permissions@acm.org.
ICMR'16, June 06-09, 2016, New York, NY, USA© 2016 ACM. ISBN 978-1-4503-4359-6/16/06…$15.00
DOI: http://dx.doi.org/10.1145/2911996.2912054

and their aggregation. On the other hand, recent studies (e.g., [10])have shown that aggregating features derived from a deep neuralnetwork is also effective for 2D image retrieval. Also, for shapebased 3D model retrieval (3DMR) on which we focus in this paper,
extracting local features from a 3D model and aggregating them per
3D model remains as one of the most promising approaches. We
thus think that improving feature aggregation is still quite
meaningful for progress of multimedia information retrieval.
Goal of local feature aggregation is to generate a feature vector thatreflects, as well as possible, distribution of local features in theirfeature space. Feature aggregation generally consists of two steps,i.e., encoding and pooling. A set of codewords are chosen, typically,by clustering the set of local features. A local feature is encoded byusing codewords around the feature and statistics associated withthe codewords. Such statistics as density, mean, and/or variance,
can be used for the encoding. For pooling, encoded local features
are accumulated into a single feature per 3D model. For efficientinter-feature comparison, aggregated feature better be compact.
Many algorithms for feature aggregation have been proposed [12].
They can be roughly classified into two groups; sparse coding (SC)based approach and higher-order statistics (HS)-based approach.
The SC-based approach (e.g., [5][19][26]) sparsely encodes a local
feature by using weighted linear sum of its neighboring codewords.
Locality-constrained Linear (LL) coding [26] is one of the mostrepresentative SC-based methods. Bag-of-Features (BF) [5] is a
special case of the SC-based approach in which a local feature is
assigned to its nearest single codeword by vector quantization. On
the other hand, the HS-based approach (e.g., [14][23][27]) tries to
encode local features by using higher-order statistics around thecodewords. Fisher Vector (FV) coding [23] employs mean andvariance of local features around the codeword. Vector of LocallyAggregated Descriptor (VLAD) [14] approximates the FV by using
only mean around the codeword.
These feature aggregation algorithms described above have beensuccessfully utilized for recognition or retrieval of 2D images or 3Dmodels. However, they are not necessarily optimal in terms ofaccuracy since the codebook learning and the feature encoding
steps are processed separately. That is, as a pre-processing, acodebook (i.e., a set of codewords) is first learned by clustering theset of local features. Then, each local feature is encoded by using
the learned codebook. This “greedy” framework is adopted by mostof the feature aggregation algorithms including BF, LL1, FV,
VLAD, and Triangular Encoding (TE) [4]. The greedy approach
could reduce expressive power of the aggregated features.
1
Although joint optimization framework is proposed in [26], LL is typicallyused along with a codebook learned by k-means clustering in the literature.

Takahiko Furuya, Ryutarou Ohbuchi, Accurate Aggregation of Local Features by using K-sparse Autoencoder for 3D Model Retrieval, shortpaper, ACM International Conference on Multimedia Retrieval 2016 (ICMR 2016), June 6-9, New York, NY, USA. (2016)a large set of local features
kSA
extracted from all the encoder decoder
3D models in a databasea local feature

forward propagation

a set of local features kSA
extracted fromencodera 3D modela set ofa set of local features
ankSA
encodedextracted fromaggregatedencoder decoder
local features feature
a 3D model

forward propagation

sum-poolingback propagation(a) training kSA (pre-processing).

(b) DkSA aggregation.

parameter spacefor kSAafter
fine-tuning

displacement
as aggregatedfeaturebeforefine-tuning
forward propagation
computingdisplacement ofparameters
back propagation(c) PkSA aggregation.

Figure 1. Overview of the proposed feature aggregation algorithms using k-Sparse Autoencoder.
In this paper, aiming for more accurate aggregation of local features,
we propose a pair of novel SC-based feature aggregation algorithms.
The algorithm is based on k-Sparse Autoencoder (kSA) proposedby Makhzani et al. [20]. A kSA-based feature aggregation can be
more accurate than the existing ones for two reasons. Firstly, kSA
can jointly optimize codebook learning and feature encoding as
kSA is trained by alternating encoding of local features andupdating of codewords (i.e., updating of connection weights of kSA
neurons) so that reconstruction error of the local features isminimized. Similar joint optimization framework is employed by
Sparse Autoencoder (SA) [21] and KSVD [2], which are popularsparse coding algorithms. Secondly, feature encoding by kSA could
enhance saliency of local features since the kSA has two constraints
considered to aid accurate feature encoding, that are, k-sparsenessand non-negativity. K-sparseness constrains feature encoding to use
a small number (k) of codewords correlated to the local feature.
Non-negativity forces coefficients weighting the codewords to be
non-negative. Several state-of-the-art SC-based algorithms, e.g.,
LL coding and Localized Soft-assignment (LS) coding [19],showed these constraints improve accuracy of encoded features.

parameters, or weights, caused by the fine-tuning. Table 1
compares characteristics of SC-based feature aggregation
algorithms we will evaluate in the experimental section. The DkSA
and the PkSA have potential to yield expressive aggregated features
since both of them satisfy all the criteria essential for accuratefeature aggregation, i.e., joint optimization of codebook and feature
encoding, k-sparse constraint, and non-negative constraint.
Exploiting reconstruction error of local features for PkSA is novel
for the group of SC-based feature aggregation algorithms.

Our two kSA-based feature aggregation algorithms are called
Database-adaptive kSA (DkSA) aggregation and Per-data-adaptivekSA (PkSA) aggregation. DkSA (Figure 1b) is a straight-forward
approach for feature aggregation using kSA. As a pre-processing(Figure 1a), kSA is trained by using numerous and diverse local
features sampled from the database (hence database-adaptive).
After training, each of a set local features extracted from a 3Dmodel is encoded by using the database-adapted kSA. The set of
encoded local features are accumulated into a single feature vector
per 3D model by sum-pooling for efficient comparison. PkSA
(Figure 1c), on the other hand, aims at generating aggregatedfeatures more expressive than DkSA by exploiting reconstruction
error of local features derived from the kSA. kSA is first trained in
the same manner as DkSA (Figure 1a). Then, the database-adapted
kSA is additionally trained, or fine-tuned, by using a set of local
features extracted from each 3D model (hence per-data-adaptive).
An aggregated feature for the 3D model is a set of displacements of

2. PROPOSED ALGORITHM
2.1 Database-adaptive kSA Aggregation

Table 1. Comparison of sparse coding-based feature
aggregation algorithms.
joint
exploiting
optimization
algorithms k-sparseness non-negativity
reconstructionof codebook
error
and encoding
SA [21]
No
No
Yes
Yes
TE [4]No
No
No
Yes
KSVD [2]
No
No
Yes
Yes
1
LL [26]
No
No
Yes
Yes
DkSA (ours)
No
Yes
Yes
Yes
PkSA (ours)
Yes
Yes
Yes
Yes

Experimental evaluation under a 3DMR scenario shows that the
proposed kSA-based feature aggregation algorithms significantly
outperform existing SC-based and HS-based counterparts.
Contributions of this paper can be summarized as follows.
 Proposition of two k-Sparse Autoencoder based feature
aggregation algorithms, that are, DkSA and PkSA, for moreaccurate aggregation of local features.
 Quantitative evaluation of DkSA and PkSA using multiple
local features and multiple benchmark datasets for 3DMR.

As with the previous feature aggregation methods, DkSA (Figure1b) aggregation consists of two processing steps, that are, encoding
and pooling of local features. At the encoding stage, a set of local
features extracted from a 3D model is first normalized by using
ZCA-whitening [3] so that local features have zero means and all
the pair of dimensions are uncorrelated. Then, each whitened local
feature is fed into the trained kSA to obtain unit activations in thehidden layer. Feature encoding is performed as h = σ(y) = σ(xWe),
where h is an encoded local feature (or unit activation in the hidden
layer), x is a input (whitened) local feature, We is a weight matrix
for encoder, and σ is a non-linear activation function. By using
hidden layer with n units, d-dimensional local feature x is encoded
into n-dimensional feature (n > d). Note that our kSA includes bias
terms for both encoder and decoder, although the bias term isomitted from the equation for notation simplicity.
Activation function σ strongly influences saliency of encoded local
features. Our activation function consists of Rectified Linear Units(ReLU) [16] and k-largest value selection. Specifically, yi, which isi-th element of y, is first rectified by using ReLU, i.e.,zi =max(0, yi). ReLU ensures non-negativity of feature encoding.
Then, k largest elements in {zi | i = 1, …, n} are kept while the rest
is set to 0.
At the pooling stage, the set of encoded local features is aggregatedinto a single feature vector per 3D model by summing. Therefore,
the number of dimensions for aggregated feature is also n. Finally,

Takahiko Furuya, Ryutarou Ohbuchi, Accurate Aggregation of Local Features by using K-sparse Autoencoder for 3D Model Retrieval, shortpaper, ACM International Conference on Multimedia Retrieval 2016 (ICMR 2016), June 6-9, New York, NY, USA. (2016)
the aggregated feature vector is normalized by using power
normalization followed by L2 normalization as with [23]. Cosine
similarity is used to compare a pair of DkSA-aggregated features.
DkSA has two hyper-parameters that require manual tuning, i.e.,
the number of units in the hidden layer n and the number of nonzero elements in the hidden layer k. In the experiments, n is
typically set to 8,000 and small k (e.g.,k=3) is used.
Training of the kSA is performed as follows. The kSA is trained byminimizing mean squared reconstruction error of local features. We
use T=250k local features randomly sampled from the database fortraining. We use Stochastic Gradient Descent (SGD) with minibatch. Each mini-batch contains 200 local features randomly
selected from the set of T training local features. To adaptively
assign a learning rate to each parameter (i.e., each connection
weight between neurons), we use AdaGrad algorithm [6] with an
initial learning rate = 0.2. Training is regularized with weight decaycoefficient λ = 10–4. The optimization is iterated for 50 epochs.

2.2 Per-data-adaptive kSA Aggregation
PkSA (Figure 1c) aims at generating more accurate aggregatedfeatures than DkSA by exploiting reconstruction error of local
features. For pre-processing, kSA is trained in the same manner as
DkSA except for the initial learning rate = 0.05 for AdaGrad. We
use this initial learning rate since it yielded slightly higher retrieval
accuracy in the preliminary experiments.
To aggregate a set of local features extracted from a 3D model M,
the database-adapted kSA is additionally trained, or fine-tuned, byusing the local features of the 3D model M. The kSA parameters
We for encoder and Wd for decoder are fine-tuned to reconstruct
the local features of the 3D model M well. Then, we computedisplacement, or differential, of parameters due to fine-tuning as
ΔWd = W*d – Wd, where W*d and Wd indicate weight matrices ofthe decoder after and before fine-tuning, respectively. The
displacement matrix ΔWd is then vectorized to form a PkSAaggregated feature for the 3D model M. The PkSA-aggregatedfeature vector has D=(n+1)×d dimensions where “+1” means that
our kSA has bias terms. We can also use parameter displacement of
the encoder, i.e., ΔWe = W*e − We, for aggregation. However,preliminary experiments showed that decoder displacement ΔWd
consistently outperformed encoder displacement ΔWe.
Note that fine-tuning for PkSA aggregation is performed per 3Dmodel. That is, the database-adapted kSA is duplicated for each 3Dmodel, and each duplicated kSA is fine-tuned by using the set of
local features extracted from the 3D model. For fine-tuning, we
employ Steepest Descent (SD) algorithm; all the local features of
the 3D model are used to compute gradients of the parameters. We
use SD instead of SGD (with mini-batch) so as to exclude
randomness from the result of fine-tuning. We use AdaGrad withinitial learning rate = 0.05 and fine-tuning is iterated for 50 epochs.
Since PkSA aggregation produces higher-dimensional aggregatedfeature vectors than DkSA aggregation, we perform dimension
reduction of the PkSA-aggregated features for efficient interfeature comparison. We use Sparse Random Projection (SRP) [1]algorithm to compress the PkSA-aggregated features down to 8,000
dimensions. Preliminary experiments showed that the SRP down to8,000 dimensions did not degrade retrieval accuracy. Comparison
among the dimension-reduced PkSA-aggregated feature vectors isdone in a manner identical to that of DkSA; aggregated features are
power-normalized and then L2-normalized for Cosine similarity.
PkSA has the same hyper-parameters as DkSA, i.e., the number ofunits in the hidden layer n and the number of non-zero elements in

the hidden layer k. In the experiments, n is to 500 and k is set toaround 10, which is slightly larger than k for DkSA.

3. EXPERIMENTS AND RESULTS
3.1 Experimental Setup
To validate efficacy of the proposed feature aggregation algorithms,
we use four benchmark databases for 3DMR; the Princeton ShapeBenchmark (PSB) [24], the Engineering Shape Benchmark (ESB)[13], the SHREC 2011 Non-Rigid watertight meshes dataset(SH11NR) [17], and the SHREC 2014 Large-scale Comprehensive
3D shape retrieval dataset (SH14LC) [18]. For all the benchmarks,
one of 3D models in the database is used as a query and remaining
3D models are used as retrieval targets. We use Mean Average
Precision (MAP) [%] as an index of retrieval accuracy. Since
optimization of the proposed algorithms using SGD haverandomness, all the experiments for DkSA and PkSA are performed
three times and their average MAP are reported. Table 2
summarizes the number of non-zero units k used for DkSA and
PkSA aggregation. For another hyper-parameter, i.e., the number ofhidden units, we use n=8,000 for DkSA and n=500 for PkSA unlessotherwise stated.
Table 2. Number of non-zero units k used for experiments.
DkSA
PkSA

POD

LSF

3
20

3
10

SI
3
10

RoPS
3
20

DSIFT MO1SIFT
1
3
5
20

We compare the proposed feature aggregation algorithms against
six existing feature aggregation algorithms, i.e., BF [5], LL [26],
FV [23], VLAD (VL) [14], SV [27], and DM [9]. We also apply
three sparse coding algorithms to local feature aggregation, i.e.,Sparse Autoencoder (SA) [21], Triangular Encoding (TE) [4], andKSVD [2]. For sparse coding-based aggregation methods including
BF, LL, SA, TE, and KSVD, we learn 8k codewords. On the other
hand, for higher-order statistics-based methods including VL, FV,and SV, the number of codewords are determined so that
dimensionality of aggregated feature vector becomes about 300k.
For DM, we use 250k local features to generate a manifold graphfor aggregation. All the aggregated feature vectors are powernormalized and are then L2-normalized. We use Cosine similarity
for comparison among feature vectors.
We use six local features for 3DMR, i.e., Position and Orientation
Distribution (POD) [9], Spin Image (SI) [15], Local Statistical
Feature (LSF) [22], Rotational Projection Statistics (RoPS) [10],Dense SIFT (DSIFT) [7], and Multi-Orientation One SIFT
(MO1SIFT) [8]. They are classified into two groups; local 3D
geometric feature including POD, SI, LSF, RoPS, and local 2D
visual feature including DSIFT and MO1SIFT. Each local feature
extraction method has hyper-parameters, e.g., the number of local
features per 3D model or the number of rendering views. In thispaper, we follow the experimental settings by Furuya et al. [9].

3.2 Experimental Results
3.2.1 Comparison with Other Feature Aggregation
Algorithms
Table 3 compares the proposed feature aggregation algorithms
against the other feature aggregation algorithms for the PSB dataset.
Part of the table is due to [9]. Table 3 shows our kSA-based feature
aggregation algorithms outperform the existing SC-based methods,i.e., BF, SA, TE, KSVD, and LL. We try to explain the results byusing Table 1. TE has non-negativity but it lacks k-sparseness forfeature encoding. KSVD has k-sparseness (we used k=5 for theexperiment) and it learns codebook and feature encoding jointly.

Takahiko Furuya, Ryutarou Ohbuchi, Accurate Aggregation of Local Features by using K-sparse Autoencoder for 3D Model Retrieval, shortpaper, ACM International Conference on Multimedia Retrieval 2016 (ICMR 2016), June 6-9, New York, NY, USA. (2016)
However, feature encoding by KSVD could produce negative
coefficients. LL has both k-sparseness (we used k=15) and nonnegativity. But feature encoding would be not optimal because it
uses a codebook learned by k-means clustering. In comparison, the
DkSA and PkSA satisfy both k-sparseness and non-negativity offeature encoding. Also, DkSA and PkSA jointly optimize codebook
learning and feature encoding. PkSA outperforms DkSA for fiveout of six features in Table 3 as the former exploits reconstruction
error of local features for aggregation.
Table 3 also shows DkSA and PkSA outperform the state-of-the-art
HS-based aggregation methods for most cases. Joint optimization
framework of the proposed algorithms would produce moreaccurate aggregated features than the “greedy” approach adopted
by the HS-based methods in Table 3.
Table 3. Comparison of MAP [%] (PSB dataset).
POD
47.8
37.2
40.2
50.0
53.1
52.9
52.9
55.3
60.1
57.1
61.5

BF
SA
TE
KSVD
LL
FV
VL
SV
DM
DkSA
PkSA

DSIFT MO1SIFT
54.0
51.9
46.9
42.3
45.5
41.3
55.7
57.4
57.6
56.5
61.7
54.2
60.9
50.6
63.8
53.2
61.1
64.7
56.7
63.7
63.8
62.7

LSF
33.0

SI
38.1

RoPS
40.5

33.8
36.7
38.3
40.2
41.4
35.3
43.0

41.0
44.9
45.5
49.6
47.3
37.4
48.8

46.4
46.3
42.7
48.4
50.4
52.1
55.1

Figure 2 plots retrieval accuracies against the number ofdimensions for aggregated features. We extracted POD featuresfrom the 3D models. Both DkSA and PkSA show higher retrieval
accuracy than the state-of-the-art feature aggregation methods
including LL, VL, FV, and SV. Also, PkSA significantly
outperforms the DM which is one of the most accurate feature
aggregation algorithm for 3DMR [9].

Table 4. Comparison of MAP [%] (POD feature).
algorithms
BF
LL
FV
VL
SV
DM
DkSA
PkSA

PSB

ESB
47.8
53.1
52.9
52.9
55.3
60.1
57.1
61.5

52.1
53.0
54.4
49.8
54.0
57.2
57.4
59.4

SH11NR
87.3
94.7
96.7
95.9
96.5
95.8
96.0
96.4

SH14LC
39.6
44.7
44.8
44.0
45.3
50.7
47.7
50.9

Table 5. Computation time [s] for feature aggregation.
algorithms
VL
SV
DkSA
PkSA

pre-processing
41.5
342.6
798.6
226.9

feature aggregation per 3D model
0.04
0.08
0.29
0.75

both codebook learning and feature encoding. Computations of
DkSA and PkSA are accelerated by using a GPU. We use our own
implementations based on the CUDA library for DkSA and PkSA.
Table 5 compares efficiency of feature aggregation algorithms. In
the table, pre-processing for VL and SV mean learning a codebook
with 3k codewords. Meanwhile, pre-processing for DkSA and
PkSA includes whitening of local features and training kSA.
Number of hidden units n for DkSA and PkSA are set to 8k and 500
respectively. As for feature aggregation, we measured computation
time for aggregating 3k POD features per 3D model. DkSA and
PkSA are slower than VL and SV in aggregating local features.
PkSA is the slowest among the four algorithms since it requires
fine-tuning of kSA for aggregation. However, by the help of GPU,
computation time of aggregation for both DkSA and PkSA are
within 1 second, which is acceptable for 3DMR.
Querying a 3D model database by using the proposed algorithms is
also efficient. We measured computation time per query for the
SH14LC database which includes 8,987 3D models. A retrieval
consists of three processes, i.e., extracting local features from thegiven query 3D model, aggregating the local features of the query,and computing similarities among the query and the 3D models inthe database. 3DMR using DkSA took 0.61s per query and 3DMR
using PkSA took 1.12s per query.

6560
MAP [%]

Implementations of VL and SV are based on the VLFeat library[25]. They employ parallel computing on the multi-core CPUs for55
BF-POD
FV-POD
SV-POD
DkSA-POD

50
45
0100,000

200,000

LL-POD
VL-POD
DM-POD
PkSA-POD
300,000

Aggregated feature dimensons

Figure 2. Comparison of feature aggregation methods (PSB).
In the experiments above, we used the only one benchmark
database, i.e., the PSB. Table 4 shows generalization capability ofthe proposed algorithms for a variety of 3D model databases
described in Section 3.1. We use POD local feature for theexperiment. Table 4 shows the PkSA aggregation performs well,
but not always the best, among the previous state-of-the-art feature
aggregation algorithms.

3.2.2 Efficiency
We evaluate efficiency of the proposed kSA-based feature
aggregation algorithms. We use a PC having two Intel Xeon E52650V2 CPUs, 256GB DRAM, and a GeForce GTX 980 GPU.

4. CONCLUSION AND FUTURE WORKIn this paper, for more accurate aggregation of local features, we
proposed two novel feature aggregation algorithms DkSA and
PkSA that employ k-Sparse Autoencoder (kSA). These algorithms
jointly optimize codebook learning and feature encoding. Also,
feature encoding using the kSA can enhance saliency of local
features due to two constraints, i.e., k-sparseness and nonnegativity, on unit activation in the hidden layer of the kSA.
Furthermore, PkSA aggregation exploits reconstruction error of
local features per data object for better retrieval accuracy.
Quantitative evaluation using multiple local features and multiple
benchmarks for 3DMR showed that our proposed algorithms
perform equal or better than existing state-of-the-art feature
aggregation algorithms. As a future work, we will evaluate
effectiveness of the kSA-based aggregation under 2D image
retrieval or recognition setting.

Takahiko Furuya, Ryutarou Ohbuchi, Accurate Aggregation of Local Features by using K-sparse Autoencoder for 3D Model Retrieval, shortpaper, ACM International Conference on Multimedia Retrieval 2016 (ICMR 2016), June 6-9, New York, NY, USA. (2016)

5. ACKNOWLEDGMENTS
This research is supported by JSPS Grant-in-Aid for Young
Scientists (B) #16K16055 and JSPS Grant-in-Aid for Scientific
Research on Innovative Areas #26120517.

6. REFERENCES[1] Achlioptas, D. 2003. Database-friendly random projections:
Johnson-Lindenstrauss with binary coins. Journal ofComputer and System Sciences, 66(4), 671–687.
[2] Aharon, M., Elad, M., Bruckstein, A. 2006. K-SVD: An
Algorithm for Designing Overcomplete Dictionaries forSparse Representation. IEEE Transactions on Signal
Processing, 54(11), 4311–4322.
[3] Bell, A., Sejnowski, T.J. 1996. Edges are the `Independent
Components' of Natural Scenes. Proc. NIPS 1996.
[4] Coates, A., Ng, A.Y., Lee, H. 2011. An analysis of singlelayer networks in unsupervised feature learning. Proc.
AISTATS 2011, 215–223.
[5] Csurka, G. et al. 2004. Visual Categorization with Bags of
Keypoints, Proc. ECCV 2004 workshop on Statistical
Learning in Computer Vision, 59–74.
[6] Duchi, J., Hazan, E., Singer, Y. 2011. Adaptive subgradientmethods for online learning and stochastic optimization. The
Journal of Machine Learning Research, 12, 2121–2159.
[7] Furuya T., Ohbuchi. R. 2009, Dense sampling and fastencoding for 3D model retrieval using bag-of-visual features,
Proc. ACM CIVR 2009, Article No. 26.
[8] Furuya, T., Ohbuchi, R. 2014. Fusing Multiple Features for
Shape-based 3D Model Retrieval, Proc. BMVC 2014.
[9] Furuya, T., Ohbuchi, R. 2015. Diffusion-on-Manifold
Aggregation of Local Features for Shape-based 3D Model
Retrieval. Proc. ICMR 2015, 171–178.
[10] Gong, Y., Wang, L., Guo, R., Lazebnik, S. 2014. Multi-Scale
Orderless Pooling of Deep Convolutional Activation
Features. Proc. ECCV 2014, 392–407[11] Guo, Y. et al. 2013. Rotational Projection Statistics for 3D
Local Surface Description and Object Recognition, IJCV,
105(1), 63–86.
[12] Huang, Y., Wu, Z., Wang, L., Tan, T. 2014. Feature Coding
in Image Classification, A Comprehensive Study. IEEE
TPAMI, 36(3), 493–506.
[13] Jayanti, S., Kalyanaraman, Y., Iyer, N., Ramani, K. 2006.
Developing an engineering shape benchmark for CAD
models, Proc CAD, 38(9), 939–953.
[14] Jégou, H., Douze, M., Schmid, C., P. Perez. 2010.
Aggregating local descriptors into a compact image
representation, Proc. CVPR 2010, 3304–3311.
[15] Johnson, A.E., Hebert, M. 1999. Using spin images for
efficient object recognition in cluttered 3D scenes, PatternAnalysis and Machine Intelligence, 21(5), 433–449.
[16] Krizhevsky, A., Sutskever, I., Hinton, G.E. 2012. ImageNet
Classification with Deep Convolutional Neural Networks.
Proc. NIPS 2012, 1097–1105.
[17] Lian, Z. et al. 2011. SHREC'11 Track: Shape Retrieval on
Non-rigid 3D Watertight Meshes, Proc. EG 3DOR 2011, 79–88.
[18] Li, B. et al. 2014. Large Scale Comprehensive 3D Shape
Retrieval, Proc. EG 3DOR 2014, 131–140.
[19] Liu, L., Wang, L., Liu, X. 2011. In defense of softassignment coding, Proc. ICCV 2011, 2486–2493.

[20] Makhzani, A., Frey B. 2012. k-Sparse Autoencoders,arXiv:1312.5663.
[21] Ng, A.Y. 2011. Sparse auto-encoder. CS294A Lecture notes.
[22] Ohkita, Y., Ohishi, Y., Furuya, T., Ohbuchi, R. 2012. Nonrigid 3D Model Retrieval Using Set of Local Statistical
Features, Proc. ICME 2012 Workshop on Hot Topics in 3D
Multimedia, 593–598.
[23] Perronnin, F., Sánchez, J., Mensink, T. 2010. Improving thefisher kernel for large-scale image classification, Proc.
ECCV 2010, Part IV, 143–156.
[24] Shilane, P., Min, P., Kazhdan, M., Funkhouser, T. 2004. The
Princeton Shape Benchmark, Proc. SMI 2004, 167–178.
[25] Vedaldi, A., Fulkerson, B. 2010. Vlfeat: an open and portablelibrary of computer vision algorithms. Proc. ACM MM 2010,
1469–1472.
[26] Wang, J. et al. 2010, Locality-constrained Linear Coding for
Image Classification, Proc. CVPR 2010, 3360–3367.
[27] Zhou, X., Yu, K., Zhang, T., Huang, T.S. 2010. ImageClassification using Super-Vector Coding of Local Image
Descriptors, Proc. ECCV 2010, 141–154.