Low-Bit Representation of Linear Classifier Weights
for Mobile Large-Scale Image Classification
PS2-36

Yoshiyuki Kawano and Keiji Yanai
The University of Electro-Communications, Tokyo, Japan

ObjectiveImplementationx2x• Implement standalone large-scale
object recognition with 1000/10000 classes
on a mobile phone.
• Fisher vector and linear classifier• Multi-class classification: one-vs-rest

Local descriptors
・ RootHOG - gradient・ Color – moment

RootHoG

1-vs-rest
linearFisherclassifiers
Vectorwithquantized
Color patches
weights
Processing over 4 coresin parallel(We are working on DCNN-based mobile object recognition.
4bit is OK currently. We will present it at the other places.)

local patch: 16x16 and 24x24
dense sampling every 6 pixel

Feature vector (coding)
GMM: K=64/128/256
Spatial Pyramid: 1x1 + 2x2 (=5)(K=128 with no SP for 10k classification )Takes only 0.270 seconds(D: dim of FV, C:num. of classes)• Too many training parameters (𝐷 ∙ 𝐶)
• Limited memory and storage

Quadcore-CPU
smartphone

after classification

Multi-threads over quadcore(Galaxy S5 (2.6GHz, 4core, Android4.4.2))# class GMM SP Color-FV HoG-FV
Mem.
Mem.
(32bit float)

levelour findings• 2-bit representation of classifier weights
led to only slight performance loss.

1000
1000
1000
10184

64
128
256
128

222140960
20480
40960
4096

68MB 4.3MB
136MB 8.5MB
272MB 17.1MB
278MB 17.4MB• No problem on server scalability

Represent each element of all the weightvectors of all the linear classifiersin low bits by scalar-quantization
Distribution of weights

DatasetILSVRC2012(1000), ImageNet 10k

Modification of the way of quantization
helps improve performance with 2-bit.

𝑑Very-high-dimensional weight vectors
still exhibit discriminative power after
low-bit quantization.

We have resolve it !

Low-bit representation of
linear classifier weights(A linear SVM can be used as an alternative.)

Until 4-bit, no performance loss.
With 2-bit, slight loss .
With 1-bit, great loss.

メモリ量を1/8に削減、性能劣化は僅か• Quick response

AROW (online learning)with 1-vs-rest and late fusion

1000-class classification rate with different bits

・ 効果
• anywhere without internet• Large-size training parameters• Low computational power

Linear Classifier

Experimental results

Standalone mobile obj. rec.
Client-side recognition vs. server-side

76801536030720
3072(2bit)・ Color-FV (7680d/15360d/30720d)
・ HOG-FV (10240d/20480d/40960d)(DCNN has the same characteristics in general.)(old[2] : no subtraction of average with fixed 𝛼 = 1)

Performance with the same memory size
using different bits and GMM size

Processing time
(Galaxy S5 (2.4GHz, quadcore))

0.591 sec.

0.330 sec.
0.159 sec.

Linear classifier
𝑦 =𝒘∙𝒙2bit,6432 bit ⇒ 2 bit(1/16)
32bit(float),644bit,642bit,1281bit,2562bit,1281bit,256

1-bit quantization with 256-GMM
achieved the best result.
However, processing time forfeature extraction increased much.

(𝛼 = 2.2𝜎 (estimated by CV))value00

range for 2bit
− ∞ < 𝑤𝑖 < −𝛼01−𝛼 ≤ 𝑤𝑖 < 0100≤ 𝑤𝑖 < 𝛼11𝛼 ≤ 𝑤𝑖 < ∞The same 𝛼 is used for all the weights
For one-vs-rest classification,
reconstruction of original weights is not needed.

10k-class classification results with 2-bit
quantization and Product quantization (PQ)

Combination of PQ and 2-bit scalarquantization is more effective.
Difference to PQ: PQ needs to refera codebook table at evaluation time,
while the proposed method does not.

[2] Y. Kawano and K. Yanai. ILSVRC on a smartphone. IPSJ Trans. CVA, 6:83–87, 2014.
[3] H. Jegou, M. Douze, and C. Schmid. Product quantization for nearest neighbor search. IEEE Trans．PAMI, 33(1):117–128, 2011.