Inter-Image Communication for
Weakly Supervised Localization
Xiaolin Zhang, Yunchao Wei, Yi Yang

ReLER, AAII, University of Technology Sydney
http://arxiv.org/abs/2008.05096

ContentBackground
Motivation
Inter-Image Communication
Experiment

Weakly Supervised Localization

Convolution

GAP…

SoftMax

GAP

Convolutional Neural Network

CrossEntropy

Bird

Infer

Global Average Pooling

Localization Map

Bounding Boxo Given an image-level label, WSL is to find the location of the target object.
o In practice, classification networks are trained to extract localization maps. Bounding boxes areinferred offline.
o The fundamental target of WSL is to improve the localization maps.

Methods for extracting localization maps
CAMConv1*1…

FCN

clsFeaturemaps

GAP cls…

Featuremaps

FC

…

…

FCN

GAP

ACoL

SelectZhou, B., Khosla, A., A., L., Oliva, A., Torralba, A.: Learning Deep Features for Discriminative Localization. IEEE CVPR (2016)
Zhang, X., Wei, Y., Feng, J., Yang, Y., Huang, T.: Adversarial complementary learning for weakly supervised object localization. In: IEEECVPR (2018)

Motivation
Current
birdObservations and Facts• Some regions belonging to the object are nothighlighted• In general, vectors in orange/purple are responsible
to the scores in orange/purple.
• Images are processed independently
Intuition of I2C𝑓!

• Push the vectors belonging to the same object close• Push the object vectors across images in a same
category close (Inter-image communication)
Zhang, X., Wei, Y., Kang, G., Yang, Y., Huang, T.: Self-produced guidance for weaklysupervised object localization. In: ECCV. Springer (2018)

Ourbirdcar

Classification Net

Class Centers….

….

Global Consistency(GC)Classification Net

Stochastic Consistency (SC)

Stochastic Consistency (SC)• Push feature vectors of the objects
from two images close• Operate within a minibatch
Global Consistency (GC)
• Push feature vectors approach theirclass centers• One center vector for each category• Operate across minibatches

Backbone

…

𝑓!
CNNs

GAP

Softmax

Cross-entropy
Loss

#𝑀"

#𝐼"

𝐹"

Object
Seeds

𝐹$

#

CNNs

Backbone
𝑓!
#𝐼$

…

𝑀$

GAP

Softmax

Cross-entropy
LossBackbone

…

𝑓!
CNNs

GAP

Softmax

Cross-entropy
Loss

#𝑀"

#𝐼"

#𝑉"

𝐹"

SC Loss

Select object vectors#𝑉$𝐹$#CNNsBackbone
𝑓!
#𝐼$

…

𝑀$

GAP

Softmax

Cross-entropy
LossBackbone

CNNs

…

𝑓!
GAP

Softmax

Cross-entropy
Loss

#𝑀"

#𝐼"

#𝑉"

𝐹"

SC Loss

Select object vectors
#𝑉$

𝐹$

AVG

GC Loss𝑎#

CNNs

Backbone
𝑓!
#

𝐼$

…

𝑤#
GAP

#𝑀$

Softmax

Cross-entropy
Loss

…

𝑓!
CNNs

GAP

Softmax

Cross-entropy
Loss

#𝑀"

#𝑉"

𝐹"

SC Loss

Select object vectors
#

𝑉$

𝐹$

AVG

#𝑓!

…

𝑀$
CNNs

GC Loss𝑎#𝑤#
GAP

Softmax

Cross-entropy
LossObject Seed

Ours

ACoL

Image

Visualization Comparison

Predicted boxes are in green, while Ground-truth boxes are in red.

Comparison of Localization Error on ILSVRCComparison of Localization Error on CUB

Are SC and GC really effective?

How do the hyper-variables affect the localization
performance?

𝜆!
Classification

Localization

Gt-Known LocalizationΔ(𝑒𝑟𝑟) =0.27

×40×40×40

𝜆"Classification

Localization

×1000

×1000

Gt-Known Localization

×1000

𝐾
Classification

Localization

Gt-Known Localization

×33×33×33

Thank You