Drop-Bottleneck: Learning Discrete Compressed Representation for Noise-Robust Exploration
Authors: Jaekyeom Kim
Minjung Kim
Dongyeon Woo
Gunhee Kim
Venue: ICLR 2021 Poster (2021)
Keywords: Reinforcement learning
Information bottleneck

Abstract: We propose a novel information bottleneck (IB) method named Drop-Bottleneck, which discretely drops features that are irrelevant to the target variable. Drop-Bottleneck not only enjoys a simple and tractable compression objective but also additionally provides a deterministic compressed representation of the input variable, which is useful for inference tasks that require consistent representation. Moreover, it can jointly learn a feature extractor and select features considering each feature dimension's relevance to the target task, which is unattainable by most neural network-based IB methods. We propose an exploration method based on Drop-Bottleneck for reinforcement learning tasks. In a multitude of noisy and reward sparse maze navigation tasks in VizDoom (Kempka et al., 2016) and DMLab (Beattie et al., 2016), our exploration method achieves state-of-the-art performance. As a new IB framework, we demonstrate that Drop-Bottleneck outperforms Variational Information Bottleneck (VIB) (Alemi et al., 2017) in multiple aspects including adversarial robustness and dimensionality reduction.

Published as a conference paper at ICLR 2021
Drop-Bottleneck: LEARNING DISCRETE COMPRESSED REPRESENTATION FOR NOISE-ROBUST EXPLORATION
Jaekyeom Kim, Minjung Kim, Dongyeon Woo & Gunhee Kim Department of Computer Science and Engineering Seoul National University, Seoul, Republic of Korea
jaekyeomBsmu.cet kr, mi n jung. kim snu kr. (oody0325 http://vision. snu. ac. /tt.kr/et/s////ttt
gunhee @snu kr
ABSTRACT
We propose novel information bottleneck (IB) method named Drop-Bottlenekk, which discretely drops features that are irrelevant to the target variable. Drop- Bottleneck not only enjoys simple and tractable compression objective but also additionally provides a deterministic compressed representation the input vari- able, which is useful for inference tasks that require consistent representation. Moreover, it can jointly learn a feature extractor and select features considering each feature dimension's relevance the target task, which is unattainable by most neural on network- -based IB methods. We propose an exploration method based for reinforcement learning tasks. In a multitude of noisy and reward sparse maze navigation tasks in VizDoom (Kempka et al. (2016) and DM- Lab (Beattie al., 2016), our exploration method achieves tfate-of-the-ar per- formance. As a new IB framework, we demonstrate that forms Variational Information Bottleneck (VIB) (Alemi et al.) (207)) in multiple aspects including adversarial robustness and dimensionality reduction.
outper-
INTRODUCTION
Data with noise or noisy-TV problem (Burda et al. (019a) one of well-known such phenomena in reinforcement learning. If observations from the environment are modified to contain a TVV screen, which changes its channel randomly based on the agent's actions, the performance of curiosity- based exploration
information easily harm the training of model; for instance, the
methods dramatically degrades (Burda et al.) 2019a (b]
Kim et al. 2019 Savinov et al. 2019).
The information bottleneck (IB) theory (Tishyy et al. 2000 Tishby & Zaslavsky 2015) provides framework for dealing with such task-irrelevant information, and has been actively adopted to ex- ploration in reinforcement learning (Kim et al. 2019 [gl et al. (009).. For input variable X and target variable Y. the IB theory introduces another variable Z. which is a compressed representation of X. The IB objective trains Z to contain less information about X but more information about Y as possible, where the two are quantified by mutual information terms of I (Z and [(Z; Y), re- spectively. IB methods such Variational Information Bottleneck (VIB) (Alemi et al. 2017, Chalk et al. 2016) and Information Dropout (Achille & Soatto 2018) show that the compression of the input variable X can be done by neural networks.
In this work, propose novel information bottleneck method named hoop-Bottleneck that com- presses the input variable by discretely dropping a subset of its input features that are irrelevant to the target variable. Drop-Bottleneck Bottleneck provides some nice properties follows:
The compression term of Drop-Bottie Boop-Bottleneck's objective is simple and is optimized tractable solution.
Prop-Bottleneck provides a deterministic compressed representation that still maintains majority of the learned indistinguishability i.e. compression. It is useful for inference tasks that require the input representation to be consistent and stable.
Prop-Bottleneck jointly trains feature extractor and performs feature selection, as it learss the feature-wise drop probability taking into account each feature dimension's 's relevance to the target task. Hence, unlike the compression provided by most neural network-based IB

---

Published as a conference paper at ICLR 2021
methods, our deterministic representation reduces the feature dimensionality, which makes the following inference better efficient with less amount of data.
Compared to VIB, both of Prop-Bottleneck's ' original (stochastic) and deterministic com- pressed representations can greatly improve the robustness to adversarial examples.
Based on the newly proposed Drop-Bottlenech we design an exploration method that is robust against noisy observations in very sparse reward environments for reinforcement learning. Our ex- ploration maintains an episodic memory and generates intrinsic rewards based on the predictability of new observations from the compressed representations of the ones in the memory. As result, our method achieves state-of- the-art performance on multiple environments of VizDoom (Kempka et al., 2016) and DMLab (Beattie al. 2016). We also show that combining our exploration method with VIB instead of Prop-Bottleneck degrades the performance by meaningful margins.
Additionally, we empirically compare with VIB to show Drop-Bottlenecks superior robustness to adversarial examples and ability to reduce feature dimensionality for inference with ImageNet dataset (Russakovsky et al.) 2015). We also demonstrate that Drop-Bottleneck" deterministic rep- resentation can be a reasonable replacement for its original representation in terms of the learned indistingusshability, with Occluded CIFAR dataset (Achille & Soatto 2018).
RELATED WORK
2.1
INFORMATION BOTTLENECK METHIOSS
There have been a number of IB methods that approximations special forms of the original IB objective. Variational Information Bottleneck (VIB) (Alemi et al. (017) approximates the original IB objective by establishing variational bounds on the compression and prediction terms. Chalk al.) 2016) propose the same variational bound on the IB objective in the context of sparse coding tasks. Conditional Entropy Bottleneck (CEB) and Variational Conditional Entropy Bottleneck (VCEB) (Fischer 2020; Fischer& & Alemi 2020 use an alternative form of the original IB objective derived under the Minimum Necessary Information (MNI) criterion preserve only necessary amount of information. The IB theory (Tishby et al.) 2000) has been used for various problems that require restriction of information or dealing with task-irrelevant information. Information Dropout (Achille) & Soatto| 2018) relates the IB principle to multiple practices in deep learning, including Dropout, disentanglement and variational autoencoding. Moyer et al. (2018) obtain representations invariant to specific factors under the variational autoencoder (VAE) (Kingma & Welling (013)) and VIB frameworks. [Amjad & Geiger (2019) discuss the use of IB theory for classification tasks from theoretical point of view. Dai al. (2018) employ IB theory for compressing neural networks by pruning neurons in networks. |Schulz et al. (2020) propose an attribution method that determines each input feature's importance by enforcing compression of the input variable via the IB framework.
Similar to our goal, some previous research has proposed variants of the original IB objective. De- terministic information bottleneck (DIB) (Strouse & Schwab, (017)) replaces the compression term with an entropy term and solves the new objective using a deterministic encoder. Nonlinear in- formation bottleneck (NIB) (Kolchinsky et al. (019)) modifies the IB objective by squaring the compression term and uses a non non-parametric upper bound on the compression term. While DIB is always in the deterministic form, we can flexibly choose the stochastic one for training and the deterministic one for test. Compared NIB, which is more computationally demanding than VIB due to its non-parametric upper bound, our method is faster.
2.2 REINFORCEMENT LEARNING WITH INFORMATION BOTTLENECK METHODS
The IB theory has been applied to several reinforcement learning (RL) tasks. Variational discrimi- nator bottleneck (Peng et al. 2019) regulates the discriminator's accuracy using the IB objective to improve adversarial training. and use it for imitation learning. Information Bottleneck Actor Critic (ggl et al., 2019) employs VIB to make the features generalize better and encourage the compres- sion of states input to the actor-critic algorithm. the VIB framework to train a compressor that compresses the representation of states, which is still informative about the value function, and uses the compressiveness exploration signals. In- foBot (Goyal et al. 2019) proposes a conditional version of VIB to improve the transferability of a goal-conditioned policy by minimizing the policy's 's dependence on the goal. Variational bandwidth
suriosity-Bottlenedd (Kim et al. 2019) employs
2

---

Published as a conference paper at ICLR 2021
bottleneck (Goyal al. 2020) uses modified, conditional version of VIB, and solves RL tasks with privileged inputs (i.e. valuable information that comes with a cost). Our exploration method differs from these methods in two aspects. First, we propose new information bottleneck method that is not limited to exploration in RL but generally applicable to the problems for which the IB theory is used. Second, method generates exploration signals based the noise-robust predictability i.e. the predictability between noise-robust representations of observations.
PROP-BOTTLENECK
3.1 PRELIMINARIES OF INFORMATION BOTTLENECK
Given an input random variable X, the information bottleneck (IB) framework (Tishby et al. 2000
Tishby & Zaslavsky 2015) formalizes problem of obtaining X's compressed representation Z, which still and only preserves information relevant to the target variable Y. Its objective function is
minimize - (2; + (I;
(1)
where 3 is a Lagrangian multiplier. The first and second terms are prediction and compression terms, respectively. The prediction term I(Z; Y) encourages
preserve task-relevant information while the compression term (Z; X) compresses the input information as much as possible.
As reviewed in the previous section, there have been several IB methods (Alemi et al. 2017 Chalk
al., 2016 Achille Soatto| 2018 Strouse Schwab 2017; Kolchinsyy al. 2019). among
which the ones derived using variational inference such as Variational Information Bottleneck (VIB)
(Alemi al. 2017) have become dominant due to applicability to general problems.
3.2
BOTTLEEOT
We propose novel information bottleneck method called Drop- Coop-Botteeeck (DB), where the input information is compressed by discretely dropping a subset input features. Thus, its compres- sion objective is simple and easy to optimize. Moreover, its representation is easily convertible to deterministic version for inference tasks (Section 3.3). and allows joint training with a feature ex- tractor (Section 3.4). While discrete dropping of features has been explored by prior works including Dropout (Srivastava et al. 2014), DB differs in that its goal is to assign different drop probabilities feature variables based their relevance to the target variable.
For input variable X [X1,. Xa] and drop probability p [p1..... Pal 1] we define its compressed representation as 7 = Cp( = (c(X1,P1). ..l.........c......
c( ((Xa.Pa)) pa)] such that
c(X X p.) = Bernoulli(1 - p:)) X,, where b - d d-
for i = 1,...,,, d. That is, the compression procedure drops the i-th input feature (i.e. replaced by zero) with probability p;. Since the drop probability is to be learned, we use a scaling factor b to keep the scale of Z constant. We use a single scaling factor for all feature dimensions in order to preserve the relative scales between the features.
With the definition in Equation
2), we derive the compression term of DB to minimize as
1(2:))
1 1(2 X XalZI..... Zz-1)
(3)
[[[ X; Zi...., Z(-1) + I(Z;; X1, XI,..,, X 1/ 2, 2,,1,
(4)
.... .2,1))1)) < Σ I(Z;:X,) = i(Z:X)
(5)
km1
1m
using that Z, X X,+1..... X1 .,.,................... Z, 1, X, and Z..ZZ..... 1X, Note that f(z:x) I(Z;X) Ei (Z.)) H Za) TC(Z) where TC(Z) is the
total correlation of and H(-) denotes the entropy, and i(Z;X) = I(Z:X) Xf..,,,, are in- dependent. To provide rough view the gap, due to the joint optimization with the compression
3

---

Published as a conference paper at ICLR 2021
term f(z; X) and the prediction term [[ Y), Z becomes likely preserve less redundant and less correlated features, and (C) could decrease as the optimization progresses.
Finally, DB's new compression term, 1 f(z; X), is computed
d
i(Z:X) = Σ ((, Σ (H())) H(x))2))
(6)
i=1
(X,,) H.-H((,l1) 0) - (1-PP) HX,, = =x,))
(7)
(H(),) P (X,) (1-p:).) p)) Σ H(X;) - pt).
(8)
From Equation Equation (8) we use the two ideas: (i) HX,; = 0) (X,) because
Z; = 0 means it contains no information about X,, and (ii) (X; 6x; means Zi preserves the feature (i.e. Z, fully identifies X2) and thus their conditional entropy becomes zero. Importantly, DB's compression term is computed as the simple tractable expression in Equation (8). As the goal of the compression term is to penalize T [Z; not H(X), the drop probability p is the only parameter optimized with our compression term. Thus, each H (X;) can be computed with any entropy estimation method such as the binning- maased estimation, which involves quantization for continuous X,, since the computation has need to be differentiable.
bXx)) = O because Z; =
However, one issue of Equation (8) is that Z is not differentiable with respect to p Bernoulli distributions are not differentiable. We thus use the Concrete relaxation (Maddison et al. 2017 Jang al. 2016 of the Bernoulli distribution update p via gradients from Z:
Bernoullipp) x= (Λ (logp log(1 p) log - log(1 - -2)) =
(9)
where Uniform (0, 1) and À is temperature for the Concrete distribution. Intuitively, p is trained to assign a high drop probability to the feature that is irrelevant to or redundant for predicting the target variable Y.
3.3 DETERMINISTIC COMPRESSED REPRESENTATION
With Drop-Bottleneck, one can simply obtain the deterministic version of the compressed represen- tation as Z (X) [o(XI.P)). ...., e(Xa, Pa)] for
d
p(X., = b. (p; < 0.5) Xi, where b Σx (pa < 0.5)
(10)
1 is defined similarly to with a minor exception that the scaling is done based on the actual, deter- ministic number of the dropped features. The deterministic compressed representation Z useful for inference tasks that require stability or consistency of the representation as well as reducing the feature dimensionality for inference, we demonstrate in Section 5.4
3.4 TRAINING WITH DROP-BOTTLENECK
We present how Drop-Bottenecc (DB) is trained with the full IB objective allowing joint training with a feature extractor. Since DB proposes only a new compression term, any existing method for maximizing the prediction term [(2; Y) can be adopted. We below discuss an example with Deep Infomax (Hjelm et al. 2019) since our exploration method uses this framework (Section 4 Deep Infomax, instead of T ((;Y). maximizes its Sensen-Shannon mutual information estimator
ISD (Z:Y)= V
(Z, Ý))] - Erzery (Z, + log log4)
(11)
2
where T& is discriminator network with parameter ab and ((-) is the softplus function. Finally, the IB objective with Drop- Poop-Bottleneck becomes
minimize -1/50(Z;Y)
(Z;Y) 3 H(X;) p:),
(12)
4

---

Published as a conference paper at ICLR 2021
which be optimized via gradient descent. To make p more freely trainable, suggest simple element- parameterization of p as p; = (p,) and initializing pl ~ Uniform(a, a, b). We obtain the input variable X from feature extractor, whose parameters trained via the prediction term, jointly with and sb. In next section, we will discuss its application to hard exploration problems for reinforcement learning.
4 ROBUST EXPLORATION WITH DROP-BOTTLENECK
Based on DB, we propose an exploration method that is robust against highly noisy observations in very sparse reward environment for reinforcement learning tasks. We first define a parametric feature extractor fe that maps state to X. For transitions A, S') where S, A, and S' are current states, actions and next states, respectively, we use the DB objective with
== fo'")"
Z Cp(), Y T=C(ff((S).
(13)
For every transition (S, A, S'). : the compression term I - IICCp(fo(S) f(())) encourages Cp to drop unnecessary features of the next state embedding fa(S") as possible. The prediction term Y(Z;Y) = (())) (p)) makes the compressed representations of the current and next state embeddings, (p)) and (p(fo(S"), informative about each other.
Applying Equation 133 to Equation
12). the Drop-Bottleneck objective for exploration becomes
minimize - 7S5D 1ASP (Cp(U Cp((s))))))(()))) + Σ H(((((S))))(( -p:)-
(14)
While fe. p. and T3 are being trained online, we use them for the agent's exploration with the help of episodic memory inspired by Savinov et al.] (2019). Starting from an empty episodic memory M, we add the learned feature of the observation each step. For example, time step f, the episodic Cp Cp(fa(st11) observations from that episode. We then quantify the degree of novelty of a new observation as an intrinsic reward. Specifically, the intrinsic reward for is computed utilizing the Deep Infomax
memory is . = (Cp(fo(s)), Cp(ff(82))....
where 8 8t-1
are the earlier
discriminator Tes which trained to output a large value for joint (or likely) input and a small value for marginal (or arbitrary) input:
(s)) t
1
Σ (9(s,, 8y) + g(( st)l s.t. g(x,y) ( ((-T)CC)(((())
(15)
where g(8,,8) 84) and g(5.. 8)) denote the unlikeliness of 8 being the next and the previous state of e.- respectively. Thus, intuitively, for s, that close to a region covered by the carlier observations in the state space, r (st) becomes low, and vice versa. results in solid exploration method capable of handling noisy environments with very sparse rewards. For computing the intrinsic reward, we the DB's deterministic compressed representation of states to provide stable exploration signals to the policy optimization.
EXPERIMENTS
We carry out three types experiments to evaluate Drop-Bottlenech (DB) in multiple aspects. First, we apply DB exploration to multiple VizDoom (Kempka et al. 2016) and DMLab (Beattic et al.) 2016) environments with three hard noise settings, where we compare with state-of-the-art methods as well as its VIB variant. Second, we empirically show that DB is superior to VIB for adversar- ial robustness and feature dimensionality reduction in ImageNet classification (Russakovsky et al.) 2015), and we juxtapose DB with VCEB, which employs a different form of the IB object, for the same adversarial robustness test, in Appendix A Finally, in Appendix B we make another compari- with VIB in terms of removal of task-irrelevan information and the validity of the deterministic compressed representation, where Appendix provides the visualization of the iaskiirreeevant in- formation removal on the same task.
5.1 EXPERIMENTAL SETUP FOR EXPLORATION TASKS
For the exploration tasks on VizDoom (emmkka et al. 2016) and DMLab (Beattic et al. 2016), the proximal policy optimization (PPO) algorithm (Schulman et al. 2017) as the base RI.
5

---

Published as a conference paper at ICLR 2021
method. We employ ICM from Pathak et al. (2017), and EC and ECO from Savinov al. (2019) as baseline exploration methods. ICM learns a dynamics model of the environment and uses the prediction errors exploration signals for the agent. EC and ECO are curiosity methods that use episodic memory to produce novelty bonuses according to the reachability of new observations, and show the state-of-the-art exploration performance summary, we compare with four baseline methods: PPO, PPO + ICM, PPO + EC, and PPO ECO. Additionally, we report the performance of our method combined with VIB instead of DB.
VizDoom and DMLab navigation tasks. In
We conduct experiments with three versions of noisy-TV settings named "Image Action", "Noise" and "Noise Action", as proposed in Savinov et al. (2019). We present more details of noise and their observation examples in Appendix E. Following Savinov et al. 2019)) we resize observations as 160 x 120 only for the episodic curiosity module while as 84x x 84 for the PPO policy and exploration methods. We use the official source code of Savinov et al. (2019) to implement and configure the baselines (ICM, EC and ECO) and the three noise settings.
For the feature extractor fe. we use the same CNN with the policy network of PPO from Mnih al. (2013). The only modification is to use d 128 i.e. 128-dimensional features instead of 5T2 to make features lightweight enough to be stored in the episodic memory. The Deep Infomax diseriminator To consists of three FC hidden layers with 64, 32, 16 units each, followed by a final FC layer for prediction. We initialize the drop probability p with p; a (p²) and p¹p Uniform(a, b)
where a = -2, 1. We collect samples and update p, Tip fa with Equation (14) every 10.8K and 21.6K time steps in VizDoom and DMLab, respectively. with batch size of 512. We duplicate the compressed representation 50 times with differently sampled drop masks, which help better training of the feature extractor, the drop probability and the discriminator by forming diverse subsets of features. Please refer to Appendix B for more details of our experimental setup.
5.2 EXPLORATION IN NOISY STATIC MAZE ENVIRONMENTS
VizDoom (eempka et al. 2016) provides a static 3D maze environment. We experiment on the My- WayHome task with nine different settings by combining three reward conditions ("Dense", "Sparse" and "Very Sparse") in Pathak et al.] (017)) and three noise settings in the previous section. In the "Dense", "Sparse" and "Very Sparse cases, the agent is randomly spawned in near, medium and very distant room, respectively. Thus, "Dense" is a relatively easy task for the agent to reach the goal even with a short random walk, while "Sparse" and "Very Sparse" require the agent to perform a series of directed actions, which make the goal-oriented navigation difficult.
Table T and Figure compare the DB exploration with three baselines, PPO, PPO + ICM, and PPO ECO on the VizDoom tasks, in terms of the final performance and how quickly they learn. The results suggest that even in the static maze environments, the three noise settings can degrade the performance of the exploration by large margins. On the other hand, our method with DB shows robustness to such noise or task-irrelevant information, and outperforms the baselines in all the nine challenging tasks, whereas our method combined with VIB does not exhibit competitive results.
5.3 EXPL.ORATION NOISY AND RANDOMLY GENERATED MAZE ENVIRONMENTS
As more challenging exploration task, we employ DMLab (Savinov al.) 2019), which are general and randomly generated maze environments where at the beginning of every episode, each maze is procedurally generated with random goal location. Thanks to the random map generator, each method is evaluated on test mazes that are independent of training mazes. As done in [Svvinov et al.) (219)) we test with six settings according to two reward conditions ("Sparse" and "Very Sparse") and the three noise settings. In the "Sparse" scenario, the agent is (re-)spawned at a random location when each episode begins or every time it reaches the goal i.e. the sparse reward; the agent should reach the goal as many times possible within the fixed episode length. The "Very Sparse" is its harder version: the agent does not get (re-) (re-)spawned near or the same room with the goal.
Table I compares between different exploration methods on the DMLab tasks. The results demon- strate that our DB exploration method achieves state-of-the tate-of-thee-rt performance with significant margins from the baselines on all the 6 tasks, and performs better than our method equipped with VIB as well. The plots too suggest that our method provides stable exploration signals to the agent under different environmental and noise settings. As mentioned in Section 5.1 our method can achieve better per-
https://gi thb.. com goog| e- resea /ep sodi C- curiosity

---

Published as a conference paper at ICLR 2021
Table Comparison of the average episodic sum of rewards in VizDoom (over runs) and DMLab (over 30 runs) under three noise settings: Image Action (IA), Noise (N) and Noise Action (NA). The values are measured after 6M and 20M action-repeated) steps for VizDoom and DMLab respectively, with no seed tuning done. Baseline results for DMLab are cited from Savinov et al.) 2019). Grid Oracle provides the performance upper bound by the oracle method for DMLab tasks.
VizDoom
DMLab
Method
Dense
Sparse
Very Spurse
Sparse
Very Spurse
IA
NA
IA
NA
1A
NA
1A
NA
IA
PPO (Schilman
7017
1,00
1.00
1.00
0.90
0.00
0.00
0.00
0.00
8.5
11.6
9.8
6.3
8.7
PPO FICM Famak ef al 2017
0.87
1.00
0.00
0.50
0.40
0.00
0.73
0.20
6.9
7.6
4.9
5.7
PPO EC (Savisov et 2019
13.1
18.7
14.3
7.4
13.4
11.3
Savinoo
12.5
PPO + Ours (VIB)
1,00
1.00
1.00
0.21
0.61
0.40
0.37
0.70
0.67
28.2
31.9
27.1
23.5
25.4
22.3
PPO Ours (Drop Bottleeck)
1,00
1.00
1.00
0.90
1.00
0.99
0.90
1,00
0.90
30.4
32.7
30.6
28.8
29.1
26.9
PPO Grid Oracle'
37.4
38.8
39.3
36,3
35.5
35.4
Valines Thame Aakia
Yallow Saae
Valliae Moie Ardie
Dillat Trags Araer
Doltas New
DoBLab Skew Jetna
(a)
(b)
280
(c)
720 Oues (V)))
(Deup Bartlaask)
Figure I: Reward trajectories as a function of training step (in millions) for VizDoom (columns 1-3) and DMLab (columns 4-6) with (a) Very Sparse, (b) Sparse and (c) Dense settings. For VizDoom tasks, we show all the 10 runs per method. For DMLab tasks, we show the averaged episode rewards over 30 runs of our exploration with the 95% confidence intervals.
1/4
SAF
13M
11M
ISM
2017
0.9
0.5
0.0
0.6
0.8
0.6
Figure 2: Evolution examples of the drop probability distribution p on Very Sparse DMLab envi- ronments with (left) Image Action, (middle) Noise and (right) Noise Action settings. Each figure shows a histogram per p value according to training iterations (the more front is the more recent).
formance even using much lower resolution observations of 84 x 84 than 160 120 of EC and ECO. Also, excluding the policy network, our method maintains 0.5M parameters, which significantly smaller compared ECO with 13M parameters. Please refer to Appendix D for an ablation study,
Figure 2 shows evolution examples of the drop probability distribution over training time steps. It overviews the role of drop probability p in DB. As the joint training of the feature extractor fo with p progresses, gets separated into high- and low-value groups, where the former drops task-inrelevat or redundant features and the latter preserves rask-relevant features. This suggests that in the DMLab environments, the DB objective of Equation (14) successfully encourages dropping the features unrelated to transition between observations and also the deterministic compressed representation becomes stable as the training progresses.
7

---

Published as a conference paper at ICLR 2021
Table Results of the adversarial robustness for Drop- Bottleneck (DB) and Variational Information Bottle- neck (VIB) (Alemi et al. 2017) with the targeted & and Lo attacks (Carlini & Wagner, (017). Succ. de- notes the attack success rate % (lower is better), and Dist. is the average perturbation distance over success- ful adversarial examples (higher is better).
- V1D
Orignal fostice dimestioality
DB
Fosture diarmiiannii used Dd (diterm)
DD (detm.))
80
1,024
1,024
Numbor of fostures
768
Accuracy (56)
70-
512
Attack 8
VIB
DB
DB (determ.)
206
type
Succ.
Dist.
Suce.
Dist.
Suce.
19
8
0.0001
99.5
0.806
100.0
0.929
99.5
0.923
60
0,0003162
99.5
0.751
100.0
0.911
100.0
0.911
10
10
10
10
10
99.5
1.097
100.0
1.134
12
0.003162
99.5
0.842
27.0
3.434
23.0
2.565
0.01
100.0
0.936
18.5
6.847
20.0
6.551
0,03162
100.0
0.695
11.0
2.160
39.5
1.953
Figure 3: Classification accuracy of
0.1
99.5
0.874
85,5
2.850
85.5
2.348
Dnception-ResNeee equipped with VIB (Alemi et al. 2017) and DB on ImageNet
0.0001
99.5
0.015
91.0
0.013
95.5
0.009
0.0003162 62
99.5
0.017
85.0
0.016
91.5
0.009
validation set (Russakovsky et al. 2015).
0.001
100.0
0.017
62.5
0.020
70.0
0.012
6.
0.003162
97.5
0.017
1.5
0.009
1.5
0.020
DB (determ.) quickly drops many feature
0.01
87.0
0.019
2.0
0.022
2.0
0.013
dimensions with increased B, while VIB
0.03162
25.0
0.121
8.5
0.022
8.0
0.023
retains them at 1024 regardless of B.
0.1
15.5
0.202
23.0
0.017
23.0
0.019
5.4 COMPARISON WITH VIB: ADVERSARIAL ROBUSTNESS & DIMENSION REDUCTION
We experiment with image classification on ImageNet (Russakovsky et al. 2015) to compare Drop- Bottleneck (DB) with Variational Information Bottleneck (VIB) (Alemi et al. 2017), the most widely-used IB framework, regarding the robustness to adversarial attacks and the reduction of feature dimensionality. We follow the experimental setup from Alemi et al.) (0117) with some ex- ceptions. We use 31 = 0.9 and no learning rate decay for DB's s Adam optimizer (Kingma & Bal 2013). For prediction, we use one Monte Carlo sample of each stochastic representation. Addition- ally, we provide a similar comparison with the mutual information based feature selection method.
Robustness to adversarial attacks. Following Alemi et al. (017). we employ the targeted & and to adversarial attacks from Carlini & Wagner (2017). For each method, we determine the first 200 validation images on ImageNet that are classified correctly, and apply the attacks to them by selecting uniformly random attack target classes. Please refer to Appendix E..2 for further details.
Table 2 shows the results. For the targeted & attacks, choosing the value of 8 from [0.003162, 0.1] provides the improved robustness of DB with the maximum at 3 = 0.01. On the other hand, VIB has no improved robustness in all ranges of B. For the targeted fxc attacks, DB can reduce the attack success rate even near to 0% (e.g. 3 = 0.003162 or 0.01). Although VIB decreases the attack success rate to 15.5% at 3 0.1, VIB already suffers from the performance degradation at B = 0.1 compared to DB (Figure 3) and increasing 33 accelerates VIB's s degradation even further. Note that the validation accuracies of both VIB and DB are close zero at 3 0.3162.
Dimensionality reduction. Figure 3 compares the accuracy of DB and VIB by varying 3 on the ImageNet validation set. Overall, their accuracies develop similarly with respect to B3 while VIB is slightly better in the lower range of 3. DB produces better accuracy in the higher range of B. Note that DB (determ.) shows the almost identical accuracy plot with DB. Importantly. DB (determ.) still achieves a reasonable validation accuracy (2 75%) using only a few feature dimensions (e.g. 8) out of the original 1024 dimensions. This suggests that DB's deterministic compressed representation can greatly reduce the feature dimensionality for inference with only a small trade-off with the performance. It is useful for improving the efficiency of the model after the training is complete. On the other hand, VIB has no such capability. Finally, as Figure 3 shows, the trade-off between the dimensionality reduction and the performance can be controlled by the value of B.
Comparison with feature selection. As the deterministic representation of DB, DB (determ.), pro- vides the dimensionality reduction, we also empirically compare DB with the univariate mutual information- based feature selection method for obtaining the feature space with reduced dimen- sionality. In the experiments, the same features provided to DB and VIB are given as input to the feature selection method well, and for more straightforward comparison, we let the feature
8

---

Published as a conference paper at ICLR 2021
Table Results of the adversarial robustness for Drop- Bottleneck (DB) and the mutual pooommtiion-basee fea- ture selection with the targeted lg and &o attacks (Car- lini & Wagner) 2017), using the same number of fea- tures. Succ. denotes the attack success rate in (lower is better), and Dist. is the average perturbation distance over successful adversarial examples (higher is better).
Accuraey (%)
40
Attack of type features
MI-based FS
DB (determ.)
Suce.
Dist.
Suce.
Dist.
+ MI-bused feature selection
196
99.5
1.484
99.5
0.923
* DB (determ.)
70
100.0
1.323
100.0
0.941
19
99.5
1.161
100.0
1.134
99.5
1.164
20.0
6.551
2
2
2
2
2
2"
2
97.0
1.202
39.5
1.953
Number of features
97.0
1.127
85.5
2.348
Figure 4: Classification accuracy of Inception-ResNCN equipped with the mutual information-based feature selec- tion and DB on ImageNet validation set (Russakovsky et al. (2015), using the same number of features.
196
99.5
0.016
95.5
0.009
70
100.0
0.014
91.5
0.009
19
99.5
0.013
70.0
0.012
Foo
99.5
0.014
2.0
0.013
97.0
0.016
8.0
0.023
97.0
0.015
23.0
0.019
lection method preserve the same number of features as DB (determ.). Refer to Appendix for further details of the feature selection procedure. Figure 4 shows the classification accuracy of the two methods for the same numbers of features. The results show that while the mutual information- based feature selection method could provide a marginal performance benefit when larger subset of the pre-trained features preserved, DB significantly better at retaining the accuracy with a small number of feature dimensions. For instance, DB achieves the accuracy over 71% even with features, but the accuracy feature selection method drops from R 68% to z 10% when the number of features is < 28. Also, make comparison of the adversarial robustness; Table suggests that the features preserved with the feature selection method show almost no robustness to the targeted la and &x attacks, where every attack success rate is 2 97%. On the other hand, DB (determ.) can reduce the success rate to 20% for the & and to 2% for the foc. attacks with 8 features.
CONCLUSION
We presented Drop-Bottleneck as novel information bottleneck method where compression is done by discretely dropping input features, taking into account each input feature's relevance to the tar- get variable and allowing its joint training with a feature extractor. We then proposed an exploration method based on Prop-Bottleneck, and it showed of-th- art performance on multiple noisy and reward-spars. navigation environments from VizDoom and DMLab. The results showed the robust- ness of Prop-Bottleeeck's compressed representation against noise or task-irrelevant information. With experiments on ImageNet, we also showed that Drop-Bottee Bott-eott achieves hetter adversarial robustness compared to VIB and can reduce the feature dimension for inference. In the exploration experiments, we directly fed the noisy observations to the policy, which can be one source of per- formance degradation in noisy environments. Therefore, applying Drop- Bottleneck to the policy network can improve its generalization further, which will be one interesting future research.
ACKNOWLEDGMENTS
We thank Myeongjang Pyeon, Hyunwoo Kim, Byeongchang Kim and the anonymous reviewers for the helpful comments. This work was supported by Samsung Advanced Institute Technol- ogy. Basic Science Research Program through the National Research Foundation of Korea (NRF) (2020R1A2B33030955555 and Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) Jackyeom Kim was supported by Hyundai Motor Chung Mong-Kot Koo Foundation. Gunhee Kim is the corresponding author.
SW StarLab).

---

Published as a conference paper at ICLR 2021
REFERENCES
Alessandro Achille and Stefano Soatto. Information dropout: Learning optimal representations through noisy computation. IEEE transactions on pattern analysis and machine intelligence, 40 (12):2897-2905, 2018.
Alexander A Alemi, lan Fischer, Joshua V Dillon, and Kevin Murphy. Deep variational information bottleneck. In Proceedings of the 5th International Conference on Learning Representations (ICLR), 2017.
Rana Ali Amjad and Bernhard Claus Geiger. Learning representations for neural network-based classification using the information bottleneck principle. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.
Charles Beattie, Joel Z Leibo, Denis Teplyashin, Tom Ward, Marcus Wainwright, Heinrich Küttler, Andrew Lefrancq, Simon Green, Víctor Valdés, Amir Sadik, al. Deepmind lab. arXiv preprint arXiv:1612.03881 2016.
Yuri Burda, Harri Edwards, Deepak Pathak, Amos Storkey, Trevor Darrell, and Alexei A Efros Large-scale study of curiosity-driven learning. In Proceedings of 7th International Conference Learning Representations (ICLR), 2019a.
Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation. In Proceedings of the 7th International Conference on Learning Representations (ICLR), 2019b.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp). pp. 39-57. IEEE, 2017.
Matthew Chalk, Olivier Marre, and Gasper Tkacik. Relevant sparse codes with variational informa- tion bottleneck. In Advances in Neural Information Processing Systems, pp. 1957-1965, 2016.
Bin Dai, Chen Zhu, and David Wipf. Compressing neural networks using the variational information bottleneck. arXiv preprint
2018.
Ian Fischer. The conditional entropy bottleneck. arXiv preprint arXiv:2002.05379, 2020.
lan Fischer and Alexander A Alemi. Ceb improves model robustness. arXiv preprint arXiv:2002.05380, 2020.
Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, volume 15, pp. 315-323, 2011.
Anirudh Goyal, Riashat Islam, Daniel Strouse, Zafarali Ahmed, Matthew M Botvinick, Hugo Larochelle, Sergey Levine, and Yoshua Bengio. Infobot: Transfer and exploration via the in- formation bottleneck. ArXiv, abs/1901.10902, 2019.
Anirudh Goyal, Yoshua Bengio, Matthew M Botvinick, and Sergey Levine. The variational band- width bottleneck: Stochastic evaluation on an information budget. ArXiv, abs/2004. 11935, 2020.
R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. Proceedings of the 7th International Conference on Learning Representa- tions (ICLR), 2019.
Maximilian Igl, Kamil Ciosek, Yingzhen Li, Sebastian Tschiatschek, Cynthia H Zhang, Sam Devlin, and Katja Hofmann. Generalization in reinforcement learning with selective noise injection and information bottleneck. In NeurlPS, 2019.
Sergey Ioffe and Christian Szegedy. Batch normalization:
Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning, 2015.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmant In Proceedings of the 4th International Conference Learning Representations (ICLR), 2016.
10

---

Published as a conference paper at ICLR 2021
Michat Kempka, Marek Wydmuch, Grzegorz Rune, Jakub Toczek, and Wojeiech Jaškowski. Viz- doom: A doom-based research platform for visual reinforcement learning. In 2016 IEEE Conference on Computational Intelligence and Games (CIG), pp. 1-8. IEEE, 2016.
Youngjin Kim, Wontae Nam, Hyunwoo Kim, Ji-Hoon Kim, and Gunhee Kim. bbttsity-bottleeekk: Exploration by distilling task-specific novelty. In International Conference on Machine Learning, pp. 0179-337983 2019.
Diederik P Kingma and Jimmy Lei Ba. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR), 2015.
Diederik P Kingma and Max Welling, arXiv: 14, 2013.
Auto-encoding variational bayes.
arXiv preprint
Artemy Kolchinsky, Brendan D Tracey, and David H Wolpert. Nonlinear information bottleneck. Entropy, 21(12):1181, 2019.
Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mutual information. Phys- ical review E, 69(6):066135 2004.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Master's thesis, University of Toronto, 2009.
Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database, 2010.
Chris Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. In Proceedings of the 5th International Conference Learning Representations (ICLR), 2017.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Mare Belle- mare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529-533 533, 2015.
Daniel Moyer, Shuyang Gao, Rob Brekelmans, Aram Galstyan, and Greg Ver Steeg. Invariant rep- resentations without adversarial training. In Advances in Neural Information Processing Systems, pp. 9084-9093, 2018.
Vinod Nair and Geoffrey E. Hinton. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference Machine Learning, pp. 807-814, 2010.
Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In International Conference on Machine Learning, 2017.
Xue Bin Peng, Angjoo Kanazawa, Sam Toyer, Pieter Abbeel, and Sergey Levine Variational dis-
criminator bottleneck: Improving imitation learning, inverse rl, and gans by constraining informa- tion flow. Proceedings of the 7th International Conference on Learning Representations (ICLR), 2019.
Brian C Ross. Mutual information between discrete and continuous data sets. PloS one, 9(2): c87357, 2014.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, al. Imagenet large scale visual recognition challenge. International journal of computer vision, 1 115(3)211-225,, 2015.
Nikolay Savinov, Anton Raichuk, Raphaël Marinier, Damien Vincent, Mare Pollefeys, Timothy Lillicrap, and Sylvain Gelly. Episodic curiosity through reachability. In Proceedings of the 7th International Conference on Learning Representations (ICLR), 2019.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 1777006044 2017.
Karl Schulz, Leon Sixt, Federico Tombari, and Tim Landgraf. Restricting the flow: Information bottlenecks for attribution. In Proceedings of the 8th International Conference on Learning Rep- resentations (ICLR), 2020.
11

---

Published as a conference paper at ICLR 2021
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based based local- ization. In Proceedings of the IEEE international conference computer vision, pp. 618-626, 2017.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: simple way prevent neural networks from overfitting. The journal of machine learning research, 15()):1929-158,, 2014.
DJ Strouse and David J Schwab. The deterministic information bottleneck. Neural computation, 29 (6):1611-1630, 2017.
Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In 2015 IEEE Information Theory Workshop (ITW), pp. 1-5. IEEE, 2015.
Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. arXiv preprint physics/0004055, 2000.
12

---

Published as a conference paper at ICLR 2021
A COMPARISON WITH VCEB: ADVERSARIAL ROBUSTNESS
In this section, we compare Drop-Bottleneck (DB) with Variational Conditional Entropy Bottleneck
(VCEB) (Fischer) 2020 Fischer & Alemi] 2020) on the same ImageNet (Russakovsky et al. 2015)
tasks for the adversarial robustness as in Section 5.4 VCEB variationally approximates the CEB
objective, which is defined as
minimize - -I(ZY) + (1(;
(16)
Note that Equation 16 is an alternative form of the original IB objective, Equation T.. as [[Z; X, Y) I(Z; Y I(Z; X X) [Z; X)) and I Y x =0 (:: Z1y X). As in Section 5.4 we employ the experimental setup from VIB (Alemi et al. 2017 with small mod- ifications the hyperparameters for the Adam optimizer (Kingma & Baj 2015) that 31 0.9 and
no learning rate decay is used. Additionally for VCEB, we apply the configurations suggested by Fischer & Alemi (2220:: 1) at test time, the mean of the Gaussian encoding instead of sampling from the distribution, and 2) reparameterize = exp(-p) and anneal the value of from 100 the final P during training. For our experiments, the annealing is performed via the first 100000 training steps, where each epoch consists of 6405 steps.
Employing the experimental setup from Alemi et al. (2017), we adopt the targeted Eg and foo ad- versarial attacks from Carlini & Wagner (2017). We determine the first 200 correctly classified validation images on ImageNet for each setting and perform the attacks where the attack target classes are chosen randomly and uniformly. Further details are described in Appendix E.2
Figure 3 visualizes the classification accuracy for each corresponding final value of and the adversarial ro- bustness results are shown in Table 4 Overall, both VCEB and DB provide meaningful robustness to the targeted lz and foe attacks. For the targeted & at- tacks, although VCEB could achieve the higher aver- age perturbation distance over successful attacks, DB and its deterministic form show better robustness com- pared to VCEB in terms the attack success rates: 18.5% (DB
80
Accuracy (%)
VCEB
70
10
Final (alter anmenling)
0.01) and 20.0% (DB (determ.) at 3 = 0.01) versus 45.0% (VCEB at the final p= 3.454). For the targeted foo attacks, DB and its determinis- tic version again achieve the lower attack success rates than VCEB: 1.5% and 2.0% (DB and DB (determ.) at B E (0.003162, 0.01) }}) versus 12.5% (VCEB at the fi- nal p = 3.454).
Figure 5: Classification accuracy of Inception-ResNet-v2 ResNet-v2 equipped with VCEB (Fischer) 2020) Fischer & Alemi 2020) on ImageNet (Russakovsky et al, 2015). p is annealed from 100 to the final over the first 100000 training steps.
B REMOVAL OF TASK-IRRELEVANT INFORMATION AND VALIDITY OF DETERMINISTIC COMPRESSED REPRESENTATION
We experiment Drop-Bottleneck (DB) and Variational Information Bottleneck (VIB) (Alemi et al.) 2017) occluded image classification tasks to show the following:
DB can control the degree of compression (i.e. degree of removal of task-irrelevant infor- mation) in the same way with VIB as the popular IB method.
DB's s deterministic compressed representation works as a reasonable replacement for its stochastic compressed representation and it maintains the learned indistinguishability better than the attempt of VIB's 's deterministic compressed representation.
We employ the Occluded CIFAR dataset using the experimental settings from Achille & Soatto 2018). The Occluded CIFAR dataset is created by occluding CIFAR-10 (Krizhessyy (009) images with MNIST (LeCun et al.. 2010) images shown in Figure 6al and each image has two labels of CIFAR and MNIST. We use modified version of All-CNN-32 (Achille & Soatto 2018) equipped with IB method (either of DB or VIB) for the feature extractor whose output dimension is d. Each
13

---

Published as a conference paper at ICLR 2021
Table Results of the adversarial robustness for Drop-Bottleneck (DB) and Variational Conditional Entropy Bottleneck (VCEB) (Fischer, [020 Fischer & Alemi] 2020) with the targeted & and foo attacks (Carlini Wagner 2017). Suce. denotes the attack success rate % (lower better), and Dist. is the average perturbation distance over successful adversarial examples (higher is better). *p for VCEB annealed from 100 to the final p over the first 100000 training steps.
Constraint on (
Constraint on (Z;
Attack type
VCEB
DB
DB (determ.)
Final p
B
Succ.
Dist.
Succ.
Dist.
Succ.
Dist.
9.210
99.0
1.200
0.0001
100.0
0.929
99.5
0.923
8.059
92.0
2.028
0.0003162
100.0
0.944
100.0
0.941
6.908
86.5
5.040
0.001
99.5
1.097
100.0
1.134
5.757
65.0
7.198
0.003162
27.0
3.434
23.0
2.565
4.605
46.0
12.016
0.01
18.5
6.847
20.0
6.551
3.454
45.0
10.744
0.03162
41.0
2.160
39.5
1.953
2.303
53.0
14.021
0.1
85.5
2.850
85.5
2.348
9.210
86.0
0.012
0.0001
91.0
0.013
95.5
0.009
8.059
64.5
0.013
0.0003162
85.0
0.016
91.5
0.009
6.908
48.0
0.016
0.001
62.5
0.020
70.0
0.012
to
5.757
29.0
0.019
0.003162
1.5
0.009
1.5
0.020
4.605
17.0
0.025
0.01
2.0
0.022
2.0
0.013
3.454
12.5
0.025
0,03162
8.5
0.022
8.0
0.023
2.303
17.5
0.027
0.1
23.0
0.017
23.0
0.019
run consists of phases. In the first phase, we train the feature extractor with a logistic classifier using both the classification loss for CIFAR and the compression objective of the IB method. Fixing the trained feature extractor, we train logistic classifier for MNIST the second phase. We train two different versions of classifiers for each of VIB and DB using stochastic deterministic compressed representation from the feature extractor. For the deterministic representation of VIB, use the mode of the output Gaussian distribution.
Figures 6dl the experimental results with d = 32, = -4, 64, and d 128. In the first phase, DB retains only subset features that concentrate more on the CIFAR part of the images. Thus, the trained feature extractor preserves less information about the MNIST parts, and the errors of the MNIST classification are high. The first observation that for both DB and VIB with the original stochastic compressed representation, nuisance plots show that increasing 8 from the minimum value to ~ 0.1/c barely changes the primary CIFAR errors but grows the nuisance MNIST errors up to 90% (i.e. the maximum error for the 10-way classification). With even larger 3, enforcing stronger compression results in the increase of the primary errors too, as shown in primary plots. This suggests that both DB and VIB provide fine controllability over the removal of task-irrelevant information.
6bg
Secondly, if we move our focus to the nuisance (deterministic) plots in Figures 6b which show the test errors on the nuisance MNIST classification with the feature extractor deterministic rep- resentation, the results become different between DB and VIB. In DB, the nuisance (deterministic) plots follow the nuisance plots the range of / where the compression takes effect (i.e. where the nuisance errors increase). Moreover, the two plots get closer as B increases. It means that Drop-Bottlene
deterministic compressed representations maintain the majority of the indistin- guishability for the task-inreeevant information learned during the first phase, especially when B is large enough to enforce some degree of the compression. On the other hand, VIB's nuisance (de- terministic) plots largely different from the nuisance plots; even the primary errors rise before the nuisance (deterministic) errors reach their maximum values. This shows that employing the mode of VIB's output distribution its deterministic representation results in loss of the learned indistinguishabilit.
In summary, we confirm that DB provides controllability over the degree of compression in a similar way as VIB. On the other hand, DB's deterministic representation can be reasonable replacement
14

---

Published as a conference paper at ICLR 2021
VIB primary
VIB muisance
VIB muisance (deterministicc)
DB primary
DB nuisance
DB nuisance (delerministie)
3
airplane
The ecnu (15
antomabile
deer
30
horse
ship
10
10
0 10
10'
10
10
107
107
10
10
10'
10"
3-d
(a) Sample images
(b) d 32
(c) d 64
(d) d = 128
Figure 6: (a) A few samples from Occluded CIFAR dataset (Achille & Soatto 2018 (b)-d)
Test error plots on the primary task (i.e. the classification of occluded CIFAR images) and on the nuisance tasks (i.e. classification of the MNIST digits). For all the three types of tasks, we use the same feature extractor trained for the primary task, where its deterministic representation is used only for training and test of the nuisance (deterministic) task.
Raw image
Primary
Nuisance (agg.)
(a) Without DB
(b) With DB
Grad-CAM (Selvaraju al. 2017) visualization for the last convolutional layer of the feature extractor the Occluded CIFAR classification task. For the visualization, d 128, and = 5.623/ d for (b) are used. Primary denotes the maps of the logits for the primary labels. Nuisance (agg.) means the maps the nuisance task aggregated over all the logits (i.e. 10 logits). (a) indicates that the feature extractor without DB trained on the primary task still outputs much information about the nuisance tasks, and thus the nuisance classifier could depend on the features extracted from the nuisance (MNIST) regions. In contrast, (b) suggests that the feature extractor with DB could learn to discard the nuisance features, so that the nuisance classifier mostly fails to learn due to the lack of nuisance-relevant features.
Figure 7:
for its original stochastic representation in terms of preserving the learned indistinguishabilit,, which is not exhibited by VIB.
C VISUALIZATION OF TASK-IRRELEVANT INFORMATION REMOVAL
In this section, we visualize the removal of task-irrelevan information with Prop-Bottlenecd (DB). To this end, we employ the Occluded CIFAR dataset (Achille & Soatto 2018) with the same exper- imental setup as in Appendix B Each image of Occluded CIFAR dataset is one of the CIFAR- 10 (Krizhevsky 2009) images occluded by MNIST (LeCun al. 2010 digit images. In the first phase of the experiments, the feature extractor and the classifier are trained on the primary (oc- cluded CIFAR classification) task in a normal way. During the second phase, the learned feature extractor is fixed, and only a new classifier is trained on the nuisance (MNIST classification) task. In Appendix B we quantitatively showed that the feature extractor with DB could focus more on the occluded CIFAR images and preserve less information about the MNIST occlusions. We take qualitative approach in this section and visualize the phenomenon using Grad-CC CAMM (Selvaraju et al,) 2017). Grad-CAM is popular for providing visual explanation given convolutional neural networks with their target values (e.g. target logits in classification tasks).
We first sample multiple test images from the Occluded CIFAR dataset, and load full, trained models, which include the feature extractor, primary classifier and nuisance classifier. We then obtain the activation maps for the last convolutional layer of the feature extractor on the primary and nuisance tasks. On the primary task, we compute the activation maps simply targeting the logits for the sample labels. However, on the nuisance task, we get the activation maps of all the logits and aggregate them by taking the maximum of the maps each pixel location. Therefore, the aggregated maps the
15

---

Published as a conference paper at ICLR 2021
nuisance task visualize the activation related to not only the logits for the true class labels but also the other logits, capturing most of the feature usage induced during the training on the nuisance task.
Figure compares two trained models: the d 128 model without DB, and the = 128 model with DB (deterministic). We use the DB model with 5 5.623/ d for the visualization, as the value is sufficiently large enough to enforce strong compression while still keeps the primary error not high. Figure 7a shows that regarding the logits for the nuisance (MNIST) task, large portion of each image including the MNIST digit is activated in most cases, and thus it indicates that the feature extractor trained without DB preserves much of the nuisance features. On the other hand, Figure 7b visualizes that the feature extractor with DB outputs notably less of the nuisance features, preventing the nuisance classifier from learning correctly.
To sum up, provide the visual demonstration that the classification task with the Occluded CIFAR dataset, the feature extractor equipped with DB trained on the primary task could discard majority of the nuisance i.e. task insk-irrelevant information given a value of 3 that is strong enough.
D ABLATION STUDY: EXPLORATION WITHOUT DROP-BOTTLENECK
We perform ablation study to show input information. We examine the performance of the same exploration method as described in Sec- tion 4 but without DB; that is, the feature vectors are fully used with dropping. In order to em- phasize the effectiveness of DB for noisy and task tuse-irreeevant information, we conduct experiments with both noisy and original (i.e. without explicit noise injection) settings.
(DB)'s ability of dealing with ineeeant
Table 5: Comparison of the average episodic sum of rewards DMLab tasks (over 30 runs), where PPO Ours Noo-Drop-Bottlenee without explicit noise injection) and three noisy settings are tested: Image Action (IA), Noise (N), Noise Action (NA) and Original (0). The values are measured after 20M (4 action-repeated) time steps, with no seed tuning done. Baseline results for DMLab are cited from Savinov et al. (019).
denotes our exploration method without DB. The original (i.e.
DMLab
Method
Very Sparse
IA
N
NA
O
PPO (Schulman et al. 2017)
6.3
8.7
6.1
8.6
PPO + ICM (Pathak et al.
2017
4.9
6.0
5.7
11.2
PPO + EC (Savinov et 2019)
7.4
13.4
11.3
24.7
PPO + ECO (Savinov et al.
2019)
16.8
26.0
12.5
40.5
PPO + Ours (No-Drop- Drop-Bottlennec)
14.9
11.7
10.3
33.0
PPO + Ours (Drop-Bottleneck)
28.8
29.1
26.9
42.7
Improvement (%)
93.3
148.7
161.2
29.4
Table 5) shows the results on "Very Sparse" DMLab environments with "Image Action", "Noise", "Noise Action" and "Original" settings. Compared to "Original" setting where observations tain only implicit, inherent noisy information irrelevant to state transitions, DB brings much more significant improvement to exploration methods in "Image Action", "Noise", and "Noise Action" settings, which inject explicit, severe transition-irellvannt information to observations. These results suggest that DB plays important role handling noisy task-irrelevant input information.
E DETAILS OF EXPERIMENTS
We describe additional details of the experiments in Section For all the experiments with DB, each entropy HX;) is computed with the binning-base estimation using 32 bins, and the drop probability p is initialized with p. = = and pk Uniform(a, b) for = =-2,
16

---

Published as a conference paper at ICLR 2021
Table Hyperparameters of PPO (Schulman al. 2017). PPO + ICM (Pathak ECO Savinov et al. 2019), and PPO + Ours for the VizDoom experiments.
2017 PPO
PPO
PPO ICM
PPO ECO
PPO + Ours
PPO
Learning rate
0.00025 0.01
0.00025 0.01
0.00025 0.01 5
0.00025
Entropy coefficient
0.01 5
Task reward scale
5
Exploration method
Training period and sample size of optimization epochs Intrinsic bonus scale
3K
120K 10
10.8K
4
0.01
0.001
Table Hyperparameters of PPO (Schulman et al. 2017). PPO + ICM (Pathak et al. EC/ECO (Savinov et al. 2019), and PPO + Ours for the DMLab experiments.
2017 PPO
PPO
PPO ICM
PPO ECVECO
PPO + Ours
PPO
Learning rale Entropy coefficient
0.00019 0.0011
0.00025 0.0042
0.00025 0.0021
0.00025 0.0011
Task reward scale
Exploration method
Training period and sample size of optimization epochs Intrinsic bonus scale
720K (BCO)
21.6K 2
10 (ECO)
0.55
0.030
0.005
E. DETAILS OF EXPLORATION EXPERIMENTS
To supplement Section 5.1 we describe additional details of the VizDoom (Kmmkka et al. 2016)
and DMLaa (Beattie al. 2016) exploration experiments. We collect training samples in a buffer and update p, Py fo with Equation (14) periodically. We use Adam optimizer (Kingma & Baj 2015) with a learning rate of 0.0001 and batch size of 512. In each optimization epoch, the training samples from the buffer are re-shuffled. For each mini-batch, we optimize the Deep In- fomax (Hjelm et al. 2019) discriminator Te with 8 extra epochs with the same samples, to make the Shannon mutual information bound tighter. This way of training T. only runs forward and backward passes on TG for the fixed output of the feature extractor for and thus can done with low computational cost. 33 is the hyperparameter that determines the relative scales of the com- pression term and the Deep Infomax Jensen-Shann Shannon mutual information estimator. 0.001/ 128 for DB and 3 0.0005/ 128 for VIB. To make experiments simpler, normalize our intrinsic rewards with the rumning mean and standard deviation.
tuned to
Table and Table 7 report the hyperparameters the methods for VizDoom and DMLab exper- iments, respectively. We tune the hyperparameters based the ones provided by Savinov et al. 2019). Unless specified, we use the same hyperparameters with Savinov et al. (011)).
Under the three noise settings suggested in Savinov et al. (2019), the lower right quadrant of every observation is occupied by TV screen follows.
"mmage Action": Every time the agent performs a specific action, it changes the channel of the TV randomly to of the predefined animal images.
"Noise": At every observation, new noise pattern is sampled and shown on the TV screen (independently from the agent's actions).
"Noise Action": Same as "Noise", but the noise pattern only changes when the agent does specific action.
Figure shows some observation examples from VizDoom and DMLab environments with "Image Action" and "Noise" settings.
17

---

Published as a conference paper at ICLR 2021
(a) VizDoom
(b) DMLab
Figure 8: Example observations from VizDoom (Kempka et al. 2016 and DMLab (Beattie et al.
2016 environments with "Image Action" (first) and "Noise" (second) settings.
E.2 DETAILS OF ADVERSARIAL ROBUSTNESS EXPERIMENTS
For the experiments, each input image is sized 299 x 299 x 3 and each pixel value is ranged [11 We perform our adversarial robustness experiments based on the official source code of Carlini & Wagner (2017). For the targeted & attack increase the number of binary search steps to 20 and use a batch size 25. Other than the two, the default hyperparameters are used.
E.3 DETAILS OF RUNNING FEATURE SELECTION METHOD
The input of the feature selection method is the same DB's and VIB's: the input features ob- tained with a pre-trained model of randomly pick 100k samples out of the total 1281167 training samples of ImageNet, and compute the relevance scores for features using those samples by estimating the mutual information between each feature and its label using the entropy estimation with the k-nearest neighbors (Kraskov al., 2004 Ross 2014). Given the computed scores, we perform the feature selection by preserving the features with the highest scores.
on ImageNet (Russakovsky et al. 2015)) We
E.4 DETAILS OF OCCLUDED IMAGE CLASSIFICATION EXPERIMENTS
We use modified version of AlI-CNN-32 (Achille & Soatto 2018
The model architecture for feature dimension d is described in Table Batch normalization (offe & Szegedy 2015) is applied to Conv layers, and the ReLU (Nar & Hinton 2010 Glorot et al. 2011) activation used every hidden layer. We use the Adam optimizer (Kingma & Ba 2015) ) with a learning rate of 0.001. To ensure the convergence, in each training, the model is trained for 200 epochs with a batch size of 100.
Table 8: The network architecture for the occluded image classification experiments.
Input image 32 x 32 x
Conv [3 x 3, 96, stride I]
Conv [3 x 3, 96, stride
Conv [3 x 3, 96, stride 2]
Feature extractor
Conv [3 x 3, 192, stride 1]
Conv [3 3, 192, stride 1]
Conv [3 3, 192, stride 2]
FC [d]
FC [2d]
DB [d]
VIB [d]
FC [10]
Classifier
softmax
18


Reviews:
[{"review_id": "l3om50A3FN", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper proposes an information bottleneck method, Drop-Bottleneck, that allows the input to be compressed by dropping each input feature with probability p_i. The model then learns the drop probability vector p = [p_1, ... , p_n], where dropping \"redundant\" features will reduce the \"compression penalty\" term I(XZ). The approach is demonstrated in experiments in (1) robust exploration setting for RL, (2) adversarial attacks on ImageNet, and (3) an experiment showing that their approach is able to maintain performance on ImageNet with reduced dimensionality.\n\nEvaluation:\nI found this paper to be clear and the experiments seem reasonable. I don't know of any prior work that takes this approach. I'm not an RL expert so I won't comment on the strength of the RL results, other than that their methods were clear and they seem to have been careful and fair in choosing baselines. My biggest criticism is that the robustness experiments on ImageNet compare to VIB (Alemi et al 2017) as a baseline, but they should really compare to the more recent adversarial results on Conditional Entropy Bottleneck (another information bottleneck approach that outperforms VIB) given in Fischer and Alemi 2020 (https://arxiv.org/pdf/2002.05380.pdf). I'd like to see that added to the revised version.", "extra_scores": {}, "extra_text": {}}, {"review_id": "e3zIMNLiXNB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a new Information Bottleneck objective, which compresses the latent by learning to drop features similar to DropOut. Unlike DropOut, a different probability is learnt for each latent feature/dimension using Concrete Relaxation. The experiments show that the works well.\n \nOverall, I score this paper as an accept. While the approach is limited to dropping input features, which does not make it a general IB objective, it seems to work very well in the presented RL experiments as well as show robustness that is better than DVIB’s. Moreover, the paper is clearly written and engaging.\n \n### Strengths\n \nThe paper proposes a simple yet effective idea. Using a Concrete Relaxation to learn DropOut probabilities has been done before, but the idea to have a separate probability per latent dimension is novel. The Drop-Bottleneck objective works directly on the input/latent layer, which means that the compression objective is easy to compute. This is nice because IB terms are usually cumbersome to compute. (However, this also requires the input/latent to already be disentangled, as dropping out features is limited in its expressiveness.) \n \nThe RL experiments on VizDoom and DMLab are convincing as are the ones on ImageNet. The additional experiment on the Occluded CIFAR-10 dataset in the appendix is also well thought-out and shows the advantage of this straightforward method over DVIB.\n \nMoreover, once trained, features can be dropped out deterministically if so required, which allows for proper compression and consistent behaviour.\n \n### Questions\n \nGiven the simple conceptual idea, this reviewer would be interested to see an ablation with using other methods of enforcing sparsity in the latent. Could L1 regularization of the latent activations be used instead of the $I[Z; X]$ term?\n \nDB cannot provide the same generality as other IB objectives: the input (latent) has to be sufficiently disentangled already as the objective itself does not encourage further disentanglement by itself. Do the ImageNet experiments use extracted/pre-trained embeddings as latents?\n\n---\n### Rebuttal\n\nI thank the authors for their reply. I'm more confident this is a good paper now.", "extra_scores": {}, "extra_text": {}}, {"review_id": "DbrrqFFw9Z", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThe paper contributes a novel method, Drop-Bottleneck (DB), for discretely dropping input features that are irrelevant for predicting the target variable. Key idea is to instantiate the compression term of the information bottleneck framework with learned term that sets irrelevant feature dimensions to 0. To this end, a drop probability is learned for each dimension. Dimensions that have a lower probability than 0.5 (a fixed threshold) of being relevant are set to 0.\n\nStrong Points:\n- The paper is well-written and easy to understand.\n- Experiments show that DB works better than VIB in VizDoom and DMLab when a noisy-TV noise is added to the input images. Different noisy-TV noises are considered: changing the image when the agent performs an action, adding random noise to the TV, and adding random noise when the agent performs an action.\n- Experiments also show that the obtained representation is more robust against l_2 and l_inf attacks in ImageNet. Furthermore, the experiments show that the approach can drop many ImageNet features while almost preserving the accuracy of a ResNet.\n- The paper comes with code in the supplementary material.\n\nWeak Points:\n- I found the reinforcement learning experiments not convincing since only a fixed region of the input is modified by noise (ie. the noisy TV). Hence, the approach essentially identifies irrelevant pixel locations. Such a problem could be solved by a simple pre-processing step. The method won't work if the location of the noise changes. In general, limitations of the work are not discussed.\n- The experiments on ImageNet are more interesting. However, the fact that individual dimensions (i.e. specific pixel locations) are identified as irrelevant is still a limiting factor. Furthermore, the experiments do not fit to the focus of this paper on reinforcement learning.\n- The paper does not discuss connections of the presented approach to prior works for (discrete) feature selection. It only discusses connections to prior bottleneck methods.\n- The paper does not perform experiments on datasets with meaningful features where a feature selection makes more sense than for specific pixels in images.\n\nAdditional feedback:\n- It could be stated explicitly that H refers to the entropy. Currently, it is only implicitly defined in Eq. 6.\n- I think it would be better to also cite Jang et. al (Categorical Reparameterization with Gumbel-Softmax) for the Concrete relaxation of the Bernoulli distribution.\n- Figure 1 should either be improved or removed. I don't see much additional insights that can be gained from this figure.\n- Of course, it would be very interesting to see if the drop probabilities correlate with the location of the noise inputs. It would be great if such an analysis could be added. This could replace Figure 1.", "extra_scores": {}, "extra_text": {}}, {"review_id": "2QRdNvb6ZN0", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Summary**\nThis paper proposes the Drop-Bottleneck (DB) method that performs feature selection during the training with the mutual information. It achieves the state-of-the-art result in few reinforcement learning tasks, and trains a more robust model.\n\n**Originality and significance aspect**\nThis paper combines mainly two ideas 1) classic feature selection (choose Xis to drop) with respect to the mutual information (between X and Y) 2) Information Bottleneck (IB) formulation that maximizes the prediction-term mutual information term (between Z and Y) and minimizes the compression information (between X and Z) simultaneously. It is actually fairly close to the core idea of the feature selection because it finds the compression by dropping the original feature or not unlike the other IB compression methods. In other words, DB is a modernized version of the feature selection that automatically drops a feature based on the IB-style loss. I would say the idea is not entirely new (somewhat limited), but it could be still useful to the community. \n\nIn the significance aspect, I wanted to see authors to apply this method on other noisy setup tasks (e.g. computer vision tasks with noise), outside of the reinforcement learning tasks. The improvement on the RL tasks seems to be substantial; however, I would like to know how DB performs when there is correlation between feature dimensions (see below clarification question to see more details).\n\n**Quality and clarity aspect**\n\nThe paper was overall easy to follow. Here are a few questions to authors:\n\n* What if we just drop the feature space only using the mutual information between X and Y and drop them to achieve a similar number of features that was resulted by DB -- it is basically the classic mutual information feature selection. Would that perform as good as DB? Can you make a comparison?  I think this should be one of the baseline. If that's similar to DB, what would be the benefit of DB?\n* Does DB always minimize \\hat{I}(Z; X) -- with the independent assumption? When some of the Xis are correlated (e.g. consider a vision task), there could be quite a gap between I(Z; X) and the independent assumption version. \n\n**Recommendation**\nI think the paper is at the borderline. Looking forward to seeing more discussion with the classic feature selection method and some evaluation on tasks outside of RL (if possible). I would be happy to revisit my score\n\n\n-----\n**Post rebuttal comment**\nI thank the authors for the rebuttal. Authors have addressed my concerns and clarified some of the confusing points that I had. I would like to recommend this paper to be accepted.\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes to enhance the robustness of RL and supervised learning algorithms to noise in the observations by dropping input features that are irrelevant for the task. It relies on the information bottleneck framework (well derived in the paper) and learns a parametric compression of the input features that sets them to zero if they are not relevant for the taskn. The method is extensively evaluated on several RL tasks (exploration in VizDoom and DMLab with a noisy “TV” distractor) and supervised tasks (ImageNet or CIFAR-10 classification with noise).

Reviewers have praised the idea, derivation and writing, as well as the extensive experiments on RL and supervised tasks. Critique focused on:
* the contrived nature of the TV noise (localised always in the same corner of the image -- a standard evaluation according to the authors),
* lack of comparison with other feature selection methods,
* lack of comparison with Conditional Entropy Bottleneck (done during rebuttal),
* more general noise than just specific pixels (clarified by the authors as being the features coming out of a convnet)

Given that the reviewers’ comments were largely addressed by the authors, and given the final scores of the paper, I will recommend acceptance.

Decision: Accept (Poster)