A Rate-Distortion Theory of Adversarial Examples

1 INTRODUCTION . The mathematical concept of information , as introduced by Shannon ( 1948 ) in the context of communication over noisy channels , has revolutionized fields like psychology and neuroscience . Despite the fact that deep learning has been heavily influenced by these fields , information theory is not the established framework for its development . Built on mainly empirically derived design principles , modern neural networks achieve impressive results . However , a theoretical understanding as to when good performance can be expected is lacking ( Zhang et al. , 2017 ; Arpit et al. , 2017 ) . In particular , the inability to characterize generalization imposes hard constraints on the responsible deployment of deep learning in performance-critical settings where fault tolerance is required . The challenge of obtaining such a characterization arises from the difficulty of interpreting ( Bau et al. , 2017 ) and quantifying the uncertainty of the predictions ( Gal , 2016 ; Guo et al. , 2017 ) . Furthermore , the adversarial examples phenomenon ( Biggio et al. , 2013 ; Szegedy et al. , 2014 ; Nguyen et al. , 2015 ; Athalye et al. , 2018 ) shows that solely maximizing test accuracy can result in potentially unsafe model behavior . Many studies have attempted to understand generalization via model complexity and introduced various measures to quantify it . However , Neyshabur et al . ( 2017 ) concludes that a combination of several measures is required to explain some of the empirically observed phenomena . A more compact description of generalization was proposed by Tishby & Zaslavsky ( 2015 ) , who analyzed the learning problem in terms of mutual information . This measure incorporates many of the previously proposed characteristics , effectively summarizing the interplay between the data , network architecture , and the optimization procedure . We adopt this view for the image classification problem and make the following contributions : 1 . Identifying a DNN with a noisy communication channel we formalize the trade-off between a model ’ s prediction accuracy and sensitivity to adversarial examples through ratedistortion theory . Practically speaking , this is the information bottleneck ( IB ) trade-off in the context of supervised deep learning for image classification . We attribute sensitivity to adversarial examples to excess capacity , i.e. , complexity of input representations . 2 . We explain the mechanisms by which explicit regularization strategies determine the model ’ s capacity , drawing on an intuitive sphere packing argument for the Gaussian communication channel . Weight decay has a natural interpretation as a power constraint on the channel , while batch normalization acts as an information-theoretic “ short-circuit ” between adjacent layers . 3 . We demonstrate the practical utility of the trade-off mentioned in contribution 1 on the Street View House Numbers ( SVHN ) dataset . We propose evaluating robustness in terms of fault tolerance by plotting the information transmission rate versus the signal-to-noise ratio for unbounded and unseen “ worst case ” noise . Furthermore , we demonstrate successful communication across the DNN in the reverse direction by generating plausible messages from only the label , confirming the expectation that a communication channel should work bidirectionally . 2 THEORY . We begin by formulating the machine learning problem in the framework of information theory , focusing in particular on image classification , with the model being a feedforward deep neural network ( DNN ) trained in the supervised setting . We find that this problem has a natural interpretation as that of communication over noisy channels and use it as a concrete example for the following analysis ( which is , however , not limited to image classification ) . We assume basic familiarity with information-theoretic quantities , such as entropy and mutual information . The model ’ s input ( image ) , the corresponding desired output ( true label ) and the actual output ( predicted label ) are represented by the random variables X , Y and Ŷ , respectively . The goal of the classification task is to adjust the parameters of the model such that the prediction Ŷ given X is as close as possible to the true label Y for previously unseen inputs . The mapping from input X to prediction Ŷ is formally characterized through the conditional probability distribution P ( Ŷ |X ) . We model each of the nL layers in a DNN as a continuous communication channel , and describe the corresponding output as a random variable Tj , j = 1 , . . . , nL , as in Tishby & Zaslavsky ( 2015 ) . The DNN is the series connection of such channels . In the communication setting , the label Y is the intended message , and the image X can be interpreted as a highly redundant and noisy encoding of Y . The decoded message Ŷ is obtained from the final layer TnL . A central quantity characterizing a physical channel is its capacity , defined as the maximum transmission rate at which communication is possible with arbitrarily low probability of error . Consider that in order to transmit information over a channel , the message must first be encoded in terms of signals that the channel can actually represent – for example , voltage pulses of varying amplitude . These signals are composed to sequences of length n , or “ codewords ” , used to represent the message . An ideal continuous channel allows for infinitely many codewords that are all uniquely decodable , and thus achieves error-free information transmission , corresponding to infinite capacity . In practice , channel capacity is restricted by two factors : the available power and the noise , e.g. , from the environment . The most common model used in theory to characterize real channels is the Gaussian channel with additive white noise Z ∼ N ( 0 , N ) , depicted in Figure 1 ( a ) , with a constraint on the average power of the signal that is proportional to the signal amplitude x2i : 1 n ∑n i=1 x 2 i ≤ P . Shannon ( 1948 ) proved that the information capacity of the Gaussian channel is a function of the signal-to-noise ratio ( SNR ) P/N : C = 1 2 log2 ( 1 + P N ) bits per transmission . ( 1 ) The maximal number of codewords that can be decoded with an arbitrarily small probability of error under these constraints can be estimated via a simple geometrical argument : An input sequence x = { xi } ni=1 and the corresponding received sequence y = { yi } ni=1 = { ( xi + zi ) } ni=1 can be represented as two points in an n-dimensional space of encodings . The average power constraint forces x to lie inside a sphere of radius √ nP centered around the origin . Since y differs from x only due to noise with variance N , it will be found with high probability in a sphere of radius √ nN centered around x . In this way , the sphere with radius √ n ( P +N ) is partitioned into decoding spheres with radius √ nN , one for each codeword x . The maximum number of non-confusable codewords M is given by the number of non-intersecting decoding spheres that can be “ packed ” into the large sphere , obtained from the ratio of sphere volumes : M = ( P +N N ) n/2 = 2 n 2 log2 ( 1+ P N ) , ( 2 ) where the exponent in the final expression is precisely nC , since capacity C is the maximal achievable rate . In image classification setting , the noise is already built into X , as indicated in Figure 1 ( b ) . The fact that we wish to transmit only the message , modeled as a latent cause variable Φ , observable only with noise through X – but not X itself – is an essential detail that lets us reason about deterministic neural network layers as noisy channels , even though no noise is added in the channel itself after the input layer.1 An average power constraint on the input to the channel for unbounded activation functions such as the relu is equivalent to a squared L2 norm constraint on the weights – i.e. , the traditional weight decay – since the input to a DNN is typically bounded , and the input to the channel is the weighted input of the layer . The geometric sphere packing argument can be extended to understand the role of capacity in supervised deep learning : We say that the model has learned the rule if training inputs are mapped to nK non-overlapping clusters , corresponding to the associated labels . The model generalizes to the extent that test inputs are mapped to the same clusters . Achieving this already means a perfect solution to the classification task in the sense of arbitrarily small prediction error . However , this is not sufficient for fault-tolerance , or generalization in the stricter sense , since all vacant encoding space that is not assigned to any label represents “ excess capacity ” . To achieve fault-tolerance , we have to minimize excess capacity , or , in terms of the geometric interpretation , we need to maximally fill out the available encoding space . The ideal result with the densest packing of a spherical volume requires the nK non-overlapping clusters to be spherical . Spherical cluster shape can be achieved by whitening ( i.e. , sphering ) the input , and by learning projections that explain the natural variability in the data to the extent that it is relevant . Provided a sufficient power budget , the representation learning problem is that of finding projections that rotate and reflect the input into compact volumes that can be partitioned homogeneously with respect to the label . The trade-off is that making the class-specific volumes more compact requires compression of information , but we need to preserve enough information to distribute the decoding spheres among the available representation space such that they are all non-overlapping . This illustrates how the problem of reliable communication over a noisy channel is essentially that of optimal information compression . The transmission rate R is a function of the number of codewords M and their length n : R = log ( M ) /n . Increasing M for fixed channel properties ( P and N ) will increase the overlap between decoding spheres , while decreasing n corresponds to compressing the input representation . Therefore , increasing R is only possible at the cost of transmission accuracy . This fundamental trade-off is characterized analytically by rate-distortion ( RD ) theory in the form of a constrained optimization problem . The RD theory introduces a distortion measure d ( x , t ) that quantifies the distance between a random variable X and its representation T , and describes the rate R as a function of the average distortion D. A major drawback of RD theory is that it does not specify how to choose d. The IB method , which can be seen as a generalization of RD theory , answers this question by introducing the concept of the relevant variable Y . In this setting , the representation T is required to maximally preserve information that X carries about Y ( Tishby et al. , 1999 ) . This way , the constrained optimization 1 We acknowledge that stochastic feedforward DNNs are considered in the literature ( c.f . Tang & Salakhutdinov , 2013 ) , but only consider the more typical deterministic layers in our work . problem is expressed solely in terms of mutual information : L = I ( X ; T ) − βI ( Y ; T ) , ( 3 ) where I ( X ; T ) measures the compression of X and β is a positive parameter associated with the constraint on the preserved information I ( Y ; T ) . Thus , the solution to ( 3 ) characterizes an optimal trade-off between compressing X and preserving the relevant information it carries . It turns out that it is more efficient to split up the task of encoding the message into multiple stages via a successive refinement of information ( Equitz & Cover , 1991 ) , which means that we should be able to decode the relevant variable Y from any of the layers Tj with distortion Dj , and the description will be RD-optimal at each layer . In DNNs , each of the “ stages ” are the hidden layers ’ representations Tj of the input X , which induce a partition of the input space w.r.t . the relevant variable Y and coarsen it gradually in the course of learning . This corresponds to the case of no excess capacity . Remarkably , it has been suggested that optimizing all layers in a DNN simultaneously with plain stochastic gradient descent ( SGD ) is sufficient to achieve successive refinement . The process is characterized by a brief fitting or memorization phase where I ( X ; T ) and I ( Y ; T ) increase monotonically – consistent with empirical observations that DNNs begin memorizing after the first epoch ( Carlini et al. , 2018 ) – and followed by a compression phase where I ( X ; T ) decreases . If there are no constraints on the complexity of the system , there is no need for successive refinement : relevant information can always be encoded with sub-optimal representations at a higher cost , e.g. , through some degree of memorization . Such a solution achieves small distortion ( error ) with significant excess rate ( complexity ) , making it easier to transmit messages along the channel which are decoded arbitrarily at test time – i.e. , adversarial examples . It is argued that the primary mechanism facilitating compression is the inherent noise in the dataset , which increases the entropy of the weights subject to a constraint on the empirical risk . This entropy increase is logarithmic in the number of epochs , consistent with observations that we may reap substantial benefits by continuing to train well after the empirical loss is minimized , e.g. , for obtaining a maximum-margin solution ( Soudry et al. , 2018 ) . Although optimal compression may be conferred automatically by SGD given sufficient data , it is prudent to take other measures to improve learnability , e.g. , by whitening the input , to achieve this in fewer epochs .

Summary: The paper discusses on a rate distortion interpretation of adversarial examples by building the equivalence of DNN and a noisy channel. The proposed topic is very interesting. However, it is quite disappointing after reading the paper, that it does not deliver. In a sense, the reader has an impression that the paper is a collection of fractions of small thoughts and empirical observation pieces that are yet to be stringed up coherently.