Caveats for information bottleneck in deterministic scenarios

1 INTRODUCTION . The information bottleneck ( IB ) method ( Tishby et al. , 1999 ) provides a principled way to extract information that is present in one variable that is relevant for predicting another variable . Given two random variables X and Y , IB posits a “ bottleneck ” variable T that obeys the Markov condition Y − X − T . By the data processing inequality ( DPI ) ( Cover & Thomas , 2012 ) , this Markov condition implies that I ( X ; T ) ≥ I ( Y ; T ) , meaning that the bottleneck variable can not contain more information about Y than it does about X . In fact , any particular choice of the bottleneck variable T can be quantified by two terms : the mutual information I ( X ; T ) , which reflects how much T compresses X , and the mutual information I ( Y ; T ) , which reflects how well T predicts Y . In IB , bottleneck variables are chosen to maximize prediction given a constraint on compression ( Witsenhausen & Wyner , 1975 ; Ahlswede & Körner , 1975 ; Gilad-Bachrach et al. , 2003 ) , F ( r ) : = max T∈∆ I ( Y ; T ) s.t . I ( X ; T ) ≤ r , ( 1 ) where ∆ is the set of random variables T obeying the Markov condition Y − X − T . The values of F ( r ) for different r specify the IB curve . In order to explore the IB curve , one must find optimal T for different values of r. It is known that the IB curve is concave in r but may not be strictly concave . This seemingly minor issue of non-strict concavity will play a central role in our analysis . In practice , the IB curve is almost always explored not via the constrained optimization problem of Eq . ( 1 ) , but rather by maximizing the so-called IB Lagrangian , LβIB ( T ) : = I ( Y ; T ) − βI ( X ; T ) , ( 2 ) ∗Dept of Aeronautics & Astronautics , Massachusetts Institute of Technology , Cambridge , MA 02139 , USA where β ∈ [ 0 , 1 ] is a parameter that controls the trade-off between compression and prediction . The advantage of optimizing LβIB is that it avoids the non-linear constraint in Eq . ( 1 ) 1 . Several recent papers have drawn connections between IB and supervised learning , in particular classification using neural networks . In this context , X represents input vectors , Y represents the output classes , and T represents intermediate representations used by the network architecture , such as the activity of hidden layer ( s ) ( Tishby & Zaslavsky , 2015 ) . Some of these papers modify neural network training algorithms so as to optimize the IB Lagrangian ( Alemi et al. , 2016 ; Chalk et al. , 2016 ; Kolchinsky et al. , 2017 ) , thereby permitting the use of IB with high-dimensional , continuousvalued random variables . Some papers have also suggested that by controlling the amount of compression , one can tune desired characteristics of trained models such as generalization error ( Shamir et al. , 2010 ; Tishby & Zaslavsky , 2015 ; Vera et al. , 2018 ) , robustness to adversarial inputs ( Alemi et al. , 2016 ) , and detection of out-of-distribution data ( Alemi et al. , 2018 ) . Other research ( ShwartzZiv & Tishby , 2017 ) has suggested — somewhat controversially ( Saxe et al. , 2018 ) — that stochastic gradient descent ( SGD ) training dynamics may implicitly favor hidden layer mappings that balances compression and prediction , with earlier hidden layers favoring prediction over compression and latter hidden layers favoring compression over prediction . Finally , there is the general notion that intermediate representations that are optimal in the IB sense correspond to “ interesting ” or “ useful ” compressions of input vectors ( Amjad & Geiger , 2018 ) . There are also numerous application domains of IB beyond supervised learning , including clustering ( Slonim & Tishby , 2000 ) , coding theory and quantization ( Cardinal , 2003 ; Zeitler et al. , 2008 ; Courtade & Wesel , 2011 ) , and cognitive science ( Zaslavsky et al. , 2018 ) . In most of these applications , it is of central interest to explore solutions at different points on the IB curve , for example to control the number of detected clusters , or to adapt codes to available channel capacity . In some scenarios , Y may be a deterministic function of X , i.e. , Y = f ( X ) for some single-valued function f . For example , in many classification problems , it is assumed that any given input belongs to a single class , which implies a deterministic relationship between X and Y . In this paper , we demonstrate three caveats for IB that appear whenever Y is a deterministic function of X : 1 . There is no one-to-one mapping between different points on the IB curve and maximizers of the IB LagrangianLβIB for different β , thus the IB curve can not be explored by maximizingL β IB while varying β . This occurs because when Y = f ( X ) , the IB curve has a piecewise linear shape and is therefore not strictly concave . The dependence of the IB Lagrangian on the strict concavity of F ( r ) has been previously noted ( Gilad-Bachrach et al. , 2003 ; Shwartz-Ziv & Tishby , 2017 ) , but the implications and pervasiveness of this problem ( e.g. , in many classification scenarios ) has not been fully recognized . We analyze this issue and propose a solution in the form of an alternative objective function , which can be used to explore the IB curve even when Y = f ( X ) . 2 . All points on the IB curve contain uninteresting trivial solutions ( in particular , stochastic mixtures of two very simple solutions ) . This suggests that IB-optimality is not sufficient for an intermediate representation to be an interesting or useful compression of input data . 3 . For a neural network with several hidden layers that achieves a low probability of error , the hidden layers can not display a strict trade-off between compression and prediction ( in particular , different layers can only differ in the amount of compression , not prediction ) . In Appendix B , we show that the above three caveats also apply to the recently proposed deterministic IB variant of IB ( Strouse & Schwab , 2017 ) , in which the compression term is quantified using the entropy H ( T ) rather than the mutual information I ( X ; T ) . In that Appendix , we propose an alternative objective function that can be used to resolve the first problem for dIB . We also show , in Appendix C , that our results apply when Y is not exactly a deterministic function of X , but -close to one . In this case : ( 1 ) it is hard to explore the IB curve by optimizing the IB Lagrangian , because all optimizers will fall within O ( − log ) of a single “ corner ” point on the information plane ; ( 2 ) along all points on the IB curve , there are “ uninteresting ” trivial solutions that are no more thanO ( − log ) away from being optimal ; ( 3 ) different layers of a neural networks can trade-off at most O ( − log ) amount of prediction . A recent paper ( Amjad & Geiger , 2018 ) also discusses several difficulties in using IB to analyze intermediate representations in supervised learning . That paper does not consider the particular 1Note that optimizing LβIB is still a constrained problem in that p ( t|x ) must be a valid conditional probability . However , this constraint is usually easier to handle , e.g. , by using an appropriate parameterization . issues that arise when Y is a deterministic function of X , and its arguments are complementary to ours . Shwartz-Ziv & Tishby ( 2017 ) [ Sec . 2.4 ] discuss another caveat for IB in deterministic settings , concerning the relationship between sufficient statistics and the complexity of the inputoutput mapping , which is orthogonal to the three caveats analyzed here . Finally , Saxe et al . ( 2018 ) and Amjad & Geiger ( 2018 ) observed that when T is continuous-valued and a deterministic function of a continuous-valued X , I ( X ; T ) can be unbounded , making the application of the IB framework problematic . We emphasize that the caveats discussed in this paper are unrelated to that problem . Note that our results are based on analytically-provable properties of the IB curve , i.e. , global optima of Eq . ( 1 ) , and do not concern practical issues of optimization ( which may be important in realworld scenarios ) . Our theoretical results are also independent of the practical issue of estimating MI between neural network layers , an active area of recent research ( Belghazi et al. , 2018 ; Goldfeld et al. , 2018 ; Dai et al. , 2018 ; Gabrié et al. , 2018 ) , though our empirical experiments in Section 7 rely on the estimator proposed in ( Kolchinsky & Tracey , 2017 ; Kolchinsky et al. , 2017 ) . Finally , our results are also independent of issues related to the relationship between IB , finite data sampling , and generalization error ( Shamir et al. , 2010 ; Tishby & Zaslavsky , 2015 ; Vera et al. , 2018 ) . In the next section , we review some of the connections between supervised learning and IB . In Section 3 , we show that when Y = f ( X ) , the IB curve has a piecewise linear ( not strictly concave ) shape . In Sections 4 , 5 and 6 , we discuss the three caveats mentioned above . In Section 7 , we demonstrate the caveats using a neural-network implementation of IB on the MNIST dataset . 2 SUPERVISED CLASSIFICATION AND IB . In supervised learning , one is given a training dataset { xi , yi } i=1 .. N of inputs and outputs . In this case , the random variablesX and Y refer to the inputs and outputs respectively , and p̄ ( x , y ) indicates their joint empirical distribution in the training data . It is usually assumed that the x ’ s and y ’ s are sampled i.i.d . from some “ true ” distributionw ( y|x ) w ( x ) . The high-level goal of supervised learning is to use the dataset to select a particular conditional distribution qθ ( ŷ|x ) of outputs given inputs parameterized by θ that is a good approximation of w ( y|x ) . For clarity , we use Ŷ ( and ŷ ) to indicate a random variable ( and its outcome ) corresponding to the predicted outputs . We use X to indicate the set of outcomes of X , and Y to indicate the set of outcomes of Y and Ŷ . Supervised learning is called classification when Y takes a finite set of values , and regression when Y is continuousvalued . Here we focus on classification , where the set of possible output values can be written as Y = { 0 , 1 , . . . , m } . We expect our results to also apply in some regression scenarios ( with some care regarding evaluating MI measures , see Footnote 2 ) , but leave this for future work . In practice , many supervised learning architectures use some kind of intermediate representation to make predictions about the output , such as hidden layers in neural networks . In this case , the random variable T represents the activity of some particular hidden layer in a neural network . Let T be a ( possibly stochastic ) function of the inputs , as determined by some parameterized conditional distribution qθ ( t|x ) , so that T obeys the Markov condition Y −X − T . The mapping from inputs to hidden layer activity can be either deterministic , as traditionally done in neural networks , or stochastic , as used in some architectures ( Alemi et al. , 2016 ; Kolchinsky et al. , 2017 ; Achille & Soatto , 2018 ) 2 . The mapping from hidden layer activity to predicted outputs is represented by the parameterized conditional distribution qθ ( ŷ|t ) , so the full Markov condition between true outputs , inputs , hidden layer activity , and predicted outputs is Y −X − T − Ŷ . The overall mapping from inputs to predicted outputs implemented by a neural network with a hidden layer can be written as qθ ( ŷ|x ) : = ∫ qθ ( ŷ|t ) qθ ( t|x ) dt . ( 3 ) Note that T does not have to represent a particular hidden layer , but could instead represent the activity of the neurons in a set of contiguous hidden layers , or in fact any arbitrary set of neurons that separate inputs from predicted outputs ( in the sense of conditional independence , so that Eq . ( 3 ) holds ) . More generally yet , T could also represent some intermediate representation in a non-neuralnetwork supervised learning architecture , again as long as Eq . ( 3 ) holds . 2If T is continuous-valued and a deterministic function of a continuous-valuedX , then one should consider noisy or quantized mappings from inputs to hidden layers , such that I ( X ; T ) is finite . For classification problems , training often involves selecting θ to minimize cross-entropy loss , sometimes while stochastically sampling the activity of hidden layers . When T is the last hidden layer , or when the mapping from T to the last hidden layer is deterministic , cross-entropy loss can be written LCE ( θ ) = − 1 N ∑ i ∫ qθ ( t|xi ) log qθ ( ŷi|t ) dt = Ep̄θ ( Y , T ) [ − log qθ ( Ŷ |T ) ] , ( 4 ) where E indicates expectation , p̄θ ( y , t ) : = 1N ∑ i δ ( y , yi ) qθ ( t|xi ) is the empirical distribution of outputs and hidden layer activity , and δ is the Kronecker delta . Eq . ( 4 ) can also be written as LCE ( θ ) = H ( p̄θ ( Y |T ) ) +DKL ( p̄θ ( Y |T ) ‖qθ ( Ŷ |T ) ) ( 5 ) whereH is ( conditional ) Shannon entropy , DKL is Kullback-Leibler ( KL ) divergence , and p̄θ ( y|t ) is defined via the definition of p̄θ ( y , t ) above . Since KL is non-negative , LCE ( θ ) is an upper bound on the conditional entropy H ( p̄θ ( Y |T ) ) . During training , one can decrease LCE either by decreasing the conditional entropy H ( p̄θ ( Y |T ) ) , or decreasing the KL term by changing qθ ( Ŷ |T ) to better approximate p̄θ ( Y |T ) . Minimizing LCE thus minimizes an upper bound on H ( p̄θ ( Y |T ) ) , which is equivalent to maximizing a lower bound on Ip̄θ ( Y ; T ) = H ( p̄ ( Y ) ) −H ( p̄θ ( Y |T ) ) ( since H ( p̄ ( Y ) ) is a constant that doesn ’ t depend on θ ) . We can now make explicit the relationship between supervised learning and IB . In IB , one is given a joint distribution p ( x , y ) , and then seeks a bottleneck variable T that obeys the Markov condition Y − X − T and minimizes I ( X ; T ) while maximizing I ( Y ; T ) . In supervised learning , one is given an empirical distribution p̄ ( x , y ) and defines an intermediate representation T that obeys the Markov condition Y −X−T ; during training , cross-entropy loss is minimized , which is equivalent to maximizing a lower bound on I ( Y ; T ) ( given assumptions of Eq . ( 4 ) hold ) . To complete the analogy , one might choose θ so as to also minimize I ( X ; T ) , i.e. , choose hidden layer mappings that provide compressed representations of input data ( Alemi et al. , 2016 ; Chalk et al. , 2016 ; Kolchinsky et al. , 2017 ; Achille & Soatto , 2018 ) . In fact , we use such an approach in our experiments on the MNIST dataset in Section 7 . In many supervised classification problems , there is only one correct class label associated with each possible input . Formally , this means that given the empirical training dataset distribution p̄ ( y|x ) , one can write Y = f ( X ) for some single-valued function f . This relationship between X and Y may arise because it holds under the “ true ” distribution w ( y|x ) from which the training dataset is sampled , or simply because each input vector x occurs at most once in the training data ( as happens in most real-world classification training datasets ) . Note that we make no assumptions about the relationship between X and Ŷ under qθ ( ŷ|x ) , the distribution of predicted outputs given inputs implemented by the supervised learning architecture , and our analysis holds even if qθ ( ŷ|x ) is nondeterministic ( e.g. , the softmax function ) . It is important to note that not all classification problems are deterministic . The map from X to Y may be intrinsically noisy or non-deterministic ( e.g. , if human labelers can not agree on a class for some x ) , or noise may be intentionally added as a regularization technique ( e.g. , as done in the label smoothing method ( Szegedy et al. , 2016 ) ) . Moreover , one of the pioneering papers on the relationship between IB and supervised learning ( Shwartz-Ziv & Tishby , 2017 ) analyzed an artificially-constructed classification problem in which p ( y|x ) was explicitly defined to be noisy ( see their Eq . 10 ) . However , we believe that many if not most real-world classification problems do in fact exhibit a deterministic relation between X and Y . In addition , as we show in Appendix C , even if the relationship between X and Y is not perfectly deterministic but close to being so , then the three caveats discussed in this paper still apply in an approximate sense .

Summary: This paper is about potential problems of the information bottleneck principle in cases where the output variable Y is a deterministic function of the inputs X. Such a deterministic relationship between outputs and inputs induces the problem that the the IB "information curve" (i.e. I(T;Y) as a function of I(X;T)) is piece-wise linear and, thus, no longer strictly concave, which is crucial for non-degenerate ("interesting") solutions. The authors argue that most real classification problems indeed show such a deterministic relation between the class labels and the inputs X, and they explore several issues that result from such pathologies.