Information-theoretic generalization bounds for black-box learning algorithms

We derive information-theoretic generalization bounds for supervised learning algorithms based on the information contained in predictions rather than in the output of the training algorithm . These bounds improve over the existing informationtheoretic bounds , are applicable to a wider range of algorithms , and solve two key challenges : ( a ) they give meaningful results for deterministic algorithms and ( b ) they are significantly easier to estimate . We show experimentally that the proposed bounds closely follow the generalization gap in practical scenarios for deep learning . 1 Introduction . Large neural networks trained with variants of stochastic gradient descent have excellent generalization capabilities , even in regimes where the number of parameters is much larger than the number of training examples . Zhang et al . [ 41 ] showed that classical generalization bounds based on various notions of complexity of hypothesis set fail to explain this phenomenon , as the same neural network can generalize well for one choice of training data and memorize completely for another one . This observation has spurred a tenacious search for algorithm-dependent and data-dependent generalization bounds that give meaningful results in practical settings for deep learning [ 17 ] . One line of attack bounds generalization error based on the information about training dataset stored in the weights [ 39 , 4 , 22 , 6 , 33 , 14 , 23 , 27 ] . The main idea is that when the training and testing performance of a neural network are different , the network weights necessarily capture some information about the training dataset . However , the opposite might not be true : A neural network can store significant portions of training set in its weights and still generalize well [ 32 , 40 , 21 ] . Furthermore , because of their information-theoretic nature , these generalization bounds become infinite or produce trivial bounds for deterministic algorithms . When such bounds are not infinite , they are notoriously hard to estimate , due to the challenges arising in estimation of Shannon mutual information between two high-dimensional variables ( e.g. , weights of a ResNet and a training dataset ) . This work addresses the aforementioned challenges . We first improve some of the existing information-theoretic generalization bounds , providing a unified view and derivation of them ( Sec . 2 ) . We then derive novel generalization bounds that measure information with predictions , rather than with the output of the training algorithm ( Sec . 3 ) . These bounds are applicable to a wide range of methods , including neural networks , Bayesian algorithms , ensembling algorithms , and non-parametric approaches . In the case of neural networks , the proposed bounds improve over the existing weightbased bounds , partly because they avoid a counter-productive property of weight-based bounds that information stored in unused weights affects generalization bounds , even though it has no effect on generalization . The proposed bounds produce meaningful results for deterministic algorithms and are significantly easier to estimate . For example , in case of classification , computing our most efficient bound involves estimating mutual information between a pair of predictions and a binary variable . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . We apply the proposed bounds to ensembling algorithms , binary classification algorithms with finite VC dimension hypothesis classes , and to stable learning algorithms ( Sec . 4 ) . We compute our most efficient bound on realistic classification problems involving neural networks , and show that the bound closely follows the generalization error , even in situations when a neural network with 3M parameters is trained deterministically on 4000 examples , achieving 1 % generalization error . 2 Weight-based generalization bounds . We start by describing the necessary notation and definitions , after which we present some of the existing weigh-based information-theoretic generalization bounds , slightly improve some of them , and prove relations between them . The purpose of this section is to introduce the relevant existing bounds and prepare grounds for the functional conditional mutual information bounds introduced in Sec . 3 , which we consider our main contribution . All proofs are presented in Appendix A. Preliminaries . We use capital letters for random variables , corresponding lowercase letters for their values , and calligraphic letters for their domains . If X is a random variable , X̄ denotes an independent copy of X . For example , if ( X , Y ) is a pair of random variables with joint distribution PX , Y , then the joint distribution of ( X̄ , Ȳ ) will be PX̄ , Ȳ = PX̄ ⊗ PȲ = PX ⊗ PY . A random variable X is called σ-subgaussian if E exp ( t ( X − EX ) ) ≤ exp ( σ2t2/2 ) , ∀t ∈ R. For example , a random variable that takes values in [ a , b ] almost surely , is ( b− a ) /2-subgaussian . Given probability measures P and Q defined on the same measurable space , such that P is absolutely continuous with respect to Q , the Kullback–Leibler divergence from P to Q is defined as KL ( P ‖ Q ) = ∫ log dPdQdP , where dPdQ is the Radon-Nikodym derivative of P with respect to Q . If X and Y are random variables defined on the same probability space , then KL ( X ‖ Y ) denotes KL ( PX ‖ PY ) . The Shannon mutual information between random variables X and Y is I ( X ; Y ) = KL ( PX , Y ‖ PX ⊗ PY ) . In this paper , all information-theoretic quantities are measured in nats , instead of bits . Throughout the paper [ n ] denotes the set { 1 , 2 , . . . , n } . Finally , if A = ( a1 , . . . , an ) is a collection , then A−i , ( a1 , . . . , ai−1 , ai+1 , . . . , an ) . Theorems proved in the subsequent sections will be relying on the following lemma . Lemma 1 . Let ( Φ , Ψ ) be a pair of random variables with joint distribution PΨ , Φ . If g ( φ , ψ ) is a measurable function such that EΦ , Ψ [ g ( Φ , Ψ ) ] exists and g ( Φ̄ , Ψ̄ ) is σ-subgaussian , then∣∣EΦ , Ψ [ g ( Φ , Ψ ) ] − EΦ , Ψ̄ [ g ( Φ , Ψ̄ ) ] ∣∣ ≤√2σ2I ( Φ ; Ψ ) . ( 1 ) Furthermore , if g ( φ , Ψ̄ ) is σ-subgaussian for each φ and the expectation below exists , then EΦ , Ψ [ ( g ( Φ , Ψ ) − EΨ̄ g ( Φ , Ψ̄ ) ) 2 ] ≤ 4σ2 ( I ( Φ ; Ψ ) + log 3 ) , ( 2 ) and P ( ∣∣g ( Φ , Ψ ) − EΨ̄ g ( Φ , Ψ̄ ) ∣∣ ≥ ) ≤ 4σ2 ( I ( Φ ; Ψ ) + log 3 ) 2 , ∀ > 0 . ( 3 ) The first part of this lemma is equivalent to Lemma 1 of Xu and Raginsky [ 39 ] , which in turn has its roots in Russo and Zou [ 29 ] . The second part generalizes Lemma 2 of Hafez-Kolahi et al . [ 13 ] by also providing bounds on the expected squared difference . 2.1 Generalization bounds with input-output mutual information . Let S = ( Z1 , Z2 , . . . , Zn ) ∼ Dn be a dataset of n i.i.d . examples , R ∈ R be a source of randomness ( a random variable independent of S ) and A : Zn × R → W be a training algorithm . Let W = A ( S , R ) be the output of the training algorithm applied on the dataset S with randomness R. Given a loss function ` : W ×Z → R , the empirical risk is Lemp ( A , S , R ) = 1n ∑n i=1 ` ( W , Zi ) and the population risk is L ( A , S , R ) = EZ′∼D ` ( W , Z ′ ) , where Z ′ is a test example independent from S and R. The generalization gap , also call generalization error , is L ( A , S , R ) −Lemp ( A , S , R ) . In this setting , Xu and Raginsky [ 39 ] establish the following information-theoretic bound on the absolute value of the expected generalization gap . Theorem 2.1 ( Thm . 1 of Xu and Raginsky [ 39 ] ) . If ` ( w , Z ′ ) , where Z ′ ∼ D , is σ-subgaussian for all w ∈ W , then |ES , R [ L ( A , S , R ) − Lemp ( A , S , R ) ] | ≤ √ 2σ2I ( W ; S ) n . ( 4 ) We generalize this result by showing that instead of measuring information with the entire dataset , one can measure information with a subset of size m chosen uniformly at random . For brevity , hereafter we call subsets chosen uniformly at random just “ random subsets ” . Theorem 2.2 . Let U be a random subset of [ n ] with size m , independent of S and R. If ` ( w , Z ′ ) , where Z ′ ∼ D , is σ-subgaussian for all w ∈ W , then |ES , R [ L ( A , S , R ) − Lemp ( A , S , R ) ] | ≤ Eu∼U √ 2σ2 m I ( W ; Su ) , ( 5 ) and ES , R ( L ( A , S , R ) − Lemp ( A , S , R ) ) 2 ≤ 4σ2 n ( I ( W ; S ) + log 3 ) . ( 6 ) With a simple application of Markov ’ s inequality one can get tail bounds from the second part of the theorem . Furthermore , by taking square root of both sides of ( 6 ) and using Jensen ’ s inequality on the left side , one can also construct an upper bound for the expected absolute value of generalization gap , ES , R |L ( A , S , R ) − Lemp ( A , S , R ) | . These observations apply also to the other generalization gap bounds presented later in this work . Note the bound on the squared generalization gap is written only for the case of m = n. It is possible to derive squared generalization gap bounds of form 4σ 2 m ( Eu∼U I ( W ; Su ) + log 3 ) . Unfortunately , for small m the log 3 constant starts to dominate , resulting in vacuous bounds . Picking a small m decreases the mutual information term in ( 5 ) , however , it also decreases the denominator . When settingm = n , we get the bound of Xu and Raginsky [ 39 ] ( Thm . 2.1 ) . Whenm = 1 , the bound of ( 5 ) becomes 1n ∑n i=1 √ 2σ2I ( W ; Zi ) , matching the result of Bu et al . [ 6 ] ( Proposition 1 ) . A similar bound , but for a different notion of information , was derived by Alabdulmohsin [ 2 ] . Bu et al . [ 6 ] prove that the bound with m = 1 is tighter than the bound with m = n. We generalize this result by proving that the bound of ( 5 ) is non-descreasing in m. Proposition 1 . Let m ∈ [ n− 1 ] , U be a random subset of [ n ] of size m , U ′ be a random subset of size m+ 1 , and φ : R→ R be any non-decreasing concave function . Then EU φ ( 1 m I ( W ; Su ) ) ≤ EU ′ φ ( 1 m+ 1 I ( W ; Su′ ) ) . ( 7 ) When φ ( x ) = √ x , this result proves that the optimal value for m in ( 5 ) is 1 . Furthermore , when we use Jensen ’ s inequality to move expectation over U inside the square root in ( 5 ) , then the resulting bound becomes √ 2σ2 m Eu∼U I ( W ; Su ) and matches the result of Negrea et al . [ 22 ] ( Thm . 2.3 ) . These bounds are also non-decreasing with respect to m ( using Proposition 1 with φ ( x ) = x ) . Thm . 2.1 can be used to derive generalization bounds that depend on the information between W and a single example Zi conditioned on the remaining examples Z−i = ( Z1 , . . . , Zi−1 , Zi+1 , . . . , Zn ) . Theorem 2.3 . If ` ( w , Z ′ ) , where Z ′ ∼ D , is σ-subgaussian for all w ∈ W , then |ES , R [ L ( A , S , R ) − Lemp ( A , S , R ) ] | ≤ 1 n n∑ i=1 √ 2σ2I ( W ; Zi | Z−i ) , ( 8 ) and ES , R ( L ( A , S , R ) − Lemp ( A , S , R ) ) 2 ≤ 4σ2 n ( n∑ i=1 I ( W ; Zi | Z−i ) + log 3 ) . ( 9 ) This theorem is a simple corollary of Thm . 2.2 , using the facts that I ( W ; Zi ) ≤ I ( W ; Zi | Z−i ) and that I ( W ; S ) is upper bounded by ∑n i=1 I ( W ; Zi | Z−i ) , which is also known as erasure information [ 35 ] . The first part of it improves the result of Raginsky et al . [ 26 ] ( Thm . 2 ) , as the averaging over i is outside of the square root . While these bounds are worse that the corresponding bounds of Thm . 2.2 , it is sometimes easier to manipulate them analytically . The bounds described above measure information with the output W of the training algorithm . In the case of prediction tasks with parametric methods , the parameters W might contain information about the training dataset , but not use it to make predictions . Partly for this reason , the main goal of this paper is to derive generalization bounds that measure information with the prediction function , rather than with the weights . In general , there is no straightforward way of encoding the prediction function into a random variable . However , when the domain Z is finite , we can encode the prediction function as the collection of predictions on all examples of Z . This naturally leads us to the next setting ( albeit with a different motivation ) , first considered by Steinke and Zakynthinou [ 33 ] , where one first fixes a set of 2n examples , and then randomly selects n of them to form the training set . We use this setting to provide prediction-based generalization bounds in Sec . 3 . Before describing these bounds we present the setting of Steinke and Zakynthinou [ 33 ] in detail and generalize some of the existing weight-based bounds in that setting .

Summary: The paper derives information-theoretic bounds for learning algorithms. The bounds are based on the predictions of the learned model, rather than the weights (output), which makes them tighter by the data-processing inequality and can yield non-vacuous bounds in cases where weight-based mutual information bounds are vacuous. The authors validate the analysis with a few experiments.