Information-theoretic generalization bounds for black-box learning algorithms

We derive information-theoretic generalization bounds for supervised learning algorithms based on the information contained in predictions rather than in the output of the training algorithm . These bounds improve over the existing informationtheoretic bounds , are applicable to a wider range of algorithms , and solve two key challenges : ( a ) they give meaningful results for deterministic algorithms and ( b ) they are significantly easier to estimate . We show experimentally that the proposed bounds closely follow the generalization gap in practical scenarios for deep learning . 1 Introduction . Large neural networks trained with variants of stochastic gradient descent have excellent generalization capabilities , even in regimes where the number of parameters is much larger than the number of training examples . Zhang et al . [ 41 ] showed that classical generalization bounds based on various notions of complexity of hypothesis set fail to explain this phenomenon , as the same neural network can generalize well for one choice of training data and memorize completely for another one . This observation has spurred a tenacious search for algorithm-dependent and data-dependent generalization bounds that give meaningful results in practical settings for deep learning [ 17 ] . One line of attack bounds generalization error based on the information about training dataset stored in the weights [ 39 , 4 , 22 , 6 , 33 , 14 , 23 , 27 ] . The main idea is that when the training and testing performance of a neural network are different , the network weights necessarily capture some information about the training dataset . However , the opposite might not be true : A neural network can store significant portions of training set in its weights and still generalize well [ 32 , 40 , 21 ] . Furthermore , because of their information-theoretic nature , these generalization bounds become infinite or produce trivial bounds for deterministic algorithms . When such bounds are not infinite , they are notoriously hard to estimate , due to the challenges arising in estimation of Shannon mutual information between two high-dimensional variables ( e.g. , weights of a ResNet and a training dataset ) . This work addresses the aforementioned challenges . We first improve some of the existing information-theoretic generalization bounds , providing a unified view and derivation of them ( Sec . 2 ) . We then derive novel generalization bounds that measure information with predictions , rather than with the output of the training algorithm ( Sec . 3 ) . These bounds are applicable to a wide range of methods , including neural networks , Bayesian algorithms , ensembling algorithms , and non-parametric approaches . In the case of neural networks , the proposed bounds improve over the existing weightbased bounds , partly because they avoid a counter-productive property of weight-based bounds that information stored in unused weights affects generalization bounds , even though it has no effect on generalization . The proposed bounds produce meaningful results for deterministic algorithms and are significantly easier to estimate . For example , in case of classification , computing our most efficient bound involves estimating mutual information between a pair of predictions and a binary variable . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . We apply the proposed bounds to ensembling algorithms , binary classification algorithms with finite VC dimension hypothesis classes , and to stable learning algorithms ( Sec . 4 ) . We compute our most efficient bound on realistic classification problems involving neural networks , and show that the bound closely follows the generalization error , even in situations when a neural network with 3M parameters is trained deterministically on 4000 examples , achieving 1 % generalization error . 2 Weight-based generalization bounds . We start by describing the necessary notation and definitions , after which we present some of the existing weigh-based information-theoretic generalization bounds , slightly improve some of them , and prove relations between them . The purpose of this section is to introduce the relevant existing bounds and prepare grounds for the functional conditional mutual information bounds introduced in Sec . 3 , which we consider our main contribution . All proofs are presented in Appendix A. Preliminaries . We use capital letters for random variables , corresponding lowercase letters for their values , and calligraphic letters for their domains . If X is a random variable , X̄ denotes an independent copy of X . For example , if ( X , Y ) is a pair of random variables with joint distribution PX , Y , then the joint distribution of ( X̄ , Ȳ ) will be PX̄ , Ȳ = PX̄ ⊗ PȲ = PX ⊗ PY . A random variable X is called σ-subgaussian if E exp ( t ( X − EX ) ) ≤ exp ( σ2t2/2 ) , ∀t ∈ R. For example , a random variable that takes values in [ a , b ] almost surely , is ( b− a ) /2-subgaussian . Given probability measures P and Q defined on the same measurable space , such that P is absolutely continuous with respect to Q , the Kullback–Leibler divergence from P to Q is defined as KL ( P ‖ Q ) = ∫ log dPdQdP , where dPdQ is the Radon-Nikodym derivative of P with respect to Q . If X and Y are random variables defined on the same probability space , then KL ( X ‖ Y ) denotes KL ( PX ‖ PY ) . The Shannon mutual information between random variables X and Y is I ( X ; Y ) = KL ( PX , Y ‖ PX ⊗ PY ) . In this paper , all information-theoretic quantities are measured in nats , instead of bits . Throughout the paper [ n ] denotes the set { 1 , 2 , . . . , n } . Finally , if A = ( a1 , . . . , an ) is a collection , then A−i , ( a1 , . . . , ai−1 , ai+1 , . . . , an ) . Theorems proved in the subsequent sections will be relying on the following lemma . Lemma 1 . Let ( Φ , Ψ ) be a pair of random variables with joint distribution PΨ , Φ . If g ( φ , ψ ) is a measurable function such that EΦ , Ψ [ g ( Φ , Ψ ) ] exists and g ( Φ̄ , Ψ̄ ) is σ-subgaussian , then∣∣EΦ , Ψ [ g ( Φ , Ψ ) ] − EΦ , Ψ̄ [ g ( Φ , Ψ̄ ) ] ∣∣ ≤√2σ2I ( Φ ; Ψ ) . ( 1 ) Furthermore , if g ( φ , Ψ̄ ) is σ-subgaussian for each φ and the expectation below exists , then EΦ , Ψ [ ( g ( Φ , Ψ ) − EΨ̄ g ( Φ , Ψ̄ ) ) 2 ] ≤ 4σ2 ( I ( Φ ; Ψ ) + log 3 ) , ( 2 ) and P ( ∣∣g ( Φ , Ψ ) − EΨ̄ g ( Φ , Ψ̄ ) ∣∣ ≥ ) ≤ 4σ2 ( I ( Φ ; Ψ ) + log 3 ) 2 , ∀ > 0 . ( 3 ) The first part of this lemma is equivalent to Lemma 1 of Xu and Raginsky [ 39 ] , which in turn has its roots in Russo and Zou [ 29 ] . The second part generalizes Lemma 2 of Hafez-Kolahi et al . [ 13 ] by also providing bounds on the expected squared difference . 2.1 Generalization bounds with input-output mutual information . Let S = ( Z1 , Z2 , . . . , Zn ) ∼ Dn be a dataset of n i.i.d . examples , R ∈ R be a source of randomness ( a random variable independent of S ) and A : Zn × R → W be a training algorithm . Let W = A ( S , R ) be the output of the training algorithm applied on the dataset S with randomness R. Given a loss function ` : W ×Z → R , the empirical risk is Lemp ( A , S , R ) = 1n ∑n i=1 ` ( W , Zi ) and the population risk is L ( A , S , R ) = EZ′∼D ` ( W , Z ′ ) , where Z ′ is a test example independent from S and R. The generalization gap , also call generalization error , is L ( A , S , R ) −Lemp ( A , S , R ) . In this setting , Xu and Raginsky [ 39 ] establish the following information-theoretic bound on the absolute value of the expected generalization gap . Theorem 2.1 ( Thm . 1 of Xu and Raginsky [ 39 ] ) . If ` ( w , Z ′ ) , where Z ′ ∼ D , is σ-subgaussian for all w ∈ W , then |ES , R [ L ( A , S , R ) − Lemp ( A , S , R ) ] | ≤ √ 2σ2I ( W ; S ) n . ( 4 ) We generalize this result by showing that instead of measuring information with the entire dataset , one can measure information with a subset of size m chosen uniformly at random . For brevity , hereafter we call subsets chosen uniformly at random just “ random subsets ” . Theorem 2.2 . Let U be a random subset of [ n ] with size m , independent of S and R. If ` ( w , Z ′ ) , where Z ′ ∼ D , is σ-subgaussian for all w ∈ W , then |ES , R [ L ( A , S , R ) − Lemp ( A , S , R ) ] | ≤ Eu∼U √ 2σ2 m I ( W ; Su ) , ( 5 ) and ES , R ( L ( A , S , R ) − Lemp ( A , S , R ) ) 2 ≤ 4σ2 n ( I ( W ; S ) + log 3 ) . ( 6 ) With a simple application of Markov ’ s inequality one can get tail bounds from the second part of the theorem . Furthermore , by taking square root of both sides of ( 6 ) and using Jensen ’ s inequality on the left side , one can also construct an upper bound for the expected absolute value of generalization gap , ES , R |L ( A , S , R ) − Lemp ( A , S , R ) | . These observations apply also to the other generalization gap bounds presented later in this work . Note the bound on the squared generalization gap is written only for the case of m = n. It is possible to derive squared generalization gap bounds of form 4σ 2 m ( Eu∼U I ( W ; Su ) + log 3 ) . Unfortunately , for small m the log 3 constant starts to dominate , resulting in vacuous bounds . Picking a small m decreases the mutual information term in ( 5 ) , however , it also decreases the denominator . When settingm = n , we get the bound of Xu and Raginsky [ 39 ] ( Thm . 2.1 ) . Whenm = 1 , the bound of ( 5 ) becomes 1n ∑n i=1 √ 2σ2I ( W ; Zi ) , matching the result of Bu et al . [ 6 ] ( Proposition 1 ) . A similar bound , but for a different notion of information , was derived by Alabdulmohsin [ 2 ] . Bu et al . [ 6 ] prove that the bound with m = 1 is tighter than the bound with m = n. We generalize this result by proving that the bound of ( 5 ) is non-descreasing in m. Proposition 1 . Let m ∈ [ n− 1 ] , U be a random subset of [ n ] of size m , U ′ be a random subset of size m+ 1 , and φ : R→ R be any non-decreasing concave function . Then EU φ ( 1 m I ( W ; Su ) ) ≤ EU ′ φ ( 1 m+ 1 I ( W ; Su′ ) ) . ( 7 ) When φ ( x ) = √ x , this result proves that the optimal value for m in ( 5 ) is 1 . Furthermore , when we use Jensen ’ s inequality to move expectation over U inside the square root in ( 5 ) , then the resulting bound becomes √ 2σ2 m Eu∼U I ( W ; Su ) and matches the result of Negrea et al . [ 22 ] ( Thm . 2.3 ) . These bounds are also non-decreasing with respect to m ( using Proposition 1 with φ ( x ) = x ) . Thm . 2.1 can be used to derive generalization bounds that depend on the information between W and a single example Zi conditioned on the remaining examples Z−i = ( Z1 , . . . , Zi−1 , Zi+1 , . . . , Zn ) . Theorem 2.3 . If ` ( w , Z ′ ) , where Z ′ ∼ D , is σ-subgaussian for all w ∈ W , then |ES , R [ L ( A , S , R ) − Lemp ( A , S , R ) ] | ≤ 1 n n∑ i=1 √ 2σ2I ( W ; Zi | Z−i ) , ( 8 ) and ES , R ( L ( A , S , R ) − Lemp ( A , S , R ) ) 2 ≤ 4σ2 n ( n∑ i=1 I ( W ; Zi | Z−i ) + log 3 ) . ( 9 ) This theorem is a simple corollary of Thm . 2.2 , using the facts that I ( W ; Zi ) ≤ I ( W ; Zi | Z−i ) and that I ( W ; S ) is upper bounded by ∑n i=1 I ( W ; Zi | Z−i ) , which is also known as erasure information [ 35 ] . The first part of it improves the result of Raginsky et al . [ 26 ] ( Thm . 2 ) , as the averaging over i is outside of the square root . While these bounds are worse that the corresponding bounds of Thm . 2.2 , it is sometimes easier to manipulate them analytically . The bounds described above measure information with the output W of the training algorithm . In the case of prediction tasks with parametric methods , the parameters W might contain information about the training dataset , but not use it to make predictions . Partly for this reason , the main goal of this paper is to derive generalization bounds that measure information with the prediction function , rather than with the weights . In general , there is no straightforward way of encoding the prediction function into a random variable . However , when the domain Z is finite , we can encode the prediction function as the collection of predictions on all examples of Z . This naturally leads us to the next setting ( albeit with a different motivation ) , first considered by Steinke and Zakynthinou [ 33 ] , where one first fixes a set of 2n examples , and then randomly selects n of them to form the training set . We use this setting to provide prediction-based generalization bounds in Sec . 3 . Before describing these bounds we present the setting of Steinke and Zakynthinou [ 33 ] in detail and generalize some of the existing weight-based bounds in that setting .

Summary: The authors build upon recent literature on information theoretic bounds on generalization error and specifically the conditional mutual information bound by Steinke and Zakynthinou [28]. They propose an improved bound that is tigther than [28] and easier to compute in practice. It is achieved by replacing the weight-parameter vector in the mutual information expression of [28] with a prediction function evaluated on a subset of in-sample (training) data and a subset of out-of-sample data.  This enables the authors to give performance guarantees on deterministic training algorithms. Several toy examples and experimental results are presented to illustrate the advantages of the improved generalization bound.