Title: Recursive PAC-Bayes: A Frequentist Approach to Sequential Prior Updates with No Information Loss
Venue: NeurIPS 2024
Accepted: yes
Keywords: PAC-Bayes

data-dependent prior
Abstract: PAC-Bayesian analysis is a frequentist framework for incorporating prior knowledge into learning. It was inspired by Bayesian learning, which allows sequential data processing and naturally turns posteriors from one processing step into priors for the next. However, despite two and a half decades of research, the ability to update priors sequentially without losing confidence information along the way remained elusive for PAC-Bayes. While PAC-Bayes allows construction of data-informed priors, the final confidence intervals depend only on the number of points that were not used for the construction of the prior, whereas confidence information in the prior, which is related to the number of points used to construct the prior, is lost. This limits the possibility and benefit of sequential prior updates, because the final bounds depend only on the size of the final batch.

We present a novel and, in retrospect, surprisingly simple and powerful PAC-Bayesian procedure that allows  sequential prior updates with no information loss. The procedure is based on a novel decomposition of the expected loss of randomized classifiers. The decomposition rewrites the loss of the posterior as an excess loss relative to a downscaled loss of the prior plus the downscaled loss of the prior, which is bounded recursively. As a side result, we also present a generalization of the split-kl and PAC-Bayes-split-kl inequalities to discrete random variables, which we use for bounding the excess losses, and which can be of independent interest. In empirical evaluation the new procedure significantly outperforms state-of-the-art.

Recursive PAC-Bayes: A Frequentist Approach to Sequential Prior Updates with No Information Loss

Abstract

PAC-Bayesian analysis is a frequentist framework for incorporating prior knowledge into learning. It was inspired by Bayesian learning, which allows sequential data processing and naturally turns posteriors from one processing step into priors for the next. However, despite two and a half decades of research, the ability to update priors sequentially without losing confidence information along the way remained elusive for PAC-Bayes. While PAC-Bayes allows construction of datainformed priors, the final confidence intervals depend only on the number of points that were not used for the construction of the prior, whereas confidence information in the prior, which is related to the number of points used to construct the prior, is lost. This limits the possibility and benefit of sequential prior updates, because the final bounds depend only on the size of the final batch.

We present a novel and, in retrospect, surprisingly simple and powerful PACBayesian procedure that allows sequential prior updates with no information loss. The procedure is based on a novel decomposition of the expected loss of randomized classifiers. The decomposition rewrites the loss of the posterior as an excess loss relative to a downscaled loss of the prior plus the downscaled loss of the prior, which is bounded recursively. As a side result, we also present a generalization of the split-kl and PAC-Bayes-split-kl inequalities to discrete random variables, which we use for bounding the excess losses, and which can be of independent interest. In empirical evaluation the new procedure significantly outperforms state-of-the-art.

1 Introduction

PAC-Bayesian analysis was born from an attempt to derive frequentist generalization guarantees for Bayesian-style prediction rules (Shawe-Taylor and Williamson, 1997, McAllester, 1998). The motivation was to provide a way to incorporate prior knowledge into the frequentist analysis of generalization. PAC-Bayesian bounds provide high-probability generalization guarantees for randomized classifiers. A randomized classifier is defined by a distribution ρ on a set of prediction rules H , which is used to sample a prediction rule each time a prediction is to be made. Bayesian posterior is an example of a randomized classifier, whereas PAC-Bayesian bounds hold generally for all randomized classifiers. Prior knowledge is encoded through a prior distribution π on H , and the complexity of a posterior distribution ρ is measured by the Kullback-Leibler (KL) divergence from the prior, KL( ρ ∥ π ) . PAC-Bayesian generalization guarantees are optimized by posterior distributions ρ that optimize a trade-off between empirical data fit and divergence from the prior in the KL sense.

Selection of a 'good' prior plays an important role in the PAC-Bayesian bounds. If one manages to foresee which prediction rules are likely to produce low prediction error and allocate a higher prior mass for them, then the bounds are tighter, because the posterior only needs to make a small deviation from the prior. But if the prior mass on well-performing prediction rules is small, the bounds are loose. A major technique to design good priors is to use part of the data to estimate a good prior and the rest of the data to compute a PAC-Bayes bound. It is known as data-dependent or data-informed priors (Ambroladze et al., 2007). However, all existing approaches to data-informed priors have three major disadvantages. The first is that the bounds are computed on 'the rest of the data' that were not used in construction of the prior. Thus, the sample size in the bounds is only a fraction of the total sample size. Therefore, empirically data-informed priors are not always helpful. In many cases starting with an uninformed prior and using all the data to compute the posterior and the bound turns to be superior to sacrificing part of the data for prior construction (Ambroladze et al., 2007, Mhammedi et al., 2019). The second disadvantage is that all the confidence information about the prior is lost in the process. In particular, a prior trained on a few data points is treated in the same way as a prior trained on a lot of data. And a third related disadvantage is that sequential data processing provides no benefit, because the bounds only depend on the size of the last chunk and all the confidence information from processing earlier chunks is lost in the process.

Our main contribution is a new (and simple) way of decomposing the loss of a randomized classifier defined by the posterior. We write it as an excess loss relative to a downscaled loss of the randomized classifier defined by the prior plus the downscaled loss of the randomized classifier defined by the prior. The excess loss can be bounded using PAC-Bayes-Empirical-Bernstein-style inequalities (Tolstikhin and Seldin, 2013, Mhammedi et al., 2019, Wu et al., 2021, Wu and Seldin, 2022), whereas the loss of the randomized classifier defined by the prior can be bounded recursively. The recursive bound can both use the data used for construction of the prior and 'the rest of the data', and thereby preserves confidence information on the prior. Our contribution stands out relative to all prior work on PAC-Bayes, and in fact all prior work on frequentist generalization bounds, because it makes sequential data processing and sequential prior updates meaningful and beneficial.

We note that while several recent papers experimented with sequential posterior updates by using martingale-style analysis, in all these works the prior remained fixed and only the posterior was changing (Chugg et al., 2023, Biggs and Guedj, 2023, Rodríguez-Gálvez et al., 2024). The work on sequential posterior updates is orthogonal to our contribution and can be combined with it. Namely, it is possible to apply sequential posterior updates in-between sequential prior updates. Another line of work used tools from online learning to derive PAC-Bayesian bounds (Jang et al., 2023), and in this context Haddouche and Guedj (2023) have used sequential prior updates, but their bounds hold for a uniform aggregation of sequentially constructed posteriors, which is different from standard posteriors studied in our work. The confidence bounds in their work come primarily from aggregation rather than confidence in individual posteriors in the sequence (the denominator of their bounds depends on the number of aggregated posteriors). The need to construct and maintain a large number of posteriors has a negative impact on the computational efficiency. Our work is the first one allowing sequential prior updates without loss of confidence information.

An additional side contribution of independent interest is a generalization of the split-kl and PACBayes-split-kl inequalities of Wu and Seldin (2022) from ternary to general discrete random variables. It is based on a novel representation of discrete random variables as a superposition of Bernoulli random variables.

The paper is organized in the following way. In Section 2 we briefly survey the evolution of datainformed priors in PAC-Bayes and present our main idea behind Recursive PAC-Bayes; in Section 3 we present our generalization of the split-kl and PAC-Bayes-split-kl inequalities, which are later used to bound the excess losses; in Section 4 we present the Recursive PAC-Bayes bound; in Section 5 we present an empirical evaluation; and in Section 6 we conclude with a discussion.

2 The evolution of data-informed priors and the idea of Recursive PAC-Bayes

̸

In this section we briefly survey the evolution of data-informed priors, and then present our construction of Recursive PAC-Bayes. We consider the standard classification setting, with X being a sample space, Y a label space, H a set of prediction rules h : X → Y , and ℓ ( h ( X ) , Y ) = 1 ( h ( X ) = Y ) the zero-one loss function, where 1 ( · ) denotes the indicator function. We let D

denote a distribution on X × Y and S = { ( X 1 , Y 1 ) , . . . , ( X n , Y n ) } an i.i.d. sample from D . Let L ( h ) = E ( X,Y ) ∼D [ ℓ ( h ( X ) , Y )] be the expected and ˆ L ( h, S ) = 1 n ∑ n i =1 ℓ ( h ( X i ) , Y i ) the empirical loss.

Let ρ be a distribution on H . A randomized classifier associated with ρ samples a prediction rule h according to ρ for each sample X ∈ X , and applies it to make a prediction h ( X ) . The expected loss of such randomized classifier, which we call ρ , is E h ∼ ρ [ L ( h )] and the empirical loss is E h ∼ ρ [ ˆ L ( h, S )] . For brevity we use E ρ [ · ] to denote E h ∼ ρ [ · ] .

We use KL( ρ ∥ π ) to denote the Kullback-Leibler divergence between two probability distributions, ρ and π (Cover and Thomas, 2006). For p, q ∈ [0 , 1] we further use kl( p ∥ q ) = KL((1 -p, p ) ∥ (1 -q, q )) to denote the Kullback-Leibler divergence between two Bernoulli distributions with biases p and q .

The goal of PAC-Bayes is to bound E ρ [ L ( h )] . Below we present how the bounds on E ρ [ L ( h )] have evolved. In Appendix A we also provide a graphical illustration of the evolution.

Uninformed priors Early work on PAC-Bayes used uninformed priors (McAllester, 1998). An uniformed prior π is a distribution on H that is independent of the data S . A classical, and still one of the tightest bounds, is the following.

Theorem 1 (PAC-Bayeskl Inequality, Seeger, 2002, Maurer, 2004) . For any probability distribution π on H that is independent of S and any δ ∈ (0 , 1) :

where P is the set of all probability distributions on H , including those dependent on S .

A posterior ρ that minimizes E ρ [ L ( h )] has to balance between allocating higher mass to prediction rules h with small ˆ L ( h, S ) and staying close to π in the KL( ρ ∥ π ) sense. Since π has to be independent of S , typical uninformed priors aim 'to leave maximal options open' for ρ by staying close to uniform.

Data-informed priors Ambroladze et al. (2007) proposed to split the data S into two disjoint sets, S = S 1 ∪ S 2 , and use S 1 to construct a data-informed prior π and compute a bound on E ρ [ L ( h )] using π and S 2 . Since in this approach π is independent of S 2 , Theorem 1 can be applied. The advantage is that π can use S 1 to give higher mass to promising classifiers, thus relaxing the regularization pressure KL( ρ ∥ π ) and making it easier for ρ to allocate even higher mass to well-performing classifiers (those with small ˆ L ( h, S 2 ) ). The disadvantage is that the sample size in the bound (the n in the denominator) decreases from the size of S to the size of S 2 . Indeed, Ambroladze et al. observed that the sacrifice of S 1 for prior construction does not always pay off.

Data-informed priors + excess loss Mhammedi et al. (2019) observed that if we have already sacrificed S 1 for the construction of π , we could also use it to construct a reference prediction rule h ∗ , typically an Empirical Risk Minimizer (ERM) on S 1 . They then employed the decomposition

and used S 2 to give a PAC-Bayesian bound on E ρ [ L ( h ) -L ( h ∗ )] and a single-hypothesis bound on L ( h ∗ ) . The quantity E ρ [ L ( h ) -L ( h ∗ )] is known as excess loss . The advantage of this approach is that when L ( h ∗ ) is a good approximation of E ρ [ L ( h )] , the excess loss has lower variance than the plain loss E ρ [ L ( h )] and, therefore, is more efficient to bound, whereas the single-hypothesis bound on L ( h ∗ ) does not involve the KL( ρ ∥ π ) term. Therefore, it is generally beneficial to use excess losses in combination with data-informed priors. However, as with the previous approach, sacrificing S 1 to learn π and h ∗ means that the denominator in the bounds ( n in Theorem 1) reduces to the size of S 2 , and it does not always pay off. (We note that the excess loss is not binary and not in the [0 , 1] interval, and in order to exploit small variance it is actually necessary to apply a PAC-Bayes-Empirical-Bernstein-style inequality (Tolstikhin and Seldin, 2013, Mhammedi et al., 2019, Wu et al., 2021) or the PAC-Bayes-split-kl inequality (Wu and Seldin, 2022) rather than Theorem 1, but the point about reduced sample size still applies.)

Recursive PAC-Bayes (new) We introduce the following decomposition of the loss

As before, we decompose S into two disjoint sets S = S 1 ∪ S 2 . We make the following major observations:

The quantity E π [ L ( h ′ )] on the right is 'of the same kind' as E ρ [ L ( h )] on the left.

We can take an uninformed prior π 0 and apply Theorem 1 (or any other suitable PAC-Bayes bound) to bound E π [ L ( h ′ )] . (The KL term in the bound on E π [ L ( h ′ )] will be KL( π ∥ π 0 ) .)

We can restrict π to depend only on S 1 , but still use all the data S in calculation of the PACBayes bound on E π [ L ( h ′ )] , because π is a posterior relative to π 0 , and a posterior is allowed to depend on all the data, and in particular on any subset of the data. Therefore, the empirical loss E π [ ˆ L ( h ′ , S )] can be computed on all the data S , and the denominator of the bound in Theorem 1 can be the size of S , and not the size of S 2 . This is what we call preservation of confidence information on π , because all the data S are used to construct a confidence bound on E π [ L ( h ′ )] , and not just S 2 . This is in contrast to the bound on L ( h ∗ ) in the approach of Mhammedi et al. (2019), which only allows to use S 2 for bounding L ( h ∗ ) . Note that while we use all the data S in calculation of the bound, we only use S 1 and E π [ ˆ L ( h ′ , S 1 )] in the construction of π . Nevertheless, we can still use the knowledge that we will have n samples when we reach the estimation phase, i.e., when constructing π we can leave the denominator of the bound at n , allowing more aggressive deviation from π 0 .

If we restrict π to depend only on S 1 , then it is a valid prior for estimation of any posterior quantity E ρ [ · ] based on S 2 . Thus, if we also restrict γ to depend only on S 1 , we can use any PACBayes-Empirical-Bernstein-style inequality or the PAC-Bayes-split-kl inequality to estimate the excess loss E ρ [ L ( h ) -γ E π [ L ( h ′ )]] based on S 2 , i.e., based on E ρ [ ˆ L ( h, S 2 ) -γ E π [ ˆ L ( h ′ , S 2 )]] . If γ E π [ L ( h ′ )] is a good approximation of E ρ [ L ( h )] and E ρ [ L ( h )] is not close to zero, then the excess loss E ρ [ L ( h ) -γ E π [ L ( h ′ )]] is more efficient to bound than the plain loss E ρ [ L ( h )] .

In general, since E ρ [ L ( h )] is expected to improve on E π [ L ( h ′ )] , it is natural to set γ < 1 . However, γ is not allowed to depend on S 2 , because otherwise ˆ L ( h, S 2 ) -γ E π [ ˆ L ( h ′ , S 2 )] becomes a biased estimate of L ( h ) -γ E π [ L ( h ′ )] . We discuss the choice of γ in more detail when we present the bound and the experiments.

Biggs and Guedj (2023) have proposed a sequential martingale-style evaluation of a martingale version of E ρ [ L ( h ) -L ( h ∗ )] and L ( h ∗ ) in the approach of Mhammedi et al., but it has not been shown to yield significant improvements yet. The same 'martingalization' can be directly applied to our decomposition, but to keep things simple we stay with the basic decomposition.

Finally, we note that we can split S 1 further and apply (1) recursively to bound E π [ L ( h ′ )] .

To set notation for recursive decomposition, we use π 0 , π 1 , . . . , π T to denote a sequence of distributions on H , where π 0 is an uninformed prior and π T = ρ is the final posterior. We use γ 2 , . . . , γ T to denote a sequence of coefficients. For t ≥ 2 we then have the recursive decomposition

To construct π 1 , . . . , π T we split the data S into T non-overlapping subsets, S = S 1 ∪ · · · ∪ S T . We restrict π t to depend on U train t = ⋃ t s =1 S s only, and we use U val t = ⋃ T s = t S s to estimate (recursively) E π t [ L ( h )] (see Figures 1 and 2 in Appendix A for a graphical illustration). Note that S t is used both for construction of π t and for estimation of E π t [ L ( h )] (it is both in U train t and U val t ), resulting in efficient use of the data. It is possible to use any standard PAC-Bayes bound, e.g., Theorem 1, to bound E π 1 [ L ( h )] , and any PAC-Bayes-Empirical-Bernstein-style bound or the PAC-Bayes-split-kl bound to bound the excess losses E π t [ L ( h ) -γ t E π t -1 [ L ( h )]] . The excess losses take more than three values, so in the next section we present a generalization of the PAC-Bayes-split-kl inequality to general discrete random variables, which may be of independent interest. The Recursive PAC-Bayes bound is presented in Section 4.

3 Split-kl and PAC-Bayes-split-kl inequalities for discrete random variables

The kl inequality is one of the tightest concentration of measure inequalities for binary random variables. Letting kl -1 , + (ˆ p, ε ) := max { p : p ∈ [0 , 1] and kl(ˆ p ∥ p ) ≤ ε } denote the upper inverse of kl and kl -1 , -(ˆ p, ε ) := min { p : p ∈ [0 , 1] and kl(ˆ p ∥ p ) ≤ ε } the lower inverse, it states the following.

Theorem 2 ( kl Inequality (Langford, 2005, Foong et al., 2021, 2022)) . Let Z 1 , · · · , Z n be independent random variables bounded in the [0 , 1] interval and with E [ Z i ] = p for all i . Let ˆ p = 1 n ∑ n i =1 Z i be the empirical mean. Then, for any δ ∈ (0 , 1) :

While the kl inequality is tight for binary random variables, it is loose for random variables taking more than two values due to its inability to exploit small variance. To address this shortcoming Wu and Seldin (2022) have presented the split-kl and PAC-Bayes-split-kl inequalities for ternary random variables. Ternary random variables naturally appear in a variety of applications, including analysis of excess losses, certain ways of analysing majority votes, and in learning with abstention. The bound of Wu and Seldin is based on decomposition of a ternary random variable into a pair of binary random variables and application of the kl inequality to each of them. Their decomposition yields a tight bound in the binary and ternary case, but loose otherwise. The same decomposition was used by Biggs and Guedj (2023) to derive a slight variation of the inequality, with the same limitations. We present a novel decomposition of discrete random variables into a superposition of binary random variables. Unlike the decomposition of Wu and Seldin, which only applies in the ternary case, our decomposition applies to general discrete random variables. By combining it with kl bounds for the binary elements we obtain a tight bound. The decomposition is presented formally below and illustrated graphically in Figure 3 in Appendix A.

3.1 Split-kl inequality

Let Z ∈ { b 0 , . . . , b K } be a ( K + 1) -valued random variable with b 0 < b 1 < · · · < b K . For j ∈ { 1 , . . . , K } define Z | j = 1 ( Z ≥ b j ) and α j = b j -b j -1 . Then Z = b 0 + ∑ K j =1 α j Z | j . For a sequence Z 1 , . . . , Z n of ( K + 1) -valued random variables with the same support, let Z i | j = 1 ( Z i ≥ b j ) denote the elements of binary decomposition of Z i .

Theorem 3 (Splitkl inequality for discrete random variables) . Let Z 1 , . . . , Z n be i.i.d. random variables taking values in { b 0 , . . . , b K } with E [ Z i ] = p for all i . Let ˆ p | j = 1 n ∑ n i =1 Z i | j . Then for any δ ∈ (0 , 1) :

Proof. Let p | j = E [ ˆ p | j ] , then p = b 0 + ∑ K j =1 α j p | j and

where the first inequality is by the decomposition of p and the second inequality is by the union bound and Theorem 2.

3.2 PAC-Bayes-Split-kl inequality

Let f : H×Z → { b 0 , . . . , b K } be a ( K + 1) -valued loss function. (To connect it to the earlier examples, in the binary prediction case we would have Z = X × Y with elements Z = ( X,Y ) and f ( h, Z ) = ℓ ( h ( X ) , Y ) , but we will need a more general space Z later.) For j ∈ { 1 , . . . , K } let f | j ( · , · ) = 1 ( f ( · , · ) ≥ b j ) . Let D Z be an unknown distribution on Z . For h ∈ H let F ( h ) = E D Z [ f ( h, Z )] and F | j ( h ) = E D Z [ f | j ( h, Z )] . Let S = { Z 1 , . . . , Z n } be an i.i.d. sample according to D Z and ˆ F | j ( h, S ) = 1 n ∑ n i =1 f | j ( h, Z i ) .

Theorem 4 (PAC-Bayes-Split-kl Inequality) . For any distribution π on H that is independent of S and any δ ∈ (0 , 1) :

where P is the set of all possible probability distributions on H that can depend on S .

Proof. We have f ( · , · ) = b 0 + ∑ K j =1 α j f | j ( · , · ) and F ( h ) = b 0 + ∑ K j =1 α j F | j ( h ) . Therefore,

where the first inequality is by the decomposition of F and the second inequality is by the union bound and application of Theorem 1 to F | j (note that f | j is a zero-one loss function).

4 Recursive PAC-Bayes bound

Now we derive a Recursive PAC-Bayes bound based on the loss decomposition in equation (2). We aim to bound E π t [ L ( h ) -γ t E π t -1 [ L ( h ′ )]] , which we denote by

where D× π t -1 is a product distribution on X × Y × H and h ′ ∈ H is sampled according to π t -1 . We further define

then F γ t ,π t -1 ( h ) = E D× π t -1 [ f γ t ( h, ( X,Y,h ′ ))] . In order to apply Theorem 4, we represent f γ t as a superposition of binary functions. For this purpose we let { b t | 0 , b t | 1 , b t | 2 , b t | 3 } = {-γ t , 0 , 1 -γ t , 1 } and define f γ t | j ( h, ( X,Y,h ′ )) = 1 ( f γ t ( h, ( X,Y,h ′ )) ≥ b t | j ) . We let F γ t ,π t -1 | j ( h ) = E D× π t -1 [ f γ t | j ( h, ( X,Y,h ′ ))] , then F γ t ,π t -1 ( h ) = -γ t + ∑ 3 j =1 ( b t | j -b t | j -1 ) F γ t ,π t -1 | j ( h ) .

Now we construct an empirical estimate of F γ t ,π t -1 | j ( h ) . We first let ˆ π t -1 = { h π t -1 1 , h π t -1 2 , . . . } be a sequence of prediction rules sampled independently according to π t -1 . We define U val t ◦ ˆ π t -1 = {( X i , Y i , h π t -1 i ) : ( X i , Y i ) ∈ U val t } . In words, for every sample ( X i , Y i ) ∈ U val t we sample a prediction rule h π t -1 i according to π t -1 and put the triplet ( X i , Y i , h π t -1 i ) in U val t ◦ ˆ π t -1 . The triplets ( X i , Y i , h π t -1 i ) correspond to the random variables Z in Theorem 4. We note that | U val t | = | U val t ◦ ˆ π t -1 | , and we let n val t = | U val t | . We define the empirical estimate of F γ t ,π t -1 | j ( h ) as ˆ F γ t | j ( h, U val t ◦ ˆ π t -1 ) = 1 n val t ∑ ( X,Y,h ′ ) ∈ U val t ◦ ˆ π t -1 f γ t | j ( h, ( X,Y,h ′ )) . Note that E D× π t -1 [ ˆ F γ t | j ( h, U val t ◦ ˆ π t -1 )] = F γ t ,π t -1 | j ( h ) , therefore, we can use Theorem 4 to bound E π t [ F γ t ,π t -1 ( h )] using its empirical estimates. We are now ready to state the bound.

Theorem 5 (Recursive PAC-Bayes Bound) . Let S = S 1 ∪ · · · ∪ S T be an i.i.d. sample split in an arbitrary way into T non-overlapping subsamples, and let U train t = ⋃ t s =1 S s and U val t = ⋃ T s = t S s . Let n val t = | U val t | . Let π ∗ 0 , π ∗ 1 , . . . , π ∗ T be a sequence of distributions on H , where π ∗ t is allowed to depend on U train t , but not the rest of the data. Let γ 2 , . . . , γ T be a sequence of coefficients, where γ t is allowed to depend on U train t -1 , but not the rest of the data. For t ∈ { 1 , . . . , T } let P t be a set of distributions on H , which are allowed to depend on U train t . Then for any δ ∈ (0 , 1) :

where B t ( π t ) is a PAC-Bayes bound on E π t [ L ( h )] defined recursively as follows. For t = 1

For t ≥ 2 we let E t ( π t , γ t ) denote a PAC-Bayes bound on E π t [ L ( h ) -γ t E π ∗ t -1 [ L ( h ′ )]] given by

and then

Proof. By Theorem 1 we have P ( ∃ π 1 ∈ P 1 : E π 1 [ L ( h )] ≥ B 1 ( π 1 )) ≤ δ T . Further, by Theorem 4 for t ∈ { 2 , . . . , T } we have P ( ∃ π t ∈ P t : E π t [ L ( h ) -γ t E π ∗ t -1 [ L ( h ′ )]] ≥ E t ( π t , γ t ) ) ≤ δ T . The theorem follows by a union bound and the recursive decomposition of the loss (2).

Discussion

Note that π ∗ 1 , . . . , π ∗ T can be constructed sequentially, but π ∗ t can only be constructed based on the data in U train t , meaning that in the construction of π ∗ t we can only rely on E π t [ ˆ F γ t | j ( h, S t ◦ ˆ π t -1 ) ] , but not on E π t [ ˆ F γ t | j ( h, U val t ◦ ˆ π t -1 ) ] . Also note that S t is part of both U train t and U val t (see Figure 1 in Appendix A for a graphical illustration). In other words, when we evaluate the bounds we can use additional data. And even though the additional data can only be used in the evaluation stage, we can still use the knowledge that we will get more data for evaluation when we construct π ∗ t . For example, we can take

and for t ≥ 2

The empirical losses above are calculated on S t corresponding to π ∗ t , but the sample sizes n val t correspond to the size of the validation set U val t rather than the size of S t . This allows to be more aggressive in deviating with π ∗ t from π ∗ t -1 by sustaining larger KL( π ∗ t ∥ π ∗ t -1 ) terms.

Similarly, γ 2 , . . . , γ T can also be constructed sequentially, as long as γ t only depends on U train t -1 (otherwise ˆ F γ t | j ( h, S t ◦ ˆ π ∗ t -1 ) becomes a biased estimate of F γ t ,π ∗ t -1 | j ( h ) ).

We naturally want to have improvement over recursion steps, meaning B t ( π ∗ t ) < B t -1 ( π ∗ t -1 ) . Plugging this into (3), we obtain E ( π ∗ t , γ t ) + γ t B t -1 ( π ∗ t -1 ) < B t -1 ( π ∗ t -1 ) , which implies that we want γ t to be sufficiently small to satisfy γ t < 1 -E t ( π ∗ t ,γ t ) B t -1 ( π ∗ t -1 ) . At the same time, γ t should be non-negative. Therefore, improvement over recursion steps can only be maintained as long as E t ( π ∗ t , γ t ) < B t -1 ( π ∗ t -1 ) . We note that γ t B t -1 ( π ∗ t -1 ) term in (3) is linearly increasing in γ t , whereas E ( π ∗ t , γ t ) is decreasing in γ t . The value of γ t that minimizes the trade-off depends on the data. Even though it is not allowed to use U val t for tuning γ t , it is possible to take a grid of values of γ t and a union bound over the grid, and then select the best value from the grid based the value of the bound evaluated on U val t .

5 Experiments

In this section, we provide an empirical comparison of our Recursive PAC-Bayes (RPB) procedure to the following prior work: i) Uninformed priors (Uninformed), (Dziugaite and Roy, 2017); ii) Datainformed priors (Informed) (Ambroladze et al., 2007, Pérez-Ortiz et al., 2021); iii) Data-informed prior + excess loss (Informed + Excess) (Mhammedi et al., 2019, Wu and Seldin, 2022). All the experiments were run on a laptop. The source code for replicating the experiments is available at Github 1 .

We start with describing the details of the optimization procedure, and then present the results.

5.1 Details of the optimization and evaluation procedure

We constructed π ∗ 1 , . . . , π ∗ T sequentially using the optimization objective, (4) for π ∗ 1 and (5) for π ∗ 2 to π ∗ T , and computed the bound using the recursive procedure in Theorem 5. There are a few technical details concerning convexity of the optimization procedure and infinite size of the set of prediction rules H that we address next.

5.1.1 Convexification of the loss functions

The functions f γ t | j ( h, ( X,Y,h ′ )) defined in Section 4 are non-convex and non-differentiable: f γ t | j ( h, ( X,Y,h ′ )) = 1 ( f γ t ( h, ( X,Y,h ′ )) ≥ b t | j ) = 1 ( ℓ ( h ( X ) , Y ) -γ t ℓ ( h ′ ( X ) , Y ) ≥ b t | j ) . In order to facilitate optimization, we approximate the external indicator function 1 ( z ≥ z 0 ) by a sigmoid function ω ( z ; c 1 , z 0 ) = (1 + exp( c 1 ( z -z 0 ))) -1 with a fixed parameter c 1 > 0 specified in Appendix B.3.

Furthermore, since the zero-one loss ℓ ( h ( X ) , Y ) is also non-differentiable, we adopt the cross-entropy loss, as in most modern training procedures (Pérez-Ortiz et al., 2021). Specifically, for a k -class classification problem, let h : X → R k represent the function implemented by the neural network, assigning each class a real value. Let u = h ( X ) be the assignment, with u i being the i -th value of the vector. To convert this real-valued vector into a probability distribution over classes, we apply the softmax function σ : R k → ∆ k -1 , where σ ( u ) i = exp( c 2 u i ) / ∑ j exp( c 2 u j ) for some c 2 > 0 for each entry. The cross-entropy loss ℓ ce : R k × [ k ] → R is defined by ℓ ce ( u, Y ) = -log( σ ( u ) Y ) . However, since this loss is unbounded, whereas the PAC-Bayes-kl bound requires losses within [0 , 1] , we enforce a [0 , 1] -valued cross-entropy loss by mixing the output distribution with a uniform distribution σ ( u ) , i.e., ˜ σ ( u ) i = (1 -p min ) σ ( u ) i + p min /k for all i ∈ [ k ] and for some p min > 0 , and then rescaling it to [0 , 1] by taking ˜ ℓ ce ( u, Y ) = -log(˜ σ ( u ) Y ) / log( k/p min ) .

We emphasize that in the evaluation of the bound (using Theorem 5), we directly compute the zero-one loss and the f γ t | j functions without employing the approximations.

5.1.2 Relaxation of the PAC-Bayes-kl bound

The PAC-Bayeskl bound is often criticized for being unfriendly to optimization (Rodríguez-Gálvez et al., 2024). Therefore, several relaxations have been proposed, including the PAC-Bayes-classic bound (McAllester, 1999), the PAC-Bayesλ bound (Thiemann et al., 2017), and the PAC-Bayesquadratic bound (Rivasplata et al., 2019, Pérez-Ortiz et al., 2021), among others. In our optimization we have adopted the bound of McAllester (1999) instead of the kl-based bounds in Equation (5).

We again emphasize that in the evaluation of the bound we used the kl-based bounds in Theorem 5.

5.1.3 Estimation of E π [ · ]

Due to the infinite size of H and lack of a closed-form expression for E π 1 [ ˆ L ( h, S )] and E π t [ ˆ F γ t | j ( h, U val t ◦ ˆ π ∗ t -1 )] appearing in Theorem 5, we approximate them by sampling (PérezOrtiz et al., 2021). For optimization, we sample one classifier for each mini-batch during stochastic gradient descent. For evaluation, we sample one classifier for each data in the corresponding evaluation dataset. Due to approximation of the empirical quantities the final bound in Theorem 5 requires an additional concentration bound. (We note that the extra bound is only required for computation of the final bound, but not for optimization of ˆ π ∗ t .) Specifically, let ˆ π ∗ t = { h π 1 1 , h π t 2 , . . . , h π t m } be m prediction rules sampled independently according to π t . Then for any function f ( h ) taking values in [0 , 1] (which is the case for ˆ L ( h, S ) and ˆ F γ t | j ( h, U val t ◦ ˆ π ∗ t -1 ) ) and δ ′ ∈ (0 , 1) we have

It is worth noting that E π ∗ t [ f ( h )] is evaluated for a fixed π ∗ t , meaning that there is no selection involved, and therefore no KL term appears in the bound above. We, of course, take a union bound over all the quantities being estimated.

5.2 Experimental results

We evaluated our approach and compared it to prior work using multi-class classification tasks on MNIST (LeCun and Cortes, 2010) and Fashion MNIST (Xiao et al., 2017) datasets, both with 60000 training data. The experimental setup was based on the work of Dziugaite and Roy (2017) and Pérez-Ortiz et al. (2021). Similar to them we used Gaussian distributions for all the priors and posteriors, modeled by probabilistic neural networks. Technical details are provided in B.

The empirical evaluation is presented in Table 1. For the Uninformed approach, we trained and evaluated the bound using the entire training dataset directly. For the other two baseline methods, Informed and Informed + Excess Loss, we used half of the training data to train the informed prior and an ERM h ∗ for the excess loss, and the other half to learn the posterior. For our Recursive PAC-Bayes, we chose γ t = 1 / 2 for all t , and conducted experiments with T = 2 , 4 , 6 , 8 to study the impact of recursion depth. (Each value of T corresponded to a separate run of the algorithm and a separate evaluation of the bound, i.e., they should not be seen as successive refinements.) We applied a geometric data split. Specifically, for T = 2 the split was (30000, 30000) points; for T = 4 , it was (7500, 7500, 15000, 30000); for T = 6 it was (1875, 1875, 3750, 7500, 15000, 30000); and for T = 8 , it was (469, 469, 937, 1875, 3750, 7500, 15000, 30000). This approach allowed the early recursion steps, which had fewer data points, to efficiently learn the prior, while preserving enough data for fine-tuning in the later steps. Note that with this approach the value of n val t = | U val t | = ∑ T s = t | S s | , which is in the denominator of the bounds in Theorem 5, is at least n 2 .

Table 1 shows that even with only T = 2 , which corresponds to the data split used in the Informed and the Informed + Excess Loss approaches, RPB achieves better test performance than prior work. As the recursion deepens, further improvements in both the test error and the bound are observed. We note that while the bound for T = 2 is looser compared to the Informed + Excess Loss method, deeper recursion yields bounds that are tighter. Overall, deep recursion provides substantial improvements in the bound and the test error relative to prior work.

Tables 2 and 3 provide a glimpse into the training progress of RPB with T = 8 by showing the evolution of the key quantities along the recursive process. Similar tables for other values of T are provided in Appendix B.4, along with training details for other methods. The tables show an impressive reduction of the KL term and significant improvement of the bound as the recursion proceeds, demonstrating effectiveness of the approach.

6 Discussion

We have presented the first PAC-Bayesian bound that supports sequential prior updates and preserves confidence information on the prior. The work closes a long-standing gap between Bayesian and Frequentist learning by making sequential data processing and sequential updates of prior knowledge meaningful and beneficial in the frequentist framework, as it has always been in the Bayesian framework. We have shown that apart from theoretical beauty the approach is highly beneficial in practice.

The Recursive PAC-Bayes framework is extremely rich and powerful, and leads to numerous directions for future research, some of which we briefly sketch next.

The decomposition in (2) applies to any loss function, including unbounded losses. It would be interesting to find additional applications to it.

While we have restricted ourselves to the zero-one loss function to illustrate the use of PACBayes-split-kl, the results can be directly generalized to any bounded loss function by replacing PAC-Bayes-split-kl with PAC-Bayes-Empirical-Bernstein or PAC-Bayes-Unexpected-Bernstein, and deriving the corresponding analogue of Theorem 5 (which is straightforward).

We have shown that the bound works well with geometric split of the data, but there are many other ways to split the data which could be studied.

There is also a lot of space for experimentation with optimization of γ t .

It would be interesting to study how the bound will perform in sequential learning settings, where the data arrives sequentially, and thus the partition is dictated externally.

There are many interesting research directions from the computational perspective. We note that for base models with linear computational complexity (e.g., neural networks) the overhead of recursion is relatively small and optimization time of Recursive PAC-Bayes is comparable to processing all data at once or in two chunks (as in data-dependent priors). For base models with superlinear computational complexity (e.g., kernel SVMs) sequential training of several small models in the recursion may actually be cheaper than training a big model based on all the data. Moreover, since the bound in Theorem 5 holds for any sequence of distributions π ∗ 0 , π ∗ 1 , . . . , π ∗ T , the optimization in equation (5) is allowed to be approximate. Considering that the improvement of the bounds and the test loss relative to prior work was very significant, there is space to look at the trade-off between statistical power and computational complexity. Namely, it may potentially be possible to relax the approximation of arg min in equation (5) to gain computational speed-up at the cost of only a small compromise on the bounds and test losses.

We note that it is possible to start the recursion at π 0 . Namely, it is possible to use, for example, Theorem 2 to bound E π 0 [ L ( h )] using all the data, and apply the recursive decomposition (2) starting from π 1 . Whether this would yield an advantage relative to starting the recursion at π 1 , as we did, remains to be studied.

Acknowledgments and Disclosure of Funding

YWacknowledges support from the Novo Nordisk Foundation, grant number NNF21OC0070621. YZ acknowledge Ph.D. funding from Novo Nordisk A/S. BECA acknowledges funding from the ANR grant project BACKUP ANR-23-CE40-0018-01.

References

Amiran Ambroladze, Emilio Parrado-Hernández, and John Shawe-Taylor. Tighter PAC-Bayes bounds. In NeurIPS , 2007.

Felix Biggs and Benjamin Guedj. Tighter PAC-Bayes generalisation bounds by leveraging example difficulty. In Proceedings on the International Conference on Artificial Intelligence and Statistics (AISTATS) , 2023.

Ben Chugg, Hongjian Wang, and Aaditya Ramdas. A unified recipe for deriving (time-uniform) PAC-Bayes bounds. Journal of Machine Learning Research , 24(372), 2023.

Thomas M. Cover and Joy A. Thomas. Elements of Information Theory . Wiley Series in Telecommunications and Signal Processing, 2nd edition, 2006.

Gintare Karolina Dziugaite and Daniel M. Roy. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI) , 2017.

Andrew Foong, Wessel Bruinsma, David Burt, and Richard Turner. How tight can pac-bayes be in the small data regime? In NeurIPS , 2021.

Andrew Y. K. Foong, Wessel P. Bruinsma, and David R. Burt. A note on the chernoff bound for random variables in the unit interval. arXiv preprint arXiv.2205.07880 , 2022.

Maxime Haddouche and Benjamin Guedj. PAC-Bayes generalisation bounds for heavy-tailed losses through supermartingales. Transactions on Machine Learning Research Journal , 2023.

Kyoungseok Jang, Kwang-Sung Jun, Ilja Kuzborskij, and Francesco Orabona. Tighter PAC-Bayes bounds through coin-betting. In Proceedings of the Conference on Learning Theory (COLT) , 2023.

John Langford. Tutorial on practical prediction theory for classification. Journal of Machine Learning Research , 6, 2005.

Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010. URL http://yann. lecun.com/exdb/mnist/ .

Andreas Maurer. A note on the PAC-Bayesian theorem. arXiv preprint cs/0411099, 2004.

David McAllester. Some PAC-Bayesian theorems. In Proceedings of the Conference on Learning Theory (COLT) , 1998.

David McAllester. Some PAC-Bayesian theorems. Machine Learning , 37, 1999.

Zakaria Mhammedi, Peter Grünwald, and Benjamin Guedj. PAC-Bayes un-expected Bernstein inequality. In NeurIPS , 2019.

María Pérez-Ortiz, Omar Rivasplata, John Shawe-Taylor, and Csaba Szepesvári. Tighter risk certificates for neural networks. Journal of Machine Learning Research , 2021.

Omar Rivasplata, Vikram M Tankasali, and Csaba Szepesvari. Pac-bayes with backprop. arXiv preprint arXiv:1908.07380 , 2019.

Borja Rodríguez-Gálvez, Ragnar Thobaben, and Mikael Skoglund. More PAC-Bayes bounds: From bounded losses, to losses with general tail behaviors, to anytime validity. Journal of Machine Learning Research , 25(110), 2024.

Matthias Seeger. PAC-Bayesian generalization error bounds for Gaussian process classification. Journal of Machine Learning Research , 3, 2002.

John Shawe-Taylor and Robert C. Williamson. A PAC analysis of a Bayesian estimator. In Proceedings of the Conference on Learning Theory (COLT) , 1997.

Niklas Thiemann, Christian Igel, Olivier Wintenberger, and Yevgeny Seldin. A strongly quasiconvex PAC-Bayesian bound. In Proceedings of the International Conference on Algorithmic Learning Theory (ALT) , 2017.

Ilya Tolstikhin and Yevgeny Seldin. PAC-Bayes-Empirical-Bernstein inequality. In NeurIPS , 2013.

Yi-Shan Wu and Yevgeny Seldin. Split-kl and PAC-Bayes-split-kl inequalities for ternary random variables. In NeurIPS , 2022.

Yi-Shan Wu, Andres Masegosa, Stephan Sloth Lorenzen, Christian Igel, and Yevgeny Seldin. Chebyshev-cantelli pac-bayes-bennett inequality for the weighted majority vote. In NeurIPS , 2021.

Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 , 2017.

A Illustrations

In this appendix we provide graphical illustrations of the basic concepts presented in the paper.

Uninformed prior

p

S

S1

S2

π

Informed Prior

P

S

S1

S2

π; h*

Informed Prior + Excess Loss

P

S

Ep[L(h)- L(h*)]; L(h*)

S1

S2

T1 ;Y2

Eπ,[L(h)]

Recursive PAC-Bayes

S

T=2

TT2

En2 [L(h) -2En,[L(h)]

S1

S2

S3

T1;Y2

Eπ,[L(h)]

T2 ;V3

Recursive PAC-Bayes

T=3

ETZ

[L(h) - 2Eπ[L(h)]

S

T3

Ens [L(h) -3Enπ[L(h)]

S1

S2

S3

T1; Y2

Eπ,[L(h)]

T2 ;V3

Recursive PAC-Bayes

En,[(h) -v2En,[L(h)]

S

T=4

T3;V4

En,[(h) -xEnx,[L(h)]

T4

Eπ[(h)-4Enπ[L(h)]

Eπ [L(h)] =

[L(h)-3Eπ[L(h')]

+sEπ[L(h)-2Enπ[L(h)]

+Y3Y2Eπ[L(h)]

Visualization

of the data

S1

S2

S3

split:

T2

T3

Z = b2

Zl1 = 1

Zl2 = 1

Zl3 = 0

bo

b1

b2

b3

α1

α2

α3

B Experimental details

In this section, we provide the details of the datasets in Appendix B.1, our neural network architectures in Appendix B.2, and other details in Appendix B.3. We provide further statistics for all the methods on both datasets in Appendix B.4.

B.1 Datasets

We perform our evaluation on two datasets, MNIST (LeCun and Cortes, 2010) and Fashion MNIST (Xiao et al., 2017). We will introduce these two datasets in the following.

B.1.1 MNIST

The MNIST (Modified National Institute of Standards and Technology) dataset is one of the most renowned and widely used datasets in the field of machine learning, particularly for training and testing in the domain of image processing and computer vision. It consists of a large collection of handwritten digit images, spanning the numbers 0 through 9.

The MNIST dataset comprises a total of 70,000 grayscale images of handwritten digits, where the training set has 60,000 images and the test set has 10,000 images. Each image in the dataset is 28x28 pixels, resulting in a total of 784 pixels per image. The images are in grayscale, with pixel values ranging from 0 (black) to 255 (white). Each image is associated with a label from 0 to 9, indicating the digit that the image represents. The images are typically stored in a single flattened array of 784 elements, although they can also be represented in a 28x28 matrix format.

B.1.2 Fashion MNIST

The Fashion MNIST dataset is a contemporary alternative to the traditional MNIST dataset, created to provide a more challenging benchmark for machine learning algorithms. It consists of images of various clothing items and accessories, offering a more complex and varied dataset for image classification tasks.

The Fashion MNIST dataset contains a total of 70,000 grayscale images, where the training set has 60,000 images and the test set has 10,000 images. Each image in the dataset is 28x28 pixels, resulting in a total of 784 pixels per image. The images are in grayscale, with pixel values ranging from 0 (black) to 255 (white). Each image is associated with one of 10 categories, representing different types of fashion items. The categories are: 1. T-shirt/top 2. Trouser 3. Pullover 4. Dress 5. Coat 6. Sandal 7. Shirt 8. Sneaker 9. Bag 10. Ankle boot. Similar to MNIST, the images are stored in a single flattened array of 784 elements but can also be represented in a 28x28 matrix format.

B.2 Neural network architectures

For all methods, we adopt a family of factorized Gaussian distributions to model both priors and posteriors, characterized by the form π = N ( w,σ I ) where w ∈ R d denotes the mean vector, and σ represents the scalar variance. We use feedforward neural networks for the MNIST dataset (LeCun and Cortes, 2010), while using convolutional neural networks for the Fashion MNIST dataset (Xiao et al., 2017).

Both our feedforward neural network and convolutional neural network are probabilistic, and each layer has a factorized (i.e. mean-field) Gaussian distribution.

Our feedforward neural network has the following architecture:

Input layer. Input size: 28 × 28 (flattened to 784 features).

Probabilistic linear layer 1. Input features: 784, output features: 600, activation: ReLU.

Probabilistic linear layer 2. Input features: 600, output features: 600, activation: ReLU.

Probabilistic linear layer 3. Input features: 600, output features: 600, activation: ReLU.

Probabilistic linear layer 4. Input features: 600, output features: 10, activation: Softmax.

Our convolutional neural network has the following architecture:

Input layer. Input size: 1 × 28 × 28 .

Probabilistic convolutional layer 1. Input channels: 1, output channels: 32, kernel size: 3x3, activation: ReLU.

Probabilistic convolutional layer 2. Input channels: 32, output channels: 64, kernel size: 3x3, activation: ReLU.

Max pooling layer. Pooling size: 2x2.

Flattening layer. Flattens the output from the previous layers into a single vector.

Probabilistic linear layer 1. Input features: 9216, output features: 128, activation: ReLU.

Probabilistic linear layer 2 (output layer). Input features: 128, output features: 10, activation: Softmax.

B.3 Other details in the experiments

General for all methods The methods in comparisons are trained and evaluated using the procedure described in Section 2 and visually illustrated in Figure 1. We will provide some further details for each method later in the following. For all methods in comparison, we apply the optimization and evaluation method described in Section 5.1. For the approximation described in Section 5.1.1, we set the parameters c 1 = c 2 = 5 . The lower bound for the prediction p min = 1 e -5 . The δ in our bound and all the other methods is selected to be δ = 0 . 025 . As mentioned in Section 5.1.2, we use the PAC-Bayes-classic bound by McAllester in replacement of PAC-Bayeskl when doing optimization. Note that for all methods, we also have to estimate the empirical loss of the posterior E π [ · ] described in Section 5.1.3. We also allocate the budget for the union bound for the estimation such that these estimations in the bound are controlled with probability at least 1 -δ ′ , where we chose δ ′ = 0 . 01 . Therefore, the ultimate bounds for all methods hold with probability at least 1 -δ -δ ′ . Note that we do not consider such bounds during optimization but only when estimating the bounds.

For all methods, we adopt a family of factorized Gaussian distributions to model both priors and posteriors of all the learnable parameters of the classifiers, characterized by the form π = N ( w,σ I ) where w ∈ R d denotes the mean vector, and σ represents the scalar variance. For all methods, we initialize an uninformed prior π 0 = N ( w 0 , σ 0 I ) that is independent of data, where the mean is randomly initialized, and the variance σ 0 is initialized to 0.03 (Pérez-Ortiz et al., 2021).

In the training process of all methods in our experiments, we set the batch size to 250, the number of training epochs to 200, and use stochastic gradient descent with a learning rate of 0.005 and a momentum of 0.95.

Uninformed priors We take π 0 defined above as the uninformed prior. We then learn the posterior ρ from the prior using the entire training dataset S , applying a PAC-Bayes bound. We evaluate the bound using, again, the entire training dataset S .

Data-informed priors We start with the same π 0 as the uninformed prior. We train the informed prior π 1 using S 1 with | S 1 | = | S | / 2 by minimizing a PAC-Bayes bound. The posterior ρ is then learned using the informed prior π 1 and the subset S 2 with | S 2 | = | S | / 2 , again by minimizing a PAC-Bayes bound. The bound is evaluated using S 2 .

Data-informed priors + excess loss We train the informed prior π 1 and the reference classifier h ∗ using S 1 that contains half of the training dataset. π 1 is obtained by minimizing a PAC-Bayes bound with the uninformed prior π 0 , while the reference classifier h ∗ is obtained by an empirical risk minimizer (ERM). The posterior ρ is obtained by minimizing a PAC-Bayes bound on the excess loss between ρ and h ∗ . The prior used in the bound for both training and evaluation is the data-informed prior π 1 . Therefore, the data for both training and evaluation of ρ must be the other half of data S 2 .

B.4 Further results for the experiments

In this section, we report some more statistics for all methods.

For all methods, to calculate the classification loss of ρ on the testing data, E ρ [ ˆ L ( h, S test )] (Test 0-1), we sample one classifier for each data. The train 0-1 loss for all methods is computed on the entire training dataset S , while the test 0-1 loss for all methods is computed on the test dataset S test .

B.4.1 Recursive PAC-Bayes

We report the additional results of Recursive PAC-Bayes on MNIST with T = 2 in Table 4, T = 4 in Table 5, and T = 6 in Table 6. We report Recursive PAC-Bayes on Fashion MNIST with T = 2 in Table 7, T = 4 in Table 8, and T = 6 in Table 9.

∗

∗

∗

∗

∗

∗

B.4.2 Uninformed priors

We report the additional results of uninformed priors (McAllester, 1998) on MNIST and Fashion MNIST in Table 10. As described earlier in Section 2, Section 5, and Section B.3, we evaluate the bound using the entire training set.

B.4.3 Data-informed priors

We report the additional results of data-informed priors (Ambroladze et al., 2007) on MNIST and Fashion MNIST in Table 11. As described earlier in Section 2, Section 5, and Section B.3, we evaluate the bound using S 2 that is independent of the data-informed prior π 1 .

B.4.4 Data-informed priors + excess loss

We report the additional results of data-informed priors + excess loss (Mhammedi et al., 2019) on MNIST and Fashion MNIST in Table 12 and 13. As described earlier in Section 2, Section 5, and Section B.3, we evaluate the bound using S 2 that is independent of the data-informed prior π 1 and the reference prediction rule h ∗ . The bound is composed of two parts: a bound on the excess loss of ρ with respect to h ∗ (Excess bound) and a single hypothesis bound on h ∗ ( h ∗ bound). We report the two components of the bound in Table 12. We provide further details to compute these bounds from the losses of their corresponding quantities in Table 13.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We tried to write an abstract and an introduction that reflected the rest of the paper as accurately as possible.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have tried to be as honest as possible in setting the framework of this theoretical work and have explicitly mentioned the limiting assumptions of the paper, e.g. i.i.d. data.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We have clearly stated the assumptions made in the paper, and our theoretical results are accompanied by detailed proofs.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The datasets are available online and we have tried to provide as much detail as possible to make our experiments transparent and reproducible.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The code has been uploaded, the datasets are open source.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We have introduced many different notations to accurately specify the different data splits and parameters used in our framework, and have tried to detail the optimization procedure to make our results as understandable as possible.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We provided for each experiment the corresponding standard deviations over different runs of the experiments. Please refer to the tables in the paper for more details.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: As mentioned in the paper, all the experiments were run on a personal laptop.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have not violated any of the points in the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This is a theoretical paper with no specific societal impact.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The open source datasets we used in the paper were properly cited, namely MNIST (LeCun and Cortes, 2010) and Fashion MNIST (Xiao et al., 2017).

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We have not released any new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our paper did not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our paper did not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper investigates a type of PAC-Bayes bounds that makes use of sequential prior updates, without loosing the appearance of the cardinality of the whole training set.\n\nSTRENGTHS:\n* Sequential updates feel natural in that context, not only statistically as demonstrated here, but also computationally. Indeed, having sequential Monte Carlo methods in mind, it might be easier to approximate a complex posterior by slowly modifying the prior. \n* The first three sections of the paper are very nicely written, and agreeable to read for a non-expert.\n* While the proofs are straightforward consequences of earlier results, the key theorem has practical importance and, to my limited knowledge and following the authors' claim, seems to have been missed by earlier investigations.\n* Overall, I enjoyed reading the paper, and feel like I learned something.\n\nWEAKNESSES:\n## General comments\n* The crux of the paper, Section 4, is too dense and hard to read, to the point of hiding some of the degrees of freedom of the paper, and making it tedious to follow the paper line by line and check the claims. This can easily be solved; see major comments. \n* After a very nice historical introduction, I was expecting a down-to-earth comparison to recent work on PAC-Bayes and sequential posterior updates, as listed in p2 L60-68. Maybe my lack of expertise made me miss a point, but I think these approaches are introduced and discarded in a single high-level paragraph, which feels unfair compared to the (nice) long pedagogic account of Section 2.\n\n## Major\n1. p2 L60 to L68: this paragraph cites several competing martingale-based approaches to PAC-Bayes inequalities with sequential posterior updates, which are discarded a bit quickly for me to understand why. Could you elaborate and give some details on the claims of e.g. Section 6.5 of [Chugg et al., 2023] and [Haddouche and Guedj, 2023]? For instance, you could give an informal version of their main claim, and point to the limitations you address. \n2. Relatedly, why are these approaches not featured in the experimental section? Maybe the answer to bullet 1. will make it clear.\n3. I would avoid bullet lists on p4 and p7, and put the proofs of Theorems 3 and 4 in the appendix; all of this would create space for Section 4 to breathe. I was enjoying reading the paper until Section 4, and then bumped into the daunting density of Section 4 on p6, due in part to too many inline formulas and overly compact definitions. The risk is to lose many readers here, while this is the crux of the paper. With the space gained, you could e.g. use centered displays for the most important definitions. \n4. With the space created, still in Section 4, I would take time to introduce objects one by one. I believe $(\\gamma_t)$, for instance, is used before Theorem 5 without definition. Implicitly, we understand in Theorem 5 that they can be predictable functions. We should thus, for instance, have a definition of $\\gamma_t$ as a sequence of real-valued functions adapted to a filtration. Then we could introduce, for $\\gamma\\in\\mathbb{R}$, the function $f_{\\gamma}: \\mathcal{H} \\times \\mathcal{X} \\times \\mathcal{Y} \\times \\mathcal{H} \\rightarrow \\mathbb{R}$ by $f_{\\gamma}(h, x, y, h')=...$, instead of the more compact definition of $f_{\\gamma_t}(h, (X,Y,h'))$, which saves space but intertwines the definitions of several objects, functions and random variables. Similarly, I would start by defining a filtration to be able to state which function is measurable w.r.t. to what $\\sigma$-algebra.\n5. Relatedly, L232 I am not sure I understand over what distribution the expectation is taken. Slower definitions would likely make it clearer.\n6. Section 4 feels like it could be rewritten in terms of (super)martingales, although it's a matter of taste, I guess, and I think I saw a comment in the paper that you wanted to avoid doing so. However, not explicitly using martingales makes it harder to connect to previous work. For instance, your theorem 5 bears resemblance with a maximal inequality for martingales such as Ville's, as in [Chugg et al., 2023], could you comment on this?\n7. Can you confirm that I understand well Section 5.1? Basically, you describe how you approximate the sequential optimization problems in (4), for $t=1, ..., T$. This does create a sequence $(\\pi^*_t)$, to which you can apply Theorem 5. Actually, Theorem 5 does not care whether $(\\pi^*_t)$ optimizes (4), so it is OK to actually approximate the optimal sequence (4).\n8. Section 5.1.3: how do you draw $h_{t,1}\\sim\\pi_t^*$? Is $\\pi_t^*$ easy to draw from by construction?\n9. Section 5: as a side remark, I am wondering whether sequential Monte Carlo techniques could help sequentially approximating $\\pi^*_t$; see e.g. [[Chopin and Papaspiliopoulos, An Introduction to Sequential Monte Carlo, Springer, 2020]](https://link.springer.com/book/10.1007/978-3-030-47845-2).\n10. p8 L320 what is meant by \"Gaussian distributions modeled by probabilistic neural networks\"?\n11. Why do you not split the data more than in 6 shards? What happens if we go \"online\" and make $n$ shards of $1$ data point? What differs then from the \"online PAC-Bayes\" approach of [Haddouche and Guedj, 2023]?\n12. Is there any computational gain in splitting the dataset and sequentially optimizing $\\pi_t$ instead of doing plain batch PAC-Bayes, on top of the statistical gain in the tighter bound? \n\n## Minor\n* p1 L27 The Bayesian posterior\n\nQUESTIONS:\nAny of my major comments above, with a priority to items 1 and 11, and commitment to follow the suggestions of clarification in Section 4.\n\nLIMITATIONS:\nThis is fundamental work; no potential negative or environmental societal impact other than making the pdf available!", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nData-dependent priors are crucial for the tightness of PAC-Bayesian bounds in several scenarios. However, using a fraction of the data to train the prior reduces the sample size of the bounds, which can sometimes be counter-productive. This fact also discourages sequential prior updates since the bounds can rapidly degenerate. This paper elegantly solves these problems using a \"prior-dependent\" excess loss bound inspired by Mhammedi et al. (2019). Their approach allows sequential prior updating without confidence loss and surpasses previous data-dependent prior building techniques in standard benchmark datasets. \n\nAdditionally, they generalize ternary split-kl PAC-Bayes bounds to general discrete random variables.\n\nSTRENGTHS:\n1- The main result, Theorem 5, is a significant contribution to the study of data-dependent priors with many possible applications.\n\n2- The experimental results (Table 1) show a remarkable improvement over previous methods.\n\nWEAKNESSES:\nI find no major weaknesses, just a couple of comments:\n\n1- Sections 3 and 4 are very dense in notation and difficult to read (I recognize there is not much the authors can do about that).\n\n2- Since Recursive PAC-Bayes is developed for sequential prior updating, it would be nice (if possible) to have some discussion about its possible applications to streaming/online learning scenarios.\n\n3- In the experiments with data-dependent priors 1/2 of the data was used for building the prior. However [1] shows that the proportion of data used for prior building is crucial. It would be nice to have experiments with different data splitting proportions (also, I think [1] is a relevant paper that could be cited as related work).\n\n\n\n[1] Dziugaite, G. K., Hsu, K., Gharbieh, W., Arpino, G., & Roy, D. (2021). On the role of data in PAC-Bayes bounds. In International Conference on Artificial Intelligence and Statistics (pp. 604-612). PMLR.\n\nQUESTIONS:\nSee Weaknesses.\n\nLIMITATIONS:\nNo significant limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n- The paper presents a novel PAC-Bayesian procedure that allows for sequential prior updates without splitting the training dataset.\n- The proposed procedure is based on a novel decomposition of the expected loss of randomized classifiers, which rewrites the posterior loss as an excess loss relative to a downscaled loss of the prior plus the downscaled loss of the prior, which is recursively bounded.\n- The paper generalizes the split-kl and PAC-Bayes-split-kl inequalities to discrete random variables.\n- The authors confirmed that their PAC-Bayes learning method outperforms state-of-the-art methods in empirical evaluations.\n\nSTRENGTHS:\n- The paper addresses a practically important topic of adaptively optimizing the prior in PAC-Bayes learning to reduce pre-training costs while enhancing generalization performance.\n- The attempt to derive recursively bounded generalization error bounds for the optimization steps includes originality.\n\nWEAKNESSES:\nFirst, I want to note that I find it difficult to fully grasp the motivation, utility, and specific algorithms of this paper from the current manuscript, which may lead to some of the points in the Weaknesses and Questions sections being based on misunderstandings. If there are any misunderstandings, I would appreciate it if the authors could correct them and provide clearer explanations beyond what is presented in the paper.\n\n- **Limitations of Derived Bounds**:\n  - As one of the limitations in this paper, the derived bounds assume 0-1 loss, limiting their applicability to binary classification problems. \n  - The bounds are valid only at a specific time point $t$. Considering that most existing PAC-Bayes bounds provide insights into generalization error independent of iteration, this bound appears to be valid under very restrictive conditions.\n  - The definitions of symbols throughout the paper seem ambiguous, affecting readability. For instance, it is unclear what $t$ represents—whether it is training iteration, epoch, or a separate recursion step. Clarification is needed.\n\n- **Comparison with Existing Approaches**:\n  - The main issue addressed by this paper is that traditional data-dependent priors in PAC-Bayes learning are constructed by splitting the training data, thus discarding some information. However, alternative approaches not discussed in this paper use optimization methods like SGLD or Entropy-SG(L)D, which guarantee differential privacy while optimizing the prior using all training data (e.g., Dziugaite et al., 2017; Dziugaite et al., 2018). Although these methods might result in looser bounds or smaller probabilities of bound validity, they have shown good performance in similar experimental settings. While these methods differ in being pre-training approaches rather than recursive, they share the goal of optimizing the prior without splitting the training data. The lack of comparison with these methods leaves the usefulness of the proposed approach insufficiently demonstrated.\n\n- **Data Splitting and Recursive Approach**:\n  - The paper still splits data according to the value of $T$. It is unclear how this fundamentally differs from traditional data-splitting prior learning approaches. In the split strategy proposed in Sec. 5, increasing the number of split data sets with $T$ might lead to suboptimal prior distributions due to fewer data in early stages, potentially affecting generalization performance.\n  - There is no discussion on the tightness of the bounds derived in Theorem 5, especially for $t \\geq 2$. Intuitively, even though $\\gamma_{t}$ is multiplicative, the cumulative addition of the KL term may result in looser bounds as $t$ increases unless $\\gamma_{t}$ or/and the complexity value is very small. The decreasing bound values with increasing $T$ observed in the experiments may be due to the simplicity of datasets like MNIST, where KL values are initially very small. If there is another reason, it should be explained.\n\n- **Computational Efficiency**:\n  - There is no mention of computational efficiency. It seems that increasing the number of recursive steps would lead to higher computational costs.\n\n- **Convergence of generalization bounds**:\n  - It is unclear whether the bound guarantees convergence with respect to the sample size $n$, i.e., whether it converges to zero as $n \\rightarrow \\infty$. If we only consider Equation (3), even if $n_{t}^{\\mathrm{val}} \\rightarrow \\infty$ and $B_{t-1} = 0$, it seems that $-\\gamma_{t}$ would remain in $\\mathcal{E}_{t}$, preventing convergence. Is this a desirable property for a generalization error bound?\n\n- **Experimental Validation**:\n  - The experiments are limited to very simple datasets like MNIST and Fashion-MNIST, which is insufficient to convincingly demonstrate the utility of the proposed algorithm. More diverse and complex datasets should be tested to validate the approach comprehensively.\n\nCitation:\n- Dziugaite et al., 2017: Dziugaite et al. Data-dependent PAC-Bayes priors via differential privacy. NeurIPS2018. https://arxiv.org/pdf/1802.09583\n- Dziugaite et al., 2018: Dziugaite et al. Entropy-SGD optimizes the prior of a PAC-Bayes bound: Generalization properties of Entropy-SGD and data-dependent priors. ICML2018. https://arxiv.org/pdf/1712.09376\n\nQUESTIONS:\nIn addition to the concerns raised in the Weakness section, I would appreciate it if you could also address the following questions:\n\n- Can you provide a qualitative interpretation of the generalization error bound given by Equation (3)?\n- Why does the summation part in \\(\\mathcal{E}\\) above Equation (3) only sum from \\(j=1\\) to \\(j=3\\)?\n- The specific algorithm of the proposed method is not very clear. Could you provide an algorithm table?\n- What are the advantages of the proposed method compared to traditional PAC-Bayes prior pre-training approaches that fully use the training data (e.g., Dziugaite et al., 2017; Dziugaite et al., 2018)? If there is a reason for not conducting a theoretical or experimental comparison with these methods, please explain.\n- You mentioned calculating the 0-1 loss. Is this correct that you conducted binary classification using a one-vs-all approach?\n\nLIMITATIONS:\nThis study is foundational research aimed at improving the generalization by PAC-Bayes learning with prior optimization, and these aspects are discussed in the Broader Impact paragraph. The datasets used are open datasets such as MNIST and Fashion MNIST, which suggests that there are no significant concerns regarding potential negative social impacts.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

comments: SUMMARY:
PAC-Bayes bounds are extended to the recursive or streaming case by breaking the expected loss into ($a$) expected loss on the previous step times a discounting factor plus ($b$) current expected loss minus $a$. An extension of the kl inequality to many-valued (here 4-valued) random variables provides a bound on $b$.

STRENGTHS:
This is impressive work. It's technically sound and makes a major theoretical advance.

WEAKNESSES:
The proof of thm 5 doesn't account for sampling error between $\hat{\pi}^*_{t-1}$ and $\pi^*_{t-1}$. The inequality used in line 244 refers only to $\pi^*_{t-1}$ but the definitions of 
$\mathcal{E}\_{t}$ 
and 
$\\hat{F}\_{\\gamma_{t} | j}$
involve 
$\hat{\pi}^{*}\_{t-1}$. 
The method in section 5.1.3 can resolve this but it should be included in the theorem (or the theorem should be rewritten without sampling).

214: "can depend on $S$" is conceptually helpful but technically superfluous. What's really meant is that because the quantification is over all of $\mathcal{P}$ one is free to choose $\rho$ in a way that depends on $S$.

231: the summation index is misleading because it implies the index set is the product of $U^{val}\_t$ 
and $\hat{\pi}\_{t-1}$, 
whereas the actual index is $((X,Y),h')$.

QUESTIONS:
The objective in (4) is very similar to the VI objective, especially with 0-1 loss replaced by cross-entropy as in section 5.1.1. (I think replacing $kl^{-1,+}$ with Euclidean divergence and $n^{val}_t$ with $|S_t|$ would yield the VI update.) It would be useful to see an experimental comparison between recursive PAC-Bayes and recursive VI.

LIMITATIONS:
none noted
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-13