Risk Monotonicity in Statistical Learning

1 Introduction . Guarantees on the performance of machine learning algorithms are desirable , especially given the widespread deployment . A traditional performance guarantee often takes the form of a generalization bound , where the expected risk associated with hypotheses returned by an algorithm is bounded in terms of the corresponding empirical risk plus an additive error which typically converges to zero as the sample size increases . However , interpreting such bounds is not always straight forward and can be somewhat ambiguous . In particular , given that the error term in these bounds goes to zero , it is tempting to conclude that more data would monotonically decrease the expected risk of an algorithm such as the Empirical Risk Minimizer ( ERM ) . However , this is not always the case ; for example , [ 33 ] showed that increasing the sample size by one , can sometimes make the test performance worse in expectation for commonly used algorithms such as ERM in popular settings including linear regression . This type of non-monotonic behavior is still poorly understood and indeed not a desirable feature of an algorithm since it is expensive to acquire more data in many applications . Non-monotonic behavior of risk curves [ 47 ] —the curve of the expected risk as a function of the sample size—has been observed in many previous works [ 18 , 43 , 48 , 20 ] ( see also [ 33 , 52 ] for nice accounts of the literature ) . At least two phenomena have been identified as being the cause behind such behavior . The first one , coined peaking [ 30 , 19 ] , or double descent according to more recent literature [ 6 , 49 , 7 , 17 , 15 , 37 , 40 , 41 , 16 , 12 , 11 , 14 , 42 ] , is the phenomenon where the risk curve peaks at a certain sample size n. This sample size typically represents the cross-over point from an over-parameterized to under-parameterized model . For example , when the number of data points is less than the number of parameters of a model ( over-parameterized model ) , such as Neural ∗Work done while at the Australian National University . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Networks , the expected risk can typically increase until the number of data points exceeds the number of parameters ( under-parameterized model ) . The second phenomenon is known as dipping [ 32 , 31 ] , where the risk curve reaches a minimum at a certain sample size n and increases after that—never reaching the minimum again even for very large n. This phenomenon typically happens when the algorithm is trained on a surrogate loss that differs from the one used to evaluate the risk [ 8 ] . It is becoming more apparent that the two phenomena just mentioned ( double descent and dipping ) do not fully characterize when nonmonotonic risk behavior occurs [ 34 ] . [ 33 ] showed that non-monotonic risk behavior could happen outside these settings and formally prove that the risk curve of ERM is non-monotonic in linear regression with prevalent losses . The most striking aspect of their findings is that the risk curves in some of the cases they study can display a perpetual “ oscillating ” behavior ; there is no sample size beyond which the risk curve becomes monotone—see Figure 1 . In such cases , the risk ’ s non-monotonicity can not be attributed to the peaking/double descent phenomenon . Moreover , they rule out the dipping phenomenon by studying the ERM on the actual loss ( not a surrogate loss ) . The findings of [ 33 ] stress our current lack of understanding of generalization . This was echoed more particularly by [ 53 ] , who posed the following question as part of a COLT open problem : How can we provably avoid non-monotonic behavior ? While excess risk bounds are typically monotonic , this does not guarantee the monotonicity of the actual risk . In this work , we study under which assumptions on the learning problem there exist consistent and risk monotonic algorithms . We also aim to quantify the price to pay , in terms of corresponding excess risk rates , for achieving risk monotonicity . Contributions . In this work , we answer some questions posed by [ 53 ] by presenting an algorithm that is both consistent and risk-monotonic in high probability under weak assumptions on the learning problem . Our algorithm is technically a “ wrapper ” that takes as input any base learning algorithm B and makes up a new algorithm A that is risk monotonic in high probability and enjoys essentially the same excess risk rate as B. Crucially , our results show that risk monotonicity need not come at the expense of worse excess risk rates . In fact , we show that fast rates are achievable under a Bernstein condition ( Definition 3 ) . Our results hold under the general statistical learning setting with a bounded loss . We even go beyond the standard i.i.d . assumption on the loss process . Our relaxed technical condition on the loss process , which is formalized in Assumption 1 below , is reminiscent of the condition characterizing Martingale Difference Sequences ( MDS ) . In a nutshell , we will assume a setting where the instance random variables Z1 , Z2 , . . . and the loss ` satisfy , for all hypotheses h , E [ ` ( h , Zt ) ∣ Z1 , . . . , Zt−1 ] = L ( h ) , for some risk function L. This is trivially satisfied in the i.i.d . case , where L corresponds to the standard risk function . In general , this condition may be satisfied even if Z1 , Z2 , . . . are dependent or have different marginal distributions . We argue that our relaxed assumption on the loss process is the weakest assumption under which studying risk-monotonicity still makes sense . To achieve risk monotonicity under our loss process assumption , we derive a new concentration inequality/generalization bound of PAC-Bayesian flavor for MDS ( see Proposition 5 ) . This concentration inequality may be thought of as an empirical Freedman ’ s inequality [ 23 ] or as an extension of the empirical Bernstein inequality [ 35 ] to MDS . Our concentration inequalities also have the advantage of being time-uniform with the optimal dependence on the number of samples . Here , time-uniform means that the inequalities hold for all sample sizes simultaneously . While standard concentration inequalities can be turned into time-uniform ones using a union bound over the number of samples n , the resulting bounds will have a sub-optimal lnn factor instead of the optimal2 ln lnn that we are able to get . Finally , our concentration bounds are easily derived using the guarantee of a recent parameter-free online learning algorithm—FREEGRAD [ 39 ] . Our approach opens up the door for obtaining new concentration inequalities through the design of online learning algorithms . Approach Overview . Our approach to deriving the new concentration inequalities is based on the guarantee of the recent FREEGRAD algorithm . The algorithm operates in rounds , where at each round t , FREEGRAD outputs ŵt in some convex set W , say Rd , then observes a vector gt ∈ Rd , typically the sub-gradient of a loss function at the iterate ŵt . The algorithm guarantees a regret bound of the form ∑Tt=1 g ⊺ t ( ŵt −w ) ≤ Õ ( ∥w∥ √ QT ) , for all w ∈ W , where QT ∶= ∑ T t=1 ∥gt∥ 2 . What is more , FREEGRAD ’ s outputs ( ŵt ) ensure the following ( see [ 39 , Theorem 5 ] ) : ŵ⊺t gt +Φ ( St , Qt ) ≤ Φ ( St−1 , Qt−1 ) , ∀t ≥ 1 , ( 1 ) where St ∶= ∥∑ti=1 gi∥ , Qt ∶= ∑ t i=1 ∥gi∥ 2 , and Φ ( S , V ) ∶= exp ( S 2 /2 γ2+V +∣S∣ − 1 2 ln γ 2 γ2+V ) , for any γ > 0 . Instantiating this guarantee in 1d with ( gt ) set to an MDS ( Xt ) and taking ( conditional ) expectation in ( 1 ) shows that Φt ∶= Φ ( ∑ti=1Xi , ∑ t i=1X 2 i ) is a non-negative supermartingale , from which concentration results can be obtained via Ville ’ s inequality ( a generalization of Markov ’ s inequality—see Lemma 18 ) . Our proof technique is similar to the one introduced in [ 28 ] , with the difference that we use the specific shape of FREEGRAD ’ s potential function to build our supermartingale , which leads to a desirable empirical variance term in the final concentration bound . On the side of risk monotonicity , given n samples , the key idea behind our approach is to iteratively generate a sequence of distributions P1 , P2 , . . . leading up to Pn over hypotheses , where we only allow consecutive distributions , say Pk−1 and Pk to differ if we can guarantee ( with high enough confidence ) that the risk associated with Pk is lower than that of Pk−1 . To test for this , we compare the average empirical losses of hypotheses sampled from Pk−1 versus ones sampled from Pk , taking into account the potential gap between empirical and population expectations . Applying our new concentration bounds to quantify this gap not only allows us to achieve risk monotonicity under a non-i.i.d . loss process but also enables us to achieve fast excess risk rates under the Bernstein condition . For the latter , it was crucial to have an empirical loss variance term in the concentration inequality . Related Works . Much work has already been done in efforts to mitigate the non-monotonic behavior of risk curves [ 54 , 42 , 33 ] . For example , in the supervised learning setting with the zero-one loss , [ 9 ] introduced the “ memorize ” algorithm that predicts the majority label on any test instance x that was observed during training ; otherwise , a default label is predicted . [ 9 ] showed that this algorithm is risk-monotonic . However , it is unclear how their result could generalize beyond the particular setting they considered . Risk-monotonic algorithms are also known for the case where the model is correctly specified ( see [ 33 ] for an overview ) ; in this paper , we do not make such an assumption . Closer to our work is that of [ 54 ] who , like us , also used the idea of only updating the current predictor for sample size n if it has a lower risk than the predictor for sample size n−1 . They determine whether this is the case by performing statistical tests on a validation set ( or through cross-validation ) . They introduce algorithm wrappers that ensure that the risk curves of the final algorithms are monotonic with high probability . However , their results are specialized to the 0-1 loss and they do not answer the question by [ 53 ] on the existence of learners that guarantee a monotonic risk in expectation . On the side of concentration bounds for non-i.i.d . processes , our results are somewhat similar to those found in e.g . [ 27 , 26 ] . However , our technique for deriving them , which relies on the guarantee of a parameter-free online learning algorithm , is entirely different . The theoretical link between online regret and concentration inequalities was previously drawn—see e.g . [ 45 , 22 ] . However , our approach is slightly different as we use the monotonicity of an online algorithm ’ s potential function to get our concentration results . Thus , new concentration inequalities may be derived similarly by modifying the explicit potential function directly . Our approach is more similar to that of [ 28 ] who also derived concentration inequalities using guarantees of online betting algorithms that bet fractions smaller than one of their wealth at each round . 2One can not improve on the ln lnn factor by the law of iterated logarithm [ 13 ] . We allow for a parameter h ∈H in our concentration bounds to make the results useful in the statistical learning setting . These bounds are of PAC-Bayesian type and are somewhat reminiscent of those in [ 50 , 38 ] . We refer the reader to [ 25 ] for an overview of existing PAC-Bayesian bounds . Outline . In Section 2 , we introduce the setting , notation , and relevant definitions . In Section 3 , we present our new concentration inequalities for Martingale Difference Sequences and loss processes we are interested in ( those that satisfy Assumption 1 below ) . In Section 4 , we present our risk-monotonic algorithm wrapper and show that it achieves risk monotonicity in high probability . We conclude with a discussion in Section 5 . The proofs of the new concentration inequalities and risk monotonicity are differed to Appendices B and C , respectively . Appendix A presents existing and new technical results needed in some of our proofs .

Summary: The paper studies the problem of risk monotonicity, that is, the property that the population loss decreases monotonically with increasing data points. The paper proposes an algorithm that converts any consistent base algorithm into a risk-monotonic algorithm while maintaining consistency under two assumptions: (1) an assumption on the generation of samples which is weaker than i.i.d. (2) an assumption on the base algorithm requiring its posterior to not have KL divergence to a fixed prior growing super-linearly in the sample size.  The approach presented in the paper is pretty straightforward: the base algorithm is run for $k=1, \ldots, n$ times and iteratively the current hypothesis is updated to be the base learner's output only if the risk reduces from the previous step, else it remains unchanged. Finally the hypothesis at step $n$ is returned. The main challenge here is to be able to confidently account for this change using the samples. For this, the authors derive a new concentration inequality for this using an online algorithm's guarantees.