Mixability made efficient: Fast online multiclass logistic regression

1 Introduction . In online learning , a learner sequentially interacts with an environment and tries to learn based on data observed on the fly ( Cesa-Bianchi and Lugosi ( 2006 ) , Hazan et al . ( 2016 ) ) . More formally , at each iteration t ≥ 1 , the learner receives an input xt in some space X ; makes a prediction ŷt in a decision domain Ŷ and the environment reveals the output yt ∈ Y . The inputs xt and the outputs yt are sequentially chosen by the environment and can be arbitrary . No stochastic assumption ( except boundedness ) on the data sequence ( xt , yt ) 1≤t≤n is made . The accuracy of a prediction ŷt ∈ Ŷ at instant t ≥ 1 for the outcome yt ∈ Y is measured through a loss function ` : Ŷ × Y → R. The learner aims at minimizing his cumulative regret Rn ( f ) = n∑ t=1 ` ( ŷt , yt ) − n∑ t=1 ` ( f ( xt ) , yt ) , ( 1 ) with respect to any function f ∈ F in a reference class . All along this paper , we will consider parametric class of functions fθ indexed by θ ∈ Θ ⊂ Rd . We will also assume convexity of the loss according to index θ , which we denote ` x , y : θ 7→ ` ( fθ ( x ) , y ) for any ( x , y ) ∈ X × Y . In this 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . general context , many algorithms have been designed based on different assumptions and obtaining different trade-offs for regret . We review below the most relevant ones for our purpose . Assuming only convexity of the loss ` , a well-known strategy for the learner is Online Gradient Descent ( Zinkevich ( 2003 ) ) with an optimal regret of orderO ( √ n ) . However , if the loss is α-mixable , the learner may achieve the faster rate O ( 1α log ( n ) ) by using an aggregating forecaster ( see Vovk ( 2001 ) , Van Erven et al . ( 2015 ) ) . Yet , such an algorithm is not constructive in general and when it is , the computational complexity is often very high ( a notable exception is the least-squares setting ) . To reduce it , a stronger assumption has been introduced by Hazan et al . ( 2007 ) with η-exp concavity . Under the latter hypothesis , Online Newton Step ( ONS ) has a regret bounded by O ( 1η log ( n ) ) and a computational complexity of O ( n ) . However this efficiency comes at the price of deteriorating statistical performance . Indeed , η-exp concavity implies α-mixability for η ≥ α and in some cases , the gap between η and α can be very large . The most spectacular case of this phenomenon occurs for logistic regression . In this setting , the loss is defined as ` ( ŷt , yt ) = − log ( σ ( ŷt ) yt ) where σ ( z ) i = ezi∑ j e zj and the regret is computed with respect to linear functions F = { x 7→ Wx , W ∈ Θ } , where Θ = B ( RK×d , B ) is the Frobenius bounded ball of radius B > 0 . On this subset , the logistic loss is η-exp concave only for η ≤ e−B . The regret of Online Newton Step can thus be of order O ( eB log ( n ) ) . On the other hand , as remarked by Kakade and Ng ( 2008 ) ( binary case ) and Foster et al . ( 2018 ) , the logistic loss is 1-mixable and an aggregating algorithm may achieve O ( log ( nB ) ) . Nevertheless , the algorithm relies on Monte Carlo methods and must sample from log-concave probability distributions which is extremely computationally expensive ( see Table 1 ) . Therefore , in this framework , the exp-concavity assumption leads to an efficient algorithm with low dependence on B while mixability yields an inefficient algorithm with much better statistical performance . Between these two extremes , other algorithms and trade-offs have been analyzed . First , it has been shown that in some situations , Follow The Regularized Leader ( FTRL ) can achieve a fast rate without an exponential constant ( Bach et al . ( 2010 ) , Marteau-Ferey et al . ( 2019 ) , Ostrovskii et al . ( 2021 ) ) . However , several additional assumptions are necessary to achieve these rates . Unfortunately , they are essentially unavoidable . Indeed , Hazan et al . ( 2014 ) showed a polynomial lower bound for the proper algorithm ( i.e. , with linear prediction function ) in the regime B = log ( n ) . This prevents algorithms like ONS and Follow The Regularized Leader from reaching logarithmic regret without an exponential constant in B . This result motivates the search for improper algorithms with better computational complexity than Foster et al . ( 2018 ) . For binary logistic regression , two efficient improper algorithms have been proposed in the literature . First , in the i.i.d . framework , Mourtada and Gaïffas ( 2019 ) have proposed the Sample Minmax Predictor ( SMP ) . It achieves an excess risk of order O ( ( B2 + d ) /n ) with computational complexity equivalent to Follow The Regularized leader . In the online framework , Jézéquel et al . ( 2020 ) proposed AIOLI , which is based on quadratic approximations of the logistic loss as well as virtual labels to regularize . The regret is upper-bounded O ( dB log ( n ) ) and the computational complexity is O ( n ( d2 + log ( n ) ) ) . These previous works left open the question of achieving the same type of performance in a setting other than binary logistic regression . In particular , for multi-class logistic regression , no other algorithm than Foster et al . ( 2018 ) is known to achieve logarithmic regret without an exponential constant . Contributions We introduce in Section 3 a new generic online learning algorithm , that we call Gaussian Aggregating Forecaster ( GAF ) . GAF achieves logarithmic regret with a small multiplicative constant for a large class of convex loss functions ( see Theorem 1 ) . The latter includes several popular loss functions such as squared loss , binary and multi-class logistic loss . Our assumptions on the loss functions are slightly stronger than α-mixability but generally weaker than assumptions widely used in the statistical framework such as generalized self-concordance ( see Bach et al . ( 2010 ) ) . In the particular but significant setting of multi-class logistic regression , GAF has a regret bounded by O ( dKB2 + dKB log ( n ) ) and a total computational complexity of O ( nK3d2 + K2n4 ) , thus significantly improving on the O ( B6n25 ( Bn + dK ) 12 ) complexity of Foster et al . ( 2018 ) , which was the best known to date for algorithms without exponential dependence on B . Table 1 summarizes the regrets and computational cost obtained by the relevant algorithms in this framework . It is worth pointing out that , by using standard online-to-batch conversion Helmbold and Warmuth ( 1995 ) , this paper also provides new results on the excess risk in the statistical i.i.d . Indeed , even in this extensively used framework , the only existing improper algorithms without exponential constant in B so far are GAF and Foster et al . ( 2018 ) . Jézéquel et al . ( 2020 ) and Mourtada and Gaïffas ( 2019 ) are in fact restricted to binary outputs only . Our new algorithm is inspired by the aggregating forecaster of Vovk ( 2001 ) that we apply to quadratic approximations of the loss , in order to make it more efficient . The high-level idea is that these quadratic approximations replace complicated log-concave distributions with Gaussian distributions , from which it is much easier to sample and thereby drastically reducing the complexity of the algorithm . We believe that this approach can be applied to much broader contexts than those analyzed here . 2 Setting . We recall here the setting and introduce the main notations and assumptions that will be used throughout the paper . Setting and notation Our framework is formalized as a sequential game between a learner and an environment . At each forecasting instance t ≥ 1 , the learner is given an input xt ∈ X ; forms a prediction ŷt ∈ Ŷ ⊆ RK ( possibly based on the current input xt and on the past information x1 , y1 , . . . , xt−1 , yt−1 ) . Note that the prediction space Ŷ ⊆ RK may be uni-dimensional ( K = 1 ) in some settings ( least-square regression ) or multi-dimensional ( K ≥ 1 ) in some cases ( e.g. , vectorvalued regression , multi-class classification ) . Then , the environment chooses yt ∈ Y ; reveals it to the learner which incurs the loss ` ( ŷt , yt ) . Finally , the performance of the learner is assessed by the regret Rn ( θ ) = n∑ t=1 ` ( ŷt , yt ) − n∑ t=1 ` xt , yt ( θ ) , with respect to all θ ∈ Θ ⊂ Rd . Here , ` x , y ( θ ) = ` ( fθ ( x ) , y ) where F = { fθ : X → Ŷ } is the reference class of functions . For simplicity of notation , we also define ` t ( θ ) = ` xt , yt ( θ ) and Lt ( θ ) = ∑t s=1 ` s ( θ ) + λ‖θ‖2 for all t ≥ 1 and λ > 0 . In all specific examples considered in this work , the learner will be compared to linear functions fθ : x 7→ θ > Φ ( x ) , where θ ∈ Θ = B ( Rd , B ) and Φ is a function from X to Rd×K such that for any x ∈ X and i ∈ [ K ] , ‖Φ ( x ) . , i‖ ≤ R. We introduce and discuss below our main assumptions on the losses . Assumptions We assume that , for all ( x , y ) ∈ X × Y , the losses ` x , y are convex , C2 and satisfy the following assumptions . ( A1 ) The loss function ` is α-mixable . In other words , for all ( Gaussian ) probability distributions π over Rd and input x ∈ X , there exists ŷ ∈ Ŷ such that for all y ∈ Y , ` ( ŷ , y ) ≤ − 1 α log ( Eθ∼πe−α ` x , y ( θ ) ) . ( A2 ) There exists ζ > 0 such that , for all ( x , y ) ∈ X × Y and θ1 , θ2 ∈ Rd , ` x , y ( θ1 ) ≤ ` x , y ( θ2 ) +∇ ` x , y ( θ2 ) > ( θ1 − θ2 ) + eζ‖θ1−θ2‖ 2 ‖θ1 − θ2‖2∇2 ` x , y ( θ2 ) . ( A3 ) There exists β > 0 such that , for all ( x , y ) ∈ X × Y and ( θ1 , θ2 ) ∈ Θ× Rd , ` x , y ( θ1 ) ≥ ` x , y ( θ2 ) +∇ ` x , y ( θ2 ) > ( θ1 − θ2 ) + β 2 ( θ1 − θ2 ) > ∇2 ` x , y ( θ2 ) ( θ1 − θ2 ) . ( A4 ) ` is γ-smooth i.e. , for all x ∈ X , y ∈ Y , θ ∈ Rd , ∇2 ` x , y ( θ ) ≤ γI . In general , these assumptions are rather weak and related to standard assumptions of ( online ) convex optimization . ( A1 ) follows from instance from exp-concavity . As we illustrate in Section 4 , all are satisfied for broadly used loss functions such as squared loss , binary and multi-class logistic loss . We provide more details on these assumptions below . Mixability ( A1 ) is a standard assumption in online learning to achieve fast rates for the regret ( Van Erven et al. , 2015 ) , which was introduced by Vovk ( 2001 ) . It is a weaker condition than exp-concavity because ( A1 ) holds with ŷ = Eθ∼π [ θ ] for η-exp concave loss functions when η ≥ α . Assumption ( A2 ) basically prevents the Hessian from changing too quickly on θ . This condition is new but not very restrictive because , as we prove in Lemma 6 , it is weaker than other widely accepted assumptions such as generalized self-concordance ( Bach et al. , 2010 ) . The lower bound ( A3 ) is the most original assumption and is crucial to make aggregating forecaster efficient . This bound is close to the one needed in the analyses of ONS , which can be derived from η-exp-concavity . However , the key difference is that ( A3 ) uses the Hessian in the quadratic term and not the outer product of the gradient . This makes the right-hand-side closer to the third Taylor series approximation of the loss and allows much larger values for β . For example , as we will show in the section 4.2 , this change allows us to remove the exponential constant for the logistic regression setting . In this case , ( A3 ) is verified with β ' B−1 , while η-exp-concavity would require η ' e−B .

Summary: This paper considers the online convex optimization setting with mixable losses. The exponential weights algorithm is known to have constant regret with mixable losses, but it is also known to have prohibitively high running time for certain mixable losses. A mixable loss function of interest is the logistic loss used in logistic regression, for which faster algorithms such as Online Newton Step (ONS) suffer a double exponentially larger regret bound compared to exponential weights.  As a solution the authors propose a mix between ONS and exponential weights called GAF. To update, the algorithm computes a quadratic surrogate loss akin to the surrogate loss used in ONS, but uses the actual hessian rather than an approximation to the hessian to compute the surrogate loss. Another interesting trick that the authors use is that the point around which the quadratic surrogate loss is defined: it is the minimizer of the $l_2$ regularization plus the loss of the previous round and the surrogate losses of all rounds preceding the previous round.  The predictions of the algorithms are the same predictions that would be used for exponential weights. While for arbitrary priors this would pose a computational problem, the algorithm uses a gaussian prior. As is observed in prior work, with a gaussian prior and quadratic losses the posterior of exponential weights is also a gaussian. This allows the authors to devise more efficient predictions.