Fast Routing under Uncertainty: Adaptive Learning in Congestion Games via Exponential Weights

√ T ) equilibrium convergence rate ; however , this rate is suboptimal in static environments – i.e. , when the network is not subject to randomness . In this static regime , accelerated algorithms achieve anO ( 1/T 2 ) convergence speed , but they fail to converge altogether in stochastic problems . To fill this gap , we propose a novel , adaptive exponential weights method – dubbed ADAWEIGHT – that seamlessly interpolates between the O ( 1/T 2 ) and O ( 1/ √ T ) rates in static and stochastic environments respectively . Importantly , this “ best-of-both-worlds ” guarantee does not require any prior knowledge of the problem ’ s parameters or any tuning by the optimizer ; in addition , the method ’ s convergence speed depends subquadratically on the size of the network ( number of vertices and edges ) , so it scales gracefully to large , real-life urban networks . 1 Introduction . Navigation apps like Google Maps and Waze have user bases numbering in the hundreds of millions , and they may receive upwards of 104 routing requests per second ; in fact , Google Maps alone exceeded one billion monthly active users in 2019 , and its interface routinely receives up to 105 requests during rush hour in major metropolitan centers [ 10 ] . This vast number of users must be routed efficiently , in real-time , and without causing any “ ex-post ” regret at the user end ; otherwise , if a user could have experienced better travel times along a non-recommended route , they would have no incentive to follow the app recommendation in the first place . In the language of congestion games [ 40 ] , this requirement is known as a “ Wardrop equilibrium ” , and it is typically represented as a high-dimensional vector describing the traffic flow along each path in the network [ 49 ] . Ideally , this equilibrium should be computed before making a recommendation , so as to minimize the number of disgruntled users . In practice however , this is rarely possible : the state of the network typically depends on random factors that vary from one epoch to the next ( weather conditions , traffic accidents , fluctuations in the total number of commuters in the system , etc . ) , so it is generally unrealistic to expect that such a recommendation can be made in advance . Our paper takes a learning approach to this problem : routing recommendations are provided in an online manner , and they are subsequently updated “ on the fly ” once the state of the network has 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . been observed . In more detail , motivated by applications to GPS routing and navigation apps , we consider an adaptive recommendation paradigm that unfolds as follows : 1 . At each epoch t = 1 , 2 , . . . , a centralized control interface – such as Google Maps – determines a routing flow for its users and provides a recommendation accordingly . 2 . After making a recommendation , the interface observes the travel times of the network ’ s users ; based on this feedback , it updates the routing recommendation and the process repeats . Main challenges . There are several key challenges that arise in this setting . First and foremost , learning methods that are well-suited to rapidly fluctuating environments may be highly suboptimal in static networks and vice versa . Second , the problem ’ s dimensionality – the number P of available paths – is exponential in the size of the underlying network , so it is crucial to propose learning methods that remain efficient in large networks . Finally , methods that require prior knowledge of the problem ’ s parameters – e.g. , the smoothness modulus of the network ’ s latency functions – are beyond reach because such knowledge can not be realistically obtained by the optimizer . In view of all this , our paper seeks to answer the following question : Is it possible to design an adaptive , parameter-agnostic algorithm that is simultaneously optimal in static and stochastic networks , and whose convergence rate is polynomial in the network ’ s size ? Our contributions in the context of related work . Our paper proposes a novel , adaptive exponential weights algorithm – dubbed ADAWEIGHT – which enjoys the following desirable properties : 1 . In static networks , the method converges to a Wardrop equilibrium at a rate of O ( ( logP ) 32 /T 2 ) . 2 . In stochastic networks , it converges to a mean Wardrop equilibrium at anO ( ( logP ) 32 / √ T ) rate . 3 . These rates are attained without any prior tuning by the optimizer . In the above , T denotes the learning horizon ( number of epochs ) and P is the number of paths used to route traffic in the network . Thus , even though P may grow exponentially , the logarithmic dependence on P ensures that the algorithm ’ s runtime remains polynomial , and in fact , subquadratic , in the size of the network . To the best of our knowledge , ADAWEIGHT is the first method that simultaneously achieves these desiderata ; to provide the necessary context , we give below a detailed account of the related work on the topic . The static regime of our learning model matches the standard framework of Blum et al . [ 7 ] who showed that a variant of the classic exponential weights ( EW ) algorithm [ 4 , 5 , 31 , 48 ] converges to a Wardrop equilibrium at an O ( 1/ √ T ) rate ( in the Cesàro , time-averaged sense ) . This result was subsequently extended to stochastic congestion games by Krichene et al . [ 22 , 24 ] , who showed that the EW algorithm also enjoys an O ( 1/ √ T ) convergence rate to mean Wardrop equilibria ( again , in a Cesàro sense ) . As we discuss in the sequel , the convergence speed of the EW algorithm of Blum et al . [ 7 ] and Krichene et al . [ 22 ] is O ( logP/ √ T ) in both cases ; however , if the method ’ s learning rate is not chosen appropriately , the EW algorithm may lead to non-convergent , chaotic behavior , even in symmetric congestion games over a 2-link Pigou network [ 42 ] . In general equilibrium problems , the O ( 1/ √ T ) rate can not be improved ( see e.g. , [ 1 , 8 ] ) without more stringent assumptions – such as strong monotonicity and the like . However , nonatomic congestion games are well known to admit a convex potential – sometimes referred to as the Beckmann–McGuire–Winsten ( BMW ) potential [ 6 ] – so the O ( 1/ √ T ) convergence guarantee of Blum et al . [ 7 ] is not optimal in the static regime ( i.e. , when costs functions do not change overtime ) . In this static regime , the optimal smooth convex minimization rate is O ( 1/T 2 ) [ 34 , 36 ] , and it is achieved by the seminal “ accelerated gradient ” algorithm of Nesterov [ 35 ] . If applied directly to our problem , Nesterov ’ s algorithm has a catastrophic Θ ( P ) dependence on the number of paths ; however , by coupling it with a “ mirror descent ” template in the spirit of [ 38 , 50 ] , Krichene et al . [ 24 ] proposed an accelerated method with an exponential projection step that is particularly wellsuited for congestion problems . In fact , going a step further , it is possible to design an accelerated exponential weights method – ACCELEWEIGHT for short – that achieves an O ( log ( P ) /T 2 ) rate in static environments . Note also that Nakamura et al . [ 33 ] recently shows that an O ( exp ( −T ) ) rate is achievable under a stronger assumption that the BMW potential is strongly convex ( which is beyond the scope of consideration of our paper ) . Crucially , the learning rate parameter of ACCELEWEIGHT must be tuned with prior knowledge of the problem ’ s smoothness parameters ; moreover , despite its optimality in the static regime , the method fails to converge altogether in stochastic problems . The universal algorithm of Nesterov [ 39 ] provides a method to resolve the former issue , but it relies on a line-search mechanism that can not be adapted to a stochastic framework , so it does not resolve the latter . On the flip side , the accelerated stochastic approximation ( AC-SA ) algorithm of Lan [ 26 ] achieves optimal rates in both the static and stochastic regimes , but it does not provide anytime guarantees ( the iteration budget must be fixed as a function of the accuracy threshold required ) , and it assumes full knowledge of the smoothness modulus of the game ’ s cost functions ( which is not realistically available to the optimizer ) . Our approach adapts to the problem ’ s smoothness parameters via an “ inverse-sum-of-squares ” learning rate in the spirit of ADAGRAD [ 16 ] – though ADAGRAD itself lacks an acceleration mechanism , so it is suboptimal in static environments [ 29 , 30 ] . To the best of our knowledge , the first orderoptimal interpolation result was achieved by the ACCELEGRAD algorithm of Levy et al . [ 29 ] for unconstrained problems ( and assuming knowledge of a compact set containing a solution of the problem ) . The UNIXGRAD proposal of Kavis et al . [ 20 ] subsequently achieved the desired adaptation in constrained problems , but under the requirement of a bounded Bregman diameter . This requirement rules out the EW template ( the simplex has infinite entropic diameter ) , so the convergence speed of UNIXGRAD is polynomial in the number of paths , and hence unsuitable for large network instances . For convenience , we compare all these works in Table 1 above . 2 Problem setup . 2.1 . The game . Building on the classical model of Beckmann et al . [ 6 ] , we will consider a class of routing games defined by three basic primitives : ( i ) the game ’ s network structure ; ( ii ) the associated set of traffic demands ; and ( iii ) the network ’ s cost functions . The formal definition is as follows : 1 . The network structure : Consider a directed graph G ≡ G ( V , E ) with vertex set V and edge set E . The focal point of interest is a set of pre-determined origin-destination ( O/D ) pairs ( Oi , Di ) ∈ V × V indexed by i ∈ N = { 1 , . . . , N } . Each pair i ∈ N is associated to a traffic demand Mi > 0 that is to be routed from Oi to Di via a fixed set of paths ( or routes ) Pi joining Oi to Di in G. We denote the set of all such paths in the network by P : = ⋃ i∈N Pi and , for concision , we denote the corresponding cardinalities as Pi : = |Pi| and P : = |P| = ∑ i Pi . We also write Mtot : = ∑ i∈N Mi and Mmax : = maxi∈N Mi for the total and maximum traffic demand associated to network ’ s O/D pairs . 2 . Routing flows : In order to route the traffic , the set of feasible flow profiles is defined as X : = { x ∈ RP+ : ∑ p∈Pi xi , p = Mi , i = 1 , . . . , N } ( 1 ) i.e. , as the product of scaled simplices X = ∏ iMi ∆ ( Pi ) . In turn , each feasible flow profile x ∈ X induces on each edge e ∈ E a routing load µe ( x ) = ∑ i∈N ∑ p∈Pi 1 { e∈pi } xi , p , i.e. , the accumulated mass of all traffic associated to the focal set of O/D pairs that goes through e. 3 . Congestion cost : The traffic routed through a given edge e ∈ E incurs a congestion cost ( or latency ) depending on the total traffic on the edge and/or any other exogenous factors . Formally , we will collectively encode all such factors in a state variable ω ∈ Ω taking values in some ambient probability space ( Ω , F , P ) . We will further assume that each edge e ∈ E is endowed with an edge-cost function ce : R+ × Ω → R+ ; in this way , given a flow profile x ∈ X and a state ω ∈ Ω , the cost to traverse edge e ∈ E will be ce ( µe ( x ) , ω ) . Analogously , the cost to traverse a path p ∈ P will be given by the induced path-cost function cp : X × Ω→ R+ defined as cp ( x , ω ) = ∑ e∈p ce ( µe ( x ) , ω ) . The only assumption that we will make for the game ’ s cost functions is as follows : Assumption 1 . Each cost function ce ( ρ , ω ) , e ∈ E is measurable in ω and non-decreasing , bounded and Lipschitz continuous in ρ . Specifically , there exist H > 0 and L > 0 such that ce ( ρ , ω ) ≤ H and |ce ( ρ , ω ) − ce ( ρ′ , ω ) | ≤ L|ρ′ − ρ| for all ρ , ρ′ ∈ [ 0 , Mtot ] and all ω ∈ Ω , e ∈ E . Remark . Assumption 1 is a very mild regularity condition that is satisfied by most congestion models that occur in practice – including BPR , polynomial , or regularly-varying latency functions , cf . [ 7 , 11 , 13 , 22 , 25 , 27 , 40 , 41 , 43 , 45 ] and references therein . For this reason , we will treat Assumption 1 as a standing , blanket assumption and we will not mention it explicitly in the sequel . The key difference between our problem setup and other nonatomic congestion models in the literature [ 40 , 45 ] is the advent of uncertainty in the game ’ s cost functions ( as modeled by the game ’ s ambient state variable ω ∈ Ω ) . For concreteness , we demonstrate below two examples of this type of uncertainty that could arise in practice : Example 1 ( Stochastically perturbed BPR costs ) . Standard BPR models [ 13 , 27 , 41 , 43 ] , under the form cBPRe ( ρ ) = ae + be ( ρ/cape ) r , ∀ρ ≥ 0 , capture the effects of a link ’ s length , its capacity , free-flow speed , etc. , on urban network congestion . However , they neglect other miscellaneous factors such as weather conditions , accidents and other random factors which may cause a temporary increase – or decrease – in congestion . In our notation , if ωe is an additive random fluctuation for the cost of edge e ∈ E , the induced cost at a flow x ∈ X becomes c ( x , ω ) = cBPRe ( µe ( x ) ) +ωe . The noise can also appear in cases where the actual latency / congestion can only be measured up to a certain error ( a case of vital importance for Internet-like networks ) . § Example 2 ( BPR models with exogenous loads ) . In practice , the total traffic is the aggregation of all commuters , irrespective of whether they are using a navigation app or not . With this in mind , consider the problem of a navigation app making a flow recommendation x ∈ X for its users . If ωe denotes the exogenous traffic load on edge e ∈ E ( i.e. , commuters not using the app ) , the total load on e will be µe ( x ) + ωe . Thus , following the BPR model , the induced cost is ce ( x , ω ) = cBPRe ( µe ( x ) + ωe ) , i.e. , the randomness is woven implicitly in the model . § 2.2 . The mean game and equilibrium flows . In the above framework , each exogenous state variable ω ∈ Ω determines an instance of a routing game , defined formally as a tuple Γω ≡ Γω ( G , N , P , cω ) where cω is shorthand for the network ’ s cost functions { ce ( · , ω ) } e∈E instantiated at ω . Of course , in analyzing the game , each individual instance Γω is meaningless by itself , unless P assigns positive probability only to a single ω . For this reason , we will instead focus on the mean game Γ ≡ Γ ( G , N , P , C ) which has the same network and routing flow structure as every Γω , ω ∈ Ω , but whose congestion costs are determined by the mean cost functions Cp ( x ) = E [ cp ( x , ω ) ] . Now , motivated by the route recommendation problem described in the introduction , we will focus on learning equilibrium flows where the controller can guarantee Wardrop ’ s principle on average [ 49 ] , i.e. , that all traffic is routed along a path with minimal mean cost . Formally , we have : Definition 1 ( Mean equilibrium flows ) . We say that x∗ ∈ X is a mean equilibrium flow if and only if , for all i ∈ N and all p , q ∈ Pi such that x∗i , p > 0 , we have Cp ( x∗ ) ≤ Cq ( x∗ ) . Remark . Definition 1 means that , on average , no user has an incentive to deviate from the recommended route ; obviously , when the support of P is a singleton , we recover the usual definition of a Wardrop equilibrium [ 6 , 40 , 49 ] . This special case will be particularly important and we describe it in detail in the next section . Importantly , the problem of finding an equilibrium flow of a ( fixed ) routing game Γω admits a potential function – often referred to as the Beckmann–McGuire–Winsten ( BMW ) potential [ 6 , 15 ] . Specifically , for a given instance ω ∈ Ω , the BMW potential is defined as Fω ( x ) : = ∑ e∈E ∫ µe ( x ) 0 ce ( ρ , ω ) dρ for all x ∈ X , ( BMW ) and it has the property that arg minFω coincides with the set Eq ( Γω ) of equilibrium flows of Γω . In our stochastic context , a natural question that arises is whether the potential property for each fixed ω ∈ Ω can be extended to the mean game Γ when ω is randomly generated . Clearly , the most direct candidate for a potential function in this case is is the averaged BMW potential : F ( x ) = E [ Fω ( x ) ] : = E [ ∑ e∈E ∫ µe ( x ) 0 ce ( ρ , ω ) dρ ] . ( 2 ) Indeed , as we show in Appendix B , we have : Proposition 1 . A flow profile x∗ ∈ X is a mean equilibrium flow if and only if it is a minimizer of F over X ; more succinctly , Eq ( Γ ) = arg minx∈X F ( x ) . In view of the above , F provides a natural merit function for examining how close a given flow profile x ∈ X is to being an equilibrium ; on that account , all our convergence certificates in the sequel will be stated in terms ofF . Note also that since ce , e ∈ E , are continuous and non-decreasing , F is a differentiable , convex function onX . However , since the probability law P on Ω is not known , we will not assume that F ( and/or its gradients ) can be explicitly computed in general . 2.3 . The learning model . The last component of our model is the actual learning process that unfolds over time . The specific sequence of events that we will consider evolves as follows : 1 . At each stage t = 1 , 2 , . . . , the navigation interface selects a flow profile Xt ∈ X and makes the corresponding routing recommendation to its users . 2 . Concurrently , the state ωt of the network is drawn from Ω ( i.i.d . relative to P ) . 3 . The interface observes the realized congestion costs ce ( µe ( Xt ) , ωt ) along each e ∈ E ( possibly up to some error ) ; subsequently , the flow recommendation is updated , and the process repeats . We will refer to this general model as the stochastic regime . For concreteness , we discuss below two special cases that have attracted particular interest in the literature : Example 3 ( Static environments ) . In the absence of randomness , P is supported on a single instance ω ∈ Ω , so we have ωt = ω for all t = 1 , 2 , . . . , and we assume that the navigation interface measures directly C ( Xt ) = c ( Xt , ω ) . This setup matches the deterministic model of Blum et al . [ 7 ] , Fischer and Vöcking [ 17 ] and Krichene et al . [ 23 , 25 ] , and we will refer to it as the static regime . § Example 4 ( Routing games with noisy observations ) . Consider the setting where only a stochastic “ perturbed ” cost ce ( µe ( x ) , ωt ) = ce ( µe ( x ) , ω ) + ωte is observed when a flow profile x ∈ X is employed at time t ( here , ω ∈ Ω is fixed ) . When ωte is the random noise such that E [ ωte ] = 0 , ∀e , ∀t , our learning model is reduced to the routing game with noisy observations studied by Krichene et al . [ 22 , 25 ] . § For a given learning window , it might not be a priori clear whether the system is in the static or stochastic regime . As we shall see in Section 3 , standard stochastic algorithms are suboptimal in the static regime , while order-optimal deterministic algorithms may fail to converge altogether in the stochastic regime . As such , the key question that we aim to answer is as follows : is it possible to design routing algorithms that automatically adapt to the appropriate setting and provide optimal convergence guarantees in both static and stochastic environments ? We address this in Section 4 .

Summary: This paper studies the equilibrium computation of congestion games and proposes a new algorithm, named ADAWEIGHT, that achieves $O(1/\sqrt{T})$ convergence in the stochastic regime and $O(1/T^2)$ in the static regime. The algorithm has several desirable properties:   1. it attains the best rates in the stochastic and static regimes,  2. the rates depend on $\log P$, not on $P$,  3. it is parameter-agnostic (no knowledge of parameters is needed),  4. it enjoys any-time guarantee (i.e., last-iterate convergence).   The results are obtained using some tools: primal and dual extrapolation, weighted-average analysis, and an adaptive learning rate. Experiments confirm the fast convergence of ADAWEIGHT in both regimes, while the existing ACCELEWEIGHT fails to converge in the stochastic regime.   --- After rebuttal ---  I appreciate the authors' kind response. I think the paper provides solid technical results, while the contribution to the ML community is somewhat marginal. Taking those into account, I keep my score of 6. I think the paper would be strengthened if the authors could work more on the smooth trade-off between the stochastic and static regimes.