Fast Routing under Uncertainty: Adaptive Learning in Congestion Games via Exponential Weights

√ T ) equilibrium convergence rate ; however , this rate is suboptimal in static environments – i.e. , when the network is not subject to randomness . In this static regime , accelerated algorithms achieve anO ( 1/T 2 ) convergence speed , but they fail to converge altogether in stochastic problems . To fill this gap , we propose a novel , adaptive exponential weights method – dubbed ADAWEIGHT – that seamlessly interpolates between the O ( 1/T 2 ) and O ( 1/ √ T ) rates in static and stochastic environments respectively . Importantly , this “ best-of-both-worlds ” guarantee does not require any prior knowledge of the problem ’ s parameters or any tuning by the optimizer ; in addition , the method ’ s convergence speed depends subquadratically on the size of the network ( number of vertices and edges ) , so it scales gracefully to large , real-life urban networks . 1 Introduction . Navigation apps like Google Maps and Waze have user bases numbering in the hundreds of millions , and they may receive upwards of 104 routing requests per second ; in fact , Google Maps alone exceeded one billion monthly active users in 2019 , and its interface routinely receives up to 105 requests during rush hour in major metropolitan centers [ 10 ] . This vast number of users must be routed efficiently , in real-time , and without causing any “ ex-post ” regret at the user end ; otherwise , if a user could have experienced better travel times along a non-recommended route , they would have no incentive to follow the app recommendation in the first place . In the language of congestion games [ 40 ] , this requirement is known as a “ Wardrop equilibrium ” , and it is typically represented as a high-dimensional vector describing the traffic flow along each path in the network [ 49 ] . Ideally , this equilibrium should be computed before making a recommendation , so as to minimize the number of disgruntled users . In practice however , this is rarely possible : the state of the network typically depends on random factors that vary from one epoch to the next ( weather conditions , traffic accidents , fluctuations in the total number of commuters in the system , etc . ) , so it is generally unrealistic to expect that such a recommendation can be made in advance . Our paper takes a learning approach to this problem : routing recommendations are provided in an online manner , and they are subsequently updated “ on the fly ” once the state of the network has 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . been observed . In more detail , motivated by applications to GPS routing and navigation apps , we consider an adaptive recommendation paradigm that unfolds as follows : 1 . At each epoch t = 1 , 2 , . . . , a centralized control interface – such as Google Maps – determines a routing flow for its users and provides a recommendation accordingly . 2 . After making a recommendation , the interface observes the travel times of the network ’ s users ; based on this feedback , it updates the routing recommendation and the process repeats . Main challenges . There are several key challenges that arise in this setting . First and foremost , learning methods that are well-suited to rapidly fluctuating environments may be highly suboptimal in static networks and vice versa . Second , the problem ’ s dimensionality – the number P of available paths – is exponential in the size of the underlying network , so it is crucial to propose learning methods that remain efficient in large networks . Finally , methods that require prior knowledge of the problem ’ s parameters – e.g. , the smoothness modulus of the network ’ s latency functions – are beyond reach because such knowledge can not be realistically obtained by the optimizer . In view of all this , our paper seeks to answer the following question : Is it possible to design an adaptive , parameter-agnostic algorithm that is simultaneously optimal in static and stochastic networks , and whose convergence rate is polynomial in the network ’ s size ? Our contributions in the context of related work . Our paper proposes a novel , adaptive exponential weights algorithm – dubbed ADAWEIGHT – which enjoys the following desirable properties : 1 . In static networks , the method converges to a Wardrop equilibrium at a rate of O ( ( logP ) 32 /T 2 ) . 2 . In stochastic networks , it converges to a mean Wardrop equilibrium at anO ( ( logP ) 32 / √ T ) rate . 3 . These rates are attained without any prior tuning by the optimizer . In the above , T denotes the learning horizon ( number of epochs ) and P is the number of paths used to route traffic in the network . Thus , even though P may grow exponentially , the logarithmic dependence on P ensures that the algorithm ’ s runtime remains polynomial , and in fact , subquadratic , in the size of the network . To the best of our knowledge , ADAWEIGHT is the first method that simultaneously achieves these desiderata ; to provide the necessary context , we give below a detailed account of the related work on the topic . The static regime of our learning model matches the standard framework of Blum et al . [ 7 ] who showed that a variant of the classic exponential weights ( EW ) algorithm [ 4 , 5 , 31 , 48 ] converges to a Wardrop equilibrium at an O ( 1/ √ T ) rate ( in the Cesàro , time-averaged sense ) . This result was subsequently extended to stochastic congestion games by Krichene et al . [ 22 , 24 ] , who showed that the EW algorithm also enjoys an O ( 1/ √ T ) convergence rate to mean Wardrop equilibria ( again , in a Cesàro sense ) . As we discuss in the sequel , the convergence speed of the EW algorithm of Blum et al . [ 7 ] and Krichene et al . [ 22 ] is O ( logP/ √ T ) in both cases ; however , if the method ’ s learning rate is not chosen appropriately , the EW algorithm may lead to non-convergent , chaotic behavior , even in symmetric congestion games over a 2-link Pigou network [ 42 ] . In general equilibrium problems , the O ( 1/ √ T ) rate can not be improved ( see e.g. , [ 1 , 8 ] ) without more stringent assumptions – such as strong monotonicity and the like . However , nonatomic congestion games are well known to admit a convex potential – sometimes referred to as the Beckmann–McGuire–Winsten ( BMW ) potential [ 6 ] – so the O ( 1/ √ T ) convergence guarantee of Blum et al . [ 7 ] is not optimal in the static regime ( i.e. , when costs functions do not change overtime ) . In this static regime , the optimal smooth convex minimization rate is O ( 1/T 2 ) [ 34 , 36 ] , and it is achieved by the seminal “ accelerated gradient ” algorithm of Nesterov [ 35 ] . If applied directly to our problem , Nesterov ’ s algorithm has a catastrophic Θ ( P ) dependence on the number of paths ; however , by coupling it with a “ mirror descent ” template in the spirit of [ 38 , 50 ] , Krichene et al . [ 24 ] proposed an accelerated method with an exponential projection step that is particularly wellsuited for congestion problems . In fact , going a step further , it is possible to design an accelerated exponential weights method – ACCELEWEIGHT for short – that achieves an O ( log ( P ) /T 2 ) rate in static environments . Note also that Nakamura et al . [ 33 ] recently shows that an O ( exp ( −T ) ) rate is achievable under a stronger assumption that the BMW potential is strongly convex ( which is beyond the scope of consideration of our paper ) . Crucially , the learning rate parameter of ACCELEWEIGHT must be tuned with prior knowledge of the problem ’ s smoothness parameters ; moreover , despite its optimality in the static regime , the method fails to converge altogether in stochastic problems . The universal algorithm of Nesterov [ 39 ] provides a method to resolve the former issue , but it relies on a line-search mechanism that can not be adapted to a stochastic framework , so it does not resolve the latter . On the flip side , the accelerated stochastic approximation ( AC-SA ) algorithm of Lan [ 26 ] achieves optimal rates in both the static and stochastic regimes , but it does not provide anytime guarantees ( the iteration budget must be fixed as a function of the accuracy threshold required ) , and it assumes full knowledge of the smoothness modulus of the game ’ s cost functions ( which is not realistically available to the optimizer ) . Our approach adapts to the problem ’ s smoothness parameters via an “ inverse-sum-of-squares ” learning rate in the spirit of ADAGRAD [ 16 ] – though ADAGRAD itself lacks an acceleration mechanism , so it is suboptimal in static environments [ 29 , 30 ] . To the best of our knowledge , the first orderoptimal interpolation result was achieved by the ACCELEGRAD algorithm of Levy et al . [ 29 ] for unconstrained problems ( and assuming knowledge of a compact set containing a solution of the problem ) . The UNIXGRAD proposal of Kavis et al . [ 20 ] subsequently achieved the desired adaptation in constrained problems , but under the requirement of a bounded Bregman diameter . This requirement rules out the EW template ( the simplex has infinite entropic diameter ) , so the convergence speed of UNIXGRAD is polynomial in the number of paths , and hence unsuitable for large network instances . For convenience , we compare all these works in Table 1 above . 2 Problem setup . 2.1 . The game . Building on the classical model of Beckmann et al . [ 6 ] , we will consider a class of routing games defined by three basic primitives : ( i ) the game ’ s network structure ; ( ii ) the associated set of traffic demands ; and ( iii ) the network ’ s cost functions . The formal definition is as follows : 1 . The network structure : Consider a directed graph G ≡ G ( V , E ) with vertex set V and edge set E . The focal point of interest is a set of pre-determined origin-destination ( O/D ) pairs ( Oi , Di ) ∈ V × V indexed by i ∈ N = { 1 , . . . , N } . Each pair i ∈ N is associated to a traffic demand Mi > 0 that is to be routed from Oi to Di via a fixed set of paths ( or routes ) Pi joining Oi to Di in G. We denote the set of all such paths in the network by P : = ⋃ i∈N Pi and , for concision , we denote the corresponding cardinalities as Pi : = |Pi| and P : = |P| = ∑ i Pi . We also write Mtot : = ∑ i∈N Mi and Mmax : = maxi∈N Mi for the total and maximum traffic demand associated to network ’ s O/D pairs . 2 . Routing flows : In order to route the traffic , the set of feasible flow profiles is defined as X : = { x ∈ RP+ : ∑ p∈Pi xi , p = Mi , i = 1 , . . . , N } ( 1 ) i.e. , as the product of scaled simplices X = ∏ iMi ∆ ( Pi ) . In turn , each feasible flow profile x ∈ X induces on each edge e ∈ E a routing load µe ( x ) = ∑ i∈N ∑ p∈Pi 1 { e∈pi } xi , p , i.e. , the accumulated mass of all traffic associated to the focal set of O/D pairs that goes through e. 3 . Congestion cost : The traffic routed through a given edge e ∈ E incurs a congestion cost ( or latency ) depending on the total traffic on the edge and/or any other exogenous factors . Formally , we will collectively encode all such factors in a state variable ω ∈ Ω taking values in some ambient probability space ( Ω , F , P ) . We will further assume that each edge e ∈ E is endowed with an edge-cost function ce : R+ × Ω → R+ ; in this way , given a flow profile x ∈ X and a state ω ∈ Ω , the cost to traverse edge e ∈ E will be ce ( µe ( x ) , ω ) . Analogously , the cost to traverse a path p ∈ P will be given by the induced path-cost function cp : X × Ω→ R+ defined as cp ( x , ω ) = ∑ e∈p ce ( µe ( x ) , ω ) . The only assumption that we will make for the game ’ s cost functions is as follows : Assumption 1 . Each cost function ce ( ρ , ω ) , e ∈ E is measurable in ω and non-decreasing , bounded and Lipschitz continuous in ρ . Specifically , there exist H > 0 and L > 0 such that ce ( ρ , ω ) ≤ H and |ce ( ρ , ω ) − ce ( ρ′ , ω ) | ≤ L|ρ′ − ρ| for all ρ , ρ′ ∈ [ 0 , Mtot ] and all ω ∈ Ω , e ∈ E . Remark . Assumption 1 is a very mild regularity condition that is satisfied by most congestion models that occur in practice – including BPR , polynomial , or regularly-varying latency functions , cf . [ 7 , 11 , 13 , 22 , 25 , 27 , 40 , 41 , 43 , 45 ] and references therein . For this reason , we will treat Assumption 1 as a standing , blanket assumption and we will not mention it explicitly in the sequel . The key difference between our problem setup and other nonatomic congestion models in the literature [ 40 , 45 ] is the advent of uncertainty in the game ’ s cost functions ( as modeled by the game ’ s ambient state variable ω ∈ Ω ) . For concreteness , we demonstrate below two examples of this type of uncertainty that could arise in practice : Example 1 ( Stochastically perturbed BPR costs ) . Standard BPR models [ 13 , 27 , 41 , 43 ] , under the form cBPRe ( ρ ) = ae + be ( ρ/cape ) r , ∀ρ ≥ 0 , capture the effects of a link ’ s length , its capacity , free-flow speed , etc. , on urban network congestion . However , they neglect other miscellaneous factors such as weather conditions , accidents and other random factors which may cause a temporary increase – or decrease – in congestion . In our notation , if ωe is an additive random fluctuation for the cost of edge e ∈ E , the induced cost at a flow x ∈ X becomes c ( x , ω ) = cBPRe ( µe ( x ) ) +ωe . The noise can also appear in cases where the actual latency / congestion can only be measured up to a certain error ( a case of vital importance for Internet-like networks ) . § Example 2 ( BPR models with exogenous loads ) . In practice , the total traffic is the aggregation of all commuters , irrespective of whether they are using a navigation app or not . With this in mind , consider the problem of a navigation app making a flow recommendation x ∈ X for its users . If ωe denotes the exogenous traffic load on edge e ∈ E ( i.e. , commuters not using the app ) , the total load on e will be µe ( x ) + ωe . Thus , following the BPR model , the induced cost is ce ( x , ω ) = cBPRe ( µe ( x ) + ωe ) , i.e. , the randomness is woven implicitly in the model . § 2.2 . The mean game and equilibrium flows . In the above framework , each exogenous state variable ω ∈ Ω determines an instance of a routing game , defined formally as a tuple Γω ≡ Γω ( G , N , P , cω ) where cω is shorthand for the network ’ s cost functions { ce ( · , ω ) } e∈E instantiated at ω . Of course , in analyzing the game , each individual instance Γω is meaningless by itself , unless P assigns positive probability only to a single ω . For this reason , we will instead focus on the mean game Γ ≡ Γ ( G , N , P , C ) which has the same network and routing flow structure as every Γω , ω ∈ Ω , but whose congestion costs are determined by the mean cost functions Cp ( x ) = E [ cp ( x , ω ) ] . Now , motivated by the route recommendation problem described in the introduction , we will focus on learning equilibrium flows where the controller can guarantee Wardrop ’ s principle on average [ 49 ] , i.e. , that all traffic is routed along a path with minimal mean cost . Formally , we have : Definition 1 ( Mean equilibrium flows ) . We say that x∗ ∈ X is a mean equilibrium flow if and only if , for all i ∈ N and all p , q ∈ Pi such that x∗i , p > 0 , we have Cp ( x∗ ) ≤ Cq ( x∗ ) . Remark . Definition 1 means that , on average , no user has an incentive to deviate from the recommended route ; obviously , when the support of P is a singleton , we recover the usual definition of a Wardrop equilibrium [ 6 , 40 , 49 ] . This special case will be particularly important and we describe it in detail in the next section . Importantly , the problem of finding an equilibrium flow of a ( fixed ) routing game Γω admits a potential function – often referred to as the Beckmann–McGuire–Winsten ( BMW ) potential [ 6 , 15 ] . Specifically , for a given instance ω ∈ Ω , the BMW potential is defined as Fω ( x ) : = ∑ e∈E ∫ µe ( x ) 0 ce ( ρ , ω ) dρ for all x ∈ X , ( BMW ) and it has the property that arg minFω coincides with the set Eq ( Γω ) of equilibrium flows of Γω . In our stochastic context , a natural question that arises is whether the potential property for each fixed ω ∈ Ω can be extended to the mean game Γ when ω is randomly generated . Clearly , the most direct candidate for a potential function in this case is is the averaged BMW potential : F ( x ) = E [ Fω ( x ) ] : = E [ ∑ e∈E ∫ µe ( x ) 0 ce ( ρ , ω ) dρ ] . ( 2 ) Indeed , as we show in Appendix B , we have : Proposition 1 . A flow profile x∗ ∈ X is a mean equilibrium flow if and only if it is a minimizer of F over X ; more succinctly , Eq ( Γ ) = arg minx∈X F ( x ) . In view of the above , F provides a natural merit function for examining how close a given flow profile x ∈ X is to being an equilibrium ; on that account , all our convergence certificates in the sequel will be stated in terms ofF . Note also that since ce , e ∈ E , are continuous and non-decreasing , F is a differentiable , convex function onX . However , since the probability law P on Ω is not known , we will not assume that F ( and/or its gradients ) can be explicitly computed in general . 2.3 . The learning model . The last component of our model is the actual learning process that unfolds over time . The specific sequence of events that we will consider evolves as follows : 1 . At each stage t = 1 , 2 , . . . , the navigation interface selects a flow profile Xt ∈ X and makes the corresponding routing recommendation to its users . 2 . Concurrently , the state ωt of the network is drawn from Ω ( i.i.d . relative to P ) . 3 . The interface observes the realized congestion costs ce ( µe ( Xt ) , ωt ) along each e ∈ E ( possibly up to some error ) ; subsequently , the flow recommendation is updated , and the process repeats . We will refer to this general model as the stochastic regime . For concreteness , we discuss below two special cases that have attracted particular interest in the literature : Example 3 ( Static environments ) . In the absence of randomness , P is supported on a single instance ω ∈ Ω , so we have ωt = ω for all t = 1 , 2 , . . . , and we assume that the navigation interface measures directly C ( Xt ) = c ( Xt , ω ) . This setup matches the deterministic model of Blum et al . [ 7 ] , Fischer and Vöcking [ 17 ] and Krichene et al . [ 23 , 25 ] , and we will refer to it as the static regime . § Example 4 ( Routing games with noisy observations ) . Consider the setting where only a stochastic “ perturbed ” cost ce ( µe ( x ) , ωt ) = ce ( µe ( x ) , ω ) + ωte is observed when a flow profile x ∈ X is employed at time t ( here , ω ∈ Ω is fixed ) . When ωte is the random noise such that E [ ωte ] = 0 , ∀e , ∀t , our learning model is reduced to the routing game with noisy observations studied by Krichene et al . [ 22 , 25 ] . § For a given learning window , it might not be a priori clear whether the system is in the static or stochastic regime . As we shall see in Section 3 , standard stochastic algorithms are suboptimal in the static regime , while order-optimal deterministic algorithms may fail to converge altogether in the stochastic regime . As such , the key question that we aim to answer is as follows : is it possible to design routing algorithms that automatically adapt to the appropriate setting and provide optimal convergence guarantees in both static and stochastic environments ? We address this in Section 4 .

Summary: The paper considers the speed on learning to route an equilibrium for in classical network routing models with congestion. The main application could be routing suggestions made by an application such as google maps.   They consider two versions:   (1) static: when the congestion function is fixed throughout the learning process, called static process. In this case, equilibrium is the minimizer of the potential function, so algorithms minimizing the potential can lead to fast convergence to equilibrium. This leads to a $O(1/T^2)$ error rate as a function of the time.   (2) which is called stochastic, but appears to be worse case instead, where routers can use multiplicative weight, leading to a $O(1/\sqrt{T})$ error rate.   Under some smoothness assumption on the congestion functions (bounded and Lipschitz) the paper proposes a "best-of-both worlds" algorithm called ADAWeight, that claims to achieve the $O(1/T^2)$ regret in the static case and the $O(1/\sqrt{T})$ regret in case (2).