Differentiable Equilibrium Computation with Decision Diagrams for Stackelberg Models of Combinatorial Congestion Games

We address Stackelberg models of combinatorial congestion games ( CCGs ) ; we aim to optimize the parameters of CCGs so that the selfish behavior of non-atomic players attains desirable equilibria . This model is essential for designing such social infrastructures as traffic and communication networks . Nevertheless , computational approaches to the model have not been thoroughly studied due to two difficulties : ( I ) bilevel-programming structures and ( II ) the combinatorial nature of CCGs . We tackle them by carefully combining ( I ) the idea of differentiable optimization and ( II ) data structures called zero-suppressed binary decision diagrams ( ZDDs ) , which can compactly represent sets of combinatorial strategies . Our algorithm numerically approximates the equilibria of CCGs , which we can differentiate with respect to parameters of CCGs by automatic differentiation . With the resulting derivatives , we can apply gradient-based methods to Stackelberg models of CCGs . Our method is tailored to induce Nesterov ’ s acceleration and can fully utilize the empirical compactness of ZDDs . These technical advantages enable us to deal with CCGs with a vast number of combinatorial strategies . Experiments on real-world network design instances demonstrate the practicality of our method . 1 Introduction . Congestion games ( CGs ) [ 49 ] form an important class of non-cooperative games and appear in various resource allocation scenarios . Combinatorial CGs ( CCGs ) can model more complex situations where each strategy is a combination of resources . A well-known example of a CCG is selfish routing [ 50 ] , where each player on a traffic network chooses an origin-destination path , which is a strategy given by a combination of some roads with limited width ( resources ) . Computing the outcomes of players ’ selfish behaviors ( or equilibria ) is essential when designing social infrastructures such as traffic networks . Therefore , how to compute equilibria of CCGs has been widely studied [ 5 , 12 , 57 , 42 ] . In this paper , we are interested in the perspective of the leader who designs non-atomic CCGs . For example , the leader aims to optimize some traffic-network parameters ( e.g. , road width values ) so that players can spend less traveling time at equilibrium . An equilibrium of non-atomic CCGs is characterized by an optimum of potential function minimization [ 41 , 53 ] . Thus , designing CCGs can be seen as a Stackelberg game ; the leader optimizes the parameters of CCGs to minimize an objective function ( typically , the social-cost function ) while the follower , who represents the population of selfish non-atomic players , minimizes the potential function . This mathematical formulation is called the Stackelberg model in the context of traffic management [ 46 ] . Therefore , we call our model with general combinatorial strategies a Stackelberg model of CCGs . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Stackelberg models of CCGs have been studied for cases where the potential function minimization has desirable properties . For example , Patriksson and Rockafellar [ 46 ] proposed a descent algorithm for traffic management using the fact that projections onto flow polyhedra can be done efficiently . However , many practical CCGs have more complicated structures . For example , in communication network design , each strategy is given by a Steiner tree ( see [ 24 , 42 ] and Section 1.1 ) , and thus the projection ( and even optimizing linear functions ) is NP-hard . How to address such computationally challenging Stackelberg models of CCGs has not been well studied , despite its practical importance . Inspired by a recent equilibrium computation method [ 42 ] , we tackle the combinatorial nature of CCGs by representing their strategy sets with zero-suppressed binary decision diagrams ( ZDDs ) [ 39 , 32 ] , which are well-established data structures that provide empirically compact representations of combinatorial objects ( e.g. , Steiner trees and Hamiltonian paths ) . Although the previous method [ 42 ] can efficiently approximate equilibria with a Frank–Wolfe-style algorithm [ 18 ] , its computation procedures break the differentiability of the outputs in the CCG parameters ( the leader ’ s variables ) , preventing us from obtaining gradient information required for optimizing leader ’ s objective functions . Our contribution is to develop a differentiable pipeline from leader ’ s variables to equilibria of CCGs , thus enabling application of gradient-based methods to the Stackelberg models of CCGs . We smooth the Frank–Wolfe iterations using softmin , thereby making computed equilibria differentiable with respect to the leader ’ s variables by automatic differentiation ( or backpropagation ) . Although the idea of smoothing with softmin is prevalent [ 29 , 38 ] , our method has the following technical novelty : • Our algorithm is tailored to induce Nesterov ’ s acceleration , making both equilibrium computation and backpropagation more efficient . To the best of our knowledge , the idea of simultaneously making iterative optimization methods both differentiable and faster is new . • Our method consists of simple arithmetic operations performed with ZDDs as in Algorithm 2 . This is essential for making our equilibrium computation accept automatic differentiation . The per-iteration complexity of our method is linear in the ZDD size . Armed with these advantages , our method can work with CCGs that have an enormous number of combinatorial strategies . We experimentally demonstrate its practical usefulness in real-world network design instances . Our method brings benefits by improving the designs of social infrastructures . Notation . Let [ n ] : = { 1 , . . . , n } . For any S ⊆ [ n ] , 1S ∈ { 0 , 1 } n denotes a binary vector whose i-th entry is 1 if and only if i ∈ S. Let ∥ · ∥ be the ℓ2-norm . 1.1 Problem setting . We introduce the problem setting and some assumptions . For simplicity , we describe the symmetric setting , although our method can be extended to an asymmetric setting , as explained in Appendix A. Combinatorial congestion games ( CCGs ) . Suppose that there is an infinite amount of players with an infinitesimal mass ( i.e. , non-atomic ) . We assume the total mass is 1 without loss of generality . Let [ n ] be a set of resources and let S ⊆ 2 [ n ] be a set of all feasible strategies . We define d : = |S| , which is generally exponential in n. Each player selects strategy S ∈ S. Let y ∈ [ 0 , 1 ] n be a vector whose i-th entry indicates the total mass of players using resource i ∈ [ n ] . In other words , if we let z ∈ △d be a vector whose entry zS ( S ∈ S ) indicates the total mass of players choosing S , we have y : = ∑ S∈S zS1S ∈ Rn . Therefore , y is included in convex hull C : = { ∑ S∈S zS1S ∣∣ z ∈ △d } , where △d : = { z ∈ Rd ∣∣ z ≥ 0 , ∑S∈S zS = 1 } is the ( d− 1 ) -dimensional probability simplex . A player choosing S incurs cost cS ( y ) : = ∑ i∈S ci ( yi ) , where each ci : R → R is assumed to be strictly increasing ; this corresponds to a natural situation where cost ci increases as i ∈ [ n ] becomes more congested . Each player selfishly selects a strategy to minimize his/her own cost . Equilibrium and potential functions . We say z ∈ △d attains a ( Wardrop ) equilibrium if for every S ∈ S such that zS > 0 , it holds that cS ( y ) ≤ minS′∈S cS′ ( y ) , where y : = ∑ S∈S zS1S . That is , no one has an incentive to deviate unilaterally . Let f : Rn → R be a potential function defined as f ( y ) : = ∑ i∈ [ n ] ∫ yi 0 ci ( u ) du . From the first-order optimality condition , it holds that z attains an equilibrium iff y = ∑ S∈S zS1S satisfies y = argminu∈C f ( u ) . Note that minimizer y is unique since ci is strictly increasing , which means f is strictly convex ( not necessarily strongly convex ) . Stackelberg model of CCGs . We turn to the problem of designing CCGs . For i ∈ [ n ] , let ci ( yi ; θ ) be a cost function with parameters θ ∈ Θ . We assume ci ( yi ; θ ) to be strictly increasing in yi for any θ ∈ Θ and differentiable with respect to θ for any y ∈ C. Let f ( y ; θ ) =∑i∈ [ n ] ∫ yi0 ci ( u ; θ ) du be a parameterized potential function , which is strictly convex in y for any θ ∈ Θ . A leader who designs CCGs aims to optimize θ values so that an objective function , F : Θ× C → R , is minimized at an equilibrium of CCGs . Typically , F is a social-cost function defined as F ( θ , y ) = ∑ i∈ [ n ] ci ( yi ; θ ) yi , which represents the total cost incurred by all players . Since an equilibrium is characterized as a minimizer of potential function f , the leader ’ s problem can be written as follows : minimize θ∈Θ F ( θ , y ) subject to y = argmin u∈C f ( u ; θ ) . ( 1 ) Since minimizer y ( θ ) : = y is unique , we can regard F ( θ , y ( θ ) ) as a function of θ . We study how to approximate derivatives of F ( θ , y ( θ ) ) with respect to θ for applying gradient-based methods to ( 1 ) . Example 1 : traffic management . We are given a network with an origin-destination ( OD ) pair . Let [ n ] be the edge set and let S ⊆ 2 [ n ] be the set of all OD paths . Each edge in the network has cost function ci ( yi ; θ ) , where θ controls the width of the roads ( edges ) . A natural example of the cost functions is c ( yi ; θ ) = yi/θi for θi > 0 ( see , e.g. , [ 46 ] ) , which satisfies the above assumptions , i.e. , strictly increasing in yi and differentiable in θi . Once θ is fixed , players selfishly choose OD paths and consequently reach an equilibrium . The leader wants to find θ that minimizes social cost F at equilibrium , which can be formulated as a Stackelberg model of form ( 1 ) . Note that although the Stackelberg model of standard selfish routing is well studied [ 46 ] , there are various variants ( e.g. , routing with budget constraints [ 28 , 42 ] ) for which existing methods do not work efficiently . Example 2 : communication network design . We consider a situation where multi-site meetings are held on a communication network ( see , e.g. , [ 24 , 42 ] ) . Given an undirected network with edge set [ n ] and some vertices called terminals , groups of people at terminals hold multi-site meetings , including people at all the terminals . Since each group wants to minimize the communication delays caused by congestion , each selfishly chooses a way to connect all the terminals , namely , a Steiner tree covering all the terminals . If we let ci ( yi ; θ ) indicate the delay of the i-th edge , a group choosing Steiner tree S ∈ S incurs cost cS ( y ; θ ) . As with the above traffic-management example , the problem of optimizing θ to minimize the total delay at equilibrium can be written as ( 1 ) .

Summary: The problem of computing equilibria in CGs is consider an easy task whenever the strategy space has a compact representation, since it turns out that the users minimize a convex function. When having to decide on some parameters of the CG in order to optimize the performance at equilibrium we get a bi-level optimization problem, where in the upper level the designer sets the parameters and in the lower level the users optimize their function reaching an equilibrium. Solving this bi-level problem is regarded to be far more difficult. This work deals with the latter problem and combines optimization techniques to give an algorithm that performs well in practice.