Differentiable Equilibrium Computation with Decision Diagrams for Stackelberg Models of Combinatorial Congestion Games

We address Stackelberg models of combinatorial congestion games ( CCGs ) ; we aim to optimize the parameters of CCGs so that the selfish behavior of non-atomic players attains desirable equilibria . This model is essential for designing such social infrastructures as traffic and communication networks . Nevertheless , computational approaches to the model have not been thoroughly studied due to two difficulties : ( I ) bilevel-programming structures and ( II ) the combinatorial nature of CCGs . We tackle them by carefully combining ( I ) the idea of differentiable optimization and ( II ) data structures called zero-suppressed binary decision diagrams ( ZDDs ) , which can compactly represent sets of combinatorial strategies . Our algorithm numerically approximates the equilibria of CCGs , which we can differentiate with respect to parameters of CCGs by automatic differentiation . With the resulting derivatives , we can apply gradient-based methods to Stackelberg models of CCGs . Our method is tailored to induce Nesterov ’ s acceleration and can fully utilize the empirical compactness of ZDDs . These technical advantages enable us to deal with CCGs with a vast number of combinatorial strategies . Experiments on real-world network design instances demonstrate the practicality of our method . 1 Introduction . Congestion games ( CGs ) [ 49 ] form an important class of non-cooperative games and appear in various resource allocation scenarios . Combinatorial CGs ( CCGs ) can model more complex situations where each strategy is a combination of resources . A well-known example of a CCG is selfish routing [ 50 ] , where each player on a traffic network chooses an origin-destination path , which is a strategy given by a combination of some roads with limited width ( resources ) . Computing the outcomes of players ’ selfish behaviors ( or equilibria ) is essential when designing social infrastructures such as traffic networks . Therefore , how to compute equilibria of CCGs has been widely studied [ 5 , 12 , 57 , 42 ] . In this paper , we are interested in the perspective of the leader who designs non-atomic CCGs . For example , the leader aims to optimize some traffic-network parameters ( e.g. , road width values ) so that players can spend less traveling time at equilibrium . An equilibrium of non-atomic CCGs is characterized by an optimum of potential function minimization [ 41 , 53 ] . Thus , designing CCGs can be seen as a Stackelberg game ; the leader optimizes the parameters of CCGs to minimize an objective function ( typically , the social-cost function ) while the follower , who represents the population of selfish non-atomic players , minimizes the potential function . This mathematical formulation is called the Stackelberg model in the context of traffic management [ 46 ] . Therefore , we call our model with general combinatorial strategies a Stackelberg model of CCGs . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Stackelberg models of CCGs have been studied for cases where the potential function minimization has desirable properties . For example , Patriksson and Rockafellar [ 46 ] proposed a descent algorithm for traffic management using the fact that projections onto flow polyhedra can be done efficiently . However , many practical CCGs have more complicated structures . For example , in communication network design , each strategy is given by a Steiner tree ( see [ 24 , 42 ] and Section 1.1 ) , and thus the projection ( and even optimizing linear functions ) is NP-hard . How to address such computationally challenging Stackelberg models of CCGs has not been well studied , despite its practical importance . Inspired by a recent equilibrium computation method [ 42 ] , we tackle the combinatorial nature of CCGs by representing their strategy sets with zero-suppressed binary decision diagrams ( ZDDs ) [ 39 , 32 ] , which are well-established data structures that provide empirically compact representations of combinatorial objects ( e.g. , Steiner trees and Hamiltonian paths ) . Although the previous method [ 42 ] can efficiently approximate equilibria with a Frank–Wolfe-style algorithm [ 18 ] , its computation procedures break the differentiability of the outputs in the CCG parameters ( the leader ’ s variables ) , preventing us from obtaining gradient information required for optimizing leader ’ s objective functions . Our contribution is to develop a differentiable pipeline from leader ’ s variables to equilibria of CCGs , thus enabling application of gradient-based methods to the Stackelberg models of CCGs . We smooth the Frank–Wolfe iterations using softmin , thereby making computed equilibria differentiable with respect to the leader ’ s variables by automatic differentiation ( or backpropagation ) . Although the idea of smoothing with softmin is prevalent [ 29 , 38 ] , our method has the following technical novelty : • Our algorithm is tailored to induce Nesterov ’ s acceleration , making both equilibrium computation and backpropagation more efficient . To the best of our knowledge , the idea of simultaneously making iterative optimization methods both differentiable and faster is new . • Our method consists of simple arithmetic operations performed with ZDDs as in Algorithm 2 . This is essential for making our equilibrium computation accept automatic differentiation . The per-iteration complexity of our method is linear in the ZDD size . Armed with these advantages , our method can work with CCGs that have an enormous number of combinatorial strategies . We experimentally demonstrate its practical usefulness in real-world network design instances . Our method brings benefits by improving the designs of social infrastructures . Notation . Let [ n ] : = { 1 , . . . , n } . For any S ⊆ [ n ] , 1S ∈ { 0 , 1 } n denotes a binary vector whose i-th entry is 1 if and only if i ∈ S. Let ∥ · ∥ be the ℓ2-norm . 1.1 Problem setting . We introduce the problem setting and some assumptions . For simplicity , we describe the symmetric setting , although our method can be extended to an asymmetric setting , as explained in Appendix A. Combinatorial congestion games ( CCGs ) . Suppose that there is an infinite amount of players with an infinitesimal mass ( i.e. , non-atomic ) . We assume the total mass is 1 without loss of generality . Let [ n ] be a set of resources and let S ⊆ 2 [ n ] be a set of all feasible strategies . We define d : = |S| , which is generally exponential in n. Each player selects strategy S ∈ S. Let y ∈ [ 0 , 1 ] n be a vector whose i-th entry indicates the total mass of players using resource i ∈ [ n ] . In other words , if we let z ∈ △d be a vector whose entry zS ( S ∈ S ) indicates the total mass of players choosing S , we have y : = ∑ S∈S zS1S ∈ Rn . Therefore , y is included in convex hull C : = { ∑ S∈S zS1S ∣∣ z ∈ △d } , where △d : = { z ∈ Rd ∣∣ z ≥ 0 , ∑S∈S zS = 1 } is the ( d− 1 ) -dimensional probability simplex . A player choosing S incurs cost cS ( y ) : = ∑ i∈S ci ( yi ) , where each ci : R → R is assumed to be strictly increasing ; this corresponds to a natural situation where cost ci increases as i ∈ [ n ] becomes more congested . Each player selfishly selects a strategy to minimize his/her own cost . Equilibrium and potential functions . We say z ∈ △d attains a ( Wardrop ) equilibrium if for every S ∈ S such that zS > 0 , it holds that cS ( y ) ≤ minS′∈S cS′ ( y ) , where y : = ∑ S∈S zS1S . That is , no one has an incentive to deviate unilaterally . Let f : Rn → R be a potential function defined as f ( y ) : = ∑ i∈ [ n ] ∫ yi 0 ci ( u ) du . From the first-order optimality condition , it holds that z attains an equilibrium iff y = ∑ S∈S zS1S satisfies y = argminu∈C f ( u ) . Note that minimizer y is unique since ci is strictly increasing , which means f is strictly convex ( not necessarily strongly convex ) . Stackelberg model of CCGs . We turn to the problem of designing CCGs . For i ∈ [ n ] , let ci ( yi ; θ ) be a cost function with parameters θ ∈ Θ . We assume ci ( yi ; θ ) to be strictly increasing in yi for any θ ∈ Θ and differentiable with respect to θ for any y ∈ C. Let f ( y ; θ ) =∑i∈ [ n ] ∫ yi0 ci ( u ; θ ) du be a parameterized potential function , which is strictly convex in y for any θ ∈ Θ . A leader who designs CCGs aims to optimize θ values so that an objective function , F : Θ× C → R , is minimized at an equilibrium of CCGs . Typically , F is a social-cost function defined as F ( θ , y ) = ∑ i∈ [ n ] ci ( yi ; θ ) yi , which represents the total cost incurred by all players . Since an equilibrium is characterized as a minimizer of potential function f , the leader ’ s problem can be written as follows : minimize θ∈Θ F ( θ , y ) subject to y = argmin u∈C f ( u ; θ ) . ( 1 ) Since minimizer y ( θ ) : = y is unique , we can regard F ( θ , y ( θ ) ) as a function of θ . We study how to approximate derivatives of F ( θ , y ( θ ) ) with respect to θ for applying gradient-based methods to ( 1 ) . Example 1 : traffic management . We are given a network with an origin-destination ( OD ) pair . Let [ n ] be the edge set and let S ⊆ 2 [ n ] be the set of all OD paths . Each edge in the network has cost function ci ( yi ; θ ) , where θ controls the width of the roads ( edges ) . A natural example of the cost functions is c ( yi ; θ ) = yi/θi for θi > 0 ( see , e.g. , [ 46 ] ) , which satisfies the above assumptions , i.e. , strictly increasing in yi and differentiable in θi . Once θ is fixed , players selfishly choose OD paths and consequently reach an equilibrium . The leader wants to find θ that minimizes social cost F at equilibrium , which can be formulated as a Stackelberg model of form ( 1 ) . Note that although the Stackelberg model of standard selfish routing is well studied [ 46 ] , there are various variants ( e.g. , routing with budget constraints [ 28 , 42 ] ) for which existing methods do not work efficiently . Example 2 : communication network design . We consider a situation where multi-site meetings are held on a communication network ( see , e.g. , [ 24 , 42 ] ) . Given an undirected network with edge set [ n ] and some vertices called terminals , groups of people at terminals hold multi-site meetings , including people at all the terminals . Since each group wants to minimize the communication delays caused by congestion , each selfishly chooses a way to connect all the terminals , namely , a Steiner tree covering all the terminals . If we let ci ( yi ; θ ) indicate the delay of the i-th edge , a group choosing Steiner tree S ∈ S incurs cost cS ( y ; θ ) . As with the above traffic-management example , the problem of optimizing θ to minimize the total delay at equilibrium can be written as ( 1 ) .

Summary: This paper studies the problem of designing parameters of combinatorial congestion games, where selfish non-atomic agents choose an optimal route/network to optimize their own objectives. The combinatorial congestion games are a special type of potential games. The equilibrium of a congestion game with given parameters can be solved by optimizing the potential function. The problem of optimizing the game parameters is therefore a bilevel optimization problem with an equilibrium (potential function optimization problem) involved in the inner problem. Benefited from the recent development of differentiable optimization, the authors propose to solve the bilevel optimization problem by backpropagating the gradient from the argmax of the inner problem to the outer objective to run gradient descent. This end-to-end gradient descent approach can avoid solving the bilevel optimization problem with complex constraints by directly running end-to-end gradient descent.  The challenge involved in this end-to-end differentiable optimization approach is how to efficiently compute the gradient of a complex optimization problem (equilibrium computation/potential function optimization) with high-dimensional constraints. The authors leverage the Frank-Wolfe algorithm to solve the optimization problem and to get the corresponding gradient of the optimal solution. The use of the Frank-Wolfe algorithm decomposes the optimization problem into an iterative algorithm composed of multiple minimization problems, where each of them is relaxed to a softmin in order to make them differentiable. To solve the softmin problem with complex constraints, they use zero-suppressed binary decision diagrams (ZDD) to compactly represent the decision variables, which can speed up the optimization step while keeping the differentiability. The authors also utilize Nesterov's acceleration to improve the convergence of the Frank-Wolfe algorithm to $O(1/T^2)$, with the differentiability maintained.