Title: Mode Connectivity in Auction Design
Venue: NeurIPS 2023
Accepted: yes
Keywords: Differentiable Economics

Mechanism Design

Neural Network Theory

Mode Connectivity

RochetNet
Abstract: Optimal auction design is a fundamental problem in algorithmic game theory. This problem is notoriously difficult already in very simple settings. Recent work in differentiable economics showed that neural networks can efficiently learn known optimal auction mechanisms and discover interesting new ones. In an attempt to theoretically justify their empirical success, we focus on one of the first such networks, RochetNet, and a generalized version for affine maximizer auctions. We prove that they satisfy mode connectivity, i.e., locally optimal solutions are connected by a simple, piecewise linear path such that every solution on the path is almost as good as one of the two local optima. Mode connectivity has been recently investigated as an intriguing empirical and theoretically justifiable property of neural networks used for prediction problems. Our results give the first such analysis in the context of differentiable economics, where neural networks are used directly for solving non-convex optimization problems.

Mode Connectivity in Auction Design

Abstract

Optimal auction design is a fundamental problem in algorithmic game theory. This problem is notoriously difficult already in very simple settings. Recent work in differentiable economics showed that neural networks can efficiently learn known optimal auction mechanisms and discover interesting new ones. In an attempt to theoretically justify their empirical success, we focus on one of the first such networks, RochetNet , and a generalized version for affine maximizer auctions . We prove that they satisfy mode connectivity , i.e., locally optimal solutions are connected by a simple, piecewise linear path such that every solution on the path is almost as good as one of the two local optima. Mode connectivity has been recently investigated as an intriguing empirical and theoretically justifiable property of neural networks used for prediction problems. Our results give the first such analysis in the context of differentiable economics, where neural networks are used directly for solving non-convex optimization problems.

1 Introduction

Auction design is a core problem in mechanism design, with immense applications in electronic commerce (such as sponsored search auctions) as well as in the public sector (such as spectrum auctions). In a revenue maximizing auction, the auctioneer needs to design a mechanism to allocate resources to buyers, and set prices in order to maximize the expected revenue. The buyers' preferences are private and they may behave strategically by misreporting them. For this reason, it is often desirable to devise dominant strategy incentive compatible (DSIC) and individually rational (IR) mechanisms. By definition, in a DSIC mechanism, it is a dominant strategy for the buyers to report the true valuations; in an IR mechanism, each participating truthful buyer receives a nonnegative payoff.

We focus on DSIC and IR mechanisms that maximize the expected revenue, assuming that the buyers' preferences are drawn from a distribution known to the auctioneer. A classical result of

Myerson [1981] provides the optimal mechanism for the case of a single item and arbitrary number of buyers. Finding the optimal mechanisms for more general settings is a tantalizingly difficult problem. We refer the reader to the surveys by Rochet and Stole [2003], Manelli and Vincent [2007] and Daskalakis [2015] for partial results and references. In particular, no analytic solution is known even for two items and two buyers. Selling multiple items to a single buyer is computationally intractable [Daskalakis et al., 2014]. Already for two items and a single buyer, the description of the optimal mechanism may be uncountable [Daskalakis et al., 2013]. Recent work gives a number of important partial characterizations, e.g. Daskalakis et al. [2015], Giannakopoulos and Koutsoupias [2014], as well as results for weaker notions of Bayesian incentive compatibility, e.g. Cai et al. [2012b,a, 2013], Bhalgat et al. [2013].

Conitzer and Sandholm [2002, 2004] proposed the approach of automated mechanism design to use optimization and computational methods to obtain (near) optimal mechanisms for specific problems; see also Sandholm and Likhodedov [2015]. An active recent area of research uses machine learning tools. In particular, Dütting et al. [2019] designed and trained neural networks to automatically find optimal auctions. They studied two network architectures, and showed that several theoretically optimal mechanisms can be recovered using this approach, as well as interesting new mechanisms can be obtained. The first network they studied is RochetNet . This is a simple two-layer neural network applicable to the single buyer case, leveraging Rochet's [1987] characterization of the optimal mechanism. The second network, RegretNet does not require such a characterization and is applicable for multiple buyers; however, it only provides approximate incentive compatibility.

Dütting et al. [2019] coined the term 'differentiable economics' for this approach, and there has been significant further work in this direction. These include designing auctions for budget constrained buyers [Feng et al., 2018]; multi-facility location Golowich et al. [2018]; balancing fairness and revenue objectives [Kuo et al., 2020]; incorporating non-linear utility functions and other networks trained from interaction data [Shen et al., 2019] 2 ; designing revenue-maximizing auctions with differentiable matchings [Curry et al., 2022a]; contextual auction design [Duan et al., 2022]; designing taxation policies [Zheng et al., 2022], and more.

The purpose of this work is to supply theoretical evidence behind the success of neural networks in differentiable economics. The revenue is a highly non-convex function of the parameters in the neural network. Curiously, gradient approaches seem to recover globally optimal auctions despite this non-convexity. Similar phenomena have been studied more generally in the context of deep networks, and theoretical explanations have been proposed, in particular, overparametrization [Allen-Zhu et al., 2019, Du et al., 2019].

Mode connectivity Recent work has focused on a striking property of the landscape of loss functions of deep neural networks: local optimal solutions (modes) found by gradient approaches are connected by simple paths in the parameter space. We provide an informal definition of mode connectivity here.

Definition 1 ( ε -mode connected (informal)) . We say that two solutions are ε -mode connected , if they are connected by a continuous path of solutions, such that the loss function does not worsen by more than ε compared to one of the end points on the entire path.

This phenomenon was identified by Garipov et al. [2018] and by Draxler et al. [2018]. Mode connectivity can help to explain the empirical performance of stochastic gradient descent (sgd) (or ascent, in case of revenue maximization). To some extent, mode connectivity prevents a poor local minimal valley region on the function value, from which the sgd method cannot escape easily. Suppose such a bad local minimum exists. Then mode connectivity implies that there exists a path from this bad local minimum to a global minimum on which the loss function does not significantly increase. Therefore, the intuition is that from every bad local minimum, a (stochastic) gradient method would eventually be able to find a way to escape. However, we would like to emphasize that mode connectivity does not provide a formal proof of the success of sgd. It only provides a useful intuition of why sgd does not completely trapped in local optima.

Kuditipudi et al. [2019] gave strong theoretical arguments for mode connectivity. They introduce the notion of ε -dropout stability : solutions to a neural network such that in each layer, one can remove

at least half the neurons and rescale the remaining units such that the loss function increases by at most ε . Solutions that are ε -dropout stable are then shown to be ε -mode connected. Moreover, they show that noise stability (see e.g., Arora et al. [2018]) implies dropout stability, and hence, mode connectivity. Nguyen [2019] showed mode connectivity when there is a hidden layer larger than the training dataset. Shevchenko and Mondelli [2020] shows that stochastic gradient descent solutions to sufficiently overparametrized neural networks are dropout stable even if we only keep a small, randomly sampled set of neurons from each layer.

1.1 Our contributions

RochetNet In this paper, we first establish mode connectivity properties of RochetNet , the architecture in Dütting et al. [2019] for multiple items and a single buyer. These networks have a single hidden layer, corresponding to the menu. Within this hidden layer, each neuron directly corresponds to an option in the menu: which contains an allocation and a price offered to the buyer (see Figure 1). The buyer is assigned the single option, including the allocation and the price, maximizing the buyer's utility. Such an option is called active for the buyer. The loss function of RochetNet is the revenue, the expected price paid by the buyer. Despite its simplicity, the experiments on RochetNet in Dütting et al. [2019] gave impressive empirical results in different scenarios. For example, in the experiments with up to six items and uniform value distributions, RochetNet achieves almost the same revenue ( 99 . 9% ) as the Straight-Jacket Auctions in Giannakopoulos and Koutsoupias [2018], which are known to be optimal in this case. This success is not limited to a single example, as RochetNet also consistently performs well in other scenarios, including when infinite menu size is necessary. Furthermore, Dütting et al. [2019] demonstrated the usefulness of RochetNet in discovering optimal auctions in situations that were previously unexplored from a theoretical perspective.

First, in Theorem 9, we show that for linear utilities, ε -mode connectivity holds between two solutions that are ε -reducible : out of the K +1 menu options (neurons), there exists a subset of at most √ K +1 containing an active option for the buyer with probability at least 1 -ε . Assuming that the valuations are normalized such that the maximum valuation of any buyer is at most one, it follows that if we remove all other options from the menu, at most ε of the expected revenue is lost. The assumption of being ε -reducible is stronger than ε -dropout stability that only drops a constant fraction of the neurons. At the same time, experimental results in Dütting et al. [2019] show evidence of this property being satisfied in practice. They experimented with different sized neural networks in a setting when the optimal auction requires infinite menu size. Even with 10 , 000 neurons available, only 59 options were active, i.e., used at least once when tested over a large sample size. We note that this property also highlights an advantage of RochetNet over RegretNet and other similar architectures: instead of a black-box neural network, it returns a compact, easy to understand representation of a mechanism.

Our second main result (Theorem 10) shows that for n items and linear utilities, if the number of menu options K is sufficiently large, namely, (2 /ε ) 4 n , then ε -mode connectivity holds between any two solutions for any underlying distribution. The connectivity property holds pointwise: for any particular valuation profile, the revenue may decrease by at most ε along the path. A key tool in this ε -mode connectivity result is a discretization technique from Dughmi et al. [2014]. We note that such a mode connectivity result can be expected to need a large menu size. In Appendix C, we present an example with two disconnected local maxima for K = 1 .

Affine Maximizer Auctions We also extend our results and techniques to neural networks for affine maximizer auctions (AMA) studied in Curry et al. [2022b]. This is a generalization of RochetNet for multi-buyer scenarios. It can also be seen as a weighted variant of the Vickrey-Clarke-Groves (VCG) mechanism [Vickrey, 1961, Clarke, 1971, Groves, 1973]. AMA offers various allocation options. For a given valuation profile, the auctioneer chooses the allocation with the highest weighted sum of valuations, and computes individual prices for the buyers; the details are described in Section 2.2. AMAis DSIC and IR, however, is not rich enough to always represent the optimal auction.

For AMA networks, we show similar results (Theorem 13 and 14) as for RochetNet . We first prove ε -mode connectivity holds between two solutions that are ε -reducible (see Definition 11). Curry et al. [2022b] provides evidence of this property being satisfied in practice, observing (in Sec. 7.1) 'Moreover, we found that starting out with a large number of parameters improves performance, even though by the end of training only a tiny number of these parameters were actually used. ' . Secondly, we also show that if the number of menu options K is sufficiently large, namely, (16 m 3 /ε 2 ) 2 nm ,

then ε -mode connectivity holds pointwise between any two solutions. That is, it is valid for any underlying distribution of valuations, possibly correlated between different buyers.

Relation to previous results on mode connectivity Our results do not seem to be deducible from previous mode connectivity results, as we outline as follows. Previous literature on mode connectivity investigated neural networks used for prediction. The results in Kuditipudi et al. [2019], Nguyen [2019], Shevchenko and Mondelli [2020] and other papers crucially rely on the properties that the networks minimize a convex loss function between the predicted and actual values, and require linear transformations in the final layer. RochetNet and AMA networks are fundamentally different. The training data does not come as labeled pairs and these network architectures are built directly for solving an optimization problem. For an input valuation profile, the loss function is the negative revenue of the auctioneer. In RochetNet , this is obtained as the negative price of the utility-maximizing bundle; for AMA it requires an even more intricate calculation. The objective is to find parameters of the neural network such that the expected revenue is as large as possible. The menu options define a piecewise linear surface of utilities, and the revenue in RochetNet can be interpreted as the expected bias of the piece corresponding to a randomly chosen input.

Hence, the landscape of the loss function is fundamentally different from those analyzed in the above mentioned works. The weight interpolation argument that shows mode-connectivity from dropout stability is not applicable in this context. The main reason is that the loss function is not a simple function of the output of the network, but is defined by choosing the price of the argmax option. We thus need a more careful understanding of the piecewise linear surfaces corresponding to the menus.

Significance for Practitioners Wesee the main contribution of our paper in explaining the empirical success and providing theoretical foundations for already existent practical methods, and not in inventing new methods. Nevertheless, two insights a practitioner could use are as follows: (i) It is worth understanding the structure of the auction in question. If one can, e.g., understand whether ε -reducibility holds for a particular auction, this might indicate whether RochetNet or AMA are good methods to apply to this particular case. (ii) Size helps: If one encounters bad local optima, increasing the menu size and rerunning RochetNet or AMA might be a potential fix and will eventually lead to a network satisfying mode connectivity.

2 Auction Settings

We consider the case with m buyers and one seller with n divisible items each in unit supply. Each buyer has an additive valuation function v i ( S ) := ∑ j ∈ S v ij , where v ij ∈ V represents the valuation of the buyer i on item j and V is the set of possible valuations. Throughout the paper, we normalize the range to the unit simplex: we assume V = [0 , 1] , and ∥ v i ∥ 1 = ∑ j v ij ≤ 1 for every buyer i . With slight abuse of notation, we let v = ( v 11 , v 12 , · · · , v ij , · · · , v mn ) ⊤ and v i = ( v i 1 , v i 2 , · · · , v in ) ⊤ . The buyers' valuation profile v is drawn from a distribution F ∈ P ( V m × n ) . Throughout, we assume that the buyers have quasi-linear utilities : if a buyer with valuation v i receives an allocation x ∈ [0 , 1] n at price p , their utility is v ⊤ i x -p .

The seller has access to samples from the distribution F , and wants to sell these items to the buyers through a DSIC 3 and IR auction and maximize the expected revenue. In the auction mechanism, the i -th bidder reports a bid b i ∈ [0 , 1] n . The entire bid vector b ∈ [0 , 1] m × n will be denoted as b = ( b 1 , . . . , b m ) = ( b i , b -i ) , where b -i represents all the bids other than buyer i . In a DSIC mechanism, it is a dominant strategy for the agents to report b i = v i , i.e., reveal their true preferences.

Definition 2 (DSIC and IR auction) . An auction mechanism requires the buyers to submit bids b i ∈ [0 , 1] n , and let b = ( b 1 , . . . , b m ) . The output is a set of allocations x ( b ) = ( x 1 ( b ) , . . . , x m ( b )) , x i ( b ) ∈ [0 , 1] n , and prices p ( b ) = ( p 1 ( b ) , . . . , p m ( b )) ∈ R m . Since there is unit supply of each item, we require ∑ i x ij ( b ) ≤ 1 , where x ij ( b ) is the allocation of buyer i of item j .

(i) An auction is dominant strategy incentive compatible (DSIC) if v ⊤ i x i ( v i , b -i ) -p i ( v i , b -i ) ≥ v ⊤ x i ( b i , b -i ) -p i ( b i , b -i ) for any buyer i and any bid b = ( b i , b -i ) .

(ii) An auction is individually rational (IR) if v ⊤ i x i ( v i , b -i ) -p i ( v i , b -i ) ≥ 0 .

The revenue of a DSIC and IR auction is

2.1 Single Buyer Auctions: RochetNet

Dütting et al. [2019] proposed RochetNet as a DISC and IR auction for the case of a single buyer. We omit the subscript i for buyers in this case. A (possibly infinite sized) menu M comprises a set of options offered to the buyer: M = { ( x ( k ) , p ( k ) ) } k ∈K . In each option ( x ( k ) , p ( k ) ) , x ( k ) ∈ [0 , 1] n represents the amount of items, and p ( k ) ∈ R + represents the price. We assume that 0 ∈ K , and ( x (0) , p (0) ) = ( 0 , 0) to guarantee IR. We call this the default option , whereas all other options are called regular options . We will use K to denote the number of regular options; thus, |K| = K +1 .

A buyer submits a bid b ∈ [0 , 1] n representing their valuation, and is assigned to option k ( b ) ∈ K that maximizes the utility 4

This is called the active option for the buyer. Note that option 0 guarantees that the utility is nonnegative, implying the IR property. It is also easy to see that such an auction is DSIC. Therefore, one can assume that b = v , i.e., the buyer submits their true valuation; or equivalently, the buyer is allowed to directly choose among the menu options one that maximizes their utility. Moreover, it follows from Rochet [1987] that every DSIC and IR auction for a single buyer can be implemented with a (possibly infinite size) menu using an appropriate tie-breaking rule.

Given a menu M , the revenue is defined as

RochetNet RochetNet (see Figure 1) is a neural network with three layers: an input layer ( n neurons), a middle layer ( K neurons), and an output layer ( 1 neuron):

the input layer takes an n -dimensional bid b ∈ V n , and sends this information to the middle layer;

the middle layer has K neurons. Each neuron represents a regular option in the menu M , which has parameters x ( k ) ∈ [0 , 1] n and p ( k ) ∈ R + , where x ( k ) ∈ [0 , 1] n represents the allocation of option k and p ( k ) represents the price of option k . Neuron k maps from b ∈ V n to b ⊤ x ( k ) -p ( k ) , i.e., the utility of the buyer when choosing option k ;

the output layer receives all utilities from different options and maximizes over these options and 0 : max { max k { ( x ( k ) ) ⊤ b -p ( k ) } , 0 } .

We will use ❘❡✈ ( M ) to denote the revenue of the auction with menu options K = { 0 , 1 , 2 , . . . , K } , where 0 represents the default option ( 0 , 0) .

The training objective for the RochetNet is to maximize the revenue ❘❡✈ ( M ) , which is done by stochastic gradient ascent. Note, however, that the revenue is the price of an argmax option, which makes it a non-continuous function of the valuations. For this reason, Dütting et al. [2019] use a softmax -approximation of the argmax as their loss function instead. However, argmax is used for testing. In Appendix B, we bound the difference between the revenues computed with these two different activation functions, assuming that the probability density function of the distribution F admits a finite upper bound. Lemma 19 shows that the difference between the revenues for softmax and argmax is roughly inverse proportional to the parameter Y of the softmax function. This allows the practitioner to interpolate between smoothness of the loss function and provable quality of the softmax approximation by tuning the parameter Y .

-

p

1

-

p

2

· · ·

-

p

k

b

1

b

2

· · ·

b

d

x

(1)

x

(2)

x

(

k

)

· · ·

0

2.2 Affine Maximizer Auctions

Affine Maximizer Auctions (AMA) also provide a menu M with a set of options K . Each option is of the form ( x ( k ) , β ( k ) ) ∈ [0 , 1] n × m × R , where x ( k ) ij ∈ [0 , 1] represents the allocation of item i to buyer j , with the restriction that ∑ i x ( k ) ij ≤ 1 for each item j , and β ( k ) represents a 'boost' . We again assume 0 ∈ K , and ( x (0) , β (0) ) = ( 0 , 0) , and call this the default option; all other options are called the regular options .

Given the bids b i ∈ [0 , 1] n of the agents, the auctioneer computes a weighted welfare, using weights w i ∈ R + for the valuations of each agent, and adds the boost β ( k ) . Then, the allocation maximizing the weighted boosted welfare is chosen, i.e., the option with

This will also be referred to as the active option . The prices collected from the buyers are computed according to the Vickrey-Clarke-Groves (VCG) scheme. Namely,

̸

̸

̸

Here, k ( b -i ) represents the option maximizing the weighted boosted welfare when buyer i is omitted, i.e., k ( b -i ) ∈ arg max k ∈K ∑ ℓ = i w ℓ b ⊤ ℓ x ( k ) ℓ + β ( k ) . It is known that AMA is DSIC and IR. Hence, we can assume that the submitted bids b i represent the true valuations v i . We also assume the ties are broken in favor of maximizing the total payment. In case of unit weights, this is equivalent to choosing the smallest β ( k ) values, see (2) in Section 4. Given the menu M , the revenue of the AMA is

In this paper, we focus on the case when w i = 1 for all buyers. This is also used in the experiments in Curry et al. [2022b]. For this case, AMA can be implemented by a three layer neural network similar to RochetNet , with m × n input neurons. For the more general case when the weights w i can also be adjusted, one can include an additional layer that combines the buyers' allocations.

Note that for a single buyer and w 1 = 1 , AMA corresponds to RochetNet , with price p ( k ) = -β ( k ) for each menu option. Indeed, in the formula defining the price p i ( b ) , the first term is 0, as well as the sum in the second term.

Similarly to RochetNet , the loss function, which is maximized via stochastic gradient ascent, is a softmax -approximation of the revenue ❘❡✈ ( M ) , in order to avoid the discontinuities introduced by the argmax . We bound the difference in the revenue in Appendix D.3, concluding that it decreases with large parameter Y as in the RochetNet case.

2.3 Mode Connectivity

One can view the revenue as a function of the menus, i.e., the parameters in the mechanism: (i) in RochetNet , { ( x ( k ) , p ( k ) ) } k ∈K ; (ii) in AMA, { ( x ( k ) , β ( k ) ) } k ∈K . We use M to denote the set of all possible menus.

Definition 3. (Mode connectivity) Two menus M 1 , M 2 ∈ M are ε -mode-connected if there is a continuous curve π : [0 , 1] → M such that (i) π (0) = M 1 ; (ii) π (1) = M 2 ; and (iii) for any t ∈ [0 , 1] , ❘❡✈ ( π ( t )) ≥ min { ❘❡✈ ( M 1 ) , ❘❡✈ ( M 2 ) } -ε .

3 Mode Connectivity for the RochetNet

In this section we present and prove our main results for the RochetNet. For some statements, we only include proof sketches. The detailed proofs can be found in Appendix A in the supplementary material. The following definition plays an analogous role to ε -dropout stability in Kuditipudi et al. [2019].

Definition 4. A menu M with |K| = K +1 options is called ε -reducible if there is a subset K ′ ⊆ K with 0 ∈ K ′ , |K ′ | ≤ √ K +1 such that, with probability at least 1 -ε over the distribution of the valuation of the buyer, the active option assigned to the buyer is contained in K ′ .

As noted in the Introduction, such a property can be observed in the experimental results in Dütting et al. [2019]. The motivation behind this definition is that if a menu satisfies this property, then all but √ K +1 options are more or less redundant. In fact, if a menu is ε -reducible, then dropping all but the at most √ K +1 many options in K ′ results in a menu M ′ with ❘❡✈ ( M ′ ) ≥ ❘❡✈ ( M ) -ε because the price of any selected option is bounded by ∥ v ∥ 1 ≤ 1 .

As a first step towards showing the mode connectivity results, we show that 0 -reducibility implies 0 -mode-connectivity. We will then use this to derive our two main results, namely that two ε -reducible menus are always ε -mode-connected and that two large menus are always ε -mode-connected.

Proposition 5. If two menus M 1 and M 2 for the RochetNet are 0 -reducible, then they are 0 -modeconnected. Moreover, the curve transforming M 1 into M 2 is piecewise linear with only three pieces.

To prove Proposition 5, we introduce two intermediate menus ̂ M 1 and ̂ M 2 , and show that every menu in the piecewise linear interpolation from M 1 via ̂ M 1 and ̂ M 2 to M 2 yields a revenue of at least min { ❘❡✈ ( M 1 ) , ❘❡✈ ( M 2 ) } . Using that menu M 1 has only √ K +1 non-redundant options, menu ̂ M 1 will be defined by repeating each of the √ K +1 options √ K +1 times. Menu ̂ M 2 will be derived from M 2 similarly. A technical lemma makes sure that this copying can be done in such a way that each pair of a non-redundant option of M 1 and a non-redundant option of M 2 occurs exactly for one index in ̂ M 1 and ̂ M 2 .

To make this more formal, we first assume without loss of generality that K +1 is a square, such that √ K +1 is an integer. It is straightforward to verify that the theorem is true for non-squares K +1 , too. Suppose the options in M 1 and M 2 are indexed with k ∈ K = { 0 , 1 , . . . , K } . Since M 1 is 0 -reducible, there is a subset K 1 ⊆ K with 0 ∈ K 1 , |K 1 | = √ K +1 such that an option with index in K 1 is selected with probability 1 over the distribution of the possible valuations. Similarly, such a set K 2 exists for M 2 . To define the curve that provides mode connectivity, we need the following technical lemma, which is proven in Appendix A.

Lemma 6. There exists a bijection φ : K → K 1 × K 2 such that for all k ∈ K 1 we have that φ ( k ) ∈ { k } × K 2 , and for all k ∈ K 2 we have that φ ( k ) ∈ K 1 ×{ k } .

With this lemma, we can define ̂ M 1 and ̂ M 2 . Let φ the bijection from Lemma 6 and suppose M 1 = { ( x ( k ) , p ( k ) ) } k ∈K . We then define ̂ M 1 = { ( x ( φ 1 ( k )) , p ( φ 1 ( k )) ) } k ∈K , where φ 1 ( k ) is the first component of φ ( k ) . Similarly, ̂ M 2 is derived from M 2 by using the second component φ 2 ( k ) of φ ( k ) instead of φ 1 ( k ) . It remains to show that all menus on the three straight line segments from M 1 via ̂ M 1 and ̂ M 2 to M 2 yield a revenue of at least min { ❘❡✈ ( M 1 ) , ❘❡✈ ( M 2 ) } , which is established by the following two propositions; their proofs can be found in Appendix A.

Proposition 7. Let M = λM 1 +(1 -λ ) ̂ M 1 be a convex combination of the menus M 1 and ̂ M 1 . Then ❘❡✈ ( M ) ≥ ❘❡✈ ( M 1 ) . Similarly, every convex combination of the menus M 2 and ̂ M 2 has revenue at least ❘❡✈ ( M 2 ) .

The idea to prove Proposition 7 is that, on the whole line segment from M 1 to ̂ M 1 , the only active options are those in K ′ , implying that the revenue does not decrease.

Proposition 8. Let M = λ ̂ M 1 +(1 -λ ) ̂ M 2 be a convex combination of the menus ̂ M 1 and ̂ M 2 . Then, ❘❡✈ ( M ) ≥ λ ❘❡✈ ( ̂ M 1 ) + (1 -λ ) ❘❡✈ ( ̂ M 2 ) .

The idea to prove Proposition 8 is that, due to the special structure provided by Lemma 6, a linear interpolation between the menus also provides a linear interpolation between the revenues. Note that without the construction of Lemma 6, such a linear relation would be false; such an example is shown in Appendix C.

Proposition 5 directly follows from Proposition 7 and Proposition 8. Based on Proposition 5, we can show our two main theorems for the RochetNet. The first result follows relatively easily from Proposition 5.

Theorem 9. If two menus M 1 and M 2 for the RochetNet are ε -reducible, then they are ε -modeconnected. Moreover, the curve transforming M 1 into M 2 is piecewise linear with only five pieces.

Proof. We prove this result by showing that every ε -reducible menu M can be linearly transformed into a 0 -reducible menu ˜ M such that each convex combination of M and ˜ M achieves a revenue of at least ❘❡✈ ( M ) -ε . This transformation converting M 1 and M 2 to ˜ M 1 and ˜ M 2 , respectively, yields the first and the fifth of the linear pieces transforming M 1 to M 2 . Together with Proposition 5 applied to ˜ M 1 and ˜ M 2 serving as the second to fourth linear piece; the theorem then follows.

To this end, let M be an ε -reducible menu with options indexed by k ∈ K . By definition, there is a subset K ′ ⊆ K of at most √ K +1 many options such that, with probability at least 1 -ε , the assigned active option is contained in K ′ . Let ˜ M consist of the same allocations as M , but with modified prices. For an option k ∈ K ′ , the price ˜ p ( k ) = p ( k ) in ˜ M is the same as in M . However, for an option k ∈ K \ K ′ , we set the price ˜ p ( k ) > 1 in ˜ M to be larger than the largest possible valuation of any option ∥ v ∥ 1 ≤ 1 . It follows that such an option will never be selected and ˜ M is 0 -reducible.

To complete the proof, let us look at the reward of a convex combination M ′ = λM +(1 -λ ) ˜ M . If for a particular valuation v the selected option in M was in K ′ , then the same option will be selected in M ′ . This happens with probability at least 1 -ε . In any other case, anything can happen, but the revenue cannot worsen by more than the maximum possible valuation, which is ∥ v ∥ 1 ≤ 1 . Therefore, ❘❡✈ ( M ) -❘❡✈ ( M ′ ) ≤ ε · 1 = ε , completing the proof.

Theorem 10. If two menus M 1 and M 2 for the RochetNet have size at least ⌈ 4 ϵ 2 ⌉ 2 n , then they are ε -connected. Moreover, the curve transforming M 1 into M 2 is piecewise linear with only five pieces.

Proof Sketch. The full proof can be found in Appendix A.2. The intuition behind this theorem is that if menus are large, then they should contain many redundant options. Indeed, as in the previous theorem, the strategy is as follows. We show that every menu M of size at least ⌈ 4 ϵ 2 ⌉ 2 n can be linearly transformed into a 0 -reducible menu ˜ M such that each convex combination of M and ˜ M achieves a revenue of at least ❘❡✈ ( M ) -ε . This transformation converting M 1 and M 2 to ˜ M 1 and ˜ M 2 , respectively, yields the first and the fifth of the linear pieces transforming M 1 to M 2 . Together with Proposition 5 applied to ˜ M 1 and ˜ M 2 serving as the second to fourth linear piece, the theorem then follows.

However, this time, the linear transformation of M to ˜ M is much more intricate than in the previous theorem. To do so, it is not sufficient to only adapt the prices. Instead, we also change the allocations of the menu options by rounding them to discretized values. This technique is inspired by Dughmi et al. [2014], but non-trivially adapted to our setting. Since the rounding may also modify the active option for each valuation, we have to carefully adapt the prices in order to make sure that for each valuation, the newly selected option is not significantly worse than the originally selected one. Finally, this property has to be proven not only for ˜ M , but for every convex combination of M and ˜ M .

After the above rounding, the number of possible allocations for any option is bounded by ⌈ 4 ϵ 2 ⌉ n . Out of several options with the same allocation, the buyer would always choose the cheapest one, implying that the resulting menu ˜ M is 0 -reducible.

4 Mode Connectivity for the Affine Maximizer Auctions

Throughout this section, we focus on AMAs with fixed weights w i = 1 for all buyers i . Similarly to RochetNet , we have the following definition for AMAs.

Definition 11. A menu M with K + 1 options is ε -reducible if and only if there exists a subset K ′ ⊆ K , 0 ∈ K ′ , |K ′ | ≤ √ K +1 such that, with probability at least 1 -ε m over the distribution of the valuation of the buyers, (i) k ( v -i ) ∈ K ′ for any buyer i ; and (ii) k ( v ) ∈ K ′ .

Such phenomena are observed in the experiments in [Curry et al., 2022b, Section 6.3].

Our two main results, namely that two ε -reducible menus are always ε -connected and two large menus are always ε -connected, are based on the following proposition, in which we show that 0 -reducibility implies 0 -connectivity.

Proposition 12. If two menus M 1 and M 2 are 0 -reducible, then they are 0 -connected. Moreover, the curve transforming M 1 into M 2 is piecewise linear with only three pieces.

The proof idea is similar to the proof of Proposition 5 in RochetNet , but requires additional arguments due to the more intricate price structure (see Appendix D.1 for more details). Based on this proposition, now, we are able to show our two main results. First, we achieve ε -connectivity from ε -reducibility.

Theorem 13. If two AMAs M 1 and M 2 are ε -reducible, then they are ε -mode-connected. Moreover, the curve transforming M 1 to M 2 is piecewise linear with only five pieces.

Before the proof, we recall how the total payment is calculated for a valuation profile v . We choose k ( v ) as the option which maximizes the boosted welfare, ∑ i v ⊤ i x ( k ) i + β ( k ) . According to (1), the total revenue can be written as

̸

Proof of Theorem 13. Similar to the proof of Theorem 9, it is sufficient to show that every ε -reducible menu M can be linearly transformed into a 0 -reducible menu ˜ M such that each convex combination of M and ˜ M achieves a revenue of at least ❘❡✈ ( M ) -ε . This can then be used as the first and fifth linear piece of the curve connecting M 1 and M 2 , while the middle three pieces are provided by Proposition 12.

We construct ˜ M by (i) keeping all options in K ′ unchanged; (ii) for the options k ∈ K \ K ′ , we decrease β ( k ) to be smaller than -m , which implies such an option will never be selected (recall that 0 ∈ K ′ is assumed, and the option ( 0 , 0) is better than any such option). Consequently, ˜ M is 0 -reducible.

To complete the proof, let us look at the revenue of M ′ = { ( x ′ ( k ) , β ′ ( k ) ) } k ∈K , which is a convex combination of M and ˜ M : M ′ = λM + (1 -λ ) ˜ M for 0 ≤ λ < 1 . Let k ′ ( v ) = arg max k ∑ i v ⊤ i x ′ ( k ) i + β ′ ( k ) . As we decrease β ( k ) for k / ∈ K ′ , k ( v ) ∈ K ′ implies k ′ ( v ) ∈ K ′ and, additionally, option k ′ ( v ) and option k ( v ) achieve the same boosted welfare and same β . Therefore, since M is ε -reducible, with probability at least 1 -ε m , the boosted welfare of v as well as the boosted welfare of v -i for all buyers i is the same for M and for M ′ . According to the formula (2), the total payment for the profile v is the same for M and M ′ . Therefore, the loss on the revenue can only appear with probability at most ε m , and the maximum loss is at most m , which implies an ε loss in total.

Second, we show that mode connectivity also holds for those AMAs with large menu sizes, namely for K +1 ≥ ⌈ 16 m 3 ϵ 2 ⌉ 2 nm .

Theorem 14. For any 0 < ϵ ≤ 1 4 , if two AMAs M 1 and M 2 have at least K + 1 ≥ ⌈ 16 m 3 ϵ 2 ⌉ 2 nm options, then they are ε -mode-connected. Moreover, the curve transforming M 1 to M 2 is piecewise linear with only five pieces.

Proof Sketch. The full proof can be found in Appendix D.2. Similar to RochetNet , the idea of proving Theorem 14 is to discretize the allocations in the menu, then one can use Proposition 12 to construct the low-loss transformation from M 1 to M 2 by five linear pieces. To do this, one wants the loss of revenue to be small during the discretization. Consider the formula (2) of the total payment. The first two terms do not change much by a small change of the discretization. However, the last term β ( k ( v )) might be significantly affected by discretization, which may cause a notable decrease in the total payment. To avoid this, we perform a proportional discount on β , incentivizing the auctioneer to choose an allocation with a small β . By this approach, the original revenue will be approximately maintained. Furthermore, we show a linear path, connecting the original menu and the menu after discretizing, which will suffer a small loss.

5 Conclusion

We have given theoretical evidence of mode-connectivity in neural networks designed to learn auction mechanisms. Our results show that, for a sufficiently wide hidden layer, ε -mode-connectivity holds in the strongest possible sense. Perhaps more practically, we have shown ε -mode-connectivity under ε -reducibility, i.e., the assumption that there is a sufficiently small subset of neurons that preserve most of the revenue. There is evidence for this assumption in previous work in differentiable economics. A systematic experimental study that verifies this assumption under various distributions and network sizes is left for future work.

Our results make a first step in providing theoretical arguments underlying the success of neural networks in mechanism design. Our focus was on some of the most basic architectures. A natural next step is to extend the arguments for AMA networks with variable weights w i . Such a result will need to analyze a four layer network, and thus could make headway into understanding the behaviour of deep networks. Besides RochetNet , Dütting et al. [2019] also proposed RegretNet , based on minimising a regret objective. This network is also applicable to multiple buyers, but only provides approximate incentive compatibility, and has been extended in subsequent work, e.g., Feng et al. [2018], Golowich et al. [2018], Duan et al. [2022]. The architecture is however quite different from RochetNet : it involves two deep neural networks in conjunction, an allocation and a payment network, and uses expected ex post regret as the loss function. We therefore expect a mode-connectivity analysis for RegretNet to require a considerable extension of the techniques used by us. We believe that such an analysis would be a significant next step in the theoretical analysis of neural networks in differentiable economics.

Acknowledgments and Disclosure of Funding

All three authors gratefully acknowledge support by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (for all three authors via grant agreement ScaleOpt-757481; for Christoph Hertrich additionally via grant agreement ForEFront-615640). Yixin Tao also acknowledges the Grant 2023110522 from SUFE.

References

Allen-Zhu, Y. Li, and Z. Song. A convergence theory for deep learning via over-parameterization. In International Conference on Machine Learning , pages 242-252. PMLR, 2019.

Arora, R. Ge, B. Neyshabur, and Y. Zhang. Stronger generalization bounds for deep nets via a compression approach. In International Conference on Machine Learning , pages 254-263. PMLR, 2018.

Babaioff, Y. A. Gonczarowski, and N. Nisan. The menu-size complexity of revenue approximation. Games and Economic Behavior , 134:281-307, 2022.

Bhalgat, S. Gollapudi, and K. Munagala. Optimal auctions via the multiplicative weight method. In Proceedings of the fourteenth ACM Conference on Electronic Commerce , pages 73-90, 2013.

Cai, C. Daskalakis, and S. M. Weinberg. An algorithmic characterization of multi-dimensional mechanisms. In Proceedings of the forty-fourth Annual ACM Symposium on Theory of Computing , pages 459-478, 2012a.

Cai, C. Daskalakis, and S. M. Weinberg. Optimal multi-dimensional mechanism design: Reducing revenue to welfare maximization. In IEEE 53rd Annual Symposium on Foundations of Computer Science , pages 130-139. IEEE, 2012b.

Cai, C. Daskalakis, and S. M. Weinberg. Understanding incentives: Mechanism design becomes algorithm design. In IEEE 54th Annual Symposium on Foundations of Computer Science , pages 618-627. IEEE, 2013.

H. Clarke. Multipart pricing of public goods. Public Choice , pages 17-33, 1971.

Conitzer and T. Sandholm. Complexity of mechanism design. In Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence , pages 103-110, 2002.

Conitzer and T. Sandholm. Self-interested automated mechanism design and implications for optimal combinatorial auctions. In Proceedings of the 5th ACM Conference on Electronic Commerce , pages 132-141, 2004.

Curry, U. Lyi, T. Goldstein, and J. Dickerson. Learning revenue-maximizing auctions with differentiable matching. In International Conference on Artificial Intelligence and Statistics , pages 6062-6073. PMLR, 2022a.

Curry, T. Sandholm, and J. Dickerson. Differentiable economics for randomized affine maximizer auctions. arXiv preprint arXiv:2202.02872 , 2022b.

Daskalakis. Multi-item auctions defying intuition? ACM SIGecom Exchanges , 14(1):41-75, 2015.

Daskalakis, A. Deckelbaum, and C. Tzamos. Mechanism design via optimal transport. In Proceedings of the fourteenth ACM Conference on Electronic Commerce , pages 269-286, 2013.

Daskalakis, A. Deckelbaum, and C. Tzamos. The complexity of optimal mechanism design. In Proceedings of the twenty-fifth annual ACM-SIAM Symposium on Discrete Algorithms , pages 1302-1318. SIAM, 2014.

Daskalakis, A. Deckelbaum, and C. Tzamos. Strong duality for a multiple-good monopolist. In Proceedings of the Sixteenth ACM Conference on Economics and Computation , pages 449-450, 2015.

Draxler, K. Veschgini, M. Salmhofer, and F. Hamprecht. Essentially no barriers in neural network energy landscape. In International Conference on Machine Learning , pages 1309-1318. PMLR, 2018.

Du, J. Lee, H. Li, L. Wang, and X. Zhai. Gradient descent finds global minima of deep neural networks. In International Conference on Machine Learning , pages 1675-1685. PMLR, 2019.

Duan, J. Tang, Y. Yin, Z. Feng, X. Yan, M. Zaheer, and X. Deng. A context-integrated transformerbased neural network for auction design. In International Conference on Machine Learning , pages 5609-5626. PMLR, 2022.

Dughmi, L. Han, and N. Nisan. Sampling and representation complexity of revenue maximization. In Web and Internet Economics: 10th International Conference, WINE 2014 , pages 277-291. Springer, 2014.

Dütting, Z. Feng, H. Narasimhan, D. Parkes, and S. S. Ravindranath. Optimal auctions through deep learning. In International Conference on Machine Learning , pages 1706-1715. PMLR, 2019.

Feng, H. Narasimhan, and D. C. Parkes. Deep learning for revenue-optimal auctions with budgets. In Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems , pages 354-362, 2018.

Garipov, P. Izmailov, D. Podoprikhin, D. P. Vetrov, and A. G. Wilson. Loss surfaces, mode connectivity, and fast ensembling of DNNs. Advances in Neural Information Processing Systems , 31, 2018.

Giannakopoulos and E. Koutsoupias. Duality and optimality of auctions for uniform distributions. In Proceedings of the fifteenth ACM Conference on Economics and Computation , pages 259-276, 2014.

Giannakopoulos and E. Koutsoupias. Duality and optimality of auctions for uniform distributions. SIAM Journal on Computing , 47(1):121-165, 2018.

Golowich, H. Narasimhan, and D. C. Parkes. Deep learning for multi-facility location mechanism design. In International Joint Conference on Artificial Intelligence , pages 261-267, 2018.

Groves. Incentives in teams. Econometrica: Journal of the Econometric Society , pages 617-631, 1973.

Kuditipudi, X. Wang, H. Lee, Y. Zhang, Z. Li, W. Hu, R. Ge, and S. Arora. Explaining landscape connectivity of low-cost solutions for multilayer nets. Advances in Neural Information Processing Systems , 32, 2019.

Kuo, A. Ostuni, E. Horishny, M. J. Curry, S. Dooley, P.-y. Chiang, T. Goldstein, and J. P. Dickerson. Proportionnet: Balancing fairness and revenue for auction design with deep learning. arXiv preprint arXiv:2010.06398 , 2020.

M. Manelli and D. R. Vincent. Multidimensional mechanism design: Revenue maximization and the multiple-good monopoly. Journal of Economic Theory , 137(1):153-185, 2007.

B. Myerson. Optimal auction design. Mathematics of Operations Research , 6(1):58-73, 1981.

Nguyen. On connected sublevel sets in deep learning. In International Conference on Machine Learning , pages 4790-4799. PMLR, 2019.

J.-C. Rochet. A necessary and sufficient condition for rationalizability in a quasi-linear context. Journal of Mathematical Economics , 16(2):191-200, 1987.

J.-C. Rochet and L. A. Stole. The economics of multidimensional screening. Econometric Society Monographs , 35:150-197, 2003.

Sandholm and A. Likhodedov. Automated design of revenue-maximizing combinatorial auctions. Operations Research , 63(5):1000-1025, 2015.

Shen, P. Tang, and S. Zuo. Automated mechanism design via neural networks. In Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems , pages 215-223, 2019.

Shevchenko and M. Mondelli. Landscape connectivity and dropout stability of sgd solutions for over-parameterized neural networks. In International Conference on Machine Learning , pages 8773-8784. PMLR, 2020.

Vickrey. Counterspeculation, auctions, and competitive sealed tenders. The Journal of Finance , 16(1):8-37, 1961.

Zheng, A. Trott, S. Srinivasa, D. C. Parkes, and R. Socher. The AI economist: Taxation policy design via two-level deep multiagent reinforcement learning. Science Advances , 8(18), 2022.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper seeks to provide a theoretical foundation for empirical results demonstrating that optimal auctions (both known and novel) can be discovered by differentiable auction theory.  It proves that two such auction formats satisfy a condition called ‘mode connectivity’: epsilon-mode connectivity is satisfied when two (local) solutions are connected by paths in parameter space.\n\nSTRENGTHS:\n**Originality**: to my knowledge, this work is novel.\n\n**Quality**: this seems well executed.\n\n**Clarity**: well-written.\n\n**Significance**: the overall significance of the paper is hard to assess.  Insofar as it provides a theoretical basis for existing empirical results, I feel that the bar for significance is cleared.\n\nWEAKNESSES:\nMy main concern about the paper is that it identifies a perplexing regularity - that both the RochetNet and AMA results require \"five pieces\" - but then does not seek to understand that.\n\nMinor:\n - p.1: DSIC printed as DISC\n - l.79: “straight-jacket” or “strait-jacket”?\n- l.135: “each in unit supply” clearer\n - l.166: whenever I see “it is … easy to see”, I want to see at least a hint of explanation: I’ve tried to cover over results that I suspected to be true (but later discovered weren’t) this way.\n\nQUESTIONS:\n- Dominant strategy incentive compatibility is a strong concept.  I would appreciate a few sentences on what that buys them relative to e.g. Bayesian-Nash incentive compatibility.\n - l.66: re: the link between epsilon-dropout stability and epsilon-mode connectivity, is it useful to think in terms of a Lipschitz constant?\n - l.92 mentions an example with 10,000 neurons and 59 active options: can this be expressed in terms of K?\n - lines 126-127: why do the menu options define a “piecewise linear surface of utilities”?\n - My overall intuition from this is that large neural networks can overfit: a large number of neurons could, further, yield inactive options, which (as they are dominated?) can be thrown away without loss.  Is something like this correct?  If so, it would be worth emphasizing that.\n - My guess is that we need no more than one ‘contract’ per type of bidder: is this correct?  If so, it should be mentioned (if it hasn’t been).\n - To me, the most curious result is the “five pieces” element of some of the RochetNet and AMA theorems: what explains this?  How much more general is it?  (Thus, could the results of S3, S4 be presented as special cases?)  Can anything more specific be said about the pieces?\n\nLIMITATIONS:\nNone.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the mode connectivity for specific neural network architecture, i.e., RochetNet and Affine Maximizer Auctions(AMA), where the local optimal solutions produce close local optimal solutions. To be specific, \n- for linear utilities, $\\epsilon$-reducible solutions implies $\\epsilon$- mode connectivity.\n- for $n$ items and linear utilities, for large menu options $K$, then $\\epsilon$-mode connectivity holds between any solutions for any distribution. \n- Similar results also holds for AMA.\n\nSTRENGTHS:\n- The problem studied is a heated topic, and is a really interesting interdisciplinary area between auction design and neural network design. \n- The result is quite novel, and establish some difference against existing work on mode connectivity.\n\nWEAKNESSES:\nThe network architecture difference of existing work and the neural network studied in this paper needs further clarification, and more emphasis. \n\n\nThe current presentation of this paper needs a major revision, especially on our contributions sections. To name a few:\n- There are some concepts that refer to later section of the paper, and the reviewer strongly suggests moving these definitions (or create an informal version) to the introduction section for readability, e.g., $\\epsilon$-reducible appeared for line 85, 89, 110, affine maximizer for line 103. \n- line 73-83 doesn't include the results for this paper, but rather a brief intro of RochetNet. \n- For reviewers unfamiliar with the word \"options\", it's really hard to understand the sentence in line 75. \"single hidden layer where the neurons directly correspond to menu options\".\n- The reviewer strongly suggests to put figure 1 in the intro section, and adding more details to the figure would helps a lot of the understanding of the paper. \n- In section 1.1, many results/theorems are mixed with existing results, which is quite confusing. \n- The definition of mode connectivity doesn't appear until page 6 of the paper...\n\nSome other minor comments:\n- line 21, please replace \"DISC\" by \"DSIC\". \n- line 79, \"straight-jacket auction\" should be supported by citations.\n\nQUESTIONS:\n- What's the definition of the word \"option\" studied in this paper?\n- Does this $\\epsilon$ scale with the upper bound of the distribution, of the distribution is not in $[0,1]$, but in $[0,H]?\n\nLIMITATIONS:\nThis paper is theoretical, hence there are no experiment results.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe starting point for the paper is recent work in the area of “differentiable economics”, in which high-revenue strategyproof auction mechanisms are found by optimizing parameters using machine-learning-inspired gradient descent techniques.\n\nThe authors consider the problem of selling multiple goods to a single buyer. In such setting, strategyproof mechanisms can always be identified with a “menu” of (allocation, price) pairs — the bidder chooses their best choice from the menu, and carefully optimizing the menu can increase revenue. The neural architecture that represents these menu items is called “RochetNet”. They also consider multi-buyer versions of this problem, in particular searching through the space of affine maximizer auctions (AMA), which are structurally similar (there is a “menu” of possible outcomes, and a “boost” which roughly plays the role of the price, although actual per-bidder payments are calculated according to a VCG-style rule).\n\nIn these classes of auctions, the performance goal (revenue) is a very non-convex function of the auction parameters. Nevertheless, first-order optimization seems to work well in finding good or even known-optimal mechanisms. Also, empirically, allowing the auctions to learn over thousands of menu items helped performance, even though at the end of training, only a handful of these menu items were actually used (and in some cases known-optimal mechanisms might only have 3-4 menu items). The authors of this paper aim to explain these phenomena.\n\nThey build on existing work in deep learning for more standard tasks, where things work similarly: even though loss landscapes are not convex, it is possible to optimize over them using first-order methods, and overparameterizing the neural networks seems to help with this. One recent direction of theoretical work aims to explain these phenomena using the concept of “mode connectivity” — there are various results showing that if it is possible to remove lots of neurons from a trained network and get almost the same performance, then any two such solutions must be connected by a continuous path where at any point along the path, the loss function is within some constant of the two solutions. Presumably, loss landscapes with such a property should be easier to optimize over.\n\nThe authors establish similar properties for the two types of auction architectures. In particular, between any two menus where there is a small subset of options that would be chosen by a bidder with probability 1-\\epsilon, embedded in a much larger unused or redundant menu, they show that (epsilon) mode connectivity holds. Additionally, any pair of menus with sufficiently large (as a function of epsilon and the number of items) menus is also epsilon-mode connected. Analogous results hold for AMAs.This provides a nice theoretical explanation of why these auctions are unexpectedly easy to optimize successfully, and why using large menus may help optimization.\n\nSTRENGTHS:\nTwo separate papers in this area observed an interesting and useful empirical phenomenon but gave no good explanation for it. This paper gives a very solid explanation, and is extremely interesting for that reason. It also will hopefully motivate better designs and learning techniques for strategyproof auction architectures.\n\nWEAKNESSES:\nCurrently I don’t see any significant weaknesses.\n\nQUESTIONS:\nI recommend further discussion of Shen et al. “Automated mechanism design via neural networks”, which presents an architecture that for certain cases is equivalent to RochetNet. It’s cited at one point but the actual neural architecture merits discussion (it also encodes menu items).\n\nIs there an explicit explanation of exactly why mode connectivity helps optimization? Maybe it’s out there in the literature already. If so I think it’s worth devoting a few sentences to this.\n\nLIMITATIONS:\nLimitations are adequately addressed — in particular, that these techniques probably won’t extend to even more complicated/flexible auction architectures — they rely on the relatively simple “menu based” or “list of allocations + boost” structure of the particular architectures in question.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}

comments: SUMMARY:
This paper studies the mode connectivity in neural networks for design auction mechanisms. Specifically, the authors prove that locally optimal solutions are connected by a simple, piecewise linear path such that every solution on the path is almost as good as one of the two local optima.

STRENGTHS:
The paper proposes to address a significant problem: how to design an auction mechanism via machine learning.

The proof is given in detail and looks all correct.

The presentation is clear and well-structured.

WEAKNESSES:
The novelty and significance are not clear. Is the mode connectivity in auction mechanism just a special case of the mode connectivity of neural networks in general? Following this intuition, the proof seems straightforward:

The functional from a neural network to its corresponding revenue is a bounded (continuous) functional; the functional from this neural network to its training loss is also a bounded functional. Thus, it is a continuous function from training loss to revenue. Therefore, the mode connectivity of neural networks in term of the training loss implies the mode connectivity of auction mechanism in term of revenue. Some may argue that these bounded/continuous properties may not always hold, but given the neural networks are usually trained by SGD or its variants, these properties should stand. 

------

POST-REBUTTAL: my concerns are addressed.

QUESTIONS:
Please address the weakness above. I would like to increase the score if it can be cleared.

LIMITATIONS:
Please see the weaknesses above.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-08

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on justifying the empirical success of this differentiable economics, particularly in the context of menu-based methods like RochetNet and Differentiable AMA auctions.\n\nThe authors introduce the concept of the $\\epsilon$-mode connectivity property, which establishes that two locally optimal menus are connected by a simple path where the revenue loss along the path is at most $\\epsilon$. The paper demonstrates that this property holds true under two conditions:\n- If the valuations are normalized, and the menus are $\\epsilon$-reducible, meaning that there exists a small subset of menus that are active for a buyer with a probability of $1-\\epsilon$.\n- If the number of menu options is sufficiently large.\n\nSTRENGTHS:\n**Significance:** Although menu-based approaches for differentiable economics show promising empirical performance, there is a lack of understanding regarding their theoretical properties. This paper takes the first stride towards exploring these theoretical aspects. \n\n**Originality:**  While mode-connectivity has primarily been studied in the context of prediction problems involving convex losses with linear transformations in the final layer, RochetNet and Differentiable AMA minimize negated revenue loss, which involves more complex calculations. This fundamental difference sets them apart in their analysis.\n\n**Clarity and Quality:** The paper is well written and easy to follow.\n\nWEAKNESSES:\n- This paper would benefit a lot from a discussion on how exactly mode connectivity can be used to justify empirical performance. While I agree that this is a cool theoretical property, it is very unclear to me how this exactly would help\n- There is also no discussion/comments what practitioners should make of this property at all - (are there any insights on how to initialize these networks better to break permutation symmetry of menus etc or anything that would help improve training)\n- The optimal allocations are not necessarily finite (this assumption isn't state clearly wherever used). See Example 3 in [*]\n\n[*] Daskalakis, C., Deckelbaum, A., and Tzamos, C. (2017). Strong duality for a multiple-good monopolist. Econometrica, 85:735–767.\n\nQUESTIONS:\nPlease see Weaknesses.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-25"}