PreferenceNet: Encoding Human Preferences in Auction Design with Deep Learning
Authors: Neehar Peri
Michael Curry
Samuel Dooley
John P Dickerson
Venue: NeurIPS 2021 Poster (2021)
Keywords: Auction Design
Differentiable Economics
Human in the Loop
Fairness
Diversity

Abstract: The design of optimal auctions is a problem of interest in economics, game theory and computer science. Despite decades of effort, strategyproof, revenue-maximizing auction designs are still not known outside of restricted settings. However, recent methods using deep learning have shown some success in approximating optimal auctions, recovering several known solutions and outperforming strong baselines when optimal auctions are not known. In addition to maximizing revenue, auction mechanisms may also seek to encourage socially desirable constraints such as allocation fairness or diversity. However, these philosophical notions neither have standardization nor do they have widely accepted formal definitions. In this paper, we propose PreferenceNet, an extension of existing neural-network-based auction mechanisms to encode constraints using (potentially human-provided) exemplars of desirable allocations. In addition, we introduce a new metric to evaluate an auction allocations' adherence to such socially desirable constraints and demonstrate that our proposed method is competitive with current state-of-the-art neural-network based auction designs. We validate our approach through human subject research and show that we are able to effectively capture real human preferences.

PreferenceNet: Encoding Human Preferences in Auction Design with Deep Learning
Neehar Peri*, Michael Curry*, Samuel Dooley, John P. Dickerson
Center for Machine Learning, University of Maryland peri@umiacs umd. edu, (curry, sdooleyl, john) 0cs. und edu
Abstract
The design of optimal auctions is problem of interest in economies, game the- ory, and computer science. Despite decades of effort, strategyproof, revenue- maximizing auction designs are still not known outside of restricted settings. However, recent methods using deep learning have shown some success in approxi- mating optimal auctions, recovering several known solutions and outperforming strong baselines when optimal auctions are not known. In addition to maximiz- ing revenue, auction mechanisms may also seek encourage socially desirable constraints such as allocation fairness or diversity. However, these philosophical notions neither have standardization nor do they have widely accepted formal definitions. In this paper, we propose PreferenceNet, an extension of existing network- based auction mechanisms to encode constraints using (potentially numan-providee exemplars of desirable allocations. In addition, introduce new metric to evaluate an auction allocations' adherence to such socially desirable constraints and demonstrate that proposed method is competitive with current state-of-the the-art art through human subject research and show that are able effectively capture real human preferences. Our code available on GitHub
based auction designs. We validate our approach
1 Introduction
Auctions are an essential tool in many marketplaces, including those in electricity 8 advertising [14), and telecommunications
26 The design of auctions with desirable properties is thus an important theoretical and practical problem. A typical assumption is that bidders may choose to bid strategically and will successfully anticipate the behavior of other bidders. This results in potentially complicated Bayes-Nash equilibrium which may be difficult to predict. To evade this problem, common requirement that an auction should be strategyproof (i.e. bidders should incentivized to truthfully share their valuations regardless of other bidders' actions).
If the goal of auction is maximize the total welfare of all participants. the Fickrey-Clarke. Groves (VCG) mechanism is both strategyproof and welfare-maximizing [40] 7 20) Intuitively, in many cases the VCG mechanism corresponds to second-price auction. If the goal is maximize the revenue gained. the problem is more challenging. Myerson's 's work completely characterizes strategyproof, aveeuu-maximiziing auctions for single item [29]. Beyond this case, results are more limited. Some results are known for the "multiple-god" monopolist" problem, in which the auctioneer sells multiple items to single bidder [IT] 12 24 31
28 21 18]. addition, designing auctions for the related but weaker notion of Bayes-Nash incentive compatibility is reasonably well understood
There has been significant difficulty in designing strategyproof auctions involving multiple items and multiple bidders. 41 shares significant, but limited results which solve the case in which items
* The first authors contributed equally this work. 35th Conference Neural Information Pracessing Systems (NcurIPS 2021).

---

may have at most 2 values. Due to the apparent difficulty of analytically designing strategyproof, revenue maximizing auctions, recent methods instead approximate optimal auctions using machine
[37] proposes an method which guarantees exact strategyproofness in the single-agent setting. (39 also use neural networks the design of
learning approaches [3] T0 19 25
13
33
32
auctions. These methods primarily focus on standard mechanism design goals of revenue or welfare maximization, but these may not be the only goals of an auction. An auction might be conducted for public goods (i.e. electromagnetic spectrum distribution (9), where the auctioneer must consider the effect not only on the auctioneer and bidders but on third-parties as well [30]. In other cases, such as auctions involving job or credit advertisements, it might be necessary to additionally constrain the auction mechanism to ensure fairness with respect to protected characteristics [23] 6]. Recent work
has considered the problem of determining revenue- maximizing, strategyproof, fair auctions, either from a specific class 5 or with a general neural network approach [25]
The underlying notions of "fairness" used in these papers (e.g. total-variation fairness [23] are defensible but somewhat arbitrary - they are attempts formally capture human intuition. It might instead be better to attempt to capture, from data, human intuitions about fairesss or other ethical requirements that are not explicitly defined 38 16
In this paper, we introduce PreferenceNet, an extension to RegretNet that directly encodes socially de- sirable constraints from data, and captures noisy human preferences. We are motivated by advertising auctions where the allocations of the auction mechanism must satisfy human preference constraints alongside the typical goals of strategyproofness and revenue maximization. We conduct a number of experiments using synthetic data (as is typical for neural network based auction mechanisms [3) and evaluate our method on different auction settings and fairness constraints. We show that PreferenceNet is able to effectively capture each fairness constraint and matches the performance of standard approaches. We also conduct two surveys to further study human preferences. In the first survey (n=140), when given a specific definition of fairness, we ask participants to determine a given auction setting is fair in order to test the noise in human judgements. In the second survey (n=345), we elicit judgements of pairwise comparisons of two auction settings to determine preferences without priming the participants with any particular definition of fairesss. We train PreferenceNet these data and show that our approach can capture nuanced human preferences in auction design.
2 Background and Related Work
presents RegretNet, neural-network architecture for learning approximately strategyproof auctions that maximize revenue. RegretNet treats the auction mechanism as a function from bids to allocations and payments, parameterized neural network. Revenue is optimized via gradient descent on sampled truthful bid profiles; strategyproofness is enforced by computing strategic bids in an adversarial manner minimize violations. RegretNet has been modified and extended in a variety of ways, particularly to enforce additional desirable constraints. 10
RegretNet.
13
19 25
15 33
Special Case: Single-Bidder Auctions. We highlight a special case in which deep learning for auctions has been particularly successful auctioos with a single bidder. In the single-bidder setting, the set of strategyproof mechanisms can be easily characterized (34). and it is possible design neural network architectures which will always lie in this set. [13 37 present two different learning- based solutions. Both methods are guaranteed to strategyproof, and revenue can be maximized by unconstrained optimization. There are some known optimal single-agent mechanisms (we highlight those of Manelli- Vinceet (28) and Pavlov (31]. as they are relevant to auction settings we study below). Moreover, the theory of single- migle-bidder mechanisms is well understood [II] 12 18]. so it is also possible learn empirically strong mechanisms using these approaches and then prove their optimality 337 13]. Unfortunately, when moving beyond the single-agent setting, it is necessary to use more general neural network architectures for which these guarantees do not apply. Because we are interested in such settings, we focus on these general architectures in this work.
Fairness and Human Preferences. As mentioned, while revenue, strategyproofness, and individ- ual rationality are the classic goals of auction design, might also be necessary for allocations made by an auction to satisfy certain other requirements such as fairness [5] (2). However, it is not always

---

clear if these mathematical definitions of these concepts actually capture human intuitions.
36
38] considers human reactions different definitions of faimess in a classification setting. [35
tests the extent to which human participants are able to understand and apply fairness metrics. [T6]
considers the problem of learning to perform fair clustering from human-proo aomann-rovvded demonstrations.
As discussed below, we also crowdsource human opinions on fairesss in auctions and analyze the results in Section 6
3 Problem Setting
Auction Model. An auction is defined as set of agents N = -l......1 .n] bidding for items M = fl........ m.. Each agent N has a corresponding private valuation Vú randomly drawn from set of n valuations as (v1)...um, E V. In general may be functions over the power set of items 2MM However, we only consider simpler cases with additive valuations and unit-demand valuations, where the valuation is simply vector Us E Rh' of values per item.
Each agent reports a bid vector 6ç to the auctioneer, which may differ from the private valuation e,- Given the profile of bids 6 = (b)...., by) of all agents, the auction has allocation and payment rules g(b) Rm [0, 1] 1m and p(b)= Rm R We will refer the matrix of allocation probabilities, whose rows must sum to 1, as g = Likewise agent i's value of the jth item is Vi.j, and bidder i has payment function pt. Moreover, for unit-demand auctions, we restrict the allocation to allow each bidder to win, in expectation, at most I item. Given the allocation, each agent receives a utility which can in either case be represented in linear form as us (1) Ej U1. Fi.j Pi-
Desirable Auction Properties. A mechanism is individually rational (IR) when an agent is guaran- teed non-negative utility: u.(u; o) 2 0 vi N, V. A mechanism is Nominant-strategg incentive- compatible (DSIC) or strategyproof if every agent maximizes their own utility by bidding truthfully, regardless of the other agents' bids. We can define regret, the difference in utility between the bid player i actually made and the best possible strategic bid: rgt,(v) max,, u, (b; v_s) u, (#t, v-:). In addition to satisfying the IR and DSIC constraints, the auctioneer seeks maximize their expected revenue, If the auction is truly DSIC, players will bid truthfully. and as result revenue is simply Ev 1EieN (v()).
4 PreferenceNet: Encoding Human Preferences
We first explore a new metric to evaluate the adherence to socially desirable constraints in item allo- cations. Next, we describe the implementation details of PreferenceNet and important considerations when training the model.
Evaluation Metrics. There are limited evaluation criteria that quantitatively measures an auction mechanism's 's ability enforce constraints on item allocations. If the constraints are not known explicitly, one can qualitatively examine the allocation graphs to evaluate the underlying allocation function (b). However, visual inspection does not scale to larger auction settings. As shown in Figure I our proposed metric is not only able to capture the same insights as qualitative analysis, but also scales to arbitrarily large auction mechanisms.
Given the limitations of existing analysis techniques, we propose Preference Classification Accuracy (PCA), new metric to evaluate how well a learned auction model satisfies an arbitrary constraint. For a set of test bids b and allocations g(b): : mm - [0, 1] nm generated by our learned auction model, we assign a label s66) E {1. 0} to each allocation according to a ground truth labeling function based on the underlying preference. For each test bid 6, s(b) is 1 if the learned auction network satisfies the ground truth constraint. PCA is calculated by averaging the value of s(b) over n test bids. Note that this metric remains valid in cases when we know the underlying preference function (e.g. total variation fairness, entropy) well as when we are sampling from an unknown distribution (e.g. human preference elicitation). For cases where the underlying preference function is known, we can directly compute the preference score for given allocation and apply a threshold obtain a label. For cases where the preference function is unknown, as is the case in human preference elicitation, we can use the ground truth allocation-labe pair to assign preference labels s(b) to new allocations based on the nearest neighbor in the ground truth set. This metric gives formal way measure
3

---

RegretNet Total Variation Fairness
PreferenceNet Total Variation Fairness
(b)
(c)
(d)
Figure We compare the allocation plots of standard RegretNet approaches in enforeing total variation fairness (TVF) 25 23 (sub-figures a and c) with our proposed approach (sub-figures b and d) learned through exemplars of desirable allocations that satisfy TVF. Visually, the allocations for both the unit-demand Sub--iguree a and b) and additive auctions (sub-figures and d) are identical. In this case, our proposed metric verifies our visual inspection, indicating that allocations from all four models satisfy TVF with 100% accuracy. However, this qualitative analysis does not extend to larger auction settings. In contrast, our proposed metric allows us to quantify the adherence of an auction mechanism to an enforced constraint for arbitrarily large auctions.
the degree to which preference constraints are violated, which crucially can be used whether or not the constraints follow explicitly-knoww function.
Preference Elicitation. In order effectively elicit preferences, rely pairwise comparisons between allocations to identify both positive and negative exemplars. We compare each input set of of allocations against n other allocations determine if particular sample preferred over these alternatives (see Figure 2). This method of group preference elicitation reduces noise and ensures that majority of the participants. We this ranking approach
the learned preference satisfactory to generate training labels in all of our experiments as described below.
Input
(a)
0.65
0.66
0.64
0.67
0.61
0.68
(b)
0.68
0.64
0.65
0.67
0.67
0.69
Pairwisc Comparisons
Figure 2: To elicit preferences from group, from person without deterministic preferences, we use pairwise comparisons to determine labels for each allocation. In examples (a) and (b), each allocation (represented
circle) has preference score (represented by number inside the circle). We can compare the score of the input against n other valuations to determine the relative ranking of the input data point. If the input data point has a smaller score than a plurality of the points compared against, it is a negative exemplar for the implicit preference (as in (a)). Otherwise it is positive (as in (b)).

---

Training Algorithm. We present the training algorithm of PreferenceNet below. We use the same additive and unit-demand network architectures as RegretNett for arbitrary numbers of agents and items. Our training algorithm follows closely from RegretNet. PreferenceNet consists of two sub- networks: RegretNet and a 3-layer MLP with a sigmoid activation at the output. We first train the MLP using a uniformly drawn sample of allocations as inputs, and optimize the binary cross entropy loss function using ground truth labels (generated as in Figure 2 identifying positive and negative exemplars. Next, we train RegretNet using the standard training procedure with the modified loss function described in Eq. I Lastly, we sample allocations and payments from the partially trained RegretNet model every epoch and augment the MLP training set to adapt to the distributional shifts in allocations over the course of training. Our modified loss function Cp (w; >) is defined as:
Ligt
Σ VEN
A(ra)) rgt,(w) + Pr Σ
rgt; (w)² La =
Σ
8j
VEN
JEM
Co(w; -
1 +E>
Σ IEN
Pi (v(0) (+00+ Ligt La
where Lg is the output of the trained MLP. RegretNet is optimized such that it is strategyproof, revenue-mane maattuemaiiiiiing, satisfies the preference learned by the MLP (i.c. maximizes the output scores of the MLP).
For each configuration of agents and m items, we train RegretNet for a maximum of 200 epoch using 160,000 training samples. We also train the MLP with 80,000 initial training samples and iteratively retrain the MLP with 5,000 additional samples from the partially trained RegretNet every epoch. For all networks, we use the Adam optimizer and 100 hidden nodes per layer. We apply warm restarts the MI.P optimizer each time we add new training data to to prevent the model from settling in local minima. We incremented Pr every 2500 iterations and A, every 25 iterations. Finally, we report the preference classification accuracy, mean regret, and mean payments by simulating the allocations of 20,000 testing samples. We run all our experiments GPU. We refer readers to GitHub for our implementation.
NVIDIA Titan X (Pascal)
MLP Architecture. We learn implicit preference functions using a simple 3-layer multi-layer perceptron that takes as input a mini-batch of allocations and outputs a vector that scores the input [0, 1] as a measure of how closely the allocation satisfies the ground truth preference. Given that neural networks are universal function approximators, with enough parameters the MI. can represent any arbitrary preference function. In practice, we find that ReL.U non-linear activations and batch normalization are essential to effectively train this network.
Class Balanced Sampling. The distribution of positive and negative training examples for arbitrary preferences are often imbalanced - preferred examples may be concentrated in a small region of possible allocations. Often, this imbalance can make it difficult to train robust model. Commonly. neural networks with improper class balance fail learn a good decision boundary. In order to compensate for such imbalanced datasets, we can explicitly over-sample sparse classes until there are an equal number of positive and negative training examples. In practice this allows the network to quickly learn the decision boundary and allows us to train with fewer data points.
MLP Co- Training. The distribution of payments and allocations generated by RegretNet shift significantly during the course of training. As a result, the initially trained MLP may not effectively enforce the preference loss as RegretNet continues to train. In order to adapt to the distribution shift, we sample allocations from RegretNet at fixed intervals while it is training, add them back to the training set, and retrain the MLP. We generate noisy labels (27) using the existing MLP to reduce the cost of collecting expensive labels. In practice, we find that this approach effectively reinforces the decision boundary between positive and negative exemplars.
Model Validation and Selection. The optimal model minimizes regret, while maximizing payments and preference classification accuracy. In practice, satisfying all three conditions may be Often, we find that choosing a fixed epoch to evaluate does not provide consistent results. Rather, we evaluate each checkpoint on validation set and maximize the following criteria in Eq. 2
difficult.
p(b)
rg)
a * PCA + 3
max(p(()) 7 (1
max (b)) s.t. a - -B+7=1
(2)
5

---

In our experiments we set = 0.45, 3 = 0 0.45. is important to note that the maximum regret and payments are calculated over the entire training process, while the mean regret, payment, and preference classification accuracy are calculated at each epoch.
5 Sampling Synthetic Preferences
Given the lack of publicly available auction data, we generate synthetic bids as in [1] 25 37 We first validate PreferenceNet using synthetic preferences, and extend our analysis to real human preferences Section 6 In our synthetic preference experiments, we are interested in three types of fairness: total variation fairness (TVF), entropy, and a quota system. All three definitions of fairness map the allocations g()) onto R. We train auction models for each of these valuation function and compare RegretNet with our proposed model under uniform additive and unit-demand auction settings in 5 Note in Table5Jther RegretNet is trained with an additional loss function term to explicitly optimize for the preference. PreferenceNet is trained as described in 4 We describe the three definitions fairess below:
Total Variation Fairness. An auction mechanism satisfies total variation fairness if the di-dittncce between allocations for any two users is at most the distance between those users. That is, total variation fairnsss is satisfied when
Vk E (1, (1..... c). E M,
Ele(b), g(b) d (1D').
iECk
We fix in all of our experiments. We minimize violations of the above constraints.
Entropy. An auction mechanism satisfies the entropy constraint if the entropy of the normalized allocation for bid profile b
g(b),. Σ, g(b);s
g(b)... E,g(b)s
log
(4)
is maximized. We normalize across items, turning the allocation into probability distribution. This normalization ensures that entropy reflects diversity in the allocations, and not the overall number of items being allocated. Specifically, encouraging entropy ensures that the allocation for given agent will tend to be more uniformly distributed.
Quota. An auction mechanism satisfies the quota constraint if, for each item in the (normalized) allocation, the smallest allocation to any agent j is greater than some minimum threshold t:
g(b) E
min
>t
(5)
Here, we normalize the allocation so that the allocation per item will be probability distribution over agents. The intuition for this definition is that each agent must guarantee some floor across every item. Returning to the advertising example, this ensures some minimum percentage of ad impressions are seen by every demographic group.
We train a number of models to compare the performance between RegretNet and PreferenceNet. Despite leveraging an weaker, implicit signal to learn faimesss constraints, PreferenceNet is able to closely match the performance of RegretNet. Surprisingly, PreferenceNet improves upon RegretNet in some cases, indicating that with careful hyperparameter tuning, further improvements are possible. Of the three fairness constraints examined, enforcing quota is hardest for additive auctions, both RegretNet and PreferenceNet struggled for auctions with large number of agents.
Limitations. Despite its effectiveness, we highlight limitations of our approach. In general, Prefer- enceNet always optimizes for the simplest function. For example, if learning a piece-wise preference where the positive exemplars are not clustered in single region in Figure PreferenceNet tends to only satisfy part of the piece-wise function. This is unsurprising, given that neural networks are known to take shortcuts in optimization [7]. Moreover, PreferenceNet has difficulty generating allocations with tightly clustered preference scores to satisfy a particular constraint. Given that we optimize for preferences implicitly using exemplars, this behavior is understandable. We study these limitations further in the supplemental material.

---

Table 1: We evaluate both RegretNet and PreferenceNet over auctions ("u" refers unit-demand and "a" refers to additive), where n is the number of agents and m the number items. We measure three criteria (a) PCA, (b) Regret Mean (STD), (c) Payment Mean (STD). Although PreferenceNet learns each preference implicitly, it produces similar performance to our strong baseline.
TVF
Entropy
Quota
(a)
PreferenceNet NegretNet
RegretNet
PreferenceNet
RegretNet
PreferenceNet
2x2
100.0
100.0
86.4
69.6
100.0
100.0
2x4
100.0
100.0
99.7
94.9
100.0
100.0
4x2 u
100.0
100.0
100.0
94.5
100.0
4x4 u
100.0
99.3
100.0
67.4
100.0
100.0
2x2 a
100.0
100.0
99.6
99.5
75.1
100.0
2x4
100.0
100.0
100.0
100.0
36.0
100.0
4x2
100.0
100.0
99.9
100.0
.1
4x4
100.0
99.9
100.0
96.1
TVF
Entropy
Quota
(b)
RegretNet
PreferenceNet
RegretNet
PreferenceNet
RegretNet
PreferenceNet
2x2
.012 (.012)
.012 (.012)
(.02)
.011 (.01)
(.013)
.013 (.012)
2x4 u
.008 (.006)
.016 (.013)
045 (.045)
.022 (.019)
,012 (.01)
.034 (.022)
4x2 u
.015 (.009)
.028 (.013)
.025 (.025)
.056 (.018)
.026 (.017)
.819 (.136)
4x4
.033 (.024)
.067 (.037)
.031 (.031)
.029 (.014)
.037 (.023)
.432 (.145)
2x2
.005 (.004)
.006 (.004)
.006 (.006)
.013 (.008)
.008 (.007)
.05 (.031)
2x4
.008 (.011)
.008 (.009)
.007 (.007)
.008 (.009)
.01 (.01)
.078 (.032)
4x2
.038 (.076)
.014 (.011)
.036 (.036)
.122 (.052)
.017 (.013)
.017 (.013)
4x4 a
.424 (.324)
.139 (.104)
.015 (.015)
.257 (.1)
.038 (.017)
.039 (.018)
TVF
Entropy
Quota
(c)
RegretNet
PreferenceNet
RegretNet
PreferenceNet
RegretNet
PreferenceNet
2x2
4.18 (.45)
4.17 (.44)
4.26 (.37)
4.14 (.43)
4.19 (.35)
4.18 (.33)
2x4
4.37 (.29)
4.41 (.34)
4.48 (.34)
4.47 (.21)
4.44 (.16)
4.49 (.29)
4x2 u
4,93 (.21)
4.98 (.21)
4.85 (.23)
4.99 (.21)
5.16 (.24)
5.03 (.21)
4x4 u
8.81 (.39)
8.92 (.38)
8.79 (.5)
8.81 (.42)
8.8 (.3)
8.81 (.36)
2x2
.87 (.31)
.7 (.32)
88 (.32)
(331)
.73 (:37)
(.3)
2x4
1,75 (.38)
1.74 (.4)
1.77 (.45)
1.74 (.4)
1.76 (.44)
1.39 (.5)
4x2
1.1 (.34)
1.2 (.22)
1.1 (.34)
1.15 (.22)
1.3 (.23)
1.31 (.23)
4x4
2.58 (.39)
2.41 (.36)
2.26 (.3)
2.28 (.32)
2.52 (.32)
2.56 (C33)
TVF (PCA 77.8)
Entropy (PCA 91.2)
Quota (PCA 94.4)
100
300-
040
2500
200
180
250
190
(b)
Figure 3: We simulate preference elicitation where positive exemplars spread along multiple bands. The grey bands represent ground truth regions of positive exemplars, and the red bands represent ground truth regions of negative exemplars. In each plot, the green histogram represents the preference scores of the generated allocations. A model that perfectly enforcing a given preference rule will generate allocations that have valuations entirely within the grey bands.

---

Survey Results
Probit Synthetic Noise Model
Fedine
Noker Iatoo
Easy Positiv Esamples
Hard Examples
Easy Negative Examples
Decision Boundary
Figure 4: We crowdsource human annotators to examine various advertising scenarios and determine if an allocation is fair according fairness (TVF) and measure the noise in responses as function of the ambiguity of the scenario. We expect that the the label noise for a particular TVF value is inversely proportional from the decision boundary. Concretely, easy allocations will have lower noise ratios, and ambiguous allocations will have high noise ratios. We can leverage this model of uncertainty and apply to our synthetic experiments to better simulate human preference elicitation.
given definition. We simplify the definition of total variation
the distance
Soliciting Human Preferences
In this section, we detail the creation of and results derived from two human subjects surveys. The results of these surveys are used validate core assumptions of our model and provide data that use to train an auction model. We find that PreferenceNet is able to adhere to the notion of fairness expressed by the human's preferences of auction outcomes. This encouraging result validates the utility and expressiveness of PreferenceNet.
We conducted both surveys through Cint, crowdsourcing platform which connected with English- speaking participants located in the United States. After submitting an application for our human subjects research to institutional IRB, were notified that the survey review. Our survey protocols can be found in the supplemental materials. Cint compensates per survey completion (regardless of length to complete), and both surveys were set to pay $1.92. All survey results are anonymized to protect participant privacy. We include these results in the supplemental material.
exempt from IRB
Measuring Preference Noise. Our first survey was designed to test how participants would inter- pret and apply a simple fairness definition. The survey protocol was designed as follows: We primed participants to consider advertising auction that was shown to two different (vague) demographic groups. Each participant was told that an auction would be fair if each ad was presented to each group at equal rates. After some familiarization, was fair. Each participant was asked 30 such questions. The median completion time for the survey was 6 minutes with median hourly wage of $18. The 30 questions presented to the participants came from a question bank of 64 randomly generated scenarios, each with an associated TVF score.
asked the participants to determine if given scenario
Human understanding is an inherently noisy process. Despite providing the same context, we observe that survey participants understand a given definition fairness and apply to various scenarios differently. Given this data, we perform normality test using Q-Q plot as shown in the supplemental material. We find that our survey data are well correlated with the Gaussian distribution. This also well supported by our visual inspection of the noise distribution. Experimentally, participants' choice of what is fair has a decision boundary at approximately a TVF value of 0.7. Interestingly, the noise is maximal near the decision boundary as shown in Figure 4 Concretely, can model this noise using a probit model, so that the probability that that the label is unpertubed for particular TVF value increases with distance such that P(Y = 1 k (ا==|), where 4 is the CDF of the Gaussian distribution, ş is the decision boundary, a is the measured sample standard deviation, and & is an optional parameter that can scale the noise. We can use this noise model to perturb the input training data to the MI.P to better simulate real data. We explore this further in the supplemental material.
observe that
8

---

Unit Demand Auction
Additive Auction
Quita
0,005
0.004
0.009
0
0.002
0.002
0.032
0.005
0.002
0.011
0.002
0.001
0.033
Eucoee
0.004
0,002
0.010
Exropp
0,002
0.001
0.052
Ohiik
0.009
0.011
0.010
Quita
0,052
0.033
0.032
Human Preferences
PCA
Regret Mean (STD)
Payment Mean (STD)
2x2 Unit
100.0
.166 (.015)
4.20 (.37)
2x2 Additive
100.0
.005 (.004)
.87 (.31)
Figure 5: We randomly sample 20,000 bids and compute the average 1.2 distance from each model's learned allocations to identify allocation similarity. We find that human preferences are most similar to both TVF and entropy in both the unit demand and additive auction settings. Moreover, PreferenceNet is able to perfectly capture human preferences with low regret.
Group Preference Elicitation. We designed our second survey to ask similar questions to the preference noise survey above. However, there were two primary differences: (1) did not prime the participants with a fairness definition, and (2) the participants were presented with two scenarios and were asked which they thought more fair. Each participant was asked 30 of these pairwise questions on the questions described above (full protocol details in the supplemental materials). This survey had 345 participants who had median completion time of minutes and a median pay of $15 per hour.
Notions of fairness and diversity have neither standardized nor widely accepted formal definitions [38 35 35|36)] 22). 'he purpose this survey is to elicit preferences from group and train an auction model whose allocations resemble the group preference. Using the survey data, apply preference elicitation strategy as described in Section generate training labels for each sample. After training PreferenceNet enforce the group preference for both the unit-demand and additive auction settings, we find that the group preference is more similar to TVF and entropy. As shown in Figure [ human preferences are not perfectly captured by these typical models, both because group preferences rarely converge to a unifying model, and preference elicitation is a noisy process.
Conclusion
Although surveying people to elicit their preferences can effectively help us model ambiguous definitions of socially beneficial constraints, we must be careful about the framing the survey questions and the choice of audiences survey.
Ethical Implications. Humans are inherently biased, so we need to be cognizant of the effects these latent biases might have over preferences for fairness. Moreover, sampling human preferences facilitates opportunities for data poisoning attacks, in which malicious survey respondent could try to negatively impact the survey collection process. In general, we can mitigate both of these issues by sampling at scale to avoid problems with noisy labels, although this brings additional cost. Most importantly, we must involve stakeholders to ensure that their preferences are validated through the learned model in iterative process.
In this paper we present PreferenceNet, a novel extension to RegretNet that makes easier to learn preferences from data to encode socially desirable constraints for auction design. We introduce new metric empirically measure how closely a learned mechanism enforces a particular constraint, and show that our proposed method is able to effectively capture human preferences.

---

Acknowledgments and Disclosure of Funding
This research was supported in part by NSF CAREER Award IIS-1846237 NSF D-ISN Award #2039862, NSF Award CCF-1852352, NIH R01 Award N0M-013039-01, NIST MSE Award #20126334, DARPA GARD #HR001 12020007, DoD WHS Award #HQ00342000035, and Google Faculty Research Award. We thank the authors of ProportionNet for sharing their codebase and Kevin Kuo and Uro Lyi for their feedback in writing this paper.
References
[1] Gianluca Brero, Benjamin Lubin, and Sven Seuken. "Machine Learning- Leamiing-owwered Iterative Combinatorial Auctions". In: (Nov. 2019).
08042 [cs. GT]
[2] Yang Cai, Constamtinos Daskalakis, and S Matthew Weinberg. "An algorithmic characterization of multi-dimeenional mechanisms". In: Proceedings of the forty-fourth annual ACM symposium
Theorr of computing. 2012, pp. 459-478.. 478.
[3] Yang Cai, Constantinos Daskalakis, and S Matthew Weinberg. "Optimal multi cilli-diiensional mechanism design: Reducing revenue to welfare maximization" In: 2012 IEEE 53rd Annual
Symposium on Foundations of Computer Science. IEEE. 2012, pp. 130-139.
[4] Yang Cai, Constantinos Daskalakis, and S Matthew Weinberg. "Understanding incentives: Mechanism design becomes algorithm design". In: 2013 IEEE 54th Annual Symposium on Foundations of Computer Science. IEEE. 2013, pp. 618-627.
[5] I. Elisa Celis, Anay Mehrotra, and Nisheeth K Vishnoi. "Toward controlling discrimination in online ad auctions". In: International Conference Machine Learning (ICML). 2019.
[6] Shuchi Chawla and Meena Jagadeesan. "Fairness in ad auctions through inverse proportional- ity". . In: arXiv preprint arXiv:2003, 13966 (2020).
[7] Edward H Clarke. "Multipart pricing of public goods". In: Public choice (1971), pp. 17-33.
[8] Peter Cramton. "Electricity market design". In: Oxford Review of Economic Policy 33.4 (2017), pp. 589-612.
[9] Peter Cramton. "The FCC spectrum auctions: An early assessment". In: Journal of Economics & Management Strategy 6.3 (1997), pp. 431-495.
[10] Michael Curry, Ping-yeh Chiang, Tom Goldstein, and John Dickerson. "Certifying Strate- gyproof Auction Networks". . In: Neural Information Processing Systems (NeurIPS), 2020.
[11] Constantinos Daskalakis, Alan Deckelbaum, and Christos Tzamos. "Mechanism design via optimal transport". In: Proceedings of the fourteenth ACM conference on Electronic commerce. 2013, pp. 269-286.
[12] Constantinos Daskalakis, Alan Deckelbaum, and Christos Tzamos. "Strong Duality for Multiple-Good Monopolist". In: Econometrica 85.3 (2017), pp. 735-767.
[13] Paul Duetting, Zhe Feng, Harikrishna Narasimhan, David C. Parkes, and Sai Srivatsa Ravin- dranath. "Optimal Auctions through Deep Learning". In: International Conference on Machine Learning (ICML).. 2019.
[14] Benjamin Edelman, Michael Ostrovsky, and Michael Schwarz. "Internet advertising and the generalized second-price auction: Selling billions of dollars worth keywords". In: American Economic Review 97.1 (2007). pp. 2422259.
[15] Zhe Feng, Harikrishna Narasimhan, and David C. Parkes. "Deep Learning for Revenue- Optimal Auctions with Budgets"" In: International Conference on Autonomous Agents and MultiAgent Systems (AAMAS). 2018.
[16] Sainyam Galhotra, Sandhya Saisubramanian, and Shlomo Zilberstein. "Learning to Generate Fair Clusters from Demonstrations".. In: (Feb. 2021). arXiv: 02112.00997 [stat. .M.)
[17] Robert Geirhos, Jömn Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. "Shortcut learning deep neural networks". In: Nature Machine Intelligence 2.11 (2020), pp. 663-673.
[18] Yiannis Giannakopoulos and Elias Koutsoupias. "Duality and optimality of auctions for uniform distributions". In: Economics and Computation (EC). 2014.
[19] Noah Golowich, Harikrishna Narasimhan, and David C. Parkes. "Deep Learning for Multi- Facility Location Mechanism Design". In: International Joint Conference on Artificial Intelli- gence (IJCAI). 2018
10

---

[20] Theodore Groves. "Incentives in teams". In: Econometrica: Journal of the Econometric Society
(1973). pp. 617-631.
[21] Nima Haghpanah and Jason D. Hartline. "Reverse Mechanism Design". In: CoRR his/1404.134) (2014). arXiv: 1404.1341
[22] Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miro Dudik, and Hanna Wallach. "Improving fairness in machine learning systems: What do industry practitioners need?" In: Conference on Human Factors in Computing Systems (CHI). 2019.
[23] Christina Ilvento, Meena Jagadeesan, and Shuchi Chawla. "Multi-Caten Category Fairness in Spon- sored Search Auctions". In: Conference on Fairness, Accountability, and Transparency (FAceT). 2020.
[24] Ian A Kash and Rafael M Frongillo. "Optimal auctions with restricted allocations". In: Eco- nomics and Computation (EC). 2016.
[25] Kevin Kuo, Anthony Ostuni, Elizabeth Horishny, Michael J. Curry, Samuel Dooley, Ping- yeh Chiang, Tom Goldstein, and John P. Dickerson. "ProportionNet: Balancing Fairness and Revenue for Auction Design with Deep Learning". In: arXiv preprint arXiv:2010.063398 (2020).
[26] Kevin Leyton-Brown, Paul Milgrom, and Ilya Segal. "Economics and computer science of a radio spectrum reallocation". In: Proceedings of the National Academy of Sciences (PNAS) 114.28 (2017), pp. 7202-7209.
[27] Jingling Li, Mozhi Zhang. Keyulu Xu, John P. Dickerson, and Jimmy Ba. "Noisy Labels Can Induce Good Representations", In: CoRR abs/2012. 12896 (2020). URL: arq/abs/2012 201.b128661
[28] Alejandro M. Manelli and Daniel R. Vincent. "Bundling as an optimal selling mechanism for multiple-good monopolist". In: J. Econ. Theory 127.1 (2006), pp. 1-35.
[29] Roger B Myerson. "Optimal auction design". In: Mathematics of Operations Research 6.1 (1981), pp. 58-73.
[30] Congressional Budget Office. The Budget and Economic Outlook: Fiscal Years 2001-2010.
Appendix B. 2000.
[31] Gregory Pavlov. "Optimal mechanism for selling two goods". In: The BE Journal of Theoretial Economics 11.1 (2011).
[32] Jad Rahme, Samy Jelassi, Joan Bruna, and S Matthew Weinberg. "A Permutationn-Equuivriaal Neural Network Architecture For Auction Design". In: (2020).
[33] Jad Rahme, Samy Jelassi, and S Matthew Weinberg. " "Auction learning as a two-player game".
In: International Conference Learning Representations (ICLR). 2021.
[34] Jean-Charles Rochet. AA necessary and sufficient condition for rationalizability in quasi-linear context".. In: Journal of mathematical Economics 16.2 (1987), pp. 191-200.
[35] Debjani Saha, Candice Schumann, Duncan C McElfresh, John P Dickerson, Michelle L Mazurek, and Michael Carl Tschantz. "Measuring Non-Expert Comprehension of Machine Learning Fairness Metrics". In: International Conference on Machine Learning (ICML). 2020.
[36] Nripsuta Ani Saxena, Karen Huang, Evan DeFilippis, Goran Radanovic, David C Parkes, and Yang Liu. "How do faimesss definitions fare? Examining public attitudes towards algorithmic definitions of fairness". In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. 2019, pp. 99-106.
[37] Weiran Shen, Pingzhong Tang, and Song Zuo. "Automated mechanism design via neural networks". In: Proceedings of the 18th International Conference on Autonomous Agents and Multiagent Systems. 2019, pp. 215-223.
[38] Megha Srivastava, Hoda Heidari, and Andreas Krause. "Mathematical Notions vs. Human Perception of Faimess: A Descriptive Approach to Faimess for Machine Learning.. In: (Feb. 2019). Xiv: 1902. 04783 [Cs
[39] Andrea Tacchetti, DJ Strouse, Marta Garnelo, Thore Graepel, and Yoram Bachrach. "A neural architecture for designing truthful and efficient auctions". In: arXiv preprint arXiv: (2019).
[40] William Vickrey. "Counterspeculation, auctions, and competitive sealed tenders". In: The Journal of finance 16.1 (1961). pp. 8-37.
[41] Andrew Chi-Chih Yao. "Dominant-strates versus bayesian multi-item auctions: Maximum
revenue determination and comparison", In: Economics and Computation (EC). 2017.
11


Reviews:
[{"review_id": "oAPMrPAy9r0", "reviewer": "Reviewer_wcGk", "summary": "This paper describes how we can encode socially desirable constraints in auction mechanisms learned using the regretNet framework.   The paper has two main contributions:  \n(1) A metric that quantifies how well these mechanisms adhere to these constraints.  \n(2) An neural network and a training procedure (called the preferenceNet) that can encode these constraints using exemplars of the desired allocation\n\nThe authors demonstrate the efficacy of the proposed approach by showing that it can match the performance of standard approaches on both synthetic preferences as well as human preferences", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n- Unlike existing/related work, they attempt to capture socially desirable properties through data (rather than formally defining them).  \n- Results are quite promising - they either match or beat standard approaches (despite only being trained using weaker and noisy signals) \n- Significance: This paper studies an important problem at the intersection of mechanism design and human in the loop learning\n- Clarity: The paper is well written and easy to understand\n\n\nComments:\n- My main concern is the inability of preferenceNet to learn the preference function exactly (which happens to play a key role in the training pipeline)\n- It is unclear how $\\alpha$,  $\\beta$ and $\\gamma$ were chosen.\n- Is there an intuition behind why some constraints are harder than others?\n\nMinor Comments:   \n- typo on line 193: evalate-> evaluate\n- The augmented Lagrangian method is usually the sum of squares of the penalty term (rather than the square of the sum of penalty terms)\n $$\\frac{\\rho_r}{2}\\left(\\sum_{i} rgt_i(w)\\right)^2 => \\frac{\\rho_r}{2} \\sum_{i} \\left(rgt_i(w)\\right)^2$$\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes. (For limitations see above)", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3 hours"}, "extra_text": {}}, {"review_id": "UVCWaST-yt", "reviewer": "Reviewer_mN2t", "summary": "This paper studies of designing revenue optimizing auctions subject to the an additional constraint on fairness. Instead of algebraically capturing the fairness of an allocation, the authors train a neural network termed PreferenceNet that assign a fairness score to any allocation. The neural network for strategy-proof, revenue maximizing auction (RegretNet) is trained to trade-off revenue and regret with the preference loss.  The author evaluate this on small instances (2-4 bidders and 2-4 items) and contrast revenue, regret and fairness of simple RegretNet with one constrained to produce fair allocations. \n\nOne primary motivation for a deep neural network based PreferenceNet is that it can capture real world preferences from humans. To explore this aspect, the authors conducted real world studies to get user preference samples and trained a PreferenceNet on that. They also correlate preference learned from humans with mathematical models of preference. ", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The key idea in this paper is to train a deep neural network for scoring an allocation for fairness. This network is used to score allocations produced by RegretNet which is the deep neural network for computing revenue optimizing incentive compatible allocations. The authors provide empirical evaluation of their idea on small instances. \n\nThe paper is closely related to the paper by Kuo et al. ProportionNet: Balancing Fairness and Revenue for Auction Design with Deep Learning. The key distinction is that paper uses only TVF and the allocation is scored using the mathematical definition, while in this work the authors approximate it using a neural network. \n\nI think this approach is interesting. The authors should explain their approach more fully and also evaluate it on a diverse set of inputs. In particular, fairness can become more restrictive if the advertisers preference for the different goods are very different. So it will be interesting to explore valuations that not identically distributed or with a larger spread of values such as exponential distributions.  \n\nAnother aspect that is worth evaluating is looking into what type of auctions are designed - how is the optimal auction different with the additional fairness constraint. \n\nGiven these concerns, I believe the authors could strengthen paper a lot with a more thorough evaluation of their model. \n\nI also have a minor clarification question. The learning process describes re-training preference net about a few rounds of training regret-net. Is that relevant even for a preference-net trained for real world data. How does that work - does it require new data gathering or some sort of heuristic based approach, what if at all would be the impact of getting that part wrong.\n\nMinor:\n- Many of the references cite the authors as author A et al. It is preferred if the authors could include the full list of authors. \n- the definition of TVF is not clear. What's c, C_k, d^k(j, j')?\n- please clarify that the experiment evaluation was done on bids drawn IID from U[0, 1].\n\nUpdate based on author's response:\nAuthors provided a lot more evaluation of their approach with non-identical distributions, larger instance sizes. The authors should include this new data in the paper as it provides a more thorough evaluation of the approach. For example, we do see larger gaps emerge in the metrics with larger instances.  I had a some more questions about the results. \n1) In some cases RegretNet PCA is lower than PreferenceNet, is this expected. Shouldn't RegretNet perform better?\n2) Can we still say that the two approaches are similar if the PCA is very far from each other or revenue is at times more than 5-6% different?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Authors do discuss limitations of their approach - in terms of human preference data gathering and model training. Particular for human preference data gathering - authors discussed the noise introduced and how they handled that. The authors also discuss the limitations of the PreferenceNet model and the situations in which it fails. One aspect lacking is looking deeper into the model trained and what it teaches about optimal auction design - this aspect is what makes the RegretNet approach interesting. Alternately the authors could discuss how this approach could scale to instances of arbitrary size. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "T5oENu6w98n", "reviewer": "Reviewer_hyQW", "summary": "This paper covers a somewhat interesting and novel problem: learning constraints of an auction setting, and then learning to run an auction that respects those. Fairness in allocations is cited as a justification for this in practice, where it may be beneficial to learn from a human’s choices to determine how to decide who should win in the auction, for example to determine similar allocations across demographics.\n\nThe approach builds on the RegretNet line of work that learns to run revenue maximizing auctions. The approach broadly is to learn a feasibility constraint that is scored by human subjects, and then use that in the RegretNet architecture to learn to generate auctions that respect that constraint. In this work, the motivating constraint is a binary classification of fairness. The implementation is a composition of learning then constraint, followed by leveraging RegretNet to learn according to that function. \n\nTo support the approach, the authors first compare PreferenceNet to RegretNet trained with the exact constraints, and see similar performance suggesting some amount of usefulness. The authors then train PreferenceNet on responses from human subjects in two surveys regarding the binary qualitative fairness of allocations. \n", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper delivers on a novel use for machine learning in auctions. The significance of the approach remains a little open to interpretation: I don’t know that fairness is the constraint that makes the most sense for this approach, but there is no reason for the approach to be limited to fairness as the measure used - in such cases is it better to have the biases explicit or implicit as a result of human responses? In some ways it seems that the decision of what is fair is moved from the explicit constraint to the survey question, either explicitly here or implicitly in any use of this approach.\n\nThe paper is generally well written and comprehensive in exploring the performance of PreferenceNet in the fairness area, both comparing to a baseline RegretNet with perfect knowledge of the constraint and then examples trained based on human input. \n\nEvaluating the architecture choices is outside of my area of expertise. \n\nSmall notes:\nFig 1: doesn't seem to be particularly helpful as a comparison, or I am missing the intuition that would give this a half page here.\nLine 193: eval[u]ate", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The work has one primary potential for negative societal impact: human responses should not be expected to be unbiased. The authors do address this, though perhaps a stronger statement regarding where such approaches would be deemed to be appropriate would be helpful. ", "review_text": "", "extra_scores": {"ethical_concerns": "(addressed above)", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "IeQ0zFjOh1D", "reviewer": "Reviewer_fFgo", "summary": "This paper proposes PreferenceNet, which is a new deep learning approach for designing revenue-maximizing auctions that encodes human preferences. PreferenceNet extends RegretNet [Duetting et al. 2019] with modified loss functions and additional constraints. ", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall I find the paper well-written and clear to follow.  The PreferenceNet proposed by the authors extends from RegretNet, with additional MLP layers that are trained to capture implicit human preferences, modified loss functions and constraints. The authors also conducted  two surveys to further study human preferences for auction outcomes, and I find the conduction of the surveys useful and important, as they provide actual feedbacks from human participants. Below are a few questions and concerns I have.\n\nFirst is about the scalability and flexibility of PreferenceNet. In the current formulation, the ground truth labeling function based on the underlying preference ( s(b) ) is constrained to be a binary label, and the input is a whole vector of allocations for the given bid. However, in practice the decision (\"good\" or \"bad\") could be hard to provided by human, especially when the number of items is large. For example, human could be bad at deciding if two vectors of allocations are actually similar or not. This can make the preference feedbacks obtained very noisy. I would also be curious to see more details on experimentally how much additional computation PreferenceNet requires compared to the RegretNet.\n\nSecond is, the deep learning based approach estimates violations of the strategyproofness and fairness constraints using a gradient-based method which is effective during training, but may not accurately reflect the true extent to which strategyproofness or fairness constraints are violated. Therefore I find the design of the loss functions and constraints for PreferenceNet intuitive, but it is unclear theoretically how these affect the generalization bound for strategyproofness. Besides, will the fairness constraints render the optimal auction infeasible in some cases? Is it possible that each agent has incompatible preferences, and PreferenceNet cannot satisfy the overall preferences for all agents? I would be interested in seeing more elaboration on this side.\n\ntypos:\nEq(3) c, C_k undefined\nEq(4) numerator missing index", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have addressed the limitations and potential negative societal impact of this work. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The review team sentiment is positive about the paper: it is a new dimension to the problem of automated auction design via NN that makes it more practical and allows us to capture subtle notions like the perception of fairness? There were initial concerns around scalability and experiments on harder (non-iid) datasets, but those were adequately addressed by the authors in the discussion. Please incorporate those in the final version.

There are additional concerns on whether biases could be introduced in the model and that through this methodology biases would be harder to identify/correct since they are not explicitly written in a formula, but hidden in the data. While I share the concern of the reviewers, I think this is inherent from this type of approach, so we need to live with that. 

Finally, our recommendation is to accept the paper.
Decision: Accept (Poster)