SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Christos Boutsikas
Purdue University cboutsik@purdue.edu
Petros Drineas Purdue University pdrineas@purdue.edu
Marios Mertzanidis Purdue University mmertzan@purdue.edu
Alexandros Psomas Purdue University somas@cs.purdue.edu
Paritosh Verma Purdue University verma136@purdue.edu
We consider the problem of a revenue-maximizing seller with a large number of items  $m$  for sale to  $n$  strategic bidders, whose valuations are drawn independently from high-dimensional, unknown prior distributions. It is well-known that optimal and even approximately-optimal mechanisms for this setting are notoriously difficult to characterize or compute, and, even when they can be found, are often rife with various counter-intuitive properties. In this paper, following a model introduced recently by Cai and Daskalakis [CD22], we consider the case that bidders' prior distributions can be well-approximated by a topic model. We design an active learning component, responsible for interacting with the bidders and outputting low-dimensional approximations of their types, and a mechanism design component, responsible for robustifying mechanisms for the low-dimensional model to work for the approximate types of the former component. On the active learning front, we cast our problem in the framework of Randomized Linear Algebra (RLA) for regression problems, allowing us to import several breakthrough results from that line of research, and adapt them to our setting. On the mechanism design front, we remove many restrictive assumptions of prior work on the type of access needed to the underlying distributions and the associated mechanisms. To the best of our knowledge, our work is the first to formulate connections between mechanism design, and RLA for active learning of regression problems, opening the door for further applications of randomized linear algebra primitives to mechanism design.
The design of revenue-optimal auctions is a central problem in Economics and Computer Science. In this problem, a revenue-maximizing seller has  $m$  heterogeneous items for sale to  $n$  strategic bidders. Each bidder  $i$  has a type  $\mathbf{t}_i \in \mathbb{R}^d$  which contains enough information to encode the bidder's willingness to pay for every subset of items. Bidders' types are private information, and thus, in order to provide meaningful guarantees on the seller's revenue, the standard approach in Economics is to make a Bayesian assumption: types are drawn from a joint distribution  $\mathcal{D}$ .
Assuming a single item for sale and bidders' types that are drawn independently from known distributions, Myerson's seminal work [Mye81] provides a closed-form solution for the revenue-optimal mechanism. Beyond this single-item case, however, multi-item mechanism design remains an active research agenda, even forty years later. Optimal mechanisms are no longer tractable, in any sense of the word, as well as exhibit various counter-intuitive properties [MV07, DDT13, DDT15, BCKW15, HR15, HN19, PSCW22]; see [Das15] for a survey. On the other
hand, significant research effort has culminated in numerous compelling positive results, such as simple and approximately optimal auctions [CHK07, CHMS10, CMS15, Yao15, RW15, CM16, CDW16, CZ17, KW19, BILW20], as well as efficient algorithms for computing near-optimal auctions  $\left[\mathrm{AFH}^{+}12\right.$  CDW12a, CDW12b, CDW13a, CDW13b], even with just sampling access to the type distribution [CR14, HMR15, DHP16, MR16, CD17, GHZ19, BCD20, GW21].
Despite all this progress, however, there are key challenges in applying these results in practice. First, the computational complexity, sample complexity, approximation guarantees, and communication complexity (i.e., the amount of information the bidder should communicate to the seller) of these results often depend on the number of items  $m$ , which could be prohibitively large (e.g., think of  $m$  as the number of items on Amazon.com). Second, bidders' type distributions are typically high-dimensional, or otherwise complex objects, that are not known nor can they be sampled. Instead, the designer might have an estimated distribution  $\hat{\mathcal{D}}$ , e.g., through market research, that is close to the real distribution  $\mathcal{D}$ . Motivated by these issues, Cai and Daskalakis [CD22] introduce a model where the true type distribution  $\mathcal{D}_i$  of bidder  $i$  is close to a structured distribution  $\hat{\mathcal{D}}_i$ . Specifically, they assume that there is an underlying design matrix  $\mathbf{A} \in \mathbb{R}^{m \times k}$  of  $k$  "archetypes," with  $k \ll m$ . Intuitively, bidder  $i$  can be approximated by a linear combination of  $k$  archetypal bidders. This same assumption has been central in the study of recommender systems. In this model, [CD22] give a framework for transforming a mechanism  $\hat{\mathcal{M}}$  for the low-dimensional distribution  $\hat{\mathcal{D}}_z$  into a mechanism for the true type distribution with good revenue guarantees, and whose computational and communication complexity does not depend on the number of items  $m$ .
The impact of their work notwithstanding, the above results require strong structural assumptions on: the design matrix  $\mathbf{A}$ ; the bidders' valuation functions; and very specific access to (or exact knowledge of) the structured distribution  $\hat{\mathcal{D}}_z$  and the mechanism  $\hat{\mathcal{M}}$ . Our work connects the recommender system approaches for mechanism design with recent progress on Randomized Linear Algebra for active learning for regression problems. We relax, and even remove these restrictive assumptions, and open the door to future exploration of more elaborate recommender system models in the context of mechanism design, using randomized linear algebra primitives.
The framework and results [CD22]. To place our results in context, we start with a brief overview of [CD22], which considers a setting where the type distribution  $\mathcal{D}_i$  of bidder  $i$  is close to a distribution  $\hat{\mathcal{D}}_i$  in the Prokhorov distance, under the  $\ell_{\infty}$  norm (Definition 1). Here,  $\hat{\mathcal{D}}_i$  first samples a vector  $\mathbf{z} \in [0,1]^k$  from a low-dimensional distribution  $\hat{\mathcal{D}}_{z,i}$ , and then outputs  $\mathbf{A}\mathbf{z}$ , where  $\mathbf{A} \in \mathbb{R}^{m \times k}$  is a known matrix. The proposed framework has a learning and a mechanism design component.
Their learning component consists of a communication-efficient query protocol  $\mathcal{Q}$  for interacting with each bidder  $i$  such that, if the type  $\mathbf{t}_i$  of bidder  $i$  satisfies  $\| \mathbf{t}_i - \mathbf{A}\mathbf{z}\|_{\infty} \leq \varepsilon$ , the query protocol outputs a vector  $\mathcal{Q}(\mathbf{t}_i)$  such that  $\| \mathcal{Q}(\mathbf{t}_i) - \mathbf{z}\|_{\infty} \leq \zeta_{\infty}$ . [CD22] give such query protocols under strong (distinct) conditions on  $\mathbf{A}$ , and specifically when  $\mathbf{A}$ : (i) satisfies an assumption similar to the separability condition of [DS03], (ii) is generated from a distribution where each archetype is an independent copy of an  $m$ -dimensional Gaussian, or (iii) is generated from a distribution where each archetype is an independent copy of an  $m$ -dimensional, bounded distribution with weak dependence. We discuss these restrictions in more detail in Appendix D.4.1. The query complexity, as well as the error  $\zeta_{\infty}$ , depend on them, but, importantly, are independent of the number of items  $m$ .
Their mechanism design component is a refinement of a robustification result of [BCD20]. For this transformation to work, one needs to interact with the mechanism and the underlying distributions using highly non-trivial operations, which are computationally demanding, and require exact knowledge of bidders' valuation functions. In our work we overcome this issues by developing new reductions and plugging them in the framework established by [BCD20] and [CD22]. The overall interplay between the different mechanism design and active regression components can be seen in Fig. 1.
Combining the two components, [CD22] obtain mechanisms for  $\mathcal{D}$ , for constrained-additive bidders. In these mechanisms, each bidder is required to answer a small number of simple Yes/No queries
1 By efficient we mean a query protocol that asks each bidder a small amount of queries. Se also Definition 3. Recall that for any (integer)  $1\leq p <   \infty$  and a vector  $\mathbf{x}\in \mathbb{R}^d$ $\| \mathbf{x}\| _p^p = \sum_{i = 1}^d |\mathbf{x}_i|^p$  ; for  $p = \infty$ $\| \mathbf{x}\|_{\infty} = \max_{i = 1\dots d}|\mathbf{x}_{i}|$  . See [CD22] for an exact expression for  $\zeta_{\infty}$  A function is constrained-additive if  $v(t,S) = \max_{T\in \mathcal{I}\cap 2^S}\sum_{j\in T}t_j$  , where  $\mathcal{I}$  is a downward-closed set system.
of the form "are you willing to pay  $p$  for item  $j$ ?", such that the loss in revenue and the violation in incentive compatibility do not depend on the number of items  $m$ .
Randomized Linear Algebra (RLA) for active learning. RLA for active learning has focused on solving regression problems of the form
$$
\mathbf {z} = \arg \min _ {\mathbf {x}} \| \mathbf {t} - \mathbf {A x} \| _ {p},
$$
for  $\ell_p$  norms with  $1\leq p < \infty$ , by querying only a subset of elements of the vector  $\mathbf{t}$ . In prior work on RLA for active learning, the focus has been on recovering an approximate solution that achieves a relative error or constant factor approximation to the optimum loss. We adapt these bounds to our setting, which could include noisy instead of exact queries, and prove bounds for the  $\ell_p$  norm error of the exact minus the approximate solution. Specifically, we bound  $\| \mathcal{Q}(\mathbf{t}_i) - \mathbf{z}\| _p\leq \zeta_p$ . We provide bounds on  $\zeta_{p}$  for all integers  $1\leq p < \infty$ . Our bounds depend on the modelling error  $\| \mathbf{t} - \mathbf{A}\mathbf{x}\| _p$  and some measure of the query noise (see Definitions 4 and 5); both dependencies are expected. Importantly, our bounds hold for arbitrary archetype matrices, very much unlike the work of [CD22], which focused on very restricted classes of matrices. A single property of the archetype matrix, the smallest singular value with respect to the  $\ell_p$  induced matrix norm,  $\sigma_{\mathrm{min},p}(\mathbf{A})$ , can characterize the quality of the error. As  $\sigma_{\mathrm{min},p}(\mathbf{A})$  decreases, the error  $\zeta_p$  grows. Intuitively,  $\sigma_{\mathrm{min},p}(\mathbf{A})$  is a measure of independence of the archetypes, with small values corresponding to linearly dependent archetypes. The query complexity needed to achieve error  $\zeta_p$  is almost linear (up to logarithmic factors) on  $k$  for  $p = 1$  and  $p = 2$ , and grows with  $k^{p / 2}$  for  $p\geq 3$ . Our query complexity bounds have no dependency on  $m$  (number of items) or  $d$  (dimensionality of a type) enabling us to produce results way beyond constrained-additive valuations, as  $d = 2^m$  dimensions suffice to encode arbitrary valuation functions.
It is critical to highlight that our ability to provide bounds on the approximation error for arbitrary archetype matrices is, at least partly, due to leveraging information from the archetype matrix  $\mathbf{A}$ . Specifically, we use this information to select which bidder types to query, instead of just querying types uniformly at random. This information involves the computation or approximation of the well-studied leverage scores of  $\mathbf{A}$  for  $p = 2$  and of the so-called Lewis weights for all other values of  $p$  (see Section 3.1). We do note that in our framework, the errors due to the modeling of the bidder type  $\mathbf{t}$  as the product  $\mathbf{Az}$  and the query noise are always bounded by the respective  $\ell_p$  norm. Thus, our models are more restrictive compared to the  $\ell_{\infty}$  norm models of [CD22]. However, to the best of our knowledge, even assuming such restrictive models, the results and tools of prior work do not extend to arbitrary archetype matrices. This is precisely the gap that is bridged by our work, using RLA for active learning of  $\ell_p$  norm regression for  $1 \leq p < \infty$ .
**Mechanism Design.** On the mechanism design front, our main contribution is relaxing the assumptions of [CD22, BCD20] on the type of access needed to the low-dimensional distribution and the mechanism for it. Specifically, we further refine the robustification result of Brustle et al. [BCD20] and remove the need for the aforementioned strong oracle.
The main difficulty of transforming mechanisms for one distribution into mechanisms for another distribution is that the two distributions might not share the same support. The crux of the issue is that the incentive constraints are very delicate; a small change in the underlying distribution may drastically change the agents' valuation distribution over the mechanisms' outcomes. One way to tame the distribution of outcomes is to map bids that are not in the support of the initial distribution, to bids that are. Brustle et al. [BCD20] do this by "optimally misreporting" on behalf of the bidder, by calculating  $\mathrm{argmax}_{\mathbf{t}_i' \in \text{supp}(\widetilde{\mathcal{D}}_{z,i})} \mathbb{E}_{\mathbf{b}_{-i} \sim \widetilde{\mathcal{D}}_{z,-i}}[u_i(v_i, \hat{\mathcal{M}}(\mathbf{t}_i', \mathbf{b}_{-i}))]$ , where  $\widetilde{\mathcal{D}}_{z,i}$  is a rounded-down version of  $\hat{\mathcal{D}}_{z,i}$ . As we've discussed, for this operation to be viable, many things need to be assumed about what the designer knows and can compute about  $\hat{\mathcal{M}}, \hat{\mathcal{D}}_{z,i}$ , and the bidder's valuation function. Our approach, instead, leverages the fact that when two distributions are close in Prokhorov distance, under any  $\ell_p$  norm, any point on the support of one distribution is close to a point on the support of the other, with high probability. Our construction simply maps a report  $\mathbf{w}_i$  to the "valid" report (approximately) closest to  $\mathbf{w}_i$  in  $\ell_p$  distance. This operation is linear on the support size. Furthermore, our overall robustification result holds for all norms, not just  $\ell_{\infty}$ , and our construction is completely agnostic to bidders' valuation functions.
Combining the components. Combining the two components we can, without any assumptions on  $\mathbf{A}$ , given a mechanism for the low-dimensional prior, design mechanisms with comparable revenue guarantees, where each bidder is required to answer a small number of queries. Our queries ask a bidder her value for a subset of items, and our mechanism can accommodate any valuation function, significantly extending the scope of our results.
Related Work. Aside from the work of [CD22], on the mechanism design front, the most relevant works are [CD17], that consider learning multi-item auctions given "approximate" distributions, and [BCD20], that consider learning multi-item auctions when type distributions are correlated, yet admit special structure. On the RLA front, we leverage and adapt multiple recent results on approximating  $\ell_p$  regression problems in an active learning setting. We discuss prior work on RLA for active learning and its connections to our setting in Section 3.1 and Appendix D.
Let  $[n] := \{1, 2, \dots, n\}$  denote the first  $n$  natural numbers. A revenue-maximizing seller has a set  $[m]$  of  $m$  heterogeneous items for sale to  $n$  strategic bidders indexed by  $[n]$ . Bidder  $i$  has a private type vector  $\mathbf{t}_i \in \mathbb{R}^d$ , and the types of all bidders are represented by a valuation profile  $\mathbf{t} = (\mathbf{t}_1, \dots, \mathbf{t}_n)$ . Bidder  $i$  has a valuation function  $v_i: \mathbb{R}^d \times 2^{[m]} \to \mathbb{R}_+$ , that takes as input the bidder's type and a (possibly randomized<sup>4</sup>) set of items  $S \subseteq [m]$  and outputs the bidder's value for  $S$ . Note that  $d \leq 2^m$ , since expressing a valuation function requires at most one real number per subset of items. Types are drawn independently. Let  $\mathcal{D} = \times_{i \in [n]} \mathcal{D}_i$  be the distribution over bidders' types,  $\mathcal{D}_{-i} = \times_{j \in [n] / \{i\}} \mathcal{D}_j$  be the distribution of all bidders excluding  $i$ , and  $supp(\mathcal{D})$  be the support of a distribution  $\mathcal{D}$ .
Mechanisms. A mechanism  $\mathcal{M} = (x,p)$  is a tuple where  $x:\mathbb{R}_+^{nd}\to 2^{[nm]}$  is the allocation rule, and  $p:\mathbb{R}_+^{nd}\to \mathbb{R}_+^n$  is the payment rule, which map reported types to allocations of the items and payments, respectively. Specifically,  $x_{i,j}(\mathbf{b})\coloneqq (x(\mathbf{b}))_{i,j}$  denotes the probability that bidder  $i$  receives item  $j$  for input valuation profile  $\mathbf{b}$ , and  $p_i(\mathbf{b})\coloneqq (p(\mathbf{b}))_i$  denotes the amount bidder  $i$  has to pay. Let  $u_{i}(\mathbf{t}_{i},\mathcal{M}(\mathbf{b}))$  be the utility of bidder  $i$  with type  $\mathbf{t}_i$  for participating in mechanism  $\mathcal{M}$ , under reports  $\mathbf{b}$ . Bidders are risk-free and quasi-linear i.e.,  $u_{i}(\mathbf{t}_{i},\mathcal{M}(\mathbf{b})) = \mathbb{E}[v_{i}(\mathbf{t}_{i},x(\mathbf{b})) - p_{i}(\mathbf{b})]$ , where the expectation is taken over the randomness of the allocation rule. Since we only consider truthful mechanisms, unless stated otherwise, reported types will be the same as the true types.
A bidder's objective is to maximize her utility. The seller strives to design mechanisms that incentivize bidders to report truthfully. We use the following notions of truthfulness. A mechanism  $\mathcal{M}$  is  $\varepsilon$ -Bayesian Incentive Compatible ( $\varepsilon$ -BIC), if for each bidder  $i \in [n]$ , any type  $\mathbf{t}_i$  and misreport  $\mathbf{t}_i'$  we have that:  $\mathbb{E}_{\mathbf{t}_{-i} \sim \mathcal{D}_{-i}}[u_i(\mathbf{t}_i, \mathcal{M}(\mathbf{t}_i, \mathbf{t}_{-i}))] \geq \mathbb{E}_{\mathbf{t}_{-i} \sim \mathcal{D}_{-i}}[u_i(\mathbf{t}_i, \mathcal{M}(\mathbf{t}_i', \mathbf{t}_{-i}))] - \varepsilon$ . A mechanism  $\mathcal{M}$  is  $(\varepsilon, \delta)$ -BIC if for each bidder  $i \in [n]$ , and any misreport  $\mathbf{t}_i'$  we have that:
$$
\underset {\mathbf {t} _ {i} \sim \mathcal {D} _ {i}} {\mathbb {P}} \left[ \underset {\mathbf {t} _ {- i} \sim \mathcal {D} _ {- i}} {\mathbb {E}} \left[ u _ {i} \left(\mathbf {t} _ {i}, \mathcal {M} \left(\mathbf {t} _ {i}, \mathbf {t} _ {- i}\right)\right) \right] \geq \underset {\mathbf {t} _ {- i} \sim \mathcal {D} _ {- i}} {\mathbb {E}} \left[ u _ {i} \left(\mathbf {t} _ {i}, \mathcal {M} \left(\mathbf {t} _ {i} ^ {\prime}, \mathbf {t} _ {- i}\right)\right) \right] - \varepsilon \right] \geq 1 - \delta .
$$
A  $(\varepsilon, 0)$ -BIC mechanism is a  $\varepsilon$ -BIC mechanism; a 0-BIC mechanism is simply BIC. Finally, a mechanism  $\mathcal{M}$  is ex-post Individually Rational (IR) if for all valuation profiles  $\mathbf{t}$  and all bidders  $i \in n$ ,  $u_i(\mathbf{t}_i, \mathcal{M}(\mathbf{t}_i, \mathbf{t}_{-i})) \geq 0$ . The seller's objective is to maximize her expected revenue. For a mechanism  $\mathcal{M}$  and distribution  $\mathcal{D}$  we denote the expected revenue as  $Rev(\mathcal{M}, \mathcal{D}) = \mathbb{E}_{\mathbf{t} \sim \mathcal{D}}\left[\sum_{i \in [n]} p_i(\mathbf{t})\right]$ . Note that, we are calculating revenue assuming truthful reports, even for, e.g.,  $(\varepsilon, \delta)$ -BIC mechanisms.
Statistical Distance. In this work we design mechanisms that work well, as long as they are evaluated on distributions that are "close" to  $\mathcal{D}$ . Here, we define the notion of distance between two probability measures that we use throughout the paper.
Definition 1 (Prokhorov Distance). Let  $(\mathcal{U},d)$  be a metric space and  $\mathcal{B}$  be a  $\sigma$ -algebra on  $\mathcal{U}$ . For  $A\in \mathcal{B}$ , let  $A^{\varepsilon} = \{x:\exists y\in A$  s.t.  $\pi (x,y) < \varepsilon \}$  where  $\pi$  is some distance metric. Two probability measures  $P,Q$  on  $\mathcal{B}$  have Prokhorov distance: inf  $\{\varepsilon >0:P(A)\leq Q(A^{\varepsilon}) + \varepsilon$  and  $Q(A)\leq P(A^{\varepsilon}) + \varepsilon ,\forall A\in \mathcal{B}\}$ . We choose  $\pi$  to be the  $\ell_p$ -distance, and we denote the Prokhorov distance between measures  $P,Q$  as  $\pi_p(P,Q)$ .
Each subset might be selected according to a probability distribution.
The following is an equivalent characterization of Prokhorov distance due to Strassen [Str65].
Lemma 1 ([Str65]). Let  $\mathcal{D}$  and  $\mathcal{D}'$  be two distributions supported on  $\mathbb{R}^n$ .  $\pi_p(\mathcal{D},\mathcal{D}') \leq \varepsilon$  iff there exists coupling  $\gamma$  of  $\mathcal{D}$  and  $\mathcal{D}'$ , such that  $\mathbb{P}_{(\mathbf{x},\mathbf{y})\sim \gamma}[\| \mathbf{x} - \mathbf{y}\| _p > \varepsilon] \leq \varepsilon$ .
Recommendation system-inspired model. We assume that, for each bidder, there exists a known design matrix  $\mathbf{A} \in \mathbb{R}^{d \times k^5}$ , where the columns of  $\mathbf{A}$  represent  $k$  "archetypes", for a constant  $k$ . Our results hold if these matrices are different for each bidder, however, for ease of notation we will assume all bidders have the same design matrix. For each bidder  $i$  there exists a distribution  $\hat{\mathcal{D}}_{z,i}$  supported on the latent space  $[0,1]^k$ . Let  $\hat{\mathcal{D}}_i = \mathbf{A} \circ \hat{\mathcal{D}}_{z,i}$  be the distribution induced by multiplying a sample from  $\hat{\mathcal{D}}_{z,i}$  with the design matrix  $\mathbf{A}$ , i.e.,  $\hat{\mathcal{D}}_i$  is the distribution of  $\mathbf{A}\mathbf{y}$  where  $\mathbf{y} \sim \hat{\mathcal{D}}_{z,i}$ .
The valuation function over the latent types is defined as  $v_{i}^{\mathbf{A}}(\mathbf{z}_{i},S) \coloneqq v_{i}(\mathbf{A}\mathbf{z}_{i},S)$  for any bundle  $S \subseteq [m]$ . We will use the following notion of Lipschitz continuity for valuation functions.
Definition 2 (Lipschitz Valuation). A valuation function  $v(\cdot, \cdot): \mathbb{R}^d \times [0,1]^m \to \mathbb{R}_+$  is  $\mathcal{L}$ -Lipschitz, if for any two types  $\mathbf{t}, \mathbf{t}' \in \mathbb{R}^d$  and any bundle  $S \subseteq [m]$ ,  $|v(\mathbf{t}, S) - v(\mathbf{t}', S)| \leq \mathcal{L}\|\mathbf{t} - \mathbf{t}'\|_{\infty}$ .
We include a table, Table 1, with all the notation used throughout the paper in the appendix.
3 Active Learning for Regression and Mechanism Design
In this section, we state our main results, deferring all technical proofs to the appendix. We present mechanisms that are completely agnostic with respect to  $\mathcal{D}$ , the distribution from which bidders' types are drawn from. However, we have limited access (described later in this section) to  $(i)$  a design matrix  $\mathbf{A}$ ;  $(ii)$  distributions  $\hat{\mathcal{D}}_{z,i}$  over  $\mathbb{R}^k$ , where for all  $i\in [n]$ ,  $\pi_p(\mathcal{D}_i,\mathbf{A}\circ \hat{\mathcal{D}}_{z,i})\leq \varepsilon_{\mathrm{md1},p}$  for some  $\varepsilon_{\mathrm{md1},p} > 0$ ; and  $(iii)$  a mechanism  $\hat{\mathcal{M}}$  for  $\hat{\mathcal{D}}_z = \times_{i\in [n]}\hat{\mathcal{D}}_{z,i}$ . This limited access to the design matrix motivates the use of active learning, which deals precisely with settings where the algorithm is allowed to (interactively) query a subset of the available data points for their respective labels (see [MD21, MMWY22] for precise definitions of the active learning setting in regression problems). Our approach is modular and starts by building an active learning component for regression problems (Section 3.1) followed by the mechanism design component (Section 3.2). We combine the two components to get an overall mechanism for  $\mathcal{D}$  in Section 3.3.
3.1 Active learning for regression via Randomized Linear Algebra