SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem 3.1 (Informal version of Theorem C.1 in Appendix C). Under general conditions on  $f_{\mathcal{D}}$  and  $\ell$ , for any finite  $\tau$  and  $\alpha \geq 1$ , (OptProg) has a unique solution  $f^{\star}(x) \propto \exp(-I_{f_{\mathcal{D}},\ell,\alpha}(x)/\gamma^{\star})$ , where  $\gamma^{\star} > 0$  is unique and also depends on  $\alpha$  and  $\tau$ . Further,  $\mathsf{Ent}(f^{\star}) = \tau$ .
The uniqueness in Theorem 3.1 implies that, if  $\alpha = 1$ ,  $\tau = \operatorname{Ent}(f_{\mathcal{D}})$ , and  $\ell(x,v) \coloneqq \ln f_{\mathcal{D}}(x) - \ln f_{\mathcal{D}}(v)$ , then the optimal solution is  $f^{\star}(x) = f_{\mathcal{D}}(x)$ . To see this, note that in this case  $I_{f_{\mathcal{D}},\ell,\alpha}(x) = \ln f_{\mathcal{D}}(x) + \operatorname{Ent}(f_{\mathcal{D}})$ . Hence,  $f^{\star}(x) = f_{\mathcal{D}}(x)$  satisfies Theorem 3.1's conclusion (see Section C.7 for details). Thus, in the absence of risk averseness, and for an appropriate choice of resource-information parameter, the output density is the same as the true density.
Theorem 3.1 can be viewed as a significant extension of results that show how well-known probability distributions arise as solutions to the entropy-maximization framework. Indeed, the standard maximum-entropy formulation only considers the setting where the input utility is given by a single value, i.e., the distribution corresponding to  $f_{\mathcal{D}}$  is concentrated at a single point; and the risk-averseness parameter  $\alpha = 1$ . While the optimal solution to the maximum-entropy framework (2) restricted to the class of well-known loss functions, e.g.  $\ell_2^2$ -loss or linear loss, can be understood by using standard tools from convex optimization (see [48]), characterizing the optimal solution to the general formulation (OptProg) is more challenging because of several reasons: (i) The input density  $f_{\mathcal{D}}$  need not be concentrated at a single point, and hence one needs to understand conditions on  $f_{\mathcal{D}}$  when the formulation has a unique optimal solution. (ii) The loss function can be arbitrary and one needs to formulate suitable conditions on the loss function such that (OptProg) has a unique optimal solution. (iii) The risk-averseness parameter  $\alpha$  makes the loss function asymmetric (around
<sup>1</sup>A variation of (4) that we use in the empirical part is the following: For a fixed "shift"  $v_0$ , let  $\ell_{\alpha}(x, v) \coloneqq \alpha \cdot \ell(x, v + v_0)$  if  $x \geq v + v_0$  and  $\ell_{\alpha}(x, v) \coloneqq \ell(x, v + v_0)$  if  $x < v + v_0$ .
any fixed value  $v$ ) and makes the analysis of the error in the objective function non-trivial. Roughly speaking, the only restrictions, other than standard integrability assumptions, that we need on the input are: (a) Monotonicity of the loss function  $\ell(x, v)$  with respect to either  $x$  or  $v$ , (b) the growth rate of the loss function  $\ell(x, v)$  is at least logarithmic, and (c) the function  $I_{f_{\mathcal{D}}, \ell, \alpha}(x)$  has a unique global minimum, which is a much weaker assumption than convexity of the function. Note that for the  $\ell_2^2$ -loss function given by  $\ell(x, v) = (x - v)^2$ , the first two conditions hold trivially; and when the input density  $f_{\mathcal{D}}$  is Gaussian, it is not hard to show that  $I_{f_{\mathcal{D}}, \ell, \alpha}(x)$  is strongly convex and hence the third condition mentioned about holds (see Appendix H for details). These conditions are formally stated in Appendix C.1 and we show that they hold for the cases of Gaussian, Pareto, Exponential, and Laplace densities in Sections H, I, J, K respectively.
The proof of Theorem 3.1 is presented in Appendix C and the following are the key steps in it: (i) The proof starts by considering the dual of (OptProg) and shows that strong duality holds (see Appendix C.2 and Appendix C.3). (ii) The next step is to show that the optimal solution  $f^{\star}$  of (OptProg) exists and is unique. This requires proving that the dual variable  $\gamma^{\star}$  (corresponding to the entropy constraint in (OptProg)) is positive - while this variable is always non-negative, the main technical challenge is to show that it is non-zero. In fact, there are instances of (OptProg) where  $\gamma^{\star}$  is zero, and an optimal solution does not exist (or an optimal solution exists, but is not unique). (iii) The proof of  $\gamma^{\star} \neq 0$  requires us to understand the properties of the integral  $I_{f_{\mathcal{D}},\ell ,\alpha}(x)$  (abbreviated as  $I(x)$  when the parameters  $f_{\mathcal{D}},\ell ,\alpha$  are clear from the context). In Appendix C.4 we show that  $I(x)$  can be expressed as a sum of two monotone functions (see Theorem C.10). This decomposition allows us to show that the optimal value of (OptProg) is finite. (iv) In Appendix C.5, we show that the optimal value of (OptProg) is strictly larger than  $I(x^{\star})$  (Lemma C.13, Lemma C.14), where  $x^{\star}$  is the minimizer of  $I(x)$ . This requires us to understand the interplay between the growth rate of the expected loss function and the entropy of a density as we place probability mass away from  $x^{\star}$ . Indeed, these technical results do not hold true if the loss function  $\ell (x,v)$  grows very slowly (as a function of  $x / v$  or  $(|x - v|)$ ). (v) Finally, in Theorem C.15, we show that  $\gamma^{\star}$  is nonzero. This follows from the fact that if  $\gamma^{\star} = 0$ , then the optimal value of (OptProg) is equal to  $I(x^{\star})$ , which contradicts the claim in (iv) above. Once we show  $\gamma^{\star} > 0$ , the expression for the (unique) optimal solution, i.e.,  $f^{\star}(x)\propto \exp (-I(x) / \gamma^{\star})$ , follows from Theorem C.16.
We conclude this section with two remarks. 1) In Appendix E, we show that Theorem 3.1 implies that  $\tau = (\mathrm{Err}_{\ell,\alpha}(f_{\mathcal{D}}, f^{\star}) / \gamma^{\star}) + \ln Z^{\star}$ , where  $Z^{\star} := \int_{\Omega} \exp(-I_{f_{\mathcal{D}}, \ell,\alpha}(x) / \gamma^{\star}) d\mu(x)$  is the partition function or the normalizing constant that makes  $f^{\star}$  a probability density; This equation is an analog of the Gibbs equation in statistical physics and gives a physical interpretation of  $I_{f_{\mathcal{D}}, \ell,\alpha}(x)$  and  $\gamma^{\star}$ :  $\gamma^{\star}$  is the temperature and  $I_{f_{\mathcal{D}}, \ell,\alpha}(x)$  is the energy corresponding to state  $x$ . This may be useful in understanding the effects of different parameters on the output density. 2) If one wishes, one can use (OptProg) to understand the setting where a single individual is being evaluated by setting the input density  $f_{\mathcal{D}}$  to be concentrated at their true utility  $v$ . For instance, if we set the loss function to be the  $\ell_2^2$ -loss, using Theorem 3.1, one can show that for any given values of the parameters,  $\alpha$  and  $\tau$  and loss function being  $\ell_2^2$ -loss, the mean of the output density is  $v - \sqrt{\frac{\gamma^{\star}}{\pi}} \cdot \frac{\sqrt{\alpha} - 1}{\sqrt{\alpha}}$ ; see Appendix D for a proof. Therefore for  $\alpha > 1$ , the mean of the output density is strictly less than  $u$ . This gives a mapping from the "true ability" to the (mean of the) "biased ability" in this case. This mapping can be used to understand how the parameters  $\tau$  and  $\alpha$  in the evaluation process transform the true ability.
Effect of varying  $\tau$  for a fixed  $\alpha$ . We first study the effect of changing the resource-information parameter  $\tau$  on  $f^{\star}$  for a fixed value of the risk-averseness parameter  $\alpha \geq 1$ . To highlight this dependency on  $\tau$ , here we use the notation  $f_{\tau}^{\star}$  to denote the optimal solution  $f^{\star}$ . We start by noting that as  $\gamma^{\star}$  increases, the optimal density becomes close to uniform, and as it goes towards zero, the optimal density concentrates around a point  $x^{\star}$  that minimizes energy:  $\mathrm{argmin}_{x \in \Omega} I_{f_{\mathcal{D}}, \ell, \alpha}(x)$ . Note that the point  $x^{\star}$  does not depend on  $\tau$ . However,  $\gamma^{\star}$  may depend in a complicated manner on both  $\tau$  and  $\alpha$ , and it is not apparent what effect changing  $\tau$  has on  $\gamma^{\star}$ . We show that, for any fixed  $\alpha \geq 1$ , as  $\tau$  decreases, the output density gets concentrated around  $x^{\star}$  (see Theorem F.1). This confirms the intuition that as we reduce  $\tau$  by adding more resources in the evaluation process, the uncertainty in the output density should be reduced. Similarly, if  $\tau$  increases because of a reduction in the resources invested in the evaluation, the uncertainty in  $f_{\tau}^{\star}$  should increase, and hence, the output density should converge towards a uniform density. For specific densities, one can obtain sharper results. Consider the case when  $f_{\mathcal{D}}$  is a Gaussian with mean  $m$  and variance  $\sigma^2$ , and  $\ell_{\alpha}(x, v) \coloneqq \alpha (x - v)^2$  if  $x \geq v$ , and  $(x - v)^2$  if  $x < v$ . The uncertainty in the Gaussian density is captured by the variance, and hence, we expect the output density to have a higher variance when the parameter  $\tau$  is increased. Indeed,
when  $\alpha = 1$ , we show that the optimal density  $f_{\tau}^{\star}$  is a Gaussian with mean  $m$  and variance  $e^{2\tau}\sigma^2$ ; see Appendix H. Thus, if one increases  $\tau$  from  $-\infty$  to  $\infty$ , the variance of the output density changes monotonically from 0 to  $\infty$ . When  $\alpha \geq 1$ , numerically, it can be seen that for any fixed  $\alpha \geq 1$ , increasing  $\tau$  increases the variance, and also decreases the mean of  $f_{\tau}^{\star}$ ; see Figure 3. Intuitively, the decrease in mean occurs because higher variance increases the probability of the estimated value being much larger than the mean, and the risk-averseness parameter imposes a high penalty when the estimated value is larger than the true value. In fact, we show in Theorem F.5 that the variance of  $f_{\tau}^{\star}$  for any continuous input density  $f_{\mathcal{D}}$  supported on  $\mathbb{R}$  is at least  $\frac{1}{2\pi} e^{2\tau - 1}$ . This follows from the well-known fact that among all probability densities supported on  $\mathbb{R}$  with variance  $\sigma^2$ , the Gaussian density with variance  $\sigma^2$  maximizes the (differential) entropy [120]; see Section F.2 for details. In a similar vein, we show that the mean of  $f_{\tau}^{\star}$  is at least  $e^{\tau - 1}$  when the input density  $f_{\mathcal{D}}$  is supported on  $[0, \infty)$ , and hence approaches  $\infty$  as  $\tau$  goes to  $\infty$  (see Theorem F.7). This result relies on the fact that among all densities supported on  $[0, \infty)$  and with a fixed expectation  $1 / \lambda$  (for  $\lambda > 0$ ), the one maximizing the entropy is the exponential density with parameter  $\lambda$  [48].
We now consider the special setting when the input density  $f_{\mathcal{D}}$  is Pareto with parameter  $\beta > 0$  ( $f_{\mathcal{D}}(x) \coloneqq \beta x^{-\beta - 1}$  for  $x \in [1, \infty)$ ), and  $\ell_{\alpha}(x, v) \coloneqq \alpha \ln(x / v)$  if  $x \geq v$ , and  $\ln(x / v)$  if  $x < v$ . When  $\alpha = 1$ , we show that the optimal density  $f_{\tau}^{\star}$  is also a Pareto density with parameter  $\beta^{\star}$  satisfying the following condition:  $1 + (1 / \beta^{\star}) - \ln \beta^{\star} = \tau$ . Using this, it can be shown that, for  $\alpha = 1$ , both the mean and variance of  $f_{\tau}^{\star}$  monotonically increase to  $\infty$  as  $\tau$  goes to  $\infty$ ; see Appendix I. The increase in variance reflects the fact that increasing  $\tau$  increases the uncertainty in the evaluation process. Unlike the Gaussian case, where the mean of  $f_{\tau}^{\star}$  could shift to the left of 0 with an increase in  $\tau$ , the mean of the output density in this setting is constrained to be at least 1, and hence, increasing the variance of  $f_{\tau}^{\star}$  also results in an increase in its mean. Numerically, for any fixed  $\alpha \geq 1$ , increasing  $\tau$  increases both the mean and variance of  $f_{\tau}^{\star}$ ; see Figures 3 and 6.
Effect of varying  $\alpha$  for a fixed  $\tau$ . Let  $f_{\alpha}^{\star}$  denote the optimal solution  $f^{\star}$  with  $\alpha$  for a fixed  $\tau$ . We observe that, for any fixed  $x$ ,  $I_{f_{\mathcal{D}},\ell,\alpha}(x)$ , which is the expected loss when the output is  $x$ , and  $\mathrm{Err}_{\ell,\alpha}(f_{\alpha}^{\star},f_{\mathcal{D}})$  are increasing functions of  $\alpha$ ; see Appendix G. Thus, intuitively, the mass of the density should shift towards the minimizer of  $I_{f_{\mathcal{D}},\ell,\alpha}(x)$ . Moreover, the minimizer of  $I_{f_{\mathcal{D}},\ell,\alpha}(x)$  itself should reduce with increasing  $\alpha$ . Indeed, as the evaluation becomes more risk averse, the expected loss,  $I_{f_{\mathcal{D}},\ell,\alpha}(x)$ , for an estimated value  $x$ , increases. However, the asymmetry of the loss function leads to a more rapid rate of increase in  $I_{f_{\mathcal{D}},\ell,\alpha}(x)$  for larger values of  $x$ . As a result, minimizer of  $I_{f_{\mathcal{D}},\ell,\alpha}(x)$  decreases with increasing  $\alpha$ . Thus, as we increase  $\alpha$ , the output densities should shrink and/or shift towards the left. We verify this for Pareto and Gaussian densities numerically: for fixed  $\tau$ , increasing  $\alpha$  decreases the mean  $f_{\alpha}^{\star}$  for both Pareto and Gaussian  $f_{\mathcal{D}}$ ; see Figures 4 and 7. As for the variance, with fixed  $\tau$ , increasing  $\alpha$  increases the variance when  $f_{\mathcal{D}}$  is Gaussian and decreases the variance when  $f_{\mathcal{D}}$  is Pareto; see Figures 4 and 7. See also discussions in Appendices H and I.
Connection to the implicit variance model. Our results confirm that increasing  $\tau$  effectively increases the "noise" in the estimated density by moving it closer to the uniform density. The implicit variance model of [61] also captures this phenomenon. More concretely, in their model, the observed utility of the advantaged group is a Gaussian random variable with mean  $\mu$  and variance  $\sigma_0^2$ , and the observed utility of the disadvantaged group is a Gaussian random variable with mean  $\mu$  and variance  $\sigma_0^2 + \sigma^2$ . This model can be derived from our framework where the input density  $f_{\mathcal{D}}$  is Gaussian, the risk-averseness parameter  $\alpha = 1$ , the loss function is  $\ell(x, v) = (x - v)^2$  and the disadvantaged group is associated with a higher value of the resource-information parameter  $\tau$ ; see Section H for details.
Connection to the multiplicative bias model. In the multiplicative-bias model of [90], the true utility of both the groups is drawn from a Pareto distribution, and the output utility for the disadvantaged group is obtained by scaling down the true utility by a factor  $\rho > 1$ . This changes the domain of the distribution to  $[1 / \rho, \infty)$  from  $[1, \infty)$  and, hence, does not fit exactly in our model which does not allow for a change in the domain. Nevertheless, we argue that when the input density  $f_{\mathcal{D}}$  is Pareto with a parameter  $\beta$ ,  $\tau = \mathrm{Ent}(f_{\mathcal{D}})$  and  $\alpha > 1$ , and the loss function is given by  $\ell(x, v) = \ln x - \ln v$ , then output density  $f_{\alpha}^{\star}$  has a smaller mean than that of the input density. As we increase  $\alpha$ , the evaluation becomes more risk-averse and hence decreases the probability of estimating higher utility values. Hence, the mean of the output density decreases. We first show that for any fixed  $\beta$ , as the parameter  $\alpha$  increases, the output density converges to a density  $g^{\star}$ . We then show numerically that, for all the Pareto distributions considered by our study, the mean of the density  $g^{\star}$  is less than that of the output density  $f_{\alpha}^{\star}$  when  $\alpha = 1$ . We give details of this argument in Section I.
Dataset	This work	Multiplicative bias [90]	Implicit variance [61]
Vary α and τ	Fix α=1	Fix τ=Ent(fD)
JEE-2009 (Birth category)	0.09	0.15	0.21	0.14	0.10
JEE-2009 (Gender)	0.07	0.15	0.19	0.07	0.08
Semantic Scholar (Gender)	0.03	0.09	0.23	0.08	0.13
Synthetic Network	0.03	0.05	10.0	0.05	0.22
Table 1: TV distances between best-fit densities and real data (Section 4) with  $80\% -20\%$  training and testing data split: Each dataset consists of two densities  $f_{G_1}$  and  $f_{G_2}$  of utility, corresponding to the advantaged and disadvantaged groups. We fix  $f_{\mathcal{D}} = f_{G_1}$  and report the best-fit TV distance between  $f_{G_2}$  and densities output by (a) our model, (b) the multiplicative bias model, and (c) the implicit variance model. We compare our model to variants where we fix  $\alpha = 1$  and  $\tau = \mathrm{Ent}(f_{\mathcal{D}})$ . Our model achieves a small TV distance on all datasets.
Ability to capture biases in data. First, we evaluate our model's ability to output densities that are "close" to the densities of biased utility in one synthetic and two real-world datasets.
Setup and discussion. In all datasets we consider, there are natural notions of utility for individuals: scores in college admissions, number of citations in research, and degree in (social) networks. In each dataset, we fix a pair of groups  $G_{1}$  and  $G_{2}$  (defined by protected attributes such as age, gender, and race) and consider the empirical density of utilities  $f_{G_1}$  and  $f_{G_2}$  for the two groups. Suppose group  $G_{1}$  is more privileged or advantaged than  $G_{2}$ . In all datasets, we observe notable differences between  $f_{G_1}$  and  $f_{G_2}$  that advantaged  $G_{1}$  (e.g.,  $f_{G_1}$ 's mean is at least 34% higher than  $f_{G_2}$ 's).
Implementation details. Our goal is to understand whether our model can "capture" the biases or differences between  $f_{G_1}$  and  $f_{G_2}$ . To evaluate this, we fix  $f_{\mathcal{D}} = f_{G_1}$ , i.e.,  $f_{G_1}$  is the true density, and compute the minimum total variation distance between  $f_{G_2}$  and a density output by our model, i.e.,  $\min_{\alpha,\tau} d_{\mathrm{TV}}(f_{\alpha,\tau}^{\star}, f_{G_2})$ ; where  $f_{\alpha,\tau}^{\star}$  is the solution to (OptProg) with inputs  $\alpha$  and  $\tau$ . (The total variation distance between two densities  $f$  and  $g$  over  $\Omega$  is  $\frac{1}{2} \int_{\Omega} |f(x) - g(x)| d\mu(x)$  [97].) To illustrate the importance of both  $\alpha$  and  $\tau$ , we also report the TV-distances achieved with  $\alpha = 1$  (no skew) and with  $\tau = \tau_0 \coloneqq \mathsf{Ent}(f_{\mathcal{D}})$  (vacuous constraint), i.e.,  $\min_{\tau} d_{\mathrm{TV}}(f_{1,\tau}^{\star}, f_{G_2})$  and  $\min_{\alpha} d_{\mathrm{TV}}(f_{\alpha,\tau_0}^{\star}, f_{G_2})$ , respectively. As a further comparison, we also report the minimum TV distances achieved by existing models of biases in evaluation processes: the multiplicative-bias model and the implicit variance model [90, 61]. Concretely, we report  $\min_{\mu,\rho} d_{\mathrm{TV}}(f_{\mathcal{D},\mu,\rho}, f_{G_2})$  and  $\min_{\mu,\sigma} d_{\mathrm{TV}}(f_{\mathcal{D},\mu,\sigma}, f_{G_2})$  where  $f_{\mathcal{D},\mu,\rho}$  is the density of  $\rho v + \mu$  for  $v \sim f_{\mathcal{D}}$  and  $f_{\mathcal{D},\mu,\sigma}$  is the density of  $v + \mu + \sigma \zeta$  where  $v \sim f_{\mathcal{D}}$  and  $\zeta \sim \mathcal{N}(0,1)$ .
Below we present brief descriptions of the datasets; detailed descriptions and additional implementation appear in Appendix L.
Dataset 1 (JEE-2009 scores). Indian Institutes of Technology (IITs) are, arguably, the most prestigious engineering universities in India. Admission into IITs is decided based on students' performance in the yearly Joint Entrance Exam (JEE) [20]. This dataset contains the scores, birth category (official SES label [135]), and (binary) gender of all students from JEE-2009 (384,977 total) [91]. We consider the score as the utility and run two simulations with birth category ( $G_{1}$  denotes students in GEN category) and gender ( $G_{1}$  denotes male students) respectively as the protected attributes. We set  $\Omega$  as the discrete set of possible scores. We fix  $\ell_{2}^{2}$ -loss as  $f_{G_{1}}$  and  $f_{G_{2}}$  appear to be Gaussian-like (unimodal with both a left-tail and a right-tale; see Figure 13 in Appendix L).
Dataset 2 (Semantic Scholar Open Research Corpus). This dataset contains the list of authors, the year of publication, and the number of citations for 46,947,044 research papers on Semantic Scholar. We consider the total first-author citations of an author as their utility and consider their gender (predicted from first name) as the protected attribute ( $G_{1}$  denotes male authors). We fix  $\Omega = \{1,2,\ldots\}$  and  $\ell$  as the log-ratio loss as  $f_{G_1}$  and  $f_{G_2}$  have Pareto-like density.
Dataset 3 (Synthetic network data). We generate a synthetic network with a biased variant of the Barabási-Albert model [5, 17, 35, 93]. The vertices are divided into two groups  $G_{1}$  and  $G_{2}$ . We start with a random graph  $G_{0}$  with  $m = 50$  vertices where each vertex is in  $G_{1}$  w.p.  $\frac{1}{2}$  independently. We extend  $G_{0}$  to  $n = 10,000$  vertices iteratively: at each iteration, one vertex  $u$  arrives,  $u$  joins  $G_{1}$  w.p.  $\frac{1}{2}$  and otherwise  $G_{2}$ , and  $u$  forms one edge with an existing vertex  $v$  -where  $v$  is chosen w.p.  $\propto d_v$  if  $v \in G_{1}$  ( $d_v$  is  $v$ 's current degree) and  $\propto \frac{1}{2} d_v$  otherwise. We use a vertex's degree as its utility, fix  $\Omega = \{1,2,\ldots\}$ , and use log-ratio loss as  $f_{G_1}$  and  $f_{G_2}$  have Pareto-like density.
Observations. We report the TV distances for all simulations in Table 1 (also see Figure 11 and Figure 12) for the plots of the corresponding best-fit densities). We observe that across all simulations our model can output densities that are close in TV distance  $(\leq 0.09)$  to  $f_{G_2}$ . Moreover, both  $\alpha$  and  $\tau$  parameters are important, and dropping either can increase the TV distance significantly (e.g., by 1.5 times on JEE-2009 data with birth category and 2.66 times on the Semantic Scholar data). Compared to the implicit variance model, our model has a better fit on the JEE-2009 (Birth category), Semantic Scholar, and Synthetic Network data because the implicit variance model does not capture skew and in these datasets  $f_{G_1}$  is a skewed version of  $f_{G_2}$ . Compared to the multiplicative bias model, our model has a better fit on the JEE-2009 (Birth category), as here the utilities have Gaussian-like distributions due to which multiplicative bias largely has a translation effect. Finally, on the JEE-2009 (Gender) data, our model's performance is similar to multiplicative bias and implicit variance models because in this data  $f_{G_1}$  and  $f_{G_2}$  are similar (TV distance  $\leq 0.08$ )
Effect of interventions on selection. Next, we illustrate the use of our model to study the effectiveness of different bias-mitigating interventions in downstream selection tasks (e.g., university admissions, hiring, and recommendation systems).