Bias in Evaluation Processes: An Optimization-Based Model
Authors: L. Elisa Celis
Amit Kumar
Anay Mehrotra
Nisheeth K Vishnoi
Venue: NeurIPS 2023 poster (2023)
Keywords: bias
evaluation
maximum entropy
selection
TL;DR: We model biases in evaluation processes (e.g., hiring) as an optimization problem subject to an information constraint, analyze and validate the solutions of the problem, and use it to study the efficacy of bias-mitigating interventions in selection.

Abstract: Biases with respect to socially-salient attributes of individuals have been well documented in evaluation processes used in settings such as admissions and hiring. We view such an evaluation process as a transformation of a  distribution of the true utility of an individual for a task to an observed distribution and model it as a solution to a loss minimization problem subject to an information constraint. Our model has two parameters that have been identified as factors leading to biases: the resource-information trade-off parameter in the information constraint and the risk-averseness parameter in the loss function.  We characterize the distributions that arise from our model and study the effect of the parameters on the observed distribution. The outputs of our model enrich the class of distributions that can be used to capture variation across groups in the observed evaluations. We empirically validate our model by fitting real-world datasets and use it to study the effect of interventions in a downstream selection task. These results contribute to an understanding of the emergence of bias in evaluation processes and provide tools to guide the deployment of interventions to mitigate biases.

Bias in Evaluation Processes: An Optimization-Based Model
L. Elisa Celis Yale University
Amit Kumar IIT Delhi
Anay Mehrotra Yale University
Nisheeth K. Vishnoi Yale University
Abstract
Biases with respect to socially-salient attributes of individuals have been well documented evaluation processes used in settings such as admissions and hiring. We view such evaluation process as transformation of distribution of the true utility of an individual for a task to an observed distribution and model it as a solution loss minimization problem subject an information constraint, Our model has two parameters that have been identified as factors leading to biases: the resource-information trade-off parameter in the information constraint and the risk-averseness parameter in the loss function. We characterize the distributions that arise from our model and study the effect of the parameters on the observed distribution. The outputs of our model enrich the class of distributions that can be used to capture variation across groups in the observed evaluations. We empirically validate our model by fitting real-world datasets and use it to study the effect interventions in downstream selection task. These results contribute to an understanding of the emergence of bias in evaluation processes and provide tools to guide the deployment of interventions to mitigate biases.
Introduction
Evaluation processes arise in numerous high-stakes settings such as hiring, university admissions, and fund allocation decisions [20, 30, 90, 122]. Specific instances include recruiters estimating the hireability of candidates via interviews [121, 30], reviewers evaluating the competence of grant applicants from proposals [147, 18), and organizations assessing the scholastic abilities of students via standardized examinations [99, 19]. In these processes, an evaluator estimates an individual's value institution. The evaluator need not be a person, they can be committee, exam, or even a machine learning algorithm [51, 122, 145]. Moreover, outcomes of real-world evaluation processes have at least some uncertainty randomness [30, 18. 79]. This randomness arise both, due to the features of the individual (e.g., their test scores or grades) that an evaluator takes as input [31, 76, 124), as well as, due the evaluation process itself [50, 30, 140].
Biases against individuals certain disadvantaged groups have been well-documented in evaluation processes [147, 74, 104, 110, 30]. For instance, in employment decisions and peer review, women receive systematically lower competence scores than men, even when qualifications are the same [147, 110], in standardized tests, the scores show higher variance in students from certain genders [21, 112], and in risk assessment-a a type of evaluation evaluation-widely used tools were twice as likely to misclassify Black defendants being at high risk of violent recidivism than White defendants [6]. Here, neither the distribution of individuals' true evaluation depends on their socially- attributes nor is the process trying to bias evaluations, yet biases consistently arise [147, 74, 104, 110, 30]. Such evaluations are increasingly used by MI systems to learn or make decisions about individuals, potentially exacerbating inequality [51, 122, 145). This raises the question of explaining the emergence of biases in evaluation processes which is important to understand how to mitigate them, and is studied here.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Related work. A wide body of work has studied reasons why such differences may arise and how to mitigate the effect of such biases [75, 57, 54, 32, 88, 28]. For one, socioeconomic disadvantages (often correlated with socially-salient attributes) have been shown to impact an individual's 's ability to perform in an evaluation process, giving rise to different performance distributions across groups [55, 16]. Specifically, disparities in access to monetary resources are known to have a significant impact on individuals' SAT scores [55]. Moreover, because of differences between socially-salient attributes of individuals and evaluators, the same amount of resources (such as time cognitive effort) spent by the evaluator and the individual, can lead to different outcomes for individuals in different groups [64, 95, 8]. For instance, it can be cognitively more demanding, especially in coneccontrr evaluation processes, for the evaluator to interact with individuals who have a different cultural background than them, thus impacting the evaluations [95, 86, 70, 58, 142, 115, 113]. Further, such biases in human evaluations can also affect learning algorithms through biased past data that the algorithms take input [75, 57, 54, 28).
Another factor that has been identified as a source of bias is "risk averseness:" the tendency to perceive lower magnitude of increase in their utility due to profit than the magnitude of decrease in their utility due to a loss of the same magnitude as the profit [84, 144, 151]. Risk averseness is known to play role in high-stakes decisions such as who to hire, who to follow on social networks, and whether to pursue higher education [71, 23, 14]. In evaluations with an abundance of applicants, overestimating the value of an individual can lead to downstream loss (e.g.. because individual is hired or admitted) whereas under- enter--stimating may not have a significant loss [139, 65]. Thus, in the presence of risk averseness, the outputs of evaluation processes may skew the output evaluations to lower or higher values. The same skew can also arise from the perspective of individuals [13, 33, 1 11]]. For instance, when negotiating salaries, overestimating their salary can lead to adverse effects in the form of evaluators being less inclined to work with the individual or in extreme cases denying employment [13, 33]. Moreover, these costs have been observed be higher for women than for men, and are one of the prominent explanations for why women negotiate less frequently [13, 33].
A number of interventions to mitigate the adverse effects of such biases in evaluation processes have been proposed. These include representational constraints that, across multiple individuals, increase the representation of disadvantaged and minority groups in the set of individuals with high evaluations [46, 135, 131, 19, 77, 27, 116), structured evaluations which reduce the scope of unintended biases in evaluations [123, 68, 147, 15], and anonymized evaluations that, when possible, blind the decision makers to the socially-salient attributes of individuals being evaluated [72].
Mathematically, some works have modeled the outcomes of evaluation processes based on empirical observations [12, 22, 90, 61]. For instance, the implicit variance model of [61] models differences in the amount of noise in the utilities for individuals in different groups Here, the output estimate is drawn from a Gaussian density whose mean is the true utility v (which can take any real value) and whose variance depends the group of the individual being evaluated: The variance is higher for individuals in the disadvantaged group compared to individuals in the advantaged group. Additive and multiplicative skews in the outputs of evaluation processes have also been modeled [90, 22] (also
Appendix A). [90] consider true utilities v > O distributed according to the Pareto density and they model the output as v/p for some fixed p 2 1: where p is larger for individuals in the disadvantaged group. These models have been influential in the study of various downstream tasks such as selection [90, 61, 38, 129, 67, 106, 108, 29], ranking [40], and classification [28] in the presence of biases.
Our contributions. We propose a new optimization-base process transforms an (unknown) input density fp representing the true utility of an individual or a population to an observed distribution in the presence of information constraints or risk aversion. Based on the aforementioned studies and insights in social sciences. our model has two parameters: the resource-ing coouurceinformation parameter (T E R) in the information constraint and the risk-averseness parameter (a 2 1) in the objective function; see (OptProg) in Section 2. The objective measures the inaccuracy of the estimator with respect to the true density fD. and involves a given loss function a is higher (worse) for individuals in groups facing higher risk aversion. The constraint places a lower bound of T on the amount of information (about the density of the true value v) that the individual and evaluator can acquire or exchange in their interaction - 7 is higher for individuals in groups that require more resources to gain unit information. We measure the amount of information in the output density by its differential entropy. Our model builds on the maximum maximum-entropy framework in statistics and information theory [80] and is derived in Section 2 and can be viewed as extending this theory to output rich family of biased densities.
approach to model how an evaluation
l and the parameter
2

---

In Section 3. we show various properties of the output densities of our model. We prove that the solution to (OptProg) unique under general conditions and characterize the output density as a function of ÍD. l, т, and see Theorem 1. By varying the loss function and the true density, our framework can not only output standard density functions (such as Gaussian, Pareto, Exponential, and Laplace), but also their appropriate "noisy" and "skewed" versions, generalizing the models studied in [90, 22, 61]. Subsequently, we investigate how varying the parameter T affects the output density in Section 3. We observe that when F -o, there is effectively no constraint, and the output is concentrated at a point. For any fixed a, as T increases, the output density spreads-its variance and/or mean increases. We also study the effect of increasing on the output density. We observe that when the true density is Gaussian or Pareto, the mean of the output density decreases as a increases for any fixed т. Thus, individuals in the group with higher a and/or face higher noise and/or skew in their evaluations as predicted by our model.
Empirically, we evaluate our model's ability to emulate biases present in real-world evaluation processes using two real-world datasets (JEE-2009 Scores and the Semantic Scholar Open Research Corpus) and one synthetic dataset (Section 4). For each dataset, we report the total variation (TV) distance between the densities biased utilities in the data and the best-fitting densities output by our framework and earlier models. Across all datasets, we observe that our model can output densities that are close to the density of biased utilities in the datasets and has a better fit than the models of [61, 90]; Table 1. Further, well-studied bias bias- mitigating interventions: equal representation (ER) and proportional representation (PR) constraints, and two additional interventions suggested by our work: decreasing the resource- information parameter r and reducing the risk-averseness parameter o. ER and PR are constraints on the allowable outcomes, 7 can be decreased by, e.g., training the evaluators to improve their efficiency, and a can be decreased using, e.g., structured interviews [30]. We observe that for each intervention, there are instances of selection, where it outperforms all other interventions (Figure 1). Thus, our model can be used as a tool to study the effectiveness of different types of interventions in downstream tasks and inform policy; see also Appendix L.I and Appendix B.
downstream selection task, we evaluate the effectiveness of two
2
Model
The evaluation processes we consider have two stakeholders-an evaluator and an individual-along with a socictal context that affects the process. In an evaluation process, an evaluator interacts with an individual to obtain an estimate of the individual's utility or value. We assume that each individual's 's true utility e is drawn from a probability distribution. This not only captures the case that the same individual may have variability in the same evaluation (as is frequently observed in interviews, examinations, and peer-review [31, 50, 76, 124, 30, 140, 18]) but also the case that v corresponds to the utility of an individual drawn from population. For simplicity, we consider the setting where e real-valued and its density is supported on a continuous subset Ω C R. This density gives rise to distribution over Ω with respect to the Lebesgue measure μ over E. For instance. P could be the set of all real numbers R, the set of positive real number Ry0, an open interval such as [1, oo), or closed interval [a, b). Following prior work modeling output densities [90, 40, 61], we assume that the true utility of all individuals is drawn from the same density fp.
We view an evaluation process as a transformation of an (unknown) true density fo into an observed density fe over Q. In real- world evaluation processes, this happens through various means: by processing features of an individual (e.g., past performance on exams or past employment), through interaction between the evaluator and the individual (e.g., in oral examinations), or by requesting the individual to complete an assessment or test [121, 99, 19]. We present an optimization- based model that captures some of the aforementioned scenarios and outputs fe. The parameters of this model encode factors that may be different for different socially-salient groups, thus, making fc group dependent even though fo is not group dependent. We derive our model in four steps.
Step I: Invoking the entropy maximization principle. In order to gain some intuition, consider a simple setting where the utility of an individual is a fixed quantity v (i.e., fo is Dirac-delta function around v). We first need to define an error or loss function é: Ω x Ω - R; given guess z of v, the loss function E(x, v) indicates the gap between the two values. We do not assume that E is symmetric
but require f(rr v) 2 0 when 2 v. Some examples of E(E, v) are (r r)², 1 vl 2/v. and (x//v). The right choice of the loss function can be context-dependent, e.g. (a - -r)² is commonly
used loss function for real-valued data, and In(x/v) )) is sometimes better at capturing relative error for
3

---

heavy-tailed distributions over positive domains [83]. For a density f for x; Be-, ((2, v) denotes the expected error of the evaluation process. One can therefore consider the following problem: Given a value A, can we find an I such that Ryz ((x v) I A? This problem is under-specified as there may be (infinitely) many densities I satisfying this constraint. To specify f uniquely, we appeal to the maximum entropy framework in statistics and information theory [81]: Among all the feasible densities, one should select the density f which has the maximum entropy. This principle leads to the selection of a density that is consistent with our constraint and makes no additional assumption. We use the notion of the (differential) entropy of a density with respect to the Lebesgue measure μ on R:
Eft == - -zZen [(x) In (())d)(x))
where f(x) In f(x) = 0 whenever f(x) = 0. Thus, we get the following optimization problem:
argmax,: f: density on 5 Ent(f), s.t., Egn ((x, u) s A.
This optimization problem is well-studied and it is known that by using different loss functions, we can derive many families of densities [102. 148]. For instance, for (zx v) (æ: - r)² we recover the Gaussian density with mean v, and for ((z, v) == In(x/v), we obtain a Pareto density.
Step 2: Incorporating the resource-informatio parameter. We now extend the above formulation to include information constraints in the evaluation process. In an evaluation process, both the evaluator and the individual spend resources such as time, cognitive effort, or money to communicate the information related to the utility of the individual to the evaluator. For instance, in interviews, both the interviewer and the interviewee spend time and cognitive effort. In university admissions, the university admissions office needs to spend money to hire and train application readers who, in turn, screen applications for the university's 's admission program, and the applicants need to spend time, cognitive effort, and money to prepare and submit their applications [24, 150]. The more resources are spent in an evaluation process, the more additional information about v is acquired. We model this using a innourcceifformation parameter т. which puts a lower bound on the entropy of f- Thus, we modify the optimization problem in Equation (2) in the following manner. We first flip the optimization problem to an equivalent problem where we minimize the expected loss subject to a lower bound on the entropy, Ent(ʃ), of f. A higher value of the resource-information parameter means that one needs to spend more resources to obtain the same information and corresponds to a stronger lower bound on Ent(f) (and vice-versa) in our framework.
argmin f: density Q
E [(x,, u), s.t., Ent(f) 2
When - -oo, the optimal density tends to a point or delta density around v (recovering the most information). and when t - oc, it tends to a uniform density on Z (learning nothing about v). Since differential entropy can vary from negative to positive infinity, the value of 7 is to be viewed relative to an arbitrary reference point. T may vary with the saaially-salient group of the individual in real-world contexts. For instance, in settings where the evaluator needs to interact with individuals (e.g., interviews), disparities can arise because it is less cognitively demanding for an evaluator to communicate with individuals who speak the same language as themselves, compared to individuals who speak a different language [95]. In settings where the evaluator assesses individuals based on data about their past education and employment (e.g., screening stages of hiring or in university admissions), disparities can arise because the evaluator is more knowledgeable about a specific group's sociocultural background compared to others, and would have spend more resources to gather the required information for the other groups [49, 62].
Step 3: Incorporating the risk-averseness parameter. We now introduce the parameter that captures risk averseness. Roughly speaking, risk averseness may arise in an evaluation process because of the downstream impact of the output. The evaluator may also benefit or may be held accountable for the estimated value, and hence, would be eager or reluctant to assign values much higher than the true utility [71, 23, 14]. Further, the individual may also be risk averse, e.g. during hiring interview, the risk of getting rejected may prompt the individual to quote less than the expected salary [13, 33, 111]. To formalize this intuition, for given f, we define a risk-averse loss function la : Ω x Ω + R that incorporates the parameter a 2 1 in l as follows:
(a, == a f(z, v) if a 2 e and (a u) = ((x, if < v.
Not only does this loss function penalize overestimation versus underestimation, but in addition, the more the overestimation, the more the penalization is. This intuition is consistent with the theory

---

of risk averseness [10, 119]. Our choice is related to the notion of hyperbolic absolute risk aversion [78, 109]. and one may pick other ways incorporate risk averseness in the loss function [144. 98]. As an example, if {(x, v) (x r)² then the fg-loss is la (2, (2,,0) v) = a (x - -) if x 2 v and (x - 2) otherwise. If f( v) = In (z//e), then the log-ratio loss is (a v) = a In (x/v), if 2 v;
In (z//v) otherwise. Plots of these two loss functions are in Figure 2. Thus, the analog of Equation (3) becomes
argming; density f E la(x U. v,2), s.t., Ent(f) 2T.
(5)
We note that, for the risk-averse loss function defined in (4), the following hold: For all a 2 a' 2 1, la (z, v) 2 lar (z, v) for all z,, v 92 and (a u) loy (z. v) is increasing function of for 2 Beyond (4), one could consider other fa u) satisfying these two properties in our framework; we omit the details. One can also incorporate the (opposite) notion of "risk eager," where values of lower than e are penalized more as opposed to values of x higher than e by letting a E (0, 1].
Step 4: Generalizing to arbitrary ÍD- To extend Equation (5) to the setting when v comes from general density fD. we replace the loss function by its expectation over fo and arrive the model for the evaluation process that we propose in this paper:
argmin f: density on 52 ErTz,a (fv. f) = Suea Ueen (((,,))(())()))))) fo(o)du(o), (OptProg) such that - Seen f(x) log f(x (x)dp(x) T
For given l and parameters a and this optimization framework can be viewed transforming the true utility density fo of a group of individuals to the density fE (the solution to this optimization problem). It is worth pointing out that neither the evaluator nor the individual is solving the above optimization problem rather (OptProg) models the evaluation process and the loss function E, a, and r depend on the socially-salies attribute of the group of individual; see also Appendix B.
3 Theoretical results
Characterization of the optimal solution. We first characterize the solution of the optimization problem (OptProg) in terms of fp. la. T, and a. Given a probability density fD. a parameter a 2 1, and loss function f, consider the function ff (x) == J. en fa(z, (v)dpu)) This integral captures the expected loss when the estimated utility is x. Further, for density f. the objective function of (OptProg) can be expressed as Erre, (fD, f) == Jaen (())
Theorem 3.1 (Informal version of Theorem C.1 in Appendix C). Under general conditions on fo and E. for any finite and a 2 (OptProg) has unique solution f*(x) x exp (- (z)/7*). where 7" > 0 is unique and also depends on a and t. Further, Ent(ʃ*)= =T.
The uniqueness in Theorem 3.1 implies that, if a 1, t Ent(fp), and ((x, v) = In fo(x) In (w), then the optimal solution is (z) fz) To see this, note that in this case / (z) In (z) + (f)(f Hence, f' = fD(z) satisfies Theorem 3.1 I'' conclusion (see Section C.7 for details). Thus, in the absence of risk averseness, and for an appropriate choice of assource-informant parameter, the output density is the same as the true density.
Theorem 3.1 can be viewed as a significant extension of results that show how well-known proba- bility distributions arise as solutions the catropy-maximizann maximum-entropy formulation only considers the setting where the input utility is given by a single value, i.e., the distribution corresponding fo is concentrated averseness parameter restricted to the class of well-knoww loss functions, e.g. E2 f2-loss or linear loss, can be understood by using standard tools from convex optimization (see [48]), characterizing the optimal solution to the general formulation (OptProg) is more challenging because of several reasons: (i) The input density fo need not be concentrated at single point, and hence one needs to understand conditions on fo when the formulation has a unique optimal solution. (ii) The loss function can be arbitrary and one needs to formulate suitable conditions on the loss function such that (OptProg) has unique optimal solution. (iii) The -vkeeversssssss parameter a makes the loss function asymmetric (around
framework. Indeed, the standard
single point; and the risk- framework (2)
1. While the optimal solution to the maximum-entree
A variation of (4) that we use in the empirical part is the following: For fixed "shift" vo. let (2, v) = [(z, + vo) if x 2 and La (2, v) == [(.r v + vo) x ++0 vo.
5

---

any fixed value u) and makes the analysis of the error in the objective function non-trivial. Roughly speaking, the only restrictions. other than standard integrability assumptions. that we need on the input are: (a) Monotonicity of the loss function E(x, v) with respect to either a: v. (b) the growth rate of the loss function f(x,, 2) is at least logarithmie, and (c) the function 7 ffp.fa(a) (2) has a unique global minimum, which is a much weaker assumption than convexity of the function. Note that for the Eg-loss function given by f(1, v) = (a - u)², the first two conditions hold trivially; and when the input density fo is Gaussian, it is not hard to show that fp,.. (2) is strongly convex and hence the third condition mentioned about holds (see Appendix H for details). These conditions are formally stated in Appendix C.1 and we show that they hold for the cases of Gaussian, Pareto, Exponential, and Laplace densities in Sections H, J, K respectively.
The proof of Theorem 3.1 is presented in Appendix C and the following are the key steps in it: (i) The proof starts by considering the dual of (OptProg) and shows that strong duality holds (see Appendix C.2 and Appendix C.3). (ii) The next step is to show that the optimal solution f* of (OptProg) exists and unique. This requires proving that the dual variable 7 (corresponding to the entropy constraint in (OptProg)) is positive while this variable is always non-negative, the main technical challenge is to show that it is non-zero. In fact, there are instances of (OptProg) where y" is zero, and an optimal solution does not exist (or an optimal solution exists, but is not unique). (iii) The proof of r + 0 requires us to understand the properties of the integral f.ptt (z) (abbreviated as [(x) when the parameters fD. E,
clear from the context). In Appendix C.4 we show that (xx) can be expressed as a sum of two monotone functions (see Theorem C.10). This decomposition allows us to show that the optimal value of (OptProg) is finite. (iv) In Appendix C.5, we show that the optimal value of (OptProg) is strictly larger than (x=*) (Lemma C. C.13, Lemma C. 14), where x is the minimizer of (xx). This requires the expected loss function and the entropy of a density as we place probability mass away from x". Indeed, these technical results do not hold true if the loss function [a v) grows very slowly (as function of /u or (|= vl). (v) Finally, in Theorem C.15, we show that 7* is nonzero. This follows from the fact that if 7* O, then the optimal value of (OptProg) is equal to I [xx*), which contradicts the claim in (iv) above. Once we show y" > O, the expression for the (unique) optimal solution, i.c., f (z) x exp (-I(z)/>²) )) follows from Theorem C.16.
to understand the interplay between the growth rate of
We conclude this section with two remarks. 1) In Appendix E, we show that Theorem 3.1 implies that == (Frry,, (fD. fr///'///" //*) + In Z+, where Z == So exp (=))//(((x///>)) du(x:)
is the partition function or the normalizing constant that makes f a probability density; This equation is an analog of the Gibbs equation in statistical physics and gives a physical interpretation of (.) and y*: y is the temperature and [fp.t.((I) is the energy corresponding to state x. This may be useful in understanding the effects of different parameters on the output density. 2) If one wishes, one can use (OptProg) to understand the setting where a single individual is being evaluated by setting the input density fo to be concentrated at their true utility v. For instance, if we set the loss function to be the & lo-soss, using Theorem 3.1, one can show that for any given values of the parameters, a and
and loss function being Ez-loss, the mean of the output density is v - V -
Va see Appendix D) for a proof. Therefore for a 1, the mean of the output density is strictly less than u. This gives a mapping from the "true ability" to the (mean of the) "biased ability" in this case. This mapping can be used to understand how the parameters T and a in the evaluation process transform the true ability.
Effect of varying r for a fixed a. We first study the effect of changing the resource-information
parameter on f* for fixed value of the risk-averseness parameter a 2 1. To highlight this dependency on r, here we use the notation ft to denote the optimal solution f We start by noting that as 7* increases, the optimal density becomes close to uniform, and as it goes towards zero the optimal density concentrates around a point x= that minimizes energy: argmingeo ff (z). Note that the point x= does not depend on 7. However, 7 may depend in a complicated manner on both and a, and it is not apparent what effect changing 7 has on y² We show that, for any fixed a 2 1, as r decreases, the output density gets concentrated around x* (see Theorem F.1). This confirms the intuition that as we reduce T by adding more resources in the evaluation process, the uncertainty in the output density should be reduced. Similarly, if 7 increases because of reduction in the resources invested in the evaluation, the uncertainty in f; should increase, and hence, the output density should converge towards uniform density. For specific densities, one can obtain sharper results. Consider the case when fD is Gaussian with mean and variance o², and Lo (x, v) == a(x v)² if x 2 and (x - 2)2 x < U. The uncertainty in the Gaussian density is captured by the variance, and hence, we expect the output density to have a higher variance when the parameter 7 is increased. Indeed,

---

when a 1, we show that the optimal density f? is a Gaussian with mean m and variance e²ro?; see Appendix H. Thus, one increases 7 from -00 to oo, the variance of the output density changes monotonically from 0 to co. When a 2 1, numerically, it can be seen that for any fixed 2 1, increasing 7 increases the variance, and also decreases the mean of f see Figure 3. Intuitively, the decrease in mean occurs because higher variance increases the probability of the estimated value being much larger than the mean, and the risk-averseness parameter imposes a high penalty when the estimated value is larger than the true value. In fact, we show in Theorem F.5 that the variance of fü for any continuous input density fo supported on IR is at least L -22 This follows from the 2n
well-known fact that among all probability densities supported on R with variance o². the Gaussian density with variance o² maximizes the (differential) entropy [120]; see Section F.2 for details. In similar vein, we show that the mean of f is at least e- when the input density S'D is supported on [0, oo), and hence approaches 0o as 7 goes oo (see Theorem F.7). This result relies the fact that among all densities supported on [0, oo) and with a fixed expectation 1/A (for À > 0), the one maximizing the entropy is the exponential density with parameter À [48].
We now consider the special setting when the input density /D is Pareto with parameter / > 0 (fx(x)= : 3x 5-1 for x E [1, )), and (z, v) == a In(2r/v) if x 2 v. and In(x/v) if x < When a = 1, we show that the optimal density f: is also a Pareto density with parameter 3 satisfying the following condition: 1 1+ (1/8*) In 3
Using this, it can be shown that, for a = 1, both the mean and variance of f= monotonically increase to Do as T goes to oo; see Appendix I. The increase in variance reflects the fact that increasing r increases the uncertainty in the evaluation process. Unlike the Gaussian case, where the mean of f: could shift to the left of 0 with an increase in r, the mean of the output density in this setting is constrained to be at least 1, and hence, increasing the variance of f also results in an increase in its mean. Numerically, for any fixed a 2 1, increasing I increases both the mean and variance of f see Figures 3 and 6.
Effect of varying a for a fixed т. Let fa denote the optimal solution I with a for a fixed т. We observe that, for any fixed x, Ifp.t.aaaaa (z)), which is the expected loss when the output is z, and atre,a(ft fD) are increasing functions of a; see Appendix G. Thus, intuitively, the mass of the density should shift towards the minimizer of ( Moreover, the minimizer of (z) itself should reduce with increasing a. Indeed, as the evaluation becomes more risk averse, the expected loss, I (,,,,,a()), for estimated value x. increases. However, the asymmetry of the loss function leads to a more rapid rate of increase in Ifo,,,((z) (x) for larger values of x. As a result, minimizer of [x) decreases with increasing a. Thus, as we increase a, the output densities should shrink and/or shift towards the left. We verify this for Pareto and Gaussian densities numerically: for fixed T, increasing a decreases the mean fä for both Pareto and Gaussian fD; see Figures 4 and 7. As for the variance, with fixed t, increasing a increases the variance when fo is Gaussian and decreases the variance when fo Pareto; see Figures and See also discussions in Appendices H and
Connection to the implicit variance model. Our results confirm that increasing T effectively increases the "noise" in the estimated density by moving closer to the uniform density. The implicit variance model of [61] also captures this phenomenon. More concretely, in their model, the observed utility of the advantaged group is Gaussian random variable with mean j and variance oz and the observed utility of the disadvantaged group is a Gaussian random variable with mean μ and variance of + o². This model can be derived from our framework where the input density fD is Gaussian, the risk-averseness parameter a = 1, the loss function is E(x, v) = (x - v)² and the disadvantaged group is associated with higher value of the issorrce-ifformation parameter 7; see Section H for details.
Connection to the multiplicative bias model. In the multiplicative-bia model of [90], the true utility of both the groups is drawn from a Pareto distribution, and the output utility for the disadvantaged group is obtained by scaling down the true utility by a factor p > 1. This changes the domain of the distribution to [1/p, oo) from [1, 0o) and, hence, does not fit exactly in our model which does not allow for a change in the domain. Nevertheless, we argue that when the input density fo is Pareto with parameter 3, T Ent(fp) and > 1, and the loss function is given by E(x, v) = In In v, then output density S has a smaller mean than that of the input density. As we increase o, the evaluation becomes more risk-averse and hence decreases the probability of estimating higher utility values. Hence, the mean of the output density decreases. We first show that for any fixed (3, as the parameter a increases, the output density converges to a density g". We then show numerically that, for all the Pareto distributions considered by our study, the mean of the density g" is less than that of the output density f* when a 1. We give details of this argument in Section I.
7

---

Dataset
This work
Multiplicative bias [90]
Implicit variance [61]
Vary a and
Fix a=1
Fix Entt((f))
JEE-2009 (Birth category)
0.09
0.15
0.21
0.14
0.10
JEE-2009 (Gender)
0.07
0.15
0.19
0.07
0.08
Semantic Scholar (Gender)
0.03
0.09
0.23
0.08
0.13
Synthetic Network
0.03
0.05
10.0
0.05
0.22
TV distances between best-fit densities and real data (Section 4) with 80%-20% training and testing data split: Each dataset consists of two densities fa, and fa,, of utility, corresponding to the advantaged and disadvantaged groups. We fix fD=fa, and report the best-fit TV distance between fa, and densities output by (a) our model, (b) the multiplicative bias model, and (c) the implicit variance model. We compare our model to variants where we fix a=1 and E=Ent(DD) Our model achieves a small TV distance on all datasets.
Table 1:
Empirical results
Ability to capture biases in data. First, we evaluate our model's 's ability to output densities that are "close" to the densities of biased utility in one synthetic and two real-world datasets.
Setup and discussion. In all datasets we consider, there are natural notions of utility for individuals: scores in college admissions, number of citations in research, and degree in (social) networks. In each dataset, we fix a pair of groups G₁ and G2 (defined by protected attributes such as age, gender, and race) and consider the empirical density of utilities fc, and fa, for the two groups. Suppose group G1 is more privileged or advantaged than Gz. In all datasets, we observe notable differences between fc, and fc, that advantaged 11 (e.g., fa, 's mean is at least 34% higher than fc, 's).
Implementation details. Our goal is to understand whether our model can "capture" the biases or differences between fc, and fG.- To evaluate this, we fix fo = Sc., i.e., Sc, is the true density, and compute the minimum total variation distance between Sa, and density output by our model, i.e., millerr dTv (fār where fä.- is the solution to (OptProg) with inputs and
T. (The total variation distance between two densities f and g over is 2 Jo f(x) g(x)) du(z) [97].) To illustrate the importance of both a and т, we also report the TV-distances achieved with 1 (no skew) and with r = To Ent(fp) (vacuous constraint). i.e., min, drv (ff,,.fa)
and min, drv : respectively. As a further comparison, we also report the minimum TV distances achieved by existing models of biases in evaluation processes: the multiplicative-bian model and the implicit variance model [90, 61]. Concretely, we report minn,p drv (JD.p.or Ja,) and min,,, drv (JD.p..o, (G,) where fD. is the density of pe + μ for v I and D is the density of e + μ + of where ~ fo and ( ~ (0,0
Below present brief descriptions of the datasets; detailed descriptions and additional implementa- tion appear in Appendix L.
Dataset I (JEE-2009 scores). Indian Institutes of Technology (IITs) are, arguably, the most prestigious engineering universities in India. Admission into IITs decided based on students' performance in the yearly Joint Entrance Exam (JEE) [20]. This dataset contains the scores, birth category (official SES label [135]), and (binary) gender of all students from JEE- (384,977 total) [91]. We consider the score as the utility and run two simulations with birth category (G1 denotes students in GEN category) and gender (G, denotes male students) respectively as the protected attributes. We set Ω as the discrete set of possible scores. We fix lg-loss as fa, and fc, appear to be Gaussian-like (unimodal with both a left-tail and a right-tale; see Figure 13 in Appendix L).
Dataset (Semantic Scholar Open Research Corpus). This dataset contains the list of authors, the year of publication, and the number of citations for 46, 944,444 research papers on Semantic Scholar. We consider the total first-author citations of an author as their utility and consider their gender (predicted from first name) as the protected attribute (G, denotes male authors). We fix Ω = ) and l as the log-ratio loss as fc, and fG, have Pareto-like density.
Dataset 3 (Synthetic network data). We generate synthetic network with a biased variant of the Barabási-Albed Albert model [5, 17, 35, 93]. The vertices are divided into two groups G₁ and Gg. We start with random graph Go with m=50 vertices where each vertex is in G w.p. 1/2 independently. We extend Go to 1=10, 000 vertices iteratively: at each iteration, one vertex u arrives, u joins G1 w.p. 2 and otherwise G2, and forms one edge with existing vertex e-where v is chosen w.p. xx d if V E G1 (d,, is v's current degree) and x 1 dd otherwise. We use a vertex's 's degree as its utility, fix Ω {1, 2, ... and use latioatio loss fc, and fc, have Pareto-like like density.
8

---

Observations. We report the TV distances for all simulations in Table (also see Figure 11 and Fig- ure 12) for the plots of the corresponding best-fit densities). We observe that across all simulations our model can output densities that are close in TV distance (S 0.09) to faz- Moreover, both and parameters are important, and dropping either can increase the TV distance significantly (e.g.. by 1.5 times on JEE-2009 data with birth category and 2.66 times on the Semantic Scholar data). Compared to the implicit variance model, our model has a better fit on the JEE-2009 (Birth category), Semantic Scholar, and Synthetic Network data because the implicit variance model does not capture skew and in these datasets fc, is a skewed version of fG.. Compared to the multiplicative bias model, model has better fit on the JEE-2009 (Birth category), as here the utilities have Gaussian like distributions due to which multiplicative bias largely has a translation effect. Finally, on the JEE- 2009 (Gender) data, model's 's performance is similar to multiplicative bias and implicit variance models because in this data Sa. and fa, are similar (TV distance
Effect of interventions on selection. Next, we illustrate the use of our model to study the effectiveness of different bias- mitigatigg interventions in downstream selection tasks (e.g., university admissions, hiring, and recommendation systems).
Subset selection tasks. There are various types of selection tasks [59, 4, 30J. We consider the simplest instantiation where there are n items, each item i has a true utility v, 2 O, and the goal to select size-k subset Sp maximizing Lies U4- If V= Un) is known, then this problem is straightforward: select & items with the highest utility. However, typically V is unknown and is estimated via a (human or algorithmic) evaluation process that outputs a possi- bly skewed/noisy estimate X X=(x1, Fn. of V [125, 152, 110, 1 118, 36]. Hence, the outputs is Eies Fii which may be very different from So and possible has much lower true utility: Eiese is < Eiesp Vr.
Interventions to mitigate bias. Several interventions have been proposed to counter the adverse effects of bias in selection, including, representational constraints, structured interviews, and interviewer training. Each of these interventions tackles different dimension of the selection task. Represen- tational constraints require the selection to include at least a specified number of individuals from unprivileged groups [30, 37, 135]. Structured interviews reduce the scope of unintended skews by requiring all interviewees to receive the same (type of) questions [30, 123, 68). Interviewer training aims to improve efficiency: the amount of (accurate) information the interviewer can acquire in given time [30]. Which intervention should a policymaker enforce?
Studying the effectiveness of interventions. A recent and growing line of work [90, 40, 28, 61, 67, 38, 107] evaluates the effectiveness of representational constraints under specific models of bias: they ask, given C of subsets satisfying some constraint, when does the constraint optimal set, Se.c == argmaxsec Eies Z.,, have a higher utility than the unconstrained optimal Sg, i.e., when is Liese,c 2/4 > Liese ?,? Based on their analysis [90, 40, 28, 61, 67, 38, 107] demonstrate the benefits of different constraints including, equal representation (ER), which requires the output S to satisfy S = JS n G2| and, proportional representation (PR), which requires S to satisfy |/GG. = ISmGal/iaa Gal. A feature of our model is that its parameters a and T have a physical interpretation, which enables the study of other interventions: for instance, structured interviews aim to reduce skew in evaluation, which corresponds to shifting a closer to I, and interviewer-training affects the information- cormmatton-to-resurree trade-off, i.e., reduces T.
Using our model we compare ER and PR with two new interventions: change a by 50% (a- intervention) and change 7 by 50% (r-intervention). Here, 50% is an arbitrary amount for illustration.
Setup. We consider a selection scenario based the JEE 2009 data: fix fn fc,, a', T' to be the best-fit fit parameters on the JEE 2009 data (by TV distance), and GG1 = 1000. Let far and fr be the densities obtained after applying the alpha intervention and the tau intervention respectively. (Formally, fa fa= /2.7' and f== 3r 3r//2-) We vary |G2| E (500, 1000, 1500]. For each i E G1, we draw 0; ~ fo and set 24 = v, (no bias). For each i E G2, we draw ~ SDD I3 ~ Sā xt ~ sar and x. fr coupled so that the CDFs of the respective densities U/ 2y. 2², and x are the same. We give ER and PR the utilities {z,); as input, we give a-ineervention utilities {2 }, as input, and the r-intervention utilities {27}.- For each [G] and [G2). we vary 50 < k S 1000, sample utilities and report the expected utilities of the subset output by each intervention over 100 iterations (Figure 1).
Here, 1000 the largest value for which ER satisfiable across all group sizes |G] and G2/-

---

Reduce skew (la- 11)
Decrease
Equal representation
Proportionnl representation
Relaive Rereowe unuiity
Reclaive iocresse utiiky
Relative incresse untiky
1.4
2.4
1.2
1.2-
1.2
1.0
1.0
200
800
1000
100
1000
(a) |G1] = 1000 and |G2] = 500 (b) |Gi| 1000 and [G2) = 1000 (c) |Gi| 1000 and [Gal = 2000
Figure 1: Effectiveness of different interventions on the selection-utiliyyas estimated by our model: The ar-xis shows (size of selection) and the y-axis shows the ratio of the (true) utility of the subset output with an intervention to the (truc) utility of the subset output without any intervention. The main observation across the figures is that, for each intervention, there is choice of k. and group sizes [G', and [G2]. where the intervention outperforms all other interventions. Hence, each intervention has a different effect on the latent utility selection and a policymaker can use our model to study their effect and decide which intervention to enforce. Error bars represent the standard error of the mean over 100 repetitions.
Observations. Our main observation is that there is no pair of interventions such that one always achieves a higher utility than the others. In fact, for each intervention, there is a value of k. |G1l, and [G2], such that the subset output with this intervention has higher utility than the subsets output with other interventions. Thus, each intervention has a very different effect on the latent utility of the selection and policymaker can use model to study the effects in order to systematically decide which interventions to enforce; see Appendix L.I for a case study of how policymaker could potentially use this model to study bias-mitigating interventions in the JEE context.
Conclusion, limitations, and future work
We present a new optimization- approach to modeling bias in evaluation processes ((OptProg)). Our model has two parameters, risk averseness a and resource-informatin trade-off т, which are well documented to lead to evaluation biases in a number of contexts. We show that it generate rich classes of output densities (Theorem 3.1) and discuss how the output densities depend on the two parameters (Section 3). Empirically, we demonstrate that the densities arising from our model have a good fit with the densities of biased evaluations in multiple real-world datasets and a synthetic dataset; often, leading model as a tool to evaluate different types of bias-mitigating interventions in a downstream selection aask-illustrating how this model could be used by policymakers to explore available interventions (Figure I and Appendix L.)); see also Appendix B. Our work relies on the assumptions in prior works that there are no differences (at population level) between G1 and G2; see, e.g., [89, 61, 40J. If this premise is false, then the effectiveness of interventions can be either underestimated or overestimated which may lead a policymaker to select a suboptimal intervention. That said, if all the considered interventions reduce risk aversion and/or resource constraints, then the chosen intervention should still have a positive impact on the disadvantaged group. Our model can be easily used to study multiple socially-salient groups by considering a group-specific risk-aversion parameter and group-specific information constraint. For example, if two groups G1, G2 overlap, then we can consider three disjoint subgroups G1 n G2, G. \G2 and G2 G1. Our model of evaluation processes considers scenarios where candidates are evaluated along a single dimension. It can also be applicd - in dimension-by-dimetd
better fit than models of prior works [90, 61] (Table 1). We use our
fashion to scenarios where individuals are evaluated along multiple dimensions, but the evaluation in any dimension is independent of the evaluation in other dimensions. Modeling evaluation processes involving multiple correlated dimensions is an interesting direction. While we illustrate the use of our model in a downstream selection task, utilities generated from biased evaluation processes are also used in other decision-male making tasks (such as regression and clustering), and studying the downstream impact of evaluation biases on them is an important direction. Moreover, the output of or model can be used by policymakers supermodular set aggregation setting, where the utility of the selected group is more than the sum of the individuals. Our model cannot be directly used to understand the effect of interventions in the long term. Additional work would be required to do so, perhaps as in [38], and would be an important direction for future work. Finally, any work on debiasing could be used adversarially to achieve the opposite goal. We need third-party evaluators, legal protections, and available recourse for affected parties- crucial components of any system though beyond the scope of this work.
assess the impact of interventions in the
10

---

Acknowledgments.
This project is supported in part by NSF Awards CCF-2112665 and IIS-
2045951.
References
[1] Is Income Implicit in Measures of Student Ability? https://budgttmmlel...o whattnr up enn. htt.p//...a.aaaao////aaaaaaaaaaa an 11/2200mcmm//accros///atttttss 2021.
Supreme Court guts affirmatiiieeeeeeeee a affecttvii, ending race-conscious admissoos. https:
http://www.....oo.//.o/////////a ecision, 2023.
[3] Social Security Administration, Beyond the Top 1000 Names, 2018. https://wwx ssa. ssa.go v/ //aat/bbbyynmee//mmmitt html.
[4] Rakesh Agrawal, Sreenivas Gollapudi, Alan Halverson, and Samuel Ieong. Diversifying search results. In Proceedings of the Second ACM International Conference on Web Search and Data Mining, WSDM '09, page 5-14, New York, NY, USA, 2009. Association for Computing Machinery.
[5] William Aiello, Fan R. K. Chung, and Linyuan Lu, A random graph model for massive graphs. STOC, pages 171-180. ACM, 2000.
[6] Julia Angwin, Madeleine Varner, and Ariana Tobin. Machine bias: Facebook enabled advertis- ers reach 'jew haters'. ProPublica, Sept, 2017.
[7] Another PIL seeks entrance tests in Gujarati, October 2011. http::://ebb..rrhiiiiii org/ web/201201 web/20122100612134/hhttt //rtiiless....s aftps://///tttceeeeeeeeeeeooeed indiiiiiieestt
2tmp:/aaaaaiee.ccm1011111111111 com/2011-1 0 2 1eter__eeeettttteeeeeeeee centts-regooall-- laaggaggggggaaaattttt.
simesofindia
anguage.
[8] Michael Argyle. The psychology interpersonal behaviour. Penguin UK, 1994.
[9] Arora, Hazan, and S. Kale. The multiplicative weights update method: meta-algorithm and applications. Theory of Computing, 8(1):121-164, 2012.
[10] Kenneth J. Arrow. Aspects the Theory of Risk-Bearing. YrjöJahnsson lectures. Yrjö- Jahnssonin Säätiö, Helsinki, 1965.
[11] Kenneth J. Arrow. The Theory of Discrimination, pages 1-33. Princeton University Press, Princeton, 1974.
[12] Kenneth J. Arrow. What has economics to say about racial discrimination? Journal of Economic Perspectives, 12(2):11-000, June 1998.
[13] L. Babcock and S. Laschever. Why Women Don't Ask: The High Cost of Avoiding Negotiations And Positive Strategies for Change. Little, Brown Book Group Limited, 2009.
[14] Stijn Baert. Hiring gay man, taking risk?: A lab experiment employment discrimination and risk aversion. Journal of Homosexuality, 6015) 2018. PMID: 28841095.
[15] Katherine Baldiga. Gender Differences in Willingness to Guess. Management Science, 60(2):434 448, 2014.
[16] British Business Bank and Oliver Wyman. Alone together: Entrepreneurship and diversity in the UK, October 2020. https: //wp::///www wwww british- bank. htt//ppconneente the-U
(etads/222001//Almnna www.loneettgettee---eeeeeee htt.reeeeetttrrrrerrrriiiiiaaaee K-FINAL.pdf.
[17] Albert-LLászie Barabási and Réka Albert. Emergence of scaling in random networks. Science, 286(5439):509-51.. 1999.

---

[18] Hannah Bast. How Objective is Peer Review?, November 2020. https: acm. org/b togs/blog-a wttp:/cacmmmmmeoooeeoeo how-objective -or-----rrrrrrri/ttttaaoo..
[19] Surender Baswana, Partha P Chakrabarti, V Kamakoti, Yash Kanoria, Ashok Kumar, Utkarsh Patange, and Sharat Chandran. Joint seat allocation: An algorithmic perspective, 2015.
[20] Surender Baswana, Partha Pratim Chakrabarti, Sharat Chandran, Yashodhan Kanoria, and Utkarsh Patange. Centralized Admissions for Engineering Colleges in India. In Proceedings of the 2019 ACM Conference on Economics and Computation, EC '19, page 323-324, New York, NY, USA, 2019. Association for Computing Machinery.
[21] Ariane Baye and Christian Monseur. Gender differences in variability and extreme scores in an international context. Large-scale Assessments in Education, 4(1):1, 2016.
[22] Gary S. Becker. The Economics of Discrimination. Economic Research Studies. University of Chicago Press, 2010.
[23] Christian Belzil and Marco Leonardi. Risk aversion and schooling decisions. Annals of Economics and Statistics, pages 35-70, 2013.
[24] Sarah Berger. This high school senior says he is spending $1,700 on college applications, January 2018. https://// //www//wwwwww//w wwwp:wwnbbcooooo http://o////////////////// price- of coll-oolleee-aeee appli cations. html.
[25] Marianne Bertrand and Esther Duflo. Chapter 8 field experiments on discriminatiinnalluura stilwell and jan zilinsky provided excellent research assistance. we thank abhijit banerjec for comments, are particularly grateful to betsy levy paluck, our discussant, for her detailed and thoughtful review of an earlier draft. In Abhijit Vinayak Banerjee and Esther Duflo, editors, Handbook of Field Experiments, volume of Handbook Economic Field Experiments, pages 309-393. North Hoth-Holland, 2017.
[26] Deepti Bhaskaran. CBSE's 'Udaan' initiative helping girl students crack IIT entrance exam. Mint, August 2017. htttps://www.iii
/ //ww. -aaa
IIT-e html.
[27] Scott Bland. Schumer to Introduce Rules for Diverse Senate Hiring. Politico, 2017. https:
477.
[28] Avrim Blum and Kevin Stangl. Recovering from biased data: Can fairesss constraints improve accuracy? In FORC, volume 156 of L/Plcs, pages 3:1-3:20. Schloss Dagstuhl Leibnni- Zentrum für Informatik, 2020.
[29] Niclas Boehmer, L. Elisa Celis, Lingxiao Huang, Anay Mehrotra, and Nisheeth K. Vishnoi. Subset selection based on multiple rankings the presence of bias: Effectiveness of fairness constraints for multiwinner voting score functions. In International Conference Machine Learning, ICML, volume 202 of Proceedings of Machine Learning Research, pages 2641-2688. PMLR, 2023.
[30] Iris Bohnet. What Works: Gender Equality by Design. Harvard University Press, 2016.
[31] Robert Boldt, John A Centra, and Rosalea G Courtney. The validity various methods of treating multiple sat@ scores. ETS Research Report Series, 1986(1 (1)88, 1986.
[32] Tristan L Botelho and Mabel Abraham. Pursuing quality: How search costs and uncertainty magnify gender-based double standards in multistage evaluation process. Administrative Science Quarterly, 62(4) 2017.
[33] Hannah Riley Bowles. Why Women Don't Negotiate Their Job Offers, June 2014. https: //hbr.org/ wttp://///////////////////
wtt/////////wwooe-oeeeeeeeeee howpeennoooo--entttttt----eeeeee jofofr-rrooofers. offers.
[34] Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
12

---

[35] Andrei Z. Broder, Ravi Kumar, Farzin Maghoul, Prabhakar Raghavan, Sridhar Rajagopalan. Raymie Stata, Andrew Tomkins, and Janet L. Wiener. Graph structure the web. Comput. Networks, (3(1-6))309-320) 2000.
[36] Quinn Capers IV, Daniel Clinchot, Leon McDougle, and Anthony G Greenwald. Implicit racial bias in medical school admissions. Academic Medicine, 92(3):365 369, 2017.
[37] Marilyn Cavicchia. How to fight implicit bias? With conscious thought, diversity expert tells NABE, June 2017.
[38] L. Elisa Celis, Chris Hays, Anay Mehrotra, and Nisheeth K. Vishnoi. The Effect of the Rooney Rule on Implicit Bias in the Long Term. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT '21, page 078-689, New York, NY, USA, Association for Computing Machinery.
2021.
[39] L. Elisa Celis, Vijay Keswani, and Nisheeth K. Vishnoi. Data preprocessing to mitigate bias: maximum entropy based approach. In ICML,, volume 119 of Proceedings of Machine Learning Research, pages 1359. PMLR, 2020.
[40] L. Elisa Celis, Anay Mehrotra, and Nisheeth K. Vishnoi. Interventions for ranking in the presence of implicit bias. In FA7*, pages 369-380. ACM, 2020.
[41] Christopher P Chambers and Federico Echenique. A Characterisation of 'Phelpsian' Statistical Discrimination. The Economic Journal, (1(()))) 2032, 2020.
[42] Tessa E. S. Charlesworth and Mahzarin Banaji. Patterns of Implicit and Explicit Attitudes: I. Long-Term Change and Stability From 2007 to 2016. Psychological Science, 30(2):174-192, 2019. PMID: 30605364.
[43] Erick Chastain, Adi Livnat, Christos Papadimitriou, and Umesh Vazirani. Algorithmss, games, and evolution. Proceedings of the National Academy of Sciences, 111(29):10220-10222 2014.
[44] Mansi Choksi. Inside India's Cram City. The New York Times, January 2023. https:
/ //web. wrchiva. http://errreeeeee/////0020020000
http//ttt////// //t////www http///wwttttttt///////// wwt.iieste............. com/2023/01/ 18/ 18/magazine//ndiia cram schooommmsssssseeaaeaaaro-sees-- -otalllsskktthhee html.
[45] Stephen Coate and Glenn C. Loury. Will affirmative-action policies eliminate negative stereo- types? The American Economic Review, 13(5):1220-12444 1993.
[46] Brian W. Collins. Tackling unconscious bias in hiring practices: The plight of the rooney rule. NYUL Rev., 82:870, 2007.
[47] Conduct IIT entrance test in Tamil also: PMK, March 2012. https::/web..... archive. org/ http://httts::/ww../tt www.ind.u artic1e2961807 ece..
teb/201446002020//hhppp
[48] Keith Conrad. Probability distributions and maximum entropy. Entropy, 6(452):10, 2004.
[49] Bradford Cornell and Ivo Welch. Culture, information, and screening discrimination. Journal of political Economy, 104(3):542-571, 1996.
[50] Jason Dana, Robyn Dawes, and Nathanial Peterson. Belief in the unstructured interview: The persistence of an illusion. Judgment and Decision Making, 855):512-520, 2013.
[51] Jeffrey Dastin. Amazon scraps secret Al recruiting tool that showed bias against women, October 2019. https://reut.rs//NNeza
[52] Debraj Deb. Tripura govt to sponsor students with NEET, JEE coaching under 'Super 30' scheme. The Indian Express, May 2020. http:/:/iinnnaaaedeeeeee com/ http//lttceeeeeeee/eeeeee //duc wintttssswwihth weiith-neet-te jee- pecee--je--cccciine cowching- under
super-30-scchme- seoo-scheme--o faoeeee-rooo-tisst -tt.
[53] Delhi govt. to provide free coaching for NEET, JEE aspirants, February 2022. https:
http::/cce.e..aaaoaaaooo.ooooorr coae- ing-for-nees -eeor-neet-teeee ceooo-nee---ee-sppiiee petttsjeeeaapiiaattttarrtttttosr
wwww.thehinda. http:///ww..eeeeettt/e//////aeae
13

---

[54] William Dieterich, Christina Mendoza, and Tim Brennan. Compas risk scales; Demonstrating accuracy equity and predictive parity. Northpoint Inc, 2016.
[55] Ezekiel Dixon-Romia Howard T Everson, and John McArdle. Race, poverty and sat scores: Modeling the influences of family income on black and white high school students' sat performance. Teachers College Record, 115(4):1-33, 2013.
[56] Miroslav Dudik. Maximum entropy density estimation and modeling geographic distributions of species, 2007.
[57] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In ITCS, pages 214-226, New York, NY, USA, 2012. ACM.
[58) Tracy D Fells and C Robert Showalter. Work-related stress in american trial judges. Journal of the American Academy of Psychiatry and the Law Online, 22(1):71-83, 1994.
[59] Edith Elkind, Piotr Faliszewski, Piotr Skowron, and Arkadii Slinko. Properties of multiwinner voting rules. AAMAS 2014, pages 53-60, 2014.
[60] Kim Elsesser. Lawsuit Claims SAT And ACT Are Biased-Herd What Research Says, 2019. https:/ //www//wwwwww./ forbes. httpa////aeaeetttaaaaa////////// and act- are- biased-
claims-aammmaiii sat hawpaaa--ooorrrcdr-sssaaaaaaaa/a
[61] Vitalii Emelianov, Nicolas Gast, Krishna P. Gummadi, and Patrick Loiseau. On fair selection
in the presence of implicit variance. EC, pages 649-675. ACM, 2020.
[62] Episode 2 of Inside the Yale Admissions Office Podcast, 2020. hetps://admissionn yal edu/podcast.
[63] Robert Epstein and Ronald Robertson. The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections. Proceedings of the National Academy of Sciences, 112(33):E4112-EA. E4421, 2015.
[64] Frederick Erickson and Jeffrey Schultz. The Counselor as gatekeeper: Social interaction in inverviews. Academic Press, 1982.
[65] Sean Fahey. The Real Cost Of Bad Hiring Decisions (And How To Avoid Making Them), March 2022. https://ww forbes. wwwccoco//eeeeeee of bad-hiring decisions and-how to- g- 4tthem/7sh-11112266655
[66] Hanming Fang and Andrea Moro. Chapter Theories of Statistical Discrimination and Affirmative Action: A Survey. Jess Benhabib, Alberto Bisin, and Matthew O. Jackson, editors, Handbook of Social Economics, volume 1, pages 133-200. North-Holland, 2011.
[67] Nikhil Garg, Hannah Li, and Faidra Monachou. Standardized tests and affirmative action: The role of bias and variance. Proceedings of the 2021 ACM Conference on Fairness, Account- ability, and Transparency, FAceT '21, page 261, New York, NY, USA, 2021. Association for Computing Machinery.
[68] Atul Gawande. The Checklist Manifesto: How to Get Things Right. Henry Holt and Company, 2010.
[69] Tamar Szabó Gendler. On the epistemic costs of implicit bias. Philosophical Studies, 156(1):33, 2011.
[70] Daniel T. Gilbert and J. Gregory Hixon. The trouble of thinking: Activation and application of stereotypic beliefs. Journal of Personality and Social Psychology, 60:509-517, 1991.
[71] Anand M Goel and Anjan V Thakor, Overconfidence, ceo selection, and corporate governance. the Journal of Finance, 6376)227772 2008.
[72] Claudia Goldin and Cecilia Rouse. Orchestrating Impartiality: The Impact of "Blind" Audi- tions on Female Musicians. American Economic Review, 90(4):715-7411 September 2000.
14

---

[73] Anthony G Greenwald and Mahzarin R Banaji. Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychological review, 102(1):4, 1995.
[74] Anthony Greenwald and Linda Hamilton Krieger. Implicit bias: Scientific foundations. Califorria Law Review, 94(4):945-967, 2006.
[75] Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning- In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett, editors, Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages 3315-3323, 2016.
[76] Abigail Johnson Hess. Rich students get better SAT scores here's why, October 2019. https:// //wp://wwwwww.o http///wwwbcccccco///o//oooo eres ares-why... .html.
[77] The White House. Fact Sheet: President Obama Announces New Commitments from Investors, Companies, Universities, and Cities to Advance Inclusive Entrepreneurship at First-Ever White House Demo Day, August 2015.
[78] J.E. Ingersoll. Theory of Financial Decision Making. G Refereence,Ifformaaie and Interdis- ciplinary Subjects Series. Rowman & Littlefield, 1987.
[79] Abigail Z. Jacobs and Hanna Wallach. Measurement and fairness. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT '21, page 375-385, New York, NY, USA. 2021. Association for Computing Machinery.
[80] Edwin T. Jaynes. Information theory and statistical mechanics. Physical Review, 106:620-630, May 1957.
[81] Edwin T. Jaynes. On the rationale of maximum-entrop methods. Proceedings of the IEEE, 70(9):939-955 952, 1982.
[82] Kailash Jeenger. Reservation Is About Adequate Representation, Not Poverty Eradication. The Wire, May 2020. https://thewire. in wwwpaax/uupreeeeeeeee.//m -ww.---encc--eeeeeeeeeeeeee--eee
[83] Yangqing Jia and Trevor Darrell. Heavy-tailed distances for gradient based image descriptors. In J. Shawe-Tayy Taylor, R. Zemel, P. Bartlett, F. Pereira, and K.Q. Weinberger, editors, Advances Neural Information Processing Systems, volume 24. Curran Associates, Inc., 2011.
[84] D. Kahneman, S.P. Slovic, P. Slovic, A. Tversky, and Cambridge University Press. Judgment Under Uncertainty: Heuristics and Biases. Cambridge University Press, 1982
[85]
Irwin Katz. Gordon Allport's "The Nature of Prejudice". Political Psychology, 12(1):125-157, 1991.
[86] Giora Keinan. Decision making under stress: Scanning of alternatives under controllable and uncontrollable threats. Journal of Personality and Social Psychology, 52:639 6444,44444 1987.
[87] M.E. Kite and B.F. Whitley. Psychology of Prejudice and Discrimination: 3rd Edition. Taylor & Francis, 2016.
[88] Jon Kleinberg and Sendhil Mullainathan. Simplicity creates inequity: Implications for fairness, stereotypes, and interpretability. In Proceedings of the 2019 ACM Conference on Economics and Computation, EC '1, page 807-808, New York, NY, USA, 2019. Association for Com- puting Machinery.
[89] Jon M. Kleinberg, Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair determination of risk scores. In ITCS, volume 67 of LIPles, pages 43:1-43:23, Schloss Dagstuhl- Seeibniz-Zentrum für Informatik, 2017.
[90] Jon M. Kleinberg and Manish Raghavan. Selection problems in the presence of implicit bias. ITCS, volume of L.Plcss pages 33:1-33:17. Schloss Dagstuhl Leibniz-Zentrum für Informatik, 2018.
15

---

[91] Mr. Rajeev Kumar. Rti complaint, 2009. Decision No. CIC/SC/CC2099/00008833322,,, Com- plaint No. CIC/SG/C/2009/000088..
[92] Rahul Kumar. SC, ST, OBC representation in Indian education is dismal, upper-caste nexus persists. The Print, March 2021. https: pttpss//theprint in/ catpcaapus--oe sow.s/s-oooiieecssstto----- opresentation- indian education on- iss--ss-iiii disallimal hatssmmmluppprrree -ww.eaaeee----eeeeeeeeeeee s/627217/.
[93] Ravi Kumar, Prabhakar Raghavan, Sridhar Rajagopalan, D Sivakumar, Andrew Tomkins, and Eli Upfal. Stochastic models for the web graph. In Proceedings 41st Annual Symposium on Foumdations of Computer Science, pages 57-65. IEEE, 2000.
[94] Didier Lairez. A short derivation of boltzmann distribution and gibbs entropy formula from the fundamental postulate, 2023.
[95] Kevin Lang. A Language Theory of Discrimination" The Quarterly Journal of Economics, 101(2):363332, 05 1986.
[96] Flavien Léger. gradient descent perspective on sinkhorn. Applied Mathematics & Optimiza- tion, 84(2):1843 18851 2021.
[97] David Asher Levin, Yuval Peres, and Elizabeth Lee Wilmer. Markov chains and mixing times. American Mathematical Soc., 2009.
[98] Haim Levy and Myles Robinson. Stochastic Dominance: Investment Decision Making Under Uncertainty, volume 34. Springer, 2006.
[99] Tamar Lewin. A New SAT Aims to Realign With Schoolwork, March 2014. https://wwt nytimes. wwl.mss.com/ae cww.cmmmm--0/ http////////ducattii// Lege-board. html.
changes aerttt-annunnnddb-yeee
[100] Falk Lieder and Thomas L. Griffiths. Resource-rational analysis: Understanding human cogmition as the optimal use of limited computational resources. Behavioral and Brain Sciences, 43:e1, 2020.
[101] W Lippmann. Public opinion. new york city. Harcourt, Brace. Lipson, J., Omidian, P.(1992, September). Health issues of Afghan refugees in California. The Western Journal of Medicine. 157(3):271-275, 1922.
[102] J. H. C. Lisman and M. C. A. van Zuylen. Note on the generation of most probable frequency distributions. Statistica Neerlandica, 26(1):19-23, March 1972.
[103) List of languages by number of native speakers in India. https: //en wwt....iieeiiii..amemmmmmmmm.aae org/wik
i i/List_of _anguugges___r_a___________ by__nmbee__eee_ of ltt___nvv_sspeaerrreeee Laataakrrr______ntee in__ini_niiii India.
[104] Karen Lyness and Madeline Heilman. When fit is fundamental: and promotions of upper-level female and male managers. Journal of Applied Psychology, 91(4):777, 2006.
performance evaluations
[105] Michael W. Mahoney and Lorenzo Orecchia. Implementing regularization implicitly via approximate eigenvector computation. In ICML, pages 121-128. Omnipress, 2011.
[106] Anay Mehrotra, Bary S. R. Pradelski. and Nisheeth K. Vishnoi. Selection in the presence of implicit bias: The advantage of intersectional constraints. In FAceT '22: 2022 ACM Conference Fairness, Accountability, and Transparency, Seoul, Republic of Korea, June 21 24, 2022, pages 599-609, ACM, 2022.
[107] Anay Mehrotra, Bary S. R. Pradelski, and Nisheeth K. Vishnoi. Selection the Presence of Implicit Bias: The Advantage of Intersectional Constraints. In FAccT, page To appear. ACM, 2022.
[108] Anay Mehrotra and Nisheeth K. Vishnoi. Maximizing submodular functions for recommenda- tion in the presence of biases. In Ying Ding, Jie Tang, Juan F. Sequeda, Lora Aroyo, Carlos Castillo, and Geert-Jan Houben, editors, Proceedings of the ACM Web Conference 2023, WWW 2023, Austin, TX. USA, 30 April 2023 4 May 2023, pages 3625-3636. ACM, 2023.
16

---

[109] Robert C. Merton. Optimum consumption and portfolio rules in a continuous time model would like to thank p. a. samuelson, m. solow, p. a. diamond, j. mirrlees, J. flemming. and d. scheffman for their helpful discussions. of course, all errors are mine. aid from the national science foundation is gratefully acknowledged. an earlier version of the paper was presented at the second world congress of the econometric society, cambridge, england. In W.T. ZIEMBA and R.G. VICKSON, editors, Stochastic Optimization Models in Finance, pages 621-661. Academic Press, 1975.
[110] Corinne A. Moss-Racusin, John F. Dovidio, Victoria L. Brescoll, Mark Graham, and Jo Handelsman. Science faculty's s subtle gender biases favor male students. Proceedings of the National Academy of Sciences, 109(41):16444-166))
2012.
[111] Julie A Nelson. Are women really more risk-averse than men? a re-analysis of the literature using expanded methods. Journal of economic surveys, 29(3):566-555 2015.
[112] E. O'Dea, M. Lagisz, M. D. Jennions, and S. Nakagawa. Gender differences in individual variation in academic grades fail to fit expected patterns for stem. Nature Communications, 9(1):3777, 2018.
[113] Tom O'Neil. Ask the expert: How to write a CV, 2012. http://veb. http:://weebb.rciieeeeeee archive.org/ver T017020206565//tttp::/
economi st. com/node/21559500
[114] Paula Onuchic. Recent contributions to theories of discrimination. arXiv preprint (XXix:2205005990 2022.
[115] Philipp Oreopoulos and Diane Dechief. Why do some employers prefer to interview matthew, but not samir? new evidence from toronto. montreal, and vancouver. Canadian Labour Market and Skills Researcher Network Working Paper 95, 2012.
[116] Christina Passariello. Tech Firms Borrow Football Play to Increase Hiring of Women, Septem- ber 2016. blttpa://wwre waj www.em/attillee wwcccelehheeeeeeeeeeeeeeeeee firms--r-------- boroo-obooooox-ttbaallllll---l-- play-t 0- hwwccresse-iirng--ee wome--------------------------e- 1474963562.
[117] Edmund S. Phelps. The statistical theory of racism and sexism. The American Economic Review, 62(4))659-661, 1972.
[118] J.R. Posselt. Inside Graduate Admissions: Merit, Diversity, and Faculty Gatekeeping. Harvard University Press, 2016.
[119] John Pratt. Risk Aversion in the Small and the Large. In Peter Diamond and Michael Rothschild, editors, Uncertainty in Economics, pages 59-79. Academic Press, 1978.
[120] Proof: Normal distribution maximizes differential entropy for fixed variance. https://stat proofbook. gi http:/io////omaaaaaaeeeeee maxent. htmll
[121] Elaine D Pulakos. Selection assessment methods. United stated of America: Society for Human Resource Management (SHRM) Foundation, 2005.
[122] Manish Raghavan, Solon Barocas, Jon M. Kleinberg, and Karen Levy. Mitigating bias in algorithmic hiring: evaluating claims and practices. In Mireille Hildebrandt, Carlos Castillo, Elisa Celis, Salvatore Ruggieri, Linnet Taylor, and Gabriela Zanfir-Fortuna, editors, FAT* '20: Conference on Fairness, Accountability, and Transparency, Barcelona, Spain, January 27-30, 2020, pages 469-481, Barcelona, Spain, 2020. ACM.
[123] Barbara Reskin and Debra Branch McBrier. Why Not Ascription? Organizations' Em- ployment of Male and Female Managers. American Sociological Review, 65(2):210-233, 2000.
[124] Retaking the SAT, Manhattan Review. http::///wa..aaaaattttrrrerooooo ing.
[125] Dan-Olof Rooth. Automatic associations and discrimination hiring: Real world evidence. Labour Economics, 17(3):523-534, 2010. 2010.
17

---

[126] Walter Rudin. Real and Complex Analysis. McGraw-Hill, 1986.
[127] Melody Sadler, Joshua Correll, Bernadette Park, and Charles M Judd. The world is not black and white: Racial bias in the decision to shoot in a multiethnic context. Journal of Social Issues, 68(2):286 313, 2012.
[128] Sahitya parishad demands entrance tests in gujarati, September 2011. https://web. archiv e. http://ww0001000001110/////////a a11-0-222/oduccacc/////l11111112 lan.uagagggassseee entrance cests-raght veer veer-chaudhary.
timesof india indiatimes com/2 -anguggggnnnn-aguueeeeeeeeeeeee
[129] Jad Salem and Swati Gupta. Closing the gap: Group-aware parallelization for the secretary problem with biased evaluations. Available at SSRN 3444283, 2019.
[130] Howard Schuman, Charlotte Steeh, Lawrence Bobo, and Maria Krysan. Racial Attitudes in America: Trends and Interpretations. Social trends the United States. Harvard University Press, 1985.
[131] Deepa Seetharaman. Facebook Is Testing the 'Rooney Rule' Approach to Hiring. The Wall Street Journal, June 2015.
[132] Herbert A Simon. Bounded rationality and organizational learning. Organization science, 1(1):125-134, 1991.
[133] Mohit Singh and Nisheeth K. Vishnoi. Entropy, optimization and counting. In Proceedings of the 46th Annual ACM Symposium on Theory of Computing, pages 50-59. ACM, 2014.
[134] Aarzoo Snigdha. JEE Main 2021: Over 45,000 Students Writing Exam In Regional Languages. NDTV, February 2021. https: //wp:.//wwwww// ndtv com/ wwtp///occccnneeeeeeeeeeeeeeeeee main- 2021 over-4 5000 students- writing-
-eaiinngg-------e exam- An-regional Baanguages-237888
[135] Thomas Sowell. Affirmative Action Around the World: An Empirical Study. Yale University Press, 2008.
[136] Damian Straszak and Nisheeth K. Vishnoi. Maximum entropy distributions: Bit complexity and stability. COLT, 2019.
[137] Damian Straszak and Nisheeth K. Vishnoi. Iteratively reweighted least squares and slime mold dynamics: connection and convergence. Mathematical Programming, 194(1):685-71 2022.
[138] R.H. Thaler. Misbehaving: The Making of Behavioral Economics. W. W. Norton, 2015.
[139] The Cost of Bad Hire, February 2019. https:://// http:///wwww...thhheeee www.noowwteerr.. htt.://wsssereeeedee/rr///tt///a ws
cost-of a- bad hire. .hmml.
[140] The Utter Uselessness of Job Interviews, April 2017. iot-joieneeeeeeeess html.
http://mmmsccm//211////// com/2017/0 htt.:aaeeeesssseee---o-oooooeeee
[141] UP govt free coaching for JEE, NEET aspirants begins today, February 2021. https: //www. wwwp::liveiitt.comm
spirants- opppaaats-bpgiii--ooooooyl111313 html.
[142] AD Van Knippenberg, AP Dijksterhuis, and Diane Vermeulen. Judgement and memory of criminal act: The effects of stereotypes and cognitive load. European Journal of Social Psychology, 29(2-3): 191-201, 1999.
[143] Nisheeth K. Vishnoi. Algorithms for Convex Optimization. Cambridge University Press, 2021.
[144] J. Neumann, O. Morgenstern, H.W. Kuhn, and A. Rubinstein. Theory of Games and Economic Behavior: 60th Anniversary Commemorative Edition. Princeton Classic Editions. Princeton University Press, 2007.
[145] Joseph Walker. Meet the New Boss: Big Data, September 2012. httpp://wwwt wwsp:wwwwwwjjjw.../oooooo.....ss cwm/ #tte://s000777334444044666222222
18

---

[146] Thomas E. Weisskopf. Impact of Reservation on Admissions to Higher Education in India. Economic and Political Weekly, 39(39):433--4349 2004.
[147] Christine Wennerås and Agnes Wold. Nepotism and Sexism in Peer-Review. 387(6631):3413433 May 1997.
Nature,
[148] Wikipedia contributors. Maximum entropy probability distribution Wikipedia, the free encyclopedia. https://en wttp://wwwwiaaaa..aaoooaaaaaaaaa wappttileeeaaiiimmet. ent 2023. [Online; accessed
http:/oowaatty....s.rrrrooooooo. 16-May-2023J.
[149] Joan C Williams. Double jeopardy? an empirical study with implications for the debates over implicit bias and intersectionality. Harvard Journal of Law & Gender, 37:185, 2014.
[150] Sarah Wood. Colleges With The Highest Application Fees, January 2022. https://www us news wwwp////caaaiioooeeeetteeooeeee. wowlleeeethhesshottt-teeee eges- the- highhstttaapppee wtw.ggeesttapppccinnooeeeeeeee
[151] Ruixun Zhang, Thomas Brennan, and Andrew W Lo. The origin of risk aversion. Proceedings of the National Academy of Sciences, 111(50):17777--7788) 2014.
[152] Jonathan C. Ziegert and Paul J. Hanges. Employment discrimination: The role of implicit attitudes, motivation, and a climate for racial bias. Journal of Applied Psychology, 90(3):553- 562, 2005.
19

---

Contents
Introduction
2 Model
3 Theoretical results
Empirical results
5
Conclusion, limitations, and future work
10
A Other related work
22
Specific examples our model
23
Characterization of optimal solution the optimization problem
24
C.1 The primal optimization problem and assumptions
24
C.2 Dual formulation and weak duality
26
C,3 Strong duality
27
C,4 Properties of the integral ((x)
................................
29
C.5 Positivity of the optimal dual variable
31
C.6 Optimality conditions and uniqueness of optimal solution
................................
33
C.7 Conditions under which optimal solution is fo
34
Derivation of output density for single individual
35
E
Connection to the Gibbs equation
36
Effect of changing the resource-information parameter 7
36
F.1 Effect of decreasing 7
36
F.2 Effect of increasing
38
G Effect of changing the risk-averseness parameter
39
Gaussian density
39
Pareto density
42
J Exponential density
45
K Laplace density
47
Implementation details and additional empirical results
48
L.1 Case Study: Evaluating bias-mitigating interventions in IITJEE admissions
48
L.2 Additional plots for simulations in Section 4
51
20

---

L.3
Implementation details
51
L.3.1
Our framework and other models
................................
51
L.3.2
Computational resources used
53
L.3.3
JEE-2009 Scores
53
L.3.4
Semantic Scholar Open Research Corpus
54
21

---

A Other related work
Models of bias in Economics. There are two prominent models of discrimination in the Economics literature: baste-based discrimination and statistical discrimination [114]. These capture different types of biases [117, 45, 49, 12, 22, 41, 11] (also see [66, 25, 114]). Taste-based discrimination [22] models explicit biases (e.g., racism and sexism) and, in the vanilla taste-based discrimination model, individuals are divided into two groups and a decision- makeer pays an additional cost for interacting with individuals in the disadvantaged group. This additional additive cost diminishes the value of disadvantaged individuals for the decision- maker.. While we do not model explicit biases, such an additive bias also arises in our model of evaluation processes with specific parameter choices, suggesting that additive biases may also arise due to resource constraints and risk averseness (Appendix C). Statistical discrimination models how group-wise disparities in the noise in the inputs to a Bayesian pecision-makes propagate to systematic disparities in decisions [117, 11]. Our mathematical model can be viewed as giving an explanation of why such disparities may arise in the input.
Implicit biases in Psychology. There is long and rich history of the study of implicit (and explicit) biases in Psychology, e.g., [85, 101. 74, 104, 69, 127, 149]. This body of works proposes various theories about why implicit biases arise [69] and their relation real-world stimuli [130, 63, 42]. We refer the reader to [73, 87] for an overview. [69] explains that the ease of categorizing individuals into categories (defined by, e-g., color, race, or gender) provides an incentive to the evaluator to use their prior (possibly biased) knowledge, and this leads to implicit biases. In contrast, we show that even when the evaluator has the same prior estimate for all social groups, biases can arise in evaluation processes when the different groups. Further, since resource constraints and risk averseness are not specific to the setting of single evaluator, our model of evaluation processes also models scenarios where the evaluator denotes a group or an organization.
connaation-toresouure
trade-off or the degree of risk averseness is different for
Optimization and human behavior. The use of optimization to model human behavior dates back to (at least) von Neumann and Morgenstern's 's work that showed that, under a small number of assumptions, the behavior of an evaluator is as if they are optimizing the expected value of a "utility function" [144]. Since then, numerous apparent deviations from this theory were discovered [84]. These deviations, broadly referred to irrational behavior or cognitive biases, laid the foundation of Behavioral Economics [138]. Several theories have been proposed to account for the deviations in human behavior from utility maximization. Including prospect theory that models risk averseness of individuals the empirical observation that humans process losses and gains of equal amounts (monetary or otherwise) asymmetrically [84] bounded rationality that reconciles irrational human behavior by proposing that humans solve underlying optimization problems approximately (instead of optimally) [132], and resource rational analysis that proposes that humans trade-off the utility of with the costs (e.g., such as effort and time) required to find a solution with higher utility [100]. These works model hidden costs and constraints humans that lead to deviations from the traditional "rational" utility maximization. Like this work, these works also use optimization to explain human behavior, but while they focus on irrational behaviors and cognitive biases, our work focuses on biases with respect to socially-salient attributes in evaluation processes.
Other entropy-based models. Maximum entropy distributions have been widely deployed in machine learning [56] and theoretical computer science [133]. Maximum entropy distributions have been shown to be "stable" [136]. The maximum-entropy framework over discrete distributions has been used to preprocess data to debias it [39]. In our optimization program, we use entropy to measure the amount of "information" in a distribution An
this appears as a constraint in our program. viewpoint is also prevalent in explanations of other phenomena. For instance, various optimization algorithms can be captured using an optimization- and entropy- -based viewpoint [9, 96, 143], it is also known to arise in explanations of biological phenomena [43], and leads to Page rank and other popular random-waid walk-based procedures [105]. Finally, taking an optimization viewpoint when studying a dynamical system also has additional benefits, such providing "potential function" that functions that gives an efficient and interpretable method of tracking the progress of complex dynamical system [137].
22

---

Plot of C,(x, 0) for different a when / 6=63
d= 2 Plot of fx(t, 5) for different a when é(x v)
a=1
a= 1
a= 2 a= 3
d=
a=4
4
5
0
-2
-1
3
4
5
(a) La(e, 0) when E(2, v) (x v)²
(b) La La (2,,5) when ((z, v) Iu Ia
Figure 2: Plots of risk-averse loss function (a, u) for different a and l.
B Specific examples of our model
In this section, we present some specific examples of mechanisms by which information constraints and risk aversion lead to bias. One context is college admissions. is well known that SAT scores are implicitly correlated with income (and, hence, test preparation) in addition to student ability [1]. While the true ability may be v. the score is skewed depending on the amount/quality of test preparation, which depends on socioeconomic status that may be correlated to vocially-salien attributes. The parameter a in our model can be used to encode this. As for т, while an evaluator may know what a GPA means at certain universities well known to them, they may not understand what GPA means for students from fesser-known schools. This lack of knowledge can be overcome, but takes effort/time, and without effort entrenches the status quo.
Another example is the evaluation of candidates using a standardized test. In time-constrained settings, a high value of the aesouce-informatte candidates may not be able to comprehend question as well as someone from an advantaged group. This could be due to various factors including less familiarity with the language used in the test or the pattern of questions, as opposed to someone who had the resources to invest a training program for the test. Similarly, high value of the iise-everseness parameter captures that an evaluator, when faced with a choice of awarding low or high marks an answer given by candidate from the disadvantaged group, is less likely to give high marks. More concretely, suppose there are several questions in a test, where each question is graded either 0 or 1. Assume that a candidate has true utility e E [0, 1J. and hence, would have received an expected score for each of the questions if one were allowed to award grades in the continuous range [0, 1]. However, the fact that the true scores have be rounded to either 0 or 1 can create a bias for the disadvantaged group. Indeed, the probability that an evaluator rounds such an answer to I may be less than v the averseessss parameter measures the extent which this probability gets scaled down.
parameter 7 for the disadvantaged group indicates that such
Most of the prior works on interventions in selection settings have focused on adding representational constraints for disadvantaged groups. Such constraints, often framed as form of affirmative action, could be beneficial but may not be possible to implement in certain contexts. For instance, in a landmark 2023 ruling, the US Supreme Court effectively prohibited the use of race-based affirmative action in college admissions [2]. Our work, via a more refined model of how bias might arrive at population level in evaluation processes, allows for evaluating additional interventions that focus on procedural fairness; this allows working towards diversity and equity goals without placing affirmative-action-lik constraints.
In this framing, we can consider decreasing either a or T. Improving either would work towards equity, but which ones to target or to what extent and via which method would be context-dependent and vary in cost. decrease in a can be achieved by reducing risk-averseness in the evaluation process; e.g. by investing in better facilities for disadvantaged groups, or making the evaluation process blind to group membership. A reduction in r may follow by allocating additional resources to the evaluation process, e.g., by letting candidate choose an evaluation in their native language. Our framework allows a policymaker to study these trade-offs, and we discuss specific examples in Appendix L.
23

---

C Characterization of optimal solution to the optimization problem
In this section, we present the formal version of Theorem 3.1. We first state the general conditions under which this result is true, and give an outline of its proof. Recall that an instance of the optimization problem is given by a tuple = (Rt, L,,,,,,,,,,,,, a; 7), where S2 is a closed interval in R (i.e., Ω is one of [a, b). (a, 20), (-o0, b). (- oo) for some suitable real values or b), fo is the density of the true utility that is Lebesgue measurable, é is the loss function, T is the sesource-information parameter and a 2 1 is the risk-averseness parameter. The main result, Theorem C.I. states that under mild conditions, the instance I has unique optimal solution. We present applications of Theorem C.I when the input density is Gaussian, Pareto, Exponential, and Laplace in Sections H, I, J, and K respectively.
In Section C.I, we state the primal optimization formulation and the assumptions needed by The- orem C.I. Clearly, we need that the instance I has a non-empty set of solutions, i.e., there exists a density of entropy at least T (assumption (AO)). In Section C.2, we state the dual of the convex program PrimalOpt and show that weak duality holds. This proof requires integrability of the loss function with respect to the measure induced by the density fo (assumption (A3)). This assumption also ensures that the optimal value does not become infinite.
In Section C.3, we show that strong duality holds. We use Slater's condition to prove this. This proof also shows that there exist optimal Lagrange dual variables. However, this does not show that there is an optimal solution f* to the primal convex program. In fact, one can construct examples involving natural loss functions and densities where strong duality holds, but the optimal solution does not exist. In order to prove the existence of an optimal solution f to the instance I, we show that the optimal Lagrange dual variable 7 (corresponding to the entropy constraint in PrimalOpt) is strictly positive. This proof requires several technical steps.
We first study properties of the function [f)),.(())) in Section C.4, which is the expected loss if the estimated value is x. It is easy to verify that the objective function of PrimalOpt, ErTy, (fr. f). is the integral of Tf (..a,,(()) (x) (a:) over f2. Therefore, the optimal solution f" (ar) would place higher probability mass on regions where (zz) is small. Under natural monotonicity conditions on the loss function (assumption (A1)), show that ((x) can be expressed as sum of an increasing and a decreasing function. This decomposition shows that for natural loss functions, e.g., those which or concave or convex in each of the coordinates, the function (())) is unimodal; we state this as an assumption (A5) to take care of general loss function settings.
In Section C.5, we show that 7* > O. This proof hinges on the following result: For every instance == (9. fD, f, a,,)) r) with finite optimal value, there is a positive constant 77 such that the optimal value is at least [fo of the function I fo. (2) and bounded mean (and median) of fo (assumption A4). Somewhat surprisingly, this result also requires that the loss function f(u, x²) grows at least logarithmically as function of Iu- - x (assumption (A2)). Without this logarithmic growth, an optimal solution need not exist: may happen that there near-optimal solutions that place vanishingly small probability mass outside the global minimum x' of If. (r). Thus, even though we have an entropy constraint, there will be a sequence of solutions, whose error converges to the optimal value, that converges to delta function.
.7, where x" is the minimizer of fo,taa())) This requires unimodality
In Section C.6, we the positivity of o explicitly write down expression for the optimal primal solution. We use strict convexity of the feasible region for PrimalOpt to show that the optimal solution is unique. Finally, in Section C.7, we show that if fo has bounded entropy, then there is suitable choice for the loss function f, and parameters a and r, such that we recover fo as the unique optimal solution in the resulting instance.
C.1 The primal optimization problem and assumptions
We re-write the optimization problem OptProg here. An instance I of this problem is given by tuple I = [D. at, r), where a closed interval in R. SD is the density of the true utility that is Lebesgue measurable, is the loss function, is the resource-information parameter and a 2 is the risk-averseness parameter. Recall that y is the Lebesgue measure on TR. Note that Ω can be of the form (-oc, oo), or (a, oc) for a real (-o0, bj for real b, or [a, b] for real a, b, a < b.
24

---

The primal problem for I = (2, f,,,,, r) is as follows. Here, Dom denotes the set {f S2 +
R Rgo).
Errg,o (fD.))= = / 11 (((e))) fo(v)du(v)
min feDom
(PrimalOpt)
such that Ent(ʃ) ((+))((())
(6)
J f(x)du(z)=L =1
We state the assumptions and justify each of these:
AO (Strict feasibility of the instance) The interval Q has length strictly larger than et. This is satisfied trivially if Q is infinite.
In order ensure that the set of feasible solutions to instance of PrimalOpt is non-empty, we require that Ω has length at least e"; otherwise even the uniform distribution St shall have entropy less than The strict inequality is needed ensure strong duality (Slater's 's condition).
Remark:
A1 (Monotonicity of the loss function) We assume that the loss function : Ω + IR is continuous and f(x, ar) for all Q. We consider types of loss functions, TYPEP and TYPEN. The TYPEP loss functions have the following monotonicity property: For any fixed v E: 22, {(v, x) strictly increases as |U - x increases. It follows that f(v, x) 2 0 with equality if and only if v = I. The TYPEN loss functions have the following property: For fixed u 02, ((V,2) is strictly increasing function of x. It follows that f(v, x) 0 if el are of TYPEP,
x 2 v and (v,,I) < 0 < U. For example, E(x, v) = (r- -x-
whereas f(x. v) = In Inv, (x - v) are of TYPEN.
Remark: These are natural monotonicity properties. A TYPEP loss function ensures that the optimal density to an instance I of PrimalOpt assigns higher values to points where the imput density fo is more concentrated. A TYPEN loss function ensures that the optimal density does not place high probability mass at points that are much larger than the mean of fD.
A2 (Growth rate of the loss function) We would like to assume that the loss function E(x; v) grows reasonably rapidly as |x: v| increases. It turns out that the "right" growth rate would be at least logarithmic. For instance, we could require that [é(z, v) is 2[ In(x - v) However, unless assume some form of the triangle inequality, such lower bound would not imply a lower bound on [((x2, v) E( v)) for any Ø2 21, v E Q. Thus, we make the following assumption: There is constant C, such that for all 22, 21, V S with I1 x2l 2 C, v & (I1, I2), [((22, ((x1, w)] 2 In l 211 Ozi:
where Om,, is value which depends # only. For example, when E(x, v) (x - v), the above is satisfied with 02- = 0, C = 1; and when E(x, v) In x - In v, the above property holds with 0,- In xr = Remark: This subtle condition, and needed to ensure that an optimal solution exists. Consider for example, an instance I with Q = [0, oo). fo being the exponential density, ensure that In(x + +2) does not
f(z, v) = In In(x + 2) In In(y + 2) (the "+2" factor become negative). The parameters o,, can be arbitrary. Let e >> 0 be an arbitrarily small constant. Now, consider a solution that places
E probability mass the remaining E probability mass uniformly over an interval of length e/ to achieve entropy T. Since the loss function grows slowly, the expected loss of this solution decreases with decreasing E. Thus, even though strong duality holds in this instance, optimal solution does not exist. In fact, we have sequence of solutions, with error converging to the optimal value of I, converging to the delta-function at O.
and spreads
A3 (Integrability of the loss function) We assume that the function E(z, v)f is in (x) for every & E Ω. In other words, for each æ E Ω.
/ (0(,, ())lf())d))))) < co.
25

---

This is needed in order to carry out basic operations like the swapping of integrals (Fubini's Theorem) on the objective function. Further, this ensures that the function does not become infinite under reasonable conditions.
Remark:
A4 (Bounded mean and half-radius) We assume that the true utility density fo has a bounded mean m. Moreover, we assume that there is finite value It such that at least half of the probability mass of fo lies in the range [m R. + R...e.
Jamin-Rmntt R.
So(u)du(v) 1/2.
This condition ensures that f/D decays at a reasonable rate (though much milder than requiring bounded variance).
Remark:
A5 (Unique global minimum for estimated loss function) Let (, denote the expected loss when the estimated value is æ, i.c.,
(1))a())== = (0((1)))oo())dd
We assume that this function has unique minimum on Ω. Morcover, if xx denotes argmin ren ff. (x), we also assume that for other local minima x of this function, ff (x) is larger than ff,,ta("" by fixed constant. We state this condition formally as follows: Given any 5 > 0, there an 66 > 0 such that for all x satisfying fx x² > 6, we (f, t
have I fx.
Remark: This condition needed to ensure the uniqueness of the optimal solution. The optimal density for an instance I = (Ω, fD. l, T,),T) tries to place higher probability mass in regions where Lf low. Therefore, unique global minimum (and well- separatedness from other local minima) is needed to ensure that the optimal solution is unique. When the loss function is TYPEN, this condition is always satisfied (unless the optimal value is -c0). For TYPEP loss function, this condition is satisfied if the loss function [(x, v) concave or convex in x (for each fixed value of u), e.g. when f(z, v) = (z - v) 1 v). |n In v/.
The following is the formal version of Theorem 3.1.
Theorem C.1 (Characterization of optimal density). Consider an instance I of the optimization problem PrimalOpt defined closed interval Q TR. Let 0., T, IDD denote the loss function, risk- averseness parameter, resource-inge infororatiionnonn parameter, and the density of the true utility respectively in I. If assumptions (A5) are satisfied, then there is unique solution f to the instance I. This solution satisfies the following condition:
f" (z) x exp (- (Itm)laalt)
(8)
where r is the Lagrange variable corresponding to the entropy constraint (6) and is strictly positive. Moreover, Ent(f*)=1
C.2 Dual formulation and weak duality
Let y 2 and R denote the Lagrange variables for the constraints (6) and (7) respectively. Then, the Lagrangian is
= EEr-,Erraaaa (fr.a + Ent(f)) + o ( f(x)du(x))1)
Given 7 2 0, Φ, define
g(7.0) = inf L(f o).
(9)
feDom
The dual problem is
max
9(7, Φ).
(DualOpt)
NE VEREREER
We first show weak duality.
26

---

Theorem C.2 (Weak duality). Consider an instance I = (Ω,, fo, (,,,,,,TT) of PrimalOpt satisfying
assumption (A3). Let g(%,,) Φ) be as defined in (9). Then,
max rERgo.dER
Φ(7,8)= S
min
(fD:
J.f. feasible for PrimatOpt
Proof. We assume that the primal problem has feasible solution, otherwise the desired inequality follows trivially. Let be a feasible solution to PrimalOpt and let 7, & be real values with ? 2 Then,
g(y, 6) s o)
= -Errea + T(T - + (S, f(x))((())
S (FD,,
where the first inequality follows from the definition of g(%, Φ), and the last inequality follows from the fact that is a feasible solution to the instance I of PrimalOpt.
C.3
Strong duality
Theorem C.3 (Strong duality). Consider an instance I (S, of PrimalOpt satisfying assumptions (A0), (A1) and (A3). Let fr and (n",ø") x²) be the optimal solutions to PrimalOpt and DualOpt respectively. Then g( ,6) (FD,,
Proof. We first observe that there is a feasible solution to the instance I. This is because, by assumption (A0), Ω must contain an interval of length at least e". Now, we define f as the uniform distribution on T. Then Ent(f) = v, and hence, is a feasible solution the instance I. We now argue that (rre, (JD,J) is finite.
Claim C.4. Consider the function and the interval I defined above. Then, Errz, (fD,
Proof. Let a, b denote the left and the right end-points of the interval respectively. For any a E and v S2, we claim that (x, v) < max(e (a, v), la (b, v)). First, consider the case when the loss function is TYPEP. If is at least I, then f(r, v) < E(a, v); otherwise f(x, v) S 6(b, v). If the loss function is TYPEN, then we know that it is an increasing function of x, and therefore. f(x, v) < e(b, v). Thus, we see that for any x E I, E Ω (a(r,)) I max(e, v), (b, s (ea(a, v) + (b, (b, v){.
Therefore (recall that is the uniform distribution on I).
1 Errz, (JD, 1) 11 S, Ma, (v))d())))))))))++ [/] J, ebb ())((())p(() <0,,
where the last inequality follows from assumption (A3).
Thus, we see that the optimal value of PrimalOpt is either finite or oo. it is -oo, then weak duality implies that DualOpt has optimal value -o as well. Thus, we have shown strong duality in this case.
For the rest of the proof, assume that the optimal value, pt, of PrimalOpt is finite. We show strong duality using Slater's condition. Towards this, we define two sets A and B, both of which contained in TR33 Define
A: := vu, r Ent(f), v= / f(x)d=(x) - 1,t Err f), for some f E Dom
and
B (0.000))4<<11
It is easy to see that B is convex. We show that A is also convex.
27

---

Claim C.S. The set A as defined above is a convex subset of IR..
Proof. The proof follows from the fact that Ent(f) is a concave function of f and (rrea(DD. is linear in f. Formally, suppose (u1, v1, 11), (uz, 22, 22, b2)
E A and À E [0, 1]. We need to show that (u, v, f) = X(u,, (1,t1) (1 ()( V2, (2) E A. By the definition of the set A, there exist fi, f2 E Dom such that
ui 2 r -Ent(fi), Vi 1 [(()dd((x) 1, 4 (FD, Sc), i (1,2).
Consider f = Af1 + (1- Then f is also density and in Dom. Further, (TD,,aff..J Brer fi) (1 Er)ErTe,a fe) Ata + (1 A)tz t.
Similarly, since is concave,
Ent(ʃ) - 1 (()) -> S1/
Thus, (u,e, EA
We argue that A and B disjoint. Indeed, otherwise, there is E Dom such that is density, Ent( 2 т and Errz,.//,, J) < p", a contradiction. By the hyperplane separation ())(())))))))))))))))) (x1, (2, xa) = such that A and B lie on different sides of this hyperplane. In other words, for every (51, 22,,22,03 v3) E A.
theorem [34, 143], there is hyperplane (5, o, D)
P0 + δx² + VI3 2 c,
and for every (21, B,
jen doz vrrs S c.
Therefore, for each f E Dom,
( - Ent(f)) + { (FD: 2c.
(10)
and
Dp S c.
(11)
Claim C.6. 5) and v are non-negative.
Proof. Suppose, for the sake of contradiction, that γ < Consider f E Dom as given by Claim C.4. Choose values (æ1, 22, T3) as follows: is a large enough value greater than 7 - Ent(f), = 0, I3 Errz,aaf. f). Claim C.4 shows that 23 is finite. Since (I1,42, Øz) E A. 50, + 1/23 a. Since 7 < 0, we can choose 2ı large enough to make 77 + irg go below a, which is a contradiction. Similarly, we can show that 2 O.
Thus, two cases arise (i) D > O, or (ii) ú = O. First, consider the case when > 0. Define 7* = 3/0,0" o" = C/ū. Inequalities (10) and (11) show that for all f E Dom.
7" Ent(f)) o- (())du())) - -1) Erre,
ie., 00'"000''0 p- It follows from weak duality (Theorem C.2) that g(>", x³)) p" and the dual optimum value is equal to p".
Now consider the case when iz = 0. Again, inequalities (10) and (11) show that for all f E Dom,
5(r Ent(f)) + o ( f(x)d=(x) - -1) 2
28

---

Next, we observe that there is an fo Dom such that fo is density and Ent(fo) > т. Indeed, let fo be the uniform distribution over an interval of length strictly larger than e" (such an interval exists by assumption (A0)). Substitution f = fo in the above inequality, and assuming 7 >> 0, the l.h.s, of the above inequality becomes strictly less than O, which is contradiction. Therefore, it must be the case that ? = 0. Hence, we see that for all f E Dom,
(/s ((o)du(()))) - 20.
Since all the three quantities 7, Φi i cannot be O, it must be the case that 0. But for any density fo E Dom, by suitably scaling it by positive real, we can make the quantity (Sa [x))d((x) strictly larger than strictly smaller than O, which is again contradiction. Hence, have concluded that // cannot be O. This completes the proof of strong duality.
Corollary C.7. Consider an instance I = (Q,, fD, a, of PrimalOpt and assume that the optimal value p" is finite. Then there exists solution (r", 6*) to DualOpt such that p" u(+""07
Proof. This follows from the proof of Theorem C.3. When p" is finite, the parameter > and, hence, "" 5/0,00 o² é/ū as defined in the proof of this result satisfy the property that g(7*, Φ*)
p'
We would now like to prove that, assuming that the optimal primal value is finite, the optimal dual variable > is non-zero. This allows write explicit expression for optimal solution to PrimalOpt. We first need to understand the properties of the following integral, which was defined in assumption (A5):
[f = - -So fa(z, fv))dd)(()))
When the parameters fD, E a will clear from the context, we shall often abbreviate fo. (x) as (xx).
C.4 Properties of the integral (x))
We study some of the key properties of the integral / (x)). We shall assume that assumptions (A0)-A44 hold. The integral [xx) can be split into two parts:
[(x) = a /
Jan(-oc,21 (l. (1,,))o(o))p((0)+
[nla,,o)) f(x, ((),))((()))((v))
=="(())
Lemma C.8. The integral (x) is strictly increasing function xf x. Further, I (x2) ( aIn(12 - 22) oz, (x1)), for all x2, E Ω satisfying D2 2 C. Here, FD denotes the cumulative distribution function (c.d.f)) of fD.
Recall that the parameter 0z, appears in the assumption (A3).
Proof. Consider æ1, with 01 T2. Then
(x2) (x1) I (E(x2, v) - ((=1,)))fo())dll + / Jen
((02, (1o))))))))))p())))
By assumption (A1), ((22, v) > f(r1, v) for all v S x and ((x2, v) 2 0 for all v S x2, we see that (x2)) > (x1)). Suppose 22 x > C. Then {(12, E(11, v) 2 In Ir2 x1 - 0z₁ Therefore, using A3, the first integral above at least
a(In(122 - 0m,) (11)) f fo(u)du(u) = alI(222 - (
29

---

Lemma C.9. The integral (z) is a strictly increasing function of x when the loss function is TYPEN, and a strictly decreasing function of z when the loss function is TYPEP.
Proof. Consider æ1, 22 with 21 < x2- Then
MZ 1 ( = J. 32
J1200007 ( (1(2),0) ((0)))(((1,,,)))) ((=1,,)))
( (el.
First, consider the case of TYPEN loss function. Then for all v 2 (x1 v) - ((v and hence, the first integrand above is positive. We also know that for all v, E(22.)) v) z {(x1, v), and hence, the second integrand above is also positive. This shows that / (xx) is increasing function of a when the loss function is TYPEN.
Now consider the case when the loss function is TYPEP. For any value v 2 æ1, ((11 e) 2 E(v,, v) 0. Similarly, for 2 22, l(22, v) (()1 Thus, (x2) (B(x1) in this case.
Combining Lemma C.8 and Lemma C.9, we get the main result about the variation of ((x).
Theorem C.10 (Monotonicity of [xx) with respect to x)). Assume that conditions (A0) (A44 hold.
The function I(x) is a continuous function of x. For a TYPEN loss function, [(x) is a monotonically strictly increasing function of with (xx) going
goes to -oo (assuming Q is unbounded
from below). For a TYPEP loss function, [xx) is the sum of an increasing and decreasing function, and has global minimum Q Further, (ax) goes as goes to co -o (assuming these values lie in Qt).
Proof. First, consider the case of TYPEN loss functions. It follows from Lemma C.8 and Lemma C.9 that (xx) is strictly increasing function of x. The integrand in (xx) is [[x o]e(xr, v) (D
where I[] denotes the indicator function. As shown in the proof of Lemma C.8, this is a monotonically decreasing function of x. Therefore, the monotone convergence theorem [126] implies that I 1'(x) goes to 0 as x goes to -oc. Similarly, I "I() goes to -oo as I goes to -c. Similarly, in the case of TYPEP loss function, I (x)) goes to and R (xx) goes to oo as goes to -oo. Thus, [(x:) goes to co.
Similarly, 7 (z) goes to co as x goes to co. Since [xx) is the sum of an increasing and decreasing function, and infinite as goes to Do or 20, it must have global minimum Q.
Corollary C.11. Consider an instance I == (S2, ID, T) of PrimalOpt. If the loss function is TYPEP, then the optimal value is always finite. If the loss function is TYPEN and the optimal value is finite, then Ω is of the form (a, oc) or (a, b) for some a, E TR.
Proof. First, consider the case when the loss function is TYPEP. We exhibit a solution / with finite objective value. Let f be the uniform distribution over interval A of length e (by assumption (A0), such an interval always exists), Since / (z) is continuous, it achieves a maximum value on A A-let this value be p. Then ErIz, (fr. f) S p. Thus, we see that the optimal value for this instance is finite.
Now we prove the second assertion. Suppose, for the sake of contradiction, that the loss function is TYPEN and fi is unbounded from below. We claim that the optimal value for this instance is -oo, which will be a contradiction. To see this, consider a density f E Dom which is uniform on an interval = [s, i] of length e (by assumption (A0).. The entropy of this density is 7 and, hence, this is feasible solution to the instance I. However,
1 (fD. = [ (())(())d)())= = et S, (<))dp()) s 1(t).
Thus, we can keep moving the interval A to the left, which would mean that [tt would tend -00 (using Theorem 10). This shows that the optimal value of this instance is -00.
We shall use the following fact about the finiteness of optimal value when condition (A5) is also satisfied.
30

---

Claim C.12. Consider an instance I = (Ω,, a, of PrimalOpt satisfying conditions (A0) (A5). Then the optimal value always finite.
Proof. If the loss function is TYPEP, this follows from Corollary C.11. In the case of a TYPEN loss function, the optimal value is finite unless Ωt is unbounded from below. In this case, Theorem C.10 shows that argmingen (x) does not exist, and hence, assumption (A5) violated.
C.5 Positivity of the optimal dual variable
We now prove that the optimal dual variable y" is strictly positive. In this section, we assume that assumptions (A0)-(A5) hold. For this, we need certain technical results.
Lemma C.13. Consider an instance I= (9,, fD. T) of PrimalOpt with the loss function being TYPEP. Let x* be argmingeg 1 (z). Then there is value r > such that any feasible solution
must have Errz, aJD, 2 21(1211" 1(x)) +
Proof. Let xx denote the unique global minimum of I (xx on Q (using assumption (A5)). We show that there is small enough value & > such that any feasible solution f must place at least amount of probability mass outside Te = [x* - e, x² + e). This suffices for the following reason. We know by the assumption (A5) that there is positive value Ç such that [(x:) > I(x*) + for all x $ Ie. Thus,
ErD,,a(JD,I)) Jant. (())s())d)(1)+ + S. J (())S()))p(()))
2 1(x*) +
Hence, we can choose the desired value 7 to be EÇ.
It remains to find such e. We consider such a value and assume that feasible solution places strictly more than & probability mass inside Te. We need one notation: For an interval A, define
- Jmn (()dd(()))
Ent = =
Let be the mean of fo and let R be the half-radius of fD, i.e., fo places at least half of its mass in the interval im R, m + R] n f. Let Ag be a large enough interval containing x and the interval im R, m R. assume that the end-points of Ag at least Ag/ /2 away from any point in [m R, m + Rt.. We consider intervals of growing size around Ao. Let Ao = [50, 40]- Let A be the union of the intervals on both sides of Ag, each length e2/s_ Similarly, having defined A;-1 (which will be union of two intervals). define A; to be the union of two intervals both sides of A;-1, each of length e²".
Consider the feasible solution f. Let 3 be the total probability mass placed by f on A.. Thus,
Ent(f) Ent,, (f) Ent (f) + Σ Ent
We bound each of the terms above. Note that Ent (f) is maximized when we distribute B, mass uniformly over As. Similarly, Enty, (f) is at most In(e) and Ent I In |Ael = D. Thus, we that
A < In(eD) 8
S Ent(s) < In(eD) + Σ In
Σ
B In [A
In other words,
Σ 1
Observe that we can still choose E and, hence, we will choose such that the r.h.s. above becomes as large we want (ideally, much larger than p*). Observe that any point in A, is at least |A,-1]
31

---

distance away from any point in [m- R, m + RJ. Therefore, (and this is where we use non-negativity of the loss function, which has been assumed be TYPEP)
ETD,, Σ
B 2
(In A 0z)
3 ln [A
where we have used the fact that In [A.] = 2 In Ai-1) It follows from the above two inequalities
that if we choose 4 then E (J', ( f) 2 2p'. Hence, either / places less than mass inside Ig or its objective function value is more than 2p"
40
The above proof only worked for TYPEP loss functions. We need a similar result for TYPEN loss functions. The ideas similar, but need to the function [xx) in more subtle manner.
Lemma C.14. Consider an instance I (9,, fD. l, a, T) of PrimalOpt with the loss function being TYPEN. Let x* be argmin (z)) Then there a value 1j > O, such that any feasible solution must have Erre, (JD. 1(x") +
Proof. As in the proof Lemma C.13, we would like to argue that there a positive >> such that any feasible solution places at least amount of mass outside the interval Lg. Since the optimum is finite, Corollary C. shows that bounded from below, i.c., it of the form [a, or [a, o0). Theorem C010 now implies that x a.
Again, define the sets A, as in the proof of Lemma C.13, but now, we make Ag start from the lower limit of Ω. Thus, each of the A; will now be a single interval, with A, being to the right of A;-1- Now we use the fact that when the loss function is TYPEN, both the functions I (a) and (z) are monotonically increasing (Lemma C.8 and Lemma C.9). Further, for point E A,, Lemma C.8 shows that
(x) 1 (m + R) 2 (In |A--1l Om++)),
because the c.d.f. of fo at m + R is at least 1/2. Hence,
f Auna
(T* (²(x)) (=))))) a3: (In AA, 0m R)+
Summing the above over all > 1, and proceeding as in the proof of Lemma C.13, that by choosing a small enough e, (here to denotes the right end-point of Ao)
Σ (=))(())d)()) (x'))) (o.o))n
can be made larger than a desired quantity Q which depends only on the parameters of the input instance I. Since (R()) increasing function of also get
J (x)dd()) 2 + +
One remaining issue is that the integral on the l...s. does not include the terms for [a, to) (recall that I (xx) can be negative). However, observe that (xx) is an increasing function of x, and hence [(x) ((a) for all x. Therefore,
(Fo,,a(fo.f) (zz)((z)dp(z)
a
1/a² I(z)f(=)du(z)
too
S Srawsra (())(())((())
[(a) ((a) Q.
Again, by choosing Q large enough, we can set the above to more than (1)).
As an immediate consequence of this result, we see that for an instance of PrimalOpt satisfying conditions (A5),(A5), the optimal dual solution 7 is non-zero.
32

---

Theorem C.15. Consider an instance I I= (Ω, fD. a,T) of PrimalOpt satisfying conditions (A0)-
(A5). Then there is an optimal dual solution (7* x6) to DualOpt satisfying Y O.
Proof. Let p" denote the optimal value of this instance (it is finite by Claim C.12). Corollary C. implies that [x)) has (unique) global minimum x in Q. Assume, for the sake of contradiction, that 7*h = O. We claim that g( , = [(=)). Indeed, consider function f which places unit probability mass at x². Then is easy to verify that L(f, o*) = =xx)). However, Lemma C.13 and Lemma C.14 4 show that p* > [(x"), which is a contradiction (because by Theorem C.3, strong duality holds).
C.6 Optimality conditions and uniqueness of optimal solution
We now show that, under suitable conditions, there an optimal solution to an instance of PrimalOpt. Theorem C.16 (Optimality condition). Consider instance I of the optimization problem Pri- malOpt defined on a space Q. Let l, o,, T, f' denote the loss function, risk-averseness parameter, resource-ings coource-ifformation parameter, and the density of the true utility respectively in I. Assume that the optimal value for this instance is finite and assumptions (A0), (A3) hold. Let ("","") be an optimal solution the corresponding DualOpt with 7² > O. Consider a function I E Dom defined as follows:
(x) + In + 6" = 0,
Vr E
(12)
Then f is an optimal solution to the instance I.
Proof. Let pt denote the value of the optimal solution to Since pt is finite, Theorem C.15 shows that there is an optimal dual solution satisfying 7* > 0. Recall that
9(7"" , ,6) = min L(f, ,d)) De
Let f be the function defined by (12) (z) is well-defined since 7" O, this is where need strict positivity of y*). We argue that L( y ,0)) = 0(7",0"
Indeed, consider any other function h Dom. Define a function 0 [0, 00) - R as follows:
O(1) T-(1 (x) L.(x)), x²) (x) te(x)) ,
where e(x) = h(x) / (a:). We first claim that (t) a convex function of
Claim C.17. The function 0 [0, 1] + R is a convex function.
Proof. We observe that Err(f" +t-e, fD) is a linear function tf The function ((tfff +t-e)) is convex function of f, and ò (f))((())) + te(x))dp(x) - -1) is linear function of t. Therefore, (()) which is the sum of these three functions, is convex function tf
di 1= Along with the convexity of @(t), this implies that ((0) < ((1) and hence L(f", ,-, o+) s L(h,,",,0" A routine calculation shows that db(t) 1== is equal to
We shall show that d0(t)
J (1)). + +1 + In (x)) + Φ²) e(z)dn(x))
Inequality (12) implies that the above expression is 0. This proves the desired result. Thus, we have shown that L(ft, 7*, x+) = g(r", o*), and therefore, f is an optimal solution to the instance Z. O
We now show the uniqueness of the optimal solution to PrimalOpt. Consider an instance of this optimization problem specified by 2 loss function E, density fn and parameters a, T. Let F denote the set of feasible densities, i.c..
F = {f : f E Dom, f is density on Z and / f(x) In f(z)d((z) S -T).
92
33

---

Lemma C.18. F is strictly convex.
Proof. Let J1, /2 E F and À be parameter in [0, 1). We first show that the density f defined by f(x) = (1(x)) + ()) a E Q, is also in F. Clearly, f is a density, because (xx 2 0 and
J f())dp(x) / fi f(())d((2) + (1 - >) fa(z)du(z)) = (1-
Hence, the fact that g(v) = y Iny is strongly convex function on R2 implies that
f() In f(z) - (f) fi() + +1 ())2(2) In f2(2)),
with equality if and only f1(2) fz()). Integrating both sides, and using the fact that fi, f2 belong to F, we get
1 f(x) In f(()dd()) s - A)T
Thus, if (x) and f2(x) differ on a set of positive measure, then f(x) f(x) < <x)()> + (2)) set of positive measure. Integrating both sides, get Ent(ʃ) > This shows that the
set F is strictly convex.
Striet convexity of now allows us to show the uniqueness the optimal solution.
Theorem C.19 (Uniqueness of optimal solution). Consider an instance = (Ω), fD. l, a, of PrimalOpt satisfying the property that the optimal value is finite and there is an optimal dual solution with 7 > 0. Then there is a unique optimal solution to this instance.
Proof. Let p¹ denote the optimal value of this instance. Theorem C. already shows the existence of optimal solution. We show the uniqueness of optimal solution. We have assumed that there is an optimal dual solution (r* σ²) such that 7* > 0. The complementary slackness condition shows that for any optimal solution f to the instance Z. Ent(f) must equal т.
Suppose, for the sake of contradiction, that there are two distinct optimal solutions f and f to (i.e., (x) * f2
subset of positive measure). Consider solution f" tfi + (1 - f) for some E (0. 1). Linearity of ( f) shows that Erre,a f.f..ff f*) p" as well. Now f is also feasible solution by Lemma C.18; in fact, this lemma shows that Ent(f*) > But this contradicts
the fact that every optimal solution must have entropy equal to r. Thus, we see that there must be unique optimal solution the instance I.
Combining Lemma F.2, Theorem C.16 and Theorem C.19, we see that Theorem C. holds.
C.7 Conditions under which optimal solution is fD
We show that our optimization framework can output the true utility fo for a choice of the parameters a, r. and the loss function é.
Theorem C.20 (Parameters that recover the true utility). Consider an instance I (Ω, fD. L, T) of the optimization problem PrimalOpt. Assume that the density fo has finite entropy.
Suppose, a = 1, t Ent(fp). and the loss function E(x,v) = In fD( In fD(x). Then the density fo is the unique optimal solution to PrimalOpt I.
Proof. We claim that, for the values of dual variables 1 and Φ = Ent(ʃo) y(7, Φ) =
L(JD,, 7, 6). Towards this, we show that fo minimizes T(f, 7, over all ʃ Dom. We first show that fo satisfics the condition (12) with 7 0. 6, f fD; it shall then follow from exactly
34

---

the same arguments as in Theorem C.16 that fo grgmin 7. 6). Now, we check that fo satisfies condition (12) (recall that the loss function ((z, v) = in fD(U) - In fr(r)):
((x) + 7(1 +1 (n + x= /
+
+
(n(c) (o(o))n - in (n(z)) J ( + (1+ Ent(fr)
5 = Ent(ff) - ln fD(x) + (1+ (o Ent(fo)
This proves the desired claim. Thus,
- L(fD 7, Erre, (fD. fD).
Thus, g(7, o) is equal to the objective function value of PrimalOpt f = fD. Hence, fo is an optimal solution to PrimalOpt for I. In order to prove uniqueness, note that the above argument also yields an optimal solution (7, Φ) to DualOpt. Since y > complementary slackness conditions imply that any optimal solution must have entropy exactly equal to т. Now, arguing as in Theorem C.19, we see that there unique optimal solution.
D Derivation of output density for a single individual
In this section, consider the setting when single individual with true utility is being evaluated. In this case, the input density fD(z) is specified by the Dirac-delta function centered at v, denoted (2). Using Theorem 3.1, we characterize the output density follows:
(S, fp. E, a, r) of (OptProg) where =R, fo 8 (x) for some real f(r, u) = (x u)², and and are arbitrary real parameters. Then the optimal
Lemma D.1. Consider an instance
density given by
Ke
SxSu S
(x)
Ke
otherwise
where K the normalization constant and 7/t is the optimal dual variable for the entropy constraint.
The mean of this density equal V x va-1
Proof. We first evaluate the integral ((x) follows:
I(x) = S fo(())))))) fo(u))a(zz, u)de = a [ (z(u)(x- - u/2 1 8o(u((z u)2.
When < the first integral the r.h.s. is O, and hence, the above integral is equal to (a t² Similarly, x > v. the above integral is equal to u)² The expression for (x) in the statement of the Lemma now follows from (12).
2/m routine calculation shows that the normalization constant K is equal (I+vVa mean this density turns out to be
Now the
Va 1
Va
35

---

E
Connection to the Gibbs equation
Theorem E.1 (Gibbs equation), Consider an instance (9,, fDD,,De,, a, T) of the optimization
problem PrimalOpt that satisfies the assumptions of Theorem C.I. Then, the following holds:
-Y" In Z = Er ; + + Ent(ff).
ffD)
(13)
Here, (x) x e Hip
the solution to PrimalOpt, Z = z= Je e dp(x), and 7- > is the solution DualOpt. Recall that [(x) Ja (z ()))dd)))(()
Proof. Theorem C.I implies that there exists (x) and y* such that
f xe Ke
Thus, we let Z = _e dp(x)), then
7
(x
Z+
Thus, from the optimality condition (12), we obtain that there a o² such that
(xx) + 7" (1 In f (z)) o- =0. 0.
Since >² >> 0, we can divide by it to obtain Z = =el+e 1+ We integrate the above with respect to the
density f*(x) get
Errg, (ftr,a(",,, + - - =0.
Thus, we obtain:
7"T "T==""En((f))
+ +2" In Z"
(14)
Rearranging this equation we obtain the theorem.
In analogy with the Gibbs equation statistical physics [94], Z* can be viewed as the partition function corresponding to the energy function [xx). 7" corresponds to the lemperature, -7" In 2* corresponds to the free energy and Err(f*,J .DD) is the internal energy.
It follows from the theorem that we can write (x)
[(x) - ErTg, (f", ff)
((x)=e=+ exp
exp
(15)
Effect of changing the resource-information parameter T
F.1 Effect of decreasing
Theorem F.1 (Effect of decreasing T). Fix a 2 1, a density fo. and loss function L. Assume that
the function (.) satisfies condition (A5), and x² == argmin (t.tta Given a T, let ft denote the optimal solution to PrimalOpt. For every 8 >> 0, there exists a value T3 such that when
7 < To, the solution f² has the following property: For every 2 I x²1 2 5, we have
fx()
88.
In other words, the density outside an interval of length around has a much smaller value than
at x².
36

---

In order to prove the above result, we first show that the optimal dual value y goes to 0 as goes to -oo. Then we shall use Theorem E.1 to show that the optimal density highly sensitive to small changes in (x:).
Lemma F.2. Fix an interval Ω S IR, parameter a 2 1, density J'D of true utility and loss function Assume that the function fo.t,o has a unique global minimum. Let I, denote the instance (St fD. L, T) Let be the optimal Lagrange variable corresponding to this instance (assuming it has non-empty solution). Then
lim 0.
Proof. We first observe that the instance I, will always have a feasible solution for small enough т. Indeed, when e" is less than the length of Ω there always a feasible solution to I,. Let To be value of 7 for which the instance I, has feasible solution. For sake of brevity, let (xx) denote (o,tta( (x),, and let x² = argmin, (x)). We first argue that Err(J.: remains bounded range.
Claim F.3. Consider value of r S To. Let ft be the optimal solution to the instance I,. Then T(x*) & Errz,o (JD, f7) S ar,
Proof. The first inequality follows from the fact that
Err Erre, = (()))(()))))))) 2 T(x").
The second inequality follows from the fact that the solution f% is also feasible solution for the instance I,.
Suppose for the sake of contradiction,
lim 7 > 0. T 00
In other words, O - 77 0. It follows that there is an infinite sequence A (To
T1 >...) going to - oc such that 97, 2 0b/2 2 for all T; E A.
Consider interval of finite but non-zero length in Ω- let U [s, be such interval. Since U is closed and / (x) is continuous function, there are finite values ao. bo such that ao S [xx) S by for all U. Thus we get:
Claim F4. There is a positive real such that
(fD. f2) - 1(x) 2
exp
77
holds for all x E U.
Proof. It follows from Claim F.3 and the observation above that for all x U and all To (fDr, I(x) lies in the range [[(=*) bu. Erre, (fD. f7) + au). Since yr 2 0//2 for all A, the result follows.
The above claim along with Theorem E.1 shows that f+ > n A contains an infinite sequence of values going to -00, we can choose a value such that 1 for all z E U, which is not possible because f* is density.
T for all x E U, E A. Since
ne- 1/ But then fz(z) > [0] This proves the lemma.
D
now prove Theorem F.I. Consider the instance as stated in this statement of this theorem. Let To be value of the informatiionressouree parameter for which there is density with entropy 7 in Ω (i.e., when PrimalOpt has feasible solution). Recall that x² arrgmin seo [xx). Consider a
Assumption (A5) shows that there is value Es 0 such that I [(x) 1(x*) > for all satisfying
37

---

x x²1 2 8. Now consider an z such that |x x²1 2 8. Using Theorem E. we see that, for all 7 S To
(x) = exp T(x') [(x) fx
Σ cxp (-66/17);
7
where the last inequality follows from the fact that 7, is positive. Now, Lemma F.2 shows that there value T3 such that for all 5 - Ts. 0 Im[10] Therefore, for all æ, x²1 2 6,
f+(x) f1(x*) 8
This proves Theorem F.I.
F.2
Effect of increasing
In this section, we consider the effect of an increase in T on the variance and the mean of the optimal density.
Theorem F.5 (Effect increasing on variance). Consider a continuous density fo on TR, loss function and information-resouu
parameter a. For given risk-averse parameter т, let I, denote the instance (IR. fD. l, a. 7) of PrimalOpt. Let f be the optimal solution to the instance Iy- Then the variance of f is at least 2% 2
The proof relies on the following result.
Theorem F.6 (Gaussian maximizes entropy; Theorem 3.2 in [48]). For continuous probability densiry function f R with variance o², Ent(f) s In (2Ta") with equality if and only if Gaussian density with variance o².
Proof of Theorem F.5. Consider the instance I, and the optimal solution f for this instance. Since f² feasible solution, Ent(f;) 2 T. ft, has unbounded variance, then the desired result follows trivially. Hence, assume that f has bounded variance, say o². Now, Theorem F.6 shows that
1 1 2
Ent(ff)) < + In(2xo²))
Using the fact that Ent(fz) 2 the above inequality implies that 0² 1 2T-1 This proves the desired result.
25
We now show that the mean of the optimal density also increases with increasing when the input density JD is supported on [0, oc).
Theorem F.7 (Effect of increasing on mean). Consider a continuous density Jo on == [0, oo), loss function and information-resource parameter a. For given risk-averse parameter T, I, denote the instance (Ω, fD, E, a, r) of PrimalOpt, and f* denotes the optimal solution to the instance I, Then the mean of f is at least e-
The proof relies the following result.
Theorem E.8 (Exponential maximizes entropy; Theorem 3.3 in [48]). For continuous probability density function f on [0, oo) with mean X, Ent(f) s 1 In (A).
Proof of Theorem F.7. The proof proceeds along similar lines that of Theorem F.5. We know that Ent(f=)) 2 because it feasible solution to the instance I,. If fr has unbounded mean, we are done; therefore, assume its mean, denoted X, is finite. Theorem F.8 shows that Ent(ʃ) S + In X. Since Ent(f;) 2 T, we see that À 2 e r-1 This proves the theorem.
38

---

G Effect of changing the risk-averseness parameter a
Theorem G.1 (Monotonicity of [(x)) with respect to a)). Consider instance I (9, ID, of the optimization problem PrimalOpt. Then, for any I E Ω, ( (T) is an increasing function of a.
Proof. By definition
/ la(z, (fo())))))))))
/ A(x, ((o,))((())))())) o2E
Ia(x) = I),ca(()
Juga
Recall from (4), that for any x, U E 92 la (x, = ((x, v) if and (a u) = alla, u) when 2 v. Since our model requires f(r, v) 2 20 whenever x 2 u, (a( 2 (a, v) for any x and a 2 a'. Thus, we have that (x) 2 (x) for all x f2
Theorem G.2 (Monotonicity of Err with respect to a). Consider instance In = (Ω, JD. l, T) of the optimization problem PrimalOpt. Suppose Ia satisfies the assumption of Theorem C.I and let la be the optimal solution to instance Lex Then, the function Erre, JDD is an increasing function of a.
Proof. As noted in Section C, if the instance I satisfies the assumptions for = 1, then the instances obtained by changing a continue satisfy the assumptions needed in Theorem C.1. Thus, we may assume the optimal solution exists for each version of I where we vary a 2 1, and let fa denote the optimal density. We first show that for any fixed density f. (rre,a(f, f, fo) is an increasing function of a. This is so because
f. ( -
E(x, E(z,, (((e)))((()))) fo(()du(()
Efrea(f,fD)
+ ffe 1zz
(
la(x, ()) fo(()du(v)
and (x, v) an increasing function of a for any Ir v Q. Consider two values of the parameter a: 1 Σ < 022. Note that the instances corresponding to both a₁ and ag are feasible as only appears in the objective and, hence, does not affect feasibility. Suppose for the sake of contradiction that Errz.ag JD) < Frrg,o, ( Jā, satisfies Ent( 2 7 as it is feasible solution of the problem instance defined by 0r and, hence, is also a feasible solution for the problem instance defined by a1 This and the definition of fa, imply that ErTç,o, (fa, fo) S Erre,o, (f fD). Thus, we get Erre,a, (fāz fD) < ErTE,a (fag fD), which contradicts the (above observed) monotonicity of Errz,a (Jág J'D) with respect to a.
H Gaussian density
The Gaussian density is defined follows over Ω = IR and has parameters m R and
(u-m)) 2
fo(x)
2
I E IR
V2T02
m is the mean and a² is the variance. The differential entropy of Sa is + 2 ln(2TT²)) [148]. We consider the loss function to be l(x, v) == (z - u² First, we compute the expression of (xx which we use to verify the applicability of Theorem C.1 with the above parameters.
Lemma H.1 (Expression for (x)).. Consider an instance I = (S) fo. e, a,, of PrimalOpt where RR, (x,,0) v) == (x u)?, and fg is the Gaussian density with mean m and variance o². Then
T(x: = (a 1002 (ww² + 1)b(w) + wo(w)) + o² (w² +1)
where t := I- ((x)
.2x c T, and (xx) = S o(z)du(() denotes the cumulative
distribution function of the Gaussian density.
39

---

5- vs.
Plot of Vateodd
0.65
1.0
0.8
0.55
Vat...
0.6
0.50
0.10
0.2
0.2
0.1
0.5
0.8
1.0
0.2
0.4
0.6
0.8
1.0
(a)
(b)
Figure 3: Mean and variance of the output density i... Ee-.y [ar and Varz~, [2)] as function of when JD the standard normal density and a fixed to 2.
Proof. By definition,
/ (x f(())dd(() [ - (0 f(())d((())
((x)
We first make a change variables. Let w == z- and y == p-m Thus, the density y is Gaussian with mean 0 and variance 1. We denote this density by di. The above integral becomes
1
(w - [o( + +op [((= (w- (o()dduu(()
00
20
10
(w² +3² 8² 2wy))(()dd(())+++++ + (n [(w² (=²+²² (u--2)))(())du(()
= ao²
+o? (w + 8² 2wy)o(v)du(z) o²
1
f (w² + 2wy)
= ao²
00
+ 2
(w² + 22 2wy))(v)dd(())
-00
(a 1)0²
(10 (ww) (ww) wp(w) + 2wv((w))] + 0² (w² +
=
(a- 1)0² (ww² + ())(w) wv(w)) o² (w² +1).
Applicability of Theorem C.1. We verify that any instance I (2, fg, a, of PrimalOpt defined by Ω R, fg as Gaussian density, finite a, and satisfies the assumptions in Theorem C. Since = R, (A0) holds for any finite r. (A1) and (A2) hold due to the choice of the loss function. (A3) holds since fg has a finite variance. (A4) holds with, e.g., R 0². In Lemma H.1, we compute
T(x) = (a 1)o² (ww² + (w)) + wd(w)) + 0² (w² +
where From this expression, follows that (xx) is differentiable and
8 T = 2(a (w)(w)
82
Since a 2 and (--) is non-negative, it follows that [xx) is strongly convex and, hence, has unique global minimum. Therefore, (A5) holds. Since assumptions (A5)-(A5) hold, we invoke Theorem C.1 to deduce the form of f*.
Figure 3 plots the mean and the variance of the output density f* as function of the parameter when the input density fo is the standard normal density. Figure plots the mean and the variance of the output density
function of the parameter in this setting.
1). Consider an instance I = (Ω, JD, 1, of Pri- ma10pt where Ω TR, f(x, u) = (a u)), and IG is the Gaussian density with mean m. and variance d², Let f be the optimal solution of I. Then f* is a Gaussian with mean and variance 1 2e
Theorem H.2 (Expression for f" when
40

---

Ex-dd vs.
Nat,...
1.0
0.825
0,300
Var...ss
0.7
0.750
0.6
0.725
0,700
0.655
0.60
(a)
(b)
Figure 4: Mean and variance of the output density f. i.e. [ and Var [ir] as a function of a when So is the standard normal density and r is fixed to the entropy of fD.
Thus, for 1, increasing does not change the mean, but increases the variance of the output density.
Proof. For 1, Lemma H.1 implies that
[xx) = (ar m)² + o
As shown earlier, by Theorem C.1, f has the following form
fx))) x -(
X
where the proportionality constant and x are determined by Ja (x))dp((z) and Ent(f*) = f* is a Gaussian density with mean m and variance 3/6 and, hence, Ent(f*) = + (n
[148]. Since Ent(f*) = т, the previous equality implies that 7" = x _e2r-1 It follows that
(x)x) V2ch exp (- (=m=") which is the Gaussian density with mean and variance
1 2
Shift in mean with increase a. We consider the effect of the parameter a on the mean of the output density. We consider instance Iι (S2 fv, a, where f R, fo is the normal density ([,,o2), 2), 2) (rr 2)2 Ent( fD) /2Tre and 1. We know from the proof of Theorem H.2 that the output density f* is the same as fo and hence, has mean 0.
Now consider an instance Zz, that corresponds to the disad vantaged group and has the same parameters as that of I, except that the parameter a is larger than 1. Let fa denote the corresponding output density. We know from Theorem 1616 that the output density proportional to (=)) where % is the optimal dual variable for the entropy constraint and Lemma H.1 shows that La (x) is given by:
(x) (a 1)o² ((w" (w)(w)) + ww(w)) + o² (w² + 1),
where = (x p)/o. or. For sake of brevity, let g(w) denote (w² + 1))(w) + wo(w), and assume w.l.o.g. that μ = and = 1, i.e. the input density is standard normal. Thus, (x) can be written as (a 1)g(z) + (x² +11 1). When is large, the first term here dominates the second term long as (x)) > 1/a. Since ((x) 2 1/2 for x 2 0, Ia is relatively large for all x 2 O, and hence, (z) goes to 0 for [0. oo) as increases. In fact, g(x) is much larger than 1/ax when is larger than - /I((a) and, hence, f* should be small for such values of well. Therefore, we expect the mean of the output density f to to -co as a goes to co. Further, the mean of ft should decrease a logarithmic rate with respect a. We verify these observations numerically Figure 5.
Deriving the implicit variance model. We show that the implicit variance model can be derived from optimization framework. In particular, we prove the following result:
Theorem H.3. Consider an instance of the implicil variance model given by parameters μ., o, 0g- Consider instances In = (Ω,, Sg. f, at, TI) and Tn = (S. Sg. l, ar T2) of (OptProg), where RR fg is the normal density N (u, (p,o%)
1. f(v.2) (x v)
_[(1 In(2TTof) and
41

---

Cuiput Densty
044
Standard Normal
Moan of Ourput Denetty
Density Value
0.8
0.2
(a)
(b)
Figure 5: Figure (a) plots the input density, i.e., standard normal density, and the output density when the parameter a 1000. Figure (b) plots mean the output density as function of the parameter (on log scale).
T2 1(1 In(2 + oR))). Then the output density of (OptProg) 11 is N N(, o.) and the output density of (OptProg) is N (p, oB )²))
Proof. The result follows from Theorem H.2. For the instance Zı, Theorem H.2 shows that the output density of OptProg is the normal density with mean μ and variance
2r -1 oō-
2x
Similarly, follows that output density for the instance I2 is normal with mean μ and variance
1
252-1 = où o².
I Pareto density
The Pareto density is defined as follows over Ω [1, oo) and has parameter B >
B
fr(x)
oo).
2011
The mean of this density is 8 T-T. Thus, the mean is finite only when 3 > 1. Its differential entropy + + In [148). We consider the loss function ((x, = In - In c. First, we compute the expression for T(x) which we use to verify the applicability of Theorem C.I with the above parameters.
Lemma I.1 (Expression for Ia (z)). Consider an instance I = (St fr,fff..a l, 7) of PrimalOpt where Q 10). (( v) In In v, and fp the Pareto density with parameter 3 > Then
[(x) In
3
8
Proof. We use integration by parts to derive the following expression for T(x):
B
+ { no
I(x) = a
(Inx
In
du(u)+
(Inx In v)
du(v)
In
In
I w++I du(v) +
In In x
00 1
b0t1 du(v)
6
a
= =alnx
B -1
8
3
42

---

Plot of Eu-da) vs.
Plot of Vaty- (a) vs.
3.4
28-
Vac-AMt A1
4-
20
2-
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
(a)
(h)
Figure 6:
Mean and variance of the output density i.e., Ex (ar) and Varz-> (2t), as function of when Jo the Pareto distribution with parameter's and fixed to 2.
Applicability of Theorem C.1. Next, we verify that assumptions (A0)-(A5) hold. Since [1, oo). (A0) holds for any finite (A1) and (A2) hold due the choice of the loss function. (A3) can be shown to hold since fp a Pareto density: see this note that for any finite x
S. (((z, v) fr fo(v)du(o)
+ / (m()) fr(v)du(U) J.a (In(v) - In(x)) fo (v)du())
/ 10
$
(In(x) + (nn())) fr(u)du(u)
-In(x)
In(U)fp(o)dp(())
In(v)
=l((x) +
+
wh+i petrdu(()) du(v)
wー
= -n(())
3
In(x) +
00.
Thus, (A3) holds. (A4) holds with, e.g., R 2. By Lemma I.1,
(xx a I + a a
3r 8
Thus, [(x) is differentiable each x Ω. Moreover, for all Ω
0I(x) a a a-1 Ox
a x21 0.
3+1
[1, oo), it follows that /(x: has unique global mini- mum at x = 1. Therefore, (A5) holds. Since assumptions (A0)-(A5) hold, we invoke Theorem C.1 deduce the form of f*. Figure 6 plots the mean and the variance of the output density f function of the parameter when the input density fo is the standard normal density. Figure plots the mean and the variance of the output density function of the parameter
Since the derivative is positive for all
this setting.
Theorem 1.2 (Expression for with = 1). Consider an instance I = (N, e, 1, of Pri- and fe is the Pareto density with parameter B> 1. Let
malOpt where f be the optimal solution of instance I. f is a Pareto density with parameter B, satisfying the following condition:
oc), == In
1 + 3-
---In In B, =T.
43

---

Plat of Vareo vs,
1.96
2.4
1.94
1.92-
1.90
1.8
1.88
1.85
10
(a)
(b)
Figure 7: Mean and variance of the output density i.e., [g-s[[] [x]] and Var [xr]. function of a when fo the Pareto distribution with parameter and fixed the entropy fD.
Let m, and of be the mean and variance of f+ as a function T. It holds that m, is monotonically increasing in T and o² is either infinite or monotonically increasing in T.
Proof. For a Lemma I.1 implies that
[ In(x)
3
As shown earlier, one can invoke Theorem C.1 for instance I for any finite which implies that f has the following form
In(x)
(*(x) x e
X
where the proportionality constant and 7* are determined by Sa (((=dd((z)) and Ent(f*) f is a Pareto density with parameter B, 7" and, hence, has differential entropy of 1 1+ + + In क. [148]. This combined with the condition Ent(f*) implies that is the Pareto density with 3, satisfying
1
+ +
Br
3
Since + 1 + In B, is decreasing function of 3r, it follows that increasing monotonically decreases By. Note that 3, > for any finite т. The mean and variance of f* are m, - and 0² 15.-11- [B.-1)"B,-2 2J respectively. Since m, is monotonically decreasing function of 3, (for 3, > 1) and 3r is a monotonically decreasing function of it follows that m, is a monotonically increasing function of B,. The variance of finite when B, > 2. Moreover, if [ >> then of is monotonically decreasing function of 3r. Since 3r is monotonically decreasing function of r, follows that of is either infinite or a monotonically increasing function of 9,,.
3r
9.
Reduction in mean with increase in a. We show how our framework can capture similar phenomenon as the multiplicative- model of [90]. Recall that in the multiplicative-bin model, the estimated utility of the disadvantaged group is scaled down by factor 1.
Fix parameter B
1. Consider an instance I₁ given by the parameters I (Ω) [1, oo), fD. C, a,,)) T) where fo is the Pareto density with parameter B, Ent(fv) a,
3 a and f(x,, v) = In x In v. shown Theorem 1.2, the output density f* is the same fo. The proof of this result also shows that the output density f*( proportional -(x))//>' where (xx) = In x and 7 is the optimal dual variable for the corresponding entropy constraint.
The disadvantaged group is modeled by an instance Ig which has the same parameters as that of Z except that the parameter is larger than 1. In this case, Theorem I.2 shows that the optimal density fū (z) is proportional to e -Ia (w)/// where lx(x) a In
3xd - and % is the corresponding
44

---

Limiting Ousput Dersiry input density
Menn Pareto Denaty
Mose of limiing Deruur Dondity
Moan the deceiity
drestry value
65
2.5
45
14 16
22
2.4
24
(a)
(h)
Figure 8: Figure (a) plots the input Pareto density with parameter 3 = 1.5 and the corresponding limiting
output density. Figure (b) plots the mean of the input Pareto density and the corresponding limiting output density as function of the parameter B.
optimal dual variable. For large (z) a(((x) + Bas) Hence, the output density (for large a) given by
fx) = K (1(0)
(1()+
where K is normalization constant. The normalization constant K and the ratio Y" /a are given by two constraints: (i) integral of fo over the domain should be and (ii) the entropy of ft should be equal to r Ent(fv) fo). This shows that the ratio % /a tends to a constant for large a, and hence, the output density converges the density given by
g'(x) = Ke -C Ke-C(((
We plot the output density (z) for 3 = 1.5 in Figure 8(a). In Figure 8(b), we observe that for all considered values of the parameter 3, the mean of the limiting output density g always remains below that of the input density fp. It is also worth noting that for large 3, the gap between the mean of the input density and that of the (limiting) output density diminishes. Intuitively, this happens because as 3 increases, the mean of the input (Pareto) density gets closer to (recall that the mean of a Pareto density with parameter /3 is equal to h-و ). Now the output density also places more mass closer to 1, but gets restricted because of two conditions: (i) the entropy of the output density must be the same as that of the corresponding imput density. and (ii) it cannot place any probability mass on values below 1. Hence, there is not much "room" for the output density to place extra probability mass on values close to I (as compared to the corresponding Pareto density). Hence its mean cannot go much below that of the corresponding Pareto density.
Exponential density
The exponential density is defined as follows over S = [0, oo) and has a parameter À > 0:
f (I))
de An r E [0, oo).
À is referred to as the "rate" parameter. The mean of the exponential density is 7. The differential entropy of JEsp 1 In À [148]. We consider the loss function f(x, v) == x v. First, we compute the expression of [xx which use to verify the applicability of Theorem C.I with the above parameters.
Lemma J.1 (Expression for ((x)). Consider an instance I fExp: E, a, T) of PrimalOpt where Ω = 00 cc). fEsp is the Exponential density with rate parameter X. and l(xx v) == Then
1 ((x) ā (a(Ax 1)+ (a 1)e-2a)
45

---

Proof. The desired integral is
Xa ! (x (x- ule Ae dx(pp) X f-d (s- do du(o).
We do a change of variable, with y = x - above, to get
Ace Sf" ye du(w) + f ye (u(v)) = Aave Ax
Ay Ay 17
Ay 17
Ay
x2 10
12
a(Ax
1)
ap
Au
-x
(a(de - 1) + (ax
1)e Az)
Applicability of Theorem C.1. We show that assumptions (A5) hold. Since f2 = [0, oc). (A0) holds for any finite (A1) and (A2) hold due the choice of the loss function. (A33 can be shown to hold since fExp is an Exponential density: To see this note that for any finite x E f
L 100,, v)) FE (U)d(u)))
/ (or (a-vi) v) SExp(v)du(()) [e (u-x) x)) fExp (v)dp(v)
$ { (lar + (el) (E)p(U)d
|a||
+
00.
Thus, (A3) holds. (A4) holds with, e.g., R = 3.- By Lemma J.1,
(xx) 1 7 (a(xs- - 1) + (a -1)e=>g)
From this expression, it follows that (xx) differentiable at each x E St Moreover, for x E Ω
8 11(x) A(o= -1ee
8r
For any > 1, this derivative is positive for all E Ω and, hence, [(x) strictly convex whenever and, thus, has a unique global minimum. If unique global minimum at = 0 over Ω = 10, oc). Combining with the > i case, it follows that
then [(2) = x This function has
(A5) holds. Since assumptions (A5) hold, one can invoke Theorem C. to deduce the form of f
Theorem J.2 (Expression for fr when = 1). Consider instance I (S2, fEssp= e, 1, of PrimalOpt where Ω = [0, oo), 8(1, v) == x - v, and fExp is the Exponential with rate parameter X. Let f* be the optimal solution of Then f is the Exponential density with mean e
Thus, for a = increasing increases the rate parameter of the output density.
Proof. Since a 1, Lemma J.1 implies that
[(x) 1 X
As shown earlier, one can invoke Theorem C.1 for instance I for any finite t, which implics that f* has the following form
f*)x) exp (--- exp 7"
where the proportionality constant and 7* are determined by Sa I (x)du(x) = and Ent(f*) = T.
Since I is an Exponential density with rate parameter 7. its entropy is Ent(f*) = + In (7*)
[148]. Since Ent((*) T, the previous equality implies that 7* --1, It follows that f*(x)
dlT exp (-xp(T-TT which is the Exponential density with mean e=1
46

---

K Laplace density
The Laplace density is defined as follows over Q = R and has parameters a E R and b >> 0:
1 #
fc(a) = 2b
x E IR.
is referred the "location" parameter and as "diversity." The differential entropy of fc is
1 ++ In(26) [148]. We consider the loss function to be ((z, v) = lr - al al for a E R. First, we compute the expression for (x) that we use to show that Theorem C.I is applicable with the above parameters.
Lemma K.1 (Expression for I (x)), Consider instance I (2, a, of PrimalOpt where = IR fc is the Laplace density with parameters a E R and > 0, and ((x, v) = lar of - al. Then
[(x) = ab(w - 1) + (la=ي) E
if otherwise
b(w 1) + b 611m)) e"
where w == I-a b
Proof.
The desired integral
25 (|x al - 10 al)e-lu-ll/ du(c))
2b / (|x al - |v al)e-lv-al du(v).
1
We perform a change of variables with w = 2 and y = b
Then the above integral becomes
ab
lylee- du(y) + I 0
(lwl - Iylee 1
(lw) Iylle -ie-iildph vldp())
Now two cases arise (i) w 2 0, or (ii) w S O. First, consider the case when w 2 0. Then the above integral becomes:
1 / (w ddply) (w + dp( + (" (w g)e *dp(y) + 2
2
a(w 1) + a(w -
1)
2
(a 1)b
= ab(w - 1) +
e
2
In the second case, we get
a
2 ( (-w w +
+
0
(-w+
g) +
1 (-w -)- -ddlly)
2
a E"
1 1 w
2
(1 =b
-ww + 1)b
2
Applicability of Theorem C.I. We show that for any finite E R and b > 0, assumptions (A0)-
(A5) hold for the instance I = (2, Sc, E, a, PrimalOpt where TR, fc is the Laplace density with parameters E IR and b > 0, and f(z, v) lx - al - lv a). Since f = E. (A0) holds for any
47

---

finite т. (A1) and (A2) hold due to the choice of the loss function. (A3) can be shown to hold since fc is Laplace density: To see this note that for any finite I f2
J (((x, -
000
al + 10 al) fc(v)du()) (1x
0
b 2 < 00.
= r
al
+
Thus, (A3) holds. (A4) holds with, e.g.. R b (2n. By Lemma K.1,
[(x) = ab(w - 1) + اLري e if 2 b(w 1) + 6 امعااع e" otherwise
where Z-a 5 One can check that (1x) is continuous and differentiable at each x E f2 (a).
01(x) 2 Hence, it follows that [xx) has Moreover, for all < a, @1(z] and for all x 2 a, üz de
unique global minimum at z Therefore, (A5) holds. Since assumptions (A0)-(A5) hold, invoke Theorem C.1 to deduce the form of the optimal density.
Theorem K.2 (Expression for f* when of PrimalOpt where S = R. fn is the Laplace density with parameters E IR and b > O, and f(x, v) I al I1 a). Let f be the optimal solution of Then f is the Laplace density with parameters (a, e" -0ee----0 /2).
1). Consider an instance
(L,, fc, l, 1)
Thus, for a 1, increasing r does not change the location parameter, but increases the "diversity" parameter of the output density.
Proof. Since a 1, Lemma K.1 implies that
(xx) = |r 2
As shown earlier, one can invoke Theorem C.1 for instance I for any finite т, which implies that f" has the following form
|xr a 2
p - al al )
f*(x) & exp
x exp
where the proportionality constant and 7* are determined by Sg (*(z)d((z) = and Ent(f*) = Clearly, f* is a Laplace density with the diversity parameter y", its entropy is Ent(f*) 1 In (2 [148]. On the other hand, since Ent(f*) T, the previous equality implies that y- _e""
follows that (*(x) cl-T exp (a, fe--))
2/2-al which the Laplace density with parameters -xxp(r-1))
L Implementation details and additional empirical results
In this section, we present additional discussions and evaluations of intervention in the JEE setting (Appendix L.1), plots omitted from Section 4 (Appendix L.2). and implementation details of our model (Appendix L.3). The code for this paper is available at https: attps://githuu com AnayMehro tra/Bias- in- Evaluation. ieraatiiioPPrrosss...
L.1
Case Study: Evaluating bias-mitigating interventions in ITT-JEE admissions
In this section, we continue our study of the effectiveness of different interventions in downstream selection task. Like in Section 4, we consider selection based on the JEE 2009 scores, but here consider representational constraints actually used in admissions to IITs. We also discuss additional interventions being implemented by the Indian state and central governments to reduce inequity in JEE scores.
48

---

Recall that, the Indian Institutes of Technology (IITs) are a group of engineering institutes in India. In 2009, there were 15 IITs and today this has grown to 23. Undergraduate admissions at IITs are decided based on the scores of candidates in the Joint Entrance Exam (JEF). JEE is conducted once every year. 2009, the scores, (binary) genders, and birth categories of all candidates who appeared in JEE 2009 were released in response to a Right to Information application filed in June 2009 [91]. The birth category of the candidates is an official socioeconomic status label recognized by the government of India [135].
Here, we focus on two groups of candidates: the candidates in the general (GEN) category (the most privileged) and candidates not in the general category. We begin by discussing some of the interventions in place to reduce inequity in JEE scores and subsequent admissions at IITs.
Interventions used in IIT admissions. The Indian constitution allows the central government and state governments enforce affirmative action in the form of quotas lower bound constraints for official SES groups at educational institutes, employments, and political bodies [82, 146]. In 2005 lower-bound interventions were introduced in the admissions process at the IITs. Concretely, in 2009, out of the 7,440 seats, 3,688 (49.6%) were reserved for students who are not in the GEN category. This means that least 3,688 out of the 7, 440 students admitted into IITs must not be in the GEN category. Note that this allows more than 3,688 or even all admitted students to be outside the GEN category. We call this constraint the Reservation constraint and, in this section, we study its effectiveness compared to other forms of interventions.
Apart from reservations, number of other interventions have also been proposed and/or implemented to reduce biases in the JEE We discuss two other types of interventions next.
Interventions to reduce skew. Private coaching institutes that train students for JEE have been criticized for being exorbitantly expensive and, hence, inaccessible for students in low SES groups [44]. Lack of accessibility to training resources can reduce the scores of candidates in low SES groups- creating a skew in the scores. To improve accessibility to training, in 2022, the Delhi government established new program that will provide free training to students enrolled in government- funded schools [53]. Similar programs have also been introduced in other states [52, 141] and by school education boards that span multiple states [26]. In the context of our model, these interventions can be seen as reducing this skew in the evaluation process.
Interventions to reduce information constraint. A criticism of JEE is that it is only offered in the English and Hindi languages. This is undesirable because only 44% of Indians report English or Hindi as their first language and, according to the 2011 census, less than 68% of Indians list one of these languages among the three languages they are most comfortable with [103]. IITs have been repeatedly criticized for not offering the exam in regional languages [128, 7, 47J. The main concern is that the current exam reduces the performance of students less familiar with English and Hindi In the context of our model, this can be thought of placing a stronger information constraint on candidates who do not speak English or Hindi as a first language: these students would need to spend higher cognitive load to understand the questions. This constraint not only acts during the exam but also during the preparation period because students speaking regional languages (and not English or Hindi), have devote additional time to learning either English or Hindi in addition to the technical material for the exam.
While the JEE exam itself has not been offered in regional languages yet. Recently, in 2021, the screening test that candidates have to clear before appearing in JEE was offered in 11 regional languages in addition to English and Hindi [134].
In this section. we compare the effectiveness of the above three interventions Reservation for lower SES groups, interventions to reduce skew (change a by A,, percent), and interventions to reduce information-constrain (change by A, percent).
Setup (Group sizes and k)). Admissions into IITss are highly selective, For instance, in 2009, 384,977 applicants (232,334 from GEN; 60%) took the exam and just 7,440 (2%) were admitted to IITs. The admission is based on the candidates' All India Rank (henceforth just rank)- denotes the candidate's 's position in the list of candidates ordered in decreasing order of their scores in JEE. Let G2 be the group of students in GEN category and G2 be all other students. To study the impact of different interventions for admissions into IITs, we fix group sizes and l match real numbers: |G1 232, 334, [G2| 152, 643, and k = 400. We focus on the set of candidates who scored at least 80 (out of 480) on the exam. (The threshold 80 ensures that at least 10k candidates outside
49

---

Decrease skew (la- II)
Reservation
Decrease
Reservation
Relative increase in utility
23333 By 152645, LTER:N50, a=2.667 202.2.66.r=0000 0.00
Relative increase in utility
252314, 151452443 MER-500 2m2666, r5
1.08
1.06
1.2
1.04
1.1
1.02
1.00-
1.0
0.0
0.2
0.4
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Au
A.
(a) Changing by A. percent
(b) Changing by A, percent
Figure 9: Effectiveness of different interventions on the selection-utilee as estimated by our model: We vary the strengths of the interventions (A., E [0, 1] and A, E 1]) and report the expected utilities of the subset output by three interventions. The z-axis shows the strength r (Figure 9())). The y-axis shows the ratio the (true) utility of the subset output with an intervention to the (true) utility of the subset output without any intervention. Our main observation is that for each of the three interventions, there is value of the percentage change in and (i.c., A. and A, respectively) for which the intervention outperforms the other expects specific intervention (e-g.. providing free coaching) to have on the parameters a and т. they can use our framework as tool to inform their decision about which intervention to enforce. Error bars represent the standard error of the mean over 100 repetitions.
intervention changing a (Figure 9())) or
interventions. Hence, depending
the amount of change policymaker
the GEN category are considered and this is significantly lower than the k-th highest score of 167). We fix fo to be the density of utilities of all candidates in G1 who scored least 80. Since fo has a Pareto-like density (see Figure 10), we fix E(x, v) = In(x) - In(u). We fix Ω to be the set of all possible scores and fc, to be the density of all candidates in G2 who scored at least 80. As in
Section 4, we select a and T that lead to the density closest in TV distance to fa,- The rest of the setup is the same in Section
Unlike the main body, here, we only consider high-scoring candidates (those with a score of at least 80) because JEË is highly selective (k/n < 0.02) and, hence, to have meaningful results the estimated density fs should have a good fit to the density from the real-data on the top 2% quantile, ie., the right tail. To ensure this, specifically consider the right tail of the distribution (by dropping candidates with a score below 80).
Observations and discussion. We vary A. j0, 1] and A, E [0. 1] and report the expected utilities of the subset output by all three interventions over 100 iterations in Figure 9. Our main observation is that for each of the three interventions, there is a value of the percentage change in a and 7 (i.e., Ar and A, respectively) for which the intervention outperforms the other two interventions. Hence, depending on the amount of change policymaker expects specific intervention (e.g., providing free coaching) to have on the parameters a and т, they can use our framework as a tool to inform their decision about which intervention enforce. Further, we observe that, expected, increasing A. and A,, i.c., the percentage of change in a and t, improves the utility achieved by the corresponding interventions.
Limitations and further discussion. Next, we discuss some of the limitations of our study. First, we note that interventions such as increasing the accessibility of education can not only reduce inequity
Students in GEN category
Students not in GEN category
600-
4004
Figure 10: The densities of scores of students in GEN category (blue) and students not in GEN category (orange) in (GE-2009-oni for students scoring at least 80 points out of 480.
50

---

in JEE but can also have positive effects on other exams and hiring. Hence, such interventions can have larger positive (or negative) impact than suggested by our simulations. Studying these auxiliary effects is beyond the scope of this paper. Further, our study also does not model the response of the students, e.g., how interventions affect the students' incentive invest in skill development? Finally, our model only predicts the effect of and on utility distributions. These predictions may not be accurate and careful post-deployment evaluation may be required to accurately assess the effectiveness of different interventions.
L.2 Additional plots for simulations in Section
In this section, present plots of the best-fit densities output by our framework on different datasets.
JEE-2009 Dataset (Birth Category) Best Fits
0.020
Biased utilities
This work
0.015
Implicit Variance Model
Multiplicative Bias Model
0.010
0.005
0.000
100
0
100
200
300
400
Best-fit distribution category)
10
and 1.51) with JEE-2009 (Birth
JEF-2009 Dataset (Gender) Best Fits
Biased utilities
0.015
This work
Implicit Variance Model
0.010
Multiplicative Bias Model
0.005
0.000
100
0
100
200
300
400
(b) Best-fit distribution (a: = 3 10 and = 1.51) with JEE-2009 (Gender)
Figure 11:
Illustration of the best-fit distribution output by our framework for the JEE-2009 dataset. Captions of subfigures report the best fit and 7.
L.3
Implementation details
L.3.1
Our framework and other models
In this section, we give implementation details of our model. Recall that our model outputs the density which is the optimal solution of the following optimization program.
51

---

Semantic Scholar Dataset Best Fits
0.06
Biased utilities
This work
Implicit Variance Model
0.04
Multiplicative Bias Model
0.02
0.00
0.0
0.1
0.2
0.3
(a) Best-fit distribution (a 22.78 and 2.09) with Semantic Scholar Open Research Corpus
Synthetic Network Dataset Best Fits
Biased utilities
0.8
This work
Implicit Variance Model
0.6
Multiplicative Bias Model
0.4
0.2
0.0
0.0
0.1
0.2
0.3
0.4
0.5
(b) Best-fit distribution (a = 1.92 and r 3.19) with synthetic network data
Figure 12:
Illustration of the best-fit distribution output by our framework for the Semantic Scholar Open Research Corpus. Captions of subfigures report the best fit and T.
argmin deasity B Errz,a (Jv, 1) Sa [.o la ())(d))((()))) (o()ddu()) (OptPooAAA))))) such that fo (x) log f(x)du(x) 2 T.
An instance of this program is specified by the following parameters.
A domain Ω C (e.g., = and l [1, oo));
2. A true density fo over Ω with respect to the Lebesgue measure μı;
3. A loss function f: Ω x + R (e.g., {(z, v) = (a v)² and E(x, v) = In (x/v)):
4. A risk-averseness (or risk-eagerness) parameter a > 0; and
5. A aesource-informaiin parameter > 0.
Recall that la is risk-averse loss defined by l and a as in (4). For our simulations, we consider the shifted variant of la mentioned in Section 2: given shift parameter vo IR, loss function é: Q + R. and parameter a > 0
aa ((. ((1 u vo)
f otherwise.
(x,
(x,
52

---

Let Jo,T, be the optimal solution to the instance Ia, = (Ω, f,, Un) of (OpiProg).
Algorithmic task. Given "target" density JT (denoting the density of biased utilities in the data), risk-averse loss function fa. and true density /D. the goal of our implementation is to find a., T and og that minimize the total variation distance between fT and f'
(a , 26) = argmin drv (a fr). a,7,0
Algorithmic approach and implementation. We perform grid-search over all three parameters o. r, and tig- Given specific a, T, and Up. to solve the above problem, we use the characterization in Theorem 3.1 to find fa, Recall that the optimal solution of (OptProg) is of the following form
Fa (4) C exp (- (1)/7")
where va. == So la (z, (z))d))) and C,," > 0 are constants that are uniquely specified by the following two equations
La (x...ro()))pp()) (z)dp(x)) =1 and Sā. (2) log (f (z)) du(x)
Algorithmically, finding C and yt requires computing a double integral over Q. In all of the simulations in Section 4, 52 is a discrete domain, so these integrals reduce to summations and we compute them exactly. We also provide an implementation of our algorithm for continuous domains. The implementation for continuous domains uses the quad function in scipy to compute the integrals. For the grid search itself, we varied a over [10-1, 10"), t over [10-1, 10), and uo over S2 We found this range to be sufficient for our simulation, but it would be interesting to design principled way of specifying the ranges given other parameters and target density fT.
Implementation details of multiplicative bias model [90] and implicit variance model [61]. Recall that the multiplicative bias and the implicit variance models are specified by parameters p and a respectively: given a fixed true value v E R, the output of the multiplicative bias model is v/p and the output of the implicit variance model is v + where Ç a zero-mean normal random variable with variance o². In addition, we allow both models to introduce a shift co. For the multiplicative bias model, given a true density fo and a target density fT, we compute (p that solves argmin p.3o drv (Jp.ao where fo.0 is the density of (v/p) + for e ~ fD. For the implicit variance model, given true density So and target density JT, we compute (a, vn) that solves argmin,,.o dTv(Ja,00: ST) where fa,v is the density of e + 0o + Ç for v ~ I'D and zero-mean normal random variable ( with variance o², For both models, we compute the optimal parameters using grid search: vary up over Ω (1/p) over (0, 1]. and over [10-2 2 10).
L.3.2
Computational resources used
All simulations were run on a MacBook Pro with 16 GB RAM and an Apple M2 Pro processor.
L.3.3
JEE-2009 Scores
Additional discussion of the dataset. The JEE-2009 test scores were released in response to Right to Information application filed in June 2009 [91]. This dataset contains the scores of all students from JEE-2009 (384,977 total) [91]: we used the version available provided by [40]. In addition to the scores, for each student, the data contains their self-reported (binary) gender and their birth category. The birth category of student is an officially designated indicator of their sociocconomic group, where the general (GEN) category is the most privileged; see [135, 19] for more details.
We observe that students not in the GEN category have significantly lower average scores than students in the GEN category (18.2 vs. 35.1); this may not imply that students not in the GEN category would perform poorly if admitted. Indeed, among students of equal true "potential," those from underprivileged groups are known to perform poorer on standardized tests [60]. In the Indian context, this could be due to many reasons, including that in India, fewer students outside the GEN category attend primary school compared to students from the general category, and on average lower amount of money is spent on the education of students in the non-general category compared to the general category [92].
53

---

Gencral group
0,04
Disadvaningod groups
0.02
0.010
0.06
0.004
0,000
Tosal Sc000
Figure 13: Distribution of scores in the JEE dataset for different protected groups based on birth category. See Section for a discussion of the dataset.
L.3.4 Semantic Scholar Open Research Corpus
Cleaning and predicting author names. We follow the procedure used by [40]. Concretely, we remove papers without publication year (1.86% of total) and predict author gender using their first name from a publicly available dataset [3], containing first names and gender of everyone born between 1890 to 2018 and registered with the US social security administration (USSSA). We remove authors whose first name has or fewer characters, these names are likely to be abbreviations (retaining 75% the total), and then categorize an author as female (respectively male) if more than 0.9 fraction of the people of the same first name are female (respectively male) in the USSSA data. We drop all uncategorized authors (32.25% the remaining). This results in 3,900,934 women and 5,074,426 men (43.46% females). We present the tradeoff between the total number of authors retained and & in Figure 14.
Counting the number of citations. We aim to ensure that the citation counts we compute correspond the total citations received by an author over their lifetime (so far). Since the dataset only contains citations from 1980 onwards, we remove authors who published their first paper before 1980 as the dataset does not have information about their earlier citations. This is the same as the cleaning procedure used by [40]. We present the resulting citation-distributiios for male and female authors respectively in Figure 15.
Namberr of Aurbee Betuned
Woman
Unkcowwn
Threshold
Figure 14: The tradeoff between the threshold 6 used for clearing the Semantic Scholar Open Research Corpus and the number of authors retained. Details appear Appendix L.3.4.
54

---

Male
Prohabilit Mass Flanction
Fomale
Total Glaatoos (Pigers after 1980)
Figure 15: Distributions of total citations of men and women in the Semantic Scholar Open Research Corpus. Details appear in Appendix L.3.4.
55


Reviews:
[{"review_id": "Tqi73zF1jI", "reviewer": "Reviewer_G1nQ", "summary": "In this paper, the authors model evaluation processes that estimate the density of value for an individual (on a task) as a loss minimization problem subject to constraints. The authors proceed to derive various properties of the output densities of their model and evaluate it on two real world datasets.", "questions": "- So how would a policy maker use this model? It isnt immediately clear to me as someone unfamiliar with the literature", "limitations": "The authors note some limitations to their work - though I didn't see much discussion of the misuse of their model and it's negative societal impact.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- This is a good solution that seems to provide clarity to a difficult and important problem\n- Strong (though limited) empirical section, and ablation studies on the effects of $\\tau$ and $\\alpha$", "weaknesses": "- I found it really hard to read this paper as someone not too familiar with the field, in particular I thought the intro could use some more clarity. There's a bit of measure theory  too that I wonder is necessary -\n- Id like to see a theorem about the performance of this model relative to others, it seems like its fairly to compare it to some of the Related Work mentioned. Though this might be fixed by adding more clarifications --", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your feedback. We are glad that you find our work as a good solution to an important and difficult problem. We address your specific questions below.\n\n**\"hard to read this paper as someone not too familiar with the field\"**  We apologize that you found it hard to read parts of the paper. Given the multitude of disciplines touched by this paper, it was difficult to write. We chose a style similar to some of the most related works [81,57,41]. We take your feedback seriously and will try to simplify some of the expressions while preparing the final version.\n\n**\"...theorem about the performance of this model relative to others...\"** Thanks for the suggestion. The two most-related prior models are the implicit variance model of [57] and the beta-bias model of [81]. In the implicit variance model, the observed utility of the advantaged group is a Gaussian random variable with mean $\\mu$ and variance $\\sigma_0^2$, and the observed utility of the disadvantaged group is a Gaussian random variable with mean $\\mu$ and variance $\\sigma_0^2 + \\sigma^2$. This model is a special case of our framework  as captured by the following theorem (this is a direct corollary of Lemma F.2 and we will include it in the final version):\n\n\n**Theorem:** Consider an instance of the implicit variance model given by parameters $\\mu, \\sigma, \\sigma_0$. Consider instances  $I_1=(\\Omega, f_{\\mathcal{G}}, \\ell, \\alpha, \\tau_1)$ and $I_2=(\\Omega, f_{\\mathcal{G}}, \\ell, \\alpha, \\tau_2)$ of OptProg, where  $\\Omega=\\mathbb{R}$, $f_{\\mathcal{G}}$ is the normal density $N(\\mu, \\sigma_0^2)$, $\\alpha=1$, $\\ell(v,x) = (x-v)^2$, $\\tau_1=\\frac{1}{2}(1+\\ln (2 \\pi \\sigma_0^2))$ and  $\\tau_2 = \\frac{1}{2}(1 + \\ln(2 \\pi (\\sigma_0^2+ \\sigma^2)))$. Then the output density of OptProg on $I_1$ is $N(\\mu, \\sigma_0^2)$ and the output density of OptProg on $I_2$ is $N(\\mu, \\sigma_0^2 + \\sigma^2)$.  \n \nIn the $\\beta$-bias model, the true utility of both groups is drawn from a Pareto distribution and the output utility for the disadvantaged group is obtained by scaling down the true utility by a factor $\\beta > 1$. This changes the domain of the distribution to $[\\frac{1}{\\beta},\\infty)$ from $[1,\\infty)$ and, hence, does not fit exactly in our model which does not change the domain. Nevertheless, we can show that, for any fixed $\\tau$,  increasing $\\alpha$ reduces the *mean* of the output density, effectively capturing the $\\beta$-bias model at a population level. Formally, the following theorem follows from the calculations in Section G; we will include it in the final version. Recall that the Pareto density with parameter $\\gamma$ over $[1, \\infty)$ is defined as $f_\\gamma(x) = \\frac{\\gamma}{x^{\\gamma+1}}, x \\in [1, \\infty).$ \n\n\n**Theorem:** Consider an instance $\\cal I$ of the $\\beta$-bias model specified by the Pareto distribution $f_\\gamma$ with parameter $\\gamma$ over $[1, \\infty)$ and a parameter $\\beta > 1$. Consider the instance  $I_1 =  (\\Omega=[1, \\infty), f_\\gamma, \\ell, \\alpha_1 = 1, \\tau)$ of OptProg, where $\\tau = 1 + \\frac{1}{\\gamma} - \\ln \\gamma$ and $\\ell(x) = \\ln (x/v).$  Then the optimal solution to $I_1$ is given by $f_\\gamma$. Further, there exist values $\\alpha_2 \\geq 1$ and $\\beta_\\gamma > 1$, which is a function of $\\gamma$ only, such that if $\\beta \\leq \\beta_\\gamma$, then the output density of OptProg on the instance $I_2 = (\\Omega, f_\\gamma, \\ell, \\alpha_2, \\tau)$ has expectation $1/\\beta$ times the expectation of $f_{\\gamma}$. \n\n\n**\"how would a policymaker use this model?\"** Most of the prior works on interventions in selection settings have focused on adding representational constraints for disadvantaged groups. Such constraints, often framed as a form of affirmative action, could be beneficial but may not be possible to implement in certain contexts. E.g., in a landmark 2023 ruling, the US Supreme Court effectively prohibited the use of race-based affirmative action in college admissions. Our work, via a more refined model of how bias might arrive at a population level in evaluation processes, allows for evaluating additional interventions that focus on procedural fairness; this allows working towards diversity and equity goals without placing affirmative-action-like constraints. \n\nIn this framing, we can consider decreasing either $\\alpha$ or $\\tau$. Improving either would work towards equity, but which ones to target or to what extent and via which method would be context dependent and vary in cost. A decrease in $\\alpha$ can be achieved by reducing risk-averseness in the evaluation process; e.g. by investment in better facilities for disadvantaged groups, or making the evaluation process blind to group membership. A reduction in $\\tau$ may follow by allocating additional resources to the evaluation process, e.g., by letting a candidate choose an evaluation in their native language. Our framework allows a policymaker to study these trade-offs, and we discuss specific examples in Supplementary Material H.     \n\n**\"I didn't see much discussion of the misuse of their model and it's negative societal impact.\"** In any work that focuses on debiasing, the ideas and models could be used adversarially to achieve the opposite goal. We need third-party evaluators, legal protections, and available recourse for affected parties -- crucial components of any system -- though beyond the scope  of this work. We will add a brief discussion to acknowledge these important points in the final version."}, {"review_id": "4hwMnZbThT", "reviewer": "Reviewer_7ABm", "summary": "The paper studies how to examine the group distributional difference using loss minimization. The authors propose a loss with a max-entropy constraint. ", "questions": "See weaknessses. ", "limitations": "1. The paper only considers binary sensitive groups, while in practice, sensitive groups are often overlapping with multiple attributes. \n2. How would the estimation error and model misspecification lead to negative impact?", "rating": 4, "confidence": 1, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "The paper studies an important problem of how to examine the evaluation bias in many applications such as hiring and school admissions. The authors nicely motivate the problem and have a detailed related work on how such biases arise in practice. ", "weaknesses": "The paper became quite hard to read after related work. I had a hard time understand what is the formulated problem and the reasons for many design choices are not clear to me. I do not follow the loss formulation, since it seems that the authors are trying to do a density estimation task with certain constraints on the density function. Why not minimize classic metrics like f-divergence, IPM, or use methods like GMM or kernel density estimation? There is also little explanations on why we should use max-entropy as constraint. \n\nEvaluation: It seems the experiments are a density estimation task for two groups, why usual density estimation methods cannot be applied here?\n\nI am also not convinced the proposed method can be used to examine effect of interventions, since they are dynamic and more information is needed to see the effect of interventions in the long run. \n\nTypo:  “risk averseness:”, incomplete sentence L121-122", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for appreciating the paper. We address your questions and concerns below and hope you will strengthen your support for the paper.\n     \n**\"..since it seems that the authors are trying to do a density estimation task with certain constraints on the density function. Why not minimize classic metrics like f-divergence, IPM, or use methods like GMM or kernel density estimation? There is also little explanations on why we should use max-entropy as constraint..\"** \n\nThanks for this question and sorry for the confusion. Our goal is not to do *density estimation*, but to *model* how an input density representing the utility of the population gets *transformed* by an evaluation process to an output density in the presence of risk-averseness and information constraint.  in Section 2, our model modifies the classical maximum entropy framework (Equation 2) to incorporate the resource-information parameter $\\tau$ (Equation 3) and the risk-averseness parameter $\\alpha$ (Equations 4). \n\nWe do use an $f$-divergence (differential entropy) in our formulation, albeit in the constraint as it allows us to incorporate the resource-information parameter $\\tau$. However, methods like GMMs or kernel density estimation are not relevant to what we are trying to achieve.\n\nIn Section 2, we will add further on the use of max-entropy as a constraint and also contrast our approach with density estimation.  \n\n    \n**\"It seems the experiments are a density estimation task for two groups, why usual density estimation methods cannot be applied here?\"** The empirical results are trying to find the values of the parameters $\\alpha$ and $\\tau$ in our model that best-fit the density of the biased values rather than estimating density itself, so the usual methods for density estimation do not seem useful.\n\n    \n**\"I am also not convinced the proposed method can be used to examine effect of interventions, since they are dynamic and more information is needed to see the effect of interventions in the long run.\"** As in many prior works [81,57,41], the proposed model can be used to understand the effect of interventions in one round. Indeed, to understand the effect of interventions in the long term, additional work would be required, perhaps evaluated as in [40], and would be an important direction for future work. We will add this as a limitation in the final version. \n\n    \n**\"The paper only considers binary sensitive groups, while in practice, sensitive groups are often overlapping with multiple attributes.\"** Indeed, we limit our discussion and simulations to binary-sensitive groups. However, our model easily extends to multiple sensitive groups by considering a group-specific risk-aversion parameter and a group-specific information constraint. Prior models of biased densities [81, 57] also limit their discussion to binary-sensitive groups, and similar extensions (with group-specific parameters) have also been proposed for them. For example, if two groups $G_1,G_2$ overlap, then we can consider three disjoint subgroups $G_1\\cap G_2,$ $G_1\\backslash G_2$ and $G_2 \\backslash G_1$. We will add a remark in the final version.\n\n   \n**\"How would the estimation error and model misspecification lead to negative impact?\"** Estimation errors and model misspecification can lead to errors in the assessments of interventions that possibly, in turn, leads to the deployment of suboptimal interventions. These errors are not specific to our evaluation method and one can try to assess and reduce their negative impact via validation methods such as train-test splits, control trials, and ablation studies. We will include a discussion about such errors in the final version. "}, {"review_id": "ruqkvVNFhS", "reviewer": "Reviewer_2dLt", "summary": "This work presents a theoretical model to quantify bias in the task of evaluating candidates (ie minimizing loss while subject to an information constraint). It presents a formula/representation of the problem, parametrized by roughly \"real-world\" factors of 1) resource-information tradeoffs; and 2) risk-aversion. After presenting some properties of this model, the work loosely applies it to quantify types of bias in real-world datasets (eg standardized testing by class/gender; citations; etc). Finally, it explores how different real-world-inspired interventions (eg Rooney Rule for representational constraint, structured interviews for standardization) could impact the state of bias in the parametrized models.\n\nedit: I have had the rebuttal and feel better about the derivations. I remain borderline accept", "questions": "- Overall this paper seems very interesting, though I couldn't follow all of the derivations/formulas. The empirical part, alone, doesn't have enough insight, but I do like applying the theoretical parametrized model to the real-world datasets to discuss how to interpret them. I'm inclined to accept, though I would feel more comfortable about this paper if one of the other reviewers were able to speak to the derivations' validity and contribution.", "limitations": "- Some of the modeling choices were noticeably reductive. For instance, on page 8, Section 4.2 uses college admissions as an example of a subset selection task. However, it suggests modeling the task is the sum of each accepted applicant's individual values, rather than considering network effects (eg Scott Page's \"The Diversity Bonus\" where a team of different people can contribute different kinds of knowledge to solution resulting in a better result than if the top-2 individual values had high overlap & therefore didnt offer complementary strengths). Of course, that is just nitpicking one example, however my general concern is that this work may or may not ultimately prove useful enough for policymakers. It might end up being the case that any tasks that are tractable enough to model mathematically are poor fits for social science dynamics in practice)", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- I quite like the design decision to choose model parameters and intervention types that are inspired by plausible tradeoffs and concerns in evaluation bias.\n- This paper would not be as strong if it were just the formulas/models without trying to measure any grounding in real-world datasets.", "weaknesses": "- Although engaging with real-world datasets empirically is commendable, the analysis conducted was rather light. It didn't offer particularly novel insights or alternative ways of thinking about bias in the data (eg could try to demonstrate how the model can help policymakers with actionable interventions; I don't think a policymaker would be able to gain such insight with the current presentation of results/discussion)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for appreciating the model, the empirical work, and the presentation of the paper. We have addressed your questions and concerns below and hope that you will consider supporting our paper further.\n\n\n**\"I'm inclined to accept .. \"**  Thanks. Please take a look at the review of reviewer gzsA concerning the validity and contributions of the derivations.\n\n**\" ... how the model can help policymakers with actionable interventions ...\"** Most of the prior works on interventions in selection settings have focused on adding representational constraints for the disadvantaged groups. Such constraints, often framed as a form of affirmative action, could be beneficial but may not be possible to implement in certain contexts. E.g., in a landmark 2023 ruling, the US Supreme Court effectively prohibited the use of race-based affirmative action in college admissions. Our model posits that reason for the emergence of differences in distributions of abilities of different groups could be because of the difference in the values of the information-resource parameter $\\tau$ and/or the risk-averseness parameter $\\alpha$ of the group. The theoretical results establish the impact of these parameters on the mean/variance of the output distributions. Thus, in all, the model allows a  policymaker to evaluate interventions beyond affirmative action that focus on procedural fairness; this allows working towards diversity and equity goals without placing affirmative-action-like constraints.\n    \n\nIn this framing, we can consider decreasing either $\\alpha$ or $\\tau$. Improving either would work towards equity, but which ones to target or to what extent and via which method would be context dependent and vary in cost. A decrease in $\\alpha$ can be achieved by reducing risk-averseness in the evaluation process; e.g. by investment in better facilities for disadvantaged groups, or making the evaluation process blind to group membership. A reduction in $\\tau$ may follow by allocating additional resources to the evaluation process, e.g., by letting a candidate choose an evaluation in their native language. Our framework allows a policymaker to study these trade-offs, and we discuss specific examples in Supplementary Material H. \n\n\n\n    \n**\"a team of different people can contribute different kinds of knowledge to solution resulting in a better result\"** Sorry for the confusion, summing up the utility of each accepted applicant's individual utility in the example of subset selection task is not a modeling choice made in this paper. It is borrowed from prior works [81,57,41]. \n    \nThe focus of this paper is to give a model of how population level differences in utility distributions can arise in evaluation processes. In the empirics, the main goal was to show that the model can explain biases in real-world data sets. The goal of showing the subset selection application was to guide policymakers on the deployment of interventions to mitigate biases. \n\nThat said, we appreciate your suggestion. Indeed, works in sociology (such as the one you suggest) have shown that one of the benefits of diverse teams is that complementary skill sets increase the overall production -- the sum is more than the parts. This corresponds to having a supermodular set aggregation function in the selection task. The output of the model can be used by policymakers to assess the impact of interventions in this supermodular setting as well. We will mention this in the final version."}, {"review_id": "3uENpjQdUj", "reviewer": "Reviewer_gzsA", "summary": "The paper proposes an optimization-based framework for modeling bias in evaluations. The perspective of the paper is to provide a well-founded and interpretable model of evaluations that can replicate biases observed in real settings without invoking an intrinsic utility for producing biased evaluations. This model is a generalization of previous models for evaluation bias which assume a particular parameterized model for the output density (e.g. Gaussian or Pareto). The primary utility of this model is to help policy-makers study the effectiveness of different interventions aimed to reduce bias, such as requiring proportional fairness of evaluations or decreasing the informational cost of evaluations. The authors apply their model to data such as IIT JEE-2009 scores, obtaining a closer fit with realized scores than previous models, and test out different interventions.\n\nSpecifically, the paper models the density of evaluations f_{\\mathcal{E}} as the output of an optimization problem, in which the evaluator seeks to produce an evaluation as close to the true value v of the individual as possible, which is drawn from a density f_{D}. Closeness is measured by a loss function \\ell(x,v) between evaluation x and quality v. This optimization problem is subject to constraints and adjustments which introduce bias, which are well-founded in the literature. The first is information constraints. Evaluations are noisy in practice because it is difficult and costly to obtain a clear evaluation signal, and this cost can vary across different groups (e.g. groups that speak different languages). This is modeled as a constraint requiring that the entropy of f_{\\mathcal{E}} be lower-bounded by some \\tau. The second source of bias is risk-aversion: the evaluator's loss may not be symmetric between over and under-estimation of the true value, and may penalize over-estimation more. This is modeled in the loss function between the evaluation and the true value v, in which loss aversion (parameterized by \\alpha\\geq0) causes the evaluator to penalize over-estimation of v more than under-estimation by a factor of \\alpha. They derive an exponential functional form for the solution of this optimization problem, which is standard by the maximum-entropy principle. They also perform sensitivity analysis of how the solution changes with \\tau and \\alpha.\n\nThis modeling framework generalizes the implicit variance and multiplicative bias models. The only inputs to the model are the information constraint \\tau, risk aversion parameter \\alpha and loss function \\ell. For empirical validation, they consider two real-life datasets and one synthetic example of scores in different contexts. They fix a group G_{1} of individuals to be a baseline group and use the distribution of scores from that group to represent the true distribution of values. They consider another group G_{2} whose scores are potentially biased (and thus arising from f_{\\mathcal{E}}), and they compare how well different models of evaluation bias reproduce the distribution of G_{2} using scores from G_{1} as the true values. They find that their more generalized model can improves the fit of the score distribution of G_{2} across all datasets. They then use their model to assess various interventions for subset selection tasks, based on whether they reduce bias and allow the evaluator to pick a subset of individuals with higher expected value. These interventions include requiring proportional or equal representation, increasing \\tau, and lowering \\alpha, which all are related to interventions considered in practice. They find that optimal interventions depend case by case based on different parameters of the subset selection task (e.g. how many individuals to hire, etc.), and their model can guide policy-makers to understand when certain interventions will be more effective than others.", "questions": "When fitting \\alpha and \\tau to minimize total variation distance with the G_{2} distribution, is there any train-test splitting, or are they fit on all the data? Otherwise there might be a concern about overfitting.\n\nIn the empirical evaluation, the underlying assumption is the distribution of true values for the G_{2} individuals is exactly the same as the distribution for G_{1}. While this assumption makes sense for convenience since we don't observe the true values of G_{2}, if this assumption were false in practice, how could this affect the validity when assessing different interventions?", "limitations": "The paper primarily mentions the limitation of the work as being that it only concerns with scalar values, and mentions that the framework can be extended to multivariate values. The paper also mentions that there may be other models out there that achieve the same fit as the maximum entropy one. This is all reasonable. It does not include the limitation that the distribution of true values for G_{2} is assumed to be identical for G_{1}, in which case one cannot disentangle evaluation bias from natural differences in these distributions.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The greatest strength of the paper is in its conception. Invoking the maximum entropy principle obtains a natural generalization of previous models in the literature, without involving an excess of extra parameters. Including risk aversion makes sense too, given that it is a well-studied source of bias and better enables the framework to model skewed distributions. The method is computationally feasible since evaluation scores are 1-D and ultimately discrete, and produces realistic evaluation distributions. The optimization-based formulation retains much of the interpretability of simpler models, while crucially allowing for greater modeling capacity through computation. In contrast to previous papers, which are primarily concerned with producing simple models that illustrate a particular source of bias, this paper provides a computational framework designed to be applied to data (I believe this aspect should be emphasized more in the paper). \n\nThe paper is clearly written and the empirical benchmarking is solid with compelling examples (JEE-2009 and Semantic Scholar). The evaluation of different interventions is also a nice illustration of the usefulness and interpretability of the model.", "weaknesses": "Overall the paper could benefit from more exposition. While the paper references key sources in the literature, it does not clearly explain the mechanism by which information constraints and risk aversion lead to bias. This could be clarified by 1 or 2 specific examples. Without this, it is difficult to understand why these are the particular sources of bias incorporated in the framework and why others are not. It would also be better to spend more time discussing concrete interventions, e.g. moving some of the material from Supplemental Material H to the main text. To aid in this, would it be possible to evaluate an intervention like 'change score from out of 100 to out of 10' or 'truncate the range of scores'? If so, this could help in demonstrating the applicability of the model for helping with realistic policy decisions.\n\nThe theoretical results on the evaluator's optimization problem seem to be fairly standard and straight-forward implications of the maximum entropy principle. This is not a bad thing, but at least this section could be trimmed down and streamlined. The proof of Theorem 1 seems fairly involved, and it would be useful to explain how this setting departs from standard maximum entropy settings. The results on sensitivity analysis with respect to \\tau and \\alpha are also not surprising and this section could be shortened and crystalized to better capture how \\tau and \\alpha affect the output density (with greater focus on how the mean changes). Figure 1 is difficult to interpret and could either be removed entirely or replaced.\n\nI think the biggest need for improvement is in showing the strength of the contribution compared to previous models. I think a strong case can be made using supplemental material, but from the current draft alone the strength of the contribution is not very clear. First, it would help to use some supplemental material (e.g. Figure 6 from Supplemental Material H) into the paper to show that a wider class of models is actually necessary for modeling real life score distributions (comparing the best fits from other models). One could also look at a synthetic example with exponential or laplacian tails, but this is not necessary.\n\nAnd since the framework a generalization of previous models, it is perhaps unsurprising that it achieves greater fit to data. More than fitting the data, ultimately what matters are implications for intervention evaluation/sensitivity analysis for policy-making. If the conclusions are identical to what one obtains from less sophisticated but simpler models, it's unclear whether a more powerful modeling class actually helps. What would have been more compelling is if there are instances where the current model gives different answers for what interventions work well or not compared to simpler models, and justifying why the current model's conclusions are more sensible. In that light, it would be useful to evaluate how well the framework assesses interventions, which can be done through synthetic examples (e.g. intervening on the synthetic example and seeing how well the model predicts the effect of the intervention). ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback. We are glad that you think that our model can guide policy-makers to understand when certain interventions will be more effective than others. Thanks also for appreciating the invocation of the maximum entropy principle to generalize previous models. \n\n**\"proof of Theorem 1...”** Please see our response [here](https://openreview.net/forum?id=7b4oobeB4w&noteId=2eKb6fBuEX).\n\n\n**\"…specific examples…\"** While we do explain how changing $\\alpha$ and $\\tau$ give rise to biases, we present a couple of concrete settings to show how $\\alpha$ and $\\tau$ may arise and give rise to biases in an evaluation process [here](https://openreview.net/forum?id=7b4oobeB4w&noteId=2eKb6fBuEX).\n\n**“…concrete interventions…e.g. moving material from Supplemental Material H.”** Thanks. We will move the discussion on interventions from Supplementary Material H (Lines 1362-1381) to the main body.\n\n\nWhile such interventions (e.g. conducting the exam in multiple languages) can be assessed by our framework, to evaluate them, one needs a model of how the intervention affects $\\tau$ and $\\alpha$, and designing such models is beyond the scope of this work. \n\nThat said, for interventions that directly manipulate the values, such as truncating the range of the values as suggested by you, one can evaluate their effect on $\\alpha$ and $\\tau$ by computing the best-fit distributions to the truncated value distribution and, hence, use our framework to assess the efficacy of these interventions.\n \n\n**\"...could be shortened...to better capture how \\tau and \\alpha affect the output...\"** Thanks for the suggestions.  We will trim the discussion of sensitivity analysis with respect to $\\tau$ and $\\alpha$, which focus on their effect on the variance and the  mean of the output density. Even though the sensitivity analysis with respect to $\\tau$ and $\\alpha$ shows that the variance and the mean of the output density change as expected, proving these results are non-trivial. The key reason is that we do not have a closed form expression for the optimal density in terms of $\\alpha$ and $\\tau$. We can only express the optimal density in terms of the optimal dual variable $\\gamma^\\star$ (see Thereom 3.1), but understanding the sensitivity properties of $\\gamma^\\star$ turns out to be  non-trivial. We draw intuition from the analogy with Gibbs equation (see equation (6) and Section C in the supplementary material), which helps us in predicting the properties of $\\gamma^\\star$. Sections D, E, F, and G in the Supplementary Material are dedicated to the proofs of these results.\n    \n**\"Figure 1...could be...replaced.\"** Thanks, we simplified Figure 1: instead of the three-dimensional plots, we present two-dimensional plots that show the effect of $\\alpha$ (respectively $\\tau$) on the mean of the distributions for a fixed value of $\\tau$ (respectively $\\alpha$). The updated figure is in the one-page PDF attached with our rebuttal. \n    \n \n**“...contribution compared to previous models…would help to use...Figure 6…”** Thanks for your suggestion, we will add a version of Figure 6 that compares the best fits of our model and the best fits of previous models with the distribution of biased utilities in the final version. As expected from the TV-distance values in Table 1, the best fit achieved by our framework is at least as good as the best fit of the implicit variance model [57] and the multiplicative bias model [81], and is a better fit than these models [57, 81] on datasets in which utilities have skew (Rows 1, 3, and 4 in Table 1).\n     \n**“...instances where the current model gives different answers...compared to simpler models…”** Yes, indeed, for certain interventions that affect both $\\tau$ and $\\alpha$, our model may lead to different assessments than simpler models. For instance, in the context of standardized testing, consider the intervention that requires conducting the exam in multiple languages. This intervention can reduce the information constraint faced by the disadvantaged group (by reducing the cognitive load on the non-native speakers during the examination) and, at the same time, may also reduce risk-aversion parameter $\\alpha$ (by eliminating the need for non-native speakers to enroll in additional training for the examination language). For such interventions, simpler models–that only consider the effect of the intervention along one dimension–may underestimate its positive impact while our framework could give a more accurate assessment. We will include a discussion on this in the final version.\n    \n**\"...train-test splitting\"** Thank you for your suggestion. We did not use a train-test split when fitting $\\alpha$ and $\\tau$. We have repeated the simulation with an 80-20 train-test split and will include them in the final version. \n    \nThe results with the train-test split are similar to the results in the paper: across all datasets, the densities output densities by our model are close to the density of biased utilities in the datasets (TV distance $\\leq 0.09$) and our model has a better fit than the implicit variance model [57] and the multiplicative bias model [81] on datasets where utilities have skew (Rows 1, 3, and 4 below).\n\nConcretely, we report the TV-distances with a train-test split in the one-page PDF attached with our rebuttal. \n\n**\"..G_{2}...is ...same as...G_{1}....if this assumption were false...\"** This assumption builds on the premise that there are no differences (at a population level) between $G_{1}$ and $G_{2}$; see, e.g., [81,57,41]. If this premise is false, then the effectiveness of interventions can be either underestimated or overestimated which may lead a policymaker to select a suboptimal intervention. That said, if all the considered interventions reduce risk aversion and/or resource constraints, then the chosen intervention should still have a positive impact on the disadvantaged group. We will add a brief discussion in the limitations section on this."}, {"review_id": "E2kog9vVm2", "reviewer": "Reviewer_Aa19", "summary": "This paper studies the issue of biases present in evaluation processes like hiring and school admissions. The authors propose a model that estimates the distribution of utility that incorporates two main features: resource constraints for information, and risk-averseness of the decision-maker. They formulate an optimization problem to estimate the utility distribution with two parameters that represent the aforementioned two features. They study the effect of these two parameters on the solution of the optimization problem, and they conduct a numerical study to study the effect of interventions in a downstream selection task.", "questions": "Please let me know if I have any misinterpretations in my review.", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The problem of bias in evaluation processes is a significant problem, and I like the approach of understanding how bias can emerge in this process via a stylized model. The authors model two important phenomena, resource constraints and risk aversion, that have been shown to arise in many settings and can contribute to bias. The authors use real-world datasets to validate their study.", "weaknesses": "The exposition of the paper was poor, making it challenging to comprehend the main message of the work. I understood the paper as positing a stylized model of an evaluation process that incorporates resource-information and risk averseness, and the main question that is studied is to understand how these two features contribute to the emergence of bias. However, I did not get a satisfactory understanding of this question from reading this work. \n\n1. The main point of confusion was the lack of a formal model of bias, and an interpretation of what the main framework, OptProg, represents in reality. OptProg outputs a density of scores, $f_{\\epsilon}$, where $f_{D}$ is a “true” distribution of scores. I interpret this via the following example: the distribution of SAT scores across all students ($f_{\\epsilon}$) is not equal to the distribution of “true ability” of same students ($f_{D}$). However, only looking at the distribution seems insufficient for understanding “bias”, since it does not specify how _each_ true score gets mapped to each “biased” score. This is the source of the issue in some of my next comments. \n\n2. Section 3.2 studies how tau and alpha influence the solution to OptProg, which seems to be the main contribution of this work. However, this section relies on specific distribution examples and numerical investigations, with long discussions and alot of notation that was difficult to follow. Moreover, it fails to explain why mean and variance are the relevant statistics of interest and how they capture the notion of \"bias\". Providing explicit theorem statements would have greatly improved this section.\n\n3. The numerical study had several parts that I found confusing:\n- in Section 4.1, fitting the OptProg model to real-world datasets and evaluating the TV distance does not demonstrate the model's ability to capture \"Ability to capture biases in data\" (section title). Why does a small TV distance imply bias? \n- The model fits the best tau and alpha, but wouldn’t tau = -\\infty always be the best tau? Also, is it the case that alpha > 1 is a better fit than alpha <= 1?\n- The interventions aimed at changing tau and alpha from the best fit would increase the error in estimation (increase TV). Why would these interventions improve utility? The confusion arises from the lack of a formal model of utility and bias.\n- There was no clear conclusion from the numerical results: each intervention was the best in some regime. What should the reader take away from this, and how should a DM use these results? \n- To strengthen the study, I suggest including numerical analyses that (a) validate the model and (b) rigorously estimate alpha and tau for different groups to demonstrate the existence of this type of bias.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback. We are glad that you like our approach. There do seem to be some misinterpretations that we have clarified. We hope that you will increase your support for the paper. \n\n    \n**\"...lack of a formal model...what...OptProg...represents\"** OptProg is an abstract model of how an input distribution of true utility of a group of individuals gets transformed by an evaluation process to an output distribution in the presence of risk-averseness and information constraint. Since the corresponding parameters $\\alpha$ and $\\tau$ depend on socio-economic factors, we chose to focus on the input/output of OptProg at a population level. However, the intuition for OptProg does come from understanding how an individual's true ability $u$ gets mapped to a biased score. Indeed, we can recover this behavior from OptProg by plugging in $f_\\mathcal{D}$ to be the Dirac delta distribution around $u$ and setting $\\alpha$ and $\\tau$ appropriately. We will clarify this.\n    \n**\"...mean and variance...\"** Our work is rooted in several prior works that have argued that mean and variance of population level distributions are relevant statistics in understanding bias. E.g., [81] points out that in the distributions of lifetime citations of men and women scientists, the average number of citations is lower for women than men. As for variance, quoting from [57] -- *different groups of candidates may exhibit different variability when their quality is estimated through a given test*.\n    \nSeveral theorems on mean and variance appear in the Supplementary Material (Theorems D.1, D.5, D.7, E.1, E.2). These theorems discuss the effect of changing $\\tau$ and $\\alpha$ on the mean/variance of the output distribution. Based on your suggestion, we will include informal versions of them in the main body.\n    \n\n**\"...Why does a small TV distance imply bias?\"** Sorry, this is a typo. We intended to say that a small TV distance between the best-first distribution generated by our model and the distribution of utilities in a dataset (which is already known to be biased; see the note below) implies that our model can generate distributions that capture biases arising in real-world datasets. We will fix this.\n\nNote: All of the real-world datasets that we consider are already known to have biases in utilities across the  protected groups (defined by gender and/or birth category for the JEE-2009 dataset and defined by gender for the semantic scholar dataset); see, for instance, [41] and [57]. \n\n**\"...wouldn’t $\\tau = -\\infty$ always be the best tau?\"** Setting $\\tau=-\\infty$ just leads to the output density having the least amount of uncertainty. E.g., when $f_{\\cal D} = N(\\mu,\\sigma^2)$, $\\tau=-\\infty$,  $\\alpha=1$, $\\ell(v,x)=(v-x)^2$, the output of OptProg is the Dirac-delta function at $\\mu$. This is not the best fit solution. Whereas if we pick $\\tau$ to be equal to the entropy of $f_{\\cal D}$, the output is same as $f_{\\cal D}$ which, by definition, is the best-fit distribution. \n\nWe report the best-fit $\\tau$ in Figure 6 of the Supplementary Material.\n   \n**..is it the case that $\\alpha > 1$ is a better fit than $\\alpha \\leq 1$?\"** Not in general. For the Semantic Scholar dataset and the synthetic network data, $\\alpha>1$ is a better fit than $\\alpha\\leq 1$. However, for the JEE-2009 dataset, the opposite holds: $\\alpha\\leq 1$ is a better fit than $\\alpha>1$. We report the best-fit $\\alpha$ in Figure 6 of the Supplementary Material. Concretely, Table 1 shows that the TV-distances with $\\alpha=1$ are poorer than with the best-fit $\\alpha$ (0.12, 0.11, 0.08, and 0.02 compared to 0.08, 0.07, 0.03, and 0.01 respectively).\n\n\n    \n**\"...Why would these interventions improve utility?\"** Changing $\\tau$ and $\\alpha$ from their best-fit values (for the biased distribution values) does increase the error in estimation from the biased distribution of values, but it may decrease the error in estimation from the true distribution of values. Since utility is measured as the sum of true (and not biased) values of the selected individuals (Lines 352 – 359), changing $\\tau$ and $\\alpha$ from their best-fit values can improve utility. We will emphasize this further in the final version.\n\n**\"...each intervention was the best in some regime...how should a DM use these results?\"** That is correct. Given the regime that is of interest to a policymaker, the policymaker can use our model to study the effects in order to systematically identify the best intervention and, based on its assessed efficacy, decide whether to enforce it (Lines 395-397). As mentioned in the abstract (and the introduction), our work provides a tool for the policymaker to guide the deployment of interventions to mitigate biases. Empirical sections provide use cases that may help policymakers to see how they can use our model. Prescribing which intervention to use in which context is outside the scope of this work.\n    \n**\"...I suggest including numerical analyses that (a) validate the model and (b) rigorously estimate alpha and tau for different groups...\"** Perhaps there is some confusion, we do validate our model: We estimate $\\alpha$ and $\\tau$ for utilities of disadvantaged groups in three real-world datasets (Lines 12 - 13).  We discuss implementation details of the estimation in Lines 303-310, report the resulting TV-distances between the best-fit distribution in Table 1, and report the best-fit values and distributions in Figure 6 in the Supplementary Material. We observe that the resulting distributions output by our framework are close in TV distance (they have TV-distances $\\leq 0.08$; Table 1) to the distributions of biased utilities in the real-world data. Moreover, our model has a better fit than the implicit variance model [57] and the multiplicative bias model [81] on datasets where utilities have skew (Rows 1, 3, and 4 in Table 1) (Lines 115-116)."}]

Author rebuttal:
We thank the area chair for their time and effort in engaging with the reviewers and considering our rebuttal.

We thank all the reviewers for their excellent suggestions which will help improve the paper and for considering our rebuttal. We take the feedback of reviewers seriously and have addressed their specific questions and concerns in individual responses.

Based on suggestions by Reviewer gzsA, we give two responses below that may be of general interest. The first is an overview of the proof of Theorem 3.1 and the second is some specific examples of mechanisms by which information constraints and risk aversion lead to bias.

We also attach a pdf file that contains a new figure that we reference in response to Reviewer gzsA.

We will include all these changes in the final version.


**1. Overview of the proof of Theorem 3.1.** 
 

(i) We first consider the dual of OptProg and show that strong duality holds (see Sections B.2 and B.3). The proof of strong duality (Theorem B.3) uses Slater's condition. This step allows us to derive the form of the optimal density to OptProg and is standard (e.g., in formulations that maximize entropy subject to additional constraints). 

(ii) The next step is to show that the optimal solution (density) of OptProg exists and is unique. This requires proving that the dual variable $\gamma^\star$ (corresponding to the entropy constraint) is *positive* -- while this variable is always nonnegative, the challenge is to show that it is *nonzero*.  There are instances to OptProg where $\gamma^\star$ is zero, and an optimal solution does not exist (or an optimal solution exists, but it is not unique). Proving that $\gamma^\star$ is nonzero under general conditions is the main technical difficulty.

(iii) The next step is reducing the proof of $\gamma^\star \neq 0$ to understanding the properties of the following integral that captures the expected loss when the estimated utility is $x$ (see lines 1026--1027): $I(x) = \int_{\Omega} \ell_\alpha(x,v) f_{\cal D}(v) dv.$ The objective function of OptProg is equal to the expectation of $I(x)$ over the output density $f^\star(x)$. 

(iv) In section B.4, we show that $I(x)$ can be expressed as a sum of two monotone functions (see Theorem B.10). This decomposition allows us to show that the optimal value of OptProg is finite and $I(x)$ has a global minimizer $x^\star$ (Claim B.12). 

(v) In Section B.5, we show that the optimal value of OptProg is strictly larger than $I(x^\star)$ (Lemma B.13, B.14). This requires us to understand the interplay between  the growth rate of the expected loss function and the entropy of a density as we place probability mass away from $x^\star$. 

(vi) Finally, in Theorem B.15, we show that $\gamma^\star$ is nonzero. This follows from the fact that if $\gamma^\star=0$, then the optimal value of OptProg is equal to $I(x^\star)$, which contradicts the claim in (v) above. Once we show $\gamma^\star > 0$, the expression for the (unique) optimal solution follows from equation (13) in line  1128. 
    
To summarize, the technical difficulty in proving Theorem 3.1 lies in showing $\gamma^\star \neq 0$ and the conceptual difficulty lies in formulating general conditions under which this property holds. Proving that these conditions hold for a general class of loss functions is non-trivial. We carry out these steps for Gaussian, Pareto, Exponential, and Laplace densities in Sections F, G, I, and J respectively. 

**2. Specific examples of mechanisms by which information constraints and risk aversion lead to bias.**

One context is college admissions. It is well known that SAT scores are implicitly correlated with income (and, hence, test preparation) in addition to student ability ("Is Income Implicit in Measures of Student Ability?", Budget Model, Penn Wharton). While the true ability may be $v$, the score is skewed depending on amount/quality of test preparation, which depends on socio-economic status. The parameter $\alpha$ in our model can be used to encode this. As for $\tau$, while an evaluator may know what a GPA means at certain universities well known to them, they may not understand what GPA means for students from  lesser-known school to them. This lack of knowledge can be overcome, but takes effort/time, and without effort entrenches the status quo.


Another example is evaluation of candidates using a standardized test.  In time-constrained settings, high  value of the resource-information parameter $\tau$ for the disadvantaged group indicates that such candidates may not be able to comprehend a question as well as someone from an advantaged group. This could be due to various factors including less familiarity with the language used in the test or the pattern of questions, as opposed to someone who had the resources to invest in a training program for the test.  Similarly, a high value of the risk-averseness parameter captures that an evaluator, when faced between a choice of awarding low or high marks to an answer given by a candidate from the disadvantaged group, is less likely to give high marks. More concretely, suppose there are several questions in a test, where each question is graded either 0 or 1. Assume that a candidate has true utility $v \in [0,1]$, and hence, would have received expected score $v$ for each of the questions if one were allowed to award grades in the continuous range $[0,1]$. However, the fact that the true scores have to be rounded to either 0 or 1 can create a bias for the disdvantaged group. Indeed, the probability that an evaluator rounds such an answer to 1 may be less than $v$ -- the risk-averseness parameter measures the extent to which this probability gets scaled down.

Meta-review: This work proposes a new optimization-based approach to modeling bias in evaluations. Reviewers have raised several important concerns on the lack of clarity, and the authors present a detailed plan to address them. The proposed model is an interesting contribution, and the paper's impact can be maximized by putting further work on the exposition. I would like to echo Reviewers gzsA's comments on expanding the discussion on implications for policy-making.
Decision: Accept (poster)