Title: Matchings Under Biased and Correlated Evaluations
Venue: NeurIPS 2025
Accepted: yes
Keywords: Stable matching

Evaluation Bias

Correlated rankings

Equilibrium analysis

Representation ratio
Abstract: We study a two-institution stable matching model in which candidates from two distinct groups are evaluated using partially correlated signals that are group-biased. This extends prior work (which assumes institutions evaluate candidates in an identical manner) to a more realistic setting in which institutions rely on overlapping, but independently processed, criteria. These evaluations could consist of a variety of informative tools such as standardized tests, shared recommendation systems, or AI-based assessments with local noise.  Two key parameters govern evaluations: the bias parameter $\beta \in (0,1]$, which models systematic disadvantage faced by one group, and the correlation parameter $\gamma \in [0,1]$, which captures the alignment between institutional rankings. We study the representation ratio $\mathcal{R}(\beta, \gamma)$, i.e., the ratio of disadvantaged to advantaged candidates selected by the matching process in this setting. Focusing on a regime in which all candidates prefer the same institution, we characterize the large-market equilibrium and derive a closed-form expression for the resulting representation ratio. Prior work shows that when $\gamma = 1$, this ratio scales linearly with $\beta$.  In contrast, we show that $\mathcal{R}(\beta, \gamma)$ increases nonlinearly with $\gamma$ and even modest losses in correlation can cause sharp drops in the representation ratio. Our analysis identifies critical $\gamma$-thresholds where institutional selection behavior undergoes discrete transitions, and reveals structural conditions under which evaluator alignment or bias mitigation are most effective. Finally, we show how this framework and results enable interventions for fairness-aware design in decentralized selection systems.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors develop a large‐market stable‐matching model with two evaluating institutions whose signals are both group-biased (disadvantaged candidates’ scores are down-scaled by a factor $\\beta\\le 1$) and imperfectly aligned (correlation parameter $\\gamma\\in[0,1]$). They derive closed-form formulas for the pair of admission cut-offs that uniquely characterize every stable matching, reveal three critical $\\gamma$ thresholds that partition the parameter space into four analytic “phases,” and obtain a piecewise expression for the disadvantaged group’s representation ratio $R(\\beta,\\gamma)$. Although each institution’s threshold can behave non-monotonically, $R(\\beta,\\gamma)$ is proven to increase with either less bias (higher $\\beta$) or tighter signal alignment (higher $\\gamma$). A normalized version of $R$ cleanly separates these two effects, enabling the authors to trace Pareto frontiers of bias-reduction versus alignment-improvement interventions that achieve any target fairness level—thus offering actionable guidance on when debiasing, standardization, or coordination yields the greatest gains in representation.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n* The paper presents a complete large-market analysis, deriving closed-form formulas for both admission cut-offs *and* the resulting representation metrics; proofs are laid out carefully in the main text and extensive appendices, with clear intermediate lemmas (e.g., Theorem 3.1–3.3 and Table 1’s structural summary) that make the logical flow easy to audit .\n  \n* By turning the representation ratio into an explicit function $R(\\beta,\\gamma)$ and normalised variant $N(\\beta,\\gamma)$, the authors translate their math into concrete design tools (e.g., the Pareto-frontier plots in Fig. 3) that a practitioner could use to pick a cost-effective debiasing or standardisation strategy .\n  \n* Prior studies either assume common rankings with bias or study correlation without group-specific bias (Brilliantova & Hosseini, 2022). This paper is, to my knowledge, the first to analyse the *interaction* and to prove monotonicity of a fairness metric despite non-monotone intermediate thresholds .\n  \n* The writing is well-structured: Section 2 walks through the model step-by-step, and “Step 1 / Step 2” derivations of $s_1^\\star$ and $s_2^\\star$ (p. 6-7) are particularly reader-friendly .\n  \n* Summaries and visual aids (Table 1, Figures 2–3) make the complex piece-wise behaviour intuitively accessible .\n\n## Weaknesses\n\n* Uniform score distributions, two institutions of equal capacity, and a single prestige-dominant preference profile (all candidates prefer institution 1) abstract away many features of real markets. The authors acknowledge some of these limitations themselves , but they remain a barrier to external validity.\n  \n* All results live in an asymptotic continuum. Even a small simulation study or a synthetic-to-real case (as done by Celis et al., 2024 for the one-institution model) would have strengthened the quality argument.\n  \n* The four-case formula for $s^\\star(\\gamma)$ and the ensuing phases are relegated to dense algebra; newcomers to matching theory may struggle to interpret why the geometric regimes matter.\n  \n* Some notation choices (e.g., switching between raw, biased, and scaled utilities) require multiple passes to parse. A running toy example early on could improve approachability.\n  \n* Real clearinghouses (college admissions, labour platforms) routinely involve dozens of institutions and many protected attributes, so scaling of results is unclear.\n  \n* In the main article the probability of each candidate prefers Institution 1 over Institution 2 is equal to 1 and the general case put on the appendic. In my opinion as far as possible it is better put main and general result in the main article not in appendix. We have the same situation fo for consider the general case of $\\beta$.\n  \n* In the main article the Institution 1 selects only from group $G_1$ but it claims general case is consider in the appendix.\n  \n* The definition of fairness criteria is defined only the fraction of selected candidates from group $G_j$. Since the main point of your model is preference of institution 1 to 2. Why this point is not consider in the definition of fairness criteria\n  \n* As my understand to run intervention planning we need to estimate $(\\beta_0,\\gamma_0)$, but in this work do not explain how can estimate it by emperical data.\n  \n* Without empirical or simulated finite-market evidence, practitioners can’t judge how sharp the analytic thresholds are for real-sized cohorts.\n  \n\nThe paper is mathematically rigorous, clearly written, and addresses a problem of growing practical importance. In my view, it has substantial impact and surpasses several previously published studies on the topic. Its main drawback, however, is the overall structure. Owing to conference page limits, the manuscript exceeds 60 pages and relegates many key discussions and facts to the appendix, which diminishes the work’s immediate value. Although I am not a specialist in this field, I believe that submitting a journal version—where the principal results are included in the main text rather than the appendix—would give the paper a strong chance of acceptance and make it better suited for a journal than for a conference.\n\nQUESTIONS:\n1. Would your selection procedure change fundamentally under additive bias, or could your proofs adapt directly?\n\n2. You state: “In the finite setting, stable matchings are characterised by threshold-based rules.” Could you cite the specific sources supporting this statement?\n\n3. The main results in the article focus on the closed‐form solutions across different parameter phases. In practice, when $n$ is not large, how can these theoretical treshholds  guide the design of effective intervention methods?\n\n4. How sensitive are the $\\gamma$ thresholds and the monotonicity of $\\mathcal{R}$ to non-uniform or correlated talent distributions?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, authors study how institutional bias and inter-institutional correlations in evaluation affect the ratio of advantaged and disadvantaged populations under matching of candidates and institutions. Here, disadvantaged means that the quality may be underestimated multiplicatively by a bias factor. The main claim is that both reducing bias and increasing correlation between evaluations of institutions can improve the representation ratio of disadvantaged groups. The authors support this claim with detailed analytical results of how the bias and correlation parameters affect the representation ratio at equilibrium. Finally, their framework can extend to many natural generalizations which makes this framework nice for evaluating decentralized selection decisions on the demographics of selected candidates.\n\nSTRENGTHS AND WEAKNESSES:\n$\\textit{Strengths}$: The authors present a well-motivated problem and well-motivated model parameters that correspond to key intervention points of evaluating decentralized admission procedures. Additionally, their model is very clearly explained, assumptions are clearly stated, and the authors highlight possible generalizations whenever they state simplifying assumptions (such as uniform candidate preference for one institution over another and sufficiently unbiased institutions). The organization of Section 3 and the supporting figure and table create nice intuition which makes it easier to follow along and understand their main conclusions, even though their analysis admits somewhat messy closed form solutions. Finally, Section 4 has a nice discussion of example interventions for both $\\beta$ and $\\gamma$ and the numerical experiments support the extent to which the interventions can improve $\\mathcal{R}$ (and $\\mathcal{N}$).\n\n$\\textit{Weaknesses}$: The complex analysis of the equilibrium strategy of the second institution reduces the clarity of section 3 and some details of their analysis could be deferred to the appendix for the sake of clarity. Additionally, I was interested in more discussion on the extent to which the monotonicity of $\\mathcal{R}$ with respect to $\\gamma$ depends on the assumption that attribute 1 is drawn iid for each candidate. That is, should we expect correlation of evaluation strategy to only improve representation if the institutions are correlating on an attribute where all candidates have the same distribution over values and each are independently assigned a value?\n\nQUESTIONS:\n- Is it possible to simplify the theorem statements in the main text (and keep full statements in the appendix) to more clearly show that $s_2^*$ and $\\mathcal{R}$ have the properties necessary for intervention as explored in Section 4? \n- (Extension question) How might $\\mathcal{R}$ change over time if this model is extended over time, especially if previous admission decisions influence candidate preferences in future time steps? \n- Small question, but what happens if $c= \\frac{1}{2}$, i.e., is there a special case when both institutions can admit everyone?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study a matching problem where two sets of candidates apply to two institutions. One set of candidates is advantaged, and the other is disadvantaged. Both candidates prefer one institution over the other. Their priorities according to the institutions are randomly drawn but the priority of a candidate at one institution is correlated to their priority at the other. The disadvantaged candidates have their priorities further scaled down by a bias factor. The authors study the behavior of the representation of these students at the institutions as a function of the bias factor and the correlation factor between the priorities at different institutes. They fully characterize this behavior in a simple setting where the number of candidates is large (tending to infinity). They show that in a particular regime, the representation of the disadvantaged students increase with decreasing bias and increasing correlation.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is very well-written and studies an interesting problem. Their problem, even though it uses a simple setting, seems to require a tedious analysis but they do a good job of highlighting the more interesting parts. Their interpretation and discussion of their analytical results is also very thorough and they also discuss useful simplified examples where appropriate.\n\nMy only complaint would be that the techniques don't seem very novel or if they are, they haven't highlighted it in the main body of the paper. But that is minor and is to be expected with these kind of problems.\n\nQUESTIONS:\nI was a bit surprised that the representation ratio was positively correlated with the correlation factor gamma, given that the tone of the introduction seemed to be making a case against positive correlation. I don't know if this was intentional or not. In hindsight, this positive correlation makes sense since the two dimensions of the utilities are independent and the bias factor applies equally to both of them. \n\nWould this correlation flip if the \\beta for u_2 was slightly higher (ie less biased) than that for u_1?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}