Learning to Rank with Top-$K$ Fairness
Authors: Boyang Zhang
Quanqi Hu
Mingxuan Sun
Qihang Lin
Tianbao Yang
Venue: Accepted by TMLR ()

Abstract: Fairness in ranking models is crucial, as disparities in exposure can disproportionately affect protected groups. Most fairness-aware ranking systems focus on ensuring comparable average exposure for groups across the entire ranked list, which may not fully address real-world concerns. For example, when a ranking model is used for allocating resources among candidates or disaster hotspots, decision-makers often prioritize only the top-$K$ ranked items, while the ranking beyond top-$K$ becomes less relevant. In this paper, we propose a list-wise learning-to-rank framework that addresses the issues of inequalities in top-$K$ rankings at training time. Specifically, we propose a top-$K$ exposure disparity measure that extends the classic exposure disparity metric in a ranked list. We then learn a ranker to balance relevance and fairness in top-$K$ rankings. Since direct top-$K$ selection is computationally expensive for a large number of items, we transform the non-differentiable selection process into a differentiable objective function and develop efficient stochastic optimization algorithms to achieve both high accuracy and sufficient fairness. Extensive experiments demonstrate that our method outperforms existing methods.

Published in Transactions on Machine Learning Research (09/2025)
Learning to Rank with Top-K Fairness
Boyang Zhang
bzhan299llu.. edu
Computer Science & Engineering, Louisiona State University
Quanqi Hu
quangi-hu@ttamus edu
Computer Science & Engineering, Teros ABM University
Mingxuan Sun"
mssml 1 Blsu edu
Computer Science Engineering, Louisiona State University
Qihang Lin Business Analytics, University of lowa
edu
Tianbao Yang
tianbao-yang Blanna edu
Computer Science & Engineering, Texos ABM University
Reviewed on OpenReview: openneviee... net/ forun? id= SSPCc39XvO
Abstract
Fairness in ranking models crucial, disparities in exposure can disproportionaaely affect protected groups. Most fairness-aware ranking systems focus ensuring comparable average exposure for groups across the entire ranked list, which may not fully address real- world concerns. For example, when a ranking model is used for allocating resources among candidates or disaster hotspols, decision-makers often prioritize only the top-K ranked items, while the ranking beyond top-K becomes less relevant. In this paper, we propose a list-wise learning-to-rabe framework that addresses the issucs of incqualitics in top-K rankings at training time. Specifically, we propose top-K exposure disparity measure that extends the classic exposure disparity metric in ranked list. We then learn ranker to balance relevance and fairness in top-K rankings. Since direct top-K selection is computationally expensive for large number of items, we transform the diefeiiferen selection process into differentiable objective function and develop efficient stochastic optimization algorithms to achieve both high accuracy and sufficient fairness. Extensive experiments demonstrate that our method out performs existing methods.
1 Introduction
Top-K fairness in ranking has become critical concern since unfair rankings lead to inequalities such uncqual business opportunitics, educational placements, and resource allocation (Kulshrestha et al. 2017 Mohler el al. 2018 Shang et al. 2020). Ranking models used for decision-making lypically provide more exposure to top-ranked items than those ranked lower (Singh & Joachims 2018). For example, in educational systems, funding agencies allocate more resources to top-ranked schools Haamoon 2001). In hiring systems, only the top-K candidates may be interviewed (Kweon et al. 2024). Similarly, in disaster response or predictive policing (Mohler et al. 2018), top-K hotspol rankings determine where limited resources are deployed. In such scttings, fairness in the top-K results has direct and significant real-world impact. Due factors like historical discrimination, items in the protected group that possess a specifie attribule, such gender, are often mnder-represented within the training dataset. This can lead the model to generate rankings that exhibit substantial disparities in exposure between groups
*Corresponding author

---

Published in Transactions on Machine Learning Research (09/2025)
Fairness in ranking differs significantly from traditional fairness metrics in classification, as it requires considering position bias. Traditional fairness literature mainly focuses on ensuring equal classification outcomes, such as equalized odds and demographic parity (Hardt et al. 2016 Zafar et al. 2017). Pairwise fairness metrics AAbdlllapoouri et al., 2017 Beutel et al. 2019 Narasimhan et al. 2020: Fabris et al. 2023
which are derived from fairness definitions in binary classification, focus on preserving the pairwise relative order of items according to their relevance scores, irrespective of group membership. However, these metrics often overlook the position bias inherent in ranking, where top-ranked items receive more attention. In contrast, list-wise metrics such as Biega et al. (2018); [iinhh & Joachims (2018); Zehlike & Castillo (2020) better address fairness in ranking. While some Biega et al. 2018) focus on individual fairness, the majority of them emphasize group fairness, ensuring that different groups receive similar average exposure.
While post-prosesseeeee processing methods (Zehlike et al. 2017 Biega et al., 2018 Asudeh et al. 2019 Mehrotra & Vishnoi 2022) are proposed to address fairness by re-ordering ranked lists, these approaches face two key limitations: (1) suboptimal trade-offs between relevance and fairness, and (2) reliance on group labels during lesting. First, post-processing methods adjust rankings after the model has been trained, meaning they do not incorporate fairness constraints within the model itself, which makes it difficult to achieve the best ranking quality for given fairness level or to achieve the highest fairness for fixed ranking quality. Second, these methods access to sensitive group labels during testing, which can limit generalization and raise privacy concerns, especially when such labels are unavailable or sensitive.
In contrast, in-processing methods such as Zehlike & Castillo (2020) that integrate fairness directly into the saaning-to--rak
learning- to-rank process during training ensure Pareto efficiency. Specifically, in-processing methods can be viewed solving constrained optimization model where the quality of ranking is optimized subject to constraint that bounds the level of unfairness. This means the in-process methods guarantee that no further improvement in fairness can be made without sacrificing ranking quality. While existing in in-processing methods (Zhu et al.] 2021 Memarrast et al., 2023) account for position bias, they do not emphasize fairness in critical top- positions. This limitation underscores the need for in-processing approaches that explicitly ensure fairness within the top-K rankings.
In this paper, we propose a list-wise learning-to-ranl' framework that addresses the issue of inequalities in top-K rankings at training time for the first time. Specifically, we introduce a novel top-K exposure disparity metric, which is an extension of the group exposure disparily in ranked list (Singh & Joachims) 2018 Zehlike& & Castillo 2020). We then learn a ranker that optimizes the list to balance ranking relevance and exposure disparities at top-K positions. A direct top-K selection process, such as a naive approach based on sorting the whole list, is computationally expensive for a large number of items. To this end, we transform the non-differentiable top-K selection into a differentiable objective function and develop efficient stochastic algorithms.
To empirically validate the effectiveness of our method, we conduct comprehensive set of experiments using popular benchmark datasets. The experimental results demonstrate that our method not only achieves high ranking accuracy but also significantly alleviates exposure disparitics at top-K positions when compared to several state-of-the-art methods. To the best of our knowledge, this is the first lime an in-processing learning-tor to-nnkk framework is proposed to address both relevance and fairness in top-K rankings with a provable convergence guarantec.
2 Related Work
Ranking fairness metrics based on pairwise comparisons (Abdollahpouri et al., 2017 Beutel et al. 2019 Narasimhan et al. 2020 Fabris et al., 2023 are proposed to ensure the relative order of pair is consistent with certain fairness principles. For example, ranking algorithm is considered fair if the likelihood of a clicked item being ranked higher than another relevant unclicked item is equal across groups, provided both
items have received the same level of engagement (Beutel et. al. 2019). In contrast, list-wise approaches optimize fairness across the entire ranking list by ensuring balanced exposure and relevance for all items (Singh)
& Joachims 2018 Yang & Stoyanovich 2017 Zehlike & Castillo 2020 Kotary et al., (022). ) For example, a statistical parity- based mcasure Yang & Stoyanovich (2017) is introduced to calculate the difference in
the distribution of various groups across different. prefixes of the ranking. Studies such as (iinhh & Joachims
2

---

Published in Transactions on Machine Learning Research (09/2025)
2018): Zehlike & Castillo (020)) define equal exposure fairness, aiming to equalize the average exposures
between minority and majority groups. Some other methods (Chakraborty et al. 2022 Zhao et al. 2023) address fairness in ranking aggregation, a different area from learning-to-rank
Ranking fairness can be addressed through pre, in-, and post- post-processing approaches. Pre-processing methods aim to prevent biased models, but creating an unbiased training set is complex and can result. in reverse
discrimination (Zehlike & Castillo 2020). Post-processing methods (Islam et al. 2023 Gao et al.) 2022
Yang et al.] 2023; Mchrotra & Vishnoi| 2022 Zehlikke et al.) 2022 2017; Biega all 2018 Asudeh et al,,
2019 Vardasbi et al., 2024 Gorantla et al.) 2024) are developed to re-rank a list to satisfy specific fairness
criteria. Specifically, The FA* algorithm, introduced in Zehlike et al. (2017), adjusts ranking lists to ensure that the ratio of items from the protected group in each prefix of the top-K rankings meets or exceeds a specified minimum threshold. Additionally, the algorithm is enhanced to accommodate multiple protected groups Zeehlike et al., 2022). However, the houristic adjustments are constrained by other models and are only compatible with their spocific fairness metric. Moreover, there is no widely accepted definition of top- K fairness nor theoretical guarantee for satisfying fairness constraints (Lahoti et al. 2019).
Post-processing methods such as Celis et al. 2017 Singh & Joachims (2018 2019 Kotary et al. 2022 aim to determine a probabilistic ranking that maximizes utility while satisfying fairness constraints. For example, a doubly stochastic matrix represents the probability of item i being ranked at position j. and the optimal matrix is learned to maximize expected utility subject to group exposure fairness constraints. The matrix is solvable via linear programming, and the sampled rankings achieve exposure fairness in expectation (Singh & Joachims 2019). These methods are categorized as post-processing because they depend on an underlying predictive model to estimate item relevance. Additionally, these methods aim to estimate the probabilities of each item being ranked at any position, which are computationally expensive.
In- In-processing methods aim to address fairness directly within the ranking model during training. DELTR (Zehlike & Castillo, 2020) extends ListNet (Cao et al. 2007) to a framework that optimizes ranking accuracy and reduces unfairness, defined as discrepancies in ranking exposure between two groups. Zhu et al. (2020 introduce a debiased personalized ranking model addressing item under-recommmnddtioon bias by improving ranking-basee statistical parity and equal opportunity. Robust (Memarrast al. 2023) constructs minimax game for fair ranking by balancing fairness constraints with utility using distributional robustness principles, achieving fairness-utilityy trade-olfs. MCFR (Wang et al.] 2024) introduces meta-learning framework combining pre- and in-processing techniques with curriculum learning to address fairness across entire ranked lists. However, these approaches concentrate on the entire ranked list rather than prioritizing top-K positions.
In-processing approaches for top-K fairness are lacking. Unlike post-processing methods, we focus on in-processing approaches that provide a thcoretical guarantee for satisfying fairness constraints at top-K positions. Different from K-SONG (Qiu et al. 2022), which focuses solely on top-K ranking relevance (NDCG), propose stochastic algorithm to optimize top- ranking that achieves both high relevance, measured by NDCG, and sufficient fairness, quantified by reducing exposure disparities between minority and majority groups. Morcover, unlike other in-processing fair ranking frameworks (Memarrast et al. 2023 Zhu et al. 2020), which are not scalable due to their model complexilies, our framework is efficiently optimizable and scalable to large datasets.
3 Preliminaries
Let. Q denote a set of N queries and E Q denote a query (e.g., a query for document retrieval or a user for recommendation). Let S, = {x:li = I,..., JN,} denote a set of N₂ items (e.g., documents, products) to be rankod for 9, where x denote the embedding for each item with respoct to query q. Lot S be the sot of all query-item pairs, i.e., S {(g x?]q E Q, x² E S,). Let yf denote the relevance score (e.g., a rating) between query q and item x² and 3,- = {u. NN denote the set of all relevance scores for query q. For simplicity, we assume that the items in S, belong to two disjoint groups. Let Sy c S. denote the set of items in the minority group and S c St denote the set of items in the majority group.
Let (x w) denote a prodictive function that outputs a score for x with respect to the query q with a higher score leading to higher rank of x in an output list. The parameters of the scoring function are denoted
3

---

Published in Transactions on Machine Learning Research (09/2025)
by (e.g., deep neural network). The classical approach to obtaining good hy(x: w) is to optimize by maximizing a quality measure the output lists across all queries or, equivalently, minimizing a loss function that decreases with the output quality.
There exist multiple ways to measure the quality of ranking items in Sq with respect to query 9. One commonly used example the NDCG measure, which defined as:
NDCG 1 Σ
2 -1
(1)
7u xXES,
log(1 rr:: x.
Here, Zg is a normalization constant and r(w: S.y E (I, is the rank of x in the set Sy based on (x; w); that is,
r(w: x, Sy)
Σ ((hg(x';;)) (x; w) 0) X'ES,
(2)
where the indicator function I(-) outputs if the imput is true and 0 otherwise. Note that higher score (x; w) leads to smaller r(w; Sq), and a higher NDCG means higher quality of ranking for query q- One can thus optimize w to maximize the average NDCG over all queries to obtain a good prediction score function (x; w). However, doing so is challenging due to the discontinuity of r(w; x, S2) in w. Therefore, according to Qiu et al. (2022). one can approximate r(w; x, Sy) by a continuous and differentiable surrogate function g(w; x, S,) = Exes, (xhg(xx; w) (x; w)), where ((-) an increasing surrogate loss function of T(- 2 0). For example, the squared hinge loss ((x) (r + c) Here is margin parameter. We define [(w;'',x, x, q) = (x' - (x(x;)) and obtain a more computationally tractable NDCG loss:
29
Lg(w)= = Zy
Σ x?ES,
(3)
log 1 + g x(w::x(
and one train (x w) by minimizing the average NDCG loss across all queries in S. Another loss function that measures the quality of a ranking with respect to query is the ListNet loss (Cao et al. 2007):
exp(y:)
(w) E X,ES
log (Ā( (o(w;x, S,)),
(4)
No و exp(v?)
where ((W;x,8,) Sq) Ex'Es, exp(hig w) hy (x; w)).
Given a loss function T.g(( (W), e.g., 3 or 4 for each query g E Q, one can minimize the average loss over all queries by solving minw [ST EnEo Lo (w) obtain a good prediction score function (, w). However, we aim achieve a ranking that both high-quality and sufficiently fair. Therefore, a measure of the fairness of (x; w) needs be involved in the optimization above.
There exist multiple ways define ranking fairness. In this paper, we focus the equal exposure fairness of ranking method similar to Zehlike & Castillo (2020). Formally, according to the probability distribution over Sy induced by scores (x, w) x E S, the exposure of item x² E Sy is defined as:
exp(hg(xX:: w)) cWw,x],60 Exees, (xp(h,(x):ww))
(5)
The exposure in (5) can be interpreted as the reciprocal of a surrogate rank function, where higher-ranked items receive greater exposure. The reason is that we can convert it to 1/ Ezyes, exp(h, w) - ha(a%, w)).
As a result, the denominator can considered as a surrogate of the rank function at I using the exponential surrogate function, i.c., the higher the score (2g, w), the lower the denominator Enyes, (xp(h, w)
(2 w)), matching its rank function. The exponential surrogate is widely utilized to approximate rank functions effectively (Rudin) 2009
4

---

Published in Transactions on Machine Learning Research (09/2025)
Given a query q Q, the equal exposure fairness requires that the averaged exposures in both minority and majority groups be equal, namely,
Σ X
e(w, S.)
Σ
e(w, Sq).
(6)
=
S
E
This requirement can be satisfied by minimizing the following loss function for each query q, which measures the disparity in the averaged exposures between groups:
Ug(w)=
e(w, X.
Σ
e(w,
5 x E
x
The learning to rank with equal exposure fairness can be lormalized as an optimization problem:
Σ QE@
(,(w) + Σ U,(W), 9~
min
(8)
S
where the loss [w(w) can be or or any loss function measuring the quality of ranking and loss function (w) is defined in 77,, which can be viewed as a regularization term to ensuring equal exposure fairness, and the parameter balances the quality and the fairness of ranking.
4 Top-K K Ranking Fairness
The equal exposure fairness introduced in 6 is defined based the entire output list for each query, which may not fully address unfairness in real-world applications. For example, when the ranking is used for allocating resources among disaster hotpots, the decision-makers might only prioritize the regions that are ranked at the top-K position, while the ranking beyond K does not matter. To extend our fairness measure to this situation, we develop a general top-K ranking fairness metric to ensure fairness among different groups at top-K positions the output ranking. In particular, the score l,(x; w) satisfies top-K ranking fairness
Σ X-E22
I(x E (e), |S
Σ xPES
I(x E (x)e
S
where S% denotes the set top-K K items (ranked by (x/: w)'s) in Sg with respect to query and e(w, x². Syy) is the exposure function in (5). As shown (5), higher-ranked items receive higher exposure scores. When extending this a top-K ranking, each item within the top-K positions weighted according to its exposure score. The top-K ranking fairness loss can be defined as the squared difference between the left and right sides of equation
Ensuring the top-K ranking fairness lies in the selection of items for the top-K set, i.., x² E S% Note that the lop-K set 5% depends on the predicted scores h, (x;; w) with model parameters w, and naive approach of sorting the scores will be expensive, taking n log n time complexity for a set of n examples. Hence, one efficiently handle top-K ranking fairness. Some related methods such as Wu et al.] (2009); Qin et al. (2010) approximate the top-k indicator by w(K g(w; =?,, S,)). The & is a continuous surrogate of the indicator function. But there exists two levels of approximation errors: one in estimating the rank of an item and another in approximating Il(- 2 0) by (--).
major innovation of this paper
To reduce approximation errors, our idea is to transform the non-differentiable top-K selection operator into a differentiable one using an approach similar to Qiu et al. (2222) Specifically, let
K+ X+ N₂
(w) = arg min
Σ (x): >)+ for q E Q x,ES,
(10)
No
where c E 1). It can be casily proved that (w) uniquely defined and the K + 1-th largest score (x,'x); w)li 1....,,,, .Ny.. Hence, x! E Sk selected in the top-K positions and only hy(x"; w) X(w). In other words, the indicator [(X? sk) in g can be replaced by (xh, w) >(w) 0).
5

---

Published in Transactions on Machine Learning Research (09/2025)
The indicator function [[hg(x);: w) X(w) 0) is discontinuous,
approximate it by smooth surrogate (x?; w) (w)) (e.g,, sigmoid function) within (9) The use of smooth surrogates to replace indicator functions has been studied in [Bendekgey & Sudderth (20210; Yao et al. (0023). To make the loss function differentiable for optimization, we replace the absolute value in with the following squared difference formulation:
UK (w, (ww))
S
x!ES
(11)
Σ
(x w) (wW))
S
E
A learning rank problem with top-k K ranking fairness formulated as the following fairness regularized bilevel optimization problem:
min E4,(w) +
C N
Σ U (w.Ax(w)) (y(w)) 9
[
(12)
9E@
s.t. (y)) satisfies
for q E
However, the lower level optimization problem 10 is non-smooth and non-strongly convex, making challenging to solve numerically. Hence, we approximate the lower-level problem with smooth and strongly convex objective function. Specifically, we define:
12
(w( arg min (g(w,))== K K+E x+ T2 x²x
N₂
(13)
hggx{
+ N₂
Σ x
ri In
1xexp
ES
where TI a smoothing parameter and T2 > a strongly convexity parameter. With this approximation, design solve optimization problem below:
min F(w)
Σ q
Tog(w
C
Σ 9
Uk (w,
(c(ww)
N
(14)
s.t. (,(w) satisfies
13 for y E Q.
This is a challenging optimization problem for several reasons. First, an unbiased stochastic gradient of the objective function in [1] is unavailable due the composite structure in Lg and U Second, when N and N₂ are large, it is computationally expensive to update (g(w) for all simultancously. To address these issues, we view [14 as an instance of the bi-level finite-sum coupled compositional stochastic optimization problems. It was addressed by Qi et al.] (2021): 0 Qiu et al. (2022). Then we apply stochastic algorithm to (4). The key component of this algorithm is to construct and update the stochastic approximations of the gradients of L, and U.K with respect w using a technique called moving average estimators. In the next. two subsections, we will provide the details on how this is done for T, and U.K. respectively.
q
4.1 Stochastic approximation for gradient of ranking loss
Let L(w) := |s| Eneo (w).. When (w) is either the NDCG loss or the ListNet loss, L(W) is finite-sum composite function, that is,
L(w) = S
Σ
fax(o(w;x(.5.,)
(15)
(gxx)es

---

Published in Transactions on Machine Learning Research (09/2025)
where (w = N, x] and fq)(()) = 1 1-2"2
2, (Ng when bgg is the NDCG loss and g(:; x. = x/: S₂) and (g)(9) exp(v/) /21- exp(y))) log(Ngg) when Ly the ListNet loss. We will only illustrate how a stochastic approximation of (L(w) can be constructed when Ly is the NDCG loss because the construction for ListNet Loss is similar.
Ny
Suppose the solution iteration We. By chain rule:
VL(w))
Σ
(16)
S
(gXT)S
For large-scale ranking problems, we approximate Vg(w,;x{, by the stochastic gradient V0g,(we) B. ExER, Ve(w;;x'xx,,)) where B, is subset randomly sampled from Sq. To approximate
(g S,)), we maintain scalar (t)) at iteration to approximate g(w; for each query-item
948
pair (q, (q,x,) and update for iteration + by moving averaging scheme. That is:
(++1) u q, = (w + (1 (1- (1-70)u
(17)
where To E [0, 1] an averaging parameter. Moreover, when |S] is large, also generate subset randomly from S, denoted by B, and use it to approximate the average over S in L(w). With these stochastic estimators, we can approximate ([(w;) with
G - B
Σ
Vigi Vin
(18)
(9.X))B
4.2 Stochastic approximation for gradient of top-K fairness regularization
Let U(w) = U (w, 2, (w)). Like L(w), ((w) is also a finite-sum composite function but with an additional challenge that (w) is not given explicitly but through solving the lower level optimization in In particular, according to (5) and 11 have
13
U(w)
Σ q
falgnna (a).naaw),.o (w),.n.(()) Do (w),,
(19)
where
fa 22,22) 23) = / 21 32
2
|S, 23
(o(w) SS
exp(hg(xx35 w)),
x
99.0 (w)
(h, w) - (w)) (xp( w)),
S
X
(w) S
(x, w) (w)) exp(hy (x)::))
x E
By chain rule, we have
V (g (Wee)agae))......... (W:), Dg.bl (g.)).))..).))pp. (w:)) ( Ig.a (We)
VU(w.)
Σ
V2f9 (gq.a (Wa), Rq. ( (Wz), 9g (w.) (g.b(())
Val ( (Wx), Do.t (wx), (w)) V
QEQ
Here, Vafe represents the gradient of fg w.r.t. kth imput for 1,2,3.

---

Published in Transactions on Machine Learning Research (09/2025)
Let Bg, By and Bg be subsets randomly sampled from Sg Sy and Sy, respectively. Suppose have some estimators for (w() and (W,) denoted by yg, and Vdy,s, respectively. We then approximate V8ga(W,p) (wi)( and Vgna(w:)) respectively, by the stochastic gradients:
1
Vaqaa( (we) =
(x) - An,t) (Vhg(x(: w) VAy.c)) exp(h w))
B8
x E
(x,; k))
18
X/EB
V9g.6(w =
(N((h(()?;w)) (Vhh,(xx: (dg.e)) (x?; (20)
1
x
(xx w) exp(h, (/)
BB
E
vig (w) =
xx)(h,((23:
B3 x E
where (-) is the gradient of (.). To approximate the gradient V (.f.(99.((w)) (g (go((We). we maintain three scalars ugyas each g 2, and update them for iteration t + 1 by moving averaging scheme:
(t) (0 and uq at iteration & to approximate Iq.a (e), Pg.b (w,) and go (we),, respectively, for
u ((+1) -99a (W:) + (1 - (1-1) -
q.a
q.a.
u 9.6
(t+1) -200,.(w,) + (1 - 72) )
(1)
(21)
(t+1) (w.))(w.) + (1 - 7
where 7% E [0, 1], k 1,2,3, are averaging parameters just like To- Moreover, when N is large, we also generate subset randomly from 2. denoted by Bg, and use it to approximate the average over Q in U(w). Naturally, use the queries contained in sample B C S defined in the previous subsection as Bq. With these stochastic estimators, we can approximate VU (Ww)) with
V (i u a.b. ug V Ig.a(w.)
(t)
C
B2
Σ BEB
+ V2la(ugg
(t) (8)
u
Ug Vo..
(e)
(22)
VaSa(ugg und Wi
(e) ())
(e)
(we)
The remaining step is to approximate (w) and Viy (w). Using the implicit function theorem as shown in Ghadimi & Wang (2018), we have ('y(w)= = - - Gg (g((g((): w) (V,G w)) At itcration t, we maintain a scalar Ag. as an estimation of Å(w,), scalar 8q,, as an estimation of (i, (w.); we), and a scalar 1g.r+1 as an estimation of VAG,(An,t w.). Let G, w; B,) = N " + [8, Ex,E,, In(1 exp((hg(x:; -))):/7)) be an approximation of GA w) using mini-batch Bg. We then approximate Vdg (w:) by Vdg. == R Gg(W,, (W,, Aqus B,)) Then Ug,0+1 and 8y, are updated by moving averaging method while dn is then updated by approximate gradient step along Uq,011:
89 -74)))8. + 74 V G wi; By),
g (1 -71)"gg 74 V. G. (2 Wo: B,),
(23)
1q1411 dn.1 1111
where Noo 2 0 and 74 [).
4.3 Algorithm and convergence result
According to the previous two subsections, we have obtained the stochastic approxinations of VL(w;) and (U(w:) and thus stochastic approximation of FWW:)) We then update w, to W++1 by momentum
8

---

Published in Transactions on Machine Learning Research (09/2025)
Algorithm Stochastic Opimization of lop-K Ranking with Exposure Disparity: KSO-RED
for 0. ....., do
Draw sample batches B S and let BQ be the set of g's in B. For each E Bq, draw sample batches Bg C B8 Sa: Bg C S
4: for (g,x)) E B do
Compute (w,( and (t+1) end for
u q.
for q Bi do
8:
Compute (we,, (We,) (ws), Ugoa ugb Ul sq.+1, Ueet1 and 1411
(++1) (++1) (f+1)
9:
end for
10: Compute Gf and G3 according to
18 and 22
11: Update z²+1 = 75)2² + (Sf
CG.)
Update w w mztt1
12:
13:
end for
gradient step. This procedure is presented formally in Algorithm | where z is the momentum gradient used to update wt and " E (0, is the momentum parameter. In practice, we ignore the gradients of the top-K selectors (hh (x?; w) >g.t)) and the computation for G2=1 can be simplified.
To present the convergence property of Algorithm some assumptions on problem 12 are needed. We make the following assumptions on problem 12
Assumption 4.1.
(xx(x; w)l S B,, for a constant B, for any g and x.
hy(x: w) is Ch-Lipschitz continuous in w for any and x.
Vilt (x; w) is L,,-Lipschizz continuous in for any and x.
Va w) is Pn-Lipschitz continuous w for any and x.
Stochastic gradients (w), V9q.a(w), ( (wy B.G.(A:.,BB), VRGG(A:w; and V GhAG (A (,; B,), have bounded variance o² for any and x.
Given Assumption 4.1 similar to Qiu et al. 2022 we can prove Theorem convergence property for Algorithm
4.2
We have the following
Theorem 4.2 (Theorem 2 in Qiu et al. 2022
Suppose Assumption 4-1 holds and no. 71, 72, 73, 74, To, are set properly, Algorithm ensures that after T = 0 iterations we can find an c-stationary solution of F(w), i.e., (FW,,,
for a randomly selected E {I....,T].
5
Experiments
5.1 Datasets and methods
We evaluate our algorithm recommendation systems because several benchmark datasets are available, which contain rich item attributes such as genre and year that are useful for fairness evaluation and contain large set of items (e.g., 20K items) for evaluating large-scale ranking systems.
MoV Harper & Konstan 2015 This datasct comprises 20 million ratings from 138,000 users across 27,000 movies. After filtering, each user has rated at least 20 movies. The dataset enriches each movie entry with metadata such as name, genre, and release year.
Netffix Prize datasct (Bennett et al. 2007): Originally containing 100 million ratings for 17,770 movies from 480,189 users, we use random subset of 20 million ratings for computational feasibility, maintaining similar structure including movie name, genre, and year.

---

Published in Transactions on Machine Learning Research (09/2025)
Derived Sensitive Group Datasets: To evaluate fairness between protected and non-protected groups, we derive three subsets from the primary datasets:
MovieLens-22M-HH horror vs. non-horror
200-- documentary vs. non-documentary
Netflix-20M: movies before 1990 vs. from 1990 onwards
We introduce basclines and our proposed ones, all of which are in-processing fair ranking frameworks.
K-SONG (Qiu et al. 2022 A color-blind method achieves top-K ranking accuracy without
considering fairness.
DELTR (Zehlike & Castillo 2020 Optimizes ListNet ranking and disparate exposure ensure group fairness.
DPR-RSP and DPR-REO ZZuu et al. 2020 ; These methods focus on reducing ranking- based
statistical parity bias and mitigating item rnderrrccommennttioon bias, while maintaining recommen- dation performance.
[Mmarrast et al. 2023 An adversarial learning-to-ranle method that optimizes ranking utility while enforcing demographic parity fairness constraints through a minimax optimization framework.
Robust
NG-DE:
We integrate Qiu et al. 2022 and Zehlike & Castillo 2020 to optimize the NDCG ranking
loss and the Disparate Exposure.
SO-RED: Our Stochastic Optimization for NDCG ranking loss and Ranking Exposure Disparity defined in 7 The fairness objective only focuses on exposure instead of top-K exposure. The training objective is the same as the closest baseline DELTR. The optimization algorithm employs moving average estimators for robust gradient computation.
KSO-RED: Our top-K Stochastic Optimization for both top- NDCG and top- K Ranking Exposure
Disparity defined in 11 The code is available at: GitHub repository
5.2 Evaluation metrics and experiment setup
The contral aspoct of our evaluation is the trade-off between accuracy and fairness. Accuracy is measured using the NDCG metric defined in [ with higher values indicating better ranking performance. Fairness is measured by the top-K exposure disparity defined in Equation training objective, we use the Mean Squared Error (MSE) of the difference between the left and right sides of Equation g During evaluation, we report both MSE and Mean Absolute Error (MAE) of this difference, with smaller ler values indicating better fairness performance. We present MAE as the fairness loss in the main text, while the corresponding MSE results are included in the Appendix C
Specifically, for the fairness loss in the
For model training and evaluation, we adopt a conventional split of training, validation, and test as in Wang et al. (2020); Qiu et al. (2022). We employ the classic deep neural model NeuMF (He al. 2017) as the prediction function hy(xx;
[x; w). For all methods, we sample the same batches of queries/ /usors and the mixture of relevant and irrelevant items per query during each iteration to ensure a fair comparison. Details on our experimental setup are in Appendix A including specifics model pre-training, fine-tuning strategies, and hyperparameter selection.
For baseline models, we maintain the same hyper-parameter settings as in the original papers to ensure optimal performance and fair comparison, as detailed in Appendix [ In addition to these quantitative results, we present visualization of how our method adjusts rankings to achieve fairer outcomes, described in Appendix The computing resources and computational efficiency analysis reported in Appendices and
10

---

Published in Transactions on Machine Learning Research (09/2023)
K=50
K=100
K=200
0.58
MovieLeno 20------0-------
0.56
100 420
1000
15.00
0.53
2.54
0.50
620
0.52
0.48
M600
0.50
0.42
0.43
5.45
0.40
0.45
0.008
0.010
0.012
0.014
0.025
0.002
0.004
0.000
0,008
0.000
0.001
0.052
0.003
0.004
Fair
Far loss
Fair uss
0.00
0.55
Movielene 201--1
0.50
0.50
0.50
0.45
0.45
0.40
15.00
MC-2
0.40
0:35
0.30
0.006
0.008
0,010
0.012
0.004
0.005
0.006
0.007
0.0000 000.0000000000000 00200.00222 00300.0035
0.00
0.30
0.34
Nediix-20M
N4000
0.26
0.37
0.28
0.24
534E0
0.35
56.27
65080
0.26
0.22
0.002
0.004
0.008
0.010
0.002
0.204
3,0000
0.0005
0,0010
0.0015
0,0020
Pair
955
Figure 1: Comparison of accuracy and MAE fairness at Kop-K on testing set.
5.3 Results
As shown in Figure] 1 we evaluate a total of 7 methods on three datasets al varying top-K lengths (K=50, 100, 200). In each sub-figure, the x-axis represents top-K exposure disparity (the lower, the better), and the y-axis represents top-K NDCG (the higher, the better). Methods are labeled with different colors. The performance of each method changes when the trade-off parameter varies. The trade-off parameter in our framework is the hyper-parane pare-prreeeter C' as defined in 12 across a range from 0 to very big number (e.g., 109) to balance the quality and the fairness of ranking. A higher C' value places more emphasis on fairness, and C corresponds to a color-blind ranking algorithm with no fairness constraints, such as K-SONG.
The results indicate that our two proposed methods, in particular, KSO-RED, consistently outperform the baseline models NG-DE, DELTR, DPR-RSP, DPR-REO, and Robust across various top-K results. That is, under the same fairness loss, our methods achieve the highest NDCG accuracy. Specifically, NG-DE demonstrates superior performance in terms of NDCG compared to DELTR in most scenarios, highlighting that K-SONG achieves better NDCG ranking performance than ListNet, particularly on the two MovicLens datasets. Furthermore, our SO-RED generally performs better than DELTR, which shares the same training objective, and NG-DE, validating the effectiveness of our optimization strategy. Notably, our KSO-RED, which directly focuses on top-K optimization, exhibits outstanding performance at shorter top- lengths N NDCG@K 100, 200), highlighting its proficiency in optimizing fairness within top-K ranking. More details are in Appendix F.
11

---

Published in Transactions on Machine Learning Research (09/2025)
5.4 Ablation Study
As shown in Figure 2 conducted an ablation study to investigale the impact of the averaging parameter in the SO-RED and KSO-RED models to investigate its impact on the trade-off between accuracy and fairness loss (MAE). For simplicity in this analysis, we set all averaging parameters "1 = 7/2 = a = y to the same value. We conduct experiments on the Mafteeenss22M--- dataset, varying y from 0.2 to 1.0. The results reveal that a lower 7 value among {0.2, 0.6, 1.0} (stronger moving average ratio) leads a more equitable balance between accuracy and fairness, evidenced by the top-K NDCG-Fairness metrics. This trend underscores the importance of the averaging parameter in optimizing the trade-off between accuracy and fairness in recommendation systems.
9.000
0.56
0.58
0.955
3.54
0.500
NDCG
0.50
0.52
K3D RED (y0.2))
0.52
0.54
0.500
0.575
0.585
KS3-tED (--0.2)
KSO 480 (x-0
1.01
3.48
RED (y 41
0.50
R2D (ym2.))
0.570
0y-1-21
0.46
NG-FF (yr1)
ago 1y-1.01
50 RED (ym1.3)
NG-DF [yr1)
0.365
5C-RED (y-1.0)
NO.00 ryei)
0.44
0.002
0.004
0,005
0.000
0.010
0.012 0.016
0.000 0,001 0,002 0,003 0.004 0,005 0.006 0.007
0.0015 0.0030 0.001500.000
3.0030...0000
Fair
loss
Fair
(a) MovieLenn-220M HOM-H-K50
(b) MovieLens-20N HoL.-eeenss-----K-IIII H-0000
(c) MovieLens-20MM 20M0H-2000
Figure Comparison of different 7 values on the MovieLeens-20M-H dataset.
6 Limitations and Future Work
First, our framework requires tuning several hyperparameters, such as the averaging parameters (1o, 71 79: 71, 75), learning rate parameters (7oo.)). and the smoothing parameters (71, Second, our current formulation assumes binary protected attributes (e.g., minority majority groups). Extending the framework to multi-group settings with more than two protected groups would require modifications to the fairness constraint formulations in g9 and corresponding algorithmic adjustments. Third, our current framework primarily focuses on optimizing exposure parity. Future work could address these limitations by incorporating adaptive hyperparameter selection mothods, generalizing the approach to handle multi- ple protected groups, and exploring additional fairness metrics beyond exposure disparity to broaden the framework's applicability.
7 Conclusion
We propose novel learning-to-rank framework that addressos the issues of incqualities in top-K positions training time. We develop an efficient stochastic optimization algorithm KSO-RED with provable convergence to optimize a top-K ranking that achieves both high quality and minimized exposure disparity. Extensive experiments demonstrate that our method outperforms existing methods. Our work contributes to the development of more equitable and unbiased ranking and recommendation systems.
Acknowledgments
This work supported in part by the NSF under Grant No. 1943486, No. 2246757 and No. 2147253.
12

---

Published in Transactions on Machine Learning Research (09/2025)
References
Himan Abdollahpouri, Robin Burke, and Bamshad Mobasher. Controlling popularity bias in learning-to-ranl recommendation. In Proceedings of the eleventh ACM conference on recommender systems, pp. 42 46, 2017.
Abolfazl Asudch, HV Jagadish, Julia Stoyanovich, and Gautam Das. Designing fair ranking schemes. In Proc. of the ACM International Conference on Information and Knowledge Management (CIKM), pp. 1276,
2019.
Henry C Bendekgey and Erik Sudderth. Scalable and stable surrogates for flexible classifiers with fairness constraints. Advances in Neurol Information Processing Systems, 34:30023 30036, 2021.
James Bennett, Stan Lanning, et al. The netflix prize. In Proceedings of KDD cup and workshop, volume 2007, pp. 35. New York, 2007.
Alex Beutel, Jilin Chen, Tulsee Doshi, Hai Qian, Li Wei, Yi Wu, Lukasz Heldt, Zhe Zhao, Lichan Hong, Ed H Chi, et al. Fairness in recommendation ranking through pairwise comparisons. In Proc. of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pp. 2212 2220, 2019.
Asia J Biega, Krishna P Gummadi, and Gerhard Weikum. Equity of attention: Amortizing individual fairness in rankings. In Proc. of the International ACM SIGIR Conference on Research and Development Information Retrieval, pp. 405 405-414, 2018.
Zhe Cao, Tao Qin, Tin Liu, Ming-Fem Tsai, and Hang Li. Learning to rank: from pairwise approach to listwise approach. In Proceedings of the 24th international conference Machine leorning, pp. 129 136, 2007.
L Elisa Celis, Damian Straszak, and Nisheeth K Vishnoi. Ranking with fairness constraints. arXiv preprint arXiu: 704.00444, 2017.
Diptarka Chakraborty, Syamantak Das, Arindam Khan, and Aditya Subramanian. Fair rank aggregation. Advances in Neural Information Processing Systems, 35:23965 23978, 2022.
Linda Darling- Hammond. Inequality in teaching and schooling: How opportunity is rationed to students of color in america. BD Smedley, AY Stith, L. Colburn, C. H. Evans (Eds.), The right thing to do The smart thing to do, pp. 208 233, 2001.
A Alessandro Fabris, Gianmaria Silvello, Gian Antonio Susto, and Asia J Biega. Pairwise fairness in ranking dissatisfaction measure. In Proceedings of the Sizteenth ACM International Conference on Web Search and Data Mining, pp. 931-939, 2023.
Ruoyuan Gao, Yingqiang Ge, and Chirag Shah. Fair: Fairness-aware information retrieval evaluation. Journal of the Association for Information Science and Technology, (3110):1461-14 2022.
Saced Ghadimi and Mengdi Wang. Approximation methods for bilevel programming. arXiv preprint arXiv:1802.0222 2018.
Sruthi Gorantla, Eshaan Bhansali, Amit Deshpande, and Anand Louis. Optimizing Bearning-to-rand models for ex-post fair relevance. In Proceedings of the 4 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1525-1534 2024.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In Proc. of the Annual Conference Neurol Information Processing Systems (NewrIPS), pp. 3315 3323, 2016.
F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (ttis), 5(4):1-19, 2015.
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural collaborative filtering Tn Proc. of the International World Wide Web Conference (WWW), pp. 173 172, 2017.
13

---

Published in Transactions on Machine Learning Research (09/2025)
Md Mouimul Islam, Mahsa Asadi, and Senjuti Basu Roy. Equitable top-k results for long tail data. Proceedings of the ACM on Monagement of Dala, 144)11-24, 2023.
James Kotary, Ferdinando Fioretto, Pascal Van Hentenryck, and Ziwei Zhu. End-to-end learning for fair ranking systems. In Proceedings of the ACM Web Conference 2022, pp. 3520-3530, 2022.
Juhi Kulshrestha, Motahhare Eslami, Johnnatan Messias, Muhammad Bilal Zafar, Saptarshi Ghosh, Krishna P Gummadi, and Karrie Karahalios. Quantifying search bias: Investigating sources of bias for political searches in social media. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, pp. 417 432, 2017.
Wonbin Kweon, ScongKu Kang, Sanghwan Jang, and Hwanjo Yu. Top-personalized-k recommendation. In Proceedings of the ACM Web Conference 2024, pp. 3388 3399, 2024.
Preethi Lahoti, Krishna Gummadi, and Gerhard Weikum. ifair: Learning individually fair data represen- tations for algorithmic decision making, In 2019 feee 35th international conference on data engineering (iede), pp. 1334-1345. IEEE, 2019.
Anay Mehrotra and Nisheeth Vishnoi. Pair ranking with noisy protected attributes. Advances in Neural Information Processing Systems, 35:31711-317211 2022.
Omid Memarrast, Ashkan Rezaci, Rizal Fathony, and Brian Ziebart. Fairness for robust learning to rank. In Pacific-Ase Asia Conference on Knowledge Discovery and Data Mining, pp. 544 556. Springer, 2023.
George Mohler, Rajeov Raje, Jeremy Carter, Matthew Valasik, and Jeffrey Brantingham. A penalized likelihood method for balancing accuracy and fairness in predictive policing. In Proc. of the IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 2454 2459, 2018.
Harikrishna Narasimhan, Andrew Cotter, Maya Gupta, and Serena Wang. Pairwise fairness for ranking and regression. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 5248 5255, 2020.
Qi Qi, Youzhi Luo, Zhao Xu, Shuiwang Ji, and Tianbao Yang. Stochastic optimization of area under precision-recall curve for deep learning with provable convergence. In Advances in Neural Information Processing Systems, volume 34, 2021.
Tao Qin, Tic- Yan Liu, and Hang Li. A general approximation framework for direct optimization of information retrieval measures. Information retrieval, 13(4):375 010337-9777770 2010.
Zi-Hao Qiu, Quanqi Hu, Yongjian Zhong, Lijun Zhang, and Tianbao Yang. Large-scale stochastic optimization of ndeg surrogates for deep learning with provable convergence. arXiv preprint arXiw:2002.12/ 2022.
Cynthia Rudin. The p-norm push: A simple convex ranking algorithm that concentrates at the top of the list. 2009.
Jin Shang, Mingxuan Sun, and Nina SN Lam. List-wise fairness criterion for point processes. In Proc. of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pp. 1948 1958, 2020.
Ashudeep Singh and Thorsten Joachims. Fairness of exposure in rankings. In Proc. of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pp. 2219 2228, 2018.
Ashudeep Singh and Thorsten Joachims. Policy learning for fairness in ranking. In Proc. of the Annual Conference on Neural Information Processing Systems (NTPS), 2019.
Ali Vardasbi, Maarten de Rijke, Fernando Diaz, and Mostafa Dehghani. The impact of group membership bias on the quality and fairness of exposure in ranking. In Proceedings of the 47th International A ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1514 1524, 2024.
14

---

Published in Transactions on Machine Learning Research (09/2025)
Chenyang Wang, Min Zhang, Weizhi Ma, Yiqun Liu, and Shaoping Ma. Make it chorus: knowledge-and time-aware item modeling for sequential recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pp. 109 118, 2020.
Yuan Wang, Zhiqiang Tao, and Yi Fang. A unified meta-learning framework for fair ranking with curriculum learning. IEEE Transactions Knowledge and Data Engineering. 2024.
Mingrui Wu, Yi Chang, Zhaohui Zheng, and Hongyuan Zha. Smoothing deg for learning to rank: novel approach using smoothed hinge functions. In Proceedings of the 18th ACM conference on Information and knowledge management, pp. 1923 1926, 2009.
Ke Yang and Julia Stoyanovich. Measuring fairness in ranked outputs. In Proceedings of the 29th International Conference on Scientific and Statisticol Database Management, pp. 1 6, 2017.
Tao Yang, Zhichao Xu, and Qingyao Ai. Vertical allocation-baser fair exposure amortizing in ranking. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region, pp. 234 244, 2023.
Wei Yao, Zhanke Zhou, Zhicong Li, Bo Han, and Yong Liu. Understanding fairness surrogate functions in algorithmic fairness. avxiv preprinl ur Xix:223300 11211, 2023.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proc. of the International World Wide Web Conference (WWW), pp. 1171-1180, 2017.
Meike Zehlike and Carlos Castillo. Reducing disparate exposure in ranking: A learning to rank approach. In Proceedings of the web conference 2020, pp. 2849 2855, 2020.
Meike Zehlike, Francesco Bonchi, Carlos Castillo, Sara Hajian, Mohamed Megahed, and Ricardo Bacza- Baeza-Yates Fa* ir: A fair top-k ranking algorithm. In Proc. of the ACM International Conference on Information and Knowledge Management (CIKM), pp. 1569-1378, 2017.
Meike Zehlike, Tom Sühr, Ricardo Bacza-Yates, Francesco Bonchi, Carlos Castillo, and Sara Hajian. Fair top-k ranking with multiple protected groups. Information processing & management, 59(1):102707, 2022.
Yikai Zhao, Wenchen Han, Zheng Zhong, Yinda Zhang, Tong Yang, and Bin Cui. Double-anonymous sketch: Achieving top-k-fairness for finding global top-k frequent items. Proceedings of the ACM Management of Data, 1(1):1-26, 2023.
Ziwei Zhu, Jianling Wang, and James Caverlee. Measuring and mitigating item under-recommendattio bias in personalized ranking systems. In Proceedings of the 43rd international ACM SIGIR conference on research and development in information relrieval, pp. 449 458, 2020.
Ziwei Zhu, Jingu Kim, Trung Nguyen, Aish Fenton, and James Caverlee. Fairness among new items in cold start recommender systems. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 767-776, 2021.
15

---

Published in Transactions on Machine Learning Research (09/2023)
0.40
0.20
0.28
0.38
Notfil-20M
0.25
0.37
0.20
2043
0.36
0.24
0.28
0.35
0.34
loss INSEI
(a) K 50
(b) K 100
(c) K 200
Figure 3: Comparison of accuracy and fairness loss (MSE) at top- on the Netflix-0M dataset.
A Experimental Details
For model training and evaluation, we adopt conventional split of training, validation, and test as in Wang et al. (2020); : Qiu et al. (2022). Specifically, we adopt the testing protocol by sampling 5 rated and 300 unrated itemss per user to evaluate the NDCG and fairness metrics, whereas the training employs a similar protocol as in Wang et al. (2020). For all methods, we sample the same batches of queries users and the mixture of relevant and irrelevant items per query during each iteration to ensure a fair comparison.
The model will be pre-trained, and the resulting warm-up model will be employed by K-SONG, NG-DE, DELTR, SO-RED, and KSO-RED for fair comparison. In our fine-tuning process, to ensure consistency and fairness in model performance comparisons, the optimal pppee-paraetter settings are based on established optimal values from K-SONG. We employ the NeuMF model (He et al.) 2017) as our primary predictive function due to its proven efficacy in recommendation lasks. Initially, the model undergoes a 20-epoch pre-training with a learning rate of 0.001 and a batch size of 256. Subsequent fine-tuning reinitializes the last layer, adjusting the learning rate to 0.0004 and applying a weight decay of x 10-7 over 120 epochs with a learning rate reduction by a factor of 0.25 after 60 epochs. To streamline our experiment, we leverage the tuned results from K-SONG and adopt the best value for the hyper-parameter 70, which is set to 0.3, serving as the base model 0.6, 1).
The averaging parameters 71, 72, and 7/ are tuned from the set of {0.2,
B Parameters for Baseline Models
For baseline models DELTR, DPR-RSP, DPR-REO, and Robust we maintain the same hyper-parameter settings used in the original papers to ensure optimal performance and a fair comparison. Specifically, in DELTR (Zehlike & Castillo, 2020) we tune the fairness hyper-parameter (7) from 0 to 100M, covering their tuming range. For DPR-RSP and DPR-REO (Zhu et al.) 2020), we select 1K and 11K for the fairness hyper-parameter, as mentioned in their respective papers. In addition, in order to adapt their code to our experimental setup, align the training/teen /ttstt//tttggggg data with ours, and incorporate our evaluation metrics, we adjust certain data structures, such as query-item lists and batch processing of predictions to ensure their code could process large datasets effectively. All other hyper-parameters are kept the same as in the original papers. For Robust (Memarrast et al., 2023), we adapt our dataset and code to ensure fair and consistent evaluation. Specifically, construct query representations by generating matrices for each user and item, then concatenating user-item pair (including user, item, sensitive attributes, and the corresponding matricos) as a query for training. Given the O(m x n) lime and space complexity of this method where m is the number of users and n the number of items, directly applying it to large-scale datasets such as MovieLens20M and Netflix20M is impractical. To address this, we sample the training datasot, reducing the complexity to O(m c), where e is a fixed number of sampled items per user. This modification ensures scalability while maintaining the integrity of the original method. Importantly, retain the full test dataset. for evaluation,
16

---

Published in Transactions on Machine Learning Research (09/2025)
110200
10000
83170
83170
86784
787
DOUSE
JILZA
107228
DIZZ
97216
9/216
18137
151T
34692
3452
53373
5373
134251
14251
126312
12312
115430
115430
Item Rank
Item Rank
(a) Before incorporating fairness
(b) After incorporating fairness
Figure 4: Ranked item list for the most unfair 0.02% of rankings in the MovieLens-20M-HH datasct.
following the original code structure ensure a fair comparison. All other hyper-paramcters are kept as specified in the original paper.
C Fairness (MSE) NDCG Results
This Fig. presents the corresponding fair loss (MSE) and NDCG on the Netflix-20M. dataset. Similar the MAE results shown in the main text, KSO-RED consistently outperforms the baseline models. The MSE metric provides an alternative perspective the fairness performance by penalizing larger deviations more heavily than MAE.
D Computing Resources for the Experiment
Our main experiments were conducted on system equipped with the following hardware:
24-core Intel CPU
96 GB of memory
1 NVIDIA V100S GPU (with 32 GB memory)
1.5 TB SSD drive
E
Experiment Statistical Significance
Table 1 shows the NDCG and fair loss values with standard deviations for various methods across different values of lopK = 50, 100, 200 and configurations C' = 0, 100k. The values in parentheses represent the standard deviations for each corresponding metric, providing insight into the consistency of the performance measures.
F Ranking Visualization
In addition to these quantitative results, we present a visualization of how our method adjusts rankings achieve fairer outcomes, particularly on the most biased 0.02% of rankings for users in the MovieLens-20M-EH dataset. Figures 4h and 4p illustrate the impact of incorporating the fairness objective during training with
17

---

Published in Transactions on Machine Learning Research (09/2025)
K
50
100
200
C=0
NDCG
0.5769 (0.0521)
0.5910 (0.0465)
0.5973 (0.0430)
KSO-RED
Fair loss
0.0139 (0.0001)
0.0069 (0.0000)
0.0034 (0.0000)
C=100K
NDCG 0.5719
(0.0541)
0.5847 (0.0409)
0.5901
(0.0464)
Fair loss
0.0095 (0.0000)
0.0042 (0.0000)
0.0012 (0.0000)
C=0
NDCG 0.5769
(0.0521)
0.5910 (0.0465)
0.5973 (0.0430)
SO-RED
Fair loss C=100K
0.0139 (0.0001)
0.0069 (0.0000)
0.0034 (0.0000)
NDCG
0.5740
(0.0528)
0.5886 (0.0471)
0.5953 (0.0442)
Fair loss
0.0103 (0.0000)
0.0063 (0.0000)
0.0012 (0.0000)
K-SONG
NDCG
0.5769 (0.0521)
0.5910 (0.0465)
0.5973 (0.0430)
Fair loss
0.0139 (0.0001)
0.0069 (0.0000)
0.0034 (0.0000)
C=0
NDCG 0.5769
(0.0521) )
0.5910 (0.0465)
0.5973 (0.0430)
NG-DE
Fair loss
0.0139 (0.0001)
0.0069 (0.0000)
0.0034 (0.0000)
C=100K
NDCG
0.5707 (0.0553)
0.5864 (0.0491)
0.5914 (0.0465)
Fair loss
0.0135 (0.0001)
0.0048 (0.0000)
0.0011 (0.0000)
C-0
NDCG 0.5496
(0.0551) )
0.5531 (0.0501)
0.5581 (0.0481)
DELTR
Fair loss
0.0136 (0.0001)
0.0041 (0.0000)
0.0015 (0.0000)
C=100K
NDCG
0.5452 (0.0584)
0.5656 (0.0517)
0.5591 (0.0484)
Fair loss
0.0114 (0.0000)
0.0042 (0.0000)
0.0008 (0.0000)
Table 1: NDCG and MAE fair loss values with standard deviations for various methods.
our KSO-RED algorithm. In both figures, each row corresponds to the ranking of 305 items for a specific user, where red pixels represent items from the minority group (c.g., horror movies), and green pixels represent. items from the majority group (e.g., non-horror movies).
The two figures show test set results generated by KSO-RED, trained with two different values of C. When C 0 (Figure 4),, the rankings are color-blind, with no fairness considerations. As C increases (Figure AD), the emphasis on fairness becomes more pronounced. This visualization underscores the effectiveness of KSO-RED in adjusting rankings to ensure a more equitable distribution of exposure, particularly when higher fairness constraints are applied.
Method
Training Time (minutes)
DELTR
84
NG-DE
110
SO-RED
126
KSO-RED and K-SONG
127
Robust
1,522
DPR-RSP/DPR-RIS
>4,320 (early stop)
Table 2: Training time comparison
18

---

Published in Transactions on Machine Learning Research (09/2025)
G Computational Efficiency Analysis
We evaluate the computational efficiency of all methods on the MovieLens-20M-E datasct example. All experiments are conducted on the same hardware configuration (24-core Intel CPU, 96GB memory, NVIDIA V100S GPU with 32GB memory). KSO-RED, SO-RED, K-SONG, DELTR, and NG-DE are implemented under our framework with 120 training epochs, while Robust uses its original framework with 5 epochs.
Table 2 presents the training time comparison across different methods. Our proposed KSO-RED demonstrates competitive computational efficiency while achieving superior fairness-accuracy trade-offs.
19


Reviews:
[{"review_id": "a4gxqQTiSC", "reviewer": "Reviewer_cXtn", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1.\tThe paper addresses an underexplored problem, ensuring fairness specifically in the top K-ranked items by introducing a principled top K exposure disparity metric.\n\n2.\tThe proposed bilevel optimization with a smooth surrogate and the KSO RED optimizer is both theoretically grounded (with convergence guarantees) and practically scalable to large real-world datasets.\n\nWeaknesses:\n\n1.\tInsufficient Practical Motivation: While the problem is well formulated, the paper lacks a compelling justification for why top K fairness is particularly critical in practice. A stronger empirical or application-driven motivation would strengthen the case.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces an in-processing learning-to-rank framework that targets fairness within the top K positions by defining a top K exposure disparity metric and jointly optimizing for both relevance and fairness. To make the hard top K selection differentiable, the authors replace the non-differentiable indicator with a smooth surrogate and formulate a bilevel objective, resulting in an efficient and trainable approximation. They develop a stochastic compositional optimizer with momentum, KSO-RED, and provide a convergence guarantee to an $\\epsilon$-stationary point under standard smoothness and bounded-variance assumptions, building on techniques from K-SONG. The approach is scalable, does not require group labels at test time, and the optimization framework can be extended to other list-wise fairness notions.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Could the authors elaborate on why KSO‑RED and SO‑RED perform so similarly on MovieLens‑20M‑H? Clarifying when top‑$K$ fairness meaningfully differs from standard (full‑list) exposure fairness, and thus when the proposed method should be preferred, would help readers decide which approach to use.\n\n2. Equation (9) appears to state a fairness constraint, not a loss. Should the fairness loss be defined as the absolute difference between the group exposures instead? Please clarify the exact loss used in Figure 1.\n\n3. I believe a $\\lambda$ term is missing in the first term of Equation (10); without it, the expression seems inconsistent with the surrounding derivation. Please verify and correct.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4FoRZmVev0", "reviewer": "Reviewer_2bKo", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The claims made in the paper are generally well supported by both theoretical derivations and experimental evidence. The central claim—that the proposed KSO-RED method effectively reduces top-$K$ exposure disparity while maintaining ranking accuracy—is substantiated by empirical studies across several datasets (MovieLens-20M-H, MovieLens-20M-D, Netflix-20M)\n- The methods are clearly described, with the progression from classical NDCG and exposure fairness to the top-$K$ extension articulated step-by-step. \n\nWeaknesses\n- Proof details of Theorem 4.2 are omitted and mostly referenced to prior work (Qiu et al., 2022). The Algorithm and convergence result mainly follow a standard approach, and the theory has been well-developed in previous work. \n- For broader impact, evaluation on, for instance, real-world demographic attributes or additional public datasets (e.g., web search, recruitment) may be considered.\n- More ablation study is needed for key hyperparameters.\n\nI don't have many major comments on this paper. It appears to follow a standard methodology, and the core idea seems straightforward. For example, the optimization objective, Top-K selection operator transformation, gradient derivations, and theoretical results align closely with established approaches. While these results are correct, they don't strike me as particularly surprising. The primary contribution appears to be the adaptation and connection of existing ideas from previous work to address the Top-K ranking fairness problem. I'm somewhat uncertain about the overall significance of this specific approach within the field.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a list-wise learning-to-rank framework to improve fairness at the top-$K$ positions in ranking tasks, motivated by the real-world importance of resource allocation and exposure disparities for protected groups. The main contributions are (i) developing a new top-$K$ exposure disparity measure extending classical fairness notions, (ii) integrating this measure as a regularizer within a differentiable and scalable optimization framework using stochastic approximation, and (iii) demonstrating via experiments on standard recommendation datasets that the proposed approach can significantly reduce unfairness at the top-$K$ positions while retaining high ranking accuracy.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- You may consider including additional baselines (especially modern transformer-based ranking architectures and recent fairness-aware methods) and evaluating at least one other domain (e.g., search, hiring, credit lending) to bolster the claimed generality of the approach.\n- The use of a sigmoid surrogate to replace the indicator function in the fairness literature has been investigated in [1-2]. You may consider citing these papers in appropriate positions, such as on page 5.\n- The format of the pseudocode is incorrect.\n- You don't need to use two lines here in Eq. (9).\n\n[1] Scalable and Stable Surrogates for Flexible Classifiers with Fairness Constraints. NeurIPS 2021.  \n[2] Understanding Fairness Surrogate Functions in Algorithmic Fairness. TMLR 2024.", "broader_impact_concerns": "NA.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "n6aRJK4Z7j", "reviewer": "Reviewer_6jWS", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The paper is easy to follow.\n\n2. The paper introduces a list-wise learning-to-rank framework that explicitly addresses fairness in top-k rankings during training, an area that has received limited attention in existing literature.\n\n3. The method is rigorously validated on multiple popular benchmark datasets, demonstrating consistent improvements in both ranking accuracy and fairness compared to existing approaches.\n\n4. The proposed method includes theoretical guarantees. \n\nWeaknesses:\n\n1. The optimization algorithm appears overly complex. It would be beneficial to include pseudocode in the main paper to improve clarity and reproducibility. Will authors release the code in the future?\n\n2. Theorem 4.2 is directly taken from Theorem 2 in Qiu et al. (2022), which diminishes the novelty of the theoretical contribution.\n\n3. The experiments are conducted solely on tabular data, leaving the effectiveness and adaptability of the proposed approach to other data modalities (e.g., image data) unclear.\n\n4. Given the complexity of the algorithm, it would be helpful to report the training time to assess its computational efficiency.\n\n5. A discussion on the limitations of the proposed approach would strengthen the paper.\n\n6. While Equations (9) and (10) are non-differentiable, this does not preclude their direct incorporation into the learning objective. I recommend adding experiments to demonstrate the advantages of Equation (13) over Equations (9) and (10).\n\n7. The experimental setup relies solely on a classic deep neural network (NeuMF). The authors are encouraged to evaluate their method using more advanced models, such as transformer-based architectures, to better assess its generalizability.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This submission addresses a critical yet underexplored issue in fairness-aware ranking: mitigating disparities in top-k exposure, where the top-ranked items significantly influence real-world decisions, such as resource allocation or emergency response. Unlike most prior work that ensures average group-level fairness across the full ranking, this paper focuses on inequities that persist at the top-k positions, where visibility and impact are greatest. The authors introduce a novel top-k exposure disparity metric, extending traditional exposure fairness to more accurately reflect real-world ranking use cases. To address the computational intractability of non-differentiable top-k selection for large item sets, the authors formulate a differentiable surrogate objective and design an efficient stochastic optimization algorithm that achieves both ranking quality and fairness. Extensive experiments demonstrate that the proposed approach outperforms existing fairness-aware ranking methods in terms of accuracy and fairness at the top-k level, providing both theoretical and empirical contributions to the study of equitable ranking systems.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See Weaknesses", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)