Preference-Based Dynamic Ranking Structure Recognition
Authors: Nan Lu
Jian Shi
Xinyu Tian
Venue: NeurIPS 2025 poster ()
Keywords: dynamic ranking
Bradley-Terry model
structural change detection
preference-based data
clustering
Primary area: general_machine_learning

Abstract: Preference-based data often appear complex and noisy but may conceal underlying homogeneous structures. This paper introduces a novel framework of ranking structure recognition for preference-based data. We first develop an approach to identify dynamic ranking groups by incorporating temporal penalties into a spectral estimation for the celebrated Bradley-Terry model. To detect structural changes, we introduce an innovative objective function and present a practicable algorithm based on dynamic programming. Theoretically, we establish the consistency of ranking group recognition by exploiting properties of a random 'design matrix' induced by a reversible Markov chain. We also tailor a group inverse technique to quantify the uncertainty in item ability estimates. Additionally, we prove the consistency of  structure change recognition, ensuring the robustness of the proposed framework. Experiments on both synthetic and real-world datasets demonstrate the practical utility and interpretability of our approach.

Preference-Based Dynamic Ranking Structure Recognition
Nan Lu n1u990078gmail.. com
Jian Shi jshi@iss wwe.ssssc... ac
Xin-Yu Tian tianx@umn edu
1 State Key Laboratory of Mathematical Sciences, Academy Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, China Schooo of Statistics, University of Minnesota, Minneapolis, USA
Abstract
Preference-based data often appear complex and noisy but may conceal underlying homogeneous structures. This paper introduces novel framework of ranking structure recognition for preference-bare based data. We first develop an approach to identify dynamic ranking groups by incorporating temporal penalties into spectral estimation for the celebrated Bradley-Terry model. To detect structural changes, we introduce an innovative objective function and present a practicable algorithm based dynamic programming. Theoretically, we establish the consistency of ranking group recognition by exploiting properties of a random 'design matrix" induced by reversible Markov chain. We also tailor group inverse technique to quantify the uncertainty in item ability estimates. Additionally, we prove the consistency structure change recognition, ensuring the robustness of the proposed framework. Experiments on both synthetic and real-world datasets demonstrate the practical utility and interpretability of our approach. |]
Introduction
Preference-based data, where observations arise from pairwise or groupwise comparisons rather than absolute measurements, is prevalent across various domains. This form of data naturally appears in applications such economics M, online recommendations ? and sports analytics (3]. In addition, the use of preference-bast data in reinforcement learning from human feedback (RLHF) has led to significant improvements the performance of large language models One major advantage of preference-basee based data lies in its case of collection, as it is often more intuitive to express relative preferences rather than assign absolute scores. Many widely used datasets are inherently preference-hasee making their effective modeling and analysis a pivotal research focus. To handle such data, the celebrated Bradley- Terry model (5) is widely used for inferring latent preference scores from pairwise comparisons. This model and its extensions have been extensively studied; see [6-9.
Ranking serves as crucial tool for summarizing preference-based data, providing interpretable outcomes that facilitate decision-making in various fields. It has broad applications, including the evaluation of sports teams [10], institutions [[]] 12] recommendation systems [13 14), financial
markets [5] [6]] and bioinformatics [8]. By leveraging ranking positions, comparison results
enable the identification of nop-performing entities [99 20 while also uncovering underlying trends
and patterns 22) Items to be ranked often possess latent structures due to population homogeneity, which can be reflected in phenomena such as circular comparison results. Moreover, even slight
Corresppnndigg author. Code available
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

modification in comparisons can result in different rank [23], highlighting the importance of grouped rankings. Grouping similar items can enhance robustness and reduce sensitivity to specific comparisons. For example, group rankings recognize homogeneous entities to improve interpretability and predictive accuracy [TO] 24]. In the context of university rankings, Soh [25] argues that minor differences in scores should be ignored and that similar institutions should be assigned to the same
group. Given vire-ng comparison results, we aim to address the following questions:
Which items exhibit similar behavior and can be categorized into the same group during a specific period?
What is the ranking order of these groups at a particular time point?
When considering the temporal dimension, we often encounter situations where item groups evolve over time. For example, in basketball, player trades and coaching changes can significantly affect a team's 's performance, potentially elevating it to higher ranking tier. Similarly, the share prices of certain companies may surge due to emerging political, technological, or market factors. These rapidly evolving scenarios underscore the importance of detecting structural change points for long- term analysis. In this context, the term group change refers to shifts in group membership over time, which are crucial for accurate analysis in the dynamic grouping problem. In this work, we take closer look at changes in group structures and aim to address another critical question:
When does the underlying cluster structure experience significant changes?
There have been several studies on grouping methods for ranking problems in the BT model. Masarotto and Varin [0] first apply the fused lasso penalty to the maximum likelihood estima- tion. Vana et al. 26 utilize similar method for journal meta-ranking, and Jeon and Choi [27) extend it to the Luce model. Tian and Shi (28) further consider the problem using the spectral method. However, it is worth noting that all these grouping methods are designed for static situations. In practice, the latent abilities of sports players and institutions may vary over time. Treating data as if it were all collected simultaneously can lead to misleading results. For example, a player in his rising period and another in a declining period may exhibit similar average performances in game season, but they should not be classified as the same. Hence, this paper concentrates on the simultaneous ranking and grouping problem for the dynamic scenario. Li et al. [3] introduce a segmented static BT model, focusing on detecting the change points of score variation. In contrast, our approach allows scores to vary continuously, and our emphasis lies in recognizing the underlying structure and the changes in clustered groups over time.
We summarize our major contributions as follows.
An innovative framework for ranking group recognition. Though item abilities can be modeled using continuous functions, the ranking positions are discrete functionals of the latent abilities, posing challenges in identifying their structure. To address this, we propose novel workflow that nests recovering dynamic ranking groups in group change recognition.
A generally applicable structure change detection method. Some works study the clustering problem of different items 10 26 27), while few works consider the abrupt changes of item abilities 3. To the best of our knowledge, we are the first to consider the ranking structure changes for the BT model. We carefully design an integrated objective function, which possesses separable properties, making it permissible to develop an efficient algorithm based on dynamic programming.
Theoretical results on recognition consistency and estimator uncertainty. We characterize conditions of the variability within groups that ensure the consistency recognition of groups and establish the structure recognition consistency. We quantify the uncertainty of item ability estimators using an innovative group inverse technique.
Ranking results with enhanced interpretability and improved accuracy. Our method provides concise ranking results and group change information. The structured ranking results enable the identification of homogeneous items and dynamic group changes, which provide insights into the underlying structure. Simulation results also demonstrate that the grouping method integrates data effectively, yielding improved estimation accuracy.
Notations We write an S bn or an = Obb,) if there exists a constant e > 0 such that a, S cb, for all n. We denote on x b, if bn S an and an S bn. Besides, we write a, = o(b,)) On < b, lim an
an/b, /b, = O. We denote by [n]] = 1..... n for any positive integer n. We let I represent

---

Recovering underlying groups for certain interval
Group change detection solved utilizing dynamic programming
Observations: preference results from pairwise comparisons
min 7
J.
[(((()
P (t)
Σ L ZPP
G
+u
Σ
R - 7 7 Ii Ruthar
[mweress]
Sub problem un / Grouping on mo.teel
You Itt-lares]
Level
Sub-problem Grouping on
Level
Figure 1: Workflow of the proposed method for dynamic ranking structure recognition. The left panel depicts the data format, representing preference outcomes obtained from pairwise comparisons. The middle panel illustrates the score evolution each group over a given interval. The right panel shows the structure detection procedure, where: (1) each node can be decomposed into subproblems represented by its child nodes; (2) each child node corresponds to a subproblem and a grouping problem intervals; (3) subproblems at level C reuse results from nodes preceding their parent node at level B, thereby avoiding redundant computation.
the identity matrix, and let e,, be an 1 vector with each element equal 1. represents the vector or matrix composed entirely of zeros. For a vector v, || denotes the fo-nrmm. We let llf (Jo /t2 where f(t) is function on [0,
Ranking Structure Recognition
2.1 Dynamic Bradley- Terry Model
In dynamic scenario, we observe pairwise comparison results among items, denoted as y
(yn (4x), i.j E [m]. lx E T. The scalar (1)(() represent the comparison result at time point between items i and j. where (1)(() represents that item i wins over item and Vty(() 0
indicates the opposite. We assume that the elements of y are independent, and the comparison time Tiş of (i,j) pair is uniformly distributed over [0, T}. The Bradley-Terry model assigns positive (I), mi)(),.... ( to items, and presumes Vig (4) Bemoulli (!)),
scores π T=(() (t)
where viy (t) π (t)/(T;; + (t)) [5]. Intuitively. taking π. (t) as the ability of item i, y (t)
(t))
represents the winning rate item i. Notice that the BT model is invariant under the scaling of the scores, set Σ (t) for all t E [0 to obtain unique representation.
straightforward approach to estimate the BT model the maximum likelihood estimator pursuit of a computationally efficient solution, we opt for the spectral- based solver 66) 22 Negahban al. 6 present an insightful perspective of the spectral method, establishing connection between the pairwise comparison results and the transition of Markov chain. Specifically, by letting the nodes of a graph represent the items, and assigning the transformation probability from node to based on the frequency of item losing to j. it is proven that the stationary distribution of this random walk corresponds to the items" abilities π.. For more detailed understanding, we recommend referring to Negahban et al. [6]
29 30
In
We adopt the kernel-based estimator. Let (n(t, KK (#=²) where K(-) is the kernel function and h is the bandwidth. The transformation probability matrix P(t) is formulated
E Enyry (a.t))
ifi
(t(
n
Een Kyle(e))
E sit (t)
if
Then we have the consistent estimator #(), which a stationary distribution of the Markov chain deduced by the stochastic matrix P 22
3

---

2.2 Recognition of Dynamic Ranking Groups
We first consider recovering the ranking groups for a given interval this section, which cornerstone for analyzing the changes of ranking groups for a relatively longer time period as discussed in Section 2.3
Let B denote the number of groups. We represent these groups as G= G2, GB). forming partition of the set [n]. The items within the same group possess similar scores, while those from different groups have significant score differences. Formally, we have
81 == min min || 12l22. > max max || тÿ 12,T ke IJEG
iEG, kit JEG:
Without loss of generality, we assume that maxfi: : i G+} < minfi: : E G.} for ls < I.
We then recover the partition of items and present the score estimations simultaneously. Since any finite-state time-home homogenoos Markov chain has at least one stationary distribution, we rewrite the estimator (to) the solution of the optimization problem, min [f= P (to)# l2 such that Σ F (ta) = 1. Setting À as tuning parameter, consider the following objective function.
/' (t(()) P (1)
min "
| T 1 IT T1+1ll2.r
(2.1)
s.t. (((()) = 1, k= 1,2......
Let /1, 42, .............................. fm be m equidistant time points in [0, 2]. We use the symbol with an item subscript, such as n,, to represent the vector corresponding to the m time points (t1), Te (f2)...... n. (t(t)))" Here, the parameter m is allowed to approach infinity, allowing for the approximation of the integral. To efficiently address the constrained optimization problem, we employ technical transformation, leading us an unconstrained form with a well-developed optimization algorithm. Define the n matrix
1
if i or i = n. if and = otherwise.
Qú
0
Let 0(t) ( -e,) en) and 0 Q(*(t) tew). Let ((t) (0 (t)...... (t) and 0 (O(t,)T,... ...., O(t,.) TTT Let 0" be the corresponding true value (with substituted by x²), and 0 denote the counterpart induced by म. We define X(t) (t) IIQ and Y(t)
PT (-))et))),- Then we have the optimization problem 2.1 reformulated as
min 0
Y
X0
12
0 2 1
(2.2)
xtm)t x_(()) (t) is the matrix X(t) with where X is the mn x xm 1) matrix diag(X ( its last column removed. represents the mnn x 1 vector, (Y(1)..... Y (T.)))) This transformation directly eliminates the constraints, reducing the optimization objective to a standard adaptive group lasso problem, which possesses efficient solutions. Having obtained the solution O, we can calculate
(1) = ---QQ---l0 ())()))))) + e.e. Let S = {i: 07 + 0). S {i: Ø. + 0) and B |S] + 1. use S to denote the estimated partition points of different groups. Specifically, we let S {0) US (n). Without loss of generality, we assume S is arranged in ascending order (if not, we simply reorder S), and 5, denotes the i-th element of S. The group estimation G (G,, G2. ...,.... G31 is obtained by
G2 = (i: Sp-1 < S.).
Remark 2.1. We note that approach and the theoretical justification presented below do not rigidly require that all items within a group possess identical scores. Instead, we establish framework in which items within a group exhibit similar behavior, rendering practically flexibility.
Remark 2.2. Though we originally have the fused term among different items as (2.1) the optimization objective has the same expression as the adaptive group lasso of a linear regression model in (22). is worth noting that the similarity is somehow superficial since the design matrix X is no longer deterministic, posing difficulties for theoretical analysis.

---

To mitigate the issue of shrinkage in large coefficients resulting from the penalization term, widely utilized approach is the refit procedure [2] This method entails re-estimating the coefficients after identifying the underlying structure. Upon obtaining the group estimation G, we employ the refit strategy in the following manner. For i, j E [B). define
Egea, Eigce, Enerinn 2112 (1.2)) Di,Eo, Eized, Z4 (x(EAt)
i
P Pay (t)
Eo4s P Gii (t),
if
We can obtain the stationary distribution क6 = ( kE(B of P((). Note that have assumed the score summation items to be I to eliminate the non con-uniqueness caused by rescaling. Therefore, the refit estimator for each item is
# (1)
i G
(2.3)
Enelinn G (t)
Remark 2.3. We use the absolute group size for normalization because, as stated in Section 2.1 we impose the constraint that the sum of item abilities equals 1, i.e. Σi= मे = 1, to ensure a unique representation. Since only the ratio between scores matters in the BT model, this constraint guarantees identifiability. Furthermore, when recovering the original scores after refitting, we still expect the normalized scores to satisfy Σx= /. = 1. At the same time, we need to preserve the score ratios between items from different groups, meaning that for E G and GR. / + 2/ / + TT.)) ). To satisfy both conditions simultancously. the normalization in equation 2..3
scaled by the absolute group size |Gx]-
Remark 2.4. The refit strategy is an optional part. Treating the comparison result of items in group as one actually compensates for more information, especially in cases where is large and M is small. We also point out that refitting induces better performance, as indicated in simulations.
2.3 Recognition of Group Changes
We then focus on detecting the change points of latent clusters over an extended period. Consider a scenario with time-correlated observations occurring within the interval j0, VJ. Still consider entities whose structure needs to be determined. There are / + 1 phases, where the items' groups remain unchanged within each phase and differ between adjacent phases. In a more formal mathematical form, 1et xtt) (z)
Zn.(t) represent the latent group of items at the time point t. There 17u < .... < M1+1 V such that z(t) & z(s) for f/-1 < < 7i < s <<> 1/++1+ i e [J] and =tt) = z (s) for 7)-1 < t, < Ts. i [J + +1. The unobservable structure change points betl belong to a preset candidate set feele Without loss of generality, let (E [t] be in increasing order. In practice, the candidate set may be selected based on practical considerations, such as dividing points between seasons in sports games or uniformly distributed time points.
are J + points
To detect changes in underlying groups, it is necessary to employ clustering methods within a subinterval I C [o, vj. We utilize the clustering method proposed in Section 2.2] for dynamic ranking, by simply substituting [0, T] with Z. Let G(Z) represent the estimated group corresponding to the true structure G(Z). With slight abuse of notations, let B(I) denote the model parameter estimations f E [m], I), and let ((I) {f(t), I} be the corresponding true (ey (o)( (t.de) We introduce the negative log- likeeiilloood function for Σt K((t.a))
values. Define (t) EM
(Ti,..., nh) time point
1(x, nnn
2
Σ (1.))
(t)(t) log(
J n.
(2.4)
π
which is a natural extension of the static case. Define the function ( ((e to measure the discrepancy between observed samples and the values expected under the grouping model.
(3(t),
Let represents {[su, $1),
: ($1, 82), ... (Sp: Spti]) with so O. 8pt1 V and (s:heet)) C {& heje being a list of increasing points. We use IG(I) to represent the estimated group number
5

---

and | to denote the interval length. We recover the change points of structures by considering the objective function:
min
EIG 12|P-
(2.5)
IEP
IEP
Intuitively, the first term evaluates the goodness of fit for the parameters, the second term is the penalty of groups and the last term imposes a penalty on phase changes.
We provide a brief clarification that the framework exhibits versatility. is not confined to the ranking problem but be applied to the general detection of group changes. As long clustering method designed for subintervals is provided, the framework can effectively perform the structure change detection. Specifically, it relies on G(I) to present the grouping results and T.(3(Z), I) to assess the goodness of of the grouping method. Besides negative log-likelihood functions, ((-) can be residuals or other measurements, determined by the specific problem.
Note that the optimization objective (2.5) exhibits separability with respect to time and features an optimal substructure property. Specifically, the objective has an additive form across time, which allows to be decomposed into independent subproblems separable over time. Moreover, the optimal solution to the overall problem can be constructed from the optimal solutions of its subproblems. These two properties enable the objective to be optimized recursively, forming the basis an efficient dynamic programming solution. We can address the combinatorial problem using Algorithm [ which provides an efficient method to estimate R
(8))
Algorithm 1 Structure Change Detection
Require: Observed data V, tuning parameters 71, 72.
Ensure: Change points estimation R.
R
+1 1, b (00,...,00) 00) E RU-1 by 0. 0, EU+1 = V.
for r from to U + do
for from to r do
b - 6 + L(B(Z), I) + r1 (I]G(Z) + 72, where I = [c]
if b, then
b, b; a, +-
+ U U+1 +1
while 1: 0 do
d + ax: R = R U
return R
3. Statistical Learning Theory
3.1 Consistency of Ranking Group Estimation
In this section, we present theoretical guarantees for our estimator. Specifically, we show that the probability of correctly identifying the underlying group structure approaches one as the sample size increases. We refer to this property group consistency, desirable feature that supports the reliability of the proposed estimation method. To ensure the theoretical results, we introduce the following assumptions.
Assumption 3.1. SuPreju,2] min; x continuously differentiable, E
mUXX ((()
< K, where A 0 is constant. (I) is three times
[n].
Assumption 3.2. The kernel function is symmetric, nonnegative, and satisfies 1 K(v) du = and _ 2 (()) du < co.
Assumptions 3.1 and 3.2 are commonly used the BT model and kernel methods let [Ti5 M for i E [m). We note that our method applies to the case where the number of comparisons may vary over time, and we assume constant number of comparisons only for the
30 We
simplicity of presentation. Recall that S 0 * 0}. Let d2 2 0 be constant such that

---

[t) - 52, E T, S Define n4 as the number of items in G, and let = ns/n. Assume r. 1/B, i e [B]. Let 8
Ing(nd) n which denotes the uniform convergence rate of the KRC
estimator [9]
Theorem 3.3. Let Assumptions 3.1 and 3.2 hold. When Mh - 00,
and nMh 0, if
1. max (6, m² V ndach o(51) and 82 = o 1+s0s (m Bin B+1
B(n-B)
):
1
V
m nMh
8
< X
617m BynMK
where 8 8= max (6, 52). then we have P(õ = G) +1
We have established the group consistency property. The following two remarks clarify the conditions of the theorem and highlight the distinct features of our theoretical analysis.
Remark 3.4. The first condition characterizes the requirement for ó1 recognize the differences among groups without being impeded by estimation errors, while the requirement of 62 limits the variation within each group ensure accurate item ability estimation, The second condition requires the appropriate order of the penalized parameter X. The term 1/'m denotes the order of integral approximation enror for 10ll2 and is not essential. If the midpoint approximation is replaced by the trapezoidal rule, then 1/m is replaced by 1/m².
Remark 3.5. Unlike standard linear regression, the design matrix X in this context is derived from a series of transformations on the observed data y. This introduces challenges for theoretical analysis. Fortunately, Ptt) is an approximation of reversible Markov transition matrix P*(t) (t) (see Section C.1 where
if j: (1)
(t)(t)
n क.
(
if
That plays an important role in deducing the properties of X and P and facilitates the establishment of thcoretical guarantees.
3.2 Asymptotic Distribution of Item Ability Estimates
In this section, we discuss uncertainty quantification, that is, the asymptotic distribution of the item ability estimators. well-characterrized uncertainty quantification enables statistical inference tasks such as hypothesis testing and helps assess the reliability of the estimators.
Selecting one representative item from each group, it let (t) = (T (t),... Tea (t))
G1, G2....,in Gn. (t).... -,, (1)) // (1) and #t(()) =
#T (t)...., (.f((t) /E #.. (t). We observe that g& is the stationary distribution of the B x matrix (t), where
(ey()) T (t) (1) if
(tay(t)
Eaps P (t),
ifi=j.
Set A (1) the group inverse of I (() (see the definition of group inverse in Kirkland and Neumann (33). We have the following result.
Theorem 3.6. Under the conditions of Theorem 3.3 82 = ol (. for a fixed B and any
E (0,1), 1), we have
n (M((f(() - +T(()) D, NOO, N(..T())(()()))))
where A(t) is B(B-1) diagonal matrix with As,,((t) = 1 Te (1)
FEPL (F (1)+ (0) f K (v)dv, and
1 k <<klls B. 2 matrix with T (t, (A (1) Al)
T(I) B B(B-1)
( (1)+ a (1) 5 S B,
1

---

3.3 Consistency of Group Changes Detection
In this section, we focus on structure recognition consistency, which refers to the property that the probability of correctly identifying group structure change points approaches one as the sample size increases. This property is important provide a general analysis for arbitrary grouping methods in Theorem 3.9 and then specialize the discussion to the dynamic ranking setting in Corollary 3.10
ensures reliable detection of structural changes. We first
To guarantee the correctness of the estimated change points, we impose assumptions regarding the grouping accuracy within a given time interval Z.
Assumption 3.7. As sample size tends to infinity, we have P(G(I) = G(I)) + 1.
Assumption 3.8. म ([((2), L(3()), =0((ss,, where 03 - 0 as sample size tends to infinity.
Assumption 3.7 is intended to ensure the accurate recovery of groups. With consistent estimator 3(I). Assumption 3.8 can be satisfied by incorporating a sufficiently smooth ((-). We show in Corollary 3.10 that our method in Section 2.2 is capable of satisfying these assumptions.
Theorem 3.9. Under Assumptions 3.7 and 3.8 if n > 72, and 63 o(72), then P( = (m)))) + with sample size tending to infinity.
Intuitively, the order of 7/2 should be larger than that of dy to ensure efficient penalty and 71 is supposed to be larger than 72 to avoid missing change points. Based on Theorems 3.3 and 3.9 have the following consistency guarantee for the ranking group change detection.
Corollary 3.10. Under the conditions of Theorem 3.3 if 71 > 72 and
nMR o(72), have
P( (m))/n) + 1 as n + 0o.
4 Computational Experiments
4.1 Recognition of Dynamic Ranking Groups
We evaluate the results using the Kendall coefficient and the mean squared error (MSE) between the estimators (f), f*f(t) and π (t) to assess the estimation accuracies of rank and value. We employ sensitivity and specificity to gauge group accuracy. Specifically, sensitivity represents the proportion of correctly identified pairs within the same group, while specificity calculates the percentage correctly distinguished between different groups We compare our method (without and with refit strategy) with the static clustering method Group Rank Centrality (GRC) (8 and the original estimator Kernel conducted
Rank Centrality (KRC). All experiments Core(TM) 15-1135G7 CPU and 16GB RAM. We utilize R pacakge sparsegl (34) for analysis. We consider two different experimental settings. Detailed configurations and parameter choices are provided in Section B.I and the results are summarized in Table [
machine with 1 11th Gen Intel(R)
Table I: Simulation results for simultaneously grouping and ranking.
Kendall
MSE
Sensitivity
Specilicity
Ours
Ours (reli)
GRC
KRC
Ours
Ours (refit)
GRC
KRC
Ours
GRC
Ours
GRC
Setting (n=20, Ml=5)
0.9998
0.9998
0.9999
0.8330
0,0586
0.0466
0.1083
0.1277
99.95%
99.97%
99.99%
100.00%
Setting
(n=50, Mh= 10) 1.0000
1.0000
1.0000
0.8207
0.0421
0.0375
0.1079
0.0684
100.00%
100.00%
100.00%
100.00%
Setting (n=20, Mh=S) 0.9416 0.9484
0.7977
0.7565
0.0494
0.0359
0.2168
0.1208
100.00%
100.00%
100.00%
63.649
Setting (n=50, Ml=10)
0.9644 0.9683
0.7974
0.7742
0.0291
0.0250
0.2167
0.0606
100.00%
100.00%
100.00%
63.71%

---

Since the refit strategy is based on identified groups, the results of non-refit and refit estimators exhibit the same sensitivity and specificity values. The sensitivity and specificity of KRC are not listed as it does not exhibit a grouping effect. It can be observed that the refit estimator performs slightly better than the non-refit one, showing a larger Kendall т and a smaller MSE. The Kendall r of our method approaches one with increasing sample size, surpassing the values of the other two methods. The results of specificity highlight the necessity of a dynamic setting. It is evident that GRC cannot distinguish some groups in the second setting. Comparing the results of both settings, the Kendall r and MSE our method are superior to those of KRC. This suggests that our method effectively captures group information, yielding better estimation accuracy.
4.2 Recognition of Group Changes
Due to the lack of established methods for detecting dynamic ranking structural changes, we compare method with naive baseline that groups items within each interval between consecutive candidate change points. A dividing point is identified as a structural change point if the groupings in its adjacent intervals differ. We evaluate the experiment results using two criteria: the number of change points and the Hausdorff distance (H-dist) between the actual and estimated sets of structural change points. Detailed configurations and parameter choices provided in Section B.2 We summarize the results for two different settings as follows.
Table 2: Simulation results for structural change detection.
Ours
Naive
H-dist
J< J 3 J J J H-dist JeJ 3= J J> J
Setting
Mh=2
0.0051
0.0%
97.4%
2.6%
0,2084
0.0%
0.0%
100.0%
Mh=4 0.0004
0.0%
99.8%
0.2%
0,1800
0.0%
0.0%
100.0%
Mr=10 0.0000
0.0%
100.0%
0.0%
0,1333
0.0%
0.8%
99.2%
Setting 2
Mh=4
0.0492
0.0%
80.4%
19.6%
0.3258
0.0%
0.0%
100.0%
Mh=10
0.0076
0.0%
97.0%
3.0%
0,2842
0.0%
0.4%
99.6%
Mh=20
0.0004
0.0%
99.8%
0.2%
0.2434
0.0%
3.6%
96.4%
Table[]showe that the estimated change points quickly converge the true values the amount of observed data increases. Compared to the naive approach, our method requires significantly fewer samples to recover the true underlying structure, demonstrating the effectiveness of the proposed objective function.
5 Empirical Analysis: Ranking Structure Recognition of NBA Teams
We analyze NBA regular season data from the 2014-2015 season to the 2018-2019 seasor The candidate structure change points correspond to the season transitions and trade deadlines each season. These trade deadlines typically fall around February 20th each year and are denoted as "TradeDDI:: We identify two structure change points: the 2015-2016 trade deadline and the end of the 2016-2011 season, with results shown in Figure For each resulting phase, we plot team win rates: alphabetically ordered on the left and ordered groups on the right. Black lines separate distinct groups. The left plot appears random, while the right displays a gradient from dark to light colors, moving from the top left to the bottom right. Items within each group exhibit similar behavior, as reflected by the color uniformity within each block, supporting the validity of the detected structure. Further details are provided in Section B.4
Figure 3 displays the estimation of teams' strengths using direct estimation and our ranking structure recognition method. The figure illustrates that our method provides a concise result regarding the structure of teams and the team strengths in each group
hanps//wwwwnbb.commganess

---

2014-2055T60000
LED
200
Figure 2: The winning percentage of Team A over Team B.
0.16
0.15
Tenm
Team
BE3
ME
0.12
EXN
51
0.10
RTK
CRI
REN
Pin
(F 0.08
DTN
Fi)
0.05
MGJ
56C
MGU
-566
LAT
UT6
141
U776
0.00
2014
2015
2016
2017 Season
2018 2019
2014
2015
2016
2017
2018
2019
Season
Figure 3: Estimation of team strengths using KRC (left) and our method (right).
Conclusion
We present a novel approach that simultaneously performs grouping and ranking based on time- varying comparisons. This offers an innovative way to analyze time-varying comparison data while generating clustered ranking results that facilitate more informed decision-makee Furthermore, we propose combined penalty for group numbers and structure change points, allowing for the detection of long-term changes underlying group configurations.
Several promising research directions remain open. First, extensions of the Bradley-Terry model that incorporate contextual information could be integrated into our framework to improve ranking accuracy. Second, while our current method focuses on pairwise comparisons, many practical scenarios involve comparisons among more than candidates; extending the approach handle such settings would broaden its applicability.
Acknowledgments and Disclosure of Funding
We thank the area chairs and reviewers for their valuable comments and suggestions, which substan- tially improved this paper.
References
[1] Christopher N. Avery, Mark B. Glickman, Caroline M. Hoxby, and Andrew Mctrick. revealed preference ranking u.s. colleges and universities The Quarterly Journal of Economics, 128( 128(1):4255 467, 2012. ISSN 0033-5533.
[2] Zhou Zhao, Hanqing Lu, Deng Cai, Xiaofei He, and Yueting Zhuang. User preference learning for online social recommendation. IEEE Transactions Knowledge and Data Engineering, (8(9):2522-2533 2016.
10

---

[3] Wanshan Li, Alessandro Rinaldo, and Daren Wang. Detecting abrupt changes in sequential pairwise comparison data, Advances in Neural Information Processing Systems, 35:37851-37864, 2022.
[4] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances neural information processing systems, 35:27730-2777 2022.
[5] Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika, 39(3/4):324-345 1952.
[6] Sahand Negahban, Sewoong Oh, and Devavrat Shah. Rank Centrality: Ranking from pairwise comparisons. Operations Research, 65(1) (00(1):26622060 287, 2017. ISSN 0030-364X 1526-5463.
[7] Gunther Schauberger and Gerhard Tutz. BTLLLaso: common framework and software package for the inclusion and selection of covariates Bradley-Terr models. Journal of Statistical Software, 88:1-29, 2019.
[8] Yue Liu, Ethan X. Fang, and Junwei Lu. Lagrangian inference for ranking problems. Operations Research, 71 (11):2002-223, 2023. ISSN 0030-364X 1526-5463.
[9] Nan Lu, Ethan X Fang, and Junwei Lu, Contextual online umcertainty-aware preference learning for human feedback. arXiv preprint arXiv:2504. 19342, 2025.
[10] Guido Masarotto and Cristiano Varin. The ranking lasso and its application to sport tournaments. The Annals of Applied Statistics, 6(4): 1949-1999,,,,, 2012.
[11] Yang Zhang, Yu Xiao, Jun Wu, and Xin Lu. Comprchensive world university ranking based on ranking aggregation. Computational Statistics, 36(2): 1 139-1 152, 2020. ISSN 0943-4062 1613-9658.
[12] Liu, Songyue Lin, Manling Wu, Wenjing Lyu, and Su. Winning and losing relationship: A new method of university ranking in the case of countries along the helt and road. Complexity, 2021:1-13, 2021. ISSN 1099-0526 1076-2787.
[13] Saúl Vargas and Pablo Castells. Rank and relevance in novelty and diversity metrics for recommender systems. In Proceedings of the fifth ACM conference on Recommender systems, pages 109-116, 2011.
[14] Changhua Pei, Yi Zhang, Yongfeng Zhang, Fei Sun, Xiao Lin, Hanxiao Sun, Jian Wu, Peng Jiang, Junfong Ge, Wenwu Ou, et al. Personalized re-ranking for recommendation. In Proceedings of the 13th ACM conference on recommender systems, pages 3-11, 2019.
[15] Qiang Song, Anqi Liu, and Steve Y. Yang. Stock portfolio selection using learning-to-rank algorithms with news sentiment. Neurocomputing, 264:20-28, 2017. ISSN 09252312.
[16] Fuli Feng, Moxin Li, Cheng Luo, Ritchie Ng, and Tat-Seng Chua. Hybrid lcarning to rank for financial event ranking. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 233-243, 2021.
[17] Shili Lin. Rank aggregation methods. WIREs Computational Statistics, 255):555-570, 2010. ISSN 1939-5108 1939-0068.
[18] Kim, F. Farnoud, and O. Milenkovie, HyDRA: gene prioritization via hybrid distance- score rank aggregation. Bioinformatics, 31(7):1034-43, 2015. ISSN 1367-4881 (Electronic) 1367-4803 (Linking).
[19] Nan Lu, Jian Shi, Xin-Yu Tian, and Kai Song. Tests on dynamic ranking. Statistica Sinica, 2024, doi: 10.5705/s.202024.115.
[20] Rose Baker and Ian McHale, An empirical Bayes model for time-varying paired comparisons ratings: Who is the greatest women's tennis player? European Journal of Operational Research, (33(1))328-1) -33,, 2017. ISSN 03772217.
[21] Gerardo lñiguez, Carlos Pineda, Carlos Gershenson, and Albert-Lászil Barabási. Dynamics of ranking. Nature Communications, 13(1):1646, 2022.
[22] Xinyu Tian, Jian Shi, Xiaotong Shen, and Kai Song. A spectral approach for the dynamic Bradley-Terry model. Stat, c322 2024.
[23] Luca Faramondi, Gabriele Oliva, Roberto Setola, and Sándor Bozóki. Robustness to rank reversal in pairwise comparison matrices based on uncertainty bounds. European Journal of Operational Research, 304(2):676 688, 2023.
11

---

[24] Gerhard Tutz and Gunther Schauberger. Extended ordered paired comparison models with application to football data from German Bundesliga. ASIA Advances in Statistical Analysis, 99:209-227, 2015.
[25] Kaycheng Soh. The seven deadly sins of world university ranking: A summary from several papers. Journal Higher Education Policy and Managemens, 39(1):104 (11)):004-1,,,,, 2017.
[26] Laura Vana, Ronald Hochreiter, and Kurt Hornik. Computing journal meta-ranking using paired comparisons and adaptive lasso estimators. Scientometrics, (06(1):22--251, 2015. ISSN 0138-9130 1868-2861.
[27] Jong-June Jeon and Hosik Choi. The sparse Luce model. Applied Intelligence, 48(8):1953-1964, 2016.
[28] Xin-Yu Tian and Jian Shi. Grouped rank centrality: Ranking and grouping from pairwise comparisons simultaneously. Stot, 12(1)),626, 2023.
[29] Heejong Bong, Wanshan Li, Shamindra Shrotriya, and Alessandro Rinaldo. Nonparametric estimation in the dynamic Bradley-Terry model. In International Conference on Artificial Intelligence and Statistics, pages 3317-3326. PMLR, 2020.
[30] Chao Gao, Yandi Shen, and Anderson Y Zhang. Uncertainty quantification in the Bradley-Terry-Luuee model. Information and Inference: A Journal of the IMA, 12(2):1033 1140, 2023.
[31] Alexandre Belloni and Victor Chernozhukov. Least squares after model selection in high-dimensional sparse models. Bernoulli, 19(2)5521-57,, 2013.
[32] Charles-Albe Alban Deledalle, Nicolas Papadakis, Joseph Salmon, and Samuel Vaiter. CLEAR: Covariant least-square refitting with applications to image restoration. SIAM Journal on Imaging Sciences, 10(I): 243-284, 2017.
[33] Stephen Kirkland and Michael Neumann. Group inverses of M-matrices and their applicaaiions. CRC Press, 2012.
[34] Xiaoxuan Liang, Aaron Cohen, Anibal Sólon Heinsfeld, Franco Pestilli, and Daniel McDonald. sparsegl: An package for estimating sparse group lasso. Journal of Statistical Software, 110:1-23, 2024.
[35] Tao Wang and I LiXing Zhu, A distribution- lasso for general single-index model. Science China Mathematics, 58: 109-130, 2015.
[36] Ming Yuan and Yi Lin. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society Series B: Statistical Methodology, 68(1 (:99677 2006.
[37] Hansheng Wang and Chenlei Leng. A note on adaptive group lasso. Computational Statistics Data Analysis, 52(12) 5286, 2008
[38] Caiya Zhang and Yanbiao Xiang, On oracle property of adaptive group lasso high-dimensional linear models. Statistical Papers, 57:249-265, 2016.
[39] Yuxin Chen, Yuejic Chi, Jianqing Fan, Cong Ma, et al. Spectral methods for data science: A statistical perspective. Foundations and Trends in Machine Learning, 14(5):566 806, 2021.
[40] Lucas Maystre, Victor Kristof, and Matthias Grossglauser Pairwise comparisons with flexible time- dynamics. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1236-1246, 2019.
[41] Wei Xiong, Hanze Dong, Chenlu Ye, Ziqi Wang, Han Zhong, Heng Ji, Nan Jiang, and Tong Zhang. Iterative preference learning from human feedback: Bridging theory and practice for RI.HF under KI. -l.costtrai In Forty-first International Conference on Machine Learning, 2024.
[42] Huiying Zhong, Zhun Deng, Weijic J Su, Zhiwei Steven Wu, and Linjun Zhang. Provable multi-pan reinforcement learning with diverse human feedback. arXiv preprint arXiv:2403.05000 2024.
[43] Banghua Zhu, Michael Jordan, and Jiantao Jiao. Principled reinforcement learning with human feedback from pairwise k-wise comparisons. International Conference on Machine Learning, pages 43037- 43067. PMLR, 2023.
[44] Eglantine Karlé and Hemant Tyagi. Dynamic ranking with the BTL model: a nearest neighbor based rank centrality method, Journal of Machine Learning Research, 24(269): 1-57, 2023.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Yes, please refer our Section 6
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
13

---

Justification:
Our Appendix provides the detailed proof of all theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
The codes for the proposed algorithm and experiments will also be made publicly available upon publication.
Justification: Please refer to experiment settings in Sections and B
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
14

---

Answer: [Yes]
Justification: The codes for the proposed algorithm and experiments will be made publicly available upon publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr ec/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
//nips. //nips.cc/ppl wwcppbblc/guuiicuceccccccoe.oopp Detgguddsse/CeSbbbiiiinn
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes, please refer to our Sections 4 and B
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Yes, please refer to our Sections and
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
15

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, please refer to our Section 4
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The research conducted in the paper adheres to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This study purely contributes to the technical advancement of dynamic ranking problem and does not have any societal impacts.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
16

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Yes, please refer to our Section
5
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
17

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
18

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
19

---

A
Notations
i i G2, We include Table 3 summarize the key notations used throughout the paper.
Table 3: Notations
Symbol
Description
integers (I.......
Identity matrix
en
x 1 vector with all elements equal to
0
Zero vector matrix
Je
f2 norm of vector e
L2 norm function f()) over interval [0, 7]
)
Set of pairwise comparison results
Vey(e)
Comparison result at time f between items i and j
(2)
Score vector of items at time in the Bradley-Teme Tenly model
M (2)
Winning probability between items and at time
KO)
Kernel function
P(t)
Transformation probability matrix for the Markov chain
B
Number of groups
Group partition of items Minimum pairwise score difference between groups
82
Maximum pairwise score difference within group Tuning parameter for group recognition
A
Constant matrix used for transformations Transformation the score vector (t))
O(t)
(())
Matrix after transformation used in optimization objective for group recognition
(t)
Vector after transformation used in optimization objective for group recognition
ztt)
Latent group of items time point
Time points where the structure changes
Candidate structure change points
[TT I)
Negative log- likelihhood function at time for score vector π Partition of time interval
P
n, 12
Regularization parameters for the objective function
#
Group inverse matrix
B Supplementary to numerical results
B.1 Experiment settings for ranking group recognition
For the experiments in Section 4.1 we consider two settings evaluate our methods, as illustrated in Figure We set T = 1, B = 3, and assign the number of items in each group as 3:3:4.
Define the first follows:
1 (2 + 0.3 (Tt)
E G1,
1 [[ 0.2 sin (6rl)
म| (1) pert, (1)
E G2,
Y
n (0.25 0.075 (6T)))
E G3-
The (t) is a perturbation term whose absolute value less than 0.01 define it as:
0.01/n. /n. For the second set,
L (1.9 + (.5
i E G
क (t( - pert,(t)
(0.1 + 0.6 arotan (t))
=
Y [
0.375sin(3zt) (1.10.3755in(33th
0.45 arctan(zt))
i G3-
The point used for order estimation 0.001. The first setting represents a simple case, while the second setting is more complex with intersections of scores among different groups.
20

---

Group Group Group
Group Group Group
1.0
0.8
1.0
t
t
Figure (1) of each group in simulations.
Set = 0.05, m = 30 and vary n and M. We repeat each setting 500 times and use the extended
BIC (EBIC) criterion [35 22 choose the tuning parameter X. We note that cross-validation be used here, but with the EBIC criteria, the computational cost is much lower. Specifically, we have
RSS(A)
EBIC(A) nm log(
+ Var(Y)) + log(nm))
nm
Here, co 0.1, [.] is the round down function. We let RSS(A)
XO(A)11 and (A)())
YY
n-1 4((A) = Σ 1110(10)120 >
Σ
(m - 1),
110:12
which is commonly used, for example, in
36 37
B,2 Experiment settings for group changes recognition
For the problem of structural change detection in Section 4.2 we consider two settings: one with three stages and another with two stages. The true abilities are depicted in Figures(]Jand(] respectively. We set = 0.02 and denote the observation times within each phase as M. The experiment repeated 500 times. We employ the widely-used 10-fold cross-validation for the choice of tuning parameters Yi and 7/2
W
Items
Irems 4-5
Item 6
0.00
7-10
0.02
0.25
0.50
0.75
1.00
Figure 5: 75" 71) in the first setting.
21

---

63
Items 1-2
C2-
Trems 6-10
0.0
3.0
625
0.50
0.75
1.00
Figure 6: x (1) in the second setting.
Specifically, in setting 1, three phases considered. The score functions in phases I and III remain the same, with a proportion of items being 3:3:4.
0.2 + 0.03 sin 18m()
i 1,2,3. 3, i == 4, 6, i i= 7,8,9, 10.
x (t) =
0.1 - 0.02 sin( 18T()
0.025 0.0075 sin (18rt)
For phase II, the item proportion is 1:1.
0.15 0515 + 0.02 sin sin (18xt)
1, 1,....................
(t)
0.05 - 0.02 sin (18rt) i = 6, 10.
In the second setting, during phase I, the item proportion is 1:4.
0.25 + 0.06 (in7t))
i= 1,2,
n (t)
0.0625 0.015
i= 3 ..., 10.
For phase II, the item proportion is 1:1:3.
0.15 0.04 sin(
i 1,2, i 3, 4,
(1)
0.065 + 0.25(t 1))
0.1425 sin55tt)) 0.0625(t - 2/1110
i i= 5,..., 10.
B.3 Sensitivity of the hyperparameter choice
Hyperparameters play crucial role in the performance of the proposed method, and their selection can be challenging. Ideally, computationally effective guidelines, such as those based on information criteria, would facilitate hyperparameter tuning. However, due to the novel optimization objective in our case, establishing such rules non-trivial and warrants further investigation.
In this study, we employ cross- cross-validation, which yields good empirical performance. To evaluate the sensitivity of the method to hyperparameter choices, conduct additional experiments using parameter grids for the two experimental settings described in Section 4.2 Specifically, we repeat each combination of hyperparameters 50 times and calculate the average number of detected change points for each combination. The results are shown in Tables [
From the experimental results, we observe that the number of estimated change points varies with different values of 71 and 7/2. In particular, we find that 7/2 should be greater than 72 to ensure effective change point detection, consistent with theoretical analysis in Theorem 3.9 Moreover, for a fixed 71, the estimated number change points decreases as 72 increases. This aligns with our intuition, larger values of "/2 impose higher penalty for each additional change point. Overall, while the tuning parameters affect the change point estimation, the estimated number of change points remains relatively stable under substantial variations in the scales of 71 and 7/2.
B.4 Supplementary to empirical analysis
For the empirical study, we set the bandwidth to match the season length, and other parameters remain consistent with those used in the simulations. Tuning parameters chosen through cross-validation
22

---

Table 4: Average change point number for different values of
71 and 72 for setting 1.
71
0.02
0.04
0.06
0.08
0.1
0.2
0.4
0.6
0.8
2
4
6
8
10
020000
12
2.12
2.1
2.22
2.22
2.22
2.04
1.58
1.54
1.54
1.54
1.54
1.54
1.54
1.54
1.54
0200.0
2.1
2.08
2.2
2.2
2.2
2.02
1.58
1.54
1.54
1.54
1.54
1.54
1.54
1.54
1.54
7==0.006
2.08
2.2
2.2
2.2
2.02
1.58
1.54
1.54
1.54
1.54
1.54
1.54
1.54
1.54
12=0.008
2.1
2.08
2.2
2.2
2.2
2.02
1.58
1.54
1.54
1.54
1.54
1.54
1.54
1.54
1.54
12=0.01
2.08
2.2
2.2
2.2
2.02
1.58
1.54
1.54
1.54
1.54
1.54
1.54
1.54
1.54
72=0.02
0
2.06
2.18
2.18
2.18
1.96
1.54
1.54
1.54
1.54
1.54
1.54
1.54
1.54
1.54
72=0.04
0
2.1
2.14
2.14
1.9
1.52
1.52
1.52
1.52
1.52
1.52
1.52
1.52
1.52
72=0.06
0
0
0
2.14
2.14
1.86
1.52
1.52
1.52
1.52
1.52
1.52
1.52
1.52
1.52
12=0.08
0
0
2.12
1.82
1.52
1.52
1.52
1.52
1.52
1.52
1.52
1.52
1.52
22=0.1
0
0
0
0
1.82
1.52
1.52
1.52
1.52
1.52
1.52
1.52
1.52
1.52
12=0.2
0
0
0
0
1.52
1.52
1.52
1.52
1.52
1.52
1.52
1.52
1.52
12=0.4
0
0
0
0
0
1.52
1.52
1.52
1.52
1.52
1.52
1.52
1.52
12=0.6
0
0
0
0
0
1.52
1.52
1.52
1.52
1.52
1.52
1.52
02=0.8
0
0
0
0
0
0
1.52
1.52
1.52
1.52
1.52
1.52
12=1
0
0
0
0
1.52
1.52
1.52
1.52
1.52
Table 5: Average change point number for different values of 7/1 and Y/2 for setting 2.
71
0.02
0.04
0.06
0.08
0.1
0.2
0.4
0.6
0.8
8
10
0.002
1/
1.92
1.86
1.94
1.92
1.92
1.9
1.9
1.9
1.86
1.78
1.76
1.76
1.76
1.76
1.76
0.004
1.82
1.84
1.92
1.9
1.9
1.88
1.88
1.88
1.84
1.76
1.74
1.74
1.74
1.74
1.74
02=0.006
1.8
1.84
1.92
1.9
1.88
1.88
1.86
1.84
1.76
1.74
1.74
1.74
1.74
1.74
72=0.008
1.68
1.78
1.88
1.86
1.86
1.84
1.84
1.86
1.84
1.76
1.74
1.74
1.74
1.74
1.74
72=0.01
1.68
1.78
1.88
1.86
1.86
1.84
1.84
1.86
1.84
1.76
1.74
1.74
1.74
1.74
1.74
72=0.02
0
1.76
1.88
1.86
1.86
1.84
1.82
1.84
1.84
1.76
1.74
1.74
1.74
1.74
1.74
72=0.04
0
1.84
1.82
1.82
1.78
1.8
1.8
1.72
1.7
1.7
1.7
1.7
72=0.06
0
0
0
1.82
1.8
1.78
1.74
1.8
1.8
1.72
1.7
1.7
1.7
1.7
1.7
72=0.08
0
1.8
1.78
1.74
1.8
1.8
1.72
1.7
1.7
1.7
1.7
1.7
72=0.1
0
0
0
0
1.78
1.74
1.78
1.78
1.7
1.68
1.68
1.68
1.68
1.68
1==0.2
0
0
0
0
1.68
1.78
1.78
1.7
1.68
1.68
1.68
1.68
1.68
2240.4
0
0
0
0
0
0
1.76
1.78
1.7
1.68
1.68
1.68
1.68
1.68
72=0.6
0
0
0
0
1.74
1.7
1.68
1.68
1.68
1.68
1.68
02=0.8
0
0
0
0
0
0
1.66
1.68
1.68
1.68
1.68
1.68
72=1
0
0
0
0
0
1.68
1.68
1.68
1.68
1.68
are "1 0.04 and %/2 0.006. For the learning results demonstrated in Figure in the first period, the GSW and SAS teams occupy the first two groups due to their extremely high winning rates. In second phase, the first group includes GSW and SAS, while the second group consists of items from the top groups in the previous stage, with some changes. For instance, MEM and DAL shift to weaker groups. From the second to the third stage, a major change the leading teams is notable: ATL, CLE, SAS disappear from the top groups, and teams like DEN, MIL, PHI emerge in the top groups.
It is important to note that employing a static method yields significantly different results. For instance, the initial phase, while also identifying seven groups, the static method GRC categorizes CHI into single group, amalgamates POR into the MEM group, and groups all items from BOS to CHA (in the order presented in Figure 2 excluding POR) as unified entity. Moreover, in the third phase, the static method recognizes three groups. The first group remains unchanged, with the subsequent ten items (excluding NOP but including LAC) forming the second group, while all other teams constitute the third group.
23

---

C Technical proofs
C.1 Proof of T'eheremm
Proof. Let j((A) represent the eigenvalue of matrix A. Let As be the submatrix of A consisting of columns that correspond to items in for matrix A, and as be the subvector of æ comprising components corresponding to S for vector x. Before presenting the main theorem, we present the following lemma on the properties of and P.
Lemma C.1. Let Assumptions
3.1 and 3.2
hold. If Mh + oc, - and nMh + 0, we
have || - P() = ()), (I
) la S
1+00s
B
In-B B.I
Mmin
(1)Xs(() Z T and
[[X (t)xs(t)) (t)ll2 S V with probability tending to 1.
We first show the consistency of 0. From Theorem SI of Lu et al. [19), we have [ (t)
(8))). Combining the definition of 0, [|@(t) 0* (t) Op()).
Notice that
0 arg min Q(0) arg min マ =1Q
X01
10.11
10:122
(C.1)
Following the proof of Theorem 2.1 points out that, (C.l is equivalent to
38 Q(0) is a strictly convex function. Lemma 4.1 in 38
0,
x (Y x0) + 11
vO, + 0, 110/12
0,
and
| (Y xx0 - Al
0 1 vO, =0,
where Xj represents the columns of corresponding to 0,. Therefore, it is sufficient to prove 30, Vj E S,0 + O, and Vj & S, Ooj 0, such that
Poz 00y l12
X Xs0os) A
0, E
(C.2)
and
1X] - -xs0o))l < 1110 Vj
(C.3)
Using
C.2
we have
X Xs0os) ABus 0,
0,
where Bos
100 12 112 jes- Notice that X X is invertible. Hence,
ous = =XX - - X(X -Bus
03 + ((X -x.Y 03s) -(x -Bus.
(C.4)
As for the second term,
[XXXxs) Y 03 [ - Xs03)laa
sup
ke[m]
sup
||(X] (4) 100
(C.5)
ke[m]
24

---

where
(Y 0000-xss()003))) x0( < (Y --00-x--100 (-) + (1)0g (1)l2
=I(PP- (L) P (T)) x (1)]2 |Xss (1)
(C.6)
The proof of Theorem
22 implies
P (t) P PTPP)())))))))))))))) (t)) (f)ll2 (f))
Op(y n²Mh"
For the second term in (C.6).
X (t)00) (t)ll2
(x)=(t))la|00' (t)l2 - P (t) 1| (2 (Q- ()s=)))(2211000)) 1)s1122105s (t)ll2
(C.7)
1+000 (n Bin
From Lemma C.I. we have |C.77 x
(n B)g2. 82 = op(y
" D
n Mhh ).
1+000s
B(n B)
then C.7 is Op((
n=MK). ). The first term in
C.5 is Op(y
B n
using Lemma
C.I
Hence,
C.5 is
Op(y
B nd Mr )-
For the third term in C.4 have
|A((5XXs) Basll
sup
Bos )l
kE[m]
< sup |A(X (th (sta) Bas()))))) S sup AI| (tx) (ta (tx)lla k[[m]
ke(m)
B VB
(C.8)
n1 min |0:112
Note that
V
I m
min 1112 2
min m
I
1
max 100, 12 m sES
I
EES
min (0) + 0 m
+ (p) 2 81.
ES
Hence,
C.8
Av B Rymb,
From
Vj E S,
B
AVB3
0051l2 2 vmdi /V
Mh
no
n2 Mho = o(1) and AB3/2 1/28 = (11. then with probability tending to 1, we have Vj E S. *
0. Actually, have proved that if AByaMK vmb,
O(1), then |loss 0%1 00 Opl 7
Then we prove
C33 Assume j $ S.
x
X 1l2 S [x][ (Y Xs03)lla+ (x (Xs03 Xslas)llz sup
m (Y)
(x())
X (ss()()))
(t(t))ns(t))lle..
(C.9)
Similar to the proof of Theorem in 28 we can obtain
C.9
sup vm max
-
max
(t) (22 ons (t)ll
< sup v mn max
(t) (t) (t)) (t)l2
v m. max
(t)
(t)(1)
(C.10)
i,k
25

---

The second inequality is gotten using Co... From the proof of Theorem in (28] we have m (X so (t))sal =1) and max; [Xg ge (t))aal O(n). Therefore,
C.10
mBa Oply
wArk
On the other hand, since 00yla 1l S
+ 10,
0ills + V 10 2 62 + 6, we have
minjas 1|10,112*
Since À
m2 ö, the theorem is proved. nMh
C.2 Proof of Theorem 3.6
Proof. Let Pg take the following form:
{{
{
Enjea, Eizer, Envering ((atta))a(((tta))
if + j:
Poiy(t)
B Liye, Dized, 2iyen (x(1.2))
1-E PGia(!),
if j.
Set मे as the stationary distribution of Pc. Define Ya(() = BPg. Let Y = BP Similar to the proof of Theorem 1 in [28] using the derivative of stationary distribution, we can obtain
θT (t) (Ya
JP: (t) (1),
Tā (1) T()(
(C.11)
where
OP T 0Y Gt
OP (1) 0 by
OP (1)
3P (1)
1
(t) oY
0Y Giy (t)
) =
)a B 0Y (t)
and other elements of the derivative matrix are zero.
Then we consider the difference between (t(() and (t))) For
Yai((1) - (G)
Eneo. Eneo, EmVETrI, (())(4))
Eneo. Enso, EmeTii Rite (t(,k)) ΣneG, Ense, Enerhi Kn(t, (x)
Enec. EnEE, Enerr (t,
+ Enec, EnEG, Emer, Vi,t, (tx)
(1(
Eneo, Eise, EmsEih Ky(t,ty tx)
n,n IEG, KEG, T. (t) (T)
(t(() + Σ (t) + +(((t
(t)
T4m LEG; kEG;
(t) Yui, (t)) + (Y wij (t) You (t)) + (t) (0)
(C.12)
Using central limit theorem, we have
nPMh(YGs Ywn)) D, N(0, 1 ( (t)
TITy (TC. (t) + न. (t))2 / (v()dd)).
(C.13)
Notice that when nMhb + * Vn? wij (1) -Yrr((1)
Mhn Eneo, EiEe; EmseTi, Hi, (n,
(1)
Eneo. Eneo, Emer, K Kt(t,tt)
n.nl IEG, kEG, (t) + म. (t)
MV 2 Eiec. Enec, (ia v² K du 2n;ng
- 0.
(C.14)
Besides,
V1²MH(YE() Y2(!)
(t()) n. IEG, kEG, क: + (t)
TT,(1) (t) (t.(t) + T (t)
=Vn2Mh
SVn*Mhō, + 0.
(C.15)
26

---

Combining CC.11
C.13). C.122
C.14 and (C.5).. we have,
Vn2Mh(Fect +o(()) 2, ((t))(0,A()))
(C.16)
Set (G) = (t) (t) and (G) = (t( - (t) From Theorem 3.3 P(G = G) +11
Therefore, for every R. from
(G) E A) = (G) E A = G)P(G - G) P(L(()) A + G)P(C # G),
we can obtian
lim (G) A) = lim (G), E A G) lim P(I.(G) E A). 7-220
(C.17)
From
C.16 and CC.17). we have
Mn - (t)))) 2, ((NOTTOAAT))
C.3 Proof of Theorem 3.9
Proof. We first prove that P( C (m))-1) Suppose there are change points in the interval (3;, 1441), and assume the first one be 175. Let = (5) and J2 = (fz 8;-1).
2 Σ
U J2I1 (2)
=n
-1(GT113LU13333333330003 + + Op(ss)
(C.18)
Notice that
+ -1 G + |G" (1)6)1 U Fo] U Tzl.
and with the fact that J1 C (77-1. n3) and the right endpoint of J1 is exactly 77,, we have
GG* (Jr U T2)1 G (Ji) 1.
Hence, with probability tending to 1,
C.18
7/2 - S 72 - < 00
which is contradictory to the definition P. Therefore, there are no change points between the estimated ones. In another word, P( - (n:)/1) + 1.
The above proof classifies that all change points are in the estimation set, and we then prove that all points in the set are change points. Suppose that & & {nr then there exists such that & E (1i, 1.) Let Jj (nz, 54) and T2 (5i, 11+1).
Then we have
2 Σ (UB0)).0++ - -18111U3)) (LB
= ((G(T))I U JT) -
ual) + + + (p(5)) + > 0
(C.19)
holds with probability tending to 1. Note that 19 means that removing $; in leads to a strictly smaller value of (25). which is contradictory the definition of P.
27

---

C.4 Proof of Corollary 3.10
(t) 70 = Op(y NMA ) Vt E [o, v] by noticing the consistency of
Proof. First notice that
. 2
group estimation by Theorem
3.3
and convergence rate results in 2 Then only need to prove
that if m(()=((t)le Op()). E [0, VJ. then μ [[T I) L म Z) Op(S). In fact, we can
(1)l2
obtain
1
[TT,II - -((
((T(()) ((+((t) dt)
[I]
JET
(t)
(t)
Σ
Diy (t)(t)]logg
log( [T( + क ,() di, (C.20)
1EI nin - 1) 7
TT(11) + ,
(6,)):0ff
where
n (t) log(
(t)
= log(1 + (,(t)/ (t) (t)
logg
[1) + TT
T. (1) + TT (1)
1 (1)/T((((
Op({) using Taylor expansion. Then (C.0
Op(S).
C.5 Proof of Lemma
Proof. We first prove I P*(t)l22 = 0(1). We omit t in this part for simplicity. Let To be the stationary distribution of P i.e., To P = TT. Let = diag(mo). Then under Assumption 3.1
min; Toi | >>]lll
|I
P')a112
mmxx, Toi
| P P')a1l2
max; Toi
|TT1/22:11
|
min, TOx
|T|/2x|||
Hence, we have
|| P*) P))0l
|1 P* P*1 2x122
I --P P-1 max zf0
|T1/2x/||
max 240
|11//22||
which is the maximum singular value of II [I II 1/2 1-12, From 2.4.3 of [39], it symmetric matrix and its spectral norm equals 1 - Mmin
PP), which is (1). Further,
P-Pl S II P 121 + |P-P" p Op(1)
using Lemma of
Let W be the inversion of (I (I-))-- (Q s=. Similar to Lemma 3 in symmetric tridiagonal matrix, and for vector x,
28 W is (n - B) x (n B)
x Wx
Σ (o 1;-1)²
B n
which is corresponding diagonal-constant matrix with the minimum eigenvalue being cos (n- (n-B)w B41 ). Hence, II(Q ((QQ-)ssi) "ss 2
B (-- 1+
1+cus
Since X is a matrix with diagonal elements X (t) (t) it), sufficient prove the same for X (l). Then we conclude the results using Lemma 4 in (28
D Additional discussions
D.I Independence assumption of pairwise comparisons
We assume that pairwise comparisons, the elements of ), are independent in Section 2.1 The inde- pendence assumption among pairwise comparisons a standard and widely adopted simplification in the BT model. In many real-world applications, such sports tournaments and recommender systems, comparisons are typically collected independently across individuals or time, which renders the assumption approximately valid in practice.
28

---

For example, Masarotto and Varin [10] analyze outcomes from the National Football League and American College Hockey under the independence assumption. Similarly, Maystre et 40 apply time-varying BT model to datasets from the NBA and the Association of Tennis Professionals (ATP), also assuming independence between matches. In the context of recommender systems, Liu et al. [8] adopt the BT framework for product ranking based on independent pairwise comparisons. All of these analyses demonstrate strong performance in practical settings.
Furthermore, recent studies in reinforcement learning from human feedback (RLHF) continue to employ the BT model with the independence assumption, which has proven effective. For instance, Xiong et al. [41], Zhong et al. [4),, and Zhu et al. 43] develop RLHF algorithms using pairwise comparisons assumed be independent, and report successful outcomes across various tasks.
D.2 Smoothness condition of Assumption 3.1
We clarify that the smoothness condition in Assumption 3.1 is standard in dynamic settings and can be relaxed. In Section 2.2 it suffices for the score functions be Lipschitz continuous to ensure consistent group identification (Theorem (.3). The stronger smoothness assumption is only needed derive the asymptotic distribution of the estimators (Theorem 3.6). Importantly, the
Lipschitz condition is widely adopted in theoretical analyses of dynamic ranking problems, such as Assumption 5.2 of Bong et al. [29] and Assumption of Karlé and Tyagi
[44]
Furthermore, in Section 2.3 the general theoretical results in Theorem 3.9 rely only on Assump- tions 3.7 and 3.8 which do not require the ability trajectories to be smooth. The smoothness condition in Corollary 3.10 imposed solely to facilitate the application of Theorem 3.3 but as noted above, this can be weakened to Lipschitz condition. Alternatively, one may segmented estimation strategy over gridded time interval to estimate G(I), and then apply our proposed change detection framework without requiring any smoothness assumption.
Finally, we would like to emphasize that method performs well even in the presence of abrupt changes. This is supported by simulation results in Section where the underlying score trajectories (shown in Figures 5 and 6] Section B.2 feature nonsmooth and abrupt changes, yet our method maintains strong performance.
D.3 Optimality of Theorem 3.6
Theorem 3.6 shows that the estimation error satisfies [|a (t((l) Op((n"Mh)- which matches the optimal convergence rate. Specifically, for the refitted estimator 7.' defined in equation (2.3), we obtain the relative error rate [| (t) (()
Mh)-1/2) We analyze the result based on the effective sample size. On average, there n items per group, each compared against roughly (B 1)n others. Assuming the Epanechnikov kernel with bandwidth h, each pair contributes about 2Mh effective observations. Hence, the total number of comparisons used estimate each 7/ (1) is of order M.. Since we pool comparisons across all items within the same group to estimate each ability score, the estimation procedure leverages this aggregated information. This matches the optimal order 1/2 established in Chen et al. 39 and Karlé and Tyagi [44), where L denotes the average number of comparisons per item.
29


Reviews:
[{"review_id": "Ll9q5fAbpo", "reviewer": "Reviewer_Ha6P", "summary": "The authors describe a statistical method to estimate a dynamic ranking structure in preference data. The Bradley-Terry model is used for inferring latent preference scores. Items are grouped together based on similar temporal behavior in their preferences. Given a grouping structure, the ranking order of these groups is determined. One of the goals of this study addresses the question how significant changes in the preference structure can be detected.\n\nThe method improves over SOTA techniques by higher accuracy and enhanced interpretability.\n\nThe objective function for scoring seems to be novel and it is designed to be solved by dynamic programming which guarantees efficient solutions. The Ranking Structure Recognition of NBA Teams shows fairly rapid changes in group structure associated with trading windows or the end of a season (see fig 2).", "questions": "In this paper, the basic assumption of the Bradlex-Terry model requires independence of the pair-wise preferences $y_{ij}(t_k) \\in \\mathcal{Y}$. How well does this assumption hold for real world preference data? It would be helpful for the reader to discuss the applicability of the BT model for today's applications. \n\nWhy does the normalization of the refit estimator in eq (2.3) scale with the absolute group size $|\\hat G_k|$ and not with the relative group size? \n\nWhat are the \"separable properties\" in the objective function that admit dynamic programming as a solution strategy? There seems to be only one occurence of \"separable\" in the manuscript stating the claim; but no explanation is given\nwhy \"separable properties\" enable dynamic programming.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\nObserving dynamic preferences requires a complex dynamics on the symmetric group to estimate a partial ranking. It couples grouping of objects when they are indistinguishable and ranking the resulting different groups. The model seems to perform well in applications when compared to SOTA methods.\n\nThe authors statistically analyze the group estimates $\\hat{G}$ and prove consistency (Th 3.3) and asymptotic normality (Th 3.6). I find this analysis convincing. \n\nThe authors claim priority on the ranking structure changes for the BT model. The proposed objective function can be efficiently solved by dynamic programming due to its \"separable properties\". The experimental results document the improvements over SOTA results for group change detection. \n\n\nWeaknesses:\n\nThe paper seems to be a compressed version of a longer manuscript and it reads a bit overloaded. I had to search together the definitions to understand the essence of the objective function and some of the statistical arguments. \n\n- Some of the symbols are not clear, e.g., where is $\\tilde{S}_k$ defined.\n\n- It is also unclear (to me) why the normalization of the refit estimator in eq (2.3) scales with the absolute group size $|\\hat G_k|$ and not with the relative group size.\n\n- Algorithm 1 does not introduce a symbol for the observed data. I assumed it to be $mathcal{Y}$. Please add it to specify the data in the algorithm.\n\n- The objective function (2.1) could be described in more detail. The first part seems to measure deviations from stationarity, while the $\\lambda$ scaled part regularizes the inference. Why are ratios of item score differences proposed as regularization term? The regularization term often refers to the corresponding noise process of the function to be determined by variation. Does there exist such an interpretation for the second term in (2.1)?", "was_revised": true, "final_justification": "The answers of the authors are mostly satisfactory. \n\nThe motivation of the regularization term in eq (2.1) by fused lasso is still somewhat opaque to me. Statistical inference on rankings operates on the symmetric group with restrictions generated by the ranking data structure. Operating in this space can induce global constraints with implicit dependencies which casts some doubts on analogies to inference problems in Euclidean space. \n\nDespite this somewhat critical remarks, I find the paper interesting and will stick to my original grading.", "rebuttal": "We would like to thank you for your detailed and constructive comments. In light of the comments, we have revised the paper, which we believe has significantly improved both its quality and presentation. Below, we respond to the comments and suggestions point-by-point.\n\n1. *\"In this paper, the basic assumption of the Bradley-Terry model requires independence of the pair-wise preferences $y_{ij}(t_k)\\in\\mathcal Y$. How well does this assumption hold for real world preference data? It would be helpful for the reader to discuss the applicability of the BT model for today's applications.\"*\n\n   **Response:**  \n   Thank you for the insightful comment. The assumption of independence among pairwise comparisons is a standard and widely adopted simplification in the Bradley-Terry (BT) model. In many real-world applications, such as sports tournaments and recommender systems, comparisons are typically collected independently across individuals or time, which renders the assumption approximately valid in practice.\n\n   For example, Masarotto and Varin (2012) analyze outcomes from the National Football League and American College Hockey under the independence assumption. Similarly, Maystre et al. (2019) apply a time-varying BT model to datasets from the National Basketball Association (NBA) and the Association of Tennis Professionals (ATP), assuming independence between matches. In the context of recommender systems, Liu et al. (2023) adopt the BT framework for product ranking based on independent pairwise comparisons. All of these analyses demonstrate strong performance in practical settings.\n\n   Furthermore, recent studies in reinforcement learning from human feedback (RLHF) continue to employ the BT model with the independence assumption, which has proven effective. For instance, Xiong et al. (2023), Zhong et al. (2024), and Zhu et al. (2023) all develop RLHF algorithms using pairwise comparisons assumed to be independent, and report successful outcomes across various tasks.\n\n   While the independence assumption is widely adopted and often reasonable, we agree that extending the BT model to account for potential dependencies in comparisons is a valuable and promising direction for future research.\n\n   **References**  \n   Liu, Y., Fang, E. X., & Lu, J. (2023). Lagrangian inference for ranking problems. *Operations Research*, 71(1), 202–223.  \n   Masarotto, G., & Varin, C. (2012). The ranking lasso and its application to sport tournaments. *The Annals of Applied Statistics*, 6(4), 1949–1970.  \n   Maystre, L., Kristof, V., & Grossglauser, M. (2019). Pairwise comparisons with flexible time-dynamics. *Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining*, 1236–1246.  \n   Xiong, W., et al. (2023). Iterative preference learning from human feedback: Bridging theory and practice for RLHF under KL-constraint. *arXiv:2312.11456*.  \n   Zhong, H., et al. (2024). Provable multi-party reinforcement learning with diverse human feedback. *arXiv:2403.05006*.  \n   Zhu, B., Jordan, M., & Jiao, J. (2023). Principled reinforcement learning with human feedback from pairwise or k-wise comparisons. *International Conference on Machine Learning*, 43037-43067. PMLR.\n\n2. *\"Why does the normalization of the refit estimator in eq (2.3) scale with the absolute group size $|\\widehat G\\_k|$ and not with the relative group size? It is also unclear (to me) why the normalization of the refit estimator in eq (2.3) scales with the absolute group size $|\\widehat G\\_k|$ and not with the relative group size.\"*\n\n   **Response:**  \n   Thank you for your question. As stated in Section 2.1, we impose the constraint that the sum of item abilities is 1, i.e., $\\sum_{i=1}^n \\pi_i^* = 1$ to ensure a unique representation without loss of generality. Since only the ratio between scores matters in the Bradley-Terry model, this constraint provides identifiability. When recovering the original scores after refitting, we still require that the normalized scores satisfy $\\sum_{i=1}^n \\widehat{\\pi}_i^{\\mathrm{rf}} = 1$. At the same time, we need to preserve the score ratios between items from different groups, meaning that for $i \\in \\widehat{G}_l$ and $j \\in \\widehat{G}_k$, we have $\\frac{\\widehat{\\pi}\\_i^{\\mathrm{rf}}}{\\widehat{\\pi}\\_i^{\\mathrm{rf}} + \\widehat{\\pi}\\_j^{\\mathrm{rf}}} =\\frac{\\widehat{\\pi}\\_{\\widehat{G}\\_l}}{\\widehat{\\pi}\\_{\\widehat{G}\\_l} + \\widehat{\\pi}\\_{\\widehat{G}\\_k}}$. To satisfy both conditions simultaneously, the normalization in equation (2.3) is scaled by the absolute group size $|\\widehat{G}_k|$. We will add this clarification in the revised version.\n\n3. *\"What are the 'separable properties' in the objective function that admit dynamic programming as a solution strategy? There seems to be only one occurence of 'separable' in the manuscript stating the claim; but no explanation is given why 'separable properties' enable dynamic programming.\"*\n\n   **Response:**  \n   Thank you for your question. The term \"separable properties\" means that the objective function can be decomposed into independent subproblems. In our case, the objective is separable because it has an additive form across time, making it separable over time. Moreover, the objective satisfies the optimal substructure property, meaning that the optimal solution to the entire problem can be constructed from optimal solutions to its subproblems. These two properties, separability and optimal substructure, allow the objective to be optimized recursively, forming the basis for an efficient dynamic programming solution. We will add the explanation in the revised manuscript.\n\n4. *\"Some of the symbols are not clear, e.g., where is $\\widetilde{\\mathcal S}_k$ defined.\"*\n\n   **Response:**  \n   Thank you for pointing out this. In the revised version, we will clarify the definition more clearly as follows.\n\n   We use $\\widetilde{\\mathcal S}$ to denote the estimated partition points of different groups. Specifically, we let $\\widetilde{\\mathcal S}=\\{0\\}\\cup\\widehat{\\mathcal S}\\cup\\{n\\}$. Without loss of generality, we assume $\\widetilde{\\mathcal S}$ is arranged in ascending order (if not, we simply reorder $\\widehat S$), and $\\widetilde{\\mathcal S}_i$ denotes the $i$-th element of $\\widetilde{\\mathcal S}$. We will also include a notation table in the revised version to improve clarity.\n\n5. *\"Algorithm 1 does not introduce a symbol for the observed data. I assumed it to be $\\mathcal{Y}$. Please add it to specify the data in the algorithm.\"*\n\n   **Response:**  \n   Thank you for pointing this out. We will add $\\mathcal{Y}$ to specify the observed data in Algorithm 1.\n\n6. *\"The objective function (2.1) could be described in more detail. The first part seems to measure deviations from stationarity, while the $\\lambda$ scaled part regularizes the inference. Why are ratios of item score differences proposed as regularization term? The regularization term often refers to the corresponding noise process of the function to be determined by variation. Does there exist such an interpretation for the second term in (2.1)?\"*\n\n   **Response:**  \n   Thank you for the question. In equation (2.1), we use the score difference term $||\\pi\\_{i}(t) - \\pi\\_{i+1}(t)||\\_{2,T}$ in the regularization. Although the form is more complex, it is inspired by the fused lasso penalty introduced by Tibshirani et al. (2005), where the term $\\sum_{i}|\\beta_i-\\beta_{i-1}|$ is used to encourage similarity between adjacent coefficients. Regularization is typically employed to reduce the effect of noise and recover the underlying signal. In our context, the fused-type penalty serves a similar role: for homogeneous items that are expected to perform similarly, the penalty helps to reduce variability caused by noise within the same group.\n\n   Moreover, we incorporate an adaptive weight of the form $||\\widetilde \\pi_{i}(t)-\\widetilde \\pi_{i+1}(t)||_{2,T}^{-1}$, which is inversely proportional to the estimated score differences. This implies that item pairs with larger estimated differences, which likely reflect meaningful variation, are penalized less. In contrast, smaller estimated differences, which may be due to noise, are penalized more heavily. This adaptive weighting strategy, similar to the adaptive lasso (Zou, 2006), has been shown to improve empirical performance and provide desirable theoretical properties.\n\n   **References**  \n   Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., \\& Knight, K. (2005). Sparsity and smoothness via the fused lasso. *Journal of the Royal Statistical Society Series B: Statistical Methodology*, 67(1), 91-108.\n\n   Zou, H. (2006). *Journal of the American Statistical Association*, 101(476), 1418-1429.\n \nAgain, thank you very much for your time and valuable comments."}, {"review_id": "7YdhWgy9Z9", "reviewer": "Reviewer_RTDx", "summary": "This paper proposes a novel framework for dynamic ranking structure recognition based on the Bradley-Terry model. It simultaneously identifies item groups with similar time-varying rankings and detects structural changes over time using a spectral estimation method combined with temporal penalties. The authors design an efficient optimization procedure using adaptive group lasso and dynamic programming, and provide theoretical guarantees for consistency and asymptotic distribution. Experiments on synthetic data and NBA game results demonstrate the method’s effectiveness and interpretability.", "questions": "1. Can the authors add one additional pargraph to briefly define the \"group change\",possibly with illustrative examples to enhance clarity?\n\n2. Can the authors conduct additional experiments to evaluate the sensitivity of the method to hyperparameter choices?", "limitations": "The authors point out the limitation of current formulation only focuses on pairwise comparison, leaving the extension to multiple candidates open.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n\nThis paper tries to tack the fundmental problem - how to track groups and rankings that change over time using preference data.  It introduces a novel, separable objective function that enables efficient optimization via dynamic programming. Theoretical results, including consistency and uncertainty quantification, are rigorously established. Empirical evaluations on synthetic and real-world datasets (e.g., NBA games) show that the proposed method outperforms existing baselines in both accuracy and interpretability.\n\n\n\nWeakness:\n\nThe main weakness so far lies in the presentation. While the theoretical sections are mathematically sound, they are dense and lack intuitive explanation, making the paper difficult to follow. Including a complete notation table and illustrative diagrams could significantly enhance clarity. \n\nAdditionally, the authors mention using cross-validation to select hyperparameters—how sensitive is the method to these choices?", "was_revised": true, "final_justification": "I retain my positive score", "rebuttal": "We would like to thank you for your detailed and constructive comments. In light of the comments, we have revised the paper, which we believe has significantly improved both its quality and presentation. Below, we respond to the comments and suggestions point-by-point.\n\n1. *\"Can the authors add one additional paragraph to briefly define the 'group change', possibly with illustrative examples to enhance clarity?\"*\n\n   **Response:**\n\n   Thank you for your insightful comment. To improve clarity, we have added the following paragraph in the revised version:\n\n   Many studies have addressed the grouping problem in ranked items, which involves assigning homogeneous items to the same group (e.g., in the context of journals, players, or sports teams). However, when considering the temporal aspect, we encounter situations where these groups evolve over time. For example, in basketball, player trades and coaching changes can significantly affect the team’s performance, potentially causing a team to rise to a higher ranking tier. In this context, the term 'group change' refers to these shifts in group membership over time, which are crucial for accurate analysis in the dynamic grouping problem.\n\n2. *\"Additionally, the authors mention using cross-validation to select hyperparameters—how sensitive is the method to these choices? Can the authors conduct additional experiments to evaluate the sensitivity of the method to hyperparameter choices?\"*\n\n   **Response:**\n\n   Thank you for your question. Hyperparameters indeed play a crucial role in the performance of the method, and their selection can be challenging. Ideally, computationally effective guidelines, such as those based on information criteria like AIC, would help in choosing hyperparameters. However, due to the novel optimization objective in our case, establishing such rules is non-trivial and warrants further investigation.\n\n   In this study, we employ cross-validation, which yields good performance. To evaluate the sensitivity of the method to hyperparameter choices, we conduct additional experiments using parameter grids for the two experimental settings in Section 4.2. Specifically, we repeat each combination of hyperparameters 50 times and calculate the average number of change points for each combination. The results are shown in the tables below. (Due to response length limitations, we only report the results for Setting 1. The results for Setting 2 are similar, and we will include the full results in the revised version.)\n\n   **Estimated change point number for different values of $\\gamma_1$ and $\\gamma_2$ for setting 1:**\n\n   | $\\gamma_1$          | 0.02 | 0.04 | 0.06 | 0.08 | 0.1  | 0.2  | 0.4  | 0.6  | 0.8  | 1    | 2    | 4    | 6    | 8    | 10   |\n   |---------------------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|\n   | $\\gamma_2$=0.002    | 2.12 | 2.1  | 2.22 | 2.22 | 2.22 | 2.04 | 1.58 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 |\n   | $\\gamma_2$=0.004    | 2.1  | 2.08 | 2.2  | 2.2  | 2.2  | 2.02 | 1.58 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 |\n   | $\\gamma_2$=0.006    | 2.1  | 2.08 | 2.2  | 2.2  | 2.2  | 2.02 | 1.58 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 |\n   | $\\gamma_2$=0.008    | 2.1  | 2.08 | 2.2  | 2.2  | 2.2  | 2.02 | 1.58 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 |\n   | $\\gamma_2$=0.01     | 2.1  | 2.08 | 2.2  | 2.2  | 2.2  | 2.02 | 1.58 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 |\n   | $\\gamma_2$=0.02     | 0    | 2.06 | 2.18 | 2.18 | 2.18 | 1.96 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 | 1.54 |\n   | $\\gamma_2$=0.04     | 0    | 0    | 2.1  | 2.14 | 2.14 | 1.9  | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 |\n   | $\\gamma_2$=0.06     | 0    | 0    | 0    | 2.14 | 2.14 | 1.86 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 |\n   | $\\gamma_2$=0.08     | 0    | 0    | 0    | 0    | 2.12 | 1.82 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 |\n   | $\\gamma_2$=0.1      | 0    | 0    | 0    | 0    | 0    | 1.82 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 |\n   | $\\gamma_2$=0.2      | 0    | 0    | 0    | 0    | 0    | 0    | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 |\n   | $\\gamma_2$=0.4      | 0    | 0    | 0    | 0    | 0    | 0    | 0    | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 |\n   | $\\gamma_2$=0.6      | 0    | 0    | 0    | 0    | 0    | 0    | 0    | 0    | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 |\n   | $\\gamma_2$=0.8      | 0    | 0    | 0    | 0    | 0    | 0    | 0    | 0    | 0    | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 |\n   | $\\gamma_2$=1        | 0    | 0    | 0    | 0    | 0    | 0    | 0    | 0    | 0    | 0    | 1.52 | 1.52 | 1.52 | 1.52 | 1.52 |\n   ---\n\n   From the table above, we observe that the number of estimated change points varies with different values of $\\gamma_1$ and $\\gamma_2$. Specifically, we find that $\\gamma_1$ should be greater than $\\gamma_2$ to ensure effective change point detection, which is consistent with our theoretical analysis in Theorem 3.10. Moreover, for a fixed $\\gamma_1$, the estimated number of change points decreases as $\\gamma_2$ increases. This aligns with our intuition, as larger values of $\\gamma_2$ imply a higher penalty for each additional change point. Overall, while there is some effect of the tuning parameters on the change point estimation, the estimated number of change points remains relatively stable, even with substantial variations in the scales of $\\gamma_1$ and $\\gamma_2$.\n\n3. *\"The main weakness so far lies in the presentation. While the theoretical sections are mathematically sound, they are dense and lack intuitive explanation, making the paper difficult to follow. Including a complete notation table and illustrative diagrams could significantly enhance clarity.\"*\n\n   **Response:**\n\n   Thank you for your comment. We appreciate the feedback. To enhance the readability and intuitiveness of the theoretical sections, we will make the following improvements in the revised version.\n\n   First, we will include the following table summarizing the key notations used throughout the paper.\n\n\n   **Notation table**\n   | **Symbol**        | **Description**                                            |\n   |-------------------|------------------------------------------------------------|\n   | $[n]$             | Set of integers $\\{1, \\ldots, n\\}$                        |\n   | $\\mathbf{I}$      | Identity matrix                                            |\n   | $\\mathbf{e}_n$    | $n \\times 1$ vector with all elements equal to 1           |\n   | $\\mathbf{0}$      | Zero vector or matrix                                      |\n   | $\\|\\mathbf{v}\\|_2$| $\\ell_2$-norm of vector $\\mathbf{v}$                       |\n   | $\\|f\\|_{2,T}$    | $L_2$ norm of function $f(\\cdot)$ over the interval $[0,T]$|\n   | $\\mathcal{Y}$     | Set of pairwise comparison results                         |\n   | $y_{ij}(t)$       | Comparison result at time $t$ between items $i$ and $j$    |\n   | $\\mathbf{\\pi}^*(t)$ | Score vector of items at time $t$ in the Bradley-Terry model |\n   | $y_{ij}^*(t)$     | Winning probability between items $i$ and $j$ at time $t$  |\n   | $K(\\cdot)$        | Kernel function                                            |\n   | $\\mathbf{P}(t)$   | Transformation probability matrix for the Markov chain     |\n   | $B$               | Number of groups                                           |\n   | $G$               | Group partition of items                                   |\n   | $\\delta_1$        | Minimum pairwise score difference between groups           |\n   | $\\delta_2$        | Maximum pairwise score difference within group             |\n   | $\\lambda$         | Tuning parameter for group recognition                     |\n   | $\\mathbf{Q}$      | Constant matrix used for transformations                   |\n   | $\\mathbf{\\theta}(t)$ | Transformation of the score vector $\\mathbf{\\pi}(t)$      |\n   | $\\mathbf{X}(t)$   | Matrix after transformation used in optimization objective for group recognition |\n   | $\\mathbf{Y}(t)$   | Vector after transformation used in optimization objective for group recognition |\n   | $\\mathbf{z}(t)$   | Latent group of items at time point $t$                    |\n   | $\\eta_i$          | Time points where the structure changes                    |\n   | $\\xi_i$           | Candidate structure change points                          |\n   | $l(\\mathbf{\\pi}, t)$ | Negative log-likelihood function at time $t$ for score vector $\\mathbf{\\pi}$ |\n   | $\\mathcal{P}$     | Partition of time interval                                  |\n   | $\\gamma_1, \\gamma_2$ | Regularization parameters for the objective function     |\n\n   In addition, we will add a diagram to illustrate the entire workflow of our methodology in the revised version. Although we cannot include the diagram here due to the rebuttal rules for this year, we describe it in detail as follows. The diagram will first depict the data formality, which will be represented by a comparison graph. This graph will show the preference results obtained from the pairwise comparisons. Next, we will illustrate the grouping method by using score curves, which show the score changes of each group and their relative ranking positions. Lastly, the structure detection procedure will be illustrated using a dynamic programming solution plot, structured as a tree to reflect the algorithm's design. In this plot, the nodes represent subproblems, the branches indicate recursive calls, and the leaves correspond to the base cases of our grouping method.\n\n   Besides, we will add more transitions and intuitive interpretations of the theoretical results and key terms to enhance the readability in the revised version.\n\nAgain, thank you very much for your time and valuable comments."}, {"review_id": "qVNa6zucVX", "reviewer": "Reviewer_kwzX", "summary": "This paper considers the problem of ranking structure recognition for preference-based data under the Bradley-Terry model, where the ranking structure could undergo changes over time. It first studies ranking group recovery for a given interval by introducing an optimization objective that can be formulated as the group lasso problem. Then, it applies the proposed clustering method to recognition of group changes by considering a combinatorial optimization problem. An algorithm based on dynamic programming is developed. Under certain technical assumptions, recognition consistency and estimator uncertainty results are established. Computational experiments are provided to evaluate the proposed framework.", "questions": "See above.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThe work proposes a framework for analyzing ranking structure recognition where the underlying cluster structure may experience changes, which is interesting and has potential applications to various areas.\nIt designs an objective function that allows for recovering the unknown change point of structures, and develops a DP-based algorithm to solve the combinatorial problem. I found this part inspiring.\nWeakness:\nWhile Theorem 3.6 quantifies the uncertainty of item ability estimations, can you comment on the optimality of the result?\nNonasymptotic recovery guarantees have received considerable attention in the recent literature of ranking. Do you expect such guarantees to hold for your proposed method?\nSome notation choices may cause confusion for readers. For instance, on line 85: “\\|f(t)\\|_{2,T}=..., where $f(t)$ is a function on $[0,T]$”. Typically, $f(t)$ is perceived as a value and it might be more appropriate to use $f$ inside the norm.\nOn line 129: would it be ““induced by \\tilde{\\pi}” instead?", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank you for your detailed and constructive comments. In light of the comments, we have revised the paper, which we believe has significantly improved both its quality and presentation. Below, we respond to the comments and suggestions point-by-point.\n\n1. *\"While Theorem 3.6 quantifies the uncertainty of item ability estimations, can you comment on the optimality of the result?\"*\n\n   **Response:**  \n   Thank you for your question. Theorem 3.6 shows that the estimation error satisfies $||\\widehat{\\mathbf{\\pi}}\\_{G}(t)-\\mathbf{\\pi}^{\\*}\\_{G}(t)||_{\\infty} = O\\_p((n^2Mh)^{-1/2})$, which matches the optimal rate. Specifically, for the refitted estimator $\\widehat{\\pi}_i^{\\text{rf}}$ defined in equation (2.3), we obtain the relative error rate $||\\widehat{\\mathbf{\\pi}}^{\\text{rf}}(t)-\\mathbf{\\pi}^{\\*}(t)||\\_{\\infty}/||\\mathbf{\\pi}^{*}(t)||\\_{\\infty} = O\\_p((n^2Mh)^{-1/2})$. We analyze the result based on the effective sample size. Note that since we pool comparisons across all items in the same group to estimate each ability score, the estimation leverages this aggregated information. On average, there are $n/B$ items per group, each compared against roughly $(B-1)\\cdot n$ others. Assuming the Epanechnikov kernel with bandwidth $h$, each pair contributes about $2Mh$ effective observations. Hence, the total number of comparisons used to estimate each $\\widehat{\\mathbf{\\pi}}^{\\text{rf}}_i(t)$ is of order $n^2Mh$. Therefore, the estimation rate $(n^2Mh)^{-1/2}$ matches the optimal rate $L^{-1/2}$ established in Chen et al. (2019) and Karlé and Tyagi (2023), where $L$ denotes the average number of comparisons per item.\n\n   **References**\n\n   Chen, Y., J. Fan, C. Ma, and K. Wang (2019). Spectral method and regularized MLE are both optimal for top-K ranking. *Annals of Statistics* 47 (4), 2204–223.\n\n   Karlé, E., & Tyagi, H. (2023). Dynamic ranking with the BTL model: a nearest neighbor based rank centrality method. *Journal of Machine Learning Research* 24 (269), 1–57.\n\n2. *\"Nonasymptotic recovery guarantees have received considerable attention in the recent literature of ranking. Do you expect such guarantees to hold for your proposed method?\"*\n\n   **Response:**  \n   Thank you for your insightful comment. Yes, we believe nonasymptotic recovery guarantees are attainable for our proposed methods. In Section 3.2, we introduce the group inverse of a Markov chain, which plays a key role in deriving nonasymptotic expansions for the spectral estimator. Building on Theorem S.1 in Lu et al. (2024), we can first expand the estimator expressed through the group inverse using properties of the Markov chain. Then, following Lemma S.1 in Lu et al. (2024), we approximate the group inverse using the relative ability scores. By substituting this approximation and carefully controlling the approximation error and remainder terms, we can obtain a nonasymptotic expansion of the estimator. While a rigorous derivation requires additional calculations, the direction is promising and deserves further study.\n\n   **References**\n\n   Lu, N., Shi, J., Tian, X.-Y., & Song, K. (2024). Tests on Dynamic Ranking. *Statistica Sinica*. DOI: 10.5705/ss.202024.0153\n\n3. *\"Some notation choices may cause confusion for readers. For instance, on line 85: '$||f(t)||\\_{2,T}=...$, where $f(t)$ is a function on $[0,T]$'. Typically, $f(t)$ is perceived as a value and it might be more appropriate to use $f$ inside the norm.\"*\n\n   **Response:**  \n   Thanks for pointing out this. We will revise it to $||f||\\_{2,T}$.\n\n4. *\"On line 129: would it be 'induced by $\\widetilde{\\mathbf{\\pi}}$' instead?\"*\n\n   **Response:**  \n   Thank you for your careful reading. We will correct it accordingly.\n\nAgain, thank you very much for your time and valuable comments."}, {"review_id": "IifUmrSzEl", "reviewer": "Reviewer_Qzoa", "summary": "This paper introduces a method for recognizing dynamic ranking structures in preference-based data, where instead of fixed scores, pairwise comparisons evolve over time.The authors build on the Bradley-Terry model and develop a technique that can:\n- Group items that behave similarly over a time interval,\n- Track how those groups change over time, and\n- Rank the groups and their members in a meaningful way.\nThey tackle the challenges of noisy, time-varying data by combining spectral methods with optimization tricks like adaptive group lasso. \nExperiments on NBA data show that their method produces interpretable, accurate rankings and captures real-world structural shifts like team trades.", "questions": "NA", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 4, "strengths_and_weaknesses": "Strengths:\n- The paper introduces a dynamic framework for ranking and grouping, which is a step beyond static models and is very practical in real-world utility.\n- The proposed method combines spectral methods, adaptive group lasso, and dynamic programming in a coherent way, and apprears to be theoratically guaranteed for group recognition and change point detection.\n\nWeaknesses:\n- The paper is very hard to read with multiple theorems, remarks, assumptions. This creates barriers for reader from understanding the content. Apart from this, one of the main contribution is confusing in lacking explanation of the terms \"consistency recognition of groups\", \"structure recognition consistency\", \"uncertainty of item ability estimators\". In addition, the figures are hard to read with font size being too small. Overall, I find great space left to improve the readability of the paper.\n- An essential weakness of the paper is its strong reliance on smoothness and homogeneity assumptions in the latent ability trajectories. The method assumes that items within a group have similar, smoothly varying abilities over time, which may not hold in many real-world scenarios, especially in volatile domains like finance, user preferences, or competitive gaming. This assumption is critical to both the group identification and change point detection. If it is violated, the model's validity and interpretability could break down.", "was_revised": true, "final_justification": "The authors paritially address my concern.", "rebuttal": "We would like to thank you for your detailed and constructive comments. In light of the comments, we have revised the paper, which we believe has significantly improved both its quality and presentation. Below, we respond to the comments and suggestions point-by-point.\n\n1. *\"The paper is very hard to read with multiple theorems, remarks, assumptions. This creates barriers for reader from understanding the content. Apart from this, one of the main contribution is confusing in lacking explanation of the terms 'consistency recognition of groups', 'structure recognition consistency', 'uncertainty of item ability estimators'. In addition, the figures are hard to read with font size being too small. Overall, I find great space left to improve the readability of the paper.\"*\n\n   **Response:**  \n   Thank you for pointing out this. In the revised version, we will enhance the readability of the paper by adding clearer explanations, smoother transitions, and more intuitive interpretations of the theoretical results and key terms.\n\n   Specifically, at the beginning of Section 3.1, we will add the following paragraph to clarify the main goal:  \"We will present theoretical guarantees for our estimator in this section. Specifically, we show that the probability of identifying the correct underlying group structure approaches one as the sample size increases. We refer to this property as consistency recognition of groups, which is a desirable feature that supports the reliability of the estimation method.\"\n\n\n   After Theorem 3.3, we will include a transition to help guide the reader: \"We have now established the consistency recognition of groups. The following two remarks clarify the conditions of the theorem and highlight the distinct features of our theoretical analysis.\"\n\n   In Section 3.2, we will add the following clarification at the beginning: \"In this section, we discuss the uncertainty quantification, that is, the asymptotic distribution of the item ability estimators. A well-characterized uncertainty quantification allows for statistical inference tasks such as hypothesis testing and helps assess the reliability of the estimator.\"\n\n   In addition, at the beginning of Section 3.3, we will include this explanation: \"In this section, we focus on structure recognition consistency, which refers to the probability of correctly identifying the group structure change points approaching one as the sample size increases. This property is important as it ensures reliable detection of structural changes in the data. We first provide a general analysis for arbitrary grouping methods in Theorem 3.10, and then specialize the discussion to the dynamic ranking setting in Corollary 3.11.\"\n\n   Additionally, we will improve the figures by enlarging the font size and adjusting the layout to enhance readability.\n\n2. *\"An essential weakness of the paper is its strong reliance on smoothness and homogeneity assumptions in the latent ability trajectories. The method assumes that items within a group have similar, smoothly varying abilities over time, which may not hold in many real-world scenarios, especially in volatile domains like finance, user preferences, or competitive gaming. This assumption is critical to both the group identification and change point detection. If it is violated, the model's validity and interpretability could break down.\"*\n\n   **Response:**  \n   Thank you for your insightful comments.  \n\n   First, we would like to clarify that the smoothness condition in Assumption 3.1 is standard in dynamic settings and can be relaxed.  \n\n   In the recognition of dynamic ranking groups section, it is sufficient for the score functions to be Lipschitz continuous to ensure consistent group identification (Theorem 3.3). The stronger smoothness assumption is only used to derive the asymptotic distribution of the estimators (Theorem 3.6). Importantly, the Lipschitz condition is widely adopted in theoretical analysis of dynamic ranking problems, such as Assumption 5.2 of  Bong et al. (2020), and Assumption 1 of Karlé and Tyagi (2023).  \n\n   Furthermore, in the recognition of group changes section, the general theoretical results in Theorem 3.10 rely only on Assumptions 3.8 and 3.9, which do not require the ability trajectories to be smooth. The smoothness condition in Corollary 3.11 is imposed solely to facilitate the application of Theorem 3.3, but as noted above, this can be weakened to a Lipschitz condition. Alternatively, one may use a segmented estimation strategy over a gridded time interval to estimate $\\widehat G(\\mathcal{I})$, and then apply our proposed group change detection framework, without requiring any smoothness assumption. We fully agree that nonsmooth, even abrupt, changes may occur in real-world scenarios. In fact, this is one of the main motivations behind our development of the group structure change detection framework, which is designed to effectively handle such irregularities in the data.  \n\n   Secondly, the homogeneity condition (lines 115–116) is relatively mild. Many prior works assume that parameters are exactly equal within each group, for example, Assumption 1 in Tian and Shi (2023), whereas we allow for within-group variability and focus instead on identifying major structural differences.  \n\n   Finally, we would like to emphasize that our method performs well even in the presence of abrupt changes. This is supported by simulation results in Section 4.2, where the underlying score trajectories (shown in Figures 4 and 5 in Section A.2) feature non-smooth and abrupt changes, yet our method maintains strong performance.  \n\n   **References**  \n   Bong, H., Li, W., Shrotriya, S., & Rinaldo, A. (2020, June). Nonparametric estimation in the dynamic Bradley-Terry model. In *International Conference on Artificial Intelligence and Statistics* (pp. 3317-3326). PMLR.  \n\n   Karlé, E., & Tyagi, H. (2023). Dynamic ranking with the BTL model: a nearest neighbor based rank centrality method. *Journal of Machine Learning Research*, 24(269), 1-57.  \n\n   Tian, X. Y., & Shi, J. (2023). Grouped rank centrality: Ranking and grouping from pairwise comparisons simultaneously. *Stat*, 12(1), e626.  \n\nAgain, thank you very much for your time and valuable comments."}]

Author rebuttal:

Meta-review: The paper propose a ranking method that can track changes in preferences over time.

All reviewers agree that the paper makes interesting and publishable contributions. The main concern is that it is hard to read. The authors have promised a list of modifications to improve readability, and both the reviewers and I feel satisfied that these changes will be helpful. I urge the authors to make their best effort to implement them.
Decision: Accept (poster)