Title: Active Seriation: Efficient Ordering Recovery with Statistical Guarantees
Venue: NeurIPS 2025
Accepted: yes
Keywords: Seriation

Sorting

Active learning

Ranking
Abstract: Active seriation aims at recovering an unknown ordering of $n$ items by adaptively querying pairwise similarities. 
The observations are noisy measurements of entries of an underlying $n \times n$ permuted Robinson matrix, whose permutation encodes the latent ordering. The framework allows the algorithm to start with partial information on the latent ordering, including seriation from scratch as a special case. We propose an active seriation algorithm that provably recovers the latent ordering with high probability. Under a uniform separation condition on the similarity matrix, optimal performance guarantees are established, both in terms of the probability of error and the number of observations required for successful recovery.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the problem of recovering the correct order of items using noisy pairwise similarity comparisons, where the comparisons are chosen actively. It introduces a simple and efficient algorithm called ASII (Active Seriation by Iterative Insertion), which incrementally builds the ordering using binary search with backtracking. The authors provide theoretical guarantees showing that ASII achieves optimal recovery rates under certain signal-to-noise conditions, and they validate its performance through simulations.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths – \n1.\tThe paper introduces and formalizes the problem of active seriation, which is a natural and important extension of classical seriation to settings where pairwise similarities can be queried adaptively.\n2.\tThe authors provide clear and tight theoretical guarantees. They show exactly when recovery is possible and when it’s not, and their algorithm matches these limits up to constants.\n3.\tThe link to multi-armed bandits, especially thresholding bandits, is clever and helps ground the analysis. \n4.\tThe paper does a good job of providing intuitive explanations alongside the formal results. For example, the discussion around how the minimal gap (delta) and the noise level (sigma) affect the ability to recover the true ordering helps the reader build a solid understanding of the underlying difficulty of the problem. These kinds of insights make the theoretical contributions more accessible and meaningful.\n5.\tThe supplementary material includes detailed and well-organized proofs. It’s clear the authors have done a careful job with the analysis.\n6.\tThe paper is well-written and easy to follow. Definitions, algorithms, and results are all clearly presented.\n\n\n\nWeaknesses – \n1.\tAll experiments are conducted on synthetic data. The absence of real-world datasets makes it difficult to assess the practical utility and robustness of the method.\n2.\tWhile the paper compares ASII to Adaptive Sorting and Spectral Seriation, it does not benchmark against more recent or diverse active learning or ranking algorithms.\n3.\tThe algorithm and its guarantees (look like) assume knowledge of the noise variance (sigma), which may not be available or easy to estimate in practice. In the ‘Test’ subroutine, the number of samples T_0 used per test is chosen to ensure high-probability correctness, which should depend on the noise level (sigma).\n4.\tWhile the algorithm is adaptive to the unknown minimal gap (delta), it does not adapt to unknown noise levels. This limits its applicability in settings where sigma is not known or cannot be estimated reliably.\n5.\tWhile ASII performs well when the minimal gap (delta) is reasonably large and uniform, its reliance on local comparisons makes it less effective when (delta) is very small or varies significantly across the matrix. In such cases, batch algorithms like Adaptive Sorting, which leverage aggregate information across the entire matrix, can be more robust. This is evident in the experimental results (e.g., Scenario 2), where ASII’s performance degrades more sharply than that of its batch counterparts.\n\nQUESTIONS:\n1.\tCould the authors clarify whether the algorithm assumes sigma is known, and if not, how T_0 is chosen in practice? If sigma is unknown, can the algorithm be made adaptive to it?\n2.\tThe algorithm’s performance degrades when the minimal gap (delta) is small or varies across the matrix, while batch methods like Adaptive Sorting are more robust in such cases. Can the authors comment on whether ASII could be modified to incorporate more global or aggregate information to improve robustness in heterogeneous matrices? Even a heuristic or hybrid approach that combines local and global signals could be valuable.\n3.\tCould ASII be extended to handle partial or missing similarity data?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the problem of seriation, a noisy variant of sorting, from an active learning angle. The basic setting is as follows. There is an unknown ordering/ranking of $n$ items, where items that are closer to each other in the ranking are more similar than ones that are far apart. Specifically the similarities are encoded by a Robinson matrix, which is a symmetric matrix $A$ that is \"monotone toward the diagonal\". That is, any path in $A$ that starts at the entry $A_{n,1}$, ends at a diagonal entry, and only goes upward or to the right is monotone. Since the ordering is unknown, the algorithm only gets noisy access to some fixed permuted copy $M$ of such a Robinson matrix.\n\nIn the active seriation problem, the algorithm is active/adaptive, and in each of the $T$ rounds it selects a pair $i,j$ and obtains an independent draw from a $\\sigma$-subgaussian random variable of mean $M_{ij}$. The goal is to retrieve the permutation (over the rows) exactly.\nThis paper assumes that there is a universal lower bound $\\Delta$ on the similarities between elements and obtains tight results parameterized by this assumption. Specifically, the authors identify the SNR $\\Delta^2 T / \\sigma^2 n$ as the key quantity. They show that when this quantity is smaller than a constant times $\\log n$, any algorithm fails with constant probability. On the other hand, they devise an algorithm whose failure probability is of order $e^{-O(SNR)}$. The algorithm is based on noisy binary search and includes an interesting ingredient of backtracking, that goes backwards in the search if it detects a possible violation. I did not go over the analysis (which appears only in the appendix).\n\nSTRENGTHS AND WEAKNESSES:\n+ The studied problem is interesting, and natural for an active approach.\n\n- Technical novelty is not very high. Noisy binary search is probably the first technique that one would try in such a setting, and the analysis does not seem to carry with it very deep lessons or surprises.\n\n- The dependence in the minimum similarity $\\Delta$ seems rather limiting. For example, it could be the case that we rank a large set of elements where some pairs of them are near-identicals, and we wish to get a good ranking at least for those elements that are far enough. It would be better if you can extend your results to this regime (which I believe shouldn't be too hard).\n\n-+ There are some issues in terms of writing, although overall it is decent. One example of an issue is my first question to the authors (which claims to Theorems 3.1 and 3.2 are not completely compatible with each other). Also, given that the technical analysis is claimed to be the main contribution in this work, at least a snippet of it should appear in the main body.\n\nQUESTIONS:\n* There seems to be incompatibility between Theorem 3.1 and 3.2, specifically in the $\\log n$ term. In Theorem 3.1 you claim that any algorithm where the SNR is less than $\\log n$ fails with constant failure probability, whereas in Theorem 3.2 you obtain an algorithm that achieves constant failure prob already when the SNR is a constant. Please explain how these settings differ.\n\n* Why does the backtracking work? I was curious about more details about the theoretical analysis (and in contrast am not that interested in the experiments). Backtracking is not that common in noisy binary search approaches, and introducing such a technique could be a valuable contribution in your paper, which you don't emphasize enough.\n\n* How would your results change if you only asked for a ranking that is approximately monotone?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

comments: SUMMARY:
This paper studies the active seriation problem: i) there are n items with a hidden true ranking; ii) the learner can query two items each time, and each query returns a noisy sample of the similarity of them; iii) If in the true ranking, two items are closer, then the similarities are closer. The learner wants to recover the full ranking with high probability by actively selecting pairs to query, and tries to use the least amount of queries. 

The authors gives the lower bound and upper bound on the sample complexity (O(n\log(n)), and proposes a new algorithm achieving the optimal sample complexity.

STRENGTHS AND WEAKNESSES:
Strengths
-The problem studies in this paper is interesting and fundamental in my opinion, and the authors give clear and optimal results on the lower and upper bounds. The settings are more general than previous works. Overall, the results and novelty of this paper are pretty good. I find no major flaws

Weakness
-For readers not quite familiar with this problem, maybe give more concrete real-world examples about this paper's setting.

QUESTIONS:
Maybe give some concrete examples to explain the real-world use case of this paper's setting?

LIMITATIONS:
No negative social impacts found
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper explores the active seriation problems. On the one hand, it demonstrates the impossibility of solving active seriation problems when the SNR value is smaller than $\\log n$ orderwise, where $n$ denotes the number of items, indicating the difficulty of this problem. On the other hand, when SNR exceeds $\\log n$, this paper proposes Active Seriation by Iterative Insertion (ASII) algorithm, which matches the lower bound of the error metric. In addition, some numerical results also validate the effectiveness of the proposed algorithm.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. This discussion of SNR $\\sim \\log n$ and the results on lower bound make this paper complete. When SNR is small, the difficulty of active seriation problems is well clarified, delineating the boundaries of problem solvability. In addition, the proposed lower bound strengthens the theoretical foundation of ASII algorithm.\n\n2. The proposed ASII algorithm is both elegant and effective. Although the underlying idea is quite simple, the analysis is concrete, the detailed design, such as the improved binary search, is dedicated, the theoretical results are strong (matching the lower bound orderwisely), and the numerical results are also convincing.\n\nWeaknesses:\n\n1. The presentation is somewhat unclear. The structure of the paper is confusing, for example, the theoretical analysis of ASII algorithm appears before the statement of the algorithm itself. Moreover, many typos lead to confusions and reduce readability, e.g., (1) Should $\\Delta(M)$ be $\\Delta_M$ in Line 126? (2) How many scenarios are considered in numerical experiments, as the caption of Fig. 2 seems inconsistent with the discussion? (3) There are some minor grammar mistakes, such as “exists” in Line 274.\n\nQUESTIONS:\nPlease try to answer the following questions:\n\n1. (Main) I believe the active seriation problem is closely related to MAB problems based on my experience: The sub-Gaussian assumption, noisy observations, and the form of the upper and lower bounds are quite similar to those in the MAB literature. Although the authors also briefly mentioned this point in Line 55, the subsequent discussion is missing. My questions regarding this connection are twofold: (1) Theoretically, can the authors provide more discussions about their similarities and differences? I wonder if this problem can even be formulated as an MAB problem? If not, what is the key distinction? (2) Experimentally, can some popular MAB algorithms such as UCB, TS, be adopted as baselines here? A comparison with these methods might help demonstrate the effectiveness of the proposed algorithm.\n\n2. For the experiments: (1) The Fig. 3 cannot reflect how parameters $\\sigma, n, T$ impact the performance of ASII, which are key components in the theorem. (2) In scenario 3, AS algorithm dramatically outperforms the others, and achieves small errors ($\\le 0.2$) when $\\beta$ is small. Does this violate Theorem 3.1? (3) Can the authors provide the error bar to show the robustness of the proposed algorithm?\n\n3. Can the authors give some real-world applications to strengthen the significance of this paper?\n\nLIMITATIONS:\nPlease focus on Questions. I think there are no additional limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}