SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In each verification phase, as we traverse the dirty search in reverse order to locate  $t^*$ , at most  $T - t^* + 2$  clean comparisons suffice. In each clean search phase, the expected number of clean comparisons is the expected number of steps in clean search. Therefore, based on Lemma A.3 and Lemma A.5, the total number of clean comparisons performed in both phases is  $O\left(\sum \log (\eta_i + 2)\right)$ .
Additionally, the running time is dominated by the number of dirty and clean comparisons.
Remark A.6. The algorithm can be implemented such that the number of clean comparisons is at most that of quicksort plus  $O(n \log \log n)$ , regardless of prediction error. Thus, even with terrible predictions our algorithm matches the performance of quicksort up to a factor that tends to 1 as  $n \to \infty$ .
To achieve this, we can implement the verification step by decreasing  $t$  in geometrically increasing step sizes until a valid  $t$  has been found, and then perform a binary search for  $t^*$  between the last two attempted values of  $t$ . This reduces the number of clean comparisons in line 9 from  $O(T - t^*)$  to  $O(\log (T - t^*))$ , which is at most  $O(\log \log n)$  in expectation by Lemma A.2, and at most  $O(n\log \log n)$  for all verification steps together. The remaining clean comparisons are performed during the clean searches. In the worst case (when all dirty comparisons are incorrect) all clean searches start from the root, and together they perform exactly the same set of comparisons as quicksort (by a coupling argument between the random choices of the two algorithms: E.g., the root of the search tree corresponds to the initial uniformly random pivot of quicksort).
B Extentions of Dirty Comparison Setting
We consider two extensions of the dirty comparison setting. The first one assumes that dirty comparisons are probabilistic; the second one discusses the setting where multiple predictors are available. We briefly discuss the extension of our algorithms and results in these new settings.
B.1 Probabilistic Dirty Comparisons
Our algorithm extends to the case where dirty comparisons are probabilistic. Assume that for each pair of objects  $i, j$ , the dirty comparison between them yields an incorrect result with probability  $\eta_{ij}$ . Algorithm 1 can be directly applied in this setting, achieving the same guarantees by defining  $\eta_i = \sum_j \eta_{ij}$ . The proof remains unchanged.
If repeatedly querying the same dirty comparison multiple times yields independent results, the number of clean comparisons can be further reduced: Let  $\epsilon_{ij} \coloneqq \min \{\eta_{ij}, 1/2\}$ . When querying a dirty comparison  $2k$  times, the probability that the correct answer fails to secure a majority vote is at most  $(4\epsilon_{ij}(1 - \epsilon_{ij}))^k$ : For  $\eta_{ij} \geq 0.5$ , this bound is trivial. Otherwise, there are  $2^{2k}$  strings of length  $2k$  over the alphabet  $\{\text{correct, incorrect}\}$ , and each string that is at least half incorrect has probability at most  $(\epsilon_{ij}(1 - \epsilon_{ij}))^k$ .
So by repeating each dirty comparison query  $2k$  times, we obtain an algorithm that performs  $O(kn\log n)$  dirty comparisons and  $O\left(\sum_{i}\log \left(\sum_{j}(4\epsilon_{ij}(1 - \epsilon_{ij}))^{k}\right)\right)$  clean comparisons.
B.2 Multiple Predictors
We now discuss the setting where multiple predictors are available and prove Theorem 1.2. Suppose we have  $k$  different dirty comparison predictors. Let  $\eta_i^p$  denote the number of incorrect comparisons by predictor  $p$  for item  $i$ .
We prove Theorem 1.2 by reduction to the problem of "prediction with expert advice": In this problem, there are  $k$  experts, and each incurs a loss in the range [0, 1] per time step. An algorithm must select an expert in each round before the losses are revealed and then incurs the loss of the chosen expert. According to (Freund and Schapire, 1997, Equation(9)), their algorithm HEDGE has an expected loss of  $O(L + \log(k))$ , where  $L$  is the total loss of the best expert in hindsight.
In our case, the experts correspond to the predictors, and time steps correspond to the  $n$  iterations of the for-loop of Algorithm 1 where an item is inserted into the BST. We define the loss of expert  $p$  in the time step where  $a_i$  ought to be inserted by  $\ell_i^p = \log(1 + \tilde{\eta}_i^p) / \log(n)$ , where  $\tilde{\eta}_i^p \leq \eta_i^p$  is the number of incorrect comparisons of predictor  $p$  between  $a_i$  and the items already in the BST at this time. Note that the value of  $\tilde{\eta}_i^p$  can be determined at the end of the time step (once  $a_i$  is correctly placed in the BST) without any additional clean comparisons; the division by  $\log n$  ensures that  $\ell_i^p \in [0,1]$  as required.
The algorithm for multiple predictors proceeds as Algorithm 1, querying for dirty comparisons at a given time step the predictor  $p$  corresponding to the expert chosen by HEDGE at that time step. Recall from the analysis in Section 2.2 that the expected number of clean comparisons in this time step is then  $O(\log (1 + \tilde{\eta}_i^p))$ , which is an  $O(\log n)$  factor larger than the loss suffered by HEDGE. Consequently, by the  $O(L + \log k)$  bound on the cost of HEDGE, the total expected number of clean comparisons is  $O(\min_p\sum_i\log (1 + \eta_i^p) + \log (k)\log (n))$ . Since  $k\leq 2^{O(n / \log n)}$ , the term  $\log (k)\log (n) = O(n)$  is negligible, and Theorem 1.2 follows.
C Proof of Comparison Complexity of Displacement Sort
Proof of Theorem 1.3. We focus on the insertion process of each item  $a_i$  in Algorithm 2. Let  $d_i$  denote the number of nodes between  $a_i$  and  $a_{i-1}$  in an inorder traversal of the tree after inserting  $a_i$ , including themselves. These nodes must have their correct ranking in the final sorted list between  $p(i-1)$  and  $p(i)$ ; hence
$$
d _ {i} \leq | p (i) - p (i - 1) | + 1, \text {f o r a l l} i = 2, \dots , n.
$$
Therefore, the running time and number of comparisons among all insertions are bounded by
$$
\begin{array}{l} \sum_ {i = 2} ^ {n} O (\log (d _ {i})) \leq \sum_ {i = 2} ^ {n} O (\log (| p (i) - p (i - 1) | + 1)) \\ \leq \sum_ {i = 2} ^ {n} O (\log (| p (i) - \hat {p} (i) | + | p (i - 1) - \hat {p} (i - 1) | + \hat {p} (i) - \hat {p} (i - 1) + 1)) \\ \leq \sum_ {i = 2} ^ {n} O (\log (3 \cdot \max \{\eta_ {i} ^ {\Delta} + 1, \eta_ {i - 1} ^ {\Delta} + 1, \hat {p} (i) - \hat {p} (i - 1) + 1 \})) \\ \leq O (n) + \sum_ {i = 1} ^ {n} O (\log (\eta_ {i} ^ {\Delta} + 1)) \\ \leq O \left(\sum_ {i = 1} ^ {n} \log \left(\eta_ {i} ^ {\Delta} + 2\right)\right). \\ \end{array}
$$
The second inequality is by  $\hat{p}(i - 1) \leq \hat{p}(i)$  and the triangle inequality; the third inequality is by monotonicity of logarithm. The penultimate inequality is justified by  $\log (\max \{x, y, z\}) \leq \log x + \log y + \log z$  for all  $x, y, z \geq 1$  and
$$
\sum_ {i = 2} ^ {n} \log (\hat {p} (i) - \hat {p} (i - 1) + 1) \leq \sum_ {i = 2} ^ {n} (\hat {p} (i) - \hat {p} (i - 1)) \leq n.
$$
This concludes the proof of Theorem 1.3.
D Proof of Comparison Complexity of Double-Hoover Sort
Lemma D.1. Upon the insertion of an item  $a_i$  into  $L$ , all items in  $L \setminus L^{<i}$  are larger than  $l_i$ . Similarly, when an item  $a_i$  is inserted into  $R$ , all items in  $R \setminus R^{>i}$  are smaller than  $r_i$ . As a result, the initial interval at line 8 and line 14 are subsets of  $L^{<i}$  and  $R^{<i}$ , and hence have sizes no larger than  $\delta$ .
Proof. Assume  $a_i$  is inserted into  $L$  in round  $\delta$ . Consider any  $a_j \in L \setminus L^{<i}$  at the time of  $a_i$  insertion. Then,  $a_j$  was inserted into  $L$  previously with a smaller insertion strength  $\delta' < \delta$ . By the insert condition,  $a_j > l_{\delta'}^j$ .
Since  $L^{<i}$  is a subset of  $L^{<j}$ , the  $\delta'$ th largest item of the latter set must exist and be no smaller than the  $\delta'$ th largest items of the former set. Then, we obtain
$$
a _ {j} > l _ {\delta^ {\prime}} ^ {j} \geq l _ {\delta^ {\prime}} ^ {i} \geq l ^ {i}.
$$
Hence, the interval  $\{x\in L|l_{\delta}^{i}\leq x\leq l^{i}\}$  only contains items in  $L^{<  i}$ . Since there are  $\delta$  items in  $L^{<  i}$  that are no smaller than  $l_{\delta}^{i}$ , the interval has at most  $\delta$  items. An analogous proof shows the symmetric property in  $R$ .
Theorem D.2. For each item  $a_{i}$ , its insertion process takes  $O(\log (\min \{\eta_i^l,\eta_i^r\} +2))$  comparisons.
Proof. Each item  $a_i$  goes through some rejected insertions in earlier rounds, and then gets inserted into  $L$  or  $R$  in a certain round. We refer to them as the exploration phase and insertion phase, respectively, and prove that the number of comparisons needed in both phases is bounded by  $O(\log (\min \{\eta_i^l,\eta_i^r\} +2))$ .
First, we claim that each  $a_{i}$  is inserted into either  $L$  or  $R$  prior to or during the round with insertion strength  $\delta_i^l \coloneqq 2^{\lceil \log (\eta_i^l + 2) \rceil}$ . If  $a_{i}$  is inserted before round  $\delta_i^l$ , the claim trivially holds. The claim also holds if at round  $\delta_i^l$ ,  $L^{< i}$  contains fewer than  $\delta_i^l$  items. In the absence of these conditions,
$$
\begin{array}{l} \left| \left\{a _ {j} \in L ^ {<   i}: a _ {j} > a _ {i} \right\} \right| = \left| \left\{a _ {j} \in L: j <   i \wedge a _ {j} > a _ {i} \right\} \right| \\ \leq \left| \left\{j \in [ n ]: \hat {p} (j) \leq \hat {p} (i) \wedge p (j) > p (i) \right\} \right| \\ = \eta_ {i} ^ {l} <   \delta_ {i} ^ {l}. \\ \end{array}
$$
Hence, in round  $\delta_i^l$ ,  $a_i$  must be larger than the boundary value, which is the  $\delta_i^l$ th largest item in  $L^{<i}$ . Consequently, it will be inserted into  $L$  in round  $\delta_i^l$ .
A similar argument shows that  $a_{i}$  will be inserted prior to or during round  $2^{\lceil \log (\eta_i^r +2)\rceil}$ . Combining these two bounds, we find that  $a_{i}$  must be inserted prior to or during round  $\delta_{i}\coloneqq 2^{\lceil \log (\min \{\eta_{i}^{l},\eta_{i}^{r}\} +2)\rceil}$ . Hence, the exploration phase only needs  $O(\log (\min \{\eta_i^l,\eta_i^r\} +2))$  comparisons.
Next, we continue to examine the number of comparisons needed in the insertion phase. Suppose  $a_{i}$  is inserted into  $L$  in some round  $\delta \leq \delta_{i}^{l}$ . Then, the binary search starts with an interval of size
$$
\left| \left\{x \in L \colon l _ {\delta} ^ {i} \leq x \leq l ^ {i} \right\} \right| \leq \delta \leq \delta_ {i} ^ {l} = O \left(\min  \left\{\eta_ {i} ^ {l}, \eta_ {i} ^ {r} \right\} + 2\right).
$$
The first inequality is due to Lemma 3.1. Hence, the insertion phase of  $a_{i}$  by binary search needs  $O(\log (\min \{\eta_i^l,\eta_i^r\} +2))$  comparisons.
E Lower Bounds on Comparison Complexity
We prove the lower bounds stated in Theorem 1.5.
E.1 Optimality of Displacement Sort
In this section, we show that an exact sorting algorithm, augmented with a positional prediction, cannot sort the array with  $o(\sum_{i\in [n]}\log (\eta_i^{\Delta}))$  comparisons.
Definition E.1. Given  $n$ , a positional prediction  $\hat{p}$ , and a real number  $U$ , define the size of the  $U$ -candidate set as
$$
\operatorname {c a n d} (\hat {p}, U) := \left| \left\{A \in S _ {n}: \sum_ {i = 1} ^ {n} \log \left(\eta_ {i} ^ {\Delta} + 2\right) \leq U \right\} \right|,
$$
where  $S_{n}$  is the set of permutations of  $[n]$  (viewed as an array),  $\eta^{\Delta}$  is calculated accordingly for each  $A$  against  $\hat{p}$ .
Theorem E.2. Given any function  $f(U) = o(\max_{\hat{p} \in S_n} \log (\operatorname{cand}(\hat{p}, U)))$ , there does not exist any positional augmented sorting algorithm with comparison complexity  $O(f(U))$  for instances with  $\sum_{i=1}^{n} \log (\eta_i^{\Delta} + 2) \leq U$ .
Proof. Given the predictor  $\hat{p}$  and an upper bound  $U$  on the error, a sorting algorithm needs to determine the correct permutation  $A$  from a candidate set of size  $\operatorname{cand}(\hat{p}, U)$ . If the algorithm only uses  $x$  comparisons, then it can only distinguish  $2^x$  different outcomes; if  $2^x$  is smaller than the number of candidates, it cannot determine the correct answer in every situation, since the information given by comparisons is not sufficient to distinguish all candidates. Hence, at least  $\lceil \log \operatorname{cand}(\hat{p}, U) \rceil$  comparisons are needed.
Theorem E.3. For all  $n \leq U \leq O(n \log n)$ , we have  $\max_{\hat{p} \in S_n} \log \operatorname{cand}(\hat{p}, U) = \Omega(U)$ .
Proof. Proof by construction. Assume without loss of generality that  $U$  is a multiple of  $n$  and let  $U' = U / n$ . Take  $\hat{p} = \langle 1,2,\dots,n \rangle$ , the identity prediction. We aim to construct sufficient number of candidate permutations  $A \in S_n$ , which all fall in the  $U$ -candidate set.
Consider every permutation  $A \in S_n$  constructed in the follow way: Initially, set  $A = \langle 1,2,\ldots ,n\rangle$ . Then, divide  $A$  into  $\frac{n}{2^{U'}}$  adjacent subarrays, each containing  $2^{U'}$  items (the last subarray is potentially smaller if there are not enough remaining elements). Finally, we permute the items in each subarray arbitrarily, and retrieve  $A$  after the permutation.
In each possible outcome  $A$ ,  $\log \eta_i^\Delta \leq U'$  for all  $i$  since each item is only permuted locally. Hence, the error of  $\hat{p}$  w.r.t.  $A$  is no larger than  $n \cdot U' = U$ . Each  $A$  falls inside the  $U$ -candidate set. Counting the number of possible different outcomes in our construction, we obtain
$$
\operatorname {c a n d} (\hat {p}, U) \geq \left[ \left(2 ^ {U ^ {\prime}}\right)! \right] ^ {\frac {n}{2 ^ {U ^ {\prime}}}},
$$
where the right-hand-side represents the possible ways to permute  $2^{U^{\prime}}$  items in each subarray.
By Stirling's formula,
$$
\log \operatorname {c a n d} (\hat {p}, U) \geq \Omega \left(\frac {n}{2 ^ {U ^ {\prime}}} \cdot \left(2 ^ {U ^ {\prime}} \cdot U ^ {\prime}\right)\right) = \Omega (U).
$$
Directly combining Theorem E.2 and Theorem E.3, and noting that at least  $\Omega(n)$  comparisons are always needed to verify correctness of a sorted list, we obtain the following:
Corollary E.4. For sorting with positional predictions, there exists no algorithm with comparison complexity  $o(\sum_{i=1}^{n} \log (\eta_i^{\Delta} + 2))$ .
E.2 Optimality of Dirty Comparisons Sort and Double-Hoover Sort
We can use the same definition of  $\mathrm{cand}(\hat{p},U)$  and construction as above. Every positional prediction can be view as a total linear relation on array  $A$ , therefore induces a unique dirty-comparison predictor. Since each element is locally perturbed, we can prove that for each permutation  $A$  obtained from the construction,  $\sum_{i = 1}^{n}\log (\eta_{i} + 2)$  and  $\sum_{i = 1}^{n}\log (\min \left\{\eta_{i}^{l},\eta_{i}^{r}\right\} +2)$  are also upper bounded by  $U$ . Hence, the optimality of Algorithm 2 and 3 can be proven in the exact same way, and Theorem 1.5 follows.
F.1 Synthetic Datasets with Varying Sizes
This section presents additional experimental results to provide a more comprehensive evaluation of our proposed algorithms. As scalability is a crucial aspect, we conducted further experiments on synthetic datasets with varying sizes from  $n = 1,000$  to  $n = 1,000,000$ . Results are shown in Figure F.1, F.2, and F.3.
In these figures, the shaded area denotes the standard deviation of each run among 30 repetitions. As the results show, our proposed algorithms perform consistently with relatively small deviations.
F.2 Comparisons against Insertion Sort
The performance of Insertion Sort against our proposed algorithms is shown in Figure F.4. As the plots show, insertion sort runs fast for small errors in the decay and country settings, though the performance declines steeply when predictions deteriorate (i.e., smoothness is worse, and robustness is non-existent). The experimental results suggest that in practical settings, it is a good idea to incorporate insertion sort as a fall-back option in case predictions are extremely accurate.
(a) class setting,  $n = 1,000$
(b) class setting,  $n = 10,000$
(c) class setting,  $n = 100,000$
(d) class setting,  $n = 1,000,000$
Figure F.1: Class Settings.
(a) decay setting,  $n = 1,000$
(b) decay setting,  $n = 10,000$
(c) decay setting,  $n = 100,000$
Figure F.2: Decay Settings.
(d) decay setting,  $n = 1,000,000$
(a) good-dominating,  $n = 1,000$
(b) good-dominating,  $n = 10,000$
(c) good-dominating,  $n = 100,000$
(d) bad-dominating,  $n = 1,000$
(e) bad-dominating,  $n = 10,000$
(f) bad-dominating,  $n = 100,000$
Figure F.3: Sorting with dirty comparisons, good- and bad-dominating settings
Figure F.4: Comparison with Insertion Sort