SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We know that the minimum 0-1 loss is attained by the Bayes predictor,  $f^{*}(x) = \mathbb{1}[\mu^{\prime}(x)\geq \mu (x)]$ , so
$$
\min  _ {f} \mathcal {L} (f) = \mathcal {L} \left(f ^ {*}\right) = 1 - \frac {1}{2} \int_ {\mathcal {X}} | \mu (x) - \mu^ {\prime} (x) | \mathrm {d} x \leq 1 - \frac {1}{R} W _ {1} (\mu , \mu^ {\prime}) \tag {2}
$$
by Eq. (1). The result then follows from an algebraic rearrangement of terms in Eq. (2).
Next, we provide a concrete example (supplementing the picture in Fig. 2) for showing that item-level alignment does not necessarily imply list-level alignment.
Example A.2. Consider two uniform distributions  $\mu_S^{\mathrm{list}},\mu_T^{\mathrm{list}}$  of two lists containing three real-valued items, where the lists are
$$
\operatorname {s u p p} \left(\mu_ {S} ^ {\text {l i s t}}\right) = \{(1, 2, 3), (4, 5, 6) \}, \quad \operatorname {s u p p} \left(\mu_ {T} ^ {\text {l i s t}}\right) = \{(1, 3, 5), (2, 4, 6) \}.
$$
The item-level distributions,  $\mu_S^{\mathrm{item}}$ ,  $\mu_T^{\mathrm{item}}$ , which are obtained by aggregating the items from all lists, are both the uniform distribution of the same six items:
$$
\operatorname {s u p p} \left(\mu_ {S} ^ {\text {i t e m}}\right) = \operatorname {s u p p} \left(\mu_ {T} ^ {\text {i t e m}}\right) = \{1, 2, 3, 4, 5, 6 \}.
$$
Note that item-level representations are automatically aligned since  $\mu_S^{\mathrm{item}} = \mu_T^{\mathrm{item}}$ , but the list-level representations are not, because  $\mathrm{supp}(\mu_S^{\mathrm{list}}) \cap \mathrm{supp}(\mu_T^{\mathrm{list}}) = \emptyset$ .
B Proofs for Section 4
This section provides the proof to Theorem 4.7, the domain adaptation generalization bound for ranking with list-level alignment. First, recall the following properties of Lipschitz functions:
Fact B.1 (Properties of Lipschitz Functions).
1. If  $f: \mathbb{R}^d \to \mathbb{R}$  is differentiable, then it is L-Lipschitz (in Euclidean distance) if and only if  $\| \nabla f \|_2 \leq L$ .
2. If  $f: \mathcal{X} \to \mathbb{R}$  is  $L$ -Lipschitz and  $g: \mathcal{X} \to \mathbb{R}$  is  $M$ -Lipschitz, then  $(af + bg) \in \operatorname{Lip}(|a|L + |b|M)$ , and  $\max(f, g) \in \operatorname{Lip}(\max(L, M))$ .
3. If  $f: \mathcal{X} \to \mathcal{Y}$  is  $L$ -Lipschitz and  $g: \mathcal{Y} \to \mathcal{Z}$  is  $M$ -Lipschitz, then  $g \circ f \in \operatorname{Lip}(LM)$ .
1. For the backward direction, suppose bounded gradient norm,  $\| \nabla f \|_2 \leq L$ , then by the mean value theorem,  $\exists t \in [0, 1]$  s.t.  $f(y) - f(x) = \nabla f(z)^\top (y - x)$  with  $z := (1 - t)x + ty$ , so by Cauchy-Schwarz inequality,
$$
\left\| f (y) - f (x) \right\| _ {2} \leq \left\| \nabla f (z) \right\| _ {2} \left\| y - x \right\| _ {2} \leq L \| y - x \| _ {2}.
$$
For the forward direction, suppose  $f \in \operatorname{Lip}(L)$ , then by differentiability,  $\nabla f(x)^\top z = f(x + z) - f(x) + o(\|z\|_2)$ . Set  $z := t\nabla f(x)$ , we have
$$
t \| \nabla f (x) \| _ {2} ^ {2} = f (x + t \nabla f (x)) - f (x) + o (t \nabla f (x)) \leq L t \| \nabla f (x) \| _ {2} + o (t \| \nabla f (x) \| _ {2}),
$$
and the result follows by dividing both sides by  $t\| \nabla f(x)\| _2$  and taking  $t\to 0$
2. First,
$$
\begin{array}{l} \left| a f (x) + b g (x) - \left(a f (y) + b g (y)\right) \right| \leq | a | | f (x) - f (y) | + | b | | g (x) - g (y) | \\ \leq \left(| a | L + | b | M\right) d _ {\mathcal {X}} (x, y). \\ \end{array}
$$
Next, assume w.l.o.g.  $\max (f(x),g(x)) - \max (f(y),g(y))\geq 0$  , then
$$
\begin{array}{l} \left| \max  (f (x), g (x)) - \max  (f (y), g (y)) \right| \\ = \left\{ \begin{array}{l l} f (x) - \max  (f (y), g (y)) \leq f (x) - f (y) \leq L   d _ {\mathcal {X}} (x, y) & \text {i f} f (x) \geq g (x) \\ g (x) - \max  (f (y), g (y)) \leq g (x) - g (y) \leq M   d _ {\mathcal {X}} (x, y) & \text {e l s e} \end{array} \right. \\ \leq \max  (L, M) d _ {\mathcal {X}} (x, y). \\ \end{array}
$$
3.  $d_{\mathcal{Z}}(g \circ f(x), g \circ f(y)) \leq M d_{\mathcal{Y}}(f(x), f(y)) \leq LM d_{\mathcal{X}}(x, y)$ .
We first prove Theorem 3.3 as a warm-up (restated below), the domain adaptation generalization bound for binary classification [52], since its proof shares the same organization as that of our Theorem 4.7, and it helps familiarizing readers of prior domain adaptation results and analysis techniques.
Theorem B.2. Let binary classification problems on a source and a target domain be given by joint distributions  $\mu_S, \mu_T$  of inputs and labels,  $(X,Y) \in \mathcal{X} \times \{0,1\}$ . Define the error rate of a predictor  $f: \mathcal{X} \to [0,1]$  on  $\mu$  by
$$
\mathcal {R} (f) = \mathbb {E} _ {(X, Y) \sim \mu} [ f (X)   \mathbb {1} [ Y \neq 1 ] + (1 - f (X))   \mathbb {1} [ Y \neq 0 ] ].
$$
Let  $\mathcal{F} \subset [0,1]^{\mathcal{X}}$  be an  $L$ -Lipschitz class of predictors. Define the minimum joint error rate by  $\lambda^{*} = \min_{f^{\prime}}(\mathcal{R}_{S}(f^{\prime}) + \mathcal{R}_{T}(f^{\prime}))$ , then for all  $f \in \mathcal{F}$
$$
\mathcal {R} _ {T} (f) \leq \mathcal {R} _ {S} (f) + 2 L W _ {1} (\mu_ {S} ^ {X}, \mu_ {T} ^ {X}) + \lambda^ {*}.
$$
Proof. Define random variable  $\eta = \mathbb{1}[Y = 1]$ , then we may rewrite
$$
\mathcal {R} (f) = \mathbb {E} _ {(X, Y) \sim \mu} [ \eta - (2 \eta - 1) f (X) ].
$$
Note that because  $2\eta - 1 \in \{-1, +1\}$ ,
$$
\begin{array}{l} \mathcal {R} (f) - \mathcal {R} \left(f ^ {\prime}\right) = \mathbb {E} _ {(X, Y) \sim \mu} [ \eta - (2 \eta - 1) f (X) ] - \mathbb {E} _ {(X, Y) \sim \mu} [ \eta - (2 \eta - 1) f ^ {\prime} (X) ] \\ = \mathbb {E} _ {(X, Y) \sim \mu} [ (2 \eta - 1) (f ^ {\prime} (X) - f (X)) ] \\ \leq \mathbb {E} _ {X \sim \mu^ {X}} [ | f ^ {\prime} (X) - f (X) | ]. \\ \end{array}
$$
$$
\begin{array}{l} \mathbb {E} _ {X \sim \mu^ {x}} [ | f (X) - f ^ {\prime} (X) | ] = \mathbb {E} _ {(X, Y) \sim \mu} [ | (2 \eta - 1) (f (X) - f ^ {\prime} (X)) - \eta + \eta | ] \\ \leq \mathbb {E} _ {(X, Y) \sim \mu} [ | (2 \eta - 1) f (X) - \eta | ] + \mathbb {E} _ {(X, Y) \sim \mu} [ | - (2 \eta - 1) f ^ {\prime} (X) + \eta | ] \\ = \mathbb {E} _ {(X, Y) \sim \mu} [ \eta - (2 \eta - 1) f (X) ] + \mathbb {E} _ {(X, Y) \sim \mu} [ \eta - (2 \eta - 1) f ^ {\prime} (X) ] \\ = \mathcal {R} (f ^ {\prime}) + \mathcal {R} (f). \\ \end{array}
$$
Then by Fact B.1, the fact that the operation of taking the absolute value is 1-Lipschitz, and Definition 3.2 of  $W_{1}$  distance, for all  $f, f' \in \mathcal{F}$ ,
$$
\begin{array}{l} \mathcal {R} _ {T} (f) = \mathcal {R} _ {S} (f) + \left(\mathcal {R} _ {T} (f) - \mathcal {R} _ {T} \left(f ^ {\prime}\right)\right) - \left(\mathcal {R} _ {S} (f) + \mathcal {R} _ {S} \left(f ^ {\prime}\right)\right) + \left(\mathcal {R} _ {S} \left(f ^ {\prime}\right) + \mathcal {R} _ {T} \left(f ^ {\prime}\right)\right) \\ \leq \mathcal {R} _ {S} (f) + \mathbb {E} _ {X \sim \mu_ {T} ^ {X}} [ | f (X) - f ^ {\prime} (X) | ] - \mathbb {E} _ {X \sim \mu_ {S} ^ {X}} [ | f (X) - f ^ {\prime} (X) | ] + (\mathcal {R} _ {S} (f ^ {\prime}) + \mathcal {R} _ {T} (f ^ {\prime})) \\ \leq \mathcal {R} _ {S} (f) + \sup  _ {q \in \operatorname {L i p} (2 L)} \left(\mathbb {E} _ {X \sim \mu_ {T} ^ {X}} [ q (X) ] - \mathbb {E} _ {X \sim \mu_ {S} ^ {X}} [ q (X) ]\right) + \left(\mathcal {R} _ {S} \left(f ^ {\prime}\right) + \mathcal {R} _ {T} \left(f ^ {\prime}\right)\right) \\ \leq \mathcal {R} _ {S} (f) + 2 L W _ {1} \left(\mu_ {S} ^ {X}, \mu_ {T} ^ {X}\right) + \left(\mathcal {R} _ {S} \left(f ^ {\prime}\right) + \mathcal {R} _ {T} \left(f ^ {\prime}\right)\right), \\ \end{array}
$$
and the result follows by taking the min over  $f'$ .
Next, we prove Theorem 4.7. The main idea is that under the setup and assumptions in Section 4,  $\mathcal{R}_S$  and  $\mathcal{R}_T$  can be written as the expectation of some Lipschitz function of  $Z\sim \mu_S^Z$  and  $\mu_T^Z$ , respectively, so by Definition 3.2, their difference is upper bounded by a Lipschitz constant times  $W_{1}(\mu_{S}^{Z},\mu_{T}^{Z})$ .
Although omitted, we remark that Theorem 4.7 can be extended to the cutoff version of the ranking metric  $u$  (e.g., NDCG@p) with a simple modification of the proof. Also, a finite sample generalization bound could be obtained using, e.g., Rademacher complexity, and additionally assuming Lipschitzness of the end-to-end scoring model [5, 51].
Proof of Theorem 4.7. Fix  $g \in \mathcal{G}$ , and consider  $\mu_S^X$  for now (analogous results hold for  $\mu_T^X$ ). Which by Assumption 4.6, when restricted to  $\mathrm{supp}(\mu_S^X)$ , has an  $L_g$ -Lipschitz inverse  $g^{-1}$ . Define function  $\epsilon_{h,g}: \mathcal{Z} \to \mathbb{R}_{\geq 0}$  for a given  $h: \mathcal{Z} \to \mathbb{R}^\ell$  as
$$
\begin{array}{l} \epsilon_ {h, g} (z) = \max  _ {r \in S _ {\ell}} u (r, y _ {S} \circ g ^ {- 1} (z)) - \mathbb {E} _ {R \sim p _ {\exp (h (z))}} [ u (R, y _ {S} \circ g ^ {- 1} (z)) ] \tag {3} \\ = \max  _ {r \in S _ {\ell}} u (r, y _ {S} \circ g ^ {- 1} (z)) - \sum_ {r \in S _ {\ell}} u (r, y _ {S} \circ g ^ {- 1} (z)) \prod_ {i = 1} ^ {\ell} \frac {\exp (h (z) _ {I (r) _ {i}})}{\sum_ {j = i} ^ {\ell} \exp (h (z) _ {I (r) _ {j}})}, \\ \end{array}
$$
and note that  $\mathcal{R}_S(h\circ g) = \mathbb{E}_{X\sim \mu_S^X}[\epsilon_{h,g}(g(X))] =: \mathbb{E}_{Z\sim \mu_S^Z}[\epsilon_{h,g}(z)]$ . We show that  $\epsilon_{h,g}$  is Lipschitz provided that  $h$  is Lipschitz, from establishing the Lipschitzness of both terms in Eq. (3).
For the first term, because  $u$  is  $L_{u}$ -Lipschitz w.r.t.  $y_{S} \circ g^{-1}(z)$  and  $y_{S} \circ g^{-1}(z)$  is in turn  $L_{y}L_{g}$ -Lipschitz w.r.t.  $z$  by Assumptions 4.3 and 4.4, it follows that  $z \mapsto u(r, y_{S} \circ g^{-1}(z))$  is  $L_{u}L_{y}L_{g}$ -Lipschitz w.r.t.  $z$  for any fixed  $r \in S_{\ell}$ , and therefore so is the function  $z \mapsto \max_{r \in S_{\ell}} u(r, y_{S} \circ g^{-1}(z))$  by Fact B.1.
For the second term, we show that it is Lipschitz (in Euclidean distance) w.r.t. both  $y_{S} \circ g^{-1}(z) =: y$  and  $h(z) =: s$ . By Jensen's inequality and Fact B.1,
$$
\| \nabla_ {y} \mathbb {E} _ {R \sim p _ {\exp (s)}} [ u (R, y) ] \| _ {2} = \| \mathbb {E} _ {R \sim p _ {\exp (s)}} [ \nabla_ {y} u (R, y) ] \| _ {2} \leq \mathbb {E} _ {R \sim p _ {\exp (s)}} [ \| \nabla_ {y} u (R, y) \| _ {2} ] \leq L _ {u}.
$$
And by the definition that  $\nabla_x f(x) = [\frac{\partial}{\partial x_1} f(x), \dots, \frac{\partial}{\partial x_d} f(x)]$  for any  $f: \mathbb{R}^d \to \mathbb{R}$ ,
$$
\begin{array}{l} \| \nabla_ {s} \mathbb {E} _ {R \sim p _ {\exp (s)}} [ u (R, y) ] \| _ {2} \\ \leq \left\| \nabla_ {s} \mathbb {E} _ {R \sim p _ {\exp (s)}} [ u (R, y) ] \right\| _ {1} \\ = \sum_ {m = 1} ^ {\ell} \left| \sum_ {r \in S _ {\ell}} u (r, y) \left(\frac {\partial}{\partial s _ {I (r) _ {m}}} \prod_ {i = 1} ^ {\ell} \frac {\exp \left(s _ {I (r) _ {i}}\right)}{\sum_ {j = i} ^ {\ell} \exp \left(s _ {I (r) _ {j}}\right)}\right) \right| \\ = \sum_ {m = 1} ^ {\ell} \left| \sum_ {r \in S _ {\ell}} u (r, y) \sum_ {i = 1} ^ {\ell} \left(\frac {\partial}{\partial s _ {I (r) _ {m}}} \frac {\exp \left(s _ {I (r) _ {i}}\right)}{\sum_ {j = i} ^ {\ell} \exp \left(s _ {I (r) _ {j}}\right)}\right) \prod_ {n \neq i} \frac {\exp \left(s _ {I (r) _ {n}}\right)}{\sum_ {j = n} ^ {\ell} \exp \left(s _ {I (r) _ {j}}\right)} \right| \\ = \sum_ {m = 1} ^ {\ell} \sum_ {r \in S _ {\ell}} u (r, y) \left(\prod_ {n = 1} ^ {\ell} \frac {\exp \left(s _ {I (r) _ {n}}\right)}{\sum_ {j = n} ^ {\ell} \exp \left(s _ {I (r) _ {j}}\right)}\right) \left| \sum_ {i = 1} ^ {m} \left(\mathbb {1} [ m = i ] - \frac {\exp \left(s _ {I (r) _ {m}}\right)}{\sum_ {j = i} ^ {\ell} \exp \left(s _ {I (r) _ {j}}\right)}\right) \right| \\ \leq B \sum_ {m = 1} ^ {\ell} \sum_ {r \in S _ {\ell}} \left(\prod_ {n = 1} ^ {\ell} \frac {\exp \left(s _ {I (r) _ {n}}\right)}{\sum_ {j = n} ^ {\ell} \exp \left(s _ {I (r) _ {j}}\right)}\right) \sum_ {i = 1} ^ {m} \left(\mathbb {1} [ m = i ] + \frac {\exp \left(s _ {I (r) _ {m}}\right)}{\sum_ {j = i} ^ {\ell} \exp \left(s _ {I (r) _ {j}}\right)}\right) \\ = B \sum_ {r \in S _ {\ell}} \left(\prod_ {n = 1} ^ {\ell} \frac {\exp \left(s _ {I (r) _ {n}}\right)}{\sum_ {j = n} ^ {\ell} \exp \left(s _ {I (r) _ {j}}\right)}\right) \sum_ {i = 1} ^ {\ell} \sum_ {m = i} ^ {\ell} \left(\mathbb {1} [ m = i ] + \frac {\exp \left(s _ {I (r) _ {m}}\right)}{\sum_ {j = i} ^ {\ell} \exp \left(s _ {I (r) _ {j}}\right)}\right) \\ = B \sum_ {r \in S _ {\ell}} \left(\prod_ {n = 1} ^ {\ell} \frac {\exp \left(s _ {I (r) _ {n}}\right)}{\sum_ {j = n} ^ {\ell} \exp \left(s _ {I (r) _ {j}}\right)}\right) \left(\ell + \sum_ {i = 1} ^ {\ell} \frac {\sum_ {m = i} ^ {\ell} \exp \left(s _ {I (r) _ {m}}\right)}{\sum_ {j = i} ^ {\ell} \exp \left(s _ {I (r) _ {j}}\right)}\right) \\ = 2 B \ell \sum_ {r \in S _ {\ell}} \left(\prod_ {n = 1} ^ {\ell} \frac {\exp \left(s _ {I (r) _ {n}}\right)}{\sum_ {j = n} ^ {\ell} \exp \left(s _ {I (r) _ {j}}\right)}\right) \\ = 2 B \ell , \\ \end{array}
$$
where the second equality is due to the product rule, the third equality to  $\frac{\mathrm{d}}{\mathrm{d}x_j}\mathrm{softmax}(x)_i = \mathrm{softmax}(x)_i(\mathbb{1}[i = j] - \mathrm{softmax}(x)_j)$ , the last inequality to Assumption 4.3, and the last equality to identifying the pmf of the P-L model (Definition 4.1). Because  $h\in \operatorname {Lip}(L_h)$  by Assumption 4.5, the two results above imply that the second term in Eq. (3) is Lipschitz: for all  $z,z^{\prime}\in \mathcal{Z}$
$$
\begin{array}{l} \left| \mathbb {E} _ {R \sim p _ {\exp (h (z))}} [ u (R, y _ {S} \circ g ^ {- 1} (z)) ] - \mathbb {E} _ {R \sim p _ {\exp (h (z ^ {\prime}))}} [ u (R, y _ {S} \circ g ^ {- 1} (z ^ {\prime})) ] \right| \\ \leq \left| \mathbb {E} _ {R \sim p _ {\exp (h (z))}} [ u (R, y _ {S} \circ g ^ {- 1} (z)) ] - \mathbb {E} _ {R \sim p _ {\exp (h (z))}} [ u (R, y _ {S} \circ g ^ {- 1} (z ^ {\prime})) ] \right| \\ + \left| \mathbb {E} _ {R \sim p _ {\exp (h (z))}} [ u (R, y _ {S} \circ g ^ {- 1} (z ^ {\prime})) ] - \mathbb {E} _ {R \sim p _ {\exp (h (z ^ {\prime}))}} [ u (R, y _ {S} \circ g ^ {- 1} (z ^ {\prime})) ] \right| \\ \leq L _ {u} \left\| y _ {S} \circ g ^ {- 1} (z) - y _ {S} \circ g ^ {- 1} \left(z ^ {\prime}\right) \right\| _ {2} + 2 B \ell \| h (z) - h \left(z ^ {\prime}\right) \| _ {2} \\ \leq \left(L _ {u} L _ {y} L _ {g} + 2 B \ell L _ {h}\right) d _ {\mathcal {Z}} \left(z, z ^ {\prime}\right). \\ \end{array}
$$
Combining the two parts, we have that  $\epsilon_{h,g}$  is  $2(L_{u}L_{y}L_{g} + B\ell L_{h})$ -Lipschitz in  $z$ .
Finally, by Fact B.1 and Definition 3.2, for all  $g\in \mathcal{G}$  and  $h,h^{\prime}\in \mathcal{H}$
$$
\mathcal {R} _ {T} (h \circ g) = \mathcal {R} _ {S} (h \circ g) + (\mathcal {R} _ {T} (h \circ g) - \mathcal {R} _ {T} (h ^ {\prime} \circ g)) - (\mathcal {R} _ {S} (h \circ g) + \mathcal {R} _ {S} (h ^ {\prime} \circ g))
$$
$$
+ \left(\mathcal {R} _ {S} \left(h ^ {\prime} \circ g\right) + \mathcal {R} _ {T} \left(h ^ {\prime} \circ g\right)\right)
$$
$$
\begin{array}{l} \leq \mathcal {R} _ {S} (h \circ g) + \left(\mathcal {R} _ {T} (h \circ g) - \mathcal {R} _ {T} \left(h ^ {\prime} \circ g\right)\right) - \left(\mathcal {R} _ {S} (h \circ g) - \mathcal {R} _ {S} \left(h ^ {\prime} \circ g\right)\right) \\ + \left(\mathcal {R} _ {S} \left(h ^ {\prime} \circ g\right) + \mathcal {R} _ {T} \left(h ^ {\prime} \circ g\right)\right) \\ \end{array}
$$
$$
\begin{array}{l} = \mathcal {R} _ {S} (h \circ g) + \mathbb {E} _ {Z \sim \mu_ {T} ^ {Z}} [ \epsilon_ {h, g} (Z) - \epsilon_ {h ^ {\prime}, g} (Z) ] - \mathbb {E} _ {Z \sim \mu_ {S} ^ {Z}} [ \epsilon_ {h, g} (Z) - \epsilon_ {h ^ {\prime}, g} (Z) ] \\ + \left(\mathcal {R} _ {S} \left(h ^ {\prime} \circ g\right) + \mathcal {R} _ {T} \left(h ^ {\prime} \circ g\right)\right) \\ \end{array}
$$
$$
\begin{array}{l} \leq \mathcal {R} _ {S} (h \circ g) + \sup  _ {\substack {(z) \in \left(\mathbb {Z} _ {T}\right) ^ {2}}} (\mathbb {E} _ {Z \sim \mu_ {T} ^ {z}} [ q (Z) ] - \mathbb {E} _ {Z \sim \mu_ {S} ^ {z}} [ q (Z) ])) \\ q \in \operatorname {L i p} \left(4 \left(L _ {u} L _ {y} L _ {g} + B \ell L _ {h}\right)\right) \quad + \left(\mathcal {R} _ {S} \left(h ^ {\prime} \circ g\right) + \mathcal {R} _ {T} \left(h ^ {\prime} \circ g\right)\right) \\ \end{array}
$$
$$
\leq \mathcal {R} _ {S} (h \circ g) + 4 \left(L _ {u} L _ {y} L _ {g} + B \ell L _ {h}\right) W _ {1} \left(\mu_ {S} ^ {Z}, \mu_ {T} ^ {Z}\right) + \left(\mathcal {R} _ {S} \left(h ^ {\prime} \circ g\right) + \mathcal {R} _ {T} \left(h ^ {\prime} \circ g\right)\right),
$$
and the result follows by taking the min over  $h^\prime$
Finally, we verify the Lipschitzness of RR and NDCG.
Proof of Corollary 4.8. Because  $\mathrm{RR} \leq 1$  uniformly and  $\|y - y'\|_2 \geq 1$  for all  $y, y' \in \{0,1\}^\ell$ ,  $y \neq y'$  so  $y \mapsto \mathrm{RR}(r,y)$  is 1-Lipschitz.
Proof of Corollary 4.9. We show that NDCG is Lipschitz w.r.t.  $y$  under the assumptions. Recall that
$$
\mathrm {N D C G} (r, y) = \frac {\mathrm {D C G} (r , y)}{\mathrm {I D C G} (y)} = \left(\sum_ {i = 1} ^ {\ell} \frac {y _ {i}}{\log (r _ {i} ^ {*} + 1)}\right) ^ {- 1} \sum_ {i = 1} ^ {\ell} \frac {y _ {i}}{\log (r _ {i} + 1)},
$$
where  $r^*$  is the descending order of  $y$ .
Note that IDCG is continuous and piecewise differentiable w.r.t.  $y$  (each piece is associated with some  $r' \in S_{\ell}$  and supported on  $\{y : \arg \max_{r} \mathrm{DCG}(r, y) = r'\}$ ), then so is  $\mathrm{IDCG}^{-1}$  on the set  $\mathcal{Y}' := \{y : C^{-1} \leq \mathrm{IDCG}(y) \leq C\}$  (which is our assumption) by chain rule. Clearly, DCG is continuous and differentiable w.r.t.  $y$ , so NDCG is continuous and piecewise differentiable on  $\mathcal{Y}'$  by product rule. By Fact B.1, to show that NDCG is Lipschitz, we just need to show that its gradient norm w.r.t.  $y$  is bounded when evaluated on (the interior of) each piece.
Let  $r \in S_{\ell}$  be arbitrary, and  $y \in \mathcal{Y}'$  be s.t. it lies in the interior of the respective piece, namely, by denoting  $r^*$  the descending order of  $y$ , the interior of the set  $\{y': \arg \max_r \mathrm{DCG}(r, y') = r^*\} = \{y': y_{I(r^*)_1}' \geq y_{I(r^*)_2}' \geq \dots \geq y_{I(r^*)_\ell}'\}$  (recall that  $I(r)_i$  denotes the index of the item with rank  $i$ ). Then, for all  $k \in [\ell]$ ,
$$
\begin{array}{l} \left| \frac {\partial}{\partial y _ {k}} \mathrm {N D C G} (r, y) \right| = \left| \mathrm {I D C G} (y) ^ {- 1} \frac {\partial}{\partial y _ {k}} \sum_ {i = 1} ^ {\ell} \frac {y _ {i}}{\log \left(r _ {i} + 1\right)} - \mathrm {D C G} (r, y) \left(\frac {\partial}{\partial y _ {k}} \sum_ {i = 1} ^ {\ell} \frac {y _ {i}}{\log \left(r _ {i} ^ {*} + 1\right)}\right) ^ {- 2} \right| \\ \leq \left| \operatorname {I D C G} (y) ^ {- 1} \log \left(r _ {k} + 1\right) ^ {- 1} \right| + \left| \operatorname {D C G} (r, y) \log \left(r _ {k} ^ {*} + 1\right) ^ {2} \right| \\ \leq \left| \mathrm {I D C G} (y) ^ {- 1} \log (2) ^ {- 1} \right| + \left| \mathrm {D C G} (r, y) \log (\ell + 1) ^ {2} \right| \\ \leq C + C \log (\ell + 1) ^ {2}, \\ \end{array}
$$
hence  $\| \nabla_y\mathrm{NDCG}(r,y)\| _2\leq \sqrt{\ell} (C + C\log (\ell +1)^2)\leq \widetilde{O} (C\sqrt{\ell})$ , and NDCG is  $\widetilde{O} (C\sqrt{\ell})$ -Lipschitz w.r.t.  $y$ . Here,  $\widetilde{O}$  hides polylogarithmic terms.
C Additional Experiments and Details on Passage Reranking
We perform additional experiments for unsupervised domain adaptation on the passage reranking task considered in Section 5 in Appendix C.1, provide case studies on ListDA vs. zero-shot and QGen PL (Tables 8 and 9), hyperparameter settings in Appendix C.2, and details on the construction of training lists in Appendix C.3.
C.1 Additional Results
Pairwise Logistic Ranking Loss. Besides the listwise softmax cross-entropy ranking loss used in Section 5:
$$
\ell (s, y) = - \sum_ {i = 1} ^ {\ell} y _ {i} \log \left(\frac {\exp (s _ {i})}{\sum_ {j = 1} ^ {\ell} \exp (s _ {j})}\right),
$$
we experiment with the pairwise logistic ranking loss [8]:
$$
\ell (s, y) = - \sum_ {i = 1} ^ {\ell} \sum_ {j = 1} ^ {\ell} \mathbb {1} [ y _ {i} > y _ {j} ] \log \left(\frac {\exp (s _ {i})}{\exp (s _ {i}) + \exp (s _ {j})}\right).
$$
The results with using the pairwise logistic loss for training on the Robust04 dataset are provided in Table 2a, which are not better than using the softmax cross-entropy loss (cf. Table 1; see also [25]), hence further experiments with this loss are not pursued.
Table 2: Reranking performance of RankT5 on top 1000 BM25-retrieved passages; additional results extending Table 1.
(a) With pairwise logistic ranking loss in place of softmax cross-entropy on Robust04.
Target domain	Method	MAP	MRR@10	NDCG@5	NDCG@10	NDCG@20
Robust04	Zero-shot	0.2656	0.7894	0.5671	0.5163	0.4729
QGen PL	0.2776*	0.7975	0.5576	0.5267	0.4892*
ItemDA (MLP)	0.2766*	0.8021	0.5794	0.5355*	0.4917*
ListDA	0.2893*†‡	0.8103	0.5935*†‡	0.5524*†‡	0.5044*†‡
Source domain is MS MARCO. Gain function in NDCG is the identity map. *Improves upon zero-shot with statistical significance (p ≤ 0.05) under the two-tailed Student's t-test. † Improves upon QGen PL. ‡ Improves upon ItemDA.
(b) With ItemDA with a transformer discriminator.
Target domain	Method	MAP	MRR@10	NDCG@5	NDCG@10	NDCG@20
Robust04	0.2848	0.8018	0.5851	0.5406	0.4982
TREC-COVID	ItemDA (transformer)	0.3088	0.8907	0.8287	0.8069	0.7691
BioASQ	0.4739	0.6420	0.5256	0.5312	0.5567
Source domain is MS MARCO. Gain function in NDCG is the identity map.
(c) With ListDA + QGen PL method.
Target domain	Method	MAP	MRR@10	NDCG@5	NDCG@10	NDCG@20
Robust04	0.2851*†	0.8039†	0.5761†	0.5386†	0.4975†
TREC-COVID	ListDA + QGen PL	0.3168	0.8950	0.8539	0.8292	0.7820
BioASQ	0.6538*‡	0.5158	0.5547‡	0.5671*‡	0.5931*‡
Source domain is MS MARCO. Gain function in NDCG is the identity map. *Improves upon zero-shot with statistical significance (p ≤ 0.05) under the two-tailed Student's t-test. † Improves upon QGen PL. § Improves upon ItemDA.
Table 3: Reranking performance of RankT5 on top 1000 BM25-retrieved passages on Signal-1M.
Target domain	Method	MAP	MRR@10	NDCG@5	NDCG@10	NDCG@20
Signal-1M	BM25	0.1740	0.5765	0.3639	0.3215	0.2905
Zero-shot	0.1511	0.4804	0.3068	0.2685	0.2410
QGen PL	0.1541	0.5043	0.3238	0.2799	0.2497
ListDA	0.1456	0.4629	0.3002	0.2602	0.2328
ListDA + QGen PL	0.1549	0.5170	0.3261	0.2817	0.2505
Source domain is MS MARCO. Gain function in NDCG is the identity map.
As an implementation remark, in this set of experiments, we do not perform pairwise comparisons to obtain the predicted rank assignments during inference (where lists have length-1000) due to the high time complexity (the loss is aggregated pairwise during training, since we truncate training lists to length-31; see Appendix C.3). Whether or not the forward pass of the model involves pairwise computations does not matter to ListDA, which is applicable to any (pointwise, pairwise, or listwise) model as long as we can gather list-level representations; ListDA could be instantiated on pairwise models e.g. DuoT5 [44], although not pursued in this work.
ItemDA with Transformer Discriminator. To show that the improvements brought by ListDA over ItemDA is due to learning list-level invariant representations and not because of the use of the more expressive transformer as the discriminator vs. MLP, we experiment with a variant of ItemDA that uses the same transformer discriminator used in our ListDA runs. Although, we remark that whereas each input to the ListDA discriminator is a list of feature vectors as a length- $\ell$  sequence, each input to the ItemDA transformer discriminator is a single feature vector, or, a sequence of length-1. So, the attention mechanism of the transformer becomes pointless, and the ItemDA transformer discriminator collapses to an MLP (with skip connections and LayerNorm).
The results are presented in Table 2b (cf. Table 1). Across all datasets and metrics, no consistent improvement of ItemDA (transformer) over ItemDA (MLP) is observed.