SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} d \log 2 + d _ {\mathcal {G}} (\log d _ {\mathcal {G}} - 1) \leq (d _ {[ \mathcal {F} ]} + d _ {\mathcal {G}}) (\log d + \log M) + 2 d _ {[ \mathcal {F} ]} \log c \\ \leq \left(d _ {[ \mathcal {F} ]} + d _ {\mathcal {G}}\right) \left(\frac {d}{6 \left(d _ {[ \mathcal {F} ]} + d _ {\mathcal {G}}\right)} + \log \left(6 \left(d _ {[ \mathcal {F} ]} + d _ {\mathcal {G}}\right)\right) - 1\right) \\ + \left(d _ {[ \mathcal {F} ]} + d _ {\mathcal {G}}\right) \log M + 2 d _ {[ \mathcal {F} ]} \log c \\ = d / 6 + \left(d _ {[ \mathcal {F} ]} + d _ {\mathcal {G}}\right) \left(\log \left(6 M \left(d _ {[ \mathcal {F} ]} + d _ {\mathcal {G}}\right)\right) - 1\right) + 2 d _ {[ \mathcal {F} ]} \log c \\ \leq d / 6 + \left(d _ {[ \mathcal {F} ]} + d _ {\mathcal {G}}\right) \log \left(6 M \left(d _ {[ \mathcal {F} ]} + d _ {\mathcal {G}}\right)\right) - d _ {\mathcal {G}} + 2 d _ {[ \mathcal {F} ]} \log c \\ \end{array}
$$
where the second step follows from the first-order Taylor series expansion of the logarithm function at the point  $6(d_{[\mathcal{F}]} + d_{\mathcal{G}})$ . Therefore,
$$
\begin{array}{l} d \leq \frac {\left(d _ {[ \mathcal {F} ]} + d _ {\mathcal {G}}\right) \log \left(6 M \left(d _ {[ \mathcal {F} ]} + d _ {\mathcal {G}}\right)\right) + 2 d _ {[ \mathcal {F} ]} \log c - d _ {\mathcal {G}} \left(\log \left(d _ {\mathcal {G}}\right)\right)}{\log 2 - 1 / 6} \\ \leq 2 \left(\left(d _ {[ \mathcal {F} ]} + d _ {\mathcal {G}}\right) \log \left(6 M \left(d _ {[ \mathcal {F} ]} + d _ {\mathcal {G}}\right)\right) + 2 d _ {[ \mathcal {F} ]} \log c\right) \\ \end{array}
$$
where the last step follows from the fact that  $\log 2 - 1 / 6 > 1 / 2$ . The above concludes the proof of Lemma 3.
Notice that if  $\sigma$  is known to the learner (i.e.,  $\mathcal{G}$  is a singleton) and hence,  $d_{\mathcal{G}} = 0^7$ , then (21) becomes
$$
d \leq 2 \left(d _ {[ \mathcal {F} ]} \log \left(6 M d _ {[ \mathcal {F} ]}\right) + 2 d _ {[ \mathcal {F} ]} \log c\right) \tag {23}
$$
B.1 Proofs for Section 3.1
7Singleton hypothesis classes have VC-dimension equal to zero.
Proof. Given (19), the zero-one risk of  $f$  can be rewritten as
$$
\mathcal {R} ^ {0 1} (f) = \sum_ {l _ {i} \neq l _ {j}} E _ {l _ {i}, l _ {j}} (f) \tag {24}
$$
Since  $\sigma$  is  $M$ -unambiguous, we know that if all the  $M$  input instances have the same label and are wrongly classified as another label, then the predicted partial label will be wrong. Therefore, the zero-one partial risk is lower bounded by the probability of such events, namely the sum of the same type of classification mistake being repeated by  $M$  times. We have that:
$$
\begin{array}{l} \mathcal {R} _ {\mathsf {P}} ^ {0 1} (f; \sigma) \geq \sum_ {l _ {i} \neq l _ {j}} E _ {l _ {i}, l _ {j}} (f) ^ {M} \\ \geq \frac {\left(\sum_ {l _ {i} \neq l _ {j}} E _ {l _ {i} , l _ {j}} (f)\right) ^ {M}}{\left(c (c - 1)\right) ^ {M - 1}} \quad (\text {P o w e r M e a n i n e q u a l i t y}) \tag {25} \\ = \frac {\mathcal {R} ^ {0 1} (f) ^ {M}}{(c (c - 1)) ^ {M - 1}} \\ \end{array}
$$
where  $|\mathcal{V}| = c$ . Rearranging the inequality yields:
$$
\begin{array}{l} \mathcal {R} ^ {0 1} (f) \leq \left(\mathcal {R} _ {\mathrm {P}} ^ {0 1} (f; \sigma) (c (c - 1)) ^ {M - 1}\right) ^ {1 / M} \\ \leq \left(c ^ {2 M - 2} \mathcal {R} _ {\mathrm {P}} ^ {0 1} (f; \sigma)\right) ^ {1 / M} \tag {26} \\ = \mathcal {O} \left(\left(\mathcal {R} _ {\mathrm {P}} ^ {0 1} (f; \sigma)\right) ^ {1 / M}\right) \\ \end{array}
$$
The above concludes the first part of the proof of Lemma 1.
We now focus on the second part of the proof. If  $\sigma$  is not  $M$ -unambiguous, then from Definition 1, we know that there exists a pair of labels  $y, y' \in \mathcal{V}$  with  $y' \neq y$ , such that  $\sigma(y, \ldots, y) \neq \sigma(y', \ldots, y')$  holds. Consider now a sample  $(x_0, y)$  drawn from  $\mathcal{D}$ . If  $\mathcal{D}$  concentrates all its mass on  $x_0 \in \mathcal{X}$ , i.e.,  $\mathbb{P}_{\mathcal{D}}(x_0) = 1$ , and classifier  $f$  misclassifies  $x_0$  as  $y' \neq y$ , then  $\mathcal{R}_{\mathbb{P}}^{01}(f; \sigma) = 0$  but  $\mathcal{R}^{01}(f) = 1$ . The above concludes the proof of Lemma 1.
Theorem 6 (ERM learnability under  $M$ -unambiguity). Suppose  $\mathcal{F}$  is realizable under  $\ell_{\mathsf{P}}^{01}$  and  $[\mathcal{F}]$  has a finite Natarajan dimension  $d_{[\mathcal{F}]}$ . Then for any  $\epsilon, \delta \in (0,1)$ , there exists a universal constant  $C_0 > 0$ , such that with probability at least  $1 - \delta$ , the empirical partial risk minimizer with  $\widehat{\mathcal{R}}_{\mathsf{P}}^{01}(f; \sigma; \mathcal{T}_{\mathsf{P}}) = 0$  has a classification risk  $\mathcal{R}^{01}(f) < \epsilon$ , if
$$
m _ {\mathbb {P}} \geq C _ {0} \frac {c ^ {2 M - 2}}{\epsilon^ {M}} \left(d _ {[ \mathcal {F} ]} \log (6 c M d _ {[ \mathcal {F} ]}) \log \left(\frac {c ^ {2 M - 2}}{\epsilon^ {M}}\right) + \log \left(\frac {1}{\delta}\right)\right) \tag {27}
$$
Proof. Let  $f$  be the empirical partial risk minimizer, such that  $\widehat{\mathcal{R}}_{\mathsf{P}}^{01}(f;\sigma; \mathcal{T}_{\mathsf{P}}) = 0$  holds. For any  $\epsilon \in (0,1)$ , we know by the proof of Lemma 1 that  $\mathcal{R}^{01}(f) \leq (c^{2M-2}\mathcal{R}_{\mathsf{P}}^{01}(f;\sigma))^{1/M}$  holds. Hence, by bounding  $\mathcal{R}_{\mathsf{P}}^{01}(f;\sigma) \leq \epsilon^M / c^{2M-2}$ , the following would follow:  $\mathcal{R}^{01}(f) \leq \epsilon$ . By the standard bound for sample complexity based on VC-dimension (see, for example, Theorem 6.7 in [40]), we know that for any confidence parameter  $\delta \in (0,1)$ ,  $\mathcal{R}^{01}(f) \leq (c^{2M-2}\mathcal{R}_{\mathsf{P}}^{01}(f;\sigma))^{1/M}$  holds with probability at least  $1 - \delta$ , if
$$
m _ {\mathbb {P}} \geq m \left(\mathcal {H} _ {\mathcal {F}}, \delta , \epsilon^ {M} / c ^ {2 M - 2}\right) \tag {28}
$$
$$
m \left(\mathcal {H} _ {\mathcal {F}}, \delta , t\right) := \frac {C}{t} \left(\mathcal {H} _ {\mathcal {F}} \log \left(\frac {1}{t}\right) + \log \left(\frac {1}{\delta}\right)\right) \tag {29}
$$
where  $C$  is a universal constant. Combining (23) with (28) and (29) yields the desired result, concluding the proof of Theorem 6.
Before proving Proposition 1, we will introduce a more general definition of unambiguity:
Definition 10 (I-unambiguity). Mapping  $\sigma$  is  $I$ -unambiguous if there exists a set of distinct  $I$  indices  $\mathcal{I} = \{i_1, \ldots, i_I\} \subseteq [M]$ , such that for any  $\boldsymbol{y} \in \mathcal{Y}^M$  having the same label  $l$  in all positions in  $\mathcal{I}$ , the vector  $\boldsymbol{y}'$  that results after flipping  $l$  to  $l' \neq l$  in those positions satisfies  $\sigma(\boldsymbol{y}') \neq \sigma(\boldsymbol{y})$ .
${}^{8}$  A constant is universal if it that does not depend on the parameters of the learning problem (e.g.,  $M$  or  $c$  ).
We now prove the following lemma based on  $I$ -unambiguity:
Lemma 4. If  $\sigma$  is both  $I$ - and  $M$ -unambiguous, then we have:
$$
\mathcal {R} ^ {0 1} (f) \leq \mathcal {O} \left(\left(\mathcal {R} _ {\mathsf {P}} ^ {0 1} (f; \sigma)\right) ^ {1 / I}\right) \tag {30}
$$
Proof. Let  $\mathcal{I}$  be the disambiguation set of  $\sigma$ . Then, for each pair of labels  $l_i, l_j \in \mathcal{V}$  with  $l_i \neq l_j$ , the following holds with probability  $E_{l_i, l_j}(f)^I$ : the gold labels in  $\mathcal{I}$  are all  $l_i$ , but they are all misclassified by  $f$  as  $j$ . Since  $\sigma$  is  $I$ -unambiguous, this type of error implies a partial label error. Then, we have the following:
$$
\begin{array}{l} \mathcal{R}^{01}_{\mathsf{P}}(f;\sigma)\geq \sum_{l_{i}\neq l_{j}}E_{l_{i},l_{j}}(f)^{I}\underbrace{(1 - \mathcal{R}^{01}(f))^{M - I}}_{\text{Probability of correctly classifying the remaining $M - I$ labels.}} \\ \geq \frac {\left(\sum_ {l _ {i} \neq l _ {j}} E _ {l _ {i} , l _ {j}} (f)\right) ^ {I} \left(1 - \mathcal {R} ^ {0 1} (f)\right) ^ {M - I}}{(c (c - 1)) ^ {I - 1}} \quad (\text {P o w e r - M e a n I n e q u a l i t y}) \tag {31} \\ = \frac {\mathcal {R} ^ {0 1} (f) ^ {I} \left(1 - \mathcal {R} ^ {0 1} (f)\right) ^ {M - I}}{(c (c - 1)) ^ {I - 1}} \\ \end{array}
$$
Since  $\sigma$  is also  $M$ -unambiguous by assumption, we have from (26) that  $\mathcal{R}^{01}(f) \leq c^2 (\mathcal{R}_{\mathbb{P}}^{01}(f; \sigma))^{1 / M}$ . Hence,  $1 - \mathcal{R}^{01}(f) \geq 1 - c^2 (\mathcal{R}_{\mathbb{P}}^{01}(f; \sigma))^{1 / M}$ . Combining the above with (31), yields:
$$
\mathcal {R} _ {\mathsf {P}} ^ {0 1} (f; \sigma) \geq \frac {\mathcal {R} ^ {0 1} (f) ^ {I} \left(1 - \left(c ^ {2 M - 2} \mathcal {R} _ {\mathsf {P}} ^ {0 1} (f ; \sigma)\right) ^ {1 / M}\right) ^ {M - I}}{(c (c - 1)) ^ {I - 1}}
$$
$$
\mathcal {R} ^ {0 1} (f) \leq \left(\frac {\mathcal {R} ^ {0 1} (f) c ^ {2 I - 2}}{\left(1 - \left(c ^ {2 M - 2} \mathcal {R} _ {\mathrm {P}} ^ {0 1} (f ; \sigma)\right) ^ {1 / M}\right) ^ {M - I}}\right) ^ {1 / I}
$$
Based on the above, we conclude that
$$
\mathcal {R} ^ {0 1} (f) \leq \Phi_ {I} \left(\mathcal {R} _ {\mathrm {P}} ^ {0 1} (f; \sigma)\right) \tag {32}
$$
$$
\Phi_ {I}: t \mapsto \min  \left\{\left(\frac {t c ^ {2 I - 2}}{(1 - (c ^ {2 M - 2} t) ^ {\frac {1}{M}}) ^ {M - I}}\right) ^ {1 / I}, t ^ {\frac {1}{M}} c ^ {2} \right\} \tag {33}
$$
as  $\mathcal{R}_{\mathsf{P}}(f;\ell_{\mathsf{P}}^{01})\to 0$ . We can see that  $\Phi_I(t) = \mathcal{O}(t^{1 / I})$  as  $t\to 0$ . The above concludes the proof of Lemma 4.
We are now ready to prove Proposition 1.
Furthermore, if  $\mathcal{F}$  is realizable under  $\ell_{\mathbb{P}}^{01}$  and  $[\mathcal{F}]$  has a finite Natarajan dimension  $d_{[\mathcal{F}]}$ , then for any  $\delta \in (0,1)$  and  $\epsilon \in (0,1)$  that is sufficiently close to 0, there exists a universal constant  $C_1$ , such that with probability at least  $1 - \delta$ , the empirical partial risk minimizer with  $\widehat{\mathcal{R}}_{\mathbb{P}}^{01}(f;\sigma) = 0$  has a classification risk  $\mathcal{R}^{01}(f) < \epsilon$ , if
Proof. The first claim can be derived from Lemma 4, by setting  $I = 1$ . For the second claim, using inequality (33) and letting  $I = 1$ , we know that
$$
\mathcal {R} ^ {0 1} (f) \leq \frac {t}{(1 - (c ^ {2 M - 2} t) ^ {1 / M}) ^ {M - 1}}
$$
Suppose  $\epsilon$  is small enough so that
$$
\epsilon \leq \frac {1}{(2 M) ^ {M} c ^ {2 M - 2}} \tag {34}
$$
Now, if we could bound  $\mathcal{R}_{\mathbb{P}}^{01}(f;\sigma)\leq \epsilon /2$  , we will have that
$$
\begin{array}{l} \mathcal {R} ^ {0 1} (f) \leq \frac {\epsilon / 2}{\left(1 - \left(c ^ {2 M - 2} (\epsilon / 2)\right) ^ {\frac {1}{M}}\right) ^ {M - 1}} \\ \leq \frac {\epsilon / 2}{\left(1 - \left(c ^ {2 M - 2} \epsilon\right) ^ {\frac {1}{M}}\right) ^ {M}} \\ \leq \frac {\epsilon / 2}{1 - M \left(c ^ {2 M - 2} \epsilon\right) ^ {\frac {1}{M}}} \quad \text {(B e r n o u l l i ' s i n e q u a l i t y)} \\ \leq \epsilon \quad \text {(E q u a t i o n (3 4))} \\ \end{array}
$$
Now, our goal is to bound the sample complexity so that  $\mathcal{R}_{\mathsf{P}}^{01}(f;\sigma)\leq \epsilon /2$ . Similarly to the proof of Theorem 6, by the standard bound for sample complexity based on VC-dimension, for any  $\delta \in (0,1)$ ,  $\mathcal{R}_{\mathsf{P}}^{01}(f;\sigma)\leq \epsilon /2$  holds with probability at least  $1 - \delta$ , if
$$
m _ {\mathbb {P}} \geq m \left(\mathcal {H} _ {\mathcal {F}, \mathcal {G}}, \delta , \epsilon / 2\right) \tag {35}
$$
$$
m \left(\mathcal {H} _ {\mathcal {F}, \mathcal {G}}, \delta , t\right) := \frac {C}{t} \left(\operatorname {V C} \left(\mathcal {H} _ {\mathcal {F}, \mathcal {G}}\right) \log \left(\frac {1}{t}\right) + \log \left(\frac {1}{\delta}\right)\right) \tag {36}
$$
where  $C$  is a universal constant. Combining (23) with (35) and (36) yields the desired result. The above concludes the proof of Proposition 1.
B.2 Proofs for Section 3.2
B.2.1 Relating the top- $k$  semantic loss to the infimum/minimum loss
In this section, we show that the minimal loss can be viewed as a special case of the top- $k$  partial loss from Definition 3.
We start by recapitulating the definition of the minimal loss [29]. Given a scoring function  $f: \mathcal{X} \to \mathbb{R}^M$ , a loss function  $\ell$  and a partial labelset  $s \in 2^\mathcal{V}$ , the minimal loss [29] for standard PLL [28] is defined as:
$$
\min  _ {y \in s} \ell (f (x), y) \tag {37}
$$
Below, we show that the minimal loss coincides with the top-1 partial loss.
For multi-instance PLL, we can extend (37) in a straightforward fashion as follows:
$$
\min  _ {\sigma (\boldsymbol {y}) = s} \ell (f (\boldsymbol {x}), \boldsymbol {y}) \tag {38}
$$
If  $\ell$  is the cross-entropy loss, then the minimal loss finds the most likely label vector in the preimage of  $s$ . Now, let us focus on Definition 3. If we set  $k = 1$  in  $\ell_{\sigma}^{k}$ , then the only model for the formula  $\varphi = \pmb{y}^{(1)} := A_{1,y_1^{(1)}} \wedge \dots \wedge A_{M,y_M^{(1)}}$  assigns to each Boolean variable occurring in  $\pmb{y}^{(1)}$  value  $\top$ . The WMC of  $\varphi$  is given by
$$
\prod_ {i = 1} ^ {M} f ^ {y _ {i} ^ {(1)}} \left(x _ {i}\right) = \max  _ {\sigma (\boldsymbol {y}) = s} P _ {f (\boldsymbol {x})} \left(\boldsymbol {y} ^ {(1)}\right) \tag {39}
$$
which is also the maximum likelihood that can be obtained in the preimage of  $s$ . This implies that the top-1 semantic loss finds the minimum negative log-likelihood. Therefore, the top- $k$  partial loss equals to the minimal loss for  $k = 1$ .
Furthermore, for a partial label  $s \in 2^{\mathcal{V}}$  and a label prediction  $z$ , the infimum loss [5] is defined as
$$
L(z,s):= \inf_{y\in s}\ell (z,y)
$$
Treating the probabilistic prediction  $f(\pmb{x})$  as a generalized label, the above definition becomes equivalent to that of the minimal loss (notice that  $\mathcal{V}$  is finite, so infimum = minimum).
The above discussion shows that the minimal loss is a special case of the top- $k$  partial loss.
B.2.2 Proofs
To prove the main result of this section, we will need the following definitions and lemmas.
Definition 11 (Top- $k$  partial  $\ell^1$  loss). For an integer  $k \geq 1$ , the top- $k$  partial  $\ell^1$  loss under scoring function  $f$ , input  $\boldsymbol{x} \in \mathcal{X}^M$  and partial label  $s \in S$  is given by
$$
\tilde {\ell} _ {\mathbb {P}} ^ {k} (f (\boldsymbol {x}), s) := 1 - \operatorname {W M C} \left(\bigvee_ {i = 1} ^ {k} \boldsymbol {y} ^ {(i)}, f (\boldsymbol {x})\right) \tag {40}
$$
where  $\pmb{y}^{(1)},\dots ,\pmb{y}^{(k)}$  are as in Definition 3.
We now prove the following lemma:
Lemma 5. For any classifier  $f \in \mathcal{F}$ ,  $\pmb{x} \in \mathcal{X}^M$ ,  $s \in S$  and integer  $k \geq 1$ , we have:
$$
\ell_ {\mathsf {P}} ^ {0 1} (f (\boldsymbol {x}), s) \leq (k + 1) \tilde {\ell} _ {\mathsf {P}} ^ {k} (f (\boldsymbol {x}), s) \leq (k + 1) \ell_ {\mathsf {P}} ^ {k} (f (\boldsymbol {x}), s) \tag {41}
$$
Proof. Let us denote by  $\pmb{y}_f = ([f](x_1),\dots ,[f](x_M))$  the most likely label assignment to  $\pmb{x}$  by  $f$ . If  $\ell_{\mathsf{P}}^{01}(\pmb{y}_f,s) = 1$  holds, then  $\sigma (\pmb{y}_f)\neq s$ . Therefore  $\pmb{y}_f$  is different from the top-  $k$ $\pmb{y}^{(i)}$  vectors. Hence, the probabilities of the  $\pmb{y}^{(i)}$ 's, for  $i\in [k]$ , and  $\pmb{y}_f$  sum to at most 1, i.e.,
$$
\begin{array}{l} 1 \geq P _ {f (\boldsymbol {x})} \left(\boldsymbol {y} _ {f}\right) + \sum_ {i = 1} ^ {k} P _ {f (\boldsymbol {x})} \left(\boldsymbol {y} ^ {(i)}\right) \tag {42} \\ \geq \left(\frac {1}{k} + 1\right) \sum_ {i = 1} ^ {k} P _ {f (\boldsymbol {x})} (\boldsymbol {y} ^ {(i)}) \\ \end{array}
$$
The second inequality holds because the assignment  $\mathbf{y}_f$  has a larger score than any other label assignments in the preimage of  $s$ . Furthermore, by Lemma 2, we know that
$$
\operatorname {W M C} \left(\bigvee_ {i = 1} ^ {k} \boldsymbol {y} ^ {(i)}, f (\boldsymbol {x})\right) \leq \sum_ {i = 1} ^ {k} P _ {f (\boldsymbol {x})} \left(\boldsymbol {y} ^ {(i)}\right) \leq \frac {k}{k + 1} \tag {43}
$$
Since the zero-one loss  $\ell_{\sigma}^{01}(\pmb{y}_f, s)$  is by definition either 0 or 1, we have:
$$
\tilde {\ell} _ {\mathsf {P}} ^ {k} (f (\boldsymbol {x}), s) \geq \frac {1}{k + 1} \geq \frac {1}{k + 1} \ell_ {\mathsf {P}} ^ {0 1} (f (\boldsymbol {x}), s)
$$
Finally, from the inequality  $-\log (1 - t)\geq t\forall t > 0$  we know that  $\tilde{\ell}_{\mathsf{P}}^{k}(f(\pmb {x}),s)$  lower bounds the cross-entropy top-  $k$  loss:
$$
\ell_ {\mathbb {P}} ^ {k} (f (\boldsymbol {x}), s) \leq \ell_ {\mathbb {P}} ^ {k} (f (\boldsymbol {x}), s) \tag {44}
$$
The above concludes the proof of Lemma 5.
The following two lemmas concern the Lipschitzness of the top- $k$  loss.
Lemma 6 (Contraction lemma (Lemma 5 from [10])). Let  $N$  and  $m$  be two positive integers. Let also  $\mathcal{H}$  be a set of functions that map  $\mathcal{X}$  to  $\mathbb{R}^N$ . Suppose that for each  $i \in [m]$ , function  $\Psi_i: \mathbb{R}^N \to \mathbb{R}$  is  $\mu_i$ -Lipschitz with the 2-norm, i.e.,
$$
\left| \Psi_ {i} \left(v ^ {\prime}\right) - \Psi_ {i} (v) \right| \leq \mu_ {i} \| v ^ {\prime} - v \| _ {2} \quad \forall v, v ^ {\prime} \in \mathbb {R} ^ {N} \tag {45}
$$
Then, for any set of  $m$  points  $x_{1},\ldots ,x_{m}\in \mathcal{X}$  , the following holds:
$$
\frac {1}{m} \mathbb {E} _ {\sigma} \left[ \sup  _ {h \in \mathcal {H}} \sum_ {i = 1} ^ {m} \sigma_ {i} \Phi_ {i} (h \left(x _ {i}\right)) \right] \leq \frac {\sqrt {2}}{m} \mathbb {E} _ {\epsilon} \left[ \sup  _ {h \in \mathcal {H}} \sum_ {i = 1} ^ {m} \sum_ {j = 1} ^ {N} \epsilon_ {i j} \mu_ {i} h _ {j} \left(x _ {i}\right) \right] \tag {46}
$$
where the  $\sigma_{i}$ 's and the  $\epsilon_{ij}$ 's are independent Rademacher variables uniformly distributed over  $\{-1, +1\}$ .
Lemma 7 (Lipschitzness). For a given scoring function  $f: \mathcal{X} \times \mathcal{Y} \to \mathbb{R}$  and a vector of instances  $\boldsymbol{x} \in \mathcal{X}^M$ , let  $f(\boldsymbol{x}) = [f^y(x_i)]_{i \in [M], y \in \mathcal{Y}} \in \mathbb{R}^{M \times |\mathcal{Y}|}$  denote the vector of scores for each label. Then, for any two scoring functions  $f, f' \in \mathcal{F}$  and any sample  $(\boldsymbol{x}, s) \in \mathcal{D}_{\mathsf{P}}$ , we have:
$$
\left| \tilde {\ell} _ {\sigma} ^ {k} (f (\boldsymbol {x}), s) - \tilde {\ell} _ {\sigma} ^ {k} \left(f ^ {\prime} (\boldsymbol {x}), s\right) \right| \leq \sqrt {M k} \| f (\boldsymbol {x}) - f ^ {\prime} (\boldsymbol {x}) \| _ {2} \tag {47}
$$
Proof. Given  $(x, s)$ , the derivative of the WMC with respect to the score  $f^y(x_i)$  for a label  $y \in \mathcal{V}$  is 0 if  $A_{i,y}$  does not appear in formula  $\varphi$ . Otherwise,
$$
\begin{array}{l} \frac {\operatorname {W M C} \left(\bigvee_ {i = 1} ^ {k} \boldsymbol {y} ^ {(i)} , f (\boldsymbol {x})\right)}{\mathrm {d} \left(f ^ {y} \left(x _ {i}\right)\right)} = \sum_ {\text {M o d e l I o f} \varphi} \prod_ {A \in \varphi | I (A) = \top} \omega (A) \cdot \prod_ {A \in \varphi | I (A) = \bot} (1 - \omega (A)) \\ = \sum_ {\text {M o d e l} I \text {o f} \varphi} \mathcal {I} _ {I} \left(A _ {i, y}\right) \prod_ {A \in \varphi_ {i, y} \mid I (A) = \top} \omega (A) \cdot \prod_ {A \in \varphi_ {i, y} \mid I (A) = \bot} \left(1 - \omega (A)\right) \\ \leq \sum_ {\text {M o d e l} I \text {o f} \varphi A \in \varphi_ {i, y} | I (A) = \top} \omega (A) \cdot \prod_ {A \in \varphi_ {i, y} | I (A) = \bot} (1 - \omega (A)) \\ = \frac {\operatorname {W M C} \left(\bigvee_ {i = 1} ^ {k} \boldsymbol {y} ^ {(i)} , f (\boldsymbol {x})\right)}{\operatorname {d} \left(f ^ {y} \left(x _ {i}\right)\right)} \\ \leq 1 \\ \end{array}
$$
$$
\mathcal {I} _ {I} \left(A _ {i, y}\right) = \left\{ \begin{array}{l l} 1 & I \left(A _ {i, y}\right) = \top \\ - 1 & \text {o t h e r w i s e} \end{array} \right. \tag {48}
$$
and  $\varphi_{i,y}$  is the logic formula that deletes all occurrences of  $A_{i,y}$  in  $\varphi$ . Therefore, the 2-norm of the gradient of the mapping  $f(\pmb{x}) \mapsto \tilde{\ell}_{\mathbb{P}}^{k}(f(\pmb{x}), s)$  is upper bounded by
$$
\sqrt {(\text {n u m b e r o f B o o l e a n v a r i a b l e s i n} \varphi)} \leq \sqrt {M k} \tag {49}
$$
The boundedness of the gradient implies that the function  $f(\pmb{x}) \mapsto \mathrm{WMC}(\bigvee_{i=1}^{k} \pmb{y}^{(i)}, f(\pmb{x}))$  is Lipschitz with a Lipschitz constant  $\sqrt{Mk}$ , concluding the proof of Lemma 7.
We are now ready to present the proof of the main theorem of this section.
Proof. Bound with the partial risk. From the proof of Lemma 4 and  $I = 1$ , we know that
$$
\mathcal {R} ^ {0 1} (f) \leq \Phi \left(\mathcal {R} _ {\mathrm {P}} ^ {0 1} (f; \sigma)\right)
$$
$$
\Phi : t \mapsto \min  \left\{\frac {t}{\left(1 - \left(c ^ {2 M - 2} t\right) ^ {1 / M}\right) ^ {M - 1}}, t ^ {\frac {1}{M}} c ^ {2} \right\} \tag {50}
$$
We can see that  $\Phi$  is monotone increasing and  $\lim_{t\to 0}\Phi (t) / t = 1$
Bound the zero-one partial risk with the top- $k$  loss. From Lemma 5, we know that  $\ell_{\mathsf{P}}^{01}(f(\pmb{x}), s) \leq (k + 1)\tilde{\ell}_{\mathsf{P}}^{k}(f(\pmb{x}), s)$ . Taking expectation yields:
$$
\mathcal {R} _ {\mathsf {P}} ^ {0 1} (f; \sigma) \leq (k + 1) \mathcal {R} _ {\mathsf {P}} ^ {k} (f; \sigma)
$$
Bound with the empirical risk. Given a partially labelled dataset  $\mathcal{T}_{\mathsf{P}} = \{(x_i,s_i)\}_{i = 1}^{m_{\mathsf{P}}}$ , by the standard Rademacher complexity bounds, we know that for each  $\delta \in (0,1)$  and each  $f\in \mathcal{F}$ , the following inequality holds with probability at least  $1 - \delta$ :
$$
\mathcal {R} _ {\mathsf {P}} ^ {k} (f; \sigma) \leq \widehat {\mathcal {R}} _ {\mathsf {P}} (f; \tilde {\ell} _ {\mathsf {P}} ^ {k}; \mathcal {T} _ {\mathsf {P}}) + 2 \Re_ {m _ {\mathsf {P}}} (\mathcal {A} _ {\mathcal {F}, k}) + \sqrt {\frac {\log (1 / \delta)}{2 m _ {\mathsf {P}}}} \tag {51}
$$
where  $\mathcal{A}_{\mathcal{F},k}$  is a class of functions that take as inputs elements of the form  $(\pmb{x},s)$  with  $\pmb{x} \in \mathcal{X}^M$  and  $s \in S$ , and is defined as follows:
$$
\mathcal {A} _ {\mathcal {F}, k} := \left\{(\boldsymbol {x}, s) \mapsto \tilde {\ell} _ {\mathsf {P}} ^ {k} (f (\boldsymbol {x}), s): f \in \mathcal {F} \right\} \tag {52}
$$
By Lemma 6 with  $\Psi_{i}:f(\pmb{x}_{i})\to \tilde{\ell}_{\mathsf{P}}^{k}(f(\pmb{x}_{i}),s_{i})$ , and Lipschitzness (Lemma 7) we have:
$$
\begin{array}{l} \mathfrak {R} _ {m} (\mathcal {A} _ {\mathcal {F}, k}) = \frac {1}{m _ {\mathsf {P}}} \mathbb {E} _ {\mathcal {T} _ {\mathsf {P}}} \mathbb {E} _ {\sigma} \left[ \sup _ {f \in \mathcal {F}} \sum_ {i = 1} ^ {m _ {\mathsf {P}}} \sigma_ {i} \tilde {\ell} _ {\mathsf {P}} ^ {k} (f (\boldsymbol {x} _ {i}), s _ {i}) \right] \\ = \sqrt {k M} \frac {1}{m _ {\mathsf {P}}} \mathbb {E} _ {\mathcal {T}} \mathbb {E} _ {\epsilon} \left[ \sup  _ {f \in \mathcal {F}} \sum_ {i = 1} ^ {m _ {\mathsf {P}}} \sum_ {j = 1} ^ {M} \sum_ {y \in \mathcal {Y}} \epsilon_ {i j y} f ^ {y} \left(x _ {i j}\right) \right] \tag {53} \\ = \sqrt{kM} M\underbrace{\frac{1}{Mm_{\mathsf{P}}}\mathbb{E}_{\mathcal{T}}\mathbb{E}_{\epsilon}\left[\sup_{f\in\mathcal{F}}\sum_{i=1}^{m_{\mathsf{P}}}\sum_{j=1}^{M}\sum_{y\in\mathcal{Y}}\epsilon_{ijy}f^{y}(x_{ij})\right]}_{\text{Rademacher complexity with }M\times m_{P}\text{instances.}} \\ = \sqrt {k} M ^ {3 / 2} \Re_ {M m _ {\mathsf {P}}} (\mathcal {F}) \\ \end{array}
$$
Bound with cross-entropy. Finally, from the inequality  $-\log (1 - t) \geq t \forall t > 0$ , we know that
$$
\widehat {\mathcal {R}} _ {\mathrm {P}} \left(f; \tilde {\ell} _ {\mathrm {P}} ^ {k}; \mathcal {T} _ {\mathrm {P}}\right) \leq \widehat {\mathcal {R}} _ {\mathrm {P}} \left(f; \ell_ {\mathrm {P}} ^ {k}; \mathcal {T} _ {\mathrm {P}}\right) \tag {54}
$$
Putting the above inequalities together yields the proof of Theorem 2.
C.1 On the bounded risk assumption
In Section 4, we introduced the bounded risk assumption. Recall that the assumption requires a constant  $R < 1$ , such that for each  $i \in [n]$  and each  $f \in \mathcal{F}_i$ ,  $\mathcal{R}^{01}(f) \leq R$  holds. Below, we discuss how this assumption is achieved with a small amount of directly labeled data  $\mathcal{T}_{\mathrm{L}} = \{(x_i, y_i)\}_{i=1}^{m_{\mathrm{L}}} \text{i.i.d.}$  drawn from  $\mathcal{D}$ .