SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[39] Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. Neural attentive session-based recommendation. In CIKM, pages 747-755, 2017.
[40] Jiaxi Tang, Francois Belletti, Sagar Jain, Minmin Chen, Alex Beutel, Can Xu, and Ed H. Chi. Towards neural mixture recommender for long range dependent user sequences. In TheWebConf, pages 1782-1793, 2019.
[41] Chen Ma, Peng Kang, and Xue Liu. Hierarchical gating networks for sequential recommendation. In KDD, pages 825-833, 2019.
[42] Ziwei Fan, Zhiwei Liu, Shen Wang, Lei Zheng, and Philip S Yu. Modeling sequences as distributions with uncertainty for sequential recommendation. In CIKM, pages 3019-3023, 2021.
[43] Di-Rong Chen and Tao Sun. Consistency of multiclass empirical risk minimization methods based on convex loss. JMLR, 7:2435-2447, 2006.
[44] Jingtao Ding, Yuhan Quan, Quanming Yao, Yong Li, and Depeng Jin. Simplify and robustify negative sampling for implicit collaborative filtering. In NeurIPS, pages 1094-1105, 2020.
[45] F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. ACM TIIIS, 5(4):1-19, 2015.
[46] Julian McAuley and Jure Leskovec. Hidden factors and hidden topics: understanding rating dimensions with review text. In RecSys, pages 165-172, 2013.
[47] Guanghu Yuan, Fajie Yuan, Yudong Li, Beibei Kong, Shujie Li, Lei Chen, Min Yang, Chenyun YU, Bo Hu, Zang Li, Yu Xu, and Xiaohu Qie. Tenrec: A large-scale multipurpose benchmark dataset for recommender systems. In NeurIPS, pages 11480-11493, 2022.
[48] Zhu Sun, Di Yu, Hui Fang, Jie Yang, Xinghua Qu, Jie Zhang, and Cong Geng. Are we evaluating rigorously? benchmarking recommendation for reproducible evaluation and fair comparison. In RecSys, pages 23-32, 2020.
[49] Zhu Sun, Hui Fang, Jie Yang, Xinghua Qu, Hongyang Liu, Di Yu, Yew-Soon Ong, and Jie Zhang. Daisyrec 2.0: Benchmarking recommendation for rigorous evaluation. IEEE TPAMI, 45(7):8206-8226, 2023.
[50] Walid Krichene and Steffen Rendle. On sampled metrics for item recommendation. In KDD, pages 1748-1757, 2020.
[51] Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In AISTATS, pages 249-256, 2010.
[52] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
Theoretically Guaranteed Bidirectional Data Rectification for Robust Sequential Recommendation - Appendix -
This Appendix is divided into three sections. First, Section A empirically validates the feasibility of Assumption 1. Next, in Section B, complete proofs of all the lemmas and theorems are presented. Finally, in Section C, we provide detailed settings of baselines and additional experimental results, including the hyper-parameter analysis and the percentage of rectified data. Our code is available at: https://github.com/AlchemistYT/BirDRec.
A The Feasibility of Assumption 1
Assumption 1. The users' true preference distribution  $\eta$  fulfills the relaxed Multiclass Tsybakov Condition [1] with constants  $C > 0$ ,  $\lambda > 0$ , and  $\alpha_0 \in (0,1]$ , such that for all  $\alpha \in (0,\alpha_0]$
$$
\mathbb {P} \left[ \eta_ {p _ {1}} \left(\mathbf {x} _ {t} ^ {u}\right) - \eta_ {p _ {2}} \left(\mathbf {x} _ {t} ^ {u}\right) \leq \alpha \right] \leq C \alpha^ {\lambda}. \tag {1}
$$
The feasibility of Assumption 1 relies on large  $C$  and small  $\lambda$ . In order to estimate the values of  $C$  and  $\lambda$ , the initial step is to approximate the true preference distribution  $\eta$ . To achieve this, we first obtain a reliable dataset  $\mathcal{D}$  via the heuristic method proposed by [2]. The heuristic method measures the matching degree between the input and target of each instance from two aspects: item co-occurrence and item properties, then those instances with matching degrees lower than a threshold are filtered as unreliable data. We use a large threshold (0.9) to filter the dataset rigorously, ensuring the vast majority of the maintained instances in  $\mathcal{D}$  are reliable. Subsequently, as suggested by [3], we train a classic SRS, SASRec [4], on the filtered reliable dataset  $\mathcal{D}$  and then use the prediction of SASRec to approximate  $\eta$ . Formally, for each reliable instance  $\langle \mathbf{x}_t^u, v_t^u \rangle$  from the reliable dataset  $\mathcal{D}$ , the SRS prediction score  $f_{v_t^u}(\mathbf{x}_t^u)$  is employed as the approximation of  $\eta_{v_t^u}(\mathbf{x}_t^u)$ .
Next, we densely sample  $\alpha$  from 0.05 to 0.9 with step size 0.005 and calculate the corresponding left-hand-side probability of Eq. 1 with the following relative frequency  $F_{\alpha}$ , and collect a series of  $(\log (\alpha),\log (F_{\alpha}))$  data points:
$$
F _ {\alpha} = \frac {1}{| \mathcal {D} |} \sum_ {\left\langle \mathbf {x} _ {t} ^ {u}, v _ {t} ^ {u} \right\rangle \in \mathcal {D}} \mathbb {I} \left[ f _ {\hat {p} _ {1}} \left(\mathbf {x} _ {t} ^ {u}\right) - f _ {\hat {p} _ {2}} \left(\mathbf {x} _ {t} ^ {u}\right) \leq \alpha \right], \tag {2}
$$
where  $\hat{p}_1$  and  $\hat{p}_2$  are respectively the top- and middle-ranked items according to  $f$ , namely,  $\sum_{v_i \in \mathcal{V}} \mathbb{I}\big[f_{\hat{p}_1}(\mathbf{x}_t^u) \geq f_{v_i}(\mathbf{x}_t^u)\big] = |\mathcal{V}|, \sum_{v_i \in \mathcal{V}} \mathbb{I}\big[f_{\hat{p}_2}(\mathbf{x}_t^u) \geq f_{v_i}(\mathbf{x}_t^u)\big] = \lfloor |\mathcal{V}| / 2\rfloor$ . We then use  $\log(F_\alpha)$  to approximate  $\log(C\alpha^\lambda)$ . As shown by the blue dots in Fig. 6, the collected  $(\log(\alpha), \log(F_\alpha))$  data points are generally linearly distributed, which allows us to estimate  $C$  and  $\lambda$  with linear regression according to  $\log(C\alpha^\lambda) = \log(C) + \lambda \log(\alpha)$ . As a result, Fig. 6 shows that the estimated  $C$  and  $\lambda$  are respectively restricted in (0.55, 0.70) and (1.37, 4.01) on real-world datasets from various domains, which validates the feasibility of Assumption 1.
(a) ML-1M
(b) Beauty
(c) Yelp
Figure 1: The estimated constants  $C$  and  $\lambda$  on various datasets.
(d) QK-Vedio
B Proofs of Lemmas and Theorems
B.1 The Proof of Theorem 1
Theorem 1. Given Assumption 1, let  $\{w_h \mid 1 \leq h \leq H, 0 \leq w_h \leq 1, \sum_{h=1}^{H} w_h = 1\}$  be the weights for averaging prediction scores of different epochs.  $\forall \langle \tilde{\mathbf{x}}_t^u, \tilde{v}_t^u \rangle$ , assume  $\epsilon \leq \alpha_0 \tau_{\tilde{v}_t^u \tilde{v}_t^u}(\tilde{\mathbf{x}}_t^u)$ . Let  $\gamma = \tau_{\tilde{v}_t^u \tilde{v}_t^u}(\tilde{\mathbf{x}}_t^u) \eta_{p_2}(\tilde{\mathbf{x}}_t^u) + \sum_{v_j \neq \tilde{v}_t^u} \tau_{v_j \tilde{v}_t^u}(\tilde{\mathbf{x}}_t^u) \eta_{v_j}(\tilde{\mathbf{x}}_t^u)$ . We have:  $\mathbb{P}\left[p_1 = \tilde{v}_t^u, \sum_{h=1}^{H} [w_h f_{\tilde{v}_t^u}^h(\tilde{\mathbf{x}}_t^u)] \leq \gamma \right] \leq C(\mathcal{O}(\epsilon))^{\lambda}$ .
$$
\begin{array}{l} \mathbb {P} \left[ p _ {1} = \tilde {v} _ {t} ^ {u}, \sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {\tilde {v} _ {t} ^ {u}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right] \leq \gamma \right] \\ \leq \mathbb {P} \left[ p _ {1} = \tilde {v} _ {t} ^ {u}, \sum_ {h = 1} ^ {H} w _ {h} \left[ \tilde {\eta} _ {\tilde {v} _ {t} ^ {u}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) - \epsilon \right] \leq \gamma \right] \\ = \mathbb {P} \left[ p _ {1} = \tilde {v} _ {t} ^ {u}, \eta_ {\tilde {v} _ {t} ^ {u}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \geq \eta_ {p _ {2}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right), \sum_ {h = 1} ^ {H} w _ {h} \left[ \sum_ {v _ {j} \in \mathcal {V}} \tau_ {v _ {j} \tilde {v} _ {t} ^ {u}} \eta_ {v _ {j}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) - \epsilon \right] \leq \gamma \right] \tag {3} \\ \leq \mathbb {P} \left[ p _ {1} = \tilde {v} _ {t} ^ {u}, \eta_ {\tilde {v} _ {t} ^ {u}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \geq \eta_ {p _ {2}} (\tilde {\mathbf {x}} _ {t} ^ {u}), \tau_ {\tilde {v} _ {t} ^ {u} \tilde {v} _ {t} ^ {u}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \eta_ {\tilde {v} _ {t} ^ {u}} (\tilde {\mathbf {x}} _ {t} ^ {u}) + \sum_ {v _ {j} \neq \tilde {v} _ {t} ^ {u}} \tau_ {v _ {j} \tilde {v} _ {t} ^ {u}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \eta_ {v _ {j}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \leq \gamma + \epsilon \right] \\ = \mathbb {P} \left[ p _ {1} = \tilde {v} _ {t} ^ {u}, \eta_ {p _ {2}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \leq \eta_ {\tilde {v} _ {t} ^ {u}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \leq \frac {\gamma + \epsilon - \sum_ {v _ {j} \neq \tilde {v} _ {t} ^ {u}} \tau_ {v _ {j} \tilde {v} _ {t} ^ {u}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \eta_ {v _ {j}} (\tilde {\mathbf {x}} _ {t} ^ {u})}{\tau_ {\tilde {v} _ {t} ^ {u} \tilde {v} _ {t} ^ {u}} (\tilde {\mathbf {x}} _ {t} ^ {u})} \right] \\ \end{array}
$$
$$
\mathbb {P} \left[ p _ {1} = \tilde {v} _ {t} ^ {u}, \sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {\tilde {v} _ {t} ^ {u}} ^ {h} (\tilde {\mathbf {x}} _ {t} ^ {u}) \right] \leq \gamma \right] \leq \mathbb {P} \left[ \eta_ {p _ {2}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \leq \eta_ {p _ {1}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \leq \eta_ {p _ {2}} (\tilde {\mathbf {x}} _ {t} ^ {u}) + \frac {\epsilon}{\tau_ {\tilde {v} _ {t} ^ {u} \tilde {v} _ {t} ^ {u}} (\tilde {\mathbf {x}} _ {t} ^ {u})} \right] (4)
$$
Recall that  $\epsilon \leq \alpha_0\tau_{\tilde{v}_t^u\tilde{v}_t^u}(\tilde{\mathbf{x}}_t^u)$ , which implies  $\frac{\epsilon}{\tau_{\tilde{v}_t^u\tilde{v}_t^u}(\tilde{\mathbf{x}}_t^u)} \leq \alpha_0$ . Hence, the relaxed Multiclass Tsybakov Condition holds and the probability is bounded by  $C\left(\frac{\epsilon}{\tau_{\tilde{v}_t^u\tilde{v}_t^u}(\tilde{\mathbf{x}}_t^u)}\right)^\lambda$ , namely,  $C\big(\mathcal{O}(\epsilon)\big)^\lambda$ .
B.2 The Proof of Lemma 1
$$
\boldsymbol {E} _ {\text {D R U T}} = \underbrace {\mathbb {P} \left[ p _ {1} = \tilde {v} _ {t} ^ {u} , p _ {1} \neq v _ {m} , \frac {\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {\tilde {v} _ {t} ^ {u}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]}{\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {m}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]} <   \beta \right]} _ {\text {C a s e - 1}} + \underbrace {\mathbb {P} \left[ p _ {1} \neq \tilde {v} _ {t} ^ {u} , p _ {1} = v _ {m} , \frac {\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {\tilde {v} _ {t} ^ {u}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]}{\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {{m}}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]} \geq \beta \right]} _ {\text {C a s e - 2}} + \underbrace {\mathbb {P} \left[ p _ {1} \neq \tilde {v} _ {t} ^ {u} , p _ {1} \neq v _ {m} \right]} _ {\text {C a s e - 3}}.
$$
Lemma 1. Given Assumption 1 and the set of weights  $\{w_h \mid 1 \leq h \leq H, 0 \leq w_h \leq 1, \sum_{h=1}^{H} w_h = 1\}$ ,  $\forall \langle \tilde{\mathbf{x}}_t^u, \tilde{v}_t^u \rangle$ , assume  $\epsilon \leq \min \left[ \alpha_0 \tau_{\tilde{v}_t^u \tilde{v}_t^u} (\tilde{\mathbf{x}}_t^u), \alpha_0 \tau_{v_m v_m} (\tilde{\mathbf{x}}_t^u) \right]$ . Let  $\beta_1 =$
$\left[\frac{\tau_{\bar{v}_t^u\bar{v}_t^u}(\bar{\mathbf{x}}_t^u)\eta_{p_2}(\bar{\mathbf{x}}_t^u) + \sum_{v_j\neq \bar{v}_t^u}\tau_{v_j\bar{v}_t^u}\eta_{v_j}(\bar{\mathbf{x}}_t^u)}{\sum_{h = 1}^{H}[w_hf_{v_m}^h(\bar{\mathbf{x}}_t^u)]}\right]$  and  $\beta_{2} = \left[\frac{\sum_{h = 1}^{H}[w_{h}f_{\bar{v}_{t}^{h}}^{h}(x_{t}^{u})]}{\tau_{v_{m}v_{m}}(\bar{\mathbf{x}}_{t}^{u})\eta_{p_{2}}(\bar{\mathbf{x}}_{t}^{u}) + \sum_{v_{j}\neq v_{m}}\tau_{v_{j}v_{m}}(\bar{\mathbf{x}}_{t}^{u})\eta_{v_{j}}(\bar{\mathbf{x}}_{t}^{u})}\right].$  We
have:  $\beta \leq \beta_{1}$  guarantees the probability of Case-1 in  $E_{\mathrm{DRUT}}$  is bounded by  $C\big(\mathcal{O}(\epsilon)\big)^{\lambda}$ , and  $\beta \geq \beta_{2}$  guarantees the probability of Case-2 in  $E_{\mathrm{DRUT}}$  is bounded by  $C\big(\mathcal{O}(\epsilon)\big)^{\lambda}$ .
Proof. For Case-1 of  $E_{\mathrm{DRUT}}$ , we have:
$$
\begin{array}{l} \mathbb {P} \left[ p _ {1} = \tilde {v} _ {t} ^ {u}, p _ {1} \neq v _ {m}, \frac {\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {\tilde {v} _ {t} ^ {u}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]}{\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {m}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]} <   \beta_ {1} \right] \\ \leq \mathbb {P} \left[ p _ {1} = \tilde {v} _ {t} ^ {u}, \frac {\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {\tilde {v} _ {t} ^ {u}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]}{\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {m}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]} <   \beta_ {1} \right] \\ \leq \mathbb {P} \left[ p _ {1} = \tilde {v} _ {t} ^ {u}, \eta_ {\tilde {v} _ {t} ^ {u}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \geq \eta_ {p 2} (\tilde {\mathbf {x}} _ {t} ^ {u}), \sum_ {h = 1} ^ {H} w _ {h} \left[ \tilde {\eta} _ {\tilde {v} _ {t} ^ {u}} (\tilde {\mathbf {x}} _ {t} ^ {u}) - \epsilon \right] <   \beta_ {1} \sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {m}} ^ {h} (\tilde {\mathbf {x}} _ {t} ^ {u}) \right] \right] \\ = \mathbb {P} \left[ p _ {1} = \tilde {v} _ {t} ^ {u}, \eta_ {\tilde {v} _ {t} ^ {u}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \geq \eta_ {p _ {2}} (\tilde {\mathbf {x}} _ {t} ^ {u}), \tilde {\eta} _ {\tilde {v} _ {t} ^ {u}} (\tilde {\mathbf {x}} _ {t} ^ {u}) <   \beta_ {1} \sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {m}} ^ {h} (\tilde {\mathbf {x}} _ {t} ^ {u}) \right] + \epsilon \right] \\ = \mathbb {P} \left[ p _ {1} = \tilde {v} _ {t} ^ {u}, \eta_ {\tilde {v} _ {t} ^ {u}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \geq \eta_ {p _ {2}} (\tilde {\mathbf {x}} _ {t} ^ {u}), \sum_ {v _ {j} \in \nu} \tau_ {v _ {j} \tilde {v} _ {t} ^ {u}} \eta_ {v _ {j}} (\tilde {\mathbf {x}} _ {t} ^ {u}) <   \beta_ {1} \sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {m}} ^ {h} (\tilde {\mathbf {x}} _ {t} ^ {u}) \right] + \epsilon \right] \\ = \mathbb {P} \left[ p _ {1} = \tilde {v} _ {t} ^ {u}, \eta_ {p _ {2}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \leq \eta_ {\tilde {v} _ {t} ^ {u}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) <   \frac {\beta_ {1} \sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {m}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right] + \epsilon - \sum_ {v _ {j} \neq \tilde {v} _ {t} ^ {u}} \left[ \tau_ {v _ {j} \tilde {v} _ {t} ^ {u}} \eta_ {v _ {j}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]}{\tau_ {\tilde {v} _ {t} ^ {u} \tilde {v} _ {t} ^ {u}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right)} \right]. \tag {5} \\ \end{array}
$$
By substituting  $\beta_{1}$  with  $\frac{\tau_{\bar{v}_t^u\bar{v}_t^u}(\bar{\mathbf{x}}_t^u)\eta_{p_2}(\bar{\mathbf{x}}_t^u) + \sum_{v_j\neq \bar{v}_t^u}\tau_{v_j\bar{v}_t^u}\eta_{v_j}(\bar{\mathbf{x}}_t^u)}{\sum_{h = 1}^{H}[w_hf_{v_m}^h(\bar{\mathbf{x}}_t^u)]}$  in Eq. 7, we obtain:
$$
\begin{array}{l} \mathbb {P} \left[ p _ {1} = \tilde {v} _ {t} ^ {u}, p _ {1} \neq v _ {m}, \frac {\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {\tilde {v} _ {t} ^ {u}} ^ {h} (\tilde {\mathbf {x}} _ {t} ^ {u}) \right]}{\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {m}} ^ {h} (\tilde {\mathbf {x}} _ {t} ^ {u}) \right]} <   \beta_ {1} \right] \\ \leq \mathbb {P} \left[ \eta_ {p _ {2}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \leq \eta_ {p _ {1}} (\tilde {\mathbf {x}} _ {t} ^ {u}) <   \eta_ {p _ {2}} (\tilde {\mathbf {x}} _ {t} ^ {u}) + \frac {\epsilon}{\tau_ {\tilde {v} _ {t} ^ {u} \tilde {v} _ {t} ^ {u}} (\tilde {\mathbf {x}} _ {t} ^ {u})} \right]. \\ \end{array}
$$
Recall that  $\epsilon \leq \alpha_0\tau_{\tilde{v}_t^u\tilde{v}_t^u}(\tilde{\mathbf{x}}_t^u)$ , which implies  $\frac{\epsilon}{\tau_{\tilde{v}_t^u\tilde{v}_t^u}(\tilde{\mathbf{x}}_t^u)} \leq \alpha_0$ . Hence, the Multiclass Tsybakov Condition holds and the probability of Case-1 is bounded by  $C\left(\frac{\epsilon}{\tau_{\tilde{v}_t^u\tilde{v}_t^u}(\tilde{\mathbf{x}}_t^u)}\right)^\lambda$ , namely,  $C\big(\mathcal{O}(\epsilon)\big)^\lambda$ .
Thereafter, for Case-2 of  $E_{\mathrm{DRUT}}$ , we have:
$$
\begin{array}{l} \mathbb {P} \left[ p _ {1} \neq \tilde {v} _ {t} ^ {u}, p _ {1} = v _ {m}, \frac {\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {t} ^ {u}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]}{\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {m}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]} \geq \beta_ {2} \right] \\ \leq \mathbb {P} \left[ p _ {1} = v _ {m}, \frac {\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {t} ^ {u}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]}{\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {m}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]} \geq \beta_ {2} \right] \\ \leq \mathbb {P} \left[ p _ {1} = v _ {m}, \eta_ {m} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \geq \eta_ {p _ {2}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right), \sum_ {h = 1} ^ {H} w _ {h} \left[ \tilde {\eta} _ {m} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) - \epsilon \right] \leq \frac {\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {m}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]}{\beta_ {2}} \right] \\ = \mathbb {P} \left[ p _ {1} = v _ {m}, \eta_ {m} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \geq \eta_ {p 2} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right), \tilde {\eta} _ {m} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \leq \frac {\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {m}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]}{\beta_ {2}} + \epsilon \right] \\ = \mathbb {P} \left[ p _ {1} = v _ {m}, \eta_ {m} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \geq \eta_ {p _ {2}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right), \sum_ {v _ {j} \in \nu} \tau_ {v _ {j} v _ {m}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \eta_ {v _ {j}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \leq \frac {\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {m}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]}{\beta_ {2}} + \epsilon \right] \\ = \mathbb {P} \left[ p _ {1} = v _ {m}, \eta_ {p _ {2}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \leq \eta_ {m} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \leq \frac {\frac {\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {m}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]}{\beta_ {2}} - \sum_ {v _ {j} \neq v _ {m}} \left[ \tau_ {v _ {j} v _ {m}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \eta_ {v _ {j}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]}{\tau_ {v _ {m} v _ {m}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right)} + \frac {\epsilon}{\tau_ {v _ {m} v _ {m}} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right)} \right]. \\ \end{array}
$$
We replace  $\beta_{2}$  with  $\frac{\sum_{h = 1}^{H}[w_{h}f_{v_{t}^{u}}^{h}(\tilde{\mathbf{x}}_{t}^{u})]}{\tau_{v_{m}v_{m}}(\tilde{\mathbf{x}}_{t}^{u})\eta_{p_{2}}(\tilde{\mathbf{x}}_{t}^{u}) + \sum_{v_{j}\neq v_{m}}\tau_{v_{j}v_{m}}(\tilde{\mathbf{x}}_{t}^{u})\eta_{v_{j}}(\tilde{\mathbf{x}}_{t}^{u})}$  in Eq. 5 and continue the calculation:
$$
\begin{array}{l} \mathbb {P} \left[ p _ {1} \neq \tilde {v} _ {t} ^ {u}, p _ {1} = v _ {m}, \frac {\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {\tilde {v} _ {t} ^ {u}} ^ {h} (\tilde {\mathbf {x}} _ {t} ^ {u}) \right]}{\sum_ {h = 1} ^ {H} \left[ w _ {h} f _ {v _ {m}} ^ {h} (\tilde {\mathbf {x}} _ {t} ^ {u}) \right]} \geq \beta_ {2} \right] \\ \leq \mathbb {P} \left[ \eta_ {p _ {2}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \leq \eta_ {m} (\tilde {\mathbf {x}} _ {t} ^ {u}) \leq \eta_ {p _ {2}} (\tilde {\mathbf {x}} _ {t} ^ {u}) + \frac {\epsilon}{\tau_ {v _ {m} v _ {m}} (\tilde {\mathbf {x}} _ {t} ^ {u})} \right]. \\ \end{array}
$$
Recall that  $\epsilon \leq \alpha_0\tau_{v_m v_m}(\tilde{\mathbf{x}}_t^u)$ , which implies  $\frac{\epsilon}{\tau_{v_m v_m}(\tilde{\mathbf{x}}_t^u)} \leq \alpha_0$ . Hence, the Multiclass Tsybakov Condition holds and the probability of Case-2 is bounded by  $C\left(\frac{\epsilon}{\tau_{v_m v_m}(\tilde{\mathbf{x}}_t^u)}\right)^\lambda$ , namely,  $C\big(\mathcal{O}(\epsilon)\big)^\lambda$ .
B.3 The Proof of Theorem 2
Theorem 2 (The Upper Bound of  $E_{\mathrm{DRUT}}$ ). Given Assumption 1 and the set of weights  $\{w_h \mid 1 \leq h \leq H, 0 \leq w_h \leq 1, \sum_{h=1}^{H} w_h = 1\}$ ,  $\forall \langle \tilde{\mathbf{x}}_t^u, \tilde{v}_t^u \rangle$ , let  $\beta_1 = \left[\frac{\tau_{\tilde{v}_t^u \tilde{v}_t^u} (\tilde{\mathbf{x}}_t^u) \eta_{p_2} (\tilde{\mathbf{x}}_t^u) + \sum_{v_j \neq \tilde{v}_t^u} \tau_{v_j \tilde{v}_t^u} \eta_{v_j} (\tilde{\mathbf{x}}_t^u)}{\sum_{h=1}^{H} [w_h f_{vm}^h (\tilde{\mathbf{x}}_t^u)]}\right]$ ,
$$
\begin{array}{l}\beta_ {2} = \left[ \right. \frac {\sum_ {h = 1} ^ {H} [ w _ {h} f _ {t} ^ {u} (\tilde {\mathbf {x}} _ {t} ^ {u}) ]}{\tau_ {v _ {m} v _ {m}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \eta_ {p _ {2}} (\tilde {\mathbf {x}} _ {t} ^ {u}) + \sum_ {v _ {j} \neq v _ {m}} \tau_ {v _ {j} v _ {m}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \eta_ {v _ {j}} (\tilde {\mathbf {x}} _ {t} ^ {u})}, \xi_ {1} = | \beta - \beta_ {1} |, a n d \xi_ {2} = | \beta - \beta_ {2} |. A s s u m e\\\xi_ {2} <   \beta_ {2},   \epsilon \leq \min  \left[ \right. \alpha_ {0} \tau_ {\tilde {v} _ {t} ^ {u} \tilde {v} _ {t} ^ {u}} (\tilde {\mathbf {x}} _ {t} ^ {u}) - \xi_ {1}, \frac {\alpha_ {0} \tau_ {v _ {m} v _ {m}} (\tilde {\mathbf {x}} _ {t} ^ {u}) \beta_ {2} (\beta_ {2} - \xi_ {2}) - \xi_ {2}}{\beta_ {2} (\beta_ {2} - \xi_ {2})}, \frac 12 \left[ [ \tau_ {p _ {1} p _ {1}} (\tilde {\mathbf {x}} _ {t} ^ {u}) - \tau_ {p _ {1} v _ {m}} (\tilde {\mathbf {x}} _ {t} ^ {u}) ] [ \alpha_ {0} + \eta_ {p _ {2}} (\tilde {\mathbf {x}} _ {t} ^ {u}) ] - \right.\\\sum_ {v _ {j} \neq p _ {1}} [ \tau_ {v _ {j} v _ {m}} (\tilde {\mathbf {x}} _ {t} ^ {u}) - \tau_ {v _ {j} p _ {1}} (\tilde {\mathbf {x}} _ {t} ^ {u}) ] [ \eta_ {v _ {j}} (\tilde {\mathbf {x}} _ {t} ^ {u}) ] \left. \right]. W e h a v e: E _ {\text {D R U T}} \leq C (O (\epsilon + \xi_ {1})) ^ {\lambda} + C (O (\epsilon + \xi_ {2})) ^ {\lambda} + C (O (\epsilon)) ^ {\lambda}.\end{array}
$$