SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem 2 (The Upper Bound of  $E_{\mathrm{DRUT}}$ ). Given Assumption 1 and the set of weights  $\{w_h \mid 1 \leq h, 0 \leq w_h \leq 1, \sum_{h=1}^{H} w_h = 1\}$ ,  $\forall \langle \tilde{\mathbf{x}}_t^u, \tilde{v}_t^u \rangle$ , let  $\beta_1 = \left[\frac{\tau_{\tilde{v}_t^u \tilde{v}_t^u}(\tilde{\mathbf{x}}_t^u) \eta_{p_2}(\tilde{\mathbf{x}}_t^u) + \sum_{v_j \neq \tilde{v}_t^u} \tau_{v_j \tilde{v}_t^u} \eta_{v_j}(\tilde{\mathbf{x}}_t^u)}{\sum_{h=1}^{H} [w_h f_{v_m}^h(\tilde{\mathbf{x}}_t^u)]}\right]$ ,  $\beta_2 = \left[\frac{\sum_{h=1}^{H} [w_h f_{\tilde{v}_t^u}^h(\tilde{\mathbf{x}}_t^u)]}{\tau_{v_m v_m} (\tilde{\mathbf{x}}_t^u) \eta_{p_2}(\tilde{\mathbf{x}}_t^u) + \sum_{v_j \neq v_m} \tau_{v_j v_m} (\tilde{\mathbf{x}}_t^u) \eta_{v_j}(\tilde{\mathbf{x}}_t^u)}\right]$ ,  $\xi_1 = |\beta - \beta_1|$ , and  $\xi_2 = |\beta - \beta_2|$ . Assume  $\xi_2 < \beta_2$ ,  $\epsilon \leq \min \left[\alpha_0 \tau_{\tilde{v}_t^u \tilde{v}_t^u} (\tilde{\mathbf{x}}_t^u) - \xi_1, \frac{\alpha_0 \tau_{v_m v_m} (\tilde{\mathbf{x}}_t^u) \beta_2 (\beta_2 - \xi_2) - \xi_2}{\beta_2 (\beta_2 - \xi_2)}, \frac{1}{2} \left[ [\tau_{p_1 p_1} (\tilde{\mathbf{x}}_t^u) - \tau_{p_1 v_m} (\tilde{\mathbf{x}}_t^u)][\alpha_0 + \eta_{p_2} (\tilde{\mathbf{x}}_t^u)] - \sum_{v_j \neq p_1} [\tau_{v_j v_m} (\tilde{\mathbf{x}}_t^u) - \tau_{v_j p_1} (\tilde{\mathbf{x}}_t^u)] \eta_{v_j}(\tilde{\mathbf{x}}_t^u)]\right]\right]$ . We have:  $E_{\mathrm{DRUT}} \leq C\big(\mathcal{O}(\epsilon + \xi_1)\big)^{\lambda} + C\big(\mathcal{O}(\epsilon + \xi_2)\big)^{\lambda} + C\big(\mathcal{O}(\epsilon)\big)^{\lambda}$ .
3.3 Theorems for Rectifying Unreliable Input
Unreliable items within the input act as mosaics to bewilder SRSs when learning users' true preferences, thus impeding SRSs from predicting the true target. Recall that Theorem 1 suggests consistently low-scored targets are unlikely to be reliable. This inspires us to exploit the prediction scores given by a backward SRS for rectifying unreliable input, that is, deleting consistently low-scored (predicted by a backward SRS) items within the input. To avoid deleting all input items with an unreliable target, we rectify the input of instances that are already processed by DRUT. Formally, given  $\langle \tilde{\mathbf{x}}_t^u,\hat{v}_t^u\rangle$ , we define a backward SRS  $\tilde{f}$  that aims to predict every input item in  $\tilde{\mathbf{x}}_t^u$  based on  $\tilde{\mathbf{x}}_t^u = \{u,\hat{v}_t^u\}$ . Hence, the backward instance for predicting  $\tilde{v}_{t - l}^u$ ,  $(l\in [1,L])$  can be formulated as  $\langle \tilde{\mathbf{x}}_t^u,\tilde{v}_{t - l}^u\rangle$ . With Assumption 1 being held, and  $\eta^\prime ,\tilde{\eta}^\prime ,p_1^\prime ,p_2^\prime ,\tau^\prime ,\epsilon^\prime$  respectively being the counterparts of  $\eta ,\tilde{\eta},p_{1},p_{2},$ $\tau ,\epsilon$  , for backward sequences, we propose the following strategy.
DDUI: Detecting and Deleting Unreliable Input. Given  $\tilde{f}$  that is  $\epsilon'$ -close to  $\tilde{\eta}'$ ,  $\langle \tilde{\mathbf{x}}_t^u, \tilde{v}_{t-l}^u \rangle$ , the consistently high-scored item  $\tilde{v}_m = \arg \max_{v_i \neq \tilde{v}_{t-l}^u} \sum_{h=1}^{H} [w_h \tilde{f}_{v_i}^h (\tilde{\mathbf{x}}_t^u)]$ , and  $\beta' \in (0,1]$ , we stipulate that if the ratio  $\sum_{h=1}^{H} [w_h \tilde{f}_{\tilde{v}_{t-l}^u}^h (\tilde{\mathbf{x}}_t^u)] / \sum_{h=1}^{H} [w_h \tilde{f}_{\tilde{v}_m}^h (\tilde{\mathbf{x}}_t^u)] < \beta'$ , i.e.,  $\tilde{v}_{t-l}^u$  is consistently lower-scored than  $\tilde{v}_m$  to some extent, then item  $\tilde{v}_{t-l}^u$  should be deleted from  $\tilde{\mathbf{x}}_t^u$  in the  $H$ -th epoch. After rectifying the  $L$  items in  $\tilde{\mathbf{x}}_t^u$ , we denote the output instance of DDUI as  $\langle \hat{\mathbf{x}}_t^u, \hat{v}_t^u \rangle$ .
DDUI differs from DRUT slightly, i.e., we remove unreliable input items to bridge the succeeding and preceding items, instead of replacing them. Since unreliable input items caused by distractions are essentially interruptions in the input sequence, replacing them may introduce new interruptions.
Superior to prior works, DDUI is theoretically guaranteed to work well with bounded error, which comes from two cases: (1)  $\tilde{v}_{t - l}^{u}$  is a reliable input item ( $p_1' = \tilde{v}_{t - l}^u$ ) but is deleted; and (2)  $\tilde{v}_{t - l}^u$  is an unreliable input item ( $p_1' \neq \tilde{v}_{t - l}^u$ ) but is kept. Hence, the error of DDUI, denoted as  $E_{\mathrm{DDUI}}$ , can be correspondingly formulated as the following Eq. 8, and its bound is analyzed in Theorem 3:
$$
\boldsymbol {E} _ {\mathrm {D D U I}} = \mathbb {P} \left[ p _ {1} ^ {\prime} = \tilde {v} _ {t - l} ^ {u}, \frac {\sum_ {h = 1} ^ {H} \left[ w _ {h} \tilde {f} _ {\tilde {v} _ {t - l}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]}{\sum_ {h = 1} ^ {H} \left[ w _ {h} \tilde {f} _ {\tilde {v} _ {m}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]} <   \beta^ {\prime} \right] + \mathbb {P} \left[ p _ {1} ^ {\prime} \neq \tilde {v} _ {t - l} ^ {u}, \frac {\sum_ {h = 1} ^ {H} \left[ w _ {h} \tilde {f} _ {\tilde {v} _ {t - l}} ^ {h} \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right) \right]}{\sum_ {h = 1} ^ {H} (w _ {h} \tilde {f} _ {\tilde {v} _ {m}} ^ {h}) \left(\tilde {\mathbf {x}} _ {t} ^ {u}\right)} \geq \beta^ {\prime} \right]. \tag {8}
$$
2The proofs of Lemma 1 and following theorems can be found in the Appendix.
Figure 2: The overall architecture of the proposed BirDRec framework.
4 The Proposed BirDRec Framework
By integrating DRUT and DDUI into existing SRSs, we introduce BirDRec, a model-agnostic bidirectional data rectification framework, which can rectify both unreliable targets and input with theoretical guarantees for more robust SRSs. Yet, the complexity of BirDRec is prohibitively high due to the calculation and storage of prediction scores for each instance across different epochs in DRUT and DDUI. To ease this issue, we devise a rectification sampling strategy that avoids prediction on the full item set to replace unreliable targets or delete unreliable input, thereby reducing the time complexity of BirDRec. Meanwhile, we adopt the self-ensemble mechanism [14] to approximate the weighted average prediction scores of different epochs, thus avoiding preserving scores of all epochs and reducing the space complexity.
Framework Overview. Accordingly, the efficiency-improved BirDRec is depicted in Fig. 2. Specifically, BirDRec first leverages the self-ensembed forward SRS to rectify the target of an instance via DRUT, and then the input is rectified by the self-ensembed backward SRS via DDUI. Thereafter, the rectified instance and its  $L$  backward instances are respectively used to train the forward and backward SRSs, which are finally employed to update the corresponding self-ensembed SRSs.
4.1 Reducing Time Complexity via Rectification Sampling
The time complexity of BirDRec is primarily attributed to the search for consistently high-scored items  $(v_{m}$  and  $\tilde{v}_m)$ , which requires the calculation of prediction scores between every instance and all items. Concretely, the per-epoch time complexity of BirDRec is  $\mathcal{O}\big(N\cdot (|\mathcal{V}| + L)\big)$ , where  $N$  denotes the number of instances. To mitigate this substantial computational burden, we propose a rectification sampling strategy that circumvents the need to search the entire item set.
Rectification Sampling. Each instance  $\langle \tilde{\mathbf{x}}_t^u, \tilde{v}_t^u \rangle$  is assigned two rectification pools  $\mathcal{K}_t^u$  and  $\tilde{\mathcal{K}}_t^u$  to rectify the target and input, respectively. At first,  $\mathcal{K}_t^u$  is initialized by the  $K$  succeeding items of  $\tilde{v}_t^u$ , i.e.,  $[\tilde{v}_{t+1}^u, \dots, \tilde{v}_{t+K}^u]$ , which are potentially better substitutions for  $\tilde{v}_t^u$ . Similarly,  $\tilde{\mathcal{K}}_t^u$  is initialized with the  $K$  preceding items of  $\tilde{v}_{t-1}^u$ . Next, in each epoch, the items in  $\mathcal{K}_t^u$ , together with  $K$  additional items that are randomly sampled from  $\mathcal{V}$ , are ranked in descending order w.r.t. their weighted average prediction scores over different epochs (i.e.,  $\sum_{h=1}^{H} [w_h f_{v_i}^h (\tilde{\mathbf{x}}_t^u)]$ ). Then the top-1 item, denoted as  $\hat{v}_m$ , is adopted as the approximation of  $v_m$  in DRUT, while the top- $K$  items are retained to update  $\mathcal{K}_t^u$  for the next epoch. The rationale behind approximating  $v_m$  with  $\hat{v}_m$  is supported by Theorem 4, which indicates that the relative rank of an item over a list of randomly sampled items can approximate this item's relative rank over the full item set. That is, the top-1 item  $\hat{v}_m$  tends to be ranked highly over
Table 1: Statistics of the datasets.
Dataset	# Users	# Items	# Interactions	Avg. Length	Sparsity
ML-1M (ML)	6,040	3,417	999,611	165.5	95.16%
Beauty (Be)	22,362	12,102	198,502	8.9	99.93%
Yelp (Ye)	22,844	16,552	236,999	10.4	99.94%
QK-Vedio (QK)	30,704	41,534	2,268,935	73.9	99.82%
$\mathcal{V}$ . The updating rules of  $\tilde{\mathcal{K}}_t^u$  for DDUI is defined similarly as  $\mathcal{K}_t^u$ . As a result, the per-epoch time complexity of BirDRec is reduced from  $\mathcal{O}\big(N\cdot (|\mathcal{V}| + L)\big)$  to  $\mathcal{O}\big(N\cdot (K + L)\big)$ .
Theorem 4. Let  $\mathcal{R}$  be a list of  $K$  items randomly sampled from  $\nu$  with replacement,  $\zeta \in (0,1)$ ,  $r_H(\mathbf{x}_t^u,v_i) = \sum_{v_j\in \mathcal{V}}\mathbb{I}\big[\sum_{h = 1}^H [w_hf_{v_j}^h (\tilde{\mathbf{x}}_t^u)] > \sum_{h = 1}^H [w_hf_{v_i}^h (\tilde{\mathbf{x}}_t^u)]\big]$  be the rank of item  $v_i$  over the entire item set at the  $H$ -th epoch, and  $\hat{r}_H(\mathbf{x}_t^u,v_i) = \sum_{v_j\in \mathcal{R}}\mathbb{I}\big[\sum_{h = 1}^H [w_hf_{v_j}^h (\tilde{\mathbf{x}}_t^u)] > \sum_{h = 1}^H [w_hf_{v_i}^h (\tilde{\mathbf{x}}_t^u)]\big]$  be the rank of  $v_i$  over  $\mathcal{R}$  at the  $H$ -th epoch. We have:  $\mathbb{P}\big[|\frac{\hat{r}_H(\mathbf{x}_t^u,v_i)}{K} -\frac{r_H(\mathbf{x}_t^u,v_i)}{|\mathcal{V}|}\big|\geq \zeta \big]\leq \exp (-2K\zeta^2)$ .
4.2 Reducing Space Complexity via Self-ensemble Mechanism
The huge space cost of BirDRec is caused by the storage of prediction scores between each instance and all items in every epoch, for the sake of calculating the weighted average prediction scores.
Specifically, at the  $H$ -th epoch, the space complexity of preserving all prediction scores is  $\mathcal{O}(N \cdot |\mathcal{V}| \cdot H)$ . To save space, we thus approximate the weighted average scores with the self-ensemble mechanism [14], thereby avoiding storing prediction scores of different epochs. Formally, let  $f^H$  be an SRS parameterized by  $\theta_H$ , and  $\overline{f}^H$  be a self-ensembld SRS parameterized by  $\overline{\theta}_H = \sum_{h=1}^{H} w_h \theta_h$ . It has proven that [14] the difference between the weighted average prediction scores  $(\sum_{h=1}^{H} w_h f_{v_i}^h(\mathbf{x}_t^u))$  and the prediction of the self-ensembld SRS  $(\overline{f}_{v_i}^H(\mathbf{x}_t^u))$  is of the second order of smallness, if and only if  $w_h = \rho^{H - h}(1 - \rho)^{1 - \delta(h - 1)}$ , where  $\rho \in (0,1)$  denotes the exponential decay rate for ensembling; and  $\delta(\cdot)$  is the unit impulse function, i.e.,  $\delta(0) = 1$ , otherwise 0. With such self-ensemblde, there is no need to store SRSs of each epoch, as  $\overline{\theta}_H$  can be efficiently derived from  $\overline{\theta}_{H - 1}$  and  $\theta_H$  with the exponential moving average as follows:
$$
\bar {\boldsymbol {\theta}} _ {H} = \sum_ {h = 1} ^ {H} \rho^ {H - h} (1 - \rho) ^ {1 - \delta (h - 1)} \boldsymbol {\theta} _ {h} = \rho \bar {\boldsymbol {\theta}} _ {H - 1} + (1 - \rho) \boldsymbol {\theta} _ {H}. \tag {9}
$$
By doing so, the burden of retaining prediction scores of different epochs is reduced to maintaining an extra self-enssembled SRS. As the parameters of an SRS mainly consist of the user and item embeddings, the per-epoch space complexity of BirDRec is reduced from  $\mathcal{O}\big((|\mathcal{U}| + |\mathcal{V}|)\cdot d + (L + |\mathcal{V}|\cdot H)\cdot N\big)$  to  $\mathcal{O}\big((|\mathcal{U}| + |\mathcal{V}|)\cdot d + (L + K)\cdot N\big)$ , where  $d$  represents the embedding size. Furthermore, to reduce the number of parameters and mitigate overfitting, the (self-enssembled) forward and backward SRSs in BirDRec share the same user and item embeddings.
Datasets. We adopt four real-world datasets with varying domains, sizes, sparsity, and average sequence lengths shown in Table 1. Specifically, ML-1M (ML) [45] is a popular movie recommendation benchmark. Beauty (Be) [46] is the product review dataset collected from Amazon.com. Yelp (Ye) [10] is a business recommendation dataset released by Yelp.com. QK-Video (QK) [47] is a video recommendation dataset crawled from Tencent.com. Following [3, 8, 9], we preprocess all datasets by removing users and items whose interactions are less than 5.
Baselines. To verify the generality of BirDRec, we implement it with vanilla SRSs based on representative backbones. In particular, FPMC [15] is based on Markov Chain. GRU4Rec[17], Caser [20], and MAGNN [27] are built on RNN, CNN, and GNN, respectively. SASRec [28] and BERT4Rec [29] are based on Transformer. Meanwhile, to validate its effectiveness and efficiency, we compare BirDRec with state-of-the-art robust SRSs including BERD [8], FMLP-Rec [9], and STEAM [10], which aim to tackle unreliable targets, unreliable input, and both, respectively.
Evaluation Protocol. Following [9, 48, 49], three widely-used metrics are adopted to evaluate the ranking quality, namely, HR, NDCG, and MRR. For all these metrics, higher metric values suggest
(a) Time per Epoch
(b) Convergence Epoch
(c) Memory Usage
Figure 3: The comparison regarding time and storage costs.
(d) GPU Memory Usage
better recommendation accuracy. For each user, we preserve the last two interactions for validation and testing, while the rest are used for training. The training of each model is carried out five times to report the average results. During testing, we follow [3, 9, 27] and evaluate the ranking results over the whole item set for fair comparison [50].
Implementation Details. For all methods, Xavier initializer [51] and Adam optimizer [52] are adopted; and the best hyper-parameter settings are empirically found based on the performance on the validation set. For BirDRec, it is implemented by PyTorch with batch_size = 1024,  $d = 64$ , learning_rate = 0.01 for Yelp and 0.001 for other datasets,  $L = 5$ ,  $\beta = 0.1$ ,  $\beta' = 0.1$ ,  $K = 10$ , and  $\rho = 0.9$ . To ensure accurate rectification with reduced  $\epsilon$ , BirDRec is trained without rectification in the first 10 epochs. All the experiments are conducted on an NVIDIA Quadro RTX 8000 GPU<sup>3</sup>.
5.2 Experimental Results and Analysis
Overall Comparison. Table 2 presents the performance of vanilla SRSs built on various representative backbones. Each SRS is trained under two different settings: the original Plain setting and our
$^3$ Our source code and experiment details (e.g., parameter settings for baselines) are in the Appendix.
Table 3: Performance comparison with existing robust SRSs, where the best performance is boldfaced and the runner up is marked by \*'. Improv. means the relative improvement of BirDRec over the runner up. The significance of the improvement is determined by a paired t-test with  $p\leq 0.001$
Datasets	ML-1M	Beauty
Metrics	HR@5	HR@10	NDCG@5	NDCG@10	MRR	HR@5	HR@10	NDCG@5	NDCG@10	MRR
BERD	0.1922*	0.2814*	0.1267*	0.1554*	0.1335*	0.0507*	0.0745	0.0332	0.0406	0.0364
FMLP-Rec	0.1789	0.2685	0.1207	0.1492	0.1299	0.0501	0.0743	0.0384*	0.0448*	0.0404*
STEAM	0.1198	0.1950	0.0765	0.1006	0.0874	0.0495	0.0765*	0.0324	0.0414	0.0371
BirDRec	0.2352	0.3259	0.1631	0.1915	0.1647	0.0653	0.0903	0.0459	0.0537	0.0482
Improv.	22.37%	15.81%	28.73%	23.23%	23.37%	30.34%	18.04%	19.53%	19.87%	19.31%
Datasets	Yelp	QK-Vedio
BERD	0.0423	0.0636	0.0304	0.0360	0.0351	0.0548	0.0958	0.0352	0.0496	0.0461
FMLP-Rec	0.0499	0.0702	0.0374	0.0433	0.0422	0.0604*	0.0991	0.0391*	0.0514*	0.0480*
STEAM	0.0556*	0.0822*	0.0387*	0.0473*	0.0448*	0.0597	0.1021*	0.0371	0.0507	0.0465
BirDRec	0.0771	0.0965	0.0626	0.0687	0.0663	0.0815	0.1306	0.0523	0.0682	0.0616
Improv.	38.67%	17.40%	61.76%	45.24%	47.99%	34.93%	27.91%	33.76%	32.68%	28.33%
Figure 4: Ablation study on different variants of BirDRec.
BirDRec framework. We can note that all the baselines are boosted significantly on all datasets with the aid of BirDRec, which demonstrates the generality of BirDRec.
Table 3 compares BirDRec with state-of-the-art robust SRSs. Considering BERD and STEAM are both built upon Transformer, we also adopt a transformer-based SRS, SASRec, as the default backbone of BirDRec in the following experiments to ensure fair comparisons. The results show that BirDRec dramatically outperforms existing robust SRSs, confirming the effectiveness of our error-bounded rectification strategies in comparison to existing methods that lack theoretical guarantees.
Fig. 3 compares the time and storage cost of BirDRec and STEAM, both of which aim to handle unreliable input and targets simultaneously. In particular, Figs. 3(a) and (b) show that BirDRec executes 4.28 times faster than STEAM in each epoch and converges with fewer epochs. Meanwhile, Figs. 3(c) and (d) show that BirDRec incurs only half the storage cost of STEAM on most datasets. These results highlight the efficiency of our sampling strategy and the self-ensemble mechanism. The storage cost of BirDRec is marginally higher than STEAM's on ML-1M. This is because the advantage of our rectification sampling strategy is less obvious on datasets with small item sets (see Table 1), indicating the superior scalability of our BirDRec on large-scale item sets.
Ablation Study. To check the efficacy of essential strategies of BirDRec (DRUT, DDUI, and the self-ensemble), we add these strategies incrementally to a plain representative SRS – SASRec. Note that without self-ensemble, DRUT and DDUI only consider the prediction scores at the latest epoch for data rectification. The results are presented in Fig. 4 and similar trends can be noted with the rest SRSs on the other metrics. First, adding either DRUT or DDUI to SASRec brings dramatic improvements, implying the effectiveness of DRUT and DDUI in rectifying unreliable targets and input, respectively. Second, using DRUT and DDUI together is better than leveraging each of them solely, indicating the necessity of rectifying both unreliable targets and input for more robust SRSs. Moreover, adding the self-ensemble mechanism can further boost the accuracy, which confirms the efficacy of considering prediction scores of different epochs for data rectification. Overall, the performance gain of separately adding DRUT, DDUI, and self-ensemble is  $45.7\%$ ,  $43.3\%$ , and  $3.1\%$ , respectively. That is, DRUT and DDUI contribute more than the self-ensemble to our BirDRec.
Hyper-parameter Analysis. We analyze the impact of key hyper-parameters, i.e., the rectification thresholds  $\beta$ ,  $\beta'$ , the size  $K$  of the rectification pool, and the exponential decay rate  $\rho$ . The results are presented in Fig. 5 with several major findings (similar trends can be noted with the other metrics on the rest datasets). (i) The best choice for  $\beta$  and  $\beta'$  is 0.1, showing that small thresholds tend to maintain unreliable instances while large ones may mistakenly rectify reliable data. (ii)  $K \geq 10$  is sufficient to gain better accuracy. (iii)  $\rho \geq 0.5$  yields better performance, that is, it is beneficial to consider predictions in earlier epochs. (iv) Even if the hyper-parameters of BirDRec are not optimally set, BirDRec still dramatically outperforms the best baseline (BERD) on ML-1M.
Figure 5: Effect of key hyper-parameters  $\beta$ ,  $\beta'$ ,  $K$ , and  $\rho$  of BirDRec on ML-1M dataset.
Case Study. Fig. 6 presents a real instance that is rectified by BirDRec on ML-1M. Specifically, in the DRUT module, the target movie 'Schindler's List' consistently gets lower scores than the movie 'Terminator 2' in the rectification pool, and is then replaced by 'Terminator 2'. This is reasonable, given that 'Terminator 2' aligns more closely with the input Sci-fi movies than 'Schindler's List'. Subsequently, in the DDUI module, the input movie 'American Beauty' is consistently lower-scored than the movie 'Star Wars IV' in the rectification pool and is thus deleted. This decision is justifiable, considering that the rectified target, 'Terminator 2', generally lacks relevance to the input 'American Beauty' across various aspects, including genres, actors, directors, tags, etc.
Figure 6: The overall architecture of the proposed BirDRec framework.
Limitations of BirDRec. Although BirDRec exhibits its superiority through our extensive experiments, its limitations are two-fold. Firstly, the improvement of BirDRec on storage cost is limited for smaller item sets. This is primarily because the rectification sampling strategy with self-ensemble has less apparent advantages on datasets with smaller item sets. To be specific, the storage cost reduction for calculating weighted average prediction scores is from  $O(|V| * H)$  to  $O(K)$  for each instance. Thus if  $|V|$  is small, the benefit of this reduction will be less obvious. Secondly, although BirDRec is significantly faster than the latest robust SRS (STEAM), it is worth noting that BirDRec is 1.6 times on average slower than its backbone model (as depicted in Fig. 3) in each training epoch. This increased training time could be a practical concern in systems with extremely large-scale datasets and real-time recommendation demands.
This work, for the first time, provides theoretically guaranteed data rectification strategies to tackle both unreliable input and targets for more robust SRSs. The proposed strategies are further integrated into a model-agnostic bidirectional data rectification framework, BirDRec, that can be flexibly implemented with most existing SRSs, for robust training against unreliable data. Additionally, we devise a rectification sampling strategy to reduce the computational cost of BirDRec; meanwhile, a self-ensemble mechanism is adopted to reduce the space complexity. Extensive experiments verify the generality, effectiveness, and efficiency of the proposed BirDRec.
The work is partially supported by the National Key Research and Development Program of China (2020YFB1707900), the National Natural Science Foundation of China (Nos. U22A2025, 62072088, 62232007), Liaoning Provincial Science and Technology Plan Project - Key R&D Department of Science and Technology (No. 2023JH2/101300182), and ARC Discovery Projects DP200101441 and DP230100676.