Theoretically Guaranteed Bidirectional Data Rectification for Robust Sequential Recommendation
Authors: yatong sun
Bin Wang
Zhu Sun
Xiaochun Yang
Yan Wang
Venue: NeurIPS 2023 poster (2023)
Keywords: recommender systems
sequential recommendation
TL;DR: We propose a theoretically guaranteed model-agnostic framework to rectify unreliable targets and input for more robust sequential recommendation, with strategies to reduce time and space complexity.

Abstract: Sequential recommender systems (SRSs) are typically trained to predict the next item as the target given its preceding (and succeeding) items as the input. Such a paradigm assumes that every input-target pair is reliable for training. However, users can be induced to click on items that are inconsistent with their true preferences, resulting in unreliable instances, i.e., mismatched input-target pairs. Current studies on mitigating this issue suffer from two limitations: (i) they discriminate instance reliability according to models trained with unreliable data, yet without theoretical guarantees that such a seemingly contradictory solution can be effective; and (ii) most methods can only tackle either unreliable input or targets but fail to handle both simultaneously. To fill the gap, we theoretically unveil the relationship between SRS predictions and instance reliability, whereby two error-bounded strategies are proposed to rectify unreliable targets and input, respectively. On this basis, we devise a model-agnostic Bidirectional Data Rectification (BirDRec) framework, which can be flexibly implemented with most existing SRSs for robust training against unreliable data. Additionally, a rectification sampling strategy is devised and a self-ensemble mechanism is adopted to reduce the (time and space) complexity of BirDRec. Extensive experiments on four real-world datasets verify the generality, effectiveness, and efficiency of our proposed BirDRec.

Theoretically Guaranteed Bidirectional Data Rectification for Robust Sequential Recommendation
Yatong Sun 1.2, Xiaochun Yang',, Zhu Sun Bin Wang', Yan Wang"
School of Computer Science and Engineering, Northeastern University, China "School of Computing, Macquarie University, Australia 3 Cennter for Frontier Al Research, Institute High Performance Computing, A*STAR, Singapore yatongåstumail. neu. edu. cn, yangxc@mail memmaiiln..... edu. sunzhuntu@gmail.. binwang@mail neu. edu. en, yan. wangâmq edu.
Abstract
Sequential recommender systems (SRSs) are typically trained to predict the next item as the targer given its preceding (and succeeding) items as the input. Such a paradigm assumes that every input-target pair is reliable for training. However, users can induced to click on items that are inconsistent with their true prefer- ences, resulting in unreliable instances, i.e., mismatched input-target pairs. Current studies on mitigating this issue suffer from two limitations: (i) they discriminate instance reliability according to models trained with unreliable data, yet without theoretical guarantees that such a seemingly contradictory solution can be effective; and (ii) most methods can only tackle either unreliable input or targets but fail to handle both simultaneously. To fill the gap, we theoretically unveil the relation- ship between SRS predictions and instance reliability, whereby two brouded strategics are proposed to rectify unreliable targets and input, respectively. On this basis, we devise model-agnostic Bidirectional Data Rectification (BirDRec) framework, which can be flexibly implemented with most existing SRSs for robust training against unreliable data. Additionally, rectification sampling strategy is devised and a self-ensemble mechanism adopted to reduce the (time and space) complexity of RerDRec. Extensive experiments four real-world datasets verify the generality, effectiveness, and efficiency of our proposed BirDRec.
Introduction
Recently, the study on sequential recommender systems (SRSs) I attention as users' preferences are inherently dynamic and evolving in real-world scenarios. The goal of SRSs learning to predict the next item a user interacts with given preceding (and succeeding) items. Therefore, training instance for SRSs is typically composed of an input item sequence and its next item as the target. However, distractions in daily lives (e.g. recommendations from friends, account sharing, and accidental clicks) can induce users to click on items that are inconsistent with their true preferences, resulting in unreliable training instances with mismatched input-target pairs. The mismatch can be categorized into Complete Mismatch and Partial Mismatch when the item caused by distractions acts as an unreliable target and unreliable input an instance, respectively. To illustrate, the romantic film 'La La Land', in the first instance of Figure [ serves as an unreliable target, which is recommended by friends and completely mismatched with the previous superhero movies. By contrast, in the second instance, 'La La Land' acts as unreliable input item which renders the input sequence partially mismatched with the target superhero film. Both
has garnered much
denotes the corresponding authors
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Instance Complete Mismatch
Instance 2 Partial Mismatch
IXTAIA
Friends'
Recommendation
External
Input
Target
Distractions
Input
Target
Figure 1: Examples of two types of mismatch caused by external distractions.
types of mismatch would cause unreliable training instances that mislead SRSs to learn sequential relationships between irrelevant items and eventually undermine the recommendation accuracy.
Although there SRSs, they suffer from two core limitations. (i) They discriminate instance reliability based on the intermediate final output of model (either the SRS itself [6 g [ or additional corrector (10) that is trained with unreliable data. However, there no theoretical guarantee that such seemingly contradictory solutions be trustworthy for detecting and correcting unreliable instances. (ii) Most prior studies only focus on tackling either unreliable input 7
number of studies aiming to combat such unreliable data for more robust
[1
12
[3] or targets [8) but fail to
handle both simultaneously. Only one recently proposed method 10 attempts address this issue, but it relies on corrector trained with unreliable data, yet without any theoretical guarantees.
As such, for the first time, theoretically unveil the relationship between the SRS predictions and instance reliability, proving that a target with consistently low prediction scores is unlikely to be reliable, assuming that the randomness of user behavior is limited. It then inspires us to devise two arrorbbounded rectification strategies to (1) detect consistently low-scored targets (i.e., unreliable targets) and replace them with steadily high- high-scored items and (2) detect and delete consistently low-scored items within the input (i.e., unreliable input), where the score is estimated by a backward SRS. Note that the unreliable input items, as the interruptions in the input, are not replaced but directly removed to bridge the preceding and succeeding items. Based on these strategies, we propose model-agnostic Bidirectional Data Rectification (BirDRec) framework which contains two SRSs in opposite directions for tackling both unreliable targets and input. In addition, to reduce the time complexity, a rectification sampling strategy is devised efficiently obtain consistently high-scored items; to reduce space complexity, a self-ensemble mechanism [14] is adopted to approximate the weighted average prediction scores across different training epochs.
Contributions. (1) We are the first to provide theoretically guaranteed data rectification strategies based on SRS predictions to tackle both unreliable input and targets for more robust SRSs. (2) We devise a model-agnostic bidirectional data rectification framework that can be flexibly implemented with most existing SRSs for robust training against unreliable data. (3) We devise a rectification sampling strategy and adopt a self-ensemble mechanism to ensure better scalability of BirDRec. (4) Extensive experiments with SRSs based on representative backbones and datasets across various domains validate the generality, effectiveness, and efficiency of BirDRec.
Related Works
adopt Markov Chains to encode users' interaction sequences, assuming users' latest interactions affect the future behavior linearly and independently. Later, powerful deep learning backbones such as recurrent neural networks (RNNs) [7
Early SRSs [15] 16]
19 convolution neural networks 18
(CNNs) 200 21 22), graph neural networks (GNNs) 23
24
25
26
27), and Transformers [28 [2]
are employed to extract complex non-linear patterns within users' sequences [30]. They posit each training instance is definitely matched input-target pair and thus cannot handle unreliable data.
To resist unreliable data, existing robust SRSs can be categorized into three types. The first type focuses on handling the complete mismatch by identifying and eliminating instances with unreliable targets. For example, BERD [ empirically finds that instances with high training loss and low uncertainty tend to have unreliable targets. This idea is relevant to studies on clean label selection BI
32
35 and label correction [36] 37 38 The second type concentrates on addressing partial mismatch by reducing the importance of unreliable input when formulating users' dynamic preference representations. Accordingly, various advanced mechanisms are integrated into SRSs, such
memory networks (7). gating networks [39 40 41, autoencoders [[2), reinforcement learning (6),

---

To the best of our knowledge, there is only one recently proposed method STEAM [T0] falling into the third rype which attempts tackle both unreliable targets and input with an additional corrector producing reliable data. Nonetheless, existing robust SRSs all rely on models (either SRSs or additional correctors) trained with unreliable data, yet without theoretical proof that such seemingly contradictory solutions can be effective.
uncertainty modelling [1] 42 and Fast Fourier Transform 19.
3 Theoretical Guarantees for Rectifying Unreliable Data
This section presents theoretical guarantees for rectifying unreliable data via SRS predictions for more robust SRSs. In particular, Section 3.1 introduuces important preliminaries. Subsequently. Section (32 unveils the relationship between the prediction score of SRSs and the reliability of a target, inspiring us to propose an error-bounded strategy for handling unreliable targets; and Section 3.3 provides the error-bounded strategy for dealing with unreliable input.
3.1 Preliminaries
Problem Statement of SRSs. Let and V be the sets of users and items, respectively. Each user U chronologically interacts with sequence of items s" [6f By .... Bl where öy e V the f-th item user u interacts with and |s*| is the length of sequence s" The goal of SRSs is to predict the target item ot given the input x {u, [-t- -...., 2 1 where L is the length of xy. Thus, the training instance of SRSs can be represented as input-target pair (5.". (T). Note that we use '-" to denote the observed data that may be unreliable due to external distractions.
Core Assumptions. Ideally, each user-item interaction should be drawn from users' true preference distribution 7 without any distractions, where 1/, (x.) [P(u) e xx))). We define pi to be the true item for recommendation, i.e, the top-1 item according to v).
Ev,r Immen (xx) 2 ne, (xx)] [V%
(I)
where I[[]] is an indicator function that equals 1 if the condition is true; otherwise 0. Mcanwhile, define P2 to be the middle-ranked item (ranked [ -th) according to v, namely,
Σ "[no 2 n. (xx)) = LIV1/2J.
(2)
In general, SRSs are built upon the hypothesis that users usually select items with a tendency rather than randomly. other words, the randomness of users' true preferences is restricted, i.e., the probability gap between the top-1 and middle-ranked items regarding 1) is unlikely to be small. This assumption can be formally defined as follows.
Assumption 1.
The users' true preference distribution 17 fulffills the relaxed Multiclass Tsybakov Condition with constants C 0, X > 0, and ag (0, 1). such that for all a E (0, on).
PP [Tpe (x+) - 1pp (x S a] S Co.
(3)
The feasibility of Assumption L reliee on small C' and large X, which are satisfied on public (observed)
datasets based on our empirical analysis in the Appendix with C' (0.55, 0.70) and À E (1.37, 4.01).
Connecting 7 with SRS Predictions. Obviously, 1) the ideal corrector to rectify unreliable data, however, due to its unavailability, many existing methods [8] 10 leverage SRS predictions as the substitution with no theoretical guarantees. This urges us to explore the connection between v) and SRS predictions. To achieve this goal, we first investigate the relationship between and users' observed preference distribution ij, since SRSs are trained with the observed data that may be distorted by external distractions. Formally, 7o, (xx) PP ny [x)) where ūj is the observed target that may be unreliable. We then define a transition probability T. (x²) = IP = 0/ u//2" = Uy, X." as the chance that true target ui is flipped from item Uy to item 2; owing external distractions. Thus, for any pair (V;, U/)) E V, there is a linear relationship between 77 and îj:
My (xx))
Σ
P(ū. v//e"" uy PP(" ,, Σ Eejev Teya, (x7 Me (xx)
(4)
SyEV
To bridge n and SRS predictions via 7j. we then study the relationship between ñ and SRS predictions. Let f be an SRS at the fi-th training epoch, and the prediction of f be e-close to 7).
max 7r, (x.)) - 1t) (x:)),
(5)
x
3

---

where f1l (x:) is the predicted probability (score) of the target being e; given the input X at the h-th training epoch. Eqs. ( indicate that there is indeed connection between v and SRS predictions, laying the foundation for the proposed strategies to rectify unreliable data as what follows.
3.2 Theorems for Rectifying Unreliable Targets
We now explore how to properly use SRS predictions for rectifying unreliable targets. Prior works empirically find that unreliable targets tend to possess consistently low prediction scores at different epochs [8] [44. Yet, there is no guarantee that such prediction scores given by an SRS trained with unreliable data can be trustworthy for detecting unreliable targes. This prompts to theoretically unveil the relationship between target reliability and SRS predictions at different epochs. Specifically, Theorem proves that reliable target is unlikely keep low prediction scores during training.
Given Assumption 1, let {wx isss S H, wA E, wa 1] be the weight for averaging prediction scores of different epochs. V ( assume S aureyer (x.). Let y
Theorem 1.
Toper (x Map + Duytay Teyep (x mm (XT). We have: P p1 p... Σ Proof.
[wn fy (xa) s S (((()))
P1
wk J (87) $
SP
P1
10h Mep -c 2
87 Top
2 Mog (2)))
Σ
Σ
Toyop Bey (87) - sT
(6)
Am1
SP
P1 80 (x," Ybu (x,) Top (ap) (x7)
(x (8)) ( s +
? + Σ Enymey Tn (8 (ay (x1))
=P
P1 07' Meg (x.) S Nay (x.)
Togio (se))
By replacing y with Thpay M Mpp (x()
+ E Eny May Teg (x)) Meg (x)), we obtain:
oy. 1 [ S s P Moz (8)) S Up (xx)) S mog (81)
77)
Toyoy (8p)
Recall that é S op (x"), which indicates oz (8)) S au- Hence, the relaxed Multiclass
Tsybakov Condition holds and the probability is bounded by C( Topey (x3)) namely, C(O()))
Note that a small e relies on large observed data and a powerful f, so that (i) the observed data can accurately approximate ij, and (ii) can closely fit the observed data. Both requirements can be satisfied thanks to the large-scale datasets and deep learning advancements in recommendation.
Theorem Tindicates that the probability of reliable target (pi = i/') keeping low prediction scores across different epochs E [w,, J (xx) <)) is bounded to be low, i.e., no more than (()()))* > This inspires us to rectify unreliable targets by replacing consistently low-scored targets with steadily high-scored items the following strategy.
DRUT: Detecting and Replacing Unreliable Targets. Given an SRS f that is e-close 7j an instance (xy, i the consistently high-scored item Um = arg max., o Dim [wn]h (x²)) and H
3 E (0, 1]. we stipulate that if Ek=n fu (xT) Dh=1 [wrtth (xq) B, i.e., the target oy is consistently lower-scored than Um to some extent, then o should be replaced by Um in the H -th epoch. We denote the output instance of DRUT as , o7).
Different from existing methods, DRUT is theoretically error-bounded. Specifically, the error of DRUT, denoted as EDRUT comes from three cases: (Case-1) the true target P1 is iy but is replaced
The detailed setting of the weights is related to the self-ensemble mechanism introduced in Section 4.2

---

by Um- (Case-2) the true target p1 is Um but Dy is kept. (Case-3) the true target is neither ج. nor Um- Conrespondingly, EDRUT can be formulated as below:
Em [1o f0p (x")]
Eksn
fur (x:1)
EDRUT=F p pi vm Σ [ fh (x:))
<,
Pi
E ( Jrm (xr))
2. 28+3 Piist piztum
Case-1
Case-2
Case-3
Subsequently, Lemma I first provides the thresholds 31 and 82 that can respectively guarantee bounded probabilities for Case-l and Case-2 EDRUT. Then in Theorem 2 the chosen 3 deviates from B1 and 32, DRUT is still error-bounded
we prove that even
Lemma 1. Given Assumption 1 and the set of weights {wN 1Is h
H2O f
Wh. S 1, E. uh 13, V (xc (8)) assume e S min [auoaap or (XP)), agtee em (x:)). Let B1
Topppp (8F) (81) Enjoor Fuy 20 Ti, (87) and 82 = =[ Tummm om (81 yau (8P) + Engetm Tayem (8))),, (81)
Eh= (wn the (87))
] We
I fw.. (87))
have: B Bi guarantees the probability of Case-l in EDRUT bounded by C(O(e)) and B 2 92
guarantees the probability of Case-2 in EDRUT is bounded by C(O())).
Theorem 2 (The Upper Bound of EDRUT), Given Assumption and the set of weights {wn 15 h I H2O & wh I 1,
1 H wh = 1}, & (8), let 31 (T (Rt (pp()))))))))))))))))))))))))) (87)+ + + May Teyep my (81)
Em (1t Ib., (87))
Ehsr (wm (8.))
8/ Frlm "mn 1ip (82) )+ Lug som from Fug tm (2)) )m,, (RT))
E1 = |3 S11, and & |a - 0z|
Assume
&a < 82, 6 S min (00 Toy of (xF) - ET,
oo Han (# (12g E2 1 [bri (xt) - up ( [x+]]-
Pa E2)
Enyten str (Tugum (xF) ( Term (xF) Ml. (xP) We have: EDRUT S c(O(e+ +((())))))((
3.3 Theorems for Rectifying Unreliable Input
Unreliable items within the input act as mosaics to bewilder SRSs when learning users' true prefer- ences, thus impeding SRSs from predicting the true target. Recall that Theorem [sseoess[[suggetts consistently low-scored targets are unlikely to be reliable. This inspires us to exploit the prediction scores given by a backward SRS for rectifying unreliable input, that is, deleting consistently low-scored (predicted by a backward SRS) items within the input. To avoid deleting all input items with an unreliable target, we rectify the input of instances that are already processed by DRUT. Formally, given (x); 6/),
we define a backward SRS j that aims to predict every input item in x based on xy {u, 6
Hence, the backward instance for predicting o (1 E [.) can be formulated as (x), ( With Assumption being held, and r/. if', ph. P2z T', e' respectively being the counterparts of n, n, Pr, P2. T, c, for backward sequences, we propose the following strategy.
DDUI: Detecting and Deleting Unreliable Input.
Given f that is c'-close to n/' (X) ) the consistently high-scored item Um = arg maXe of Σ [wn (R+) and 3r E (0, 1]. we Th F.,. (xx) < 3", i.e., نيا is consistently
stipulate that if the ratio Σn [wa.7h
7h By_ (xx) Eim [wh.
lower-scored than F. to some extent, then item oi_; should be deleted from x in the H-th epoch. After rectifying the L items in X2. we denote the output instance of DDUI as
DDUI differs from DRUT slightly, i.e., we remove unreliable input items to bridge the succeeding and preceding items, instead of replacing them. Since unreliable input items caused by distractions are essentially interruptions in the input sequence, replacing them may introduce new interruptions.
Superior to prior works, DDUI is theoretically guaranteed to work well with bounded error, which comes from two cases: (1) باا is a reliable input item (p' = ) ,) but is deleted; and (2) 1y an unreliable input item (p) A o)-1) but is kept. Hence, the error of DDUI, denoted EDDUI. can be correspondingly formulated as the following Eq. [. and its bound is analyzed in Theorem
[ur.7 2i (SE") E"r Ee=n
[ur 7h (EE" Σ
EDDUL =P
p/
38 + pl. +
2 B
(8)
Iws i... (8.))
'Th proof' of Lemma I and following theorems can be found in the Appendix.
5

---

Update Parameters via Exponential Moving Average
Self-nnsemilee Forward SRS
Recify Target
Treining
DRUT
1
Forward SRS
Update
Top-1
xj
Rectification Pools
1
x
Derive Beokward Instances (2-33
Observed Instercee
Upcaae
Top
Self-nssembled Backward SRS
DDUI
Recifyy Input
Training
Backward SRS
Update Parameters via Exponential ng Average
Figure 2: The overall architecture of the proposed BirDRec framework.
Theorem 3 (The Upper Bound of EDDUI). Given Assumption 1 and the set of weights (wo 15
TTy "1_1 (8)) (8. (81)+Eu Toyey 2- Vay (87)
h H, wh S E 1 wh 1], V (xc. o2.)) let 8i
2
1on P (87)
I Iwan 2 , (87)
8 Pmmmm (R (RT) Znzzbm T (87)% (8F)
= 18' -Bi1 E2 = 8¹ - 5z) Assume & < 822
2 [ (xt) - 7 (x)) + 1/2 (8 Enjtrl (rey (8t)) (x [- We have: EDDUT & ((0(e++(1)) (((e))))()(((((((()(e +C((0(++(5) +0(00"+6))++((0()))) +((()))
$ min YouTHR oi_ (Kt))
E
ao Bma (87 1010000 +2)
4 The Proposed BirDRec Framework
By integrating DRUT and DDUI into existing SRSs, we introduce BirDRec, a model-agnostic bidirectional data rectification framework, which can rectify both unreliable targets and input with theoretical guarantees for more robust SRSs. Yet, the complexity of BirDRec is prohibitively high due the calculation and storage of prediction scores for each instance across different epochs in DRUT and DDUI. To ease this issue, we devise rectification sampling strategy that avoids prediction on the full item set replace unreliable targets or delete unreliable input, thereby reducing the time complexity of BirDRec. Meanwhile, we adopt the self-ensemble mechanism [14] to approximate the weighted average prediction scores of different epochs, thus avoiding preserving scores of all epochs and reducing the space complexity.
Framework Overview. Accordingly, the dnfciency-iiproved BirDRec is depicted in Fig. 2 Specifi- cally, BirDRec first leverages the self-ensembled forward SRS to rectify the target of an instance via DRUT, and then the input is rectified by the self-ensembled backward SRS via DDUI Thereafter, the rectified instance and its L backward instances are respectively used to train the forward and backward SRSs, which are finally employed to update the corresponding seff-ensembled SRSs.
4.1 Reducing Time Complexity via Rectification Sampling
The time complexity of BirDRec is primarily attributed to the search for consistently high-scored items (v,, and f,)) which requires the calculation of prediction scores between every instance and all items. Concretely, the per-epoch time complexity of BirDRec is O(N (N--N N([[] + L)), where N denotes the number of instances. To mitigate this substantial computational burden, we propose a rectification sampling strategy that circumvents the need to search the entire item set.
Rectification Sampling. Each instance (Se) 8i)) is assigned two rectification pools K.O' and K; to rectify the target and input, respectively. At first, KC is initialized by the K succeeding items of ey. i.e. [o..,.... [] which are potentially better substitutions for oy. Similarly, Æy is initialized with the K preceding items of 6 Next, in each epoch, the items in KCt, together with K additional items that are randomly sampled from V, are ranked in descending order w.r.t. their weighted average
prediction scores over different epochs (i.e., >H [WA. fh, (x; ). Then the top-1 item, denoted as im- adopted as the approximation of u. in DRUT, while the top- K items are retained to update KC for the next epoch. The rationale behind approximating Um with ém is supported by Theorem 4 which indicates that the relative rank of an item over a list of randomly sampled items can approximate this item's 's relative rank over the full item set. That is, the top-1 item 6n tends to be ranked highly over

---

Table 1: Statistics of the datasets.
Dataset
Users
Items
Interactions
Avg. Length
Sparsity
ML-IM (ML)
6,040
3,417
999.611
165.5
95.16%
Beauty (Be)
22,362
12,102
198.502
8.9
99.93%
Yclp (Yo)
22,844
16.552
236.999
10.4
99.94%
Vedio (QK)
30,704
41,534
2,268,935
73.9
99.82%
V. The updating rules of Ky for DDUI is defined similarly as K.. As result, the per-epoch time complexity of BirDRec is reduced from (N-((1)++)) L)) to ONN-(E( + L)).
Theorem 4. Let R. be list of K items randomll sampled from V with replacement, E (0, 1), rulxi, vi) Enjev [[>=]lww (RP) > E Eh=1lwn.d, (R7)]] be the rank of item vs over the entire item set at the H-th epoch, and (x(x), 04) Zejer 1 Iwnn (x+)) Ehellwne (R))] be the
rank of v. over R the H-th epoch. We have: P[] fo fo rw (e [2 & exp( 2K
K
IV]
4.2 Reducing Space Complexity via Self-ensemble Mechanism
The huge space cost of BirDRec is caused by the storage of prediction scores between each instance and all items in every epoch, for the sake of calculating the weighted average prediction scores.
Specifically, at the H-th epoch, the space complexity of preserving all prediction scores is O(N (VI.)). H). To save space, we thus approximate the weighted average scores with the self-ensemble mechanism [14), thereby avoiding storing prediction scores of different epochs. Formally, let f be an SRS parameterized by Hи, and f be a self-ensembled SRS parameterized by 0μ = .H
Mh=1 WA On. It has proven that [14] the difference between the weighted average prediction scores EM Sh, (x()) and the prediction of the self-ensembled SRS ( (xx") is of the second order of smallness, if and only if wh. p - p) -))-11111), where E (0, 1) denotes the exponential decay for ensembling; and 5(-) is the unit impulse function, i.e., 6(0)
otherwise With such self-ensemble, there no need to store SRSs of each epoch, as 0H can be efficiently derived from 0 and 0r with the exponential moving average follows:
0H
p)² 1-1--6 5(h-1) 13 0 = p0 1 (1- p)
By doing so, the burden of retaining prediction scores of different epochs is reduced to maintaining an extra self-ensembled SRS. As the parameters of an SRS mainly consist of the user and item embeddings, the per-epoch space complexity of BirDRec is reduced from ((((4) + [V] - (L+1V) H)- N) to (([ + [V]) d K) N), where d represents the embedding size. Furthermore, to reduce the number of parameters and mitigate overfitting, the (self-ensembled) forward and backward SRSs in BirDRec share the same user and item embeddings.
Experiments and Results
5.1 Experimental Settings
Datasets. We adopt four real-world datasets with varying domains, sizes, sparsity, and average se-
quence lengths shown in Table T Specifically, ML-1M (ML) [45] is popular movie recommendation benchmark. Beauty (Be) [6]] is the product review dataset collected from Amazon.com. Yelp
(Ye) 10 is a business recommendation dataset released by Yelp.com. QK-Video (QK) [47] is
video recommendation dataset crawled from Tencent.com. Following 3 we preprocess all datasets by removing users and items whose interactions are less than 5.
Baselines. To verify the generality BirDRec, we implement with vanilla SRSs based on repre-
sentative backbones. In particular, FPNC [5] is based on Markov Chain. GRU4Rec Caser (0)). and MAGNN 27 are built on RNN, CNN, and GNN, respectively. SASRec (28 and BERT4Rec 22 are
based on Transformer. Meanwhile, validate its effectiveness and efficiency, compare BirDRec with state-of oftte-of-theartd the-art robust SRSs including BERD [), FMLP-Rec (9. and STEAM [0], which aim to
tackle unreliable targets, unreliable input, and both, respectively.
48 three widely-used metrics are adopted to evaluate the ranking quality, namely, HR, NDCG, and MRR. For all these metrics, higher metric values suggest
Evaluation Protocol. Following
7

---

Table 2: Performance comparison with vanilla SRSs trained with the original 'Plain' setting and our BirDRec framework, Improv. means the relative improvement of BirDRec over the 'Plain' setting. The significance of the improvement is determined by paired t-test with S 0.001.
Dulascis
MI-11M
Beanty
Backbonie
Settings
HR
HR 10
NDDGes
NDCG 10
MRR
HR
HR
NIXG@5
NDOG@10
MRK
Piain
0.1317
0.2087
0.0821
0.1069
0.0929
0.0527
0.0259
0.0244
FPMC
BirDRec
0,1534
0J9972
0,1251
0.1089
0.0513
0,0750
0.0379
Impreve
16.48%
15.33%
18.39%
17.05%
13.99%
62.86%
42.31%
73.13%
63.32%
55.33%
Plain
0.1592
0,2450
0.1023
0.1272
0.1099
0.054
0,0245
DA0308
0,0279
Caser
BirDRec
0,2091
0.2872
0.1666
0.1443
0.0445
0
0,0369
0,0334
Improw
31.34%
39.39%
30.97%
31.30%
18.04%
11.15%
25.31
19.81%
19.71%
Plain
0.1597
0.1044
0.1304
0.1126
0.0293
0.0470
0,0243
0,0228
GRU4Rec
BirDRec
0,2167
0.2973
0,1495
0.1751
0.1511
0,0526
0.0761
0
Improw
38.69%
22.50%
43.20%
34 34.28%
34.19%
79.52%
61.91%
91.94%
78.6055
69.30%
Plain
0.1769
0,2662
0.1168
0.1451
0.1252
0,0475
0.0691
0.0313
0.0385
0.0343
SASRoe
BirDRee
0,2352
0.3259
0,1631
0.1915
0.0653
0.0903
0.0537
0,0482
Improv:
32.96%
22.43%
39.64%
31.98%
31.55%
37.47%
30.68%
46.65%
39.48%
40.52%
Plain
0.1736
0,2655
0.1142
0,1429
0.1219
0.0356
0.0563
0.0231
0.0293
0.0271
BERT-Rec
BirDRec
0.2319
0.3152
0.1613
0.1871
0.1622
0.0649
0.0896
0.0466
0.0544
0.0489
Improve
33.58%
18.72%
41.24%
30.93%
33.06%
82.30%
59.
101.73%
85.67%
80,44%
Plain
0.1802
0,2692
0.1196
0.1479
0.1265
0.0566
0.0798
0.0380
0.0455
0.0403
MAGNN
BirlDecc
0.2341
0.3211
0.1628
0.1903
0.1649
0.0634
0.0890
0,1435
0
0.0456
Improv
29.91
19.28%
36.12%
28.67%
30.36%
12.01%
11.53%
14.47%
13.85%
13.15%
Dalasets
Yelp
QK-Yedio
Plain
0.0502
0.0754
0.0398
0.0431
0.0425
0.0483
0,0803
0.0303
0.0405
0.0384
HPMC
BirDRoc
0.9743
0,0914
0,063
0.0686
0.0673
0.0756
0.1193
0,0483
0.0624
0,0669
Improv.
48.01%
21.22%
58.29%
59.16%
58.35%
56.52%
48.57%
59.41%
54.07%
48.18%
Plain
0.0337
0,0519
0.0228
0,0286
0.0277
0.0198
0.0831
0.0314
0.0121
0
Caser
BirDRec
0.0681
0
0.0519
0
0,0548
0.0746
0.1221
0.0479
0,0632
0,0573
Improv:
93.18%
57.61%
127.63%
99.65%
97.3%
49 80%
46.95%
52.55%
50.12%
44.70%
Plain
0.0320
0,0530
0.0198
0,0265
0.0257
0.0485
0.0816
0.0305
0,6100
0,0388
GRU4
BirDieec
0.0741
0
0.06612
0
0
0.0776
0.1253
0.0497
0,0611
0,0591
Improv.
131,56%
73.77%
204.04%%
148.68%
148 25%
60.00%
53.55%
62.95%
58.78%
$2.32
Plain
0.404
0,0574
0.0295
0.0349
D.0341
B,0511
D
0.0326
00435
0,0408
SASRec
BirDRec
0.0771
0,0965
0.0626
0,0687
0.0663
0,0815
0.1306
0.0523
0.0682
0
Improe
90.84%
68. 12%
112.20%
96.85%
94.43%
59 49%
52.21%
60.3%%
56.78%
50.98%
Plain
0.0421
0
0,0355
D.0668
0.0925
0.0473
0.0442
BERT4Roc
BirDRec
0.0734
0,0918
0
0.0657
0,0635
0
0.1297
0,0524
0.9677
0,0613
Impra:e
74.35%
53.77%
87.74%
75.20%
72.55%
46.42%
40.22%
45.02%
43.13%
38.69%
Plain
0.0436
0
0.0337
0,0388
0
0.0529
DL0880
0.0452
0.0426
MAGNN
BirDRoe
0.0560
0,0792
0,4433
0,0488
0.0465
0.0742
0.1184
0,0479
0.0619
0,0665
Improne
28.44%
31.56%
22.55%
25.77%
21.41%
40.266%
34.53%
39.24%
36.95%
32.63%
Time (6)
400
#4 Tippes Pate
300
200
GB
60
40
SASRee
STEAM
200
100
20
20
BVORN
ML
Be
Ye
QK
ML Be
Ye
QK
ML.
Be
Ye
QK
ML Be
Ye
QK
(a) Time per Epoch
(b) Convergence Epoch
(c) Memory Usage
(d) GPU Memory Usage
Figure 3: The comparison regarding time and storage costs.
better recommendation accuracy. For each user, we preserve the last two interactions for validation and testing, while the rest are used for training. The training of each model is carried out five times to report the average results. During testing, we follow 3 9 27 and evaluate the ranking results over the whole item set for fair comparison [50].
Implementation Details. For all methods, Xavier initializer [5]] and Adam optimizer
52 are
adopted; and the best hyper-parameter settings are empirically found based on the performance on the validation set. For BirDRec, it is implemented by PyTorch with batch size 1024, d = 64. learning rate- = 0.01 for Yelp and 0.001 for other datasets, L. 5, 0.1, 8 = 0.1, K = 10, and p 0.9. To ensure accurate rectification with reduced c, BirDRec is trained without rectification in the first 10 epochs. All the experiments are conducted on an NVIDIA Quadro RTX 8000 GPL 3
5.2 Experimental Results and Analysis
Overall Comparison. Table Saalepppree presents the performance of vanilla SRSs built on various representa- tive backbones. Each SRS is trained under two different settings: the original Plain setting and our
Our source code and experiment details (e.g.. parameter settings for baselines) are in the Appendix.
8

---

Table 3: Performance comparison with existing robust SRSs, where the best performance is boldfaced and the runner up marked by '*'. Improv. means the relative improvement of BirDRec over the runner up. The significance of the improvement is determined by paired t-test with p I 0.001.
Dalaveis
ML-IM
Beauty
Metries
HRes
HR
NIXG8
NIXG
MRR
HR
HR
NDXG68
NDCG
MRR
BERD
0.1922
0.2814
0.1267
0.1335
0,9745
0.0322
0.8364
FMLF-Roc
0.1789
0.2685
0.1207
0.1492
0.1299
0.0501
0.0743
0.0384"
0,0448
0,04D4
STEAM
0.119%
0.1950
0.0765
D. 0.1006
0.0874
0.0495
0.0765
0.0244
00414
0.0371
BirDRee
0,2352
0.3259
0.1631
0.1915
0.1647
0.0653
0,0903
0,0459
0,0537
0,0482
Imsprrn:
22.37%
15.81%
28.73%
23.239%
23.37%
30.3455
18.04%
19.57%
19,87%
19.31%
Datasets
Yelp
QK-Vedio
BERD
0.0425
0.9636
0.0304
0,0548
0.0958
0.0352
0.0161
FMLP-Ree
0.0499
0.8702
0.0374
0,0433
0.0422
0.0604
0.8991
0.0391
0,0514
0.0480
STEAM
0.0556
0.0822
0.3387
0.04445
0.1597
0.1021
0.0371
011507
0104655
BirDRec
0.0771
0.0965
0.0626
0,0687
0.0663
0.0815
0.1306
0.0523
0,0682
0.0616
Impprn:
38.675
17.40%
61.76%
45.24%
47.99%
34.93%
27.91%
33.76%
32.68%
28.33%
n Plain (SASRec)
DRUT
DDDU
M DRUT DDUI
DRUT DDUT + Self-ensemble (RirDRec)
0.19
NDCG@10
0.05
0..07
1DCG
0.07
0.17
0.04
0 0.05
8 0.06
0.06
20.13
0.
0.03
0.04
0..03
0.04
0.05
ML-IM
Beauty
Yelp
QK-Vodio
Figure 4: Ablation study on different variants of BirDRec.
BirDRec framework. We can note that all the baselines are boosted significantly all datasets with the aid of BirDRec, which demonstrates the generality of BirDRec.
Table 3 compares BirDRec with state-of-the-arr robust SRSs. Considering BERD and STEAM are both built upon Transformer, we also adopt a transformer-based SRS, SASRec, as the default backbone of BirDRec in the following experiments to ensure fair comparisons. The results show that BirDRec dramatically outperforms existing robust SRSs, confirming the effectiveness of our error- bounded rectification strategies in comparison to existing methods that lack theoretical guarantees.
Fig. compares the time and storage cost of BirDRec and STEAM, both of which aim handle unrelaall input and targets simultancously. In particular, Figs. B a) and (b) show that BirDRec executes 4.28 times faster than STEAM in each epoch and converges with fewer epochs. Meanwhile, Figs.Bjo and (d) show that BirDRec incurs only half the storage cost of STEAM on most datasets. These results highlight the efficiency of our sampling strategy and the self-ensemble mechanism.
The storage cost of BirDRec is marginally higher than STEAM's on ML-IM. This is because the advantage of our rectification sampling strategy is less obvious on datasets with small item sets (see Table D. indicating the superior scalability of our BirDRec on large-scale item sets.
Ablation Study. To check the efficacy of essential strategies of BirDRec (DRUT, DDUI, and the self cefmmensemlle) we add these strategies incrementally to plain representative SRS - SASRec. Note that without self-ensemble, DRUT and DDUI only consider the prediction scores the latest epoch for data rectification. The results are presented in Fig. 4 and similar trends can be noted with the rest SRSs on the other metries. First, adding either DRUT or DDUI to SASRec brings dramatic improvements, implying the effectiveness of DRUT and DDUI in rectifying unreliable targets and input, respectively. Second, using DRUT and DDUI together is better than leveraging each of them solely, indicating the necessity of rectifying both unreliable targets and input for more robust SRSs. Moreover, adding the self-ensemble mechanism can further boost the accuracy, which confirms the efficacy of considering prediction scores of different epochs for data rectification. Overall, the performance gain of separately adding DRUT, DDUI, and self-ensemble is 45.7%. 43.3%, and 3.1%, respectively. That is, DRUT and DDUI contribute more than the self-ensemble to our BirDRec.
Hlyper-parameter Analysis. We analyze the impact of key saper-parameters i.e., the rectification thresholds 8, 8', the size K of the rectification pool, and the exponential decay rate P. The results are presented in Fig. [5] with several major findings (similar trends can be noted with the other metrics on the rest datasets). (i) The best choice for 33 and 8 is 0.1, showing that small thresholds tend to maintain unreliable instances while large ones may mistakenly rectify reliable data. (ii) K 2 10 is sufficient to gain better accuracy. (iii) 2 0.5 yields better performance, that is, it is beneficial to consider predictions in earlier epochs. (iv) Even if the hyper-parameters of BirDRee are not optimally set, Ric still dramatically outperforms the best baseline (BERD) on ML-IM.

---

NDCXe t0
0.19
0.19
0.19
NDCOBR10 10
0.19
BirDee
0.17
0.17
0.17
0.17
BERD SASRee
0.15
(8)
10g10 (8))
K
Figure 5: Effect of key hyper-parament 3, 8", K, and p of BirDRec ML-IM dataset.
Case Study. Fig. 6 presents a real instance that is rectified by BirDRec on ML-IM. Specifically, in the DRUT module, the target movie 'Schindler's s List' consistently gets lower scores than the movie 'Terminator 2' in the rectification pool, and is then replaced by TTerminator 2'. This is reasonable, given that 'Terminator 2' aligns more closely with the input Sci-fi movies than 'Schindler's List'. Subsequently, in the DDUI module, the input movie 'American Beauty' is consistently lower-scored than the movie 'Star Wars IV' the rectification pool and is thus deleted. This decision justifiable, considering that the rectified target, 'Terminator 2', generally lacks relevance to the input 'American Beauty' across various aspects, including genres, actors, directors, tags, etc.
Observed Instance
Seff-enseembled Forward SRS
DRUT
Rectry Target
x
Deplace
Top:1]
Rectification Pools
BEAUTY
LOAK POR
X
Delete
Sell-ensembied Backward SRS
DDUI
Inpur
X
Input X
Target Ri
Figure 6: The overall architecture of the proposed BirDRec framework.
Limitations of BirDRec. Although BirDRec exhibits its superiority through our extensive experi- ments, its limitations are two-fold. Firstly, the improvement of BirDRec on storage cost is limited for smaller item sets. This is primarily because the rectification sampling strategy with self-ensemble has less apparent advantages on datasets with smaller item sets. for calculating weighted average prediction scores is from O([V) H) to (() for each instance. Thus if [V] is small, the benefit of this reduction will be less obvious. Secondly, although BirDRec is significantly faster than the latest robust SRS (STEAM), it is worth noting that BirDRec is 1.6 times on average slower than its backbone model (as depicted in Fig. 3 in each training epoch. This increased training time could be practical concern in systems with extremely large-scale datasets and real-time recommendation demands.
specific, the storage cost reduction
Conclusion
This work, for the first time, provides theoretically guaranteed data rectification strategies tackle both unreliable input and targets for more robust SRSs. The proposed strategies are further integrated into model-agnostic bidirectional data rectification framework, BirDRec, that can be flexibly implemented with most existing SRSs, for robust training against unreliable data. Additionally, we devise rectification sampling strategy to reduce the computational cost of BirDRec; meanwhile, self-ensemble mechanism is adopted to reduce the space complexity. Extensive experiments verify the generality, effectiveness, and efficiency of the proposed BirDRec.
Acknowledgments
The work is partially supported by the National Key Research and Development Program of China (B020YFBI)) 1707900), the National Natural Science Foundation of China (Nos. U22A2025, 62072088. 62232007), Liaoning Provincial Science and Technology Plan Project Key R&D Department of Science and Technology (No. 2023JH2/101300188), and ARC Discovery Projects DP200101441 and DP230100676.
10

---

References
[1] Shoujin Wang, Liang Hu, Yan Wang, Longbing Cao, Quan Sheng, and Mehmet Orgun. Sequential recommender systems: challenges, progress and prospects. In IJCAI, pages 6332- 6338, 2019.
[2] Hui Fang, Danning Zhang, Yiheng Shu, and Guibing Guo. Deep learning for sequential recommendation: Algorithms, influential factors, and evaluations. ACM TOIS, 39(1):1-42, 2020.
[3] Ruihong Qiu, Zi Huang, Hongzhi Yin, and Zijian Wang. Contrastive learning for representation degeneration problem sequential recommendation. In WSDMM pages 813-823, 2022.
[4] Yu Zheng, Chen Gao, Jianxin Chang, Yanan Niu, Yang Song, Depeng Jin, and Yong Li. Disentangling long and short-term interests for recommendation. In The pages 2256- 2267, 2022.
[5] Yongjun Chen, Zhiwei Liu, Jia Li, Julian McAuley, and Caiming Xiong. Intent contrastive learning for sequential recommendation. In The WheeeeCnnf, pages 2172-2182, 2022.
[6] Jing Zhang, Bowen Hao, Bo Chen, Cuiping Li, Hong Chen, and Jimeng Sun. Hierarchical reinforcement learning for course recommendation in moocs. In AAAI, pages 435-442, 2019.
[7] Xu Chen, Hongteng Xu, Yongfeng Zhang, Jiaxi Tang. Yixin Cao, Zheng Qin, and Hongyuan Zha. Sequential recommendation with user memory networks. In WSDM, pages 108-116 2018.
[8] Yatong Sun, Bin Wang, Zhu Sun, and Xiaochun Yang. Does every data instance matter? enhancing sequential recommendation by eliminating unreliable data. In IJCAI, pages 1579- 1585, 2021.
[9] Kun Zhou, Hui Yu, Wayne Xin Zhao, and Ji-Rong Wen. Filter-enhanced mlp is all you need for sequential recommendation. In The WebConf, pages 2388-2399, 2022.
[10] Yujie Lin, Chenyang Wang, Zhumin Chen, Zhaochun Ren, Xin Xin, Qiang Yan, Maarten de Rijke, Xiuzhen Cheng, and Pengjie Ren. A self-correcting sequential recommender. In The TheCeeC pages 1283-1293, 2023.
[11] Ziwei Fan, Zhiwei Liu, Yu Wang, Alice Wang, Zahra Nazari, Lei Zheng, Hao Peng, and Philip S Yu. Sequential recommendation stochastic self-attention. In The WebConf, pages 2036-2047, 2022.
[12] Chi Zhang, Yantong Du, Xiangyu Zhao, Qilong Han, Rui Chen, and Li. Hierarchical item inconsistency signal learning for sequence denoising in sequential recommendation. In CIKM, pages
2022.
[13] Yuqi Qin, Pengfei Wang, and Chenliang Li. The world is binary: Contrastive learning for denoising next basket recommendation. In SIGIR, pages 859-868, 2021.
[14] Hongjun Wang and Yisen Wang. Self-ensemble adversarial training for improved robustness. In ICLR, 2022.
[15] Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme, Factorizing personalized markov chains for next-basket recommendation, In WehCof,, pages 81 1-8800 2010.
[16] Chen Cheng, Haiqin Yang, Michael R Lyu, and Irwin King. Where you like to go next: successive point-of-interr recommendation. In IJCAL, pages 2605-2611, 2013.
[17] Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based recommendations with recurrent neural networks. In ICLR, 2016.
[18] Feng Yu, Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. A dynamic recurrent model for next basket recommendation. In SIGIR, pages 729-732, 2016.
[19] Chao-Yuan Wu, Amr Ahmed, Alex Beutel, Alexander Smola, and How Jing. Recurrent recommender networks. In WSDM, pages 495-503, 2017.

---

[20] Jiaxi Tang and Ke Wang. Personalized top-n sequential recommendation via convolutional sequence embedding. In WSDM, pages 565-573, 2018.
[21] Trinh Xuan Tuan and Minh Phuong Tu. 3d convolutional networks for session-based recom- mendation with content features. In RecSys, pages 138-146, 2017.
[22] Fajie Yuan, Alexandros Karatzoglou, loannis Arapakis, Joemon M Jose, and Xiangnan He. A simple convolutional generative network for next item recommendation. In WSDM, pages 582-590, 2019.
[23] Mengqi Zhang, Shu Wu, Xueli Yu, Qiang Liu, and Liang Wang. Dynamic graph neural networks for sequential recommendation. IEEE TKDE, 35(5):4741 -7533 2023.
[24] Jianxin Chang, Chen Gao, Yu Zheng, Yiqun Hui, Yanan Niu, Yang Song, Depeng Jin, and Yong Li. Sequential recommendation with graph neural networks. In SIGIR, pages 378-387, 2021.
[25] Chengfeng Xu, Pengpeng Zhao, Yanchi Liu, Victor S Sheng, Jiajie Xu, Fuzhen Zhuang, Junhua Fang, and Xiaofang Zhou, Graph contextualized self-attention network for session-based recommendation. In IJCAL, pages 3940-3946 2019.
[26] Shu Wu, Yuyuan Tang, Yangiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. Session-based recommendation with graph neural networks. In AAAI, pages 346-353, 2019.
[27] Chen Ma, Liheng Ma, Yingxue Zhang, Jianing Sun, Xue Liu, and Mark Coates. Memory augmented graph neural networks for sequential recommendation. In AAAI, pages 5045-5052, 2020.
[28] Wang-Chent Kang and Julian McAuley. Self-attentive sequential recommendation. ICDM, pages 197-206, 2018.
[29] Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In CIKM, pages I 1441-1450, 2019.
[30] Zhu Sun, Qing Guo, Jie Yang, Hui Fang, Guibing Guo, Jie Zhang, and Robin Burke, Research commentary on recommendations with side information: survey and research directions. Electronic Commerce Research and Applications (ECRA), 37, 2019.
[31] Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. Learning from noisy labels with deep neural networks: A survey. IEEE TNNLS, 2022.
[32] Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. 2018
In NeurlPS, pages
[33] Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor Tsang, and Masashi Sugiyama. How does disagreement help generalization against label corruption? In ICML, pages 7164-7173, 2019.
[34] Hongxin Wei, Lei Feng, Xiangyu Chen, and Bo An. Combating noisy labels by agreement: A joint training method with co-regularization. In CVPR, pages 13723-13732, 2020.
[35] Xiaobo Xia, Tongliang Liu, Bo Han, Mingming Gong, Jun Yu, Gang Niu, and Masashi Sugiyama. Sample selection with uncertainty of losses for learning with noisy labels. In ICLR, 2022.
[36] Yangdi Lu, Yang Bo, and Wenbo He. Noise attention learning: Enhancing noise robustness by gradient scaling. In NeurIPS, pages 23164-23177, 2022.
[37] Yangdi Lu and Wenbo Hc. SELC: self-ensemble label correction improves learning with noisy 2022.
labels. In IJCAI, pages
[38] Songzhu Zheng, Pengxiang Wu, Aman Goswami, Mayank Goswami, Dimitris Metaxas, and Chao Chen. Error-bounded correction of noisy labels. In ICML, pages 11447-11457, 2020.
12

---

[39] Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. Neural attentive session-based recommendation. In CIKM, pages 747-755, 2017.
[40] Jiaxi Tang, Francois Belletti, Sagar Jain, Minmin Chen, Alex Beutel, Can Xu, and Ed H. Chi. Towards neural mixture recommender for long range dependent user sequences. In The WebConf, pages 1782-1793 2019.
[41] Chen Ma, Peng Kang, and Xue Liu. Hierarchical gating networks for sequential recommenda- tion. In KDD, pages 825-833, 2019.
[42] Ziwei Fan, Zhiwei Liu, Shen Wang, Lei Zheng, and Philip S Yu. Modeling sequences as distributions with uncertainty for sequential recommendation. In CIKM, pages 3019-3023, 2021.
[43] Di-Rong Chen and Tao Sun. Consistency of multiclass empirical risk minimization methods based on convex loss. JMLR, 7:2435-244 2447, 2006.
[44] Jingtao Ding, Yuhan Quan, Quanming Yao, Yong Li, and Depeng Jin. Simplify and robustify negative sampling for implicit collaborative filtering. In NeurIPS, pages 1094-1105, 2020.
[45] Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. ACM THS, 5(4): 1-19, 2015.
[46] Julian McAuley and Jure Leskovec. Hidden factors and hidden topics: understanding rating dimensions with review text. In RecSys, pages 165-172, 2013.
[47] Guanghu Yuan, Fajie Yuan, Yudong I.,, Beibei Kong, Shujie Li, Lei Chen, Min Yang, Chenyun YU, Bo Hu, Zang Li, Yu Xu, and Xiaohu Qie. Tenrec: large-scal multipurpose benchmark dataset for recommender systems. In NeurlPS, pages 11480-11493, 2022.
[48] Zhu Sun, Di Yu, Hui Fang, Jie Yang, Xinghua Qu, Jie Zhang, and Cong Geng, Are evaluating rigorously? benchmarking recommendation for reproducible evaluation and fair comparison. In RecSys, pages 23-32, 2020.
[49] Zhu Sun, Hui Fang, Jie Yang, Xinghua Qu, Hongyang Liu, Di Yu, Yew-Soon Ong, and Jie Zhang. Daisyrec 2.0: Benchmarking recommendation for rigorous evaluation. IEEE TPAMI, (5(7):8206-8226 2023.
[50] Walid Krichene and Steffen Rendle, On sampled metrics for item recommendation. In KDD, pages
2020.
[51] Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In AISTATS, pages 249-256, 2010.
[52] Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. In ICLR, 2015.
13

---

Theoretically Guaranteed Bidirectional Data Rectification for Robust Sequential Recommendation - AAppendix
This Appendix is divided into three sections. First, Section empirically validates the feasibility of Assumption [ Next, in Section [ complete proofs of all the lemmas and theorems are presented. provide detailed settings of baselines and additional experimental results,
Finally, in Section including the hyper-parameter analysis and the percentage of rectified data. Our code available at: https:// htt.ttth...cmmmmmceeies////////
A The Feasibility of Assumption
Assumption 1. The users' true preference distribution 1) fulfills the relaxed Multiclass Tsybakov Condition [ with constants >> X and Ciu E (0, 1), such that for all (0).
yip (x!) - Mlp (x ) S sa)= S Caa.
(I)
The feasibility of Assumption relies on large C' and small X. In order to estimate the values of C and X, the initial step is to approximate the true preference distribution 7). To achieve this, we first obtain reliable dataset D via the heuristic method proposed by 22 The heuristic method measures the matching degree between the input and target each instance from two aspects: item co-occurrence and item properties, then those instances with matching degrees lower than threshold filtered as unreliable data. We use a large threshold (0.9) to filter the dataset rigorously, ensuring the vast majority of the maintained instances in D reliable. Subsequently,
suggested by (3) train classic SRS, SASRec [4], on the filtered reliable dataset 2 and then use the prediction of SASRec to
approximate n. Formally, for each reliable instance (x"" of)) from the reliable dataset D, the SRS prediction score fop (x")) is employed as the approximation of Nop (x()).
Next, densely sample from 0.05 0.9 with step size 0.005 and calculate the corresponding left-hand-side probability of Eq. [ with the following relative frequency Fa, and collect a series of (log(a), (og(a) log(Fa)) data points:
F.
Σ (x)) - (xa' S sa. (xDD
(2)
where in and Po are respectively the top- and middle-ranked items according to S. namely,
Ev,evIffp (x+)) z fe, (x,) = [V], Iff (x") 2 fe, (x")] /2 We then use log(F.) to approximate log(Ca>). As shown by the blue dots in Fig. 6 the collected (log(a), log(F.)) data points are generally linearly distributed, which allows us to estimate C and À with linear regression according to log(Ca>) log(C) + log(a)) As a result, Fig. G that the estimated and À are respectively restricted in (0.55, 70) and (1.37, 4.01) on real-world datasets from various domains, which validates the feasibility of Assumption
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Observed (log(d), log(fa))
Observed (log(a), logif
-2
Regression Line: Alog())+log(C)
Regression Line: Aloglo)))og(()
-2
-4
-3
log(Fa)
log(Fa)
-6
-8
-10
Estimated C: 0.6895
Estimated 2.7473
-6
Estimated C: 0.5579
Estimated A: 1.3720
-12
-7
-4
-3
-2
-1
-4
-3
-2
-1
log(a)
log(a)
(a) ML-IM
(b) Beauty
Observed (og(a), log(F.))
Observed (log(a). log(Fa))
-2
Regression Line: Alog())+
-2
Regression Line: Alog(r))+og(()
-4
-4
-6
log(Fa)
-8
log(Fa)
-8
-10
-10
-12
Estimated 0.6813
00000
Estimated A: 4.0051
-12
Estimated C: 0.6941
0.00
Estimated A: 3.4551
-14
-00
-16
-4
-3
-2 log(a)
-4
3
-2 log(a)
(c) Yelp
(d) VeVio
Figure 1: The estimated constants C' and on various datasets.
B Proofs of Lemmas and Theorems
B.1 The Proof of Theorem
Theorem 1. Given Assumption 1, let {wi, IISSSHA wa S1 wa = =1} be the weights for averaging prediction scores of different epochs. (Xx, i7), assume S currper (8P). Let y=
Topor Me + Toyer (&c) (e, (K7). We have: P
P(p1 p1 oū ΣH
| $ ((O(e))
Proof.
P
P1
-M
Wh. ph (x/)] S
h=1
<IP
P1 1
whh May (xx1) - S
h=1
TP
P1 Ty May (xx)))
17p (x
Wh
Σ 2y
T/y (x²) - 5
(3)
EV
<TP
P1 ... May (Sc)) 2 1pp (x,)) कy (x:) May (x:')
Σ
Toyoy (x Muy (x," $ +
P1 .y. Mpp (SE))) < Map (x,' < + Enjdee Toyey (X)) Mos (R
Topay (x:))
By replacing with Top (x:) (x2)) + Enysoy Teyoy (x²)) (x() we obtain:

---

H
P
Whh fe (x)
S
7
SP
Typ (x,) I Tpp (x,) - 1pp (x,)
+
(x
1=1
Recall that oz (x+),
which implies
og. Hence, the relaxed Multiclass
oz (KF)
Tsybakov Condition holds and the probability is bounded by C
namely, ((O(e)) O
Topor (x.)
B.2 The Proof of Lemma
[wn Sop (8())
Σx s (8.))
EDRUT seuur=Piig P p1= 0 pi
<
pirin pistum
/ fii (xP1)
> Sh (x(11
Case-1
Case-2
Case-3
Lemma 1. Given Assumption 1 and the set of weights {wh
H2O
Wh S 1, 1, Wh 1}. V VRR of), assume
min [ootap or (8T). COTen:m (x7)).
Let $1
Popppy Dipp (8T) Tuyat (oy and $2 = Tom (& Hiyg (RT) Ev. from Toyem (81 (R1))ng (87)) ] We
Eh (Ww (K())
E Em_ f (87)]
have: B 31 guarantees the probability of Case-l in EDRUT bounded by ((O())) and B 2 82
guarantees the probability of Case-2 EDRUT is bounded by ((O(e))*
Proof. For Case-1 of EDRUT. we have:
H
Wh. th (x2)
P
P1
1/8 PI
Um
< B1
WhFt fh (x+)
h=
Whn
(x
<P
P1 1t
B1
W om (x2))
-=1
<P
P1 1/
May
(x;)
1pp
(x
Wh Map (x)) - 31
wn, [wn (xx)
h=1
h=1
IP
i Tep (x2)) ス 7pp (xc]) ).
Mop (x'"
< 11
Wh eh
(x'") +
vm
hm
P
P1 0/ Map (3²")
7pp (x²)).
Toya (x:2) B1
[witth om (s:;)]
MyEV
B1 1= [wht ph (xx)
+ Deytop Tooay Boy (x+1
=P
P1 ij Mp2 (xx)' I mop (xx))
Topoy (xC)
(5)
By substituting B1 with Fo (8(') Mog (89)) (81) En sty Tuyap Pey (xE)
in Eq.
we obtain:
Whh L (x+)
PP
U. p1: + Um P1
81
[wnft fh (xx))
(6)
<IP
Mpp
(x)) < Tpp (xx)
Top a. (x:+)
Recall that < oo (x²), which implies
S Ogp Hence, the Multiclass Tsybakov
Top (87)
Condition holds and the probability of Case-1 is bounded by C( Topoo (67))) ) n.nelyyyy C(O(e))
n

---

Thereafter, for Case-2 of EDRUT, we have:
Emm hmml
Wp fh (xx)
P
P1 + r,
Um
2 82
ff
w (xx) }]
Wwh 87
(xc))
<P
P1 =
h=
2 82
Win f (xi²)
cm
Σ11
[Wnfom (xc7)
<P
P1 = Um (X;)
2 1/22 (x)) )
w/ 7i (x:)) - c] -
B2
1=1
Σ [ (xc;)
P
P1 = Um 7mm (x
Mpp (xc) 7m (x²") I
+
82
Σx=
[Wh.bb. om (xx)
P
P1 = Um 7/
(xx)) (x:) 2 1/pa (x)'),
Σ
To (K)) (x11
S
+
B3
Σ [en S (8x"))
Eugtum [Teyeen (XC Mo,) (xx))
P
P1 Um T/p2 (xx) S 7m (xx)) <
Tem (xx)
Tom (XC))
(7)
We replace 32 with
in Eq. 5 and continue the calculation:
Tem Fm (x7 plp (2.7 ) )+ w st Tayem um (x7 Ma,, (xCP)
Σx A=1
Wh, J (x()
IP
P1 + 02 P1 = Um
2 B82
Σx=1 Win fhh (xC
(8)
1/p2 (X,) S 1mmm S 11p2 (2()) + IPP
e
Tom Mm (Sc()
Recall that og Tem (x(),, which implies
(x.") S ag. Hence, the Multiclass Tsybakov
Condition holds and the probabiliy of Case-2 is bounded by C( (Rp7)) namely, ((O()) O
Towe Men
B.3 The Proof of Theorem
Theorem 2 (The Upper Bound of EDRUT). Given Assumption and the set of weights {wh 15
[Togog (82 M9p (82) + #oy Thy op my (R2)
H.O wh < 1. E Wh 1}, V (Kx, er), let
El (wrg 1h (811)
I (wa fi (xT))
8a Tumum (8P Ping (RP) Enj Tuy om (RP)e, (x4)) E1
18 Bil. and &2
3z/ Assume
IlTTI. -p (000 Σ, 1 #p (To,em (87) (e (87) Ma, (811]]. We have: EDRUT S ((((e+)1)) +0(00+)))+++((00)
< 82, min ay (87)) E1. antum (8, (82
-82- 52)
8a (2)

---

Proof. For Case-1 of EDRUT. we have:
Σ
wh Wh fh (x:)
P
PI 0q x PI
X Um
<B
[wn (x:])
h=1
H
h [x,)
Ru P1
h 1
$
wh/ th (x.')
H
11
P1 = 0 7ap (x.) 2 1p (x
Wh.
Map (x)) - < 3
Σ
Wu, Sf firm [xx²)
h=1
h=1
H
P
P1
May (x)) >
1/2 (x
May (8()
< B
WhJ.
km
(xx)
+
h=1
P1 8, 78p (x)) 2 1pp (x
Tryet Tu (x.)
< B
ww f tim
(x:) + c
B 24=1
Wii cm (xE))
+ E E
[Tey To, (x])
P1 o Up2 (xx)
Mाy (xt)
Topey (x()
(3) + E1) Eim
[wnn (xx) + E Enpztry [To Mny (xP)
<P
P1 8i, 1p (x²) S Tap (x,) <
Top (xC))
substitute 3, with Te (87 Figg (87)+
po Toy oz Tuy (x7)
and obtain:
E (wr 8 (x1)
Σ -=1
Wh fb (x
P1 0/ P1 # Um
< /
Σ
11
f/ (x:²)
h=
1im
E1 Σ -
(t (xC)
<P
P1 0/ U/2 (xx) S Mp. (xx) < 1/ps (x(")
(10)
Tax (x())
6 E1 oo (327)
1pp (xx)) S 1p (x:' < Mpp (X(1)
Recall that Tp (x+) ε1, which implies
e+E1 rapay (RP) I ag. Hence, the Multiclass Tsybakov
Condition holds and the probabiliy of Case-2 is bounded by C
c+$1 Topoy (xP)
.anallyy C(O(E++i))
5

---

For Case-2 of EDRUT, we have:
Σ
H
wh fhe (xC1)
===1
P
P1 # 0 P1
Um
2 8
>m h=
wnJ.i (xx) ]]
Σイ=1
11
whe
(xc))
<P
P1 mm
W f
(x:)
1==
-x =k=1
[wnJe.. [xx
<P
P1 Uma Tmm (xi')
2 112 (x )
wh 7'm (x el
P1 Um 7m (x)) 2 1/2 (xx). 7m (x²) S 2m==1 [wn (xx))
P
+
B
11
[wps (SEP)
P
P1 Um 7m (x7)) 2 1/p2 (x)))
Σ
vo (x:) Moy (x71)
h=1
+
Σ [wn fo (8e") )
TTay (xx)) (xt²)
P1 Um 1pp (x:) S 7m (x S
Tem Vim (xc)
Tem Vm (x²))
> h- [ovn, f (82))
Σ Siggttm
(Tay (x()) (x27)
<P
P1 = Um 1p2 (xx) S Tm (x2)) S
8q
Tem Von (xC())
Tem Vm (xc!)
11 (on (s:))
Enggttrm [Teyyan (xx (x(1)
e
=P
P1 = Um 1p (x,') S Bm (x,)) <
Te Mon (xc7)
Tem Ven (x²))
& Σm= [Wn. sth (x,)
H
+
B2 E2 Tr Nim (2))
(11)
Em Www Say (x.')
We replace 32 with
Taye Myn (X. Mpg (XF + 2 plom Teyam (27 (8F (XP] and continue the calculation:
Σ
H
Whe ch (3:2)
1=
P
P1 x 28
P1
P1
Umn
B
ΣØ
Wh pm (x())
c
E2 -1
[wh, ph
(x² )]
<P
P1 = Um Upp (x:)) S Tm (x.") S Mpp (xc +
+
(12)
Tum um (x,)
B2(B2
{2) Tem um (5c²)
E2
<IP
1pp (x() S Tp. (x+) S Tpp (x,)
+
Tom Von (x))
B2
Tu Un (x(1)
Recall that < Cip
(8. (2 S2) which implies Tey (x.)
Ba Tom (x)) S cr--
3a(32- {2)
Hence, the relaxed Multiclass Tsybakov Condition holds and the probability of Case-2 is bounded by (2)
C(
namely, C(O(e + =²))
(B E2 Tom vo (xP)

---

Finally, for Case-3 of EDRUT, we have:
Pi + t P1 *
vm
P1 + Um
Wh ph (x²)
p1 # Um: 1/p1 (xx) 2 11p2 1)p2(x (xx),
h=1
hm1
H
W//.
f (8c]1
H
SP
1p (xx) 2 17p2 (x:)) ),
Wh. ph
< (xx))
W/ firm (x,))
hm1
h=
P
Mp (xx) T/pa (xx)
w/h [7pp (x,) - []
<
Wh
(xxx) +e]
1=1
h=1
7pp (xx) 2 Tpp (x))) 7p. (xx)
< Am (xc))
P
Tpi (xx) 2 T/p2 (x(',
Σ
Tespi (X (x+))
Tayam (R)) (x1) + 2e
BEEV
eyev
F (x(') 2 Thp (x)) ), Tripi (
1pp (x,)) + Σ
Toypi (x (x")) (xx) <
(13)
Tp (x,) (x.) + Σ
Tayam (x) Mo (x,) + 2e
2ypp
1/m (xx') 2 1/p2 (x''), Upp (x Tpmm (8 Tpmum (x11) < FP
Σ (Tayem (xx)) Tuzpi (82)) [87))(1ss (8c)) 2c
1/p (s:)') 2 Tpp (x:) Upp (xc)) Eejtpr (Tes um (xx))
Tupps (x2 M/s (xx) + 2c
-P
Tpipi (xQ') Tpt vm (x²))
=P
1p2 (x:) I Tpp ) < 1p2 (xx) )
Engern [Teguan (X7) Tespi (XC) Moy (x1) + 2e [TpaPs (x1) Tpiem (8c2) M3p (x2))
Tpim (8C)) Tpin.. (xx1
Recall that c [TTipi (K7) Tprom (x2) } [au Vip (344
EnyAps [Tayem (8t) Teyai (244 )] mey (G4)
which implies Enjer (Toyein ( in (RF)) muy +2)) [TaL (xf) - om (87) moy (97)
< ag- Hence,
[Ta (-1mm (R7)]
the relaxed Multiclass Tsybakov Condition holds and the probability of Case-3 is bounded by
Sop [Teyem (87) pe (RP) Me, +2c (aa (RP)) sm (x2) } may (RT)
namelyy C(O()))
Prrpy (RP) -yrrm (RP)
B.4 The Proof of Theorem
k [wn 7A
(xT' }]
[wn. o (xP)
EDDUt =P
1/
7h (wn
Fm (xc4
<
+P
rh # 07
Σ m
[wn Fl. Rm
[x:]
g8
(14)
7

---

Theorem 3 (The Upper Bound of EDDUI). Given Assumption 1 and the set of weights {wh 15
6 -r (8t (, (87) 22, "nyor Nog (87)
5 H,0 S wh I 1, 1 h== = =1}, A (x), Bi-L) let B
=
Ik 1vn. Pi,." (x())
Ex=ille Pop (1
8/- Fm (xF 11 (KT) + Lu Th pm (xF Wi (837
Ei
18 8il. 62 8 - 5/2b Assume E2 < 122
e min (ooTer 12
E
oo Nin 187 (0)
-6))-4
2 |1 (84) (8)))))))))))) -, (8)) + 1/24 (xx)
Dejtto Fo (xt)) Tups (x) (,,,, (xx) We have: EDDUI & ((((((+(1)))++)))))))))(())))))) +c(ole + +(61) +c((())) A
Proof. For the first term of EDUI, we have:
ΣI hhm1
[w
or
[}
P
p1 1-1
[x
A=
H
<P
11
4 1 May
(x)) 2 r/23 (x))
w/ Wap (x2)) cc < 8
Σ
[wn D [xt))]
=P
14 = 0 Mag
(x.") 2 1/2 (xT" Map (x < 38 Σ [wn Fo vm
]] +e
H
=P
14 = 8 Mop (x7') 2 1/2 (
- 10 The (x+)
< 8
>
[wn (879) +
My Ev
8 hm
[wnThh (xP)) + c Enstere Toyee Th, (xP))
<P
1/ = 8 m/g (x.)) v nop (x)')
1o (x57)
(8) + E1) >
Jun.b om (52")] + E Eossoo Digp The (87)
<P
11 = 8y m/g (x,) S Vep (x <
p0 ($2")
(15)
The p (K7) (x+)+ H. Enyzot Tepop The (x7)
We substitute 3i with
and obtain:
ΣR [wnn /.... (x7)
>
H
[x])
hmm
P
P1
1 1
8"
[w T/t
[x
h 1
E >>> h=1
[Wh F/e em (x+)
<P
1 ا 1/g6 (xx)
1opn (x7)
1/g (x
Tap er
(x])
Tée 000 (xx)
é' + &
<PP
1/g (x*) S pla- (x2") < M/3s (x
+
T
oy_ (x())
(16)
Recall that é' S ToThy कo (x E. which implies Fop o (x7) S ag. Hence, the relaxed
Multiclass Tsybakov Condition holds, and the probability of the first term of EDDUI (Eq. 14 is namely, C(O(E' + *
bounded by C( ",11
c'ts'
For the second term of EDDUL we have:
Σd
Th
hmml
Wh
pu
P
pl
02 I:
Σ h=1
W
[x
8m
Σ# H1
1=
(w Do
P
pl
p1
Um
(17)
[wN Fm (xx) }]
h=
11
(wh 1i
(xx)
1=
+P
pl + Uz
pi + +ms
2 8'
[wh ch mm
(x
8

---

For the first term of Eq.
17
we have:
Σn=1
[wn op
(x+)
P
Pi #
P1
Um
2 8¹
Σ H
[wn
Fh (xc7)
h=1
Fm
Σ h=
[wh] p
(x44)
<P
P1 0
m,
H
Wh/
[227
3
Fim
11
H
[wn
F [xEtt
2h=1
<P
P1 = m, Wo..
(x,) 2 1lps (x,)
Wh 77 (xi) e7 5
3r'
h=1
=P
P1 = 8. mm (xx) 2 T/ps (x²"), ),
No (x,)) -
21==1
11
8m
(xE")
+ c²
-k=1
H
(5271)
P1 = m. 1mm. (xx)) 2 T/ps (x*))
Toyum (x)
Moy (x21)
Br
Σm=
[10, Jim (x7)
En
TTaynn (x2 (x27)
<P
P1 = m 1/23 (x²) S 1em (x+) 5
T/ Fm (xC))
Fm Fm (327)
[wn. 7h Tm (877)
Eng um
ITes Um (x (x)) (xx²)
<P
P1 = mm m/ (xx) S no.m.. (xx) 5
E2
T Um
(2((
Urn Fm (x²71
[w. 7 (327)) 8%/
Teyann (x Mhy (x27)
6'
=P
P1 8m 1/2 (xE() S T/.... (xt") x
Tiom F. (xx1)
(3Z
Fm Fm
& "こ
11
[wn] Th
(x
Em
+
8,(322
4 Jm
(x,)
(18)
MN [wan 7:r (8.7))
We replace 8/2 with
and continue the calculation:
Tom Fun
(x) 10
(x27) ) Du Thyun (xFP (KF))), (xP) (xF)
Σ H
Σ
WhJ By
(x4)
PP
P1 + 5y p4
Um
3
-
Whi
Fl. 8m
(x:
E/ Σ
[wn J Em (3
<T
1/ F., 1/1 (327)
1/6
(x S 1/p (x2")
(19)
(x))
( Fm Fm (x27)
Bun F.
E
<P
n/y (x.)) n/... (x7) S 1/2 (xx)
T F.
T F.
($c))
Recall that é' < ao 5m (3' 55
which implies - Fm (xT) Ag E F. (x))
R9 +
Hence, the relaxed Multiclass Tsybakov Condition holds and the probability of the first term of Eq. 17 bounded by C( 1 (1) -6
namely, C(O(e' + +()))²
A
(B2 1 V. Fm. (ET))
9

---

Finally, for the second term of Eq. 17. we have:
[wnJ By
(xt)
P Pi # Pi
o Um.
B¹
Σ 12
(2nh J km
(xx))
1
Σ 1=
[wn. Fo (R+)
P pí * Um. 162q (xi 2 M/l (xi))
Σ h=
22hh 7 (x+)
<P
1/4 (x)) 2 1/5 (x >
Jun. p/ (xx)) }]
<
IwiJ p (xx ]
H
H
1/4 (x)) 1/1 (
wh 17 (x7)
c'
Wh
Mim (x1) +
h=1
1/4 (x)) 1/2 (x"), 1 (xx))
Trmm (xx))
1/4 (x+) 1/2 (x+)
Tayss (xx My (xx)) <
Temm (x)) (xx)) 26'
RyEV
(20)
P 1/4 (x*) 2 M/ss (x
Σ
Tass mi
3/4 5m (x7 m/; (x:) + Σ
He.m (x)) (xx) +
PP 1/4 (x)) 2 1/25 (x") (x)) [T (xx) - To Fm (57)) <
Σ [Tém (xx) ) To (xx) ] n/o (xT) + 20
My
P m/g (x+) S v/p1 (xT) T/6g (x+))+
Σ Engheh (re.., (xx) - -Tayps (se, }} Why (kx) ) + 26' Poip (xx) - -T Fm (xt ) m/g (xx)
Irrp (z2)) - -T/4 pm (x]))
Recall that d' S 2 bil (8t) Tot um [K)))[au 1 n/63 (xF))
En ITmm (x27)
Fesey (x7) (x.)).
which implies Engren Ii (87)--Ty0 (RT)) nl, (3"))=2'([ (87)) 1/ s. (z7) (87)) (87)
og. Hence,
Teg (KT) pm (Kr7)
the relaxed Multiclass Tsybakov Condition holds and the probability of Case-3 is bounded by E Engaes [Ta,,, (K1)) Tappp (RF) né, (xT)) - Grige (87) - o.. (87) h/g (87) namely, c(O(e)))
(34p, (82) (87))-660 76gmm (87)
B.5 The Proof of Theorem 4
Theorem 4. Let R be a list of K. items randomly sampled from V with replacement, Ç E (0. 1), Tu(xp,0) = "' [E (xT) > HH [ (x7)]] be the rank of item is over the entire item set at the H-th epoch, and fx(x) v2) = E ER [[E [wn.f (8:) > Ei (wnft (xPII be the
rank of ve over R at the H-th epoch. We have: P]] (xC...)) (x(x7 2c)
191 2 <] - (x(C))
Proof. i' fn((x) (rr",,,)) than vi in R. Since the items in R randomly sampled from V with replacement, in (x(xx, v:)
u:) can be deemed as a random variable that counts the number of items ranked higher
follows a Binomial distribution Binomial(K,, TH (x(x))))) Thus, we have:
[
K ronnnxx w))
E [f vz)]
(21)
10

---

Meanwhile, fn (x",, 13) can also be viewed as the sum of K ii.d. Bernoulli random variables 11.b2.... bg. where by 1(1 < k I K) indicates the k-th sampled item for is ranked higher than Ui, and by 0 otherwise. Hence, by applying Hoeffding's inequality [
we have:
K
by I
 by
< exp( exp(-2k
(22)
K
Then by replacing Ek=1 by with i'r (x7, vi), we obtain:
Fn (x7 vc)
(xFn(xx, vs]]
26 S eep(-22(3)
(23)
K
K
Next, replace E (x" 14)
with (x [V]
according to Eq. 21
Fn (xF. es)
rn 1x8,,00 vi)
2c
S exx(-22<()).
(24)
M
C Additional Experimental Results
C.1 Hyper-parameter Settings for Baselines
For fair comparisons, we implement FPMC with PyTorch. For other baselines, we use the original code provided by the corresponding authors. All the baselines adopt Xavier [6] initializer and Adam 77] optimizer. We empirically find the optimal hyper-parameter setting for each baseline based on the performance on the validation set. The detailed hyper-parameter setting for each baseline is summarized in [
C.2 More Results on Hyper-parameter Analysis
NDC0@10
0.055
BiDRee
0.045
B
FMI-P-Re
0.035
SASRec
3
22
0.3
log (3)
log l
Figure 2: Effect of key hyper-parametert B, B,, K, and p of BirDRec on Beauty dataset.
NDCG@ 10
0.07
0.05
STEAM
0.03
SASRce
3
-2
0
4
3
2
0
5
10
15
20
25
0.1
0.3
0.5
0.7 0.9
(3)
log1a (3')
K
P
Figure 3: Effect of key
B, B', K, and of BirDRec on Yelp dataset.
0.075
NDCG@10
0.065
0.055
FMLP-Rec
0.045
2.. c 2 www. mm
-2
- 3
10
15
20
25
0.1
0.3
0.5
0.7
0.9
log (3)
log
K
P
Figure 4: Effect of key hyper-parameters 3, 3', K, and p of BirDRec on QK-Vedia dataset.
C.3 More Results on the Percentage of Rectified Instances

---

0.200
0.175
0.05
0.150
0.04
NDCG@10
0.125
NDCG@10
0.100
0.02
BiORec
0.01
BrDRec SASRec
0.025
SASRec
20
80
100
140
20
40
8D
100
120
Epoch
Epoch
(a) ML-IM
(b) Beauty
0.06
BirRecc SASRec
0.06
0.05
NDCG@11
NDCG@10
0.04
0.03
0.04
0.02
0.03
0.01
BirDRsc
0.02
SASRec
25
100 Epoch
125
150
Epoch
(c) Yelp
(d) QK-QQQQVie Video
Figure 5: Testing accuracy (NDCG@ 10) of BirDRec and SASRec with increasing epochs. The training stops if the best accuracy does not increase in 25 consecutive epochs.
0.22
Percentage of Instances
Input Recifiied
Percentage Instances
0.08
Recified
0.20
Target Reciified
0.07
Target Redified
0.18
0.06
0.05
0.16
0.04
0.14
0.03
0.12
10
50
60
20
30
40
Epoch
Epoch
(a) MI.-IM
(b) Beauty
0.13
Percentage of Instances
Input Rectified
Percentage of Instances
Input Rectified
Target Rectifid
0.12
Target Recified
0.035
0.11
0.030
0.10
0.025
0.09
0.08
0.015
0.07
0.010
0.06
10
20
30
40
10
30
40
50
60
Epoch
Epoch
(c) Yelp
(d) QK-VQQQVA -ideideo
Figure 6: The percentage of instances that are rectified with increasing epochs.
12

---

Table 1: The search space of hyper-paa perpmt-parameters and the optimal settings found by grid search for all baselines on the four real-world datasets.
Parameter
ML
Be
Ye
QK
Scarch Space
embedding size
64
64
64
128
16,32,644 64, 128]
FPMC
learning rate
10-3
10
10
10-3
[10 10-3.00-2..........
10-11
L2. regularization iigulariattoo_cccfffccnn coefficient
10
10-2
[10 10 1000:-01210000000 10000
input_lengln length
(5.
balch_size
1024
512
512
1024
128,256,512 1024]
embedding size
64
64
64
128
16,32,64,128
learning rale
10-3
3
10-3 3
10
[10
2000
Caver
L2 pregularizationnceffiiii
10
10
10
110-4 10-3, 10-10.0.0..001
input length
5, 30,
hatch_ste size
512
256
256
512
128, 228,256,512, 1024)
horizontal filter_num
16
32
4.8. 16.16,230 16,
vertical filter num
1.2.4.8.16]
embedding size
64
64
128
32,
GRU4Rec
learning_rai rate
10
10-2
10-2
10-2
[10-4, 10 10.1000000.0000000.. 10-11
1.2 pe__eellarvatiooccofffiie
10-2
10-2
10-2
10-2
[10-4 10-3, 10-2,
input_Jength
10
10
5,10, 20. 40,
batch size
1024
512
512
1024
256. 512, 1024
GRU unit numher
256
256
256
512
1024
embedding
128
64, 128)
SASRec
learning_rai rate
10-2
10
10
10
[10 [10-1 100 1010.000-..... 2010--1-11
1.2 eeguuaaiiationncccffiiii
10 2
10 2
10 2
10
10
5.10-2.10 10-11
siput_length
50
20
20
40
5. 10,00. 30. 50)
128
512
128, 512, 1024)
self attention head num
1,2.4.8) 1,2.3.4)
sclf attention block num
embedding size
64
64
64
128
16,32,64, 128]
BBRT4Rec
learning rate
4
10 4
2
3
[10 10000-10-2,,,00 10
L2 regulatizatiooncceffiicc
10
10
10
10-4 10-100-3 1.10-2 10-11
input length
5,10 20, 30,
hatch_sia size
256
256
512
1024
128,256.88 512,, 1024)
self aell_atttniion head__ea____ num
1. 1.2.3.41 1.2.3,4
self attention block__ooppa num
embedding
128
[16,3,,6,,
MAGNN
learning rate
10
10
10
10
[10 10-$,10-33 1000,10002
2,100-11000000 1.10--1
1.2 eeggulaizationnccfffiii
10
10
10
10
[10 100-3.10-3.00 1.110000.1-1111111 10-11
input_ lengt_
15,10, 40. 501
baich
1024
512
512
1024
128, 515,6,122,10
GNN layer num
1.2.3.4)
memory unit num
10
10
10
20
15, 15,
embedding size
64
64
64
128
16,32, 64,
learning rale
10-3
10-3
10-2
10-3
[10 10-$,10-3,100-
10-
BERD
L2 eerularization_ceffiiee
10
10
10
10-2
10-4 10 10 10 10
input Jength
5, 40,
hatch_size
1024
512
512
1024
128, 512, 1024)
celf_attention head
1. 1,2.3.41
attention block_ num
1.2.3,41
JJGCN_layer_mnn layer_num _um
2
1,2,3,4)
filterrraiio
0.05
0.05
0.05
0.10
0.05, 0.100,0..1,,000 0.20,,.20,0.255 0.255
sample size of Z.om
[1,2,3,4]
STEAM FMLP- Rec
embedding
64
64
64
128
[16. 128]
learning_rate rate
10
10
10
10
[10-4 10-3,10 10
1.2 pe_realarivaiionceeffiii
10
10
10-3
10-2
[10-4" 10-3 10-2 10-1
input_Jengh
50
20
20
40
5. 5,10, 40,
batch
1024
256
256
1024
128, 512, 1024] 1,2,3.4 16.32,64, 128]
learnable filter block filter_blokk
num
embedding
128
learning_ rate
10
10
10 2
10-3
[10-4 10 10-
1003,00- 10
1.2,
10-2
10-3
10-
10
[10-4 10-3 10 10
Jength
50
20
20
40
5,10 40, 50]
batch size
1024
256
256
1024
128,256, 1024
self attenttetion head num
2
2
1.2.3.4]
sell_atiention block_num
1, 2,
insertion probability deletion probability mask probability
0.2
0.4
0.4
0.2
0.1.022.3330.4400
0.1
0.1
0.1
0.1
0.1.0.2.03.0.4.00
0.4
0.5
0.5
0.3
embedding size
64
64
64
64
16,32,64, 128] [10
learning rate
3
3
10-11
BirDRec
L.2 regularization
irallaization_cccfffiienn coefficient
10
10
10
10
[10-4 10 10000001-1110000 10 15. 30.
lengh_engghhhh
butch
1024
1024
1024
1024
[128, 128, 1024 [10
threshold 9 in DRUT
10-1
10
10-1
110-4.10-3.10-2.1000
10
threshold 3' in DDUI size K of rectification pools
10-1
10-1
10
10
10-4:10-3200 100
10
10
10
1,,0
exponential decay rate
0.9
0.9
0.9
0.9
0.1.0.33.055.0...00..
13

---

References
[1] Di-Rong Chen and Tao Sun. Consistency of multiclass empirical risk minimization methods based convex loss. The Journal of Machine Learning Research, 7:2435-2447, 2006.
[2] Yatong Sun, Bin Wang, Zhu Sun, and Xiaochun Yang. Does every data instance matter? enhancing sequential recommendation by eliminating unreliable data. IJCAI, pages 1579- 1585, 2021.
[3] Songzhu Zheng, Pengxiang Wu, Aman Goswami, Mayank Goswami, Dimitris Metaxas, and Chao Chen. Error-bounded correction of noisy labels. In ICML, pages 11447-11457, 2020.
[4] Wang-Cheng Kang and Julian McAuley, Self-attentive sequential recommendation. In /CDM, pages 197-206, 2018.
[5] Wassily Hoeffding. Probability Inequalities for sums of Bounded Random Variables, pages 409 426. 1994.
[6] Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In AISTATS, pages 249-256, 2010.
[7] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
14


Reviews:
[{"review_id": "LNSe9ZC5Kx", "reviewer": "Reviewer_rz4h", "summary": "This paper focuses on the inconsistency between user’s true preference with interaction history. Specifically, the authors propose a theoretically guaranteed data rectification strategies based on SRS predictions to tackle both unreliable input and targets for more robust SRS. Besides, the authors also devise a rectification sampling strategy and adopt a self-ensemble mechanism to ensure better scalability. Experiments on four real-world datasets show the superiority of the proposed method.", "questions": "1)\tTheorem 4 indicates that the relative rank of an item over a list of randomly sampled items can approximate this item’s relative rank over the full item set. However, if K is too smaller than |V|, the randomly sampled items may still introduce so many noises?\n\n2)\tThe space complexity reduction of Self-ensemble Mechanism if from K*H to K, but not from |V|*H to K?", "limitations": "1)\tThe proposed model seems to be too complicated to be deployed in real industrial environments.\n\n2)\tBesides sequential recommendation, can this model employed in other recommendation tasks, for example, rating prediction, CTR prediction?", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1)\tThe research problem of rectification of interaction data is very interesting.\n\n2)\tTheorem analysis for the proposed modules. \n\n3)\tThe proposed model can be compatible with most existing models.\n\n4)\tExtensive experiments with respect to effectiveness and efficiency are conducted.", "weaknesses": "1)\tSome details are not clear, for example, what’s the formulation of the Self-ensemble Mechanism?\n\n2)\tThe time comparison is important is important for real application. However, Figure 3 compares SASRec, STEAM, and BirDRec simultaneously, which makes the increasing of time comparing SASRec and BirDRec not clear. More analysis is necessary. \n\n3)\tThe experimental datasets are all small datasets with less than 100K users and items. Besides, the improvements are too large which is not very common.\n\n4)\tNo online results are given. Too much hyper-parameters, which limit the applicability of the model.\n\n5)\tMany parameters are pre-defined, without careful tuning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your invaluable reviews.\n\n## Weaknesses\n\n> W1\n\nWe did have the formulation of the Self-ensemble mechanism (see lines 238-245).\n\n> W2\n\nThanks for bringing up the clarity issue with Fig 3(a). The table below shows the per-epoch time cost (seconds) of SASRec and BirDRec:\n\n| | ML-1M | Beauty | Yelp | QK-Video\n---|---|---|---|---\nSASRec | 14.1  | 5.0    | 5.7  | 44\nBirDRec | 28.5  | 7.9    | 7.5  | 74\n\nWe will clarify this by adding values for each bar in Fig 3(a) in the camera-ready.\n\n> W3\n\nThe dataset selection exactly follows the common practice in SRSs. They are representative benchmark datasets and have been widely adopted by SOTAs [1, 2, 3] in SRSs. Although the numbers of users and items are less than 100K, the total interactions are in a large scale, e.g., QK-Video has larger than 2M interactions. \n\n**BirDRec vs. non-robust SRSs.** It is common for robust SRSs to achieve large improvements over non-robust SRSs due to the negative effects of unreliable data in misleading SRSs. The table below shows the average improvements of BirDRec and the best robust SRS over six non-robust SRSs. Both methods achieve significant improvements.\n\n| |   FPMC  |  Caser  | GRU4Rec | SASRec | BERT4Rec |   MAGNN\n---|---|---|---|---|---|---\nBest Robust SRS |   31.13%  |32.72%  |42.52% | 17.73%  | 21.81%  | 10.45%\nBirDRec | 38.91% | 40.65% | 64.81% | 49.27% | 51.06% | 25.94%\n\n**BirDRec vs. Robust SRSs (see Tab 3).** The relative improvement achieved by the best SOTA robust SRS against other robust SRSs is 23.45% on HR. BirDRec wins the best SOTA robust SRS by a lift of 25.68%. The reasons are threefold. BirDRec not only detects but also rectifies unreliable data; BirDRec handles both unreliable input and targets; and BirDRec is error-bounded.\n\n\n> W4\n\nOnline evaluation for SRSs is challenging due to privacy and ethical concerns w.r.t. accessing real-world platforms and user interaction data. We follow SRS SOTAs [1, 2, 3] to report offline results on public datasets.\n\nBeyond shared hyper-parameters (e.g., embedding size, batch size), BirDRec only has **4 additional** hyper-parameters (see Fig 5). SOTA robust SRSs [1, 2] normally have **5 additional** hyper-parameters. \n\n> W5\n\nWe have carefully tuned hyper-parameters of each method with the grid search (see Fig 5 in the main paper and Tab 1 in Appendix). To improve efficiency, we accelerated this process with two strategies following SOTAs [4, 5]. **First**, we adopt the early stop mechanism [4] to terminate the model training if the performance does not increase in 20 consecutive epochs. **Second**, we split the hyper-parameters of each method into 3 independent groups: learning rate-related ones, model capacity-related ones, and the others. Then we conduct the grid search on each group separately. The optimal setting of each group forms the final optimal setting [5].\n\n## Questions\n\n> Q1\n\nTheoretically, based on Theorem 4, larger $K$ yields a more accurate estimation of the true rank. If $K$ is too smaller than $|V|$, the upper bound of Theorem 4 might be high, thus introducing noise. Empirically, we found that $K\\ge 10$ is sufficient to reach promising accuracy on all datasets. The ratio of $K$ to $|V|$ is from 1/300 to 1/4000 across different datasets. The possible reasons are twofold. \n- Each rectification pool is initialized properly with an instance's succeeding or preceding items, which are potentially better substitutions. \n- Each rectification pool is progressively improved during training as it is continuously updated by adding newly sampled high-scored items.\n\n> Q2\n\nThe reduction is from $|V|*H$ to $K$ (see lines 234-251). **Without Self-ensemble**, each instance has to maintain prediction scores with all $|V|$ items in all $H$ previous epochs to trace back historical scores and calculate the weighted average scores. As we cannot forecast which item will be added to the rectification pool. **With Self-ensemble**, the weighted average scores of $K$ items in a rectification pool can be directly approximated by a self-ensembled model. \n\n## Limitations\n\n> L1\n\nBased on Fig 3, BirDRec is much more efficient than the latest SOTA Robust SRS STEAM, and only 1.6 times slower than its backbone. Besides, BirDRec is particularly beneficial for datasets with large item sets (see lines 246-249) thus being applicable for real environments.\n\n\n> L2\n\nBirDRec can be adapted to other recommendation tasks, as long as timestamps of user-item interactions are available. Thus, the problem can be first formulated as a sequential recommendation task. Then, the expected output can be generated by properly adding extra neural layers.\n\nFor rating prediction, the output of sequential recommendation (i.e., prediction scores for all candidate items) could be first mapped into $[0, 1]$ by a softmax layer, and then re-scaled based on the rating range. For CTR prediction, each CTR can be computed by a softmax layer over the output of sequential recommendation. \n\nReferences\n\n[1] Sun et al. Enhancing sequential recommendation by eliminating unreliable data. In IJCAI 2021.\n\n[2] Lin et al. A self-correcting sequential recommender. In TheWebConf 2023.\n\n[3] Qiu et al. Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation. In WSDM 2022.\n\n[4] Goodfellow et al. Deep Learning. 2016.\n\n[5] Yoshua Bengio. Practical Recommendations for Gradient-Based Training of Deep Architectures. Neural Networks: Tricks of the Trade. 2012.\n"}, {"review_id": "2phnvLw9Ol", "reviewer": "Reviewer_E67v", "summary": "The authors proposed Bidirectional Data Rectification (BirDRec) framework that can be incorporated into existing Sequential recommender systems (SRSs) to tackle unreliable data. In addition, the authors also adopt rectification sampling strategy.\n\nThe authors also provided theoretical guaranteed for BirDRec and conducted various experiments on the robustness of the framework.\n", "questions": "Please refer to my comments above", "limitations": "N/A", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Strengths:\n\n-\tThe idea is straightforward and easy to follow\n-\tThe authors provided theoretical guaranteed for BirDRec and conducted various experiments on the robustness of the framework.\n-\tTheoretical aspects are well-written\n-\tThe authors also provided source code for verification.\n", "weaknesses": "Weaknesses:\n\n-\tSome of the experimental results are not convincing such as the performance of baselines in Table 3. The authors need to provide more explanation and clarification such as why BERD is good for Movielens1M but STEAM is better for Yelp.\n-\tIt’s not clear which parts of the architecture in Figure 2 made the performance increases dramatically\n-\tComparing BirDRec and SASRec for hyper-parameters analysis in Appendix C.2 and C.3 seems a bit ‘unfair’. In my opinion, we should compare with most recent baselines, so at least we can compare with Figure 5 in the main paper.\n-\tHow much time did it take to do grid search across models in Table 1 from Appendix Page 13?\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "\nThank you for your invaluable comments.\n\n## Weaknesses\n\n> W1\n\nThe possible reasons are twofold: the item-wise correction process of STEAM, and the longer interaction sequences on ML-1M (see Tab 1). \n\nTo correct each instance, STEAM selects one type of operation from ‘keep’, ‘delete’, and ‘insert’ for **each item** in the instance. Such correction is more likely to make mistakes on long sequences as one single wrong operation will render the entire sequence unreliable for training. By contrast, BERD ONLY cares whether the **target item** is matched with the input items given an instance, so its performance is insensitive to the length of the sequence.\n\nBesides, the mistakes made by STEAM can be more serious than BERD, as STEAM can modify reliable instances into unreliable ones by deleting relevant items or inserting irrelevant items. In contrast, BERD simply abandons some instances that might be unreliable, but would not generate new unreliable instances. Such negative impacts will be amplified on datasets with long sequences where STEAM is more inclined to make mistakes.\n\n\n> W2\n\nBased on our ablation study (see lines 296-305 and Fig 4), the performance gain of separately adding DRUT, DDUI, and Self-ensemble is 45.7%, 43.3%, and 3.1%, respectively. Thus the ranking of their importance should be DRUT, DDUI, and Self-ensemble. We will complement the ablation study by adding this in the camera-ready.\n\n\n> W3\n\nAll the figures in C.2 and C.3 show the performance comparison with SASRec itself and our BirDRec with SASRec as backbone. These figures mainly aim to investigate the effects of different hyper-parameters on BirDRec, so as to help determine the best setting for the hyper-parameters. Hence, the comparison is fair.\n\nFor a more comprehensive visualization, we have added the three recent baselines (BERD: IJCAI-21, FMLP-Rec: TheWebConf-22, STEAM: TheWebConf-23) into C.2 and C.3. Note that their performance is already available in Tab 3, and their per-epoch accuracy is available in the training logs. The results show that BirDRec still performs the best.\n\n> W4\n\nIt takes around three months to complete the grid search in Tab 1 in Appendix. This aims to help find out optimal hyper-parameter settings for all methods, thus delivering a fair and rigorous comparison. To improve efficiency, we accelerated this process with two strategies following SOTAs [1, 2]. \n\n**First**, we adopt the early stop mechanism [1] to terminate the model training if the performance does not increase in 20 consecutive epochs. **Second**, we split the hyper-parameters of each method into three independent groups, including learning rate-related ones (e.g. learning rate, batch size), model capacity-related ones (e.g. embedding size, number of attention heads), and the others. Then we conduct the grid search on each group separately [2], and the hyper-parameters not in this group are set as suggested in their original papers. Ultimately, the optimal setting of each group together forms the final optimal settings of a specific method.\n\n\nReferences\n\n[1] Goodfellow et al. Deep Learning. 2016.\n\n[2] Yoshua Bengio. Practical Recommendations for Gradient-Based Training of Deep Architectures. Neural Networks: Tricks of the Trade. 2012.\n"}, {"review_id": "nrLzzu76jp", "reviewer": "Reviewer_1jEu", "summary": "This paper proposes a bidirectional data rectification framework, called BirDRec, to address the challenges of training sequential recommender systems (SRSs) with unreliable input and targets. The authors provide two theoretically guaranteed error-bounded strategies to rectify the data, which can be flexibly implemented with most existing SRSs. They also introduce a rectification sampling strategy and self-ensemble mechanism to reduce the computational and space complexity of BirDRec. The proposed framework is evaluated on several benchmark datasets, and the results demonstrate its generality, effectiveness, and efficiency compared to state-of-the-art robust SRSs. ", "questions": "1. Could you provide additional details on the theoretical guarantees of the error-bounded strategies proposed for data rectification? \n2. Can you explain the process behind selecting the benchmark datasets used in your experiments? What criteria did you consider when choosing these datasets? Additionally, could you elaborate on the evaluation metrics or performance measures used to assess the effectiveness of the proposed framework on these benchmarks?\n3. Have you taken into account other sources of noise or errors commonly encountered in SRSs, such as cold-start, diversity, or fairness issues? If so, how does your approach address or mitigate these challenges? \n4. In real-world applications, how do you envision the proposed framework being utilized? Are there specific domains or use cases where you believe BirDRec would be particularly beneficial? Additionally, what are some potential challenges or limitations that might hinder the adoption or scalability of your framework in practical settings? It would be valuable to understand the feasibility and potential constraints of implementing BirDRec in real-world scenarios.", "limitations": "The authors missed an opportunity to compare their approach with other recent methods that address the problem of unreliable data in SRSs. By including such a comparison, the authors could provide a more comprehensive evaluation of their proposed framework and highlight its advantages or limitations in relation to existing solutions. This addition would enhance the paper's contribution to the field. ", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper introduces BirDRec, a novel bidirectional data rectification framework designed to enhance the robustness of sequential recommender systems (SRSs) by addressing unreliable data. BirDRec can seamlessly integrate with existing SRSs, offering a flexible solution for improving their performance.\n2. The authors present two error-bounded strategies within BirDRec that provide theoretical guarantees for detecting and rectifying unreliable input and targets in SRSs. These strategies contribute to the reliability and accuracy of the recommendations generated by the system.\n", "weaknesses": "1. The evaluation of the proposed framework is limited to a few benchmark datasets, and it remains uncertain how well it would generalize to different types of data or domains. It would be beneficial for the authors to include further experiments or discuss potential challenges in applying their framework to diverse datasets.\n2. While the authors attribute the unreliable data to external distractions, it is important to acknowledge that there might be other sources of noise or errors that their approach does not account for. Including a discussion on potential alternative causes of unreliable data would strengthen the paper's analysis.\n3. The paper lacks a thorough analysis of the computational and space complexity of the proposed framework. Understanding the resource requirements of the framework is crucial for assessing its scalability to larger datasets or real-time applications. It would be valuable for the authors to provide insights into the framework's efficiency and discuss any potential limitations in terms of computational resources.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "\nThanks for your reviews.\n\n> Weakness 1\n\nWe exactly follow the scientific research standards and common practice in SRSs, that is, selecting (around 3~6) representative benchmark datasets that are widely adopted by SOTAs [1, 2, 4] for evaluation. We have tried our best to cover various domains ranging from movie (ML-1M), video (QK-Video), to E-commerce (Amazon-Beauty) and location-based social networks (Yelp).\n\nDue to space limitation, we believe no one and no method can enumerate all datasets for evaluation. Thus the generalizability of SOTAs cannot be fully guaranteed on datasets that have not been tested.\n\n> Weakness 2 & Question 3\n\nOur solution is indifferent to the cause of unreliable instances. Regardless of the reasons causing an unreliable instance, it will involve either unreliable input or unreliable target, or both. All of these can be handled by BirDRec.\n\nWe acknowledge that analyzing the causes of unreliable data might help devise solutions. However, investigating why a user interacts with irrelevant items is a challenging psychological problem [3], which is beyond the scope of this paper. We may leave it as our future work.\n\n> Weakness 3\n\nThe time and space complexity of our proposed framework are respectively analyzed in Sections 4.1 and 4.2.\n\nThe main limitations of BirDRec are twofold, which will be highlighted in our camera-ready.\n\n- **Limited improvements on storage cost for smaller item sets:** As mentioned in lines 293-295, we acknowledge that the storage cost of BirDRec is marginally higher than STEAM's on ML-1M. This is primarily because the rectification sampling strategy with Self-ensemble has a less apparent advantage on datasets with smaller item sets (see lines 246-249). To be specific, the storage cost reduction for calculating weighted average prediction scores is from $O(|V|*H)$ to $O(K)$ for each instance. Thus if $|V|$ is small, the benefit of this reduction will be less obvious.\n- **Marginally higher computational cost than backbone:** Although BirDRec is significantly faster than the latest robust SRS (STEAM), it is worth noting that BirDRec is 1.6 times on average slower than its backbone model SASRec (as depicted in Fig 3) in each training epoch. This increased training time could be a practical concern in systems with **extremely large-scale** datasets and real-time recommendation demands.\n\n> Question 1\n\nThe details and proofs of all the lemmas and theorems can be found in Appendix.\n\n> Question 2\n\n**For datasets**, we exactly follow the scientific research standards and common practice in SRSs, that is, selecting (around 3~6) representative benchmark datasets that are widely adopted by SOTAs [1, 2, 4, 5] for evaluation. Typically, they are selected by considering the following criteria: domains, sizes, sparsity levels, and average sequence lengths as shown in Tab 1.\n\n**For metrics**, these evaluation metrics are selected by following SOTAs in SRSs [1, 2, 4]. Higher metric values indicate better ranking performance. We can add the following detailed explanations in Appendix.\n\nSuppose there are $M$ users, and $R_{u}$ is the full recommendation list (RL) for user $u$.  Let $R_{u}[j]$ be the j-th item in $R_{u}$, and $R_{u}[1:N]$ be the Top-$N$ RL. $i_{u}^{t}$ is $u$'s intereacted item in the test set. $I(x)$ is an indicator function whose value is $1$ when $x > 0$, and $0$ otherwise. The formal definitions of HR, NDCG, and MRR are as follows.\n\n**HR (Hit Ratio)** gives the percentage of users that can receive at least one correct recommendation from the Top-N RL:\n\n$$HR@N=\\frac{1}{M}\\sum_{u=1}^{M}I(|R_{u}[1:N]\\cap\\{i_{u}^{t}\\}|).$$\n\n**NDCG (Normalized Discounted Cumulative Gain)** evaluates the ranking performance by measuring the positions of correct recommended items:\n\n$$NDCG@N=\\frac{1}{M}\\sum_{u=1}^{M}\\frac{1}{Z}DCG@N=\\frac{1}{M}\\sum_{u=1}^{M}\\frac{1}{Z}\\sum_{j=1}^{N}\\frac{2^{I(|\\{R_{u}[j]\\}\\cap \\{i_{u}^{t}\\}|)}-1}{\\log_2(j+1)},$$\n\nwhere Z, as a normalization constant, is the maximum possible value of DCG@N.\n\n**MRR (Mean Reciprocal Rank)** also evaluates the ranking performance via the rank position of the correct recommended item $i_{u}^{t}$ (denoted by $rank_{i_{u}^{t}}$) in the RL:\n\n$$MRR=\\frac{1}{M}\\sum_{u=1}^{M}\\frac{1}{rank_{i_{u}^{t}}}.$$\n\n> Question 4\n\nBirDRec would be particularly beneficial for domains where users' behaviors can be occasionally influenced by external distractions. For example, in movie domain, users may receive recommendations from friends once in a while. Meanwhile, it also benefits from datasets with large item sets, leading to significant improvements in storage cost compared with the latest robust SRS STEAM [2].\n\n> Limitation 1\n\nWe did compare our model with 3 recent SOTA robust SRSs, i.e., BERD [1] (IJCAI-21), FMLP-Rec [4] (TheWebConf-22), and STEAM [2] (TheWebConf-23). They aim to tackle unreliable targets, unreliable input, and both, respectively. The results are in Tab 3.\n\n\nReferences\n\n[1] Sun et al. Enhancing sequential recommendation by eliminating unreliable data. In IJCAI 2021.\n\n[2] Lin et al. A self-correcting sequential recommender. In TheWebConf 2023.\n\n[3] Cosley et al. Is seeing believing? How recommender system interfaces affect users’ opinions. In SIGCHI 2003. \n\n[4] Zhou et al. Filter-enhanced mlp is all you need for sequential recommendation. In TheWebConf 2022.\n\n[5] Yuan et al. Tenrec: A Large-scale Multipurpose Benchmark Dataset for Recommender Systems. In NeurIPS 2022.\n"}, {"review_id": "AukGJT2zmM", "reviewer": "Reviewer_cUYC", "summary": "This paper aims to address the issue where the performance of sequential recommender systems (SRSs) could be harmed by mismatched input-target pairs introduced by distracted users. The paper claims that existing studies only focus on either unreliable inputs or targets but not both simultaneously. Also no theoretical guarantee of effectiveness for the existing methods are provided.  To address these issues, the paper proposes to replace/remove target/input item using a forward/backward SRS's weighted average scores accumulated over training epochs. Error bounds for both forward and backward rectifications are provided. To further reduce the time and space complexity, sampling strategy as well as self-ensembled mechanism are applied. Experiments show the proposed approach is able to improve the performance is representative SBSs on four datasets. Comparing to rectified/denoised SBSs, the performance of the proposed approach is also reported to be better. Ablation test as well as hyperparameter analysis are also provided. ", "questions": "Please see the suggestions in the \"weakness\" section.", "limitations": "No limitation is mentioned in the paper.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The ideas of a. rectifying input and target items using weighted average prediction score of forward and backward SBR, b. using sampling and self-ensembled mechanism of improving efficiency are interesting and new in my opinion.\n* The main steps leading to the error bounds are reasonable, though I did not manage to check the proofs in the supplementary in details.\n* The empirical studies that support the effectiveness of the propose approach appeals to be convincing. ", "weaknesses": "As I am not familiar with the literature of the robustness of SRS nor the rectification / denoising in SRS, I am only able to give a few suggestions:\n* Analyzing the existing work as well as the proposed approach on a synthetic dataset with ground truth could further strengthen the claims about: a. the drawbacks of the existing studies, b. the effectiveness of the proposed approach.  Conduct simulation on the synthetic dataset could validate the error bound, as well as evaluate the approximation quality of the proposed method.\n* Qualitatively evaluating the rectified items for a few instances in different dataset would further help the audience understanding the behavior of the proposed approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "\nWe appreciate your recognition of our work and your valuable feedback.\n\n\n> Weakness 1\n\n\nThank you for the interesting and insightful suggestions. \nConstructing a synthetic dataset with ground truth for data reliability is indeed of importance for robust SRSs. However, the major challenge is how to obtain ground truth annotations. There are **NO definitive ground truth rules** that can be used to accurately identify the reliability of each instance due to the inherent ambiguity of user interactions. That is, the reason behind each interaction may not be apparent even to the users themselves, as noted in existing research [1]. Thus the annotation can not be done directly via either machine or crowdsourcing or domain experts.\n\nTo this end, one possible solution is to design our own rules that are as close to the ground truth rules as possible for determining data reliability. However, we have to overcome key concerns including but not limited to the following ones:\n\n- **C1: How to guarantee the validity of the designed rules?** For example, we may design a rule: for users who are huge fans of Sci-Fi movies, if any instance of these users involves films of other genres, then the instance is unreliable. However, we may not be able to guarantee the generalizability of this rule to all these users without exceptions in the real world. So, it is challenging to quantitatively measure to what extent each rule is valid, leading to concerns about the validity of the synthetic data.\n- **C2: How to guarantee the completeness of the designed rules?** Considering the diverse user behavior patterns, it is impractical to design rules to exactly cover all the possible behavior patterns of all users in a specific domain. In this case, the synthetic data may either oversimplify or complicate the recommendation task, rendering the performance of SRSs on such datasets less meaningful.\n\nIn conclusion, we acknowledge the invaluableness of the suggestions as well as the challenges involved. We will strive to make progress in this direction as part of our future research.\n\n\n> Weakness 2\n\n\nThank you for the invaluable comments, which allow an intuitive and detailed analysis of the proposed method. We will randomly sample some unreliable instances on different datasets and visualize the rectification process and results. These will be added in the camera-ready.\n\nReference\n\n[1] Cosley et al. Is seeing believing? How recommender system interfaces affect users’ opinions. In SIGCHI 2003. \n"}, {"review_id": "M63FDSSc80", "reviewer": "Reviewer_Uqk2", "summary": "Overall comment:\nThis paper proposes two detect and rectify methods to solve the problem of unreliable instances in Sequential Recommender Systems (SRSs), including unreliable items in the input and unreliable targets. The two methods can be applied to existing SRSs models, effectively improving the model performance.\n\nStrengths of the paper include:\n\n1. The paper provides sufficient theoretical proof for the methods, which is very impressive.\n2. The discussion and clarification of related work, as well as the differences and connections with existing work, are very clear.\n3. The experimental results are sufficient, and the original model performance is improved on multiple baseline models via proposed method. The ablation study also compares the performance of the DRUT, DDUI, their combination, and whether to use self-ensemble under different conditions.\n\nWeakness of the article includes:\n\nThe paper does not discuss the limitations of the methods. In particular, I am interested in whether this method can be applied in large-scale real-world recommender systems? Are there any practical issues?", "questions": " I am interested in whether this method can be applied in large-scale real-world recommender systems? Are there any practical issues?", "limitations": "As I written in weakness:\n\nThe paper does not discuss the limitations of the methods. In particular, I am interested in whether this method can be applied in large-scale real-world recommender systems? Are there any practical issues?", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "Strengths of the paper include:\n\n1. The paper provides sufficient theoretical proof for the methods, which is very impressive.\n2. The discussion and clarification of related work, as well as the differences and connections with existing work, are very clear.\n3. The experimental results are sufficient, and the original model performance is improved on multiple baseline models via proposed method. The ablation study also compares the performance of the DRUT, DDUI, their combination, and whether to use self-ensemble under different conditions.", "weaknesses": "Weakness of the article includes:\n\nThe paper does not discuss the limitations of the methods. In particular, I am interested in whether this method can be applied in large-scale real-world recommender systems? Are there any practical issues?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "\nWe sincerely appreciate your appreciation on our work and your valuable reviews.\n\n> Question and Limitation: the paper does not discuss the limitations of the methods. In particular, I am interested in whether this method can be applied in large-scale real-world recommender systems. Are there any practical issues?\n\nThank you for highlighting the need to discuss the limitations and practical considerations of our proposed BirDRec framework. The main limitations are twofold, which will be highlighted in our camera-ready.\n\n\n- **Limited improvements on storage cost for smaller item sets:** As mentioned in lines 293-295, we acknowledge that the storage cost of BirDRec is marginally higher than STEAM's on ML-1M. This is primarily because the rectification sampling strategy with Self-ensemble has a less apparent advantage on datasets with smaller item sets (see lines 246-249). To be specific, the storage cost reduction for calculating weighted average prediction scores is from $O(|V|*H)$ to $O(K)$ for each instance. Thus if $|V|$ is small, the benefit of this reduction will be less obvious.\n- **Marginally higher computational cost than the backbone:** Although BirDRec is significantly faster than the latest robust SRS (STEAM), it is worth noting that BirDRec is 1.6 times on average slower than its backbone model SASRec (as depicted in Fig 3) in each training epoch. This increased training time could be a practical concern in systems with **extremely large-scale** datasets and real-time recommendation demands.\n\n"}]

Author rebuttal:

Meta-review: All five reviewers unanimously recommend acceptance (ranging from borderline accept to strong accept).  Reviewers request that the explanations and clarifications from the rebuttal should be integrated into the final paper version (or supplementary material).
Decision: Accept (poster)