Title: Theoretically Guaranteed Bidirectional Data Rectification for Robust Sequential Recommendation
Venue: NeurIPS 2023
Accepted: yes
Keywords: recommender systems

sequential recommendation
Abstract: Sequential recommender systems (SRSs) are typically trained to predict the next item as the target given its preceding (and succeeding) items as the input. Such a paradigm assumes that every input-target pair is reliable for training. However, users can be induced to click on items that are inconsistent with their true preferences, resulting in unreliable instances, i.e., mismatched input-target pairs. Current studies on mitigating this issue suffer from two limitations: (i) they discriminate instance reliability according to models trained with unreliable data, yet without theoretical guarantees that such a seemingly contradictory solution can be effective; and (ii) most methods can only tackle either unreliable input or targets but fail to handle both simultaneously. To fill the gap, we theoretically unveil the relationship between SRS predictions and instance reliability, whereby two error-bounded strategies are proposed to rectify unreliable targets and input, respectively. On this basis, we devise a model-agnostic Bidirectional Data Rectification (BirDRec) framework, which can be flexibly implemented with most existing SRSs for robust training against unreliable data. Additionally, a rectification sampling strategy is devised and a self-ensemble mechanism is adopted to reduce the (time and space) complexity of BirDRec. Extensive experiments on four real-world datasets verify the generality, effectiveness, and efficiency of our proposed BirDRec.

Theoretically Guaranteed Bidirectional Data Rectification for Robust Sequential Recommendation

Abstract

Sequential recommender systems (SRSs) are typically trained to predict the next item as the target given its preceding (and succeeding) items as the input . Such a paradigm assumes that every input-target pair is reliable for training. However, users can be induced to click on items that are inconsistent with their true preferences, resulting in unreliable instances, i.e., mismatched input-target pairs. Current studies on mitigating this issue suffer from two limitations: (i) they discriminate instance reliability according to models trained with unreliable data, yet without theoretical guarantees that such a seemingly contradictory solution can be effective; and (ii) most methods can only tackle either unreliable input or targets but fail to handle both simultaneously. To fill the gap, we theoretically unveil the relationship between SRS predictions and instance reliability, whereby two error-bounded strategies are proposed to rectify unreliable targets and input, respectively. On this basis, we devise a model-agnostic Bi di r ectional D ata Rec tification ( BirDRec ) framework, which can be flexibly implemented with most existing SRSs for robust training against unreliable data. Additionally, a rectification sampling strategy is devised and a self-ensemble mechanism is adopted to reduce the (time and space) complexity of BirDRec. Extensive experiments on four real-world datasets verify the generality, effectiveness, and efficiency of our proposed BirDRec.

1 Introduction

Recently, the study on sequential recommender systems (SRSs) [1, 2, 3, 4, 5] has garnered much attention as users' preferences are inherently dynamic and evolving in real-world scenarios. The goal of SRSs is learning to predict the next item a user interacts with given the preceding (and succeeding) items. Therefore, a training instance for SRSs is typically composed of an input item sequence and its next item as the target . However, distractions in daily lives (e.g. recommendations from friends, account sharing, and accidental clicks) can induce users to click on items that are inconsistent with their true preferences, resulting in unreliable training instances with mismatched input-target pairs. The mismatch can be categorized into Complete Mismatch and Partial Mismatch when the item caused by distractions acts as an unreliable target and unreliable input of an instance, respectively. To illustrate, the romantic film 'La La Land', in the first instance of Figure 1, serves as an unreliable target, which is recommended by friends and completely mismatched with the previous superhero movies. By contrast, in the second instance, 'La La Land' acts as an unreliable input item which renders the input sequence partially mismatched with the target superhero film. Both

Instance1CompleteMismatch

Instance2PartialMismatch

SUPERMAN

LEURR

IAIAIANI

Friends'

DRON2MAN

Recommendation

IRONMAN

THON MINS

External

Input

Target

Distractions

Input

Target

types of mismatch would cause unreliable training instances that mislead SRSs to learn sequential relationships between irrelevant items and eventually undermine the recommendation accuracy.

Although there are a number of studies aiming to combat such unreliable data for more robust SRSs, they suffer from two core limitations. (i) They discriminate instance reliability based on the intermediate or final output of a model (either the SRS itself [6, 7, 8, 9] or an additional corrector [10]) that is trained with unreliable data. However, there is no theoretical guarantee that such seemingly contradictory solutions can be trustworthy for detecting and correcting unreliable instances. (ii) Most prior studies only focus on tackling either unreliable input [6, 7, 11, 12, 13] or targets [8], but fail to handle both simultaneously. Only one recently proposed method [10] attempts to address this issue, but it relies on a corrector trained with unreliable data, yet without any theoretical guarantees.

As such, we, for the first time , theoretically unveil the relationship between the SRS predictions and instance reliability, proving that a target with consistently low prediction scores is unlikely to be reliable, assuming that the randomness of user behavior is limited. It then inspires us to devise two error-bounded rectification strategies to (1) detect consistently low-scored targets (i.e., unreliable targets) and replace them with steadily high-scored items and (2) detect and delete consistently low-scored items within the input (i.e., unreliable input), where the score is estimated by a backward SRS. Note that the unreliable input items, as the interruptions in the input, are not replaced but directly removed to bridge the preceding and succeeding items. Based on these strategies, we propose a model-agnostic Bi di r ectional D ata Rec tification ( BirDRec ) framework which contains two SRSs in opposite directions for tackling both unreliable targets and input. In addition, to reduce the time complexity, a rectification sampling strategy is devised to efficiently obtain consistently high-scored items; to reduce space complexity, a self-ensemble mechanism [14] is adopted to approximate the weighted average prediction scores across different training epochs.

Contributions. (1) We are the first to provide theoretically guaranteed data rectification strategies based on SRS predictions to tackle both unreliable input and targets for more robust SRSs. (2) We devise a model-agnostic bidirectional data rectification framework that can be flexibly implemented with most existing SRSs for robust training against unreliable data. (3) We devise a rectification sampling strategy and adopt a self-ensemble mechanism to ensure better scalability of BirDRec. (4) Extensive experiments with SRSs based on representative backbones and datasets across various domains validate the generality, effectiveness, and efficiency of BirDRec.

2 Related Works

Early SRSs [15, 16] adopt Markov Chains to encode users' interaction sequences, assuming users' latest interactions affect the future behavior linearly and independently. Later, powerful deep learning backbones such as recurrent neural networks (RNNs) [17, 18, 19], convolution neural networks (CNNs) [20, 21, 22], graph neural networks (GNNs) [23, 24, 25, 26, 27], and Transformers [28, 29] are employed to extract complex non-linear patterns within users' sequences [30]. They posit each training instance is a definitely matched input-target pair and thus cannot handle unreliable data.

To resist unreliable data, existing robust SRSs can be categorized into three types. The first type focuses on handling the complete mismatch by identifying and eliminating instances with unreliable targets. For example, BERD [8] empirically finds that instances with high training loss and low uncertainty tend to have unreliable targets. This idea is relevant to studies on clean label selection [31, 32, 33, 34, 35] and label correction [36, 37, 38]. The second type concentrates on addressing partial mismatch by reducing the importance of unreliable input when formulating users' dynamic preference representations. Accordingly, various advanced mechanisms are integrated into SRSs, such as memory networks [7], gating networks [39, 40, 41], autoencoders [12], reinforcement learning [6],

uncertainty modelling [11, 42], and Fast Fourier Transform [9]. To the best of our knowledge, there is only one recently proposed method STEAM [10] falling into the third type which attempts to tackle both unreliable targets and input with an additional corrector producing reliable data. Nonetheless, existing robust SRSs all rely on models (either SRSs or additional correctors) trained with unreliable data, yet without theoretical proof that such seemingly contradictory solutions can be effective.

3 Theoretical Guarantees for Rectifying Unreliable Data

This section presents theoretical guarantees for rectifying unreliable data via SRS predictions for more robust SRSs. In particular, Section 3.1 introduces important preliminaries. Subsequently, Section 3.2 unveils the relationship between the prediction score of SRSs and the reliability of a target, inspiring us to propose an error-bounded strategy for handling unreliable targets; and Section 3.3 provides the error-bounded strategy for dealing with unreliable input.

3.1 Preliminaries

Problem Statement of SRSs . Let U and V be the sets of users and items, respectively. Each user u ∈ U chronologically interacts with a sequence of items s u = [˜ v u 1 , ˜ v u 2 , ..., ˜ v u | s u | ] , where ˜ v u t ∈ V is the t -th item user u interacts with and | s u | is the length of sequence s u . The goal of SRSs is to predict the target item ˜ v u t given the input ˜ x u t = { u, [˜ v u t -L , ..., ˜ v u t -2 , ˜ v u t -1 ] } , where L is the length of ˜ x u t . Thus, the training instance of SRSs can be represented as an input-target pair ⟨ ˜ x u t , ˜ v u t ⟩ . Note that we use '~' to denote the observed data that may be unreliable due to external distractions.

Core Assumptions . Ideally, each user-item interaction should be drawn from users' true preference distribution η without any distractions, where η v i ( x u t ) = P ( v u t = v i | x u t ) . We define p 1 to be the true item for recommendation, i.e, the top1 item according to η ,

where I [ · ] is an indicator function that equals 1 if the condition is true; otherwise 0. Meanwhile, we define p 2 to be the middle-ranked item (ranked ⌊|V| / 2 ⌋ -th) according to η , namely,

In general, SRSs are built upon the hypothesis that users usually select items with a tendency rather than randomly. In other words, the randomness of users' true preferences is restricted, i.e., the probability gap between the top1 and middle-ranked items regarding η is unlikely to be small. This assumption can be formally defined as follows.

Assumption 1. The users' true preference distribution η fulfills the relaxed Multiclass Tsybakov Condition [43] with constants C > 0 , λ > 0 , and α 0 ∈ (0 , 1] , such that for all α ∈ (0 , α 0 ] ,

The feasibility of Assumption 1 relies on small C and large λ , which are satisfied on public (observed) datasets based on our empirical analysis in the Appendix with C ∈ (0 . 55 , 0 . 70) and λ ∈ (1 . 37 , 4 . 01) .

Connecting η with SRS Predictions . Obviously, η is the ideal corrector to rectify unreliable data, however, due to its unavailability, many existing methods [8, 10] leverage SRS predictions as the substitution with no theoretical guarantees. This urges us to explore the connection between η and SRS predictions. To achieve this goal, we first investigate the relationship between η and users' observed preference distribution ˜ η , since SRSs are trained with the observed data that may be distorted by external distractions. Formally, ˜ η v i (˜ x u t ) = P (˜ v u t = v i | ˜ x u t ) , where ˜ v u t is the observed target that may be unreliable. We then define a transition probability τ v j v i (˜ x u t ) = P (˜ v u t = v i | v u t = v j , ˜ x u t ) as the chance that a true target v u t is flipped from item v j to item v i owing to external distractions. Thus, for any pair ( v i , v j ) ∈ V , there is a linear relationship between η and ˜ η :

To bridge η and SRS predictions via ˜ η , we then study the relationship between ˜ η and SRS predictions. Let f h be an SRS at the h -th training epoch, and the prediction of f h be ϵ -close to ˜ η ,

where f h v i (˜ x u t ) is the predicted probability (score) of the target being v i given the input ˜ x u t at the h -th training epoch. Eqs. (4-5) indicate that there is indeed a connection between η and SRS predictions, laying the foundation for the proposed strategies to rectify unreliable data as what follows.

3.2 Theorems for Rectifying Unreliable Targets

We now explore how to properly use SRS predictions for rectifying unreliable targets. Prior works empirically find that unreliable targets tend to possess consistently low prediction scores at different epochs [8, 44]. Yet, there is no guarantee that such prediction scores given by an SRS trained with unreliable data can be trustworthy for detecting unreliable targets. This prompts us to theoretically unveil the relationship between target reliability and SRS predictions at different epochs. Specifically, Theorem 1 proves that a reliable target is unlikely to keep low prediction scores during training.

̸

Theorem 1. Given Assumption 1, let { w h ∣ ∣ 1 ≤ h ≤ H, 0 ≤ w h ≤ 1 , ∑ H h =1 w h = 1 } be the weights 1 for averaging prediction scores of different epochs. ∀⟨ ˜ x u t , ˜ v u t ⟩ , assume ϵ ≤ α 0 τ ˜ v u t ˜ v u t (˜ x u t ) . Let γ = τ ˜ v u t ˜ v u t (˜ x u t ) η p 2 (˜ x u t )+ ∑ v j =˜ v u t τ v j ˜ v u t (˜ x u t ) η v j (˜ x u t ) . We have: P [ p 1 = ˜ v u t , ∑ H h =1 [ w h f h ˜ v u t (˜ x u t ) ] ≤ γ ] ≤ C ( O ( ϵ )) λ . Proof.

̸

̸

̸

By replacing γ with τ ˜ v u t ˜ v u t (˜ x u t ) η p 2 (˜ x u t ) + ∑ v j =˜ v u t τ v j ˜ v u t (˜ x u t ) η v j (˜ x u t ) , we obtain:

Recall that ϵ ≤ α 0 τ ˜ v u t ˜ v u t (˜ x u t ) , which indicates ϵ τ ˜ v u t ˜ v u t (˜ x u t ) ≤ α 0 . Hence, the relaxed Multiclass Tsybakov Condition holds and the probability is bounded by C ( ϵ τ ˜ v u t ˜ v u t (˜ x u t ) ) λ , namely, C ( O ( ϵ ) ) λ .

Note that a small ϵ relies on large observed data and a powerful f , so that (i) the observed data can accurately approximate ˜ η , and (ii) f can closely fit the observed data. Both requirements can be satisfied thanks to the large-scale datasets and deep learning advancements in recommendation.

Theorem 1 indicates that the probability of a reliable target ( p 1 = ˜ v u t ) keeping low prediction scores across different epochs ( ∑ H h =1 [ w h f h ˜ v u t (˜ x u t ) ] ≤ γ ) is bounded to be low, i.e., no more than C ( O ( ϵ ) ) λ . This inspires us to rectify unreliable targets by replacing consistently low-scored targets with steadily high-scored items as the following strategy.

̸

DRUT: Detecting and Replacing Unreliable Targets . Given an SRS f that is ϵ -close to ˜ η , an instance ⟨ ˜ x u t , ˜ v u t ⟩ , the consistently high-scored item v m = arg max v i =˜ v u t ∑ H h =1 [ w h f h v i (˜ x u t ) ] , and β ∈ (0 , 1] , we stipulate that if ∑ H h =1 [ w h f h ˜ v u t (˜ x u t ) ] / ∑ H h =1 [ w h f h v m (˜ x u t ) ] < β , i.e., the target ˜ v u t is consistently lower-scored than v m to some extent, then ˜ v u t should be replaced by v m in the H -th epoch. We denote the output instance of DRUT as ⟨ ˜ x u t , ˆ v u t ⟩ .

Different from existing methods, DRUT is theoretically error-bounded. Specifically, the error of DRUT, denoted as E DRUT , comes from three cases: ( Case-1 ) the true target p 1 is ˜ v u t but is replaced

by v m . ( Case-2 ) the true target p 1 is v m but ˜ v u t is kept. ( Case-3 ) the true target is neither ˜ v u t nor v m . Correspondingly, E DRUT can be formulated as below:

̸

̸

̸

̸

Subsequently, Lemma 1 first provides the thresholds β 1 and β 2 that can respectively guarantee bounded probabilities for Case-1 and Case-2 of E DRUT . Then in Theorem 2, we prove that even if the chosen β deviates from β 1 and β 2 , DRUT is still error-bounded 2 .

̸

Lemma 1. Given Assumption 1 and the set of weights { w h ∣ ∣ 1 ≤ h ≤ H, 0 ≤ w h ≤ 1 , ∑ H h =1 w h = 1 } , ∀⟨ ˜ x u t , ˜ v u t ⟩ , assume ϵ ≤ min [ α 0 τ ˜ v u t ˜ v u t (˜ x u t ) , α 0 τ v m v m (˜ x u t ) ] . Let β 1 = [ τ ˜ v u t ˜ v u t (˜ x u t ) η p 2 (˜ x u t )+ ∑ v j =˜ v u t τ v j ˜ v u t η v j (˜ x u t ) ∑ H h =1 [ w h f h vm (˜ x u t )] ] and β 2 = [ ∑ H h =1 [ w h f h ˜ v u t (˜ x u t )] τ vmvm (˜ x u t ) η p 2 (˜ x u t )+ ∑ v j = vm τ v j v m (˜ x u t ) η v j (˜ x u t ) ] . We λ

̸

have: β ≤ β 1 guarantees the probability of Case-1 in E DRUT is bounded by C ( O ( ϵ ) ) , and β ≥ β 2 guarantees the probability of Case-2 in E DRUT is bounded by C ( O ( ϵ ) ) λ .

̸

̸

3.3 Theorems for Rectifying Unreliable Input

Unreliable items within the input act as mosaics to bewilder SRSs when learning users' true preferences, thus impeding SRSs from predicting the true target. Recall that Theorem 1 suggests consistently low-scored targets are unlikely to be reliable. This inspires us to exploit the prediction scores given by a backward SRS for rectifying unreliable input, that is, deleting consistently low-scored (predicted by a backward SRS) items within the input. To avoid deleting all input items with an unreliable target, we rectify the input of instances that are already processed by DRUT. Formally, given ⟨ ˜ x u t , ˆ v u t ⟩ , we define a backward SRS ⃗ f that aims to predict every input item in ˜ x u t based on ⃗ x u t = { u, ˆ v u t } . Hence, the backward instance for predicting ˜ v u t -l , ( l ∈ [1 , L ]) can be formulated as 〈 ⃗ x u t , ˜ v u t -l 〉 . With Assumption 1 being held, and η ′ , ˜ η ′ , p ′ 1 , p ′ 2 , τ ′ , ϵ ′ respectively being the counterparts of η , ˜ η , p 1 , p 2 , τ , ϵ , for backward sequences, we propose the following strategy.

̸

DDUI: Detecting and Deleting Unreliable Input . Given ⃗ f that is ϵ ′ -close to ˜ η ′ , 〈 ⃗ x u t , ˜ v u t -l 〉 , the consistently high-scored item ⃗ v m = arg max v i =˜ v u t -l ∑ H h =1 [ w h ⃗ f h v i ( ⃗ x u t ) ] , and β ′ ∈ (0 , 1] , we stipulate that if the ratio ∑ H h =1 [ w h ⃗ f h ˜ v u t -l ( ⃗ x u t ) ] / ∑ H h =1 [ w h ⃗ f h ⃗ v m ( ⃗ x u t ) ] < β ′ , i.e., ˜ v u t -l is consistently lower-scored than ⃗ v m to some extent, then item ˜ v u t -l should be deleted from ˜ x u t in the H -th epoch. After rectifying the L items in ˜ x u t , we denote the output instance of DDUI as ⟨ ˆ x u t , ˆ v u t ⟩ .

DDUI differs from DRUT slightly, i.e., we remove unreliable input items to bridge the succeeding and preceding items, instead of replacing them. Since unreliable input items caused by distractions are essentially interruptions in the input sequence, replacing them may introduce new interruptions.

̸

Superior to prior works, DDUI is theoretically guaranteed to work well with bounded error, which comes from two cases: (1) ˜ v u t -l is a reliable input item ( p ′ 1 = ˜ v u t -l ) but is deleted; and (2) ˜ v u t -l is an unreliable input item ( p ′ 1 = ˜ v u t -l ) but is kept. Hence, the error of DDUI, denoted as E DDUI , can be correspondingly formulated as the following Eq. 8, and its bound is analyzed in Theorem 3:

̸

̸

̸

UpdateParametersviaExponential MovingAverage

Self-ensembled

RectifyTarget

Training

DRUT

ForwardSRS

ForwardSRS

Update

Top-1

Rectification

ut

Pools

DeriveBackward Instances(L=3)

Observed

Instance

Self-ensembled

Update

Top-1

x

u

t

'Training

BackwardSRs

DDUI

t

T-

5

BackwardSRS

UpdateParametersviaExponential MovingAverage

̸

4 The Proposed BirDRec Framework

By integrating DRUT and DDUI into existing SRSs, we introduce BirDRec , a model-agnostic bidirectional data rectification framework, which can rectify both unreliable targets and input with theoretical guarantees for more robust SRSs. Yet, the complexity of BirDRec is prohibitively high due to the calculation and storage of prediction scores for each instance across different epochs in DRUT and DDUI. To ease this issue, we devise a rectification sampling strategy that avoids prediction on the full item set to replace unreliable targets or delete unreliable input, thereby reducing the time complexity of BirDRec. Meanwhile, we adopt the self-ensemble mechanism [14] to approximate the weighted average prediction scores of different epochs, thus avoiding preserving scores of all epochs and reducing the space complexity.

Framework Overview . Accordingly, the efficiency-improved BirDRec is depicted in Fig. 2. Specifically, BirDRec first leverages the self-ensembled forward SRS to rectify the target of an instance via DRUT, and then the input is rectified by the self-ensembled backward SRS via DDUI. Thereafter, the rectified instance and its L backward instances are respectively used to train the forward and backward SRSs, which are finally employed to update the corresponding self-ensembled SRSs.

4.1 Reducing Time Complexity via Rectification Sampling

The time complexity of BirDRec is primarily attributed to the search for consistently high-scored items ( v m and ⃗ v m ), which requires the calculation of prediction scores between every instance and all items. Concretely, the per-epoch time complexity of BirDRec is O ( N · ( |V| + L ) ) , where N denotes the number of instances. To mitigate this substantial computational burden, we propose a rectification sampling strategy that circumvents the need to search the entire item set.

Rectification Sampling . Each instance ⟨ ˜ x u t , ˜ v u t ⟩ is assigned two rectification pools K u t and ⃗ K u t to rectify the target and input, respectively. At first, K u t is initialized by the K succeeding items of ˜ v u t , i.e., [˜ v u t +1 , ..., ˜ v u t + K ] , which are potentially better substitutions for ˜ v u t . Similarly, ⃗ K u t is initialized with the K preceding items of ˜ v u t -1 . Next, in each epoch, the items in K u t , together with K additional items that are randomly sampled from V , are ranked in descending order w.r.t. their weighted average prediction scores over different epochs (i.e., ∑ H h =1 [ w h f h v i (˜ x u t )] ). Then the top1 item, denoted as ˆ v m , is adopted as the approximation of v m in DRUT, while the topK items are retained to update K u t for the next epoch. The rationale behind approximating v m with ˆ v m is supported by Theorem 4, which indicates that the relative rank of an item over a list of randomly sampled items can approximate this item's relative rank over the full item set. That is, the top1 item ˆ v m tends to be ranked highly over

̸

V . The updating rules of ⃗ K u t for DDUI is defined similarly as K u t . As a result, the per-epoch time complexity of BirDRec is reduced from O ( N · ( |V| + L ) ) to O ( N · ( K + L ) ) .

Theorem 4. Let R be a list of K items randomly sampled from V with replacement, ζ ∈ (0 , 1) , r H ( x u t , v i ) = ∑ v j ∈V I [ ∑ H h =1 [ w h f h v j (˜ x u t )] > ∑ H h =1 [ w h f h v i (˜ x u t )] ] be the rank of item v i over the entire item set at the H -th epoch, and ˆ r H ( x u t , v i ) = ∑ v j ∈R I [ ∑ H h =1 [ w h f h v j (˜ x u t )] > ∑ H h =1 [ w h f h v i (˜ x u t )] ] be the rank of v i over R at the H -th epoch. We have: P [∣ ∣ ˆ r H ( x u t ,v i ) K -r H ( x u t ,v i ) |V| ∣ ∣ ≥ ζ ] ≤ exp( -2 Kζ 2 ) .

4.2 Reducing Space Complexity via Self-ensemble Mechanism

The huge space cost of BirDRec is caused by the storage of prediction scores between each instance and all items in every epoch, for the sake of calculating the weighted average prediction scores.

Specifically, at the H -th epoch, the space complexity of preserving all prediction scores is O ( N · |V| · H ) . To save space, we thus approximate the weighted average scores with the self-ensemble mechanism [14], thereby avoiding storing prediction scores of different epochs. Formally, let f H be an SRS parameterized by θ H , and f H be a self-ensembled SRS parameterized by θ H = ∑ H h =1 w h θ h . It has proven that [14] the difference between the weighted average prediction scores ( ∑ H h =1 w h f h v i ( x u t ) ) and the prediction of the self-ensembled SRS ( f H v i ( x u t ) ) is of the second order of smallness, if and only if w h = ρ H -h (1 -ρ ) 1 -δ ( h -1) , where ρ ∈ (0 , 1) denotes the exponential decay rate for ensembling; and δ ( · ) is the unit impulse function, i.e., δ (0) = 1 , otherwise 0 . With such self-ensemble, there is no need to store SRSs of each epoch, as θ H can be efficiently derived from θ H -1 and θ H with the exponential moving average as follows:

By doing so, the burden of retaining prediction scores of different epochs is reduced to maintaining an extra self-ensembled SRS. As the parameters of an SRS mainly consist of the user and item embeddings, the per-epoch space complexity of BirDRec is reduced from O ( ( |U| + |V| ) · d + ( L + |V| · H ) · N ) to O ( ( |U| + |V| ) · d +( L + K ) · N ) , where d represents the embedding size. Furthermore, to reduce the number of parameters and mitigate overfitting, the (self-ensembled) forward and backward SRSs in BirDRec share the same user and item embeddings.

5 Experiments and Results

5.1 Experimental Settings

Datasets. We adopt four real-world datasets with varying domains, sizes, sparsity, and average sequence lengths shown in Table 1. Specifically, ML-1M (ML) [45] is a popular movie recommendation benchmark. Beauty (Be) [46] is the product review dataset collected from Amazon.com. Yelp (Ye) [10] is a business recommendation dataset released by Yelp.com. QK-Video (QK) [47] is a video recommendation dataset crawled from Tencent.com. Following [3, 8, 9], we preprocess all datasets by removing users and items whose interactions are less than 5.

Baselines. To verify the generality of BirDRec, we implement it with vanilla SRSs based on representative backbones. In particular, FPMC [15] is based on Markov Chain. GRU4Rec [17], Caser [20], and MAGNN [27] are built on RNN, CNN, and GNN, respectively. SASRec [28] and BERT4Rec [29] are based on Transformer. Meanwhile, to validate its effectiveness and efficiency, we compare BirDRec with state-of-the-art robust SRSs including BERD [8], FMLP-Rec [9], and STEAM [10], which aim to tackle unreliable targets, unreliable input, and both, respectively.

Evaluation Protocol. Following [9, 48, 49], three widely-used metrics are adopted to evaluate the ranking quality, namely, HR, NDCG, and MRR. For all these metrics, higher metric values suggest

ML

Be

Ye

QK

0

20

40

60

SASRec

STEAM

BirDRec

better recommendation accuracy. For each user, we preserve the last two interactions for validation and testing, while the rest are used for training. The training of each model is carried out five times to report the average results. During testing, we follow [3, 9, 27] and evaluate the ranking results over the whole item set for fair comparison [50].

Implementation Details. For all methods, Xavier initializer [51] and Adam optimizer [52] are adopted; and the best hyper-parameter settings are empirically found based on the performance on the validation set. For BirDRec, it is implemented by PyTorch with batch_size = 1024 , d = 64 , learning_rate = 0 . 01 for Yelp and 0 . 001 for other datasets, L = 5 , β = 0 . 1 , β ′ = 0 . 1 , K = 10 , and ρ = 0 . 9 . To ensure accurate rectification with reduced ϵ , BirDRec is trained without rectification in the first 10 epochs. All the experiments are conducted on an NVIDIA Quadro RTX 8000 GPU 3 .

5.2 Experimental Results and Analysis

Overall Comparison . Table 2 presents the performance of vanilla SRSs built on various representative backbones. Each SRS is trained under two different settings: the original Plain setting and our

0

.

13

0

.

15

0

.

17

0

.

19

ML-1M

NDCG@10

Plain (SASRec)

DRUT

DDUI

DRUT + DDUI

DRUT + DDUI + Self-ensemble (BirDRec)

0

.

03

0

.

04

0

.

05

Beauty

NDCG@10

0

.

03

0

.

04

0

.

05

0

.

06

0

.

07

Yelp

NDCG@10

0

.

04

0

.

05

0

.

06

0

.

07

QK-Vedio

NDCG@10

BirDRec framework. We can note that all the baselines are boosted significantly on all datasets with the aid of BirDRec, which demonstrates the generality of BirDRec.

Table 3 compares BirDRec with state-of-the-art robust SRSs. Considering BERD and STEAM are both built upon Transformer, we also adopt a transformer-based SRS, SASRec, as the default backbone of BirDRec in the following experiments to ensure fair comparisons. The results show that BirDRec dramatically outperforms existing robust SRSs, confirming the effectiveness of our errorbounded rectification strategies in comparison to existing methods that lack theoretical guarantees.

Fig. 3 compares the time and storage cost of BirDRec and STEAM, both of which aim to handle unreliable input and targets simultaneously. In particular, Figs. 3(a) and (b) show that BirDRec executes 4 . 28 times faster than STEAM in each epoch and converges with fewer epochs. Meanwhile, Figs. 3(c) and (d) show that BirDRec incurs only half the storage cost of STEAM on most datasets. These results highlight the efficiency of our sampling strategy and the self-ensemble mechanism. The storage cost of BirDRec is marginally higher than STEAM's on ML-1M. This is because the advantage of our rectification sampling strategy is less obvious on datasets with small item sets (see Table 1), indicating the superior scalability of our BirDRec on large-scale item sets.

Ablation Study . To check the efficacy of essential strategies of BirDRec (DRUT, DDUI, and the self-ensemble), we add these strategies incrementally to a plain representative SRS - SASRec. Note that without self-ensemble, DRUT and DDUI only consider the prediction scores at the latest epoch for data rectification. The results are presented in Fig. 4 and similar trends can be noted with the rest SRSs on the other metrics. First, adding either DRUT or DDUI to SASRec brings dramatic improvements, implying the effectiveness of DRUT and DDUI in rectifying unreliable targets and input, respectively. Second, using DRUT and DDUI together is better than leveraging each of them solely, indicating the necessity of rectifying both unreliable targets and input for more robust SRSs. Moreover, adding the self-ensemble mechanism can further boost the accuracy, which confirms the efficacy of considering prediction scores of different epochs for data rectification. Overall, the performance gain of separately adding DRUT, DDUI, and self-ensemble is 45 . 7% , 43 . 3% , and 3 . 1% , respectively. That is, DRUT and DDUI contribute more than the self-ensemble to our BirDRec.

Hyper-parameter Analysis. We analyze the impact of key hyper-parameters, i.e., the rectification thresholds β , β ′ , the size K of the rectification pool, and the exponential decay rate ρ . The results are presented in Fig. 5 with several major findings (similar trends can be noted with the other metrics on the rest datasets). (i) The best choice for β and β ′ is 0 . 1 , showing that small thresholds tend to maintain unreliable instances while large ones may mistakenly rectify reliable data. (ii) K ≥ 10 is sufficient to gain better accuracy. (iii) ρ ≥ 0 . 5 yields better performance, that is, it is beneficial to consider predictions in earlier epochs. (iv) Even if the hyper-parameters of BirDRec are not optimally set, BirDRec still dramatically outperforms the best baseline (BERD) on ML-1M.

-

4

-

3

-

2

-

1

0

0

.

15

0

.

17

0

.

19

log

10

(

β

)

NDCG@10

-

4

-

3

-

2

-

1

0

0

.

15

0

.

17

0

.

19

log

10

(

β

′

)

NDCG@10

5

10

15

20

25

0

.

15

0

.

17

0

.

19

K

NDCG@10

0

.

1 0

.

3 0

.

5 0

.

7 0

.

9

0

.

15

0

.

17

0

.

19

ρ

NDCG@10

BirDRec

BERD

SASRec

Case Study. Fig. 6 presents a real instance that is rectified by BirDRec on ML-1M. Specifically, in the DRUT module, the target movie 'Schindler's List' consistently gets lower scores than the movie 'Terminator 2' in the rectification pool, and is then replaced by 'Terminator 2'. This is reasonable, given that 'Terminator 2' aligns more closely with the input Sci-fi movies than 'Schindler's List'. Subsequently, in the DDUI module, the input movie 'American Beauty' is consistently lower-scored than the movie 'Star Wars IV' in the rectification pool and is thus deleted. This decision is justifiable, considering that the rectified target, 'Terminator 2', generally lacks relevance to the input 'American Beauty' across various aspects, including genres, actors, directors, tags, etc.

ObservedInstance

Self-ensembled

DRUT

RectifyTarget

STINOESINT

Replace

ForwardSRS

EPISODEV

TERMINATOR

SHNISINT

Top-1

AMERICAN

BEAUTY

MB

JURASSICPARK

Rectification

Pools

Kt

i

ut

Top-1

Delete

Self-ensembled

RectifyInput

BE

T

DDUI

BackwardSRS

Limitations of BirDRec. Although BirDRec exhibits its superiority through our extensive experiments, its limitations are two-fold. Firstly, the improvement of BirDRec on storage cost is limited for smaller item sets. This is primarily because the rectification sampling strategy with self-ensemble has less apparent advantages on datasets with smaller item sets. To be specific, the storage cost reduction for calculating weighted average prediction scores is from O ( | V | ∗ H ) to O ( K ) for each instance. Thus if | V | is small, the benefit of this reduction will be less obvious. Secondly, although BirDRec is significantly faster than the latest robust SRS (STEAM), it is worth noting that BirDRec is 1.6 times on average slower than its backbone model (as depicted in Fig. 3) in each training epoch. This increased training time could be a practical concern in systems with extremely large-scale datasets and real-time recommendation demands.

6 Conclusion

This work, for the first time , provides theoretically guaranteed data rectification strategies to tackle both unreliable input and targets for more robust SRSs. The proposed strategies are further integrated into a model-agnostic bidirectional data rectification framework, BirDRec, that can be flexibly implemented with most existing SRSs, for robust training against unreliable data. Additionally, we devise a rectification sampling strategy to reduce the computational cost of BirDRec; meanwhile, a self-ensemble mechanism is adopted to reduce the space complexity. Extensive experiments verify the generality, effectiveness, and efficiency of the proposed BirDRec.

Acknowledgments

The work is partially supported by the National Key Research and Development Program of China (2020YFB1707900), the National Natural Science Foundation of China (Nos. U22A2025, 62072088, 62232007), Liaoning Provincial Science and Technology Plan Project - Key R&D Department of Science and Technology (No. 2023JH2/101300182), and ARC Discovery Projects DP200101441 and DP230100676.

References

Shoujin Wang, Liang Hu, Yan Wang, Longbing Cao, Quan Z Sheng, and Mehmet Orgun. Sequential recommender systems: challenges, progress and prospects. In IJCAI , pages 63326338, 2019.

Hui Fang, Danning Zhang, Yiheng Shu, and Guibing Guo. Deep learning for sequential recommendation: Algorithms, influential factors, and evaluations. ACM TOIS , 39(1):1-42, 2020.

Ruihong Qiu, Zi Huang, Hongzhi Yin, and Zijian Wang. Contrastive learning for representation degeneration problem in sequential recommendation. In WSDM , pages 813-823, 2022.

Yu Zheng, Chen Gao, Jianxin Chang, Yanan Niu, Yang Song, Depeng Jin, and Yong Li. Disentangling long and short-term interests for recommendation. In TheWebConf , pages 22562267, 2022.

Yongjun Chen, Zhiwei Liu, Jia Li, Julian J. McAuley, and Caiming Xiong. Intent contrastive learning for sequential recommendation. In TheWebConf , pages 2172-2182, 2022.

Jing Zhang, Bowen Hao, Bo Chen, Cuiping Li, Hong Chen, and Jimeng Sun. Hierarchical reinforcement learning for course recommendation in moocs. In AAAI , pages 435-442, 2019.

Xu Chen, Hongteng Xu, Yongfeng Zhang, Jiaxi Tang, Yixin Cao, Zheng Qin, and Hongyuan Zha. Sequential recommendation with user memory networks. In WSDM , pages 108-116, 2018.

Yatong Sun, Bin Wang, Zhu Sun, and Xiaochun Yang. Does every data instance matter? enhancing sequential recommendation by eliminating unreliable data. In IJCAI , pages 15791585, 2021.

Kun Zhou, Hui Yu, Wayne Xin Zhao, and Ji-Rong Wen. Filter-enhanced mlp is all you need for sequential recommendation. In TheWebConf , pages 2388-2399, 2022.

Yujie Lin, Chenyang Wang, Zhumin Chen, Zhaochun Ren, Xin Xin, Qiang Yan, Maarten de Rijke, Xiuzhen Cheng, and Pengjie Ren. A self-correcting sequential recommender. In TheWebConf , pages 1283-1293, 2023.

Ziwei Fan, Zhiwei Liu, Yu Wang, Alice Wang, Zahra Nazari, Lei Zheng, Hao Peng, and Philip S Yu. Sequential recommendation via stochastic self-attention. In TheWebConf , pages 2036-2047, 2022.

Chi Zhang, Yantong Du, Xiangyu Zhao, Qilong Han, Rui Chen, and Li Li. Hierarchical item inconsistency signal learning for sequence denoising in sequential recommendation. In CIKM , pages 2508-2518, 2022.

Yuqi Qin, Pengfei Wang, and Chenliang Li. The world is binary: Contrastive learning for denoising next basket recommendation. In SIGIR , pages 859-868, 2021.

Hongjun Wang and Yisen Wang. Self-ensemble adversarial training for improved robustness. In ICLR , 2022.

Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. Factorizing personalized markov chains for next-basket recommendation. In TheWebConf , pages 811-820, 2010.

Chen Cheng, Haiqin Yang, Michael R Lyu, and Irwin King. Where you like to go next: successive point-of-interest recommendation. In IJCAI , pages 2605-2611, 2013.

Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based recommendations with recurrent neural networks. In ICLR , 2016.

Feng Yu, Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. A dynamic recurrent model for next basket recommendation. In SIGIR , pages 729-732, 2016.

Chao-Yuan Wu, Amr Ahmed, Alex Beutel, Alexander J. Smola, and How Jing. Recurrent recommender networks. In WSDM , pages 495-503, 2017.

Jiaxi Tang and Ke Wang. Personalized top-n sequential recommendation via convolutional sequence embedding. In WSDM , pages 565-573, 2018.

Trinh Xuan Tuan and Minh Phuong Tu. 3d convolutional networks for session-based recommendation with content features. In RecSys , pages 138-146, 2017.

Fajie Yuan, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M Jose, and Xiangnan He. A simple convolutional generative network for next item recommendation. In WSDM , pages 582-590, 2019.

Mengqi Zhang, Shu Wu, Xueli Yu, Qiang Liu, and Liang Wang. Dynamic graph neural networks for sequential recommendation. IEEE TKDE , 35(5):4741-4753, 2023.

Jianxin Chang, Chen Gao, Yu Zheng, Yiqun Hui, Yanan Niu, Yang Song, Depeng Jin, and Yong Li. Sequential recommendation with graph neural networks. In SIGIR , pages 378-387, 2021.

Chengfeng Xu, Pengpeng Zhao, Yanchi Liu, Victor S Sheng, Jiajie Xu, Fuzhen Zhuang, Junhua Fang, and Xiaofang Zhou. Graph contextualized self-attention network for session-based recommendation. In IJCAI , pages 3940-3946, 2019.

Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. Session-based recommendation with graph neural networks. In AAAI , pages 346-353, 2019.

Chen Ma, Liheng Ma, Yingxue Zhang, Jianing Sun, Xue Liu, and Mark Coates. Memory augmented graph neural networks for sequential recommendation. In AAAI , pages 5045-5052, 2020.

Wang-Cheng Kang and Julian McAuley. Self-attentive sequential recommendation. In ICDM , pages 197-206, 2018.

Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In CIKM , pages 1441-1450, 2019.

Zhu Sun, Qing Guo, Jie Yang, Hui Fang, Guibing Guo, Jie Zhang, and Robin Burke. Research commentary on recommendations with side information: a survey and research directions. Electronic Commerce Research and Applications (ECRA) , 37, 2019.

Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. Learning from noisy labels with deep neural networks: A survey. IEEE TNNLS , 2022.

Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In NeurIPS , pages 8536-8546, 2018.

Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor Tsang, and Masashi Sugiyama. How does disagreement help generalization against label corruption? In ICML , pages 7164-7173, 2019.

Hongxin Wei, Lei Feng, Xiangyu Chen, and Bo An. Combating noisy labels by agreement: A joint training method with co-regularization. In CVPR , pages 13723-13732, 2020.

Xiaobo Xia, Tongliang Liu, Bo Han, Mingming Gong, Jun Yu, Gang Niu, and Masashi Sugiyama. Sample selection with uncertainty of losses for learning with noisy labels. In ICLR , 2022.

Yangdi Lu, Yang Bo, and Wenbo He. Noise attention learning: Enhancing noise robustness by gradient scaling. In NeurIPS , pages 23164-23177, 2022.

Yangdi Lu and Wenbo He. SELC: self-ensemble label correction improves learning with noisy labels. In IJCAI , pages 3278-3284, 2022.

Songzhu Zheng, Pengxiang Wu, Aman Goswami, Mayank Goswami, Dimitris Metaxas, and Chao Chen. Error-bounded correction of noisy labels. In ICML , pages 11447-11457, 2020.

Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. Neural attentive session-based recommendation. In CIKM , pages 747-755, 2017.

Jiaxi Tang, Francois Belletti, Sagar Jain, Minmin Chen, Alex Beutel, Can Xu, and Ed H. Chi. Towards neural mixture recommender for long range dependent user sequences. In TheWebConf , pages 1782-1793, 2019.

Chen Ma, Peng Kang, and Xue Liu. Hierarchical gating networks for sequential recommendation. In KDD , pages 825-833, 2019.

Ziwei Fan, Zhiwei Liu, Shen Wang, Lei Zheng, and Philip S Yu. Modeling sequences as distributions with uncertainty for sequential recommendation. In CIKM , pages 3019-3023, 2021.

Di-Rong Chen and Tao Sun. Consistency of multiclass empirical risk minimization methods based on convex loss. JMLR , 7:2435-2447, 2006.

Jingtao Ding, Yuhan Quan, Quanming Yao, Yong Li, and Depeng Jin. Simplify and robustify negative sampling for implicit collaborative filtering. In NeurIPS , pages 1094-1105, 2020.

F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. ACM TIIS , 5(4):1-19, 2015.

Julian McAuley and Jure Leskovec. Hidden factors and hidden topics: understanding rating dimensions with review text. In RecSys , pages 165-172, 2013.

Guanghu Yuan, Fajie Yuan, Yudong Li, Beibei Kong, Shujie Li, Lei Chen, Min Yang, Chenyun YU, Bo Hu, Zang Li, Yu Xu, and Xiaohu Qie. Tenrec: A large-scale multipurpose benchmark dataset for recommender systems. In NeurIPS , pages 11480-11493, 2022.

Zhu Sun, Di Yu, Hui Fang, Jie Yang, Xinghua Qu, Jie Zhang, and Cong Geng. Are we evaluating rigorously? benchmarking recommendation for reproducible evaluation and fair comparison. In RecSys , pages 23-32, 2020.

Zhu Sun, Hui Fang, Jie Yang, Xinghua Qu, Hongyang Liu, Di Yu, Yew-Soon Ong, and Jie Zhang. Daisyrec 2.0: Benchmarking recommendation for rigorous evaluation. IEEE TPAMI , 45(7):8206-8226, 2023.

Walid Krichene and Steffen Rendle. On sampled metrics for item recommendation. In KDD , pages 1748-1757, 2020.

Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In AISTATS , pages 249-256, 2010.

Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR , 2015.

Theoretically Guaranteed Bidirectional Data Rectification for Robust Sequential Recommendation - Appendix -

This Appendix is divided into three sections. First, Section A empirically validates the feasibility of Assumption 1. Next, in Section B, complete proofs of all the lemmas and theorems are presented. Finally, in Section C, we provide detailed settings of baselines and additional experimental results, including the hyper-parameter analysis and the percentage of rectified data. Our code is available at: https://github.com/AlchemistYT/BirDRec .

A The Feasibility of Assumption 1

Assumption 1. The users' true preference distribution η fulfills the relaxed Multiclass Tsybakov Condition [1] with constants C > 0 , λ > 0 , and α 0 ∈ (0 , 1] , such that for all α ∈ (0 , α 0 ] ,

The feasibility of Assumption 1 relies on large C and small λ . In order to estimate the values of C and λ , the initial step is to approximate the true preference distribution η . To achieve this, we first obtain a reliable dataset D via the heuristic method proposed by [2]. The heuristic method measures the matching degree between the input and target of each instance from two aspects: item co-occurrence and item properties, then those instances with matching degrees lower than a threshold are filtered as unreliable data. We use a large threshold ( 0 . 9 ) to filter the dataset rigorously, ensuring the vast majority of the maintained instances in D are reliable. Subsequently, as suggested by [3], we train a classic SRS, SASRec [4], on the filtered reliable dataset D and then use the prediction of SASRec to approximate η . Formally, for each reliable instance ⟨ x u t , v u t ⟩ from the reliable dataset D , the SRS prediction score f v u t ( x u t ) is employed as the approximation of η v u t ( x u t ) .

Next, we densely sample α from 0 . 05 to 0 . 9 with step size 0 . 005 and calculate the corresponding left-hand-side probability of Eq. 1 with the following relative frequency F α , and collect a series of ( log( α ) , log( F α ) ) data points:

where ˆ p 1 and ˆ p 2 are respectively the topand middle-ranked items according to f , namely, ∑ v i ∈V I [ f ˆ p 1 ( x u t ) ≥ f v i ( x u t ) ] = |V| , ∑ v i ∈V I [ f ˆ p 2 ( x u t ) ≥ f v i ( x u t ) ] = ⌊|V| / 2 ⌋ . We then use log( F α ) to approximate log( Cα λ ) . As shown by the blue dots in Fig. 6, the collected ( log( α ) , log( F α ) ) data points are generally linearly distributed, which allows us to estimate C and λ with linear regression according to log( Cα λ ) = log( C ) + λ log( α ) . As a result, Fig. 6 shows that the estimated C and λ are respectively restricted in (0 . 55 , 0 . 70) and (1 . 37 , 4 . 01) on real-world datasets from various domains, which validates the feasibility of Assumption 1.

(a) ML-1M

(b) Beauty

(c) Yelp

(d) QK-Vedio

Observed (log(α), log(Fa))

-2

Regression Line:Alog(α)+log(C)

-4

-6

0

-8

-10

8888

EstimatedC:0.6895

Estimated入:2.7473

-12

-4

-3

-2

-1

0

log(α)

-1

Observed (log(α), log(Fα))

Regression Line:Alog(α)+log(C)

-2

-3

-4

-5

-6

Estimated C:0.5579

-7

Q

Estimated入:1.3720

-4

-3

-2

-1

0

log(α)

-2

O

Observed (log(α), log(Fa))

-2

O

Observed (log(α), log(Fa))

Regression Line:Alog(α)+log(C)

RegressionLine:Alog(α)+log(C)

-4

-4

-6

-6

-8

0

-8

-10

-10

-14

-12

0000

08

Estimated C:0.6813

Estimated入:4.0051

-12

oo

00

88

EstimatedC:0.6941

Estimated入:3.4551

-14

00

-16

-4

-3

-2

-1

0

-4

-3

-2

-1

0

log(α)

log(α)

o0

B Proofs of Lemmas and Theorems

B.1 The Proof of Theorem 1

̸

Theorem 1. Given Assumption 1, let { w h ∣ ∣ 1 ≤ h ≤ H, 0 ≤ w h ≤ 1 , ∑ H h =1 w h = 1 } be the weights for averaging prediction scores of different epochs. ∀⟨ ˜ x u t , ˜ v u t ⟩ , assume ϵ ≤ α 0 τ ˜ v u t ˜ v u t (˜ x u t ) . Let γ = τ ˜ v u t ˜ v u t (˜ x u t ) η p 2 (˜ x u t )+ ∑ v j =˜ v u t τ v j ˜ v u t (˜ x u t ) η v j (˜ x u t ) . We have: P [ p 1 = ˜ v u t , ∑ H h =1 [ w h f h ˜ v u t (˜ x u t ) ] ≤ γ ] ≤ C ( O ( ϵ )) λ .

Proof.

̸

̸

By replacing γ with τ ˜ v u t ˜ v u t (˜ x u t ) η p 2 (˜ x u t ) + ∑ v j =˜ v u t τ v j ˜ v u t (˜ x u t ) η v j (˜ x u t ) , we obtain:

̸

Recall that ϵ ≤ α 0 τ ˜ v u t ˜ v u t (˜ x u t ) , which implies ϵ τ ˜ v u t ˜ v u t (˜ x u t ) ≤ α 0 . Hence, the relaxed Multiclass Tsybakov Condition holds and the probability is bounded by C ( ϵ τ ˜ v u t ˜ v u t (˜ x u t ) ) λ , namely, C ( O ( ϵ ) ) λ .

B.2 The Proof of Lemma 1

̸

̸

̸

̸

̸

Lemma 1. Given Assumption 1 and the set of weights { w h ∣ ∣ 1 ≤ h ≤ H, 0 ≤ w h ≤ 1 , ∑ H h =1 w h = 1 } , ∀⟨ ˜ x u t , ˜ v u t ⟩ , assume ϵ ≤ min [ α 0 τ ˜ v u t ˜ v u t (˜ x u t ) , α 0 τ v m v m (˜ x u t ) ] . Let β 1 = [ τ ˜ v u t ˜ v u t (˜ x u t ) η p 2 (˜ x u t )+ ∑ v j =˜ v u t τ v j ˜ v u t η v j (˜ x u t ) ∑ H h =1 [ w h f h vm (˜ x u t )] ] and β 2 = [ ∑ H h =1 [ w h f h ˜ v u t (˜ x u t )] τ vmvm (˜ x u t ) η p 2 (˜ x u t )+ ∑ v j = vm τ v j v m (˜ x u t ) η v j (˜ x u t ) ] . We have: β ≤ β 1 guarantees the probability of Case-1 in E DRUT is bounded by C ( O ( ϵ ) ) λ , and β ≥ β 2 guarantees the probability of Case-2 in E DRUT is bounded by C ( O ( ϵ ) ) λ .

Proof. For Case-1 of E DRUT , we have:

̸

̸

̸

By substituting β 1 with τ ˜ v u t ˜ v u t (˜ x u t ) η p 2 (˜ x u t )+ ∑ v j =˜ v u t τ v j ˜ v u t η v j (˜ x u t ) ∑ H h =1 [ w h f h vm (˜ x u t )] in Eq. 7, we obtain:

̸

Recall that ϵ ≤ α 0 τ ˜ v u t ˜ v u t (˜ x u t ) , which implies ϵ τ ˜ v u t ˜ v u t (˜ x u t ) ≤ α 0 . Hence, the Multiclass Tsybakov Condition holds and the probability of Case-1 is bounded by C ( ϵ τ ˜ v u t ˜ v u t (˜ x u t ) ) λ , namely, C ( O ( ϵ ) ) λ .

̸

Thereafter, for Case-2 of E DRUT , we have:

̸

̸

̸

We replace β 2 with ∑ H h =1 [ w h f h ˜ v u t (˜ x u t )] τ vmvm (˜ x u t ) η p 2 (˜ x u t )+ ∑ v j = vm τ v j v m (˜ x u t ) η v j (˜ x u t ) in Eq. 5 and continue the calculation:

̸

Recall that ϵ ≤ α 0 τ v m v m (˜ x u t ) , which implies ϵ τ vmvm (˜ x u t ) ≤ α 0 . Hence, the Multiclass Tsybakov Condition holds and the probabiliy of Case-2 is bounded by C ( ϵ τ vmvm (˜ x u t ) ) λ , namely, C ( O ( ϵ ) ) λ .

B.3 The Proof of Theorem 2

̸

̸

̸

Proof. For Case-1 of E DRUT , we have:

̸

We substitute β 1 with τ ˜ v u t ˜ v u t (˜ x t ) η p 2 (˜ x t )+ v j =˜ v u t τ v j ˜ v u t η v j (˜ x t ) ∑ H h =1 [ w h f h vm (˜ x u t )] and obtain:

̸

̸

Recall that ϵ ≤ α 0 τ ˜ v u t ˜ v u t (˜ x u t ) -ξ 1 , which implies ϵ + ξ 1 τ ˜ v u t ˜ v u t (˜ x u t ) ≤ α 0 . Hence, the Multiclass Tsybakov Condition holds and the probabiliy of Case-2 is bounded by C ( ϵ + ξ 1 τ ˜ v u t ˜ v u t (˜ x u t ) ) λ , namely, C ( O ( ϵ + ξ 1 ) ) λ .

̸

̸

For Case-2 of E DRUT , we have:

̸

̸

̸

We replace β 2 with ∑ H h =1 [ w h f h ˜ v u t (˜ x u t )] τ vmvm (˜ x u t ) η p 2 (˜ x u t )+ ∑ v j = vm τ v j v m (˜ x u t ) η v j (˜ x u t ) and continue the calculation:

̸

Recall that ϵ ≤ α 0 τ vmvm (˜ x u t ) β 2 ( β 2 -ξ 2 ) -ξ 2 β 2 ( β 2 -ξ 2 ) , which implies ϵ τ vmvm (˜ x u t ) + ξ 2 β 2 ( β 2 -ξ 2 ) τ vmvm (˜ x u t ) ≤ α 0 . Hence, the relaxed Multiclass Tsybakov Condition holds and the probability of Case-2 is bounded by C ( ϵβ 2 ( β 2 -ξ 2 )+ ξ 2 β 2 ( β 2 -ξ 2 ) τ vmvm (˜ x u t ) ) λ , namely, C ( O ( ϵ + ξ 2 ) ) λ .

̸

̸

Finally, for Case-3 of E DRUT , we have:

̸

̸

̸

̸

̸

̸

B.4 The Proof of Theorem 3

̸

̸

̸

̸

̸

̸

̸

̸

Proof. For the first term of E DDUI , we have:

̸

̸

We substitute β ′ 1 with τ ′ ˜ v u t -l ˜ v u t -l ( ⃗ x u t ) η ′ p ′ 2 ( ⃗ x u t )+ ∑ v j =˜ v u t -l τ ′ v j ˜ v u t -l η ′ v j ( ⃗ x u t ) ∑ H h =1 [ w h ⃗ f h ⃗ vm ( ⃗ x u t )] and obtain:

Recall that ϵ ′ ≤ α 0 τ ′ ˜ v u t -l ˜ v u t -l ( ⃗ x u t ) -ξ ′ 1 , which implies ϵ ′ + ξ ′ 1 τ ′ ˜ v u t -l ˜ v u t -l ( ⃗ x u t ) ≤ α 0 . Hence, the relaxed Multiclass Tsybakov Condition holds, and the probability of the first term of E DDUI (Eq. 14) is bounded by C ( ϵ ′ + ξ ′ 1 τ ′ ˜ v u t -l ˜ v u t -l ( ⃗ x u t ) ) λ , namely, C ( O ( ϵ ′ + ξ ′ 1 ) ) λ .

For the second term of E DDUI , we have:

̸

̸

̸

̸

̸

̸

̸

For the first term of Eq. 17, we have:

̸

̸

̸

We replace β ′ 2 with ∑ H h =1 [ w h ⃗ f h ˜ v u t ( ⃗ x u t )] τ ′ ⃗ v m ⃗ v m ( ⃗ x u t ) η ′ p ′ 2 ( ⃗ x u t )+ ∑ v j = vm τ ′ v j v m ( ⃗ x u t ) η ′ v j ( ⃗ x u t ) and continue the calculation:

̸

Recall that ϵ ′ ≤ α 0 τ ′ ⃗ v m ⃗ v m ( ⃗ x u t ) β ′ 2 ( β ′ 2 -ξ ′ 2 ) -ξ ′ 2 β ′ 2 ( β ′ 2 -ξ ′ 2 ) , which implies ϵ ′ τ ′ ⃗ v m ⃗ v m ( ⃗ x u t ) + ξ ′ 2 β ′ 2 ( β ′ 2 -ξ ′ 2 ) τ ′ ⃗ v m ⃗ v m ( ⃗ x u t ) ≤ α 0 . Hence, the relaxed Multiclass Tsybakov Condition holds and the probability of the first term of Eq. 17 is bounded by C ( ϵ ′ β ′ 2 ( β ′ 2 -ξ ′ 2 )+ ξ ′ 2 β ′ 2 ( β ′ 2 -ξ ′ 2 ) τ ′ ⃗ v m ⃗ v m ( ⃗ x u t ) ) λ , namely, C ( O ( ϵ ′ + ξ ′ 2 ) ) λ .

̸

̸

Finally, for the second term of Eq. 17, we have:

̸

̸

̸

̸

Recall that ϵ ′ ≤ 1 2 [ [ τ ′ p ′ 1 p ′ 1 ( ⃗ x u t ) -τ ′ p ′ 1 ⃗ v m ( ⃗ x u t )][ α 0 + η ′ p ′ 2 ( ⃗ x u t )] -∑ v j = p ′ 1 [ τ ′ ⃗ v m ( ⃗ x u t ) -τ ′ v j p ′ 1 ( ⃗ x u t )] η ′ v j ( ⃗ x u t ) ] , which implies ∑ v j = p ′ 1 [ τ ′ ⃗ vm ( ⃗ x u t ) -τ ′ v j p ′ 1 ( ⃗ x u t ) ] η ′ v j ( ⃗ x u t )+2 ϵ ′ -[ τ ′ p ′ 1 p ′ 1 ( ⃗ x u t ) -τ ′ p ′ 1 ⃗ vm ( ⃗ x u t ) ] η ′ p ′ 2 ( ⃗ x u t ) [ τ ′ p ′ p ′ ( ⃗ x u t ) -τ ′ p ′ ( ⃗ x u t ) ] ≤ α 0 . Hence,

̸

̸

1 1 1 ⃗ vm the relaxed Multiclass Tsybakov Condition holds and the probability of Case-3 is bounded by C ( ∑ v j = p ′ 1 [ τ ′ ⃗ vm ( ⃗ x u t ) -τ ′ v j p ′ 1 ( ⃗ x u t ) ] η ′ v j ( ⃗ x u t )+2 ϵ ′ -[ τ ′ p ′ 1 p ′ 1 ( ⃗ x u t ) -τ ′ p ′ 1 ⃗ vm ( ⃗ x u t ) ] η ′ p ′ 2 ( ⃗ x u t ) [ τ ′ p ′ 1 p ′ 1 ( ⃗ x u t ) -τ ′ p ′ 1 ⃗ vm ( ⃗ x u t ) ] ) λ , namely, C ( O ( ϵ ′ ) ) λ .

B.5 The Proof of Theorem 4

Theorem 4. Let R be a list of K items randomly sampled from V with replacement, ζ ∈ (0 , 1) , r H ( x u t , v i ) = ∑ v j ∈V I [ ∑ H h =1 [ w h f h v j (˜ x u t )] > ∑ H h =1 [ w h f h v i (˜ x u t )] ] be the rank of item v i over the entire item set at the H -th epoch, and ˆ r H ( x u t , v i ) = ∑ v j ∈R I [ ∑ H h =1 [ w h f h v j (˜ x u t )] > ∑ H h =1 [ w h f h v i (˜ x u t )] ] be the rank of v i over R at the H -th epoch. We have: P [∣ ∣ ˆ r H ( x u t ,v i ) K -r H ( x u t ,v i ) |V| ∣ ∣ ≥ ζ ] ≤ exp( -2 Kζ 2 ) .

Proof. ˆ r H ( x u t , v i ) can be deemed as a random variable that counts the number of items ranked higher than v i in R . Since the items in R are randomly sampled from V with replacement, ˆ r H ( x u t , v i ) follows a Binomial distribution Binomial( K, r H ( x u t ,v i ) |V| ) . Thus, we have:

̸

̸

̸

̸

Meanwhile, ˆ r H ( x u t , v i ) can also be viewed as the sum of K i.i.d. Bernoulli random variables b 1 , b 2 , ..., b k , where b k = 1(1 ≤ k ≤ K ) indicates the k -th sampled item for R is ranked higher than v i , and b k = 0 otherwise. Hence, by applying Hoeffding's inequality [5], we have:

Then by replacing ∑ K k =1 b k with ˆ r H ( x u t , v i ) , we obtain:

Next, we replace E [ ˆ r H ( x u t , v i ) ] with K · r H ( x u t ,v i ) |V| according to Eq. 21:

C Additional Experimental Results

C.1 Hyper-parameter Settings for Baselines

For fair comparisons, we implement FPMC with PyTorch. For other baselines, we use the original code provided by the corresponding authors. All the baselines adopt Xavier [6] initializer and Adam [7] optimizer. We empirically find the optimal hyper-parameter setting for each baseline based on the performance on the validation set. The detailed hyper-parameter setting for each baseline is summarized in Table 1.

C.2 More Results on Hyper-parameter Analysis

-

4

-

3

-

2

-

1

0

0

.

035

0

.

045

0

.

055

log

10

(

β

)

NDCG@10

-

4

-

3

-

2

-

1

0

log

10

(

β

′

)

5

10

15

20

25

K

0

.

1 0

.

3 0

.

5 0

.

7 0

.

9

ρ

BirDRec

FMLP-Rec

SASRec

-

4

-

3

-

2

-

1

0

0

.

03

0

.

05

0

.

07

log

10

(

β

)

NDCG@10

-

4

-

3

-

2

-

1

0

log

10

(

β

′

)

5

10

15

20

25

K

0

.

1 0

.

3 0

.

5 0

.

7 0

.

9

ρ

BirDRec

STEAM

SASRec

-

4

-

3

-

2

-

1

0

0

.

045

0

.

055

0

.

065

0

.

075

log

10

(

β

)

NDCG@10

-

4

-

3

-

2

-

1

0

log

10

(

β

′

)

5

10

15

20

25

K

0

.

1 0

.

3 0

.

5 0

.

7 0

.

9

ρ

BirDRec

FMLP-Rec

SASRec

C.3 More Results on the Percentage of Rectified Instances

(a) ML-1M

(b) Beauty

(c) Yelp

(d) QK-Video

0.200

0.175

0.150

0.125

G

0.100

C

D

0.075

Z

0.050

BirDRec

0.025

SASRec

0

20

40

60

80

100

120140

Epoch

0.05

0.04

100

0.03

OCG

ND

0.02

0.01

BirDRec

SASRec

0

20

40

60

80

100

120

Epoch

0.06

BirDRec

SASRec

0.05

0.04

G

C

0.03

ND

0.02

0.01

0

25

50

75

100 125 150 175

Epoch

0.06

10.

0.05

G

C

0.04

ND

0.03

BirDRec

SASReC

0.02

0

10

20

30

40

50

60

70

Epoch

(a) ML-1M

(b) Beauty

(c) Yelp

(d) QK-Video

0.22

Instances

InputRectified

0.20

TargetRectified

0.18

Percentage

0.16

0.14

0.12

P

10

20

30

40

50

60

Epoch

Instances

0.08

InputRectified

0.07

TargetRectified

0.06

6

0.05

0.04

Percentag

0.03

0.02

P

0.01

10

20

30

40

50

60

Epoch

ances

0.040

InputRectified

TargetRectified

0.035

Inst

0.030

6

e

0.025

g

entas

0.020

ercer

0.015

P

0.010

10

15

20

25

30

35

40

45

Epoch

0.13

Instances

InputRectified

0.12

TargetRectified

0.11

0.10

Percentage

0.09

0.08

0.07

P

0.06

10

20

30

40

50

60

Epoch

References

Di-Rong Chen and Tao Sun. Consistency of multiclass empirical risk minimization methods based on convex loss. The Journal of Machine Learning Research , 7:2435-2447, 2006.

Yatong Sun, Bin Wang, Zhu Sun, and Xiaochun Yang. Does every data instance matter? enhancing sequential recommendation by eliminating unreliable data. In IJCAI , pages 15791585, 2021.

Songzhu Zheng, Pengxiang Wu, Aman Goswami, Mayank Goswami, Dimitris Metaxas, and Chao Chen. Error-bounded correction of noisy labels. In ICML , pages 11447-11457, 2020.

Wang-Cheng Kang and Julian McAuley. Self-attentive sequential recommendation. In ICDM , pages 197-206, 2018.

Wassily Hoeffding. Probability Inequalities for sums of Bounded Random Variables , pages 409-426. 1994.

Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In AISTATS , pages 249-256, 2010.

Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR , 2015.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nOverall comment:\nThis paper proposes two detect and rectify methods to solve the problem of unreliable instances in Sequential Recommender Systems (SRSs), including unreliable items in the input and unreliable targets. The two methods can be applied to existing SRSs models, effectively improving the model performance.\n\nStrengths of the paper include:\n\n1. The paper provides sufficient theoretical proof for the methods, which is very impressive.\n2. The discussion and clarification of related work, as well as the differences and connections with existing work, are very clear.\n3. The experimental results are sufficient, and the original model performance is improved on multiple baseline models via proposed method. The ablation study also compares the performance of the DRUT, DDUI, their combination, and whether to use self-ensemble under different conditions.\n\nWeakness of the article includes:\n\nThe paper does not discuss the limitations of the methods. In particular, I am interested in whether this method can be applied in large-scale real-world recommender systems? Are there any practical issues?\n\nSTRENGTHS:\nStrengths of the paper include:\n\n1. The paper provides sufficient theoretical proof for the methods, which is very impressive.\n2. The discussion and clarification of related work, as well as the differences and connections with existing work, are very clear.\n3. The experimental results are sufficient, and the original model performance is improved on multiple baseline models via proposed method. The ablation study also compares the performance of the DRUT, DDUI, their combination, and whether to use self-ensemble under different conditions.\n\nWEAKNESSES:\nWeakness of the article includes:\n\nThe paper does not discuss the limitations of the methods. In particular, I am interested in whether this method can be applied in large-scale real-world recommender systems? Are there any practical issues?\n\nQUESTIONS:\nI am interested in whether this method can be applied in large-scale real-world recommender systems? Are there any practical issues?\n\nLIMITATIONS:\nAs I written in weakness:\n\nThe paper does not discuss the limitations of the methods. In particular, I am interested in whether this method can be applied in large-scale real-world recommender systems? Are there any practical issues?", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to address the issue where the performance of sequential recommender systems (SRSs) could be harmed by mismatched input-target pairs introduced by distracted users. The paper claims that existing studies only focus on either unreliable inputs or targets but not both simultaneously. Also no theoretical guarantee of effectiveness for the existing methods are provided.  To address these issues, the paper proposes to replace/remove target/input item using a forward/backward SRS's weighted average scores accumulated over training epochs. Error bounds for both forward and backward rectifications are provided. To further reduce the time and space complexity, sampling strategy as well as self-ensembled mechanism are applied. Experiments show the proposed approach is able to improve the performance is representative SBSs on four datasets. Comparing to rectified/denoised SBSs, the performance of the proposed approach is also reported to be better. Ablation test as well as hyperparameter analysis are also provided.\n\nSTRENGTHS:\n* The ideas of a. rectifying input and target items using weighted average prediction score of forward and backward SBR, b. using sampling and self-ensembled mechanism of improving efficiency are interesting and new in my opinion.\n* The main steps leading to the error bounds are reasonable, though I did not manage to check the proofs in the supplementary in details.\n* The empirical studies that support the effectiveness of the propose approach appeals to be convincing.\n\nWEAKNESSES:\nAs I am not familiar with the literature of the robustness of SRS nor the rectification / denoising in SRS, I am only able to give a few suggestions:\n* Analyzing the existing work as well as the proposed approach on a synthetic dataset with ground truth could further strengthen the claims about: a. the drawbacks of the existing studies, b. the effectiveness of the proposed approach.  Conduct simulation on the synthetic dataset could validate the error bound, as well as evaluate the approximation quality of the proposed method.\n* Qualitatively evaluating the rectified items for a few instances in different dataset would further help the audience understanding the behavior of the proposed approach.\n\nQUESTIONS:\nPlease see the suggestions in the \"weakness\" section.\n\nLIMITATIONS:\nNo limitation is mentioned in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a bidirectional data rectification framework, called BirDRec, to address the challenges of training sequential recommender systems (SRSs) with unreliable input and targets. The authors provide two theoretically guaranteed error-bounded strategies to rectify the data, which can be flexibly implemented with most existing SRSs. They also introduce a rectification sampling strategy and self-ensemble mechanism to reduce the computational and space complexity of BirDRec. The proposed framework is evaluated on several benchmark datasets, and the results demonstrate its generality, effectiveness, and efficiency compared to state-of-the-art robust SRSs.\n\nSTRENGTHS:\n1. The paper introduces BirDRec, a novel bidirectional data rectification framework designed to enhance the robustness of sequential recommender systems (SRSs) by addressing unreliable data. BirDRec can seamlessly integrate with existing SRSs, offering a flexible solution for improving their performance.\n2. The authors present two error-bounded strategies within BirDRec that provide theoretical guarantees for detecting and rectifying unreliable input and targets in SRSs. These strategies contribute to the reliability and accuracy of the recommendations generated by the system.\n\nWEAKNESSES:\n1. The evaluation of the proposed framework is limited to a few benchmark datasets, and it remains uncertain how well it would generalize to different types of data or domains. It would be beneficial for the authors to include further experiments or discuss potential challenges in applying their framework to diverse datasets.\n2. While the authors attribute the unreliable data to external distractions, it is important to acknowledge that there might be other sources of noise or errors that their approach does not account for. Including a discussion on potential alternative causes of unreliable data would strengthen the paper's analysis.\n3. The paper lacks a thorough analysis of the computational and space complexity of the proposed framework. Understanding the resource requirements of the framework is crucial for assessing its scalability to larger datasets or real-time applications. It would be valuable for the authors to provide insights into the framework's efficiency and discuss any potential limitations in terms of computational resources.\n\nQUESTIONS:\n1. Could you provide additional details on the theoretical guarantees of the error-bounded strategies proposed for data rectification? \n2. Can you explain the process behind selecting the benchmark datasets used in your experiments? What criteria did you consider when choosing these datasets? Additionally, could you elaborate on the evaluation metrics or performance measures used to assess the effectiveness of the proposed framework on these benchmarks?\n3. Have you taken into account other sources of noise or errors commonly encountered in SRSs, such as cold-start, diversity, or fairness issues? If so, how does your approach address or mitigate these challenges? \n4. In real-world applications, how do you envision the proposed framework being utilized? Are there specific domains or use cases where you believe BirDRec would be particularly beneficial? Additionally, what are some potential challenges or limitations that might hinder the adoption or scalability of your framework in practical settings? It would be valuable to understand the feasibility and potential constraints of implementing BirDRec in real-world scenarios.\n\nLIMITATIONS:\nThe authors missed an opportunity to compare their approach with other recent methods that address the problem of unreliable data in SRSs. By including such a comparison, the authors could provide a more comprehensive evaluation of their proposed framework and highlight its advantages or limitations in relation to existing solutions. This addition would enhance the paper's contribution to the field.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

comments: SUMMARY:
The authors proposed Bidirectional Data Rectification (BirDRec) framework that can be incorporated into existing Sequential recommender systems (SRSs) to tackle unreliable data. In addition, the authors also adopt rectification sampling strategy.

The authors also provided theoretical guaranteed for BirDRec and conducted various experiments on the robustness of the framework.

STRENGTHS:
Strengths:

-	The idea is straightforward and easy to follow
-	The authors provided theoretical guaranteed for BirDRec and conducted various experiments on the robustness of the framework.
-	Theoretical aspects are well-written
-	The authors also provided source code for verification.

WEAKNESSES:
Weaknesses:

-	Some of the experimental results are not convincing such as the performance of baselines in Table 3. The authors need to provide more explanation and clarification such as why BERD is good for Movielens1M but STEAM is better for Yelp.
-	It’s not clear which parts of the architecture in Figure 2 made the performance increases dramatically
-	Comparing BirDRec and SASRec for hyper-parameters analysis in Appendix C.2 and C.3 seems a bit ‘unfair’. In my opinion, we should compare with most recent baselines, so at least we can compare with Figure 5 in the main paper.
-	How much time did it take to do grid search across models in Table 1 from Appendix Page 13?

QUESTIONS:
Please refer to my comments above

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the inconsistency between user’s true preference with interaction history. Specifically, the authors propose a theoretically guaranteed data rectification strategies based on SRS predictions to tackle both unreliable input and targets for more robust SRS. Besides, the authors also devise a rectification sampling strategy and adopt a self-ensemble mechanism to ensure better scalability. Experiments on four real-world datasets show the superiority of the proposed method.\n\nSTRENGTHS:\n1)\tThe research problem of rectification of interaction data is very interesting.\n\n2)\tTheorem analysis for the proposed modules. \n\n3)\tThe proposed model can be compatible with most existing models.\n\n4)\tExtensive experiments with respect to effectiveness and efficiency are conducted.\n\nWEAKNESSES:\n1)\tSome details are not clear, for example, what’s the formulation of the Self-ensemble Mechanism?\n\n2)\tThe time comparison is important is important for real application. However, Figure 3 compares SASRec, STEAM, and BirDRec simultaneously, which makes the increasing of time comparing SASRec and BirDRec not clear. More analysis is necessary. \n\n3)\tThe experimental datasets are all small datasets with less than 100K users and items. Besides, the improvements are too large which is not very common.\n\n4)\tNo online results are given. Too much hyper-parameters, which limit the applicability of the model.\n\n5)\tMany parameters are pre-defined, without careful tuning.\n\nQUESTIONS:\n1)\tTheorem 4 indicates that the relative rank of an item over a list of randomly sampled items can approximate this item’s relative rank over the full item set. However, if K is too smaller than |V|, the randomly sampled items may still introduce so many noises?\n\n2)\tThe space complexity reduction of Self-ensemble Mechanism if from K*H to K, but not from |V|*H to K?\n\nLIMITATIONS:\n1)\tThe proposed model seems to be too complicated to be deployed in real industrial environments.\n\n2)\tBesides sequential recommendation, can this model employed in other recommendation tasks, for example, rating prediction, CTR prediction?", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-09"}