Self-supervised Learning for Sequential Recommendation with Model Augmentation
Authors: Zhiwei Liu
Yongjun Chen
Jia Li
Man Luo
Philip S. Yu
Caiming Xiong
Venue: ICLR 2022 Submitted (2022)
Keywords: Sequential Recommendation
Self-supervised Learning
Contrastive Learning
Model Augmentation

Abstract: The sequential recommendation aims at predicting the next items in user behaviors, which can be solved by characterizing item relationships in sequences. Due to the data sparsity and noise issues in sequences, a new self-supervised learning (SSL) paradigm is proposed to improve the performance, which employs contrastive learning between positive and negative views of sequences. 
However, existing methods all construct views by adopting augmentation from data perspectives, while we argue that 1) optimal data augmentation methods are hard to devise, 2) data augmentation methods destroy sequential correlations, and 3) data augmentation fails to incorporate comprehensive self-supervised signals. 
Therefore, we investigate the possibility of model augmentation to construct view pairs. We propose three levels of model augmentation methods: neuron masking, layer dropping, and encoder complementing. 
This work opens up a novel direction in constructing views for contrastive SSL. Experiments verify the efficacy of model augmentation for the SSL in the sequential recommendation. 


Under review as a conference paper at ICLR 2022
SELF-SUPERVISED LEARNING FOR SEQUENTIAL REC- OMMENDATION WITH MODEL AUGMENTATION
Anonymous authors Paper under double-blind review
ABSTRACT
The sequential recommendation aims at predicting the next items user behav- iors, which be solved by characterizing item relationships in sequences. Due to the data sparsity and noise issues sequences, a new self-supervised learning (SSL) paradigm proposed to improve the performance, which employs con- trastive learning between positive and negative views of sequences. However, existing methods all construct views by adopting augmentation from data per- spectives, while we argue that 1) optimal data augmentation methods are hard to devise, 2) data augmentation methods destroy sequential correlations, and 3) data augmentation fails to incorporate comprehensive self-supervised signals. There- fore, we investigate the possibility of model augmentation to construct view pairs. We propose three levels of model augmentation methods: neuron masking, layer dropping, and encoder complementing. This work opens up a novel direction constructing views for contrastive SSL. Experiments verify the efficacy of model augmentation for the SSL in the sequential recommendation.
INTRODUCTION
The sequential recommendation (Fan et al. 2021 Liu et al. 2021c Chen et al. 2018 Tang &
Wang 2018 Zheng et al. 2019) aims at predicting future items in sequences, where the crucial part to characterize item relationships in sequences. Recent developments in sequence modeling (Fan ct al. 2021 Liu et al. 2021ch verify the superiority of Transform (Vaswani et al.) 2017). i.e. the self- aftention mechanism, revealing item correlations in sequences. A Transformer (Kang & McAuley
2018) is able to infer the sequence embedding at specified positions by weighted aggregation of item
embeddings. where the weights are learned via self-attention. Existing works (Fan et al. 2021 Wu
et al. 2020) further improve Transformer by incorporating additional complex signals.
However, the data sparsity issue (Liu al. 2021c) and noise in sequences undermine the perfor- mance of model in sequential recommendation. The former hinders performance due to insuf- ficient training since the complex structure of sequential model requires dense corpus to be adequately trained. The latter also impedes the recommendation ability of model because noisy item sequences are unable to reveal actual item correlations. To overcome both, new contrastive self-supervisc learning (SSL) paradigm (Liu et al.) 2021b; Xie et al.) 2020 Zhou et al., (2020) is proposed recently. This paradigm enhances the capacity of encoders by leveraging additional self-supervised signals. Specifically, the SSL paradigm constructs positive view pairs as two data augmentations from the same sequences (Xie et al. 2020), while negative pairs are augmentations from distinct sequences. Incorporating augmentations during training increases the amount of train- ing data, thus alleviating the sparsity issue. And the contrastive loss (Chen et al. 2020) improves the robustness of the model, which endows model with the ability against noise.
Though being effective enhancing sequential modeling, the data augmentation methods adopted in the existing SSL paradigm suffer from the following weaknesses:
Optimal data augmentation methods are hard to devise. Current sequence augmentation meth- ods adopts random sequence perturbations (Liu et al. 2021b Xie et al.] 2020 which includes crop, mask reorder, substitute and insert operations. Though a random combination of those aug- menting operations improves the performance, it is rather time-consuming search the optimal

---

Under review as a conference paper at ICLR 2022
augmentation methods from large number of potential combinations for different datasets et al. 2021b).
Liu
Data augmentation methods destroy sequential correlations, leading to less confident positive pairs. The existing SSL paradigm requires injecting perturbations into the augmented views of sequences for contrastive learning. However, because the view construction process is not opti- mized to characterize sequential correlations, two views of one sequence may reveal distinct item relationships, which should not be recognized positive pairs.
Data augmentation fails to incorporate comprehensive self-superrvsed signals. Current data aug- mentation methods are designed based on heuristics, which already requires additional prior knowledge. Moreover, since the view construction process is not optimized with the encoder, data augmentation may only reveal partial self- self-supervised signals from data perspectives. Hence, we should consider other types of views besides data augmentation.
Therefore, we investigate the possibility of model augmentation to construct view pairs for con- trastive learning, which functions complement to the data augmentation methods. We hypothe- sis that injecting perturbations into the encoder should enhance the self-supervised learning ability to existing paradigms. The reasons are threefold: Firstly, model augmentation is jointly trained with the optimization process, thus endows the end-to-end training fashion. As such, it is easy to discover the optimal view pairs for contrastive learning. Secondly, model augmentation constructs views without manipulation to the original data, which leads to high confidence of positive pairs. Last but not least, injecting perturbation into the encoder has distinct characteristics to data augmentation, which should be important complement in constructing view pairs for existing self-supervised learning scheme (Liu et al. 2021b; Zhou et al. 2020).
This work studies the model augmentation for self-supervised sequential recommendation from three levels: 1) neuron masking (dropout), which adopts the dropout layer to randomly mask par- sequence, we can perturb the output of the embedding from this layer, which thus constructs two views from model augmentation per- spective (Gao al.] 2021). 2) layer dropping. Compared with neuron masks, we randomly drop a complete layer in the encoder to inject more perturbations. By randomly dropping layers in an encoder twice, we construct two distinct views. Intuitively, layer-drop augmentation enforces the contrast between deep features and shallows features of the encoder. 3) encoder complementing, which leverages other encoders to generate sequence embeddings. Encoder complementing aug- mentation is able to fuse distinct sequential correlations revealed by different types of encoders. For example, RNN-based sequence encoder (Hidasi et al.) 2015) can better characterize direct item transition relationships, while Transformer-bise based sequence encoder models position-wic sequen- tial correlations. Though only investigating SSL. for a sequential recommendation, we remark that model augmentation methods can also be applied in other SSL scenarios. The contributions as follows:
tial neurons in layer. By operating the dropout twice
We propose new contrastive SSL paradigm for sequential recommendation by constructing views from model augmentation, which is named SRMA.
We introduce three levels of model augmentation methods for constructing view pairs.
We discuss the effectiveness and conduct comprehensive study of model augmentations for the sequential recommendation.
We investigate the efficacy of different variants of model augmentation.
2
RELATED WORK
2.1
SEQUENTIAL RECOMMENDATION
Sequential recommendation predicts future items in user sequences by encoding sequences while modeling item transition correlations (Rendle et al., 2010 Hidasi et al. (013).. Previously, Recurrent Neural Network (RNN) have been adapted sequential recommendation (Hidasi et al. 2015 Wu
et al. 2017), ostensibly modeling sequence-level item transitions. Hierarchical RNNs (Quadrana
al. 2017 incorporate personalization information. Moreover, both long-term and short-term item
2

---

Under review as a conference paper at ICLR 2022
transition correlations modelled in LSTM (Wu et al. attention models (Vaswani et al. 2017 Devlin et al. 2018) promotes the prosperity of Transformer- based sequential recommendation models. SASRec (Kang & McAuley 2018) is pionecring work adapting Transformer to characterize complex item transition correlations. BERT4Rea (Sun et al,, 2019) adopts the bidirectional Transformer layer encode sequence. ASReP (Liu et al.) (222cc) reversely pre-training Transformer to augment short sequences and fine-tune it to predict the next- item in sequences. TGSRec (Fan al. 2021 models temporal collaborative signals in sequences to recognize item relationships.
2017 Recently, the success of self-
2.2 SELF-SUPERVISED LEARNING
Self-supervised learning (SSL) is proposed recently to describe "the machine predicts any parts of its input for any observed part" (Bengio al., 202T), which stays within the narrow scope of unsu- pervised learning. To achieve the self-prediction, endeavors from various domains have developed different SSL schemes from either generative
contrastive perspectives (Liu et al. 202 For generative SSL. the masked language model is adopted in BERT (Devlin et al., 2018) to generate masked words sentences. GPT-GNN (Hu et al.] (020)) also generates masked edges to realize SSL. Other generative SSL paradigms in computer vision (Oord et al. 2016) are proposed. Compared with generative SSL., contrastive SSL schemes have demonstrated more promising performance. SimCLR (Chen al.) 2020) proposes simple contrastive learning between augmented views for im- ages, which is rather effective achieving SSL. GCC (Qiu et al., 2020) and GraphCL YYou et al.) 2020) adopts contrastive learning between views from corrupted graph structures. CL4SRec (Xie et al. 2020) and CoSeRec (Liu et al., 2021b) devise the sequence augmentation methods for SSL sequential recommendation. This paper also investigates the contrastive SSL for sequential recommendation. Instead of adopting the data augmentation for constructing views to contrast, we propose the model augmentation generate contrastive views.
PRELIMINARY
3.1 PROBLEM FORMULATION
We denote user and item sets as L/ and y respectively. Each user E l/ is associated with sequence of items in chronological order 8n [V]..... , ..., ,.... where oz E denotes the item that has interacted with time and |s,| is the total number of items. Sequential recommendation is formulated as follows:
arg max P(u su +1 = 04 su)) VVV
(1)
where .|,,|+1 denotes the next item in sequence. Intuitively, we calculate the probability of all candidate items and recommend items with high probability scores.
3.2 SEQUENTIAL RECOMMENDATION FRAMEWORK
The of generic sequential recommendation framework is sequence encoder SeqEnc(-), which transforms item sequences to embeddings for scoring. We formulate the encoding step as:
h = SeqEnc(s..,,
where h. denotes the sequence embedding of sa. To be specific, if we adopt a Transformer (Kang & McAuley 2018 Vaswani et al. 2017) as the encoder, h, is bag of embeddings, where at each position hi represents a predicted next-item. We adopt the log-likelihood loss function to optimize the encoder for next-item prediction as follows:
Lree 1) = log(a(h' e).
Σ log(1 - -hh,, en,)),
Heu
where Cree (u, f) denotes the loss score for the prediction at position in sequence su, o is the non- linear activation function, Cr... denotes the embedding for item Ut+1, and uy is the sampled negative item for s,, The embeddings of items are retrieved from the embedding layer in SeqEnc(-), which jointly optimized with other layers.
3

---

Under review as a conference paper at ICLR 2022
Contrast
Embedding
Embedding
Output
Output
Concat
Concat
Model Augmentation
Model Augmentation
Add Norm
FFN
FFN
Encader
Encoder
Neuron Mask
FFN
Pre-trained Encoder
Augmented Soq.S,
Augmented
FFN
Encoder
Encoder
Input Embedding
Sequence
Sequence
Original Sequence
(a) Contrastive Seff-supervised Learning
(b) Neuron Masking
Layer Dropping
(d) Encoder Complementing
(a) The contrastive SSL framework with model augmentation. We apply the model aug- mentation to the encoder, which constructs two views for contrastive learning. (b) the neuron mask- ing augmentation. We demonstrate the neuron masking for the Feed-Forward network. (c) the layer dropping augmentation. We add K FFN layers after the encoder and randomly drop M lay- ers (dash blocks) during each batch of training. And (d) the encoder complementing augmentation. We pre-train another encoder for generating the embedding of sequences. The embedding from the pre-trained encoder combined with the model encoder for contrastive learning.
Figure 1:
3.3 CONTRASTIVE SELF-SUPERVISED LEARNING PARADIGM
Other than the next-item prediction, can leverage other pretext tasks (Sun et al. 2019 Liu al. 2021a b) over the sequence to optimize the encoder, which harnesses the self-supervised signals within the sequence. This paper investigate the widely adopted contrastive SSL scheme (Liu al.) 2021b Xie et al.) (020). This scheme constructs positive and negative view pairs from sequences, and employs the contrastive loss (Oord al. 2018) to optimize the encoder. We formulate the SSL step as follows:
Geal (1 hew) = log exp(sim(hau.) hzw))
(4)
m=1 m exp( (ppps h22- hm))
where h2. and h denotes two views constructed for sequence s.. I is indication function. .im(..) is the similarity function, e.g. dot-product. Since each sequence has two view, we have 2N samples in batch with N sequences for training. The nominator indicates the agreement maximization between a positive pair, while the denominator is interpreted as push away those negative pairs. Existing works apply data augmentation for sequences construct views, e.g. Xie et al. (220)) propose crop, mask, and reorder a sequence and (Liu et al. 202 devises insert and substitute sequence augmentations. For sequential recommendation, since both SSL and the next- item prediction charaterize the item relationships in sequences, we add them together to optimize the encoder. Therefore, the final loss L Lree + XL Compared with them, we adopt both the data augmentation and model augmentation to generate views for contrast. We demonstrate the contrastive SSL step in Figure a).
MODEL AUGMENTATION
In this section, we introduce the model augmentation construct views for sequences. We dis- cuss three type of model augmentation methods, which are neuron mask, layer drop and encoder complement. We illustrate these augmentation methods in Figure [bb []). respectively.
4.1 NEURON MASKING
This work adopts the Transformer the sequence encoder, which passes the hidden embeddings to the next layer through a feed-forward network (FFN). During training, we randomly mask partial neurons in each FFN layer, which involves masking probability p. The large value of p leads to intensive embedding perturbations. As such, we generate a pair of views from one sequence from model perspectives. Besides, during each batch of training, the masked neurons randomly
4

---

Under review as a conference paper at ICLR 2022
selected, which results in comprehensive contrastive learning model augmentation. Note that, though we can utilize different probability values for distinct FFN layers, we enforce their neuron masking probability to be the same for simplicity. The neuron masking augmentation FFN is shown in Figure [b). Additionally, we remark that the neuron mask can be applied to any neural layers in model inject more perturbations.
4.2 LAYER DROPPING
Dropping partial layers of model decreases the depth and reduces complexity, Previous research gues that most recommender systems require only shallow embeddings for users and items (Dacremal al. 2019). Therefore, it reasonable to randomly drop fraction of layers during training. which functions as way of regularization. Additionally, existing works (Liu et al., 20200] He et al.) 2016) claim that embeddings at shallow layers and deep layers are both important reflect the compre- hensive information of the data. Dropping layers enable contrastive learning between shallow em- beddings and deep embeddings, thus being an enhancement of existing works that only contrasting
between deep features.
On the other hand, dropping layers, especially those necessary layers a model, may destroy orig- inal sequential correlations. Thus, views generating by dropping layers may not be positive pair. To this end, instead of manipulating the original encoder, we stack K FFN layers after the encoder and randomly drop M of them during each batch of training, where M K. We illustrate the layer dropping as in Figure [c). where we append K additional FFN layers after the encoder and use dash blocks denote the dropped layers.
4.3 ENCODER COMPLEMENTING
During self-supervised learning, employ one encoder to generate embeddings of two views of one sequence. Though this encoder can be effective in revealing complex sequential correlations, contrasting on one single encoder may result in embedding collapse problems for self-supervised learning (Hua et al. (022). Moreover, one single encoder is only able to reflect the item relationships from unitary perspective. For example, the Transformer encoder adopts the attentive aggregation of item embeddings to infer sequence embedding, while an RNN structure (Hidasi et al. 2015) is more suitable in encoding direct item transitions. Therefore, contrasting between views from distinct encoders enables the model to learn comprehensive sequential relationships of items.
However, embeddings from two views of a sequence with distinct encoders lead to a non-Siamese paradigm for self-supervise learning, which is hard to train and suffers the embedding collapse problem (Koch et al.) 20155 Chen & He (021I). Additionally, if two distinct encoders reveal sig- nificantly diverse sequential correlations, the embeddings are far away from each other, thus being bad views for contrastive learning (Tian et al., (020).. Moreover, though we can optimize two en- coders during training phase, it still problematic combine them for the inference of sequence embeddings to conduct recommendations.
As result, instead of contrastive learning with distinct encoders, we harness another pre-trained encoder as an encoder complementing model augmentation for the original encoder. To be more specific, we first pre-train another encoder with the next-item prediction target. Then, in the self- supervised training stage, we utilize this pre-trained encoder to generate another embedding for view. After that, we add the view embeddings from a model encoder and the pre-trained encoder. We illustrate the encoder complementing augmentation in Figure Note that we only apply this model augmentation in one branch of the SSL paradigm. And the embedding from the pre-trained encoder is re-scaled by hyper-parameter encoder. The smaller value of y implies injecting fewer perturbations from a distinct encoder. And the parameters of this tre-trained encoder are fixed during training. Hence, there is optimization for this pre-trained trained encoder and it is no longer required to take account of both encoders during the inference stage.
before adding to the embedding from the framework's s
5

---

Under review as a conference paper at ICLR 2022
5 EXPERIMENTS
5.1 EXPERIMENTAL SETTINGS
Dataset We conduct experiments on three public datasets. Amazon Sports, Amazon Toys and Games (McAuley et al. 2015) and Yelp" which are Amazon review data in Sport and Toys cate- gories, and dataset for the business recommendation, respectively. We follow common practice in (Liu al. 2021e| Xie et al. 2020 to only keep the '5-core' sequences. In total, Sports dataset has 35,598 users, 18,357 items and 296,337 interactions. Toys dataset contains 19,412 users, 11,92 11,924 items, and 167,597intrractions. Yelp dataset consists 30,431 users, 20,033 items and 316.354 inter- actions.
Evaluation Metries We follow (Wang et al. 2019 Krichene & Rendle 2020 Liu al. 202 c to
evaluate models' performances based on the whole item set without negative sampling and report standard Hit Ratio@k (HR @k) and Normalized Discounted Cumulative Gain@k (NDCG@)) on all datasets, where & E (5, 10, 20}.
Baselines We include two groups of sequential models as baselines for comprehensive compar- isons. The first group baselines are sequential models that use different deep neural architectures to encode sequences with supervised objective. These include GRU4Rec (Hidasi et al. 2015) as an RNN-based method, Caser (Tang& & Wang] 2018) CNN-based approach, and SASRec (Kang & McAuley 2018) as one of the state-of-the-ie art Transformer based solution. The second group base- lines additionally leverage SSL objective. BERT4Rec (Sun et al. 2019) employs Cloze task (Ta-- lor 1953) as a generative self-supervised learning sigal. Ree (Zhou et al. (020) uses contrastive SSL with 'mask' data augmentation to fuse correlation-ships among item, sub-sequence, and cor- respondinng attributes into the networks. We remove the components for fusing attributes for fair comparison. CL4SRec (Xie al. 2020) maximize the agreements between two sequences augmen- lation, where the data augmentation are randomly selected from 'crop', 'reorder', and 'mask' data augmentations. CoSeRec (Liu et al.) 2021b) improves the robustness of data augmentation under contrastive SSL framework by leveraging item item-correlan cotm---orraaaioss...
Implementation Details The model encoder in SRMA is the basic Transformer- encoder. We adopt the widely used SASRec encoder. The neuron masking probability is searched from (0.0, 0.1, o. 0.9). For layer dropping, the K searched from 2, 3, 4), and M is searched accordingly. As for encoder complementing, we search the re-scale hyper-parameter {0.005, 0.01, 0.05, 0.1, 0.5, 1.0} and the pre-trained encoder is selected from a I-layer Transformer and GRU encoder.
2,
from
5.2 OVERALL PERFORMANCE
We compare the proposed paradigm SRMA to existing methods with respect to the performance on the sequential recommendation. Results are demonstrated in Table [ We can observe that Transformer-basee sequence encoders, such SASRec and BERT4Rec are better than GRU4Rec or Caser sequence encoders. Because of this, our proposed model SRMA also adopts the Transformer sequence encoder. Moreover, the SSL paradigm example, the CL4SRec model, which adopts the random data augmentation, improves the perfor- mance of SASRec HR and NDCG by 13. 13.2% and 9.8% average regarding the Sports dataset, respectively. Also, since SRMA enhances the SSL with both data augmentation and model augmen- tation, SRM MA thus outperforms all other SSL. sequential recommendation models. SRMA adopts the same data augmentation methods as CL4SRec. Nevertheless, SRMA significantly outperforms CL4SRec. On the sports dataset, achieve 18.9% and 27.9% relative improvements on HR and NDCG, respectively. On the Yelp dataset, we achieve 4.5% and 6.4% relative improvements on HR and NDCG, respectively. And on Toys data, we achieve 8.6% and 13.8% relative improvements on HR and NDCG, respectively. In addition, SRMA also performs better than CoSeRec which lever- ages item correlations for data augmentation. Those results all verify the effectiveness of model augmentation in improving the SSL paradigm.
significantly improve performance. For
hhtps://www..ylpp.co/aaeee

---

Under review as a conference paper at ICLR 2022
Table I: Performance comparisons of different methods. The best score is in bold in each row, and the second best is underlined.
Dataset
Metric
GRU4Rec
Caser
SASRec
BERT4Rec
S CL4SRec CoSeRec
SRMA
HR @5
0.0162
0.0154
0.0206
0.0217
0.0121
0,0231
0.0287
0.0299
HR @ 10
0.0258
0.0261
0.0320
0.0359
0.0205
0.0369
0.0437
0.0447
HR @ 20
0.0421
0.0399
0.0497
0.0604
0.0344
0.0557
0.0635
0.0649
Sports
N
0.0103
0.0114
0.0135
0.0143
0.0084
0.0146
0.0196
0.0199
NDCG@ 10
0.0142
0.0135
0.0172
0.019
0.0111
0.0191
0.0242
0.0246
NDCG@20
0.0186
0.0178
0.0216
0.0251
0.0146
0.0238
0.0292
0.0297
HR @5
0.0152
0.0142
0.0160
0.0196
0.0101
0.0227
0.0241
0.0243
HR @ 10
0.0248
0.0254
0.0260
0.0339
0.0176
0.0384
0.0395
0.0395
@
0.0371
0.0406
0.0443
0.0564
0.0314
0,0623
0.0649
0,0646
Yelp
NDCG@S
0.0091
0.008
0.0101
0.0121
0.0068
0.0143
0.0151
0.0154
NDCG@10 10
0.0124
0.0113
0.0133
0.0167
0.0092
0.0194
0.0205
0.0207
NDCG@20
0.0145
0.0156
0.0179
0.0223
0.0127
0.0254
0.0263
0.0266
HR@5
0.0097
0.0166
0.0463
0.0274
0.0143
0.0525
0.0583
0.0598
@R@10
0.0176
0.0270
0.0675
0.0450
0.0094
0.0776
0.0812
0.0834
#R@20
0.0301
0.0420
0.0941
0.0688
0.0235
0.1084
0.1103
0.1132
Toys
NDCG@5
0.0059
0.0107
0.0306
0.0174
0.0123
0.0346
0.0399
0.0407
NDCG@ 10
0.0084
0.0141
0.0374
0.0231
0.0391
0.0428
0.0473
0.0484
NDCG@20
0.0116
0.0179
0.0441
0.0291
0.0162
0.0505
0.0547
0.0559
5.3 COMPARISON BETWEEN MODEL AND DATA AUGMENTATION
Because SRMA adopts the random sequence augmentation, we mainly focus on comparing with CL4SRec to justify the impacts of model augmentation and data augmentation. In fact, CL4SRec also implicitly uses the neuron masking model augmentation, where dropout layers are stacked within its original sequence encoder. To separate the joint effects of model and data augmentation, we create its variants 'CL4S. p= 0', which sets all the dropout ratios to be O, thus disables the neu- masking augmentation. Also, another variant 'CL4S. w/o D', which has no data augmentation are also compared. Additionally, we create two other variants of SRMA as 'SRMA w/o M' and MSRMA w/o D' by disabling the model augmentation and data augmentation respectively. 'SRMA w/o M' has additional FFN layers compared with 'CL4S, = 0'. The recommendation perfor- mance on the Sports and Toys dataset is presented in Table 2 We have the following observations. Firstly, we notice significant performance drop of the variant 'CL4S. the neuron masking augmentation is rather crucial. It benefits both the regularization the training encoder and model augmentation of SSL. Secondly, 'SRMA w/o D' outperforms other baselines on the Sports dataset and has comparable performance to 'CL4S.', which indicates model augmen- tation is of more impact in the SSL paradigm compared with data augmentation. Thirdly, SRMA performs the best against all the variants. This result suggests that we should jointly employ the data augmentation and model augmentation in SSL paradigm, which contributes to comprehensive contrastive self-supervised signals.
0', which suggests that
5.4 HYPER-PARAMER PARAMETER SENSITIVITY
In this section, we vary the hyper-parametert in neuron masking and layer dropping to draw de- tailed investigation of model augmentation.
Effect of Neuron Masking. Though all neural layers can apply the neuron masking augmenta- tion, for simplicity, we only endow the FFN layer with the neuron masking augmentation and set the masking probability p for all FFN layers in the framework. We fix the settings of layer drop- ping and the encoder complementing model augmentation and select from {0.0, 0.1, 0.2, 0.9), where 0.0 is equivalent to neuron masking. Also, compare SRMA with SASRec justify the effectiveness of the SSL paradigm. The performance curves of HR@5 and NDCG@5 on the Sports and Toys dataset demonstrated in Figure? 2 We can observe that the performance improves first

---

Under review as a conference paper at ICLR 2022
Table 2: Performance comparison w.r.t. the variants of CLASRec (CLAS.) and SRMA. M and D denote the model augmentation and data augmentation, respectively. = 0 indicates no neuron masking. The best score in each column are in bold, where the second-best are underlined.
Sports
Toys
Model
HR
NDCG
HR
NDCG
@5
@10
@5
@ 10
@5
@ @10
@5
@ 10
CL4S. w/o D
0.0162
0.0268
0.0108
0.0142
0.0444
0.0619
0.0306
0.0363
CL4S. =
0.0177
0.0292
0.0119
0.0156
0.0451
0.0654
0.0305
0.037
CL4S.
0.0231
0.0369
0.0146
0.0191
0.0525
0.0776
0.0346
0.0428
SRMA w/o D
0.0285
0.0432
0.0187
0.0234
0.0504
0.0724
0.0331
0.0402
SRMA w/o M
0.0165
0.0272
0.0104
0.0138
0.0412
0.0590
0.0279
0.0336
SRMA
0.0299
0.0447
0.0199
0.0246
0.0598
0.0834
0.0407
0.0484
0.03
0.02
0.06
0.04
NDCG@s
NDCG@5
HR@5
0.02
HR@5
0.04
0.01
0.02
0.01
0.02
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
p
Sports
Toys
Toys
Figure/ 2: Performance comparison betweeen SASRec and SRMA in HR@5 and NDCG@5 w.r.t different values of neuron masking probability p on Sports and Toys dataset.
and then drops when increasing p from 0 to 0.9. The rising of the performance score implies that the neuron masking augmentation is effective in improving the ability of the sequence encoder for a recommendation. And the dropping of the performance indicates the intensity of model augmenta- tion should not be overly aggressive, which may lead to less informative contrastive learning. As to SASRec, we recognize higher score of SASRec when p large, which indicates the optimal model augmentation should be a slightly perturbation rather than a intensive distortion. Moreover, SRMA consistently outperforms SASRec when 0.1 < p < 0.6. Since the only difference that SASRec has no SSL module, we can thus concludes that the performance gains result from the contrastive SSL step by using the neuron masking.
Effect of Layer Dropping. The layer dropping model augmentation is controlled by two hyper- parameters, the number of additional FFN layers and the number of layers to drop during training, which is denoted as K and M, respectively. Since we can only drop those additional layers, we have M < K. We select K from {1, 2, 4} while M are searched accordingly. Due to space limitation, we only report the NDCG@5 on the Sports and Toys dataset in Figure 3 We observe that K = 2, .M = 1 achieves the best performance on both datasets, which implies the efficacy of layer dropping. Additionally,
also find that the performance on K = 4 is consistently worse than K = on both datasets, which suggests that adding too many layers increases the complexity of the model, which is thus unable enhance the SSL paradigm.
5.5 ANALYSES ON ENCODER COMPLEMENTING
In this section, we investigate the effects of encoder complementing augmentation for constructing views. Recall that we combine the embedding from the model's encoder and distinct pre-trained encoder. For this complementary encoder, we select from a Transformer-bant based and GRU-based encoder. Since the model's encoder a 2-layer Transformer, this pre-trained encoder is 1-layer Transformer to maintain diversity. We first pre-train the complementary encoder based on the next- item prediction task. As such, we empower the pre-trained encoder to characterize the sequential correlations of items. The comparison is conducted shown in Table 3, The observations are as follows: Firstly, on the Sports dataset, pre- pre-training a GRU encoder complement performs the best against the other two, which indicates that injecting dis-
both Sports and Toys datasets, which
8

---

Under review as a conference paper at ICLR 2022
Sports
Toys
0.020
0.035
NDCG@S
NDCG8S
0.015
Km1 Km2
Km3
Km4
Km1
Km2
Km3
Km4
Figure 3: The NDCG@5 performance w.r.t. different K and for layer dropping augmentation on Sports and Toys dataset.
Table 3: Performance comparison among SRMA without encoder complementing (w/o Enc.), with Transformer based (-Trans) and with GRU-based (-GRU) complementary pre-trained encoder. The best score in each column is in bold.
Sports
Toys
Encoders
HR
NDCG
HR
NDCG
@5
@ 10
@5
@ 10
@5
@ 10
@5
10
w/o Enc.
0.0269
0.0401
0.0181
0.0224
0.0567
0.0806
0.0389
0.0466
-Trans
0.0268
0.0408
0,0181
0.0226
0.0588
0.0811
0.0402
0.0474
-GRU
0.0281
0.0411
0.0186
0.0228
0.0577
0.0811
0.0395
0.047
tinct encoders for contrastive learning enhance the SSL signals. Secondly, the Toys dataset, adopting a 1-layer pre-trained Transformer as the complementary encoder yields the best scores on all metrics. Besides the effectiveness of encoder complementing, this result also suggests that the complementary encoder may not be overly different from the model's encoder on some datasets, which otherwise cannot enhance the comprehensive contrastive learning between views. Lastly, both Transformer-basee and GRU-based pre-trained complementary encoders consistently outper- form SRMA without encoder complementing, which directly indicates the necessity of encoder complementing way of model augmentation.
CONCLUSION
This work proposes novel contrastive self-supervised learning paradigm, which simultaneously employs model augmentation and data augmentation to construct views for contrasting. We pro- pose three-level model augmentation methods for this paradigm, which are neuron masking, layer dropping, and encoder complementing. We adopt this paradigm to the sequential recommendation problem and propose new model SRMA. This model adopts both the random data augmentation of sequences and the corresponding three-level model augmentation for the sequence encoder. We conduct comprchensive experiments to verify the effectiveness of SRMA. The overall performance comparison justifies the advantage contrastive SSL. with model augmentation. Additionally, de- tailed investigation regarding the impacts of model augmentation and data augmentation in improv- ing the performance are discussed. Moreover, ablation studies with respect to three-level model augmentation methods are implemented, which also demonstrate the superiority of the proposed model. Overall, this work opens up a new direction in constructing views from model augmenta- tion. We believe model augmentation can enhance existing contrastive SSL paradigms with only data augmentation.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Yoshua Bengio, Yann Lecun, and Geoffrey Hinton. Deep learning for ai. 64(7), 2021. ISSN 0001- 0782.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. International conference on machine learning, pp. 1597-1607. PMLR, 2020.
Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15750-15758, 2021.
Xu Chen, Hongteng Xu, Yongfeng Zhang, Jiaxi Tang, Yixin Cao, Zheng Qin, and Hongyuan Zha. Sequential recommendation with user memory networks. In WSDM, pp. 108-116, 2018.
Maurizio Ferrari Dacrema, Paolo Cremonesi, and Dietmar Jannach. Are really making much progress? a worrying analysis recent neural recommendation approaches. In Proceedings of the 13th ACM Conference Recommender Systems, pp. 101-109, 2019.
2019..
Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04800 2018.
Ziwei Fan, Zhiwei Liu, Jiawei Zhang, Yun Xiong, Lei Zheng, and Philip S. Yu. Continuous-time sequential recommendation with temporal graph collaborative transformer. In Proceedings of the 30th ACM International Conference on Information and Knowledge Management. ACM, 2021.
Tianyu Gao, Xingcheng Yao, and Danqi Chen. Simcse: Simple contrastive learning of sentence
embeddings. arXiv preprint arXiv:: 2.Xiv:21044.00 08821, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based rec- ommendations with recurrent neural networks. arXiv preprint 151G::55110009333 2015.
Ziniu Hu, Yuxiao Dong, Kuansan Wang, Kai- Wei Chang, and Yizhou Sun. Gpt-gnn: Generative pre-training of graph neural networks. Proceedings of the 26th ACM SIGKDD International Conference Knowledge Discovery Data Mining, pp. 1857-1867, 2020.
Tianyu Hua, Wenxiao Wang, Zihui Xue, Yue Wang. Sucheng Ren, and Hang Zhao. On feature decorrelation in self-supervised learning. arXiv preprint arXiv:2105.00470, 2021.
Wang-Cheng Kang and Julian McAuley. Self-attentive sequential recommendation. In ICDM, pp- 197-206. IEEE, 2018.
Gregory Koch, Richard Zemel, Ruslan Salakhutdinov, et al. Siamese neural networks for one-shot image recognition. In ICMI. deep learning workshop, volume 2. Lille, 2015.
Walid Krichene and Steffen Rendle. On sampled metries for item recommendation. In SIGKDD, pp. 1748-1757 2020.
Meng Liu, Hongyang Gao, and Shuiwang Ji. Towards deeper graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp- 338 348,348,22 2020.
Xiao Liu, Fanjin Zhang, Zhenyu Hou, Li Mian, Zhaoyu Wang, Jing Zhang, and Jie Tang. Self- supervised learning: Generative contrastive. IEFE Transactions on Knowledge and Data En- gineering, 2021a
Zhiwei Liu, Yongjun Chen, Jia Li, Philip S Yu, Julian McAuley, and Caiming Xiong, Con- trastive self-supervised sequential recommendation with robust augmentation. arXiv preprint arXiv:2 06479, 2021b.
10

---

Under review as a conference paper at ICLR 2022
Zhiwei Liu, Ziwei Fan, Yu Wang, and Philip S. Yu. Augmenting sequential recommendation with pseudo-prior items via reversely pre-training transformer. ACM, 2021c.
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. Image-based rec- ommendations on styles and substitutes. In SIGIR, pp. 43 13-52,,20 2015.
Aaron van den Oord, Nal Kalchbrenner, Oriol Vinyals, Lasse Espeholt, Alex Graves, and Ko- ray Kavukcuoglu. Conditional image generation with pixelenn decoders. arXiv preprint 1606v 2016.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.0377 2018.
Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. Gee: Graph contrastive coding for graph neural network pre-training. In Pro- ceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1500-1160, 2020.
Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, and Paolo Cremonesi. Personalizing bession-basede recommendations with hierarchical recurrent neural networks. In RecSys, pp. 130- 137, 2017.
Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. Factorizing personalized markov chains for next-basket recommendation. In WWW, pp. 811-820, 2010.
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. Bert4rec: Sequen- tial recommendation with bidirectional encoder representations from transformer. In CIKM, pp- 1441-1450, 2019.
Jiaxi Tang and Ke Wang. Personalized top-n sequential recommendation via convolutional sequence embedding. In WSDM, pp. 565-573, 2018.
Wilson L Taylor. "cloze procedure": A new tool for measuring readability. Journalism quarterly, 30(4))415 433, 1953.
Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good views for contrastive learning? arXiv preprint arXiv:2005. 10243, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. In NIPS, pp.
2017.
Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. Neural graph collaborative filtering. In Proceedings of the 42nd international ACM SIGIR conference Research and development in Information Retrieval, pp. 165-174, 2019.
Chao-Yuan Wu, Amr Ahmed, Alex Beutel, Alexander J Smola, and How Jing. Recurrent recom- mender networks. In WSDM, pp. 495-503, 2017.
Liwei Wu, Shuqing Li, Cho-Jui Hsieh, and James Sharpnack. Ssc-pt: Sequential recommendation via personalized transformer. In RecSys, pp. 928-337. ACM, 2020.
Xu Xie, Fei Sun, Zhaoyang Liu, Jinyang Gao, Bolin Ding, and Bin Cui. Contrastive pre-training for sequential recommendation. arXiv preprint arXiv:2010.14395, 2020.
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. Advances in Neural Information Processing Systems, 33:5812-5825, 2020.
Lei Zheng, Ziwei Fan, Chun-Ta Ta Lu, Jiawei Zhang, and Philip S Yu. Gated spectral units: Modeling co-evolving patterns for sequential recommendation. In SIGIR, pp. 1077-1080, 2019.
Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM International Conference on Infor- mation & Knowledge Management, pp. 1893 1902, 2020.


Reviews:
[{"review_id": "fy8hsm3ems1", "reviewer": "Reviewer_fdBG", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strong points\n1. two model augmentation methods for generating two views of a sequence.\n2. The detailed experiments show the effectiveness of the proposed augmentation.\n3. The paper is easy to read and follow\n\n###  Weak points\n1. overclaimed contributions. The authors should reorganize the contributions in the paper. \n2. lack technical contributions. Dropping the overclaimed contributions, the contributions are marginal\n3. lack significant test, since the improvements are very small, in the order of 0.001\n", "summary_of_the_paper": "This paper  a new self-supervised learning (SSL) paradigm for sequence recommendation by  contrastive learning between positive and negative views of sequences based on model augmentation. The model augmentation methods includes neuron masking, layer dropping and encoder complementing. The proposed algorithm is evaluated with several real-world datasets, showing the efficacy of the proposed methods.\n\n", "summary_of_the_review": "### Detailed comments\n\n1. The neuron masking model augmentation has been proposed in CL4SRec, but claimed as a contribution in the paper. In this way, this model augmentation can not be also considered as a contribution. \n2. Even though overclaimed contributions are included, the contributions may not sufficient. The proposed methods look a little straightforward and motivations are not supported by analysis. \n3. The detailed setting of baselinesa are not given.\n4. Regarding encoder complements, the authors should compare the baselines with GRU encoders as additional module. This is for illustrating the effectiveness of self supervised learning compared to simple ensemble .\n5. \"‘SRMA w/o D’ outperforms other baselines on the Sports dataset and has comparable performance to ‘CL4S.’, which indicates the model augmentation is of more impact in the SSL paradigm compared with data augmentation\". The authors should discuss more about why the superority of SRMA w/o D to CL4S can indicate that the model augmentation is of more impact in the SSL paradigm compared with data augmentation. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "f-i1mkCicL", "reviewer": "Reviewer_D5iZ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. This paper put forwards three kinds of model augmentation methods: neuron masking, layer dropping, and encoder complementing.\n2. This work proposes an idea for constructing views for constrastive SSL.\nWeaknesses:\n1. What’s the motivation for using neuron masking/layer dropping/encoder complementing for model augmentation? Is there any theoretical analysis or intuition explanations?\n2. Although the authors claim the method is model augmentation, I still consider that the method is a kind of data augmentation, because this paper only uses the three levels of model augmentation to construct view pairs for training the model.", "summary_of_the_paper": "This paper proposes three levels of model augmentation methods: neuron masking, layer dropping, and encoder complementing. This work opens up a novel direction in constructing views for contrastive SSL and does experiments to verify the efficacy of model augmentation for the SSL in the sequential recommendation.", "summary_of_the_review": "This paper proposes three levels of model augmentation methods: neuron masking, layer dropping, and encoder complementing. But the novelty and contributions are limited. Besides, it fails to explain the motivation of the proposed augmentation methods.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "KrA1BbjLvNc", "reviewer": "Reviewer_1XwB", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The paper shows the effectiveness of model augmentation over data augmentation.\n2. The paper shows that the effectiveness of model augmentation can help the model to achieve better performances.\n\nWeakness:\n1. The paper only considers model augmentation, but the model augmentation is closely related to model regularization. As shown in the paper \"SSE-PT: Sequential Recommendation Via Personalized\", regularization methods like SSE can help with the better performances too. Have the authors given much thoughts about the differences between model augmentation and model regularization? Are they similar concepts? If so, can we do some comparisons in experiments show the differences in terms of effectiveness?", "summary_of_the_paper": "The paper is trying to address the sequential recommendation problem in which the goal is to predict the next items in user behavior. The paper proposes 3 levels of model augmentation methods: neuron masking, layer dropping and encoder complementing.", "summary_of_the_review": "The paper addresses an important research problem and shows that a few model augmentation techniques can help with the sequential recommendation performances. My main concerns about the paper is the lack of explanation for the differences between model augmentation and regularization techniques. Addressing the concerns will help us understand us better if the proposed methods are general enough to be applied to other research problems.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposed a self-supervised learning view for sequential recommendation with different forms of model augmentation: neuron masking, layer dropping, and encoder complementing. Overall the scores are negative. The reviewers raised concerns mostly around the motivation of the proposed approach (which wasn't fully supported by the experimental results) as well as the limited contribution (especially considering some of the augmentation strategies have been proposed in the past). One reviewer also brought out an interesting connection between model augmentation and model regularization. The authors responded that they will keep improving the paper and hopefully we will see a much improved version in the next submission.
Decision: Reject