Iterative Memory Network for Long Sequential User Behavior Modeling in Recommender Systems
Authors: Qianying Lin
Wen-Ji Zhou
Yanshi Wang
Qing Da
Qing-Guo Chen
Bing Wang
Venue: ICLR 2022 Submitted (2022)
Keywords: recommender systems
sequential behavior modeling

Abstract: Sequential user behavior modeling is a key feature in modern recommender systems, seeking to capture users' interest based on their past activities. There are two usual approaches to sequential modeling : Recurrent Neural Networks (RNNs) and the attention mechanism. As the user behavior sequence gets longer, the usual approaches encounter problems. RNN-based methods incur the problem of fast forgetting, making it difficult to model the user's interests long time ago. The self-attention mechanism and its variations such as the transformer structure have the unfortunate property of a quadratic cost with respect to the input length, which makes it difficult to deal with long inputs. The target attention mechanism, despite having only $O(L)$ memory and time complexity, cannot model intra-sequence dependencies. In this paper, we propose Iterative Memory Network (IMN), an end-to-end differentiable framework for long sequential user behavior modeling. In our model, the target item acts as a memory trigger, continuously eliciting relevant information from the long sequence to represent the user's memory on the particular target item. In the Iterative Memory Update module, the model walks over the long sequence multiple iterations and keeps a memory vector to memorize the content walked over. Within each iteration, the sequence interacts with both the target item and the current memory for both target-sequence relation modeling and intra-sequence relation modeling. The memory is updated after each iteration. The framework incurs only $O(L)$ memory and time complexity while reduces the maximum length of network signal travelling paths to $O(1)$, which is achieved by the self-attention mechanism with $O(L^2)$ complexity. Various designs of efficient self-attention mechanisms are at best $O(LlogL)$. Extensive empirical studies show that our method outperforms various state-of-the-art sequential modeling methods on both public and industrial datasets for long sequential user behavior modeling. 

Under review as a conference paper at ICLR 2022
ITERATIVE MEMORY NETWORK FOR LONG SEQUEN- TIAL USER BEHAVIOR MODELING IN RECOMMENDER SYSTEMS
Anonymous authors Paper under double-bling review
ABSTRACT
Sequential user behavior modeling is a key feature in modern recommender sys- tems, seeking capture users' interest based on their past activities. There two usual approaches to sequential modeling Recurrent Neural Networks (RNNs) and the attention mechanism. As the user behavior sequence gets longer, the usual approaches encounter problems. RNN-based methods incur the problem of fast forgetting, making difficult to model the user's interests long time ago. The self-attention mechanism and its variations such as the transformer structure have the unfortunate property of a quadratic cost with respect the input length, which makes it difficult to deal with long inputs. The target attention mechanism, despite having only (L) memory and time complexity. cannot model intra-sequence de- pendencies. In this paper, we propose Iterative Memory Network (IMN), end- to-end differentiable framework for long sequential user behavior modeling. IMN, the target item acts as memory trigger, continuously eliciting relevant in- formation from the long sequence to represent the user's memory on the particular target item. In the Iterative Memory Update module, the model early crosses the user behavior embeddings and the item embedding, walks over the long sequence multiple iterations and keeps memory vector to memorize the content walked over. Within each iteration, the sequence interacts with both the target item and the current memory for both target-sequence relation modeling and intra-sequence relation modeling. The memory is updated after each iteration. The framework in- curs only O(L) memory and time complexity while reduces the maximum length of network signal travelling paths to (1), achieved by the self- self-attention mecha- nism with (II2)) complexity. IMN outperforms various state-of-the-arr sequential modeling methods on both public and industrial datasets for long sequential user behavior modeling. is successfully deployed on an industrial E-commerce rec- ommender system, with significant 7.29% improvement in the Click-Througe Rate (CTR) for a 7-day online A/B test.
INTRODUCTION
Click-through rate (CTR) prediction is critical for recommender systems. User sequential model- ing is the key to mine users' interest for accurate predictions. As the user sequence gets longer, particularly with lengths longer than 1000, the prediction task requires extraordinary long-range de- pendency modeling, efficient memory storage, acceptable training speed and real-time inference. Recurrent Neural Networks (RNNs) and the long short-term memory (LSTM) are employed by the early sequential recommenders (Hidasi et al., Hochreeter & Schmidhuber 1997). Graves et al. (2014) prove that L.STM forgets quickly and fails generalize to sequences longer than 20. Many empirical results also verify that RNNN-based sequential recommenders are surpassed by attention- based methods (Zhou et al. 2017 Kang & McAuley 2018 Zhou et al., 20188] Pi et al. (011)).
Lately, the self-attention mechanism has proven to benefit wide range of application domains, such as machine translation Vaswani et al. 2017). speech recognition (Chan et al. 2015), reading com-
prehension (Cui et al. 2016; Lin et al. 2017) and computer vision (Xu al. 2015 Parmar al.) 2019). The self-attention mechanism attends to different positions in the sequence, captures the most important features and allows the model to handle long-range dependencies. SASRec adapts the self-

---

Under review as a conference paper at ICLR 2022
attentive Transformer architecture for sequential recommenders and outperforms convolution-basse and recurrence- based methods empirically (Kang & McAuley) 2018).
However, SASRec has a quadratic complexity with respect to the sequence length, limiting its scal-
ability to long sequences. Research on efficient self-attention is based on either sparse attention (Li
et al. 2020 Zhou al. 2020: Child et al.) 2019 Kitaev al. 2020) approximated attention
[Wang et al. 2020), and consequently incompetent against Transformer. SASRec is also unaware of
the target item during the encoding process, which could harm its performance.
Deep Interest Network (DIN) is the first architecture that adaptively learns the user interest represen- tation from historical behaviors with respect to a particular target item (Zhou al. 2018). However, DIN does not model dependencies between elements in the sequence. Research has shown the im- portance of learning long-range intra-sequence dependencies in sequential modeling (Vaswani et al,, 2017]. SASRec considers pairwise intra-sequence dependencies and the maximum length of signal traversal paths is (1)), while DIN models no intra-sequence dependencies.
In this paper, we propose the Iterative Memory Network (IMN) for long sequential user behavior modeling. The target item is a memory trigger in IMN, continuously eliciting relevant informa- tion from the sequence. IMN early crosses the target item and the user sequence, walks over the sequence for multiple iterations and repeatedly updates the memory vector with new information retrieved from the sequence. The contributions of this paper are summarized as follows:
We propose the Iterative Memory Network (IMN), an end-to-end differentiable framework for sequential recommenders. To the best of our knowledge, it outperforms the state-of-the of-thhe-ttr- art models for equal sequence lengths, with even more sigmificant advantages on long sequential user behavior modeling. The framework is scalable sequential recommenders with other objective functions. IMN is efficient with (T) complexity and 0(1) number of sequential operations, making it scalable to long sequences. IMN requires significantly less training and inference time, with greater memory efficiency. Implemented on user sequences of length 1000, IMN is deployed successfully on industrial E-commerce platform with 1500 QPS. The inference time is 30ms. There is a significant 7.29% CTR improvement over the DIN-based industrial baseline. To the best of our knowledge, IMN the first sequential recommender that early crosses the user sequence and the target item. Our ablation study validates empirically that early crossing before further sequence encoding benefits the model performance compared to delayed crossing.
IMN models both long-range intra-sqquence dependencies and target-sequence dependencies in OLL) complexity. In IMN, the memory vector encapsulates the sequence content walked over. The multi way attention between the sequence, the target item and the memory allows for intra- sequence signal passing. Self-attention achieves this with ( complexity, and the target atten- tion mechanism allows for no intra-sequence dependency modeling.
2 RELATED WORK
Sequential Recommender Systems. Sequential recommenders predict the user's next behavior based his past activities. Recurrent Neural Networks (RNNs) are introduced for early sequential recommenders (Wu et al. [2166] Hidasi et al. 2016). Yet, RNNs are difficult to parallelize and suffer from the problem of fast forgetting (Graves al. 2014). First introduced in the encoder-decoder framework, attention is shown effective to replace RNNs to encode sequences (Bahdanau et al., 2016 Vaswani et al. 2017). Attention-based recommender systems include seli-attention based methods (Kang & McAuley 2018 Zhou et al. 2017). target-attention based methods (Zhou et al.) 2018) and methods based on the integration between RNN and attention (Zhou et al.) (2019). Target-attention based methods learn the weights for sequence items with respect to the target item.
Memory Networks. Memory Networks have wide applications in Question Answering (QA), find- ing facts for a query from the knowledge database (Chaudhari et al. 2021). Neural Turing Machines mechanism for memory searching and update (Graves) et al. 2014). |Weston et al. (2014) proposes the general architecture for Memory Networks. DMN, DMTN and DMN+ are the subsequent research (Kumar et al.) 2016] Xiong et al.) 2016 Ramachan- dran & Sohmshetty, 2017). In recommender systems, MIMN uses GRU as the controller to update
(NTM) introduces the addressing-read-wrii
user memory slots with each new clicked item (Pi al. 2019
2

---

Under review as a conference paper at ICLR 2022
Figure I: Overview of the Iterative Memory Network (IMN) architecture. The Iterative Memory Update module (in purple) illustrates the repeated memory update process. multi-way attention and the Memory Enhancement module are on the left.
The details about the
PROBLEM FORMULATION
The recommender system models the user-item interaction as matrix C [Cm=]MXM where M and N are the total number of users and items respectively. The interaction is either explicit ratings (Koren 2009) or implicit feedback (Agarwal et al., 2009). The CTR prediction task is usually based on implicit feedback. We denote E U user and I as item, and the user tm clicking the item in makes Cmn I and others O. User sequential modeling predicts the probability U clicking the target item I based his past behaviors, i1, i2 ..., LL, where L is the length of the user sequence. The sequence is usually in chronological order. Our paper focuses on long sequential user behavior modeling, where the length of the user behaviors L is the scale of thousands. We use the term "target-sequence dependencies to refer to dependencies between the target item and the sequence items and "intra-sequence dependencies" refer to dependencies between the items within the sequence.
user
4 ITERATIVE MEMORY NETWORK
We illustrate the Iterative Memory Network architecture in Fig.1. The encoder layer converts behavior features, the temporal features, and the target item features to embedding vectors. The Iterative Memory Update module is the major component in our architecture, modeling both target- sequence dependencies and intra-sequence dependencies. The Memory Enhancement module re- peatedly elicits clearer memory with the target item and outputs the final user interest vector. Next, we discuss the framework in detail.
4.1 ENCODER LAYER
We use e; to denote the embedding for the sequence item i. Each user behavior consists of not only the clicked item e;, but also the action time. Different users have different action patterns, thus the action time contains important temporal information. Since it is difficult to learn good embedding directly with continuous time features, we bucketize the time feature into multiple granularities and perform categorical feature look-ups. We slice the elapsed time with respect to the ranking time into intervals whose gap length increases exponentially. In other words, we map the time in range(O, I), added
[1,2), [2,4), ..., [2 2 to categorical features 0,1,2....k 1. Positional encodings to represent the relative positions sequence items. The timestamp encoding and the positional encoding have the same dimension that of the item embedding e; be directly summed. We
3

---

Under review as a conference paper at ICLR 2022
encode the Juh user behavior ej
c e, C 1
where denotes element-wise sum-up. We denote the embedding for the target item vr- shares the item id embedding look-up table with the item id embedding e; for the sequence.
4.2 MUL.TIWAYY ATTENTION
We calculate the similarities between the sequence item, the target item and the memory. Then we concatenate them into a single feature vector. Similarities are in both Euclidean distance and the Hadamard distance (denoted by o) to capture multi-faceted similarity measures.
a(e, v, m) fe- m,
m, e v).
where e is the embedding vector for sequence items, m is the memory embedding vector and is the embedding vector for the target item.
We input the feature vector into two-layer point-wise feed-forward network, with sigmoid as the activation function. Here point-wise feed-forward means the fully-connected ferd-forward network is applied to each sequence item separately and identically.
a(e, v, m) a (w (W afe, e, m) + b()) + B(2))
We have also experimented with the softmax activation function for the second layer. There is very slight change in model performance. The multi-way attention mechanism encodes similarities between the user behavior sequence and the target item to extract the user's interest specific to particular target item. It also encodes similarities between sequence items and the memory. As the memory vector encapsulates the sequence information, the multi-way attention mechanism can model intra-sequence dependencies between items the sequence and reduce the maximum length of signal traversal paths to (()).
4.3 ITERATIVE MEMORY UPDATE MODULE
In the Iterative Memory Update module, the target item acts as the memory trigger, continuously waking up relevant memory in the sequence. The memory vector is updated after each iteration with the relevant information retrieved from this iteration. We use GRU to model the memory update process. The initial memory m is initialized from the target item vector, to represent that the user's initial memory on the target item is the target item itself.
m UT
(4)
For iteration t, we calculate the user interest representation oy weighted sum pooling of quence item vectors, with weights derived from (3).
L alee vr,
2i = =((er, e/, e,,
el:) Σ
ur.m' m me = Σ wyef
(5)
where ei, e ..., of the list of user behavior embedding vectors, or the embedding vector of target item T and m is the memory embedding vector. We use the user interest representation vi, after iteration t as the input to update the memory m mi-1
m GRU(( m m1)
Since the memory updated with a weighted sum pooling of sequence item embeddings, the mem- contains information on the sequence items. As both the memory and the sequence contains information about the sequence items, the multi-way attention models intra-sequence dependencies. The architecture models co-occurence beyond the (target item, sequence item) pair. For exam- ple, the target item is rum and the user sequence contains lime and peppermint. With the target attention mechanism, both the attention weight between the pair (peppermint, rwr) and that be- tween the pair (lime, rum) are not high. With our IMN architecture, after the first pass sequence walk, the memory vector contains information about both the peppermint and rum. When it meets the item lime again, the memory of peppermint and rum awakens the item lime since the triplet (rum, peppermint, lime) is the recipe for Mojito and likely to co-occur multiple times in different
4

---

Under review as a conference paper at ICLR 2022
samples. Hence, with lime and peppermint in the sequence, the likelihood to click rum increases. While the target attention mechanism finds the items that co-occur frequently with the target item, IMN finds the composite group of the user's behavior items for the user to click the target item. Furthermore, IMN is structurally different from Memory Networks, which updates the memory with each incoming input and uses the updatedd memory for computations the next input. The sequen- tial nature makes it difficult to parallelize, hence infeasible to deploy on recommmender systems that need real-time responses.
4.4 MEMORY ENIIANCEMENT MODULE
After N iterations of the memory update process, the user memory vector encapsulates both the information in the sequence relevant to the target item vy and the intra-sequence information. The Memory Enhancement Module enhances the user memory with the target item repeatedly to elicit more clear memory specific to the target item and remove noises.
We use GRU to model the repeated elicitation process. The GRU's initial state initialized from the memory after the Iterative Memory Update module, up m For each step, apply linear transformation W on the GRU's last hidden state, concatenate the transformed vector with the target item, and use the concatenated vector the GRU's input.
u. -GU 1-1 "r)), u =
where u' the user representation after steps in the Memory Enhancement Module.
4.5 OTHER RELEVANT EXPERIMENTS
Dot Product Multi-way Atlention. We have experimented with the dot product distance for the multi- way attention. Model performances degrade expected.
Sequence Segmentation & Hierarchical Memory Network. We segment the sequence of length 1000 into sub-sequences of length 100. We compute the user memory vector with IMN for each sub- sequence. The user memory vectors produced are treated as another sequence for IMN. There is no performance improvement.
5
EXPERIMENTS
In this section, we present the experimental setups, experimental results, ablation study, model anal- ysis, computational cost analysis, memory efficiency analysis and hyper-parameter choices detail.
5.1 DATASETS AND EXPERIMENTAL. SETUP
Amazon Dataset. We collect two subsets from the Amazon product data, Books and Movies (McAuley al. 2015). Books contains 295982 users, 647589 items and 6626872 samples. Movies contains 233282 users. 165851 items and 4829693 samples. We split each dataset into 80% training and 20% test data according to the behavior timestamp. We use the Adam optimizer, with 0.001 learning rate (Kingma & Ba 2015). The mini-batch size is 512. We use 2 parameter servers and 4 workers, with 10GiB memory for each worker. The MLP layer size is 128 x 64.
Industrial Dataset. We collect traffic logs from real-world E-commerce platform. The E- commerce platform has search and recommendation systems, with user click and purchase logs. We use 30-day samples for training and the samples of the following day for testing. There are 1.68 billion training samples and 57 million test samples. MLP layers are 512 x 256 x 128. The hidden state dimensions for GRUs are 32. The mini-batch size is 512. We the Adam optimizer, with 0.0001 as the learning rate. We use 5 parameter servers and 50 workers, with 75GiB memory for each worker.
Evaluation Metric. We use Area Under the Curve (AUC) as the performance measurement metric.
5.2 MODEL COMPARISON
YouTube DNN. YouTube DNN uses average pooling to integrate behavior embeddings to fixed- width vectors the user's interest representation (Covington et al. 2016).
5

---

Under review as a conference paper at ICLR 2022
Table 1: Model performance (AUC) on public and industrial datasets
Model
Amazon Books
Amazon Movies
Industrial
AUC
Impr
AUC
Impr
AUC
Impr
Youtube DNN
0.8374
0.00
0.8343
0.00
0.7353
0.00
DIN
0.8516
1.42
0.8603
2.6
0.7375
0.22
DIEN
0.8550
1.76
0.8654
3.22
0.7381
0.28
SASRec
0.8214
-1.60
0.8221
-1.22
0.7346
-0.07
MIMN
0.8523
1.49
0.8714
3.71
0.7377
0.24
UBR4CTR
0.8570
1.96
0.8796
4.53
0.7392
0.39
IMN 2P
0.8498
1.24
0.8821
4.78
0.7394
0,41
IMN 3P
0,8672
2.98
0,8835
4.92
0.7408
0.55
IMN 3P+
0.8692
3.18
0.8862
5.19
0.7423
0.7
DIN. DIN proposes the target attention mechanism to soft-search user sequential behaviors with respect the target item (Zhou al. 2018).
DIEN. DIEN uses attention-based GRU to model user interest evolutions (Zhou et al.
2019
SASRec. SASRec is self-attentive model based on Transformer (Kang McAuley 2018).
MIMN. MIMN uses fixed number of memory slots represent user interests. When new click
takes place, it updates the user memory slots with GRU (Pi et al. 2019).
UBR4CTR. UBR4CTR is two-stage method. The first stage retrieves relevant user behaviors from sequence with a learnable search method. The second stage feeds retrieved behaviors into DIN-based deep model (Qin et al. (020). The Amazon datasets contain no item side information, therefore we use a strengthened version of sequence selection for the first stage with multi-heat self-attention the sequence itself.
IMN 2P/3P/3P+. IMN models without the Memory Enhancement module. 2P refers to 2 itera- tions of the memory update process, and 3P refers to iterations.
IMN 3P+. 3 memory update iterations, with the Memory Enhancement module. The number of steps t 3 for the Memory Enhancement module.
5.3 EXPERIMENTAL RESULTS
We report model performances on three datasets with maximum affordable sequence lengths in Table I We summarize model performances with increasing sequence lengths 50, 100, 200, 500 and 1000 for Amazon Books in Fig Experiments with missing performance data incur Out-of- Memory(OOM) errors that stop training. We have the following important findings:
IMN 3P consistently outperforms all state-of-the-ar baselines over three datasets. This demon- strates the effectiveness of our proposed methodology, modeling intra-sequence dependencies and target-sequence dependencies simultancously. IMN 3P+, with the Memory Enhancement module, has slight improvements over IMN 3P.
IMN 3P constantly outperforms SASRec, the Transformer-basee sequential recommender, over equal sequence lengths. As seen in Fig IMN 3P outperforms the compared models over equal sequence lengths. Noticeably, IMN outperforms SASRec over equal sequence lengths afford- able by SASRec. This does make sense, considering that SASRec encodes the sequence with multi-head attention with no knowledge on the target item. In contrast, IMN is aware the target item throughout the encoding process. The ablation study in Section 5.4 verifies the benefits of early cross with the target item.
In general, methods that emphatically perform sequential updates seem have moderate per- formance gain. DIEN uses attention-based GRUs to update the user interest with each sequence item. Similarly, MIMN sequentially updates the nearest memory slots with each sequence item. F shows that DIEN constantly outperforms DIN, though the improvement is moderate on sequence length 100.

---

Under review as a conference paper at ICLR 2022
Method
Books
Movies
Industrial
AUC
AUC
AUC
w/o. attention
0.8374
0.8343
0.7352
w/o. iterative walk
0.8516
0.8613
0.7375
w/o. Euclidean dist.
0.8549
0.8753
0,7402
delayed cross w/o. m.e.
0.8423
0.8603
0.7389
DEN
w/o, m.c full
0.8672
0.8835
0.7408
0.8692
0.8862
0.7423
Figure 2: Performance with seq. lengths
Figure 3: Ablation study on the model structure
5.4 ABLATION STUDY
We conduct ablation study about the model structure and report the results Fig We remove the Memory Enhancement module to produce the ablation model IMN(w/o. m.e.). We further remove the cross with the target item to product the ablation model IMN(delayed cross & w/o. m.c.). We replace the Euclidean distance with another Hadamard distance calculation for the ablation model IMN(W/o. Euclidean dist.). We remove the itcrative update process to produce IMN(w/o. iterative walk), which becomes the same DIN. We replace the attention with average pooling to produce IMN (w/o. attention), which becomes identical to YouTube DNN. The following are our findings:
The iterative update process models intra-sequence dependencies, which benefits the model per- formance significantly. IMN(full) has remarkable improvement over IMN(w/o. iterative walk), on par with the improvement of IMN(w/o. iterative walk) over MMN(w/o.tttentio)) demon- strates the effectiveness of our proposal to model intra-sequence dependencies in addition to target-sequence dependencies in DIN.
Delayed cross with the larget ilm results in performance degradation. IMN(w/o. m.e.) outper- forms IMN(delayed cross & w/o. m.e.) by a large extent, showing that early crossing the user sequence and the target item results in performance gain. This also explains for IMN's perfor- mance improvements over SASRec, which crosses the Transformer-encodee user sequence and the target item at the very top.
Multi-faceted distance modeling benefits model performances. IMN(full) outperforms IMN(w.o. Euclidean dist.) certain extent, showing that multi-faceted distance modeling beneficial.
5.5 MODEL ANALYSIS
We analyze the compared models and summarize the encoding paradigm, complexity, minimum number of sequential operations and maximum path length in Table 2 with the observations below:
IMN is the first architecture with (CROSS, ENC) structure, which benefis model perfor- mances. We denote the cross between the user sequence and the target item as CROSS. We denote sequence encoding ENC. The encoding paradigm for DIN is (CROSS), with no quence encoding after the target attention cross. DIEN (ENC, CROSS), encoding sequence items with GRU before interest evolution modeling with respect to the target item. SASRec adapts the Transformer encoder for sequence encoding and is therefore (ENC, CROSS). Sim- ilarly, MIMN follows the (ENC, CROSS) paradigm, using a GRU-based controller update memory slots with the sequence. UBR4CTR is (CROSS) since sequence items cross the tar- get item with no further sequence encoding for the second stage. IMN is the first model with the (CROSS, ENC) paradigm. The user sequence crosses with the target item in Hadamard distance and Fuclidean distance before sequence encoding. Ablation study Fig validates empirically that the early cross benefits model performances.
IMN is efficient with ((L - d) complexity and O(1) number of sequential operations, with theo- retical shortest maximum path length ((1). SASRec, the Transformer-based sequential recom- mender, incurs O(L2-d -d) complexity. IMN incurs (L--d) complexity, since the optimal number of iterative sequence walk is seen in Section 5.8. The minimum number of sequential operations measures the amount of parallelizable computations. Pure antention-based methods are at O(1), while recurrence-basse methods are at (I.). Maximum path length refers to the maximum length
7

---

Under review as a conference paper at ICLR 2022
Table 2:
Encoding paradigm, complexity, minimum number of sequential operations, maximum
path length for compared methods. L is the sequence length and d is the model dimension.
Method
Encoding Paradigm
Complexity
Sequential Operations
Max Path Length
DIN DIEN
(CROSS)
O(L-d)
0(1)
((xc)
ENCC, CROSS)
O(I.d)
(
(
SASRec
(ENC, CROSS)
d(
0(1)
0(1)
MIMN
(ENC, CROSS)
O(L-d)
0(L)
0(L)
UBR4CTR
(CROSS)
O(L-d)
0(1)
(
IMN
(CROSS, ENC)
O(L-d)
0(1)
0(1)
DCEM
NEVN
LOOACTR:
User Sccneii Leagss
Uset Secimmo
(a) Training time (global steps/sec)
(b) Inference time (ms) (c) Peak GPU memory usage (GiB)
Figure 4: Computational and memory efficiency analysis
of signal traversal paths. SASRec is O11) due the pairwise comparisons in the self-attention mechanism. Our IMN also is (1) since the memory vector with the sequence information in- teracts with sequence items through the multi-way attention. There intra-sequence signal passing in DIN and UBR4CTR. DIEN and MIMN are at O(L) due to the sequential nature.
5.6 COMPUTATIONAL COST ANALYSIS
We report the run-time for compared methods in 4a and the inference time in Fig4b 4b The run- time efficiency measured by global steps per second during training. The real-time efficiency measured by the inference time in milliseconds. Note that the y-axis of Fbg[4bjis is the logarithmic scale. For the inference time, we only measure the forward pass cost, excluding cost. We use inputs with lengths 50, 100, 200, 500 and 1000. Experiments with missing data incur Out-of Mettof-Meemoyy(O00 (OOM) errors that stop training. All experiments are conducted on Tesla A100 GPU with 10GiB memory. We summarize
input encoding
findings below:
IMN is computationally efficient with increasing sequence lengths. IMN involves matrix oper- ations heavily, which are highly optimized to be be parallelizable on GPU. On sequence length 1000, the training time for IMN and DIN is similar, not doubling that for YouTube DNN. More importantly, the forward pass inference time at sequence length 1000 is only 3.6ms.
Methods based on sequential updates are computationally expensive for training and inference. DIEN and MIMN, the two models based on sequential updates, have significantly lower train- ing and inference speeds. The inference time for MIMN sequence length 50 is 216ms, while the industrial norm is 30ms to 50ms. In-depth analysis shows that its addressing head is time- consuming. The computational inefficiency forces MIMN to separate the user and the item side, performing user side inference prior to online scoring- At sequence length 100, DIEN's s inference time is 77ms. Unlike MIMN, DIEN cannot separate the user and item sides since it performs target attention top of the GRU encoder. This limits DIEN's scalability to longer sequences. SASRec, the self-attentive method, is efficient during inference time, but not during training time. The inference speed for SASRec is significantly lower compared methods with sequential up- dates. This is expected since self-ate attention allows for significantly more parallelizations. However, training is relatively slow for SASRec.
8

---

Under review as a conference paper at ICLR 2022
(a) IMN performances
(h) average attention entropy
Figure 5: IMN performances and attention entropy with increasing sequence walk iterations
Table 3: Online A/B result, with Raw denoting the raw data and Impr the relative improvement.
Method
CTR
UV-IPV
TCIC
CCC
Raw
Impr
Raw
Impr
Raw
Impr
Raw
Impr
Industrial baseline
4.425%
0.00
4.051%
0.00
325741
0.00
2.979
0.00
IMN
4.748%
7.29%
4.373%
7.95%
375733
15.35%
3.193
7.18%
5.7 MEMORY CONSUMPTION
We evaluate the memory efficiency by measuring the peak GPU memory usage in GiB in Fig 4c The memory limit is 10GiB. Experiments above the horizontal dotted line in Fig incur Out-ol- Memory (OOM) errors. We summarize our findings below:
IMN is efficient with memory consumption increasing linearly with sequence lengths. IMN incurs linear space complexity. The memory overhead the user memory vector, the same size as the target item. The low peak memory consumption at varying lengths testifies its memory efficiency. Self-attentive methods have the most memory usage increase with increasing sequence lengths. The YTM-based MIMN is also memory-hungry. SASRec incurs Out-Of- nut-Of-Memory (OOM) errors on sequences longer than 100. Figoo shows that memory consumption increase is the most substantial with increasing sequence lengths, testifying the ( memory bottleneck. MIMN is also memory-hungn since keeping additional user memory slots results memory overheads.
5.8 SENSITIVITY W.R.T NUMBER SEQUENCE WALK ITERATIONS
We investigate the impact of sequence walk iterations. Fig shows IMN's performances against the sequence walk iterations. AUCs increase with more iterations and stablize after to 4 iterations. Fig 5b shows the average attention entropy against sequence walk iterations. The trend coincides with that on model performance, indicating that the optimal iteration hyper-parameter 3.
6 ONLINE A/B PERFORMANCE
We report the 7-day A/B test result on the industrial recommender in Table 3 baN-based deep model with sequences of length 50. IMN is implemented user click sequences of length 1000. UV-IPV refers to the average click count per user. Total Clicked Items Count (TCIC) the number of distinet items being clicked. Clicked Categories Count (CCC) is the number of categories clicked user. TCIC and CCC are diversity measures for recommender systems.
The baseline is a
7 CONCLUSION
The self-attention mechanism incurs O(L?) memory complexity that limits the scalability to long sequences. The target attention mechanism, despite having only (T.) complexity, can not model intra-sequence dependencies. Methods with sequential updates incur high computational costs.
IMN models intra-sequence dependencies and target-sequence dependencies simultaneously in O(L) memory and time complexities. It is efficient with computational costs and memory consump- tion. It outperforms the state-of-the-re sequential recommenders for equal sequence lengths, with significant advantages on long user sequential modeling. It is deployed successfully a world E commerce recommendation platform, with a significant improvement of 7.29% CTR.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Deepak Agarwal, Bee-Chung Chen, and Pradheep Elango. Spatio-temporal models for estimating click-through rate. In WWW, 2009.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In arXiv: (rXxv:/4090047, 2016.
William Chan, Navdeep Jaitly, Quoc V. Le, and Oriol Vinyals. Listen, attend and spell. In
(tft::5508.02111 2015.
Sneha Chaudhari, Varun Mithal, Gungor Polatkan, and Rohan Ramanath. An attentive survey of attention models. In arXiv: 19ti::9994..0200 1904.02874 2021.
Rewon Child, Scott Gray, Alec Radford, and llya Sutskever. Generating long sequences with sparse transformers. In arXiv: 10509, 2019.
Paul Covington, Jay Adams, and Emre Sargin. Deep neural networks for youtube recommendations. In Recsys. 191-198, 2016.
Yiming Cui, Zhipeng Chen, Si Wei, Shijin Wang, Ting Liu, and Guoping Hu. Attention-over- attention neural networks for reading comprehension. In arXiv: 1607.44473, 2016.
Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. In arXhv:1410.5401.000
Balazs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based rec- ommendations with recurrent neural networks. In ICLR, 2016.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.
Wang-Cheng Kang and Julian McAuley. Self-attentive sequential recommendation. In CIKM, 2018.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), Proceedings of 3rd International Conference on Learning Repre- sentations, San Diego, CA, USA, 2015.
Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. In ICLR, 2020.
Yehuda Koren. Collaborative filtering with temporal dynamics. KDD, 2009.
Ankit Kumar, Peter Ondruska, Mohit lyyer, James Bradbury, Ishaan Gulrajani, Victor Zhong, Ro- main Paulus, and Richard Socher. Ask me anything: Dynamic memory networks for natural language processing. In ICML, 2016.
Xiaoya Li, Yuxian Meng, Mingxin Zhou, Qinghong Han, Fei Wu, and Jiwei Li. Sac: Accelerating and structuring self-attention via sparse adaptive connection. In NeurIPS, 2020.
Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. A structured self-attentive sentence embedding. In 1703.03300 2017.
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. Image-based rec- ommendations styles and substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development Information Retrieval, 2015.
Niki Parmar, Prajit Ramachandran, Ashish Vaswani, Irwan Bello, Anselm Levskaya, and Jon Shlens. Stand-alon self-attention in vision models. In Advances in Neural Information Processing Sys- tems 32, pages 68-80, 2019.
Qi Pi, Weijie Bian, Guorui Zhou, Xiaoqiang Zhu, and Kun Gai. Practice on long sequential user behavior modeling for click-through rate prediction. In Proceedings of the 25th ACM SIGKDD, 2019.
Jiarui Qin, Weinan Zhang, Xin Wu, Jiarui Jin, Yuchen Fang, and Yong Yu. User behavior retrieval for click-through rate prediction. In SIGIR, 2020.
10

---

Under review as a conference paper at ICLR 2022
Govardana Sachithanandam Ramachandran and Ajay Sohmshetty. Ask more: Dynamic memory tensor networks. In arXiv: 2017.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E. ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Infor- mation Processing Systems 30, pages 5998-6008, 2017.
Sinong Wang, Belinda Li, Madian Khabsa, Han Fang, and Hao Ma. Linformer: Self- attention with linear complexity. In arXiv:2006.044 04768, 2020.
Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. In arXiv:1410.3916, 2014.
Sai Wu, Weichao Ren, Chengehao Yu, Gang Chen, Dongxiang Zhang, and Jingho Zhu. Personal rec- ommendation using deep recurrent neural networks in netease. In IEEE International Conference on Data Engineering, 1218-1229, 2016.
Caiming Xiong, Stephen Merity, and Richard Socher. Dynamic memory networks for visual and textual question answering. arXiv: 1603.0641., 2016.
Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of the 32nd International Conference on Machine Learning, 2015.
Chang Zhou, Jinze Bai, Junshuai Song, Xiaofei Liu, Zhengchao Zhao, Xiusi Chen, and Jun Gao. Atrank: An attention-based user behavior modeling framework for recommendation. In AAAI, 2017.
Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference Knowledge Discovery and Data Mining, ACM, 1059-1068, 2018.
Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. Deep interest evolution network for thick-through rate prediction. In Proceedings of the 33nd AAAI Conference on Artificial Intelligence, 2019.
Haoyi Zhou, Shanghang Zhang. Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang Informer: Beyond efficient transformer for long sequence time-series forecasting. AAAI, 2020.


Reviews:
[{"review_id": "_64-lK7NndV", "reviewer": "Reviewer_Q1ux", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper proposes an \"iterative memory network\" for long user sequence modeling such as those for ctr prediction in ads and recommender systems. The basic idea is to use a iteratively updated memory vector to interact with items in the user sequence and target item, so there is higher-order interaction intra-sequence, without the need to use self-attention. Experiments are conducted on two offline datasets comparing with some attention and memory network based methods.\n\nStrength\n- Modeling long user sequence is an important problem and is a popular topic recently.\n- The proposed model is overall sound and seems to be a reasonable way for this problem.\n\nWeakness\n- The major weakness of this paper is evaluation. First, the baselines are relatively out dated. The most recent baseline was in 2019. As mentioned above, the problem is an actively researched area recently, and the DIN paper drew ~400 citations in the last 2 years. The experimental sections need more recent and competitive baselines to compare with. Otherwise the experiments look like what a paper would do 2 years go. Some important related work or baselines seem missing, for example, retrieval based methods such as \"User Behavior Retrieval for Click-Through Rate Prediction\". Work along this line also try to tackle efficiency issues for long-sequence modeling and need to be discussed and compared with. Second, only offline experiments are done on two datasets. The industrial dataset does not have much details and it is not clear what the impact is. Many papers in this area, such as DIN, performed actual online validation so this paper is on or below the borderline in this dimension.\n- The proposed method is intuitive and easy to understand, but may fit an application oriented conference better. Using GRU units to update the memory and establish intra-sequence connections is hand-waving and there is no rigorous mathematical analysis on why and how this works. This may be ok for an application paper but seems to be below the bar of ICLR.\n\nMinor comments\n- Figure 1 quality should be better.\n- Make notation more consistent, eq4 and eq6 the scripts should be both super or sub.\n", "summary_of_the_paper": "This paper proposes an \"iterative memory network\" for long user sequence modeling such as those for ctr prediction in ads and recommender systems. The basic idea is to use a iteratively updated memory vector to interact with items in the user sequence and target item, so there is higher-order interaction intra-sequence, without the need to use self-attention. Experiments are conducted on two offline datasets comparing with some attention and memory network based methods.\n\n", "summary_of_the_review": "Though the method proposed in this paper is intuitive and sound in general, the lack of rigorous algorithm analysis and significant lack in experiments in terms of setting and baselines may require the paper to be more polished before publication.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "VXr31VCIxVQ", "reviewer": "Reviewer_UHbh", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is very clear and the proposals are quite practical. However, I worry that there might not be sufficient academic novelties in this work. Particularly, by giving up the self-attentions within the user's own browsing history, the new method would have fundamental limitations in the comprehension of user intents with long histories. The original self-attention layers would summarize exponentially longer sequences as more layers are added, but the target-attention mechanism only does so linearly. Perhaps this is okay for recommender systems, when there are not much complexity in the sequential patterns, but the contribution is rather technical than academic.\n\nAdditionally, the proposed method has another limitation that it is only suitable for reranking tasks when there is a short list of candidates, instead of end-to-end retrieval tasks. This is because every candidate item has to be evaluated through a complex neural network, which is in contrast to traditional transformer models such as SASRec, when the decoder would potentially support the retrieval tasks through approximate max-inner-product search. This follows the same limitation as DIN and I thus worry that its cross-disciplinary impact could be limited.", "summary_of_the_paper": "The paper talks about a simplification of transformer architectures. The original transformers employ a dense attention between all tokens in a sequence to focus on sequence comprehension. In this work, however, the focus is on the retrieval of the last item. It therefore only evaluates the attention between the proposed candidate item and the preceding items in user history. The other novelty as I see is the replacement of Residual connections between layers with a GRU layer, though the impact is less discussed. The work is successfully deployed in industrial recommender systems, showing significant improvements in the click-through rate.", "summary_of_the_review": "Interesting practical work and great industrial success, but limited academic contributions.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "D17uL1aHIyY", "reviewer": "Reviewer_p4tQ", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength\n\n* S1: The writing is clear, easy to follow. \n* S2: The authors conducted good experiments, where they also used industrial dataset for comparison. (It will be great if the authors could also provide comparison with current production model(s)).\n\n\nWeakness:\n\n* W1: The paper has very limited contributions. The idea of memory based attention networks is not new [1, 2]. \n* W2: The paper has poor analysis (Section 5.4). The paper needs to have deeper analysis in explanation of model architecture and not only performance. For example, besides the benefits of memory based, which were already stated in few papers such as [1, 2], the authors should provide in details about the advantages of the proposed framework (e.g., from theoretical perspectives) compared to previous baselines\n* W3: The results in Figure 2 does not seem to be convincing, since the performance on AmazonBooks and AmazonMovies decrease dramatically after only few iterations. \n\n\n[1] Latent Relational Metric Learning via Memory-based Attention for Collaborative Ranking. WWW 2018. \n[2] Signed Distance-based Deep Memory Recommender. WWW 2019.\n", "summary_of_the_paper": "This paper focuses on tackle the sequential recommendation task, where the authors proposed Iterative Memory Network (IMN), an end-to- end differentiable framework for long sequential user behaviour modeling. The main contribution of the paper is the IMN framework with efficient in memory and complexity. Specifically, the authors proposed iterative memory updates module with the multi-way attention and memory enhancement module.", "summary_of_the_review": "The paper is good in terms of writing and conducting experiments. However, the contributions are very limited. This paper requires more in-depth analysis about the model performance, ablation studies, run-time/real-time comparison, etc. A huge drop in the AUC metric after only a few iterations in Figure 2 also raise a concern about the performance.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "4m3uQHEpSni", "reviewer": "Reviewer_NSbP", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths \n1. This paper is well written and easy to follow. \n2. The topic is significant. Modeling the long sequence is technically challenging. \n3. This paper proposes the Iterative Memory Network (IMN), an end-to-end differentiable framework for long sequential user behavior modeling, which shows better performance than baselins. The IMN model is technically sound and interesting. \n\n\nweaknesses\nThe compared sequential models lack some STOA, such as [1, 2]. This paper claims its advantages over transformer variants. However, the comparisons ignore the STOA models based on transformers, which does not make sense. \n\n\n[1] Sun F, Liu J, Wu J, et al. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer[C]//Proceedings of the 28th ACM international conference on information and knowledge management. 2019: 1441-1450.\n[2] Zhou K, Wang H, Zhao W X, et al. S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization[C]//Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020: 1893-1902.\n", "summary_of_the_paper": "This paper proposes the Iterative Memory Network (IMN), an end-to-end differentiable framework for long sequential user behavior modeling. Moreover, this paper conducts experiments to show its better performance. ", "summary_of_the_review": "Modeling the long sequence is a hard problem in recommendation applications. This paper proposes the IMN model to learn the user preference from a long sequence and performs better than baselines. The technical significance and novelty of the proposed model are good. The reviewer tends to accept the paper. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a new architecture named Iterative Memory Network (IMN) to encode long user behavior sequence for recommendations. Reviewers appreciate the clarity of the writing as well as practicality and the O(L) complexity of the proposed architecture, however do raise questions on novelty. Different design choices employed in the paper are not well explained. The rebuttal was not able to convince the reviewers to accept the work at this venue, but reviewers do feel the paper could fly in an application oriented venue.
Decision: Reject