Iterative Memory Network for Long Sequential User Behavior Modeling in Recommender Systems

1 INTRODUCTION . Click-through rate ( CTR ) prediction is critical for recommender systems . User sequential modeling is the key to mine users ’ interest for accurate predictions . As the user sequence gets longer , particularly with lengths longer than 1000 , the prediction task requires extraordinary long-range dependency modeling , efficient memory storage , acceptable training speed and real-time inference . Recurrent Neural Networks ( RNNs ) and the long short-term memory ( LSTM ) are employed by the early sequential recommenders ( Hidasi et al. , 2016 ; Hochreiter & Schmidhuber , 1997 ) . Graves et al . ( 2014 ) prove that LSTM forgets quickly and fails to generalize to sequences longer than 20 . Many empirical results also verify that RNN-based sequential recommenders are surpassed by attentionbased methods ( Zhou et al. , 2017 ; Kang & McAuley , 2018 ; Zhou et al. , 2018 ; Pi et al. , 2019 ) . Lately , the self-attention mechanism has proven to benefit a wide range of application domains , such as machine translation ( Vaswani et al. , 2017 ) , speech recognition ( Chan et al. , 2015 ) , reading comprehension ( Cui et al. , 2016 ; Lin et al. , 2017 ) and computer vision ( Xu et al. , 2015 ; Parmar et al. , 2019 ) . The self-attention mechanism attends to different positions in the sequence , captures the most important features and allows the model to handle long-range dependencies . SASRec adapts the self- attentive Transformer architecture for sequential recommenders and outperforms convolution-based and recurrence-based methods empirically ( Kang & McAuley , 2018 ) . However , SASRec has a quadratic complexity with respect to the sequence length , limiting its scalability to long sequences . Research on efficient self-attention is based on either sparse attention ( Li et al. , 2020 ; Zhou et al. , 2020 ; Child et al. , 2019 ; Kitaev et al. , 2020 ) or approximated attention ( Wang et al. , 2020 ) , and consequently incompetent against Transformer . SASRec is also unaware of the target item during the encoding process , which could harm its performance . Deep Interest Network ( DIN ) is the first architecture that adaptively learns the user interest representation from historical behaviors with respect to a particular target item ( Zhou et al. , 2018 ) . However , DIN does not model dependencies between elements in the sequence . Research has shown the importance of learning long-range intra-sequence dependencies in sequential modeling ( Vaswani et al. , 2017 ) . SASRec considers pairwise intra-sequence dependencies and the maximum length of signal traversal paths is O ( 1 ) , while DIN models no intra-sequence dependencies . In this paper , we propose the Iterative Memory Network ( IMN ) for long sequential user behavior modeling . The target item is a memory trigger in IMN , continuously eliciting relevant information from the sequence . IMN early crosses the target item and the user sequence , walks over the sequence for multiple iterations and repeatedly updates the memory vector with new information retrieved from the sequence . The contributions of this paper are summarized as follows : • We propose the Iterative Memory Network ( IMN ) , an end-to-end differentiable framework for sequential recommenders . To the best of our knowledge , it outperforms the state-of-the-art models for equal sequence lengths , with even more significant advantages on long sequential user behavior modeling . The framework is scalable to sequential recommenders with other objective functions . • IMN is efficient with O ( L ) complexity and O ( 1 ) number of sequential operations , making it scalable to long sequences . IMN requires significantly less training and inference time , with greater memory efficiency . Implemented on user sequences of length 1000 , IMN is deployed successfully on an industrial E-commerce platform with 1500 QPS . The inference time is 30ms . There is a significant 7.29 % CTR improvement over the DIN-based industrial baseline . • To the best of our knowledge , IMN is the first sequential recommender that early crosses the user sequence and the target item . Our ablation study validates empirically that early crossing before further sequence encoding benefits the model performance compared to delayed crossing . • IMN models both long-range intra-sequence dependencies and target-sequence dependencies in O ( L ) complexity . In IMN , the memory vector encapsulates the sequence content walked over . The multi-way attention between the sequence , the target item and the memory allows for intrasequence signal passing . Self-attention achieves this with O ( L2 ) complexity , and the target attention mechanism allows for no intra-sequence dependency modeling . 2 RELATED WORK . Sequential Recommender Systems . Sequential recommenders predict the user ’ s next behavior based on his past activities . Recurrent Neural Networks ( RNNs ) are introduced for early sequential recommenders ( Wu et al. , 2016 ; Hidasi et al. , 2016 ) . Yet , RNNs are difficult to parallelize and suffer from the problem of fast forgetting ( Graves et al. , 2014 ) . First introduced in the encoder-decoder framework , attention is shown effective to replace RNNs to encode sequences ( Bahdanau et al. , 2016 ; Vaswani et al. , 2017 ) . Attention-based recommender systems include self-attention based methods ( Kang & McAuley , 2018 ; Zhou et al. , 2017 ) , target-attention based methods ( Zhou et al. , 2018 ) and methods based on the integration between RNN and attention ( Zhou et al. , 2019 ) . Target-attention based methods learn the weights for sequence items with respect to the target item . Memory Networks . Memory Networks have wide applications in Question Answering ( QA ) , finding facts for a query from the knowledge database ( Chaudhari et al. , 2021 ) . Neural Turing Machines ( NTM ) introduces the addressing-read-write mechanism for memory searching and update ( Graves et al. , 2014 ) . Weston et al . ( 2014 ) proposes the general architecture for Memory Networks . DMN , DMTN and DMN+ are the subsequent research ( Kumar et al. , 2016 ; Xiong et al. , 2016 ; Ramachandran & Sohmshetty , 2017 ) . In recommender systems , MIMN uses GRU as the controller to update user memory slots with each new clicked item ( Pi et al. , 2019 ) . 3 PROBLEM FORMULATION . The recommender system models the user-item interaction as a matrix C = { cmn } M×N , where M and N are the total number of users and items respectively . The interaction is either explicit ratings ( Koren , 2009 ) or implicit feedback ( Agarwal et al. , 2009 ) . The CTR prediction task is usually based on implicit feedback . We denote u ∈ U as user and i ∈ I as item , and the user um clicking on the item in makes cmn 1 and others 0 . User sequential modeling predicts the probability of a user u ∈ U clicking on the target item i ∈ I based on his past behaviors , i1 , i2 , ... , iL , where L is the length of the user sequence . The sequence is usually in chronological order . Our paper focuses on long sequential user behavior modeling , where the length of the user behaviors L is on the scale of thousands . We use the term ‘ target-sequence dependencies ’ to refer to dependencies between the target item and the sequence items and ‘ intra-sequence dependencies ’ to refer to dependencies between the items within the sequence . 4 ITERATIVE MEMORY NETWORK . We illustrate the Iterative Memory Network architecture in Fig.1 . The encoder layer converts user behavior features , the temporal features , and the target item features to embedding vectors . The Iterative Memory Update module is the major component in our architecture , modeling both targetsequence dependencies and intra-sequence dependencies . The Memory Enhancement module repeatedly elicits clearer memory with the target item and outputs the final user interest vector . Next , we discuss the framework in detail . 4.1 ENCODER LAYER . We use ei to denote the embedding for the sequence item i . Each user behavior consists of not only the clicked item ei , but also the action time . Different users have different action patterns , thus the action time contains important temporal information . Since it is difficult to learn a good embedding directly with continuous time features , we bucketize the time feature into multiple granularities and perform categorical feature look-ups . We slice the elapsed time with respect to the ranking time into intervals whose gap length increases exponentially . In other words , we map the time in range [ 0,1 ) , [ 1,2 ) , [ 2,4 ) , ... , [ 2k , 2k+1 ) to categorical features 0,1,2 , ... k + 1 . Positional encodings are added to represent the relative positions of sequence items . The timestamp encoding and the positional encoding have the same dimension as that of the item embedding ei to be directly summed . We encode the jth user behavior e j b as ejb = ei ⊕ et ⊕ ep ( 1 ) where ⊕ denotes element-wise sum-up . We denote the embedding for the target item as vT . It shares the item id embedding look-up table with the item id embedding ei for the sequence . 4.2 MULTI-WAY ATTENTION . We calculate the similarities between the sequence item , the target item and the memory . Then we concatenate them into a single feature vector . Similarities are in both Euclidean distance and the Hadamard distance ( denoted by ◦ ) to capture multi-faceted similarity measures . α ( e , v , m ) = [ e−m , e− v , e ◦m , e ◦ v ] ( 2 ) where e is the embedding vector for sequence items , m is the memory embedding vector and v is the embedding vector for the target item . We input the feature vector into a two-layer point-wise feed-forward network , with sigmoid as the activation function . Here point-wise feed-forward means the fully-connected feed-forward network is applied to each sequence item separately and identically . a ( e , v , m ) = σ ( W ( 2 ) σ ( W ( 1 ) α ( e , v , m ) + b ( 1 ) ) + b ( 2 ) ) ( 3 ) We have also experimented with the softmax activation function for the second layer . There is very slight change in model performance . The multi-way attention mechanism encodes similarities between the user behavior sequence and the target item to extract the user ’ s interest specific to a particular target item . It also encodes similarities between sequence items and the memory . As the memory vector encapsulates the sequence information , the multi-way attention mechanism can model intra-sequence dependencies between items in the sequence and reduce the maximum length of signal traversal paths to O ( 1 ) . 4.3 ITERATIVE MEMORY UPDATE MODULE . In the Iterative Memory Update module , the target item acts as the memory trigger , continuously waking up relevant memory in the sequence . The memory vector is updated after each iteration with the relevant information retrieved from this iteration . We use a GRU to model the memory update process . The initial memory m0 is initialized from the target item vector , to represent that the user ’ s initial memory on the target item is the target item itself . m0 = vT ( 4 ) For iteration t , we calculate the user interest representation vtu as a weighted sum pooling of sequence item vectors , with weights derived from Eq. ( 3 ) . vtu = f ( vT , e 1 b , e 2 b , ... , e L b ) = L∑ j=1 a ( ejb , vT , m t−1 ) ejb = L∑ j=1 wje j b ( 5 ) where e1b , e 2 b , ... , e L b is the list of user behavior embedding vectors , vT is the embedding vector of target item T and m is the memory embedding vector . We use the user interest representation vtu after iteration t as the input to update the memory m t−1 . mt = GRU ( vtu , m t−1 ) ( 6 ) Since the memory is updated with a weighted sum pooling of sequence item embeddings , the memory contains information on the sequence items . As both the memory and the sequence contains information about the sequence items , the multi-way attention models intra-sequence dependencies . The architecture models co-occurence beyond the ( target item , sequence item ) pair . For example , the target item is rum and the user sequence contains lime and peppermint . With the target attention mechanism , both the attention weight between the pair ( peppermint , rum ) and that between the pair ( lime , rum ) are not high . With our IMN architecture , after the first pass of sequence walk , the memory vector contains information about both the peppermint and rum . When it meets the item lime again , the memory of peppermint and rum awakens the item lime since the triplet ( rum , peppermint , lime ) is the recipe for Mojito and likely to co-occur multiple times in different samples . Hence , with lime and peppermint in the sequence , the likelihood to click rum increases . While the target attention mechanism finds the items that co-occur frequently with the target item , our IMN finds the composite group of the user ’ s behavior items for the user to click the target item . Furthermore , IMN is structurally different from Memory Networks , which updates the memory with each incoming input and uses the updated memory for computations on the next input . The sequential nature makes it difficult to parallelize , hence infeasible to deploy on recommmender systems that need real-time responses .

Summary: This paper proposes an "iterative memory network" for long user sequence modeling such as those for ctr prediction in ads and recommender systems. The basic idea is to use a iteratively updated memory vector to interact with items in the user sequence and target item, so there is higher-order interaction intra-sequence, without the need to use self-attention. Experiments are conducted on two offline datasets comparing with some attention and memory network based methods.