Iterative Memory Network for Long Sequential User Behavior Modeling in Recommender Systems

1 INTRODUCTION . Click-through rate ( CTR ) prediction is critical for recommender systems . User sequential modeling is the key to mine users ’ interest for accurate predictions . As the user sequence gets longer , particularly with lengths longer than 1000 , the prediction task requires extraordinary long-range dependency modeling , efficient memory storage , acceptable training speed and real-time inference . Recurrent Neural Networks ( RNNs ) and the long short-term memory ( LSTM ) are employed by the early sequential recommenders ( Hidasi et al. , 2016 ; Hochreiter & Schmidhuber , 1997 ) . Graves et al . ( 2014 ) prove that LSTM forgets quickly and fails to generalize to sequences longer than 20 . Many empirical results also verify that RNN-based sequential recommenders are surpassed by attentionbased methods ( Zhou et al. , 2017 ; Kang & McAuley , 2018 ; Zhou et al. , 2018 ; Pi et al. , 2019 ) . Lately , the self-attention mechanism has proven to benefit a wide range of application domains , such as machine translation ( Vaswani et al. , 2017 ) , speech recognition ( Chan et al. , 2015 ) , reading comprehension ( Cui et al. , 2016 ; Lin et al. , 2017 ) and computer vision ( Xu et al. , 2015 ; Parmar et al. , 2019 ) . The self-attention mechanism attends to different positions in the sequence , captures the most important features and allows the model to handle long-range dependencies . SASRec adapts the self- attentive Transformer architecture for sequential recommenders and outperforms convolution-based and recurrence-based methods empirically ( Kang & McAuley , 2018 ) . However , SASRec has a quadratic complexity with respect to the sequence length , limiting its scalability to long sequences . Research on efficient self-attention is based on either sparse attention ( Li et al. , 2020 ; Zhou et al. , 2020 ; Child et al. , 2019 ; Kitaev et al. , 2020 ) or approximated attention ( Wang et al. , 2020 ) , and consequently incompetent against Transformer . SASRec is also unaware of the target item during the encoding process , which could harm its performance . Deep Interest Network ( DIN ) is the first architecture that adaptively learns the user interest representation from historical behaviors with respect to a particular target item ( Zhou et al. , 2018 ) . However , DIN does not model dependencies between elements in the sequence . Research has shown the importance of learning long-range intra-sequence dependencies in sequential modeling ( Vaswani et al. , 2017 ) . SASRec considers pairwise intra-sequence dependencies and the maximum length of signal traversal paths is O ( 1 ) , while DIN models no intra-sequence dependencies . In this paper , we propose the Iterative Memory Network ( IMN ) for long sequential user behavior modeling . The target item is a memory trigger in IMN , continuously eliciting relevant information from the sequence . IMN early crosses the target item and the user sequence , walks over the sequence for multiple iterations and repeatedly updates the memory vector with new information retrieved from the sequence . The contributions of this paper are summarized as follows : • We propose the Iterative Memory Network ( IMN ) , an end-to-end differentiable framework for sequential recommenders . To the best of our knowledge , it outperforms the state-of-the-art models for equal sequence lengths , with even more significant advantages on long sequential user behavior modeling . The framework is scalable to sequential recommenders with other objective functions . • IMN is efficient with O ( L ) complexity and O ( 1 ) number of sequential operations , making it scalable to long sequences . IMN requires significantly less training and inference time , with greater memory efficiency . Implemented on user sequences of length 1000 , IMN is deployed successfully on an industrial E-commerce platform with 1500 QPS . The inference time is 30ms . There is a significant 7.29 % CTR improvement over the DIN-based industrial baseline . • To the best of our knowledge , IMN is the first sequential recommender that early crosses the user sequence and the target item . Our ablation study validates empirically that early crossing before further sequence encoding benefits the model performance compared to delayed crossing . • IMN models both long-range intra-sequence dependencies and target-sequence dependencies in O ( L ) complexity . In IMN , the memory vector encapsulates the sequence content walked over . The multi-way attention between the sequence , the target item and the memory allows for intrasequence signal passing . Self-attention achieves this with O ( L2 ) complexity , and the target attention mechanism allows for no intra-sequence dependency modeling . 2 RELATED WORK . Sequential Recommender Systems . Sequential recommenders predict the user ’ s next behavior based on his past activities . Recurrent Neural Networks ( RNNs ) are introduced for early sequential recommenders ( Wu et al. , 2016 ; Hidasi et al. , 2016 ) . Yet , RNNs are difficult to parallelize and suffer from the problem of fast forgetting ( Graves et al. , 2014 ) . First introduced in the encoder-decoder framework , attention is shown effective to replace RNNs to encode sequences ( Bahdanau et al. , 2016 ; Vaswani et al. , 2017 ) . Attention-based recommender systems include self-attention based methods ( Kang & McAuley , 2018 ; Zhou et al. , 2017 ) , target-attention based methods ( Zhou et al. , 2018 ) and methods based on the integration between RNN and attention ( Zhou et al. , 2019 ) . Target-attention based methods learn the weights for sequence items with respect to the target item . Memory Networks . Memory Networks have wide applications in Question Answering ( QA ) , finding facts for a query from the knowledge database ( Chaudhari et al. , 2021 ) . Neural Turing Machines ( NTM ) introduces the addressing-read-write mechanism for memory searching and update ( Graves et al. , 2014 ) . Weston et al . ( 2014 ) proposes the general architecture for Memory Networks . DMN , DMTN and DMN+ are the subsequent research ( Kumar et al. , 2016 ; Xiong et al. , 2016 ; Ramachandran & Sohmshetty , 2017 ) . In recommender systems , MIMN uses GRU as the controller to update user memory slots with each new clicked item ( Pi et al. , 2019 ) . 3 PROBLEM FORMULATION . The recommender system models the user-item interaction as a matrix C = { cmn } M×N , where M and N are the total number of users and items respectively . The interaction is either explicit ratings ( Koren , 2009 ) or implicit feedback ( Agarwal et al. , 2009 ) . The CTR prediction task is usually based on implicit feedback . We denote u ∈ U as user and i ∈ I as item , and the user um clicking on the item in makes cmn 1 and others 0 . User sequential modeling predicts the probability of a user u ∈ U clicking on the target item i ∈ I based on his past behaviors , i1 , i2 , ... , iL , where L is the length of the user sequence . The sequence is usually in chronological order . Our paper focuses on long sequential user behavior modeling , where the length of the user behaviors L is on the scale of thousands . We use the term ‘ target-sequence dependencies ’ to refer to dependencies between the target item and the sequence items and ‘ intra-sequence dependencies ’ to refer to dependencies between the items within the sequence . 4 ITERATIVE MEMORY NETWORK . We illustrate the Iterative Memory Network architecture in Fig.1 . The encoder layer converts user behavior features , the temporal features , and the target item features to embedding vectors . The Iterative Memory Update module is the major component in our architecture , modeling both targetsequence dependencies and intra-sequence dependencies . The Memory Enhancement module repeatedly elicits clearer memory with the target item and outputs the final user interest vector . Next , we discuss the framework in detail . 4.1 ENCODER LAYER . We use ei to denote the embedding for the sequence item i . Each user behavior consists of not only the clicked item ei , but also the action time . Different users have different action patterns , thus the action time contains important temporal information . Since it is difficult to learn a good embedding directly with continuous time features , we bucketize the time feature into multiple granularities and perform categorical feature look-ups . We slice the elapsed time with respect to the ranking time into intervals whose gap length increases exponentially . In other words , we map the time in range [ 0,1 ) , [ 1,2 ) , [ 2,4 ) , ... , [ 2k , 2k+1 ) to categorical features 0,1,2 , ... k + 1 . Positional encodings are added to represent the relative positions of sequence items . The timestamp encoding and the positional encoding have the same dimension as that of the item embedding ei to be directly summed . We encode the jth user behavior e j b as ejb = ei ⊕ et ⊕ ep ( 1 ) where ⊕ denotes element-wise sum-up . We denote the embedding for the target item as vT . It shares the item id embedding look-up table with the item id embedding ei for the sequence . 4.2 MULTI-WAY ATTENTION . We calculate the similarities between the sequence item , the target item and the memory . Then we concatenate them into a single feature vector . Similarities are in both Euclidean distance and the Hadamard distance ( denoted by ◦ ) to capture multi-faceted similarity measures . α ( e , v , m ) = [ e−m , e− v , e ◦m , e ◦ v ] ( 2 ) where e is the embedding vector for sequence items , m is the memory embedding vector and v is the embedding vector for the target item . We input the feature vector into a two-layer point-wise feed-forward network , with sigmoid as the activation function . Here point-wise feed-forward means the fully-connected feed-forward network is applied to each sequence item separately and identically . a ( e , v , m ) = σ ( W ( 2 ) σ ( W ( 1 ) α ( e , v , m ) + b ( 1 ) ) + b ( 2 ) ) ( 3 ) We have also experimented with the softmax activation function for the second layer . There is very slight change in model performance . The multi-way attention mechanism encodes similarities between the user behavior sequence and the target item to extract the user ’ s interest specific to a particular target item . It also encodes similarities between sequence items and the memory . As the memory vector encapsulates the sequence information , the multi-way attention mechanism can model intra-sequence dependencies between items in the sequence and reduce the maximum length of signal traversal paths to O ( 1 ) . 4.3 ITERATIVE MEMORY UPDATE MODULE . In the Iterative Memory Update module , the target item acts as the memory trigger , continuously waking up relevant memory in the sequence . The memory vector is updated after each iteration with the relevant information retrieved from this iteration . We use a GRU to model the memory update process . The initial memory m0 is initialized from the target item vector , to represent that the user ’ s initial memory on the target item is the target item itself . m0 = vT ( 4 ) For iteration t , we calculate the user interest representation vtu as a weighted sum pooling of sequence item vectors , with weights derived from Eq. ( 3 ) . vtu = f ( vT , e 1 b , e 2 b , ... , e L b ) = L∑ j=1 a ( ejb , vT , m t−1 ) ejb = L∑ j=1 wje j b ( 5 ) where e1b , e 2 b , ... , e L b is the list of user behavior embedding vectors , vT is the embedding vector of target item T and m is the memory embedding vector . We use the user interest representation vtu after iteration t as the input to update the memory m t−1 . mt = GRU ( vtu , m t−1 ) ( 6 ) Since the memory is updated with a weighted sum pooling of sequence item embeddings , the memory contains information on the sequence items . As both the memory and the sequence contains information about the sequence items , the multi-way attention models intra-sequence dependencies . The architecture models co-occurence beyond the ( target item , sequence item ) pair . For example , the target item is rum and the user sequence contains lime and peppermint . With the target attention mechanism , both the attention weight between the pair ( peppermint , rum ) and that between the pair ( lime , rum ) are not high . With our IMN architecture , after the first pass of sequence walk , the memory vector contains information about both the peppermint and rum . When it meets the item lime again , the memory of peppermint and rum awakens the item lime since the triplet ( rum , peppermint , lime ) is the recipe for Mojito and likely to co-occur multiple times in different samples . Hence , with lime and peppermint in the sequence , the likelihood to click rum increases . While the target attention mechanism finds the items that co-occur frequently with the target item , our IMN finds the composite group of the user ’ s behavior items for the user to click the target item . Furthermore , IMN is structurally different from Memory Networks , which updates the memory with each incoming input and uses the updated memory for computations on the next input . The sequential nature makes it difficult to parallelize , hence infeasible to deploy on recommmender systems that need real-time responses .

Summary: The paper talks about a simplification of transformer architectures. The original transformers employ a dense attention between all tokens in a sequence to focus on sequence comprehension. In this work, however, the focus is on the retrieval of the last item. It therefore only evaluates the attention between the proposed candidate item and the preceding items in user history. The other novelty as I see is the replacement of Residual connections between layers with a GRU layer, though the impact is less discussed. The work is successfully deployed in industrial recommender systems, showing significant improvements in the click-through rate.