Predicting User Behaviors with Scene via Dual Sequence Networks

Authors: Xu Chen, Yunmeng SHU, Yuangang Pan, Shuai Xiao, Jinsong Lan, Qingwen Liu, Haojin Zhu, Ivor Tsang
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: dual sequence networks, scene-aware, behavior prediction

Abstract:
Modeling sequential user behaviors for future action prediction is crucial in improving user's information retrieval experience. Recent studies highlight the importance of incorporating contextual information to enhance prediction performance. One crucial and typical contextual information is the scene feature which we define it as sub-interfaces within an app, created by designers to provide specific functionalities, such as ''text2product search" and ''live" in e-commence apps. Different scenes exhibit distinct functionalities and usage habits, leading to significant distribution gap in user engagement across them. Popular sequential behavior models either ignore the scene feature or merely use it as attribute embeddings, which could lead to substantial information loss or cannot capture the interplay between scene and item in modeling dynamic user interests. In this work, we propose a novel Dual Sequence Prediction network (DSPnet) to effectively capture the interplay between scene and item sequences for future behavior prediction. DSPnet consists of two parallel networks dedicated to predicting scene and item sequences, and a sequence feature enhancement module to capture the interplay. Further, considering the randomness and noise in learning sequence dynamics, we introduce Conditional Contrastive Regularization (CCR) loss to capture the invariance of similar historical sequences. Theoretical analysis suggests that DSPnet can learn the joint relationships between scene and item sequences, and also show better robustness on real-world user behaviors. Extensive experiments are conducted on one public benchmark and two collected industrial datasets. The codes and collected datasets will be made public soon.

Decision: reject
Reviewer scores: 5, 5, 3

Reviews and discussion:
<thread>
creation date:
1730384572772

thread_type:
official_review

<thread_object>
creator: reviewer_od31

<content>
summary:
The paper focuses on using scene/behavior features to enhance sequential recommendation. The authors design a Dual Sequence Prediction network (DSPNet), constructing parallel item sequence encoders along with corresponding scene sequence encoders to form a dual-branch structure. They employ an improved contrastive learning approach to enhance the robustness of both sequence models, utilize adversarial loss for cross-branch alignment, and train the model by combining "next item prediction" with "next corresponding behavior prediction." The effectiveness of the proposed DSPNet is validated on one public dataset and two internal datasets.

strengths:
1. Using behavior type features as contextual information indeed enhances the effectiveness of sequential recommendation.
2. The paper is well-written and very easy to understand.
3. It includes experiments on industrial-scale datasets.

weaknesses:
1. **Limited novelty**. The effectiveness of using behavior features as contextual features in personalized recommendations is somewhat a consensus, so emphasizing this appears somewhat trivial. The proposed method is essentially a simple combination of existing practices, including contrastive learning, adversarial training, next-item, and next-scene/behavior prediction, which are lukewarm in this domain. Although a marginal technical contribution is made with conditional weights in contrastive learning, its superiority over standard contrastive loss does not seem supported by the experiments. While some theoretical results are provided, such as Lemma 1 proving that simultaneously predicting the next item and its corresponding scene/behavior minimizes ELBO, this does not appear significant enough to suffice as a novel technical contribution.

2. **Inconsistency between the dual-branch approach and motivation**. Modeling item sequences and scene/behavior sequences with dual branches seems disconnected or even in conflict with the stated motivation. Lines 93-94 mention that item and scene *simultaneously* occur in Figure 1(c), yet the method here appears to overlook this point. This presents a contradiction—how can independent modeling of item and scene sequences ensure their one-to-one correspondence? It is unclear whether the proposed **coarse-grained** sequence-level alignment/fusion is superior to a **fine-grained** alignment/fusion between item-scene/behavior pairs.

3. **Insufficient experimental comparisons**. The paper lacks crucial baselines that would support the superiority of the proposed method, such as: (1) self-supervised sequential recommendation models, like SASRec, S3-Rec [1], ICLRec [2], and DCRec [3]; (2) baselines using scene/behavior features as attributes [4,5]. Additionally, a simple baseline could involve adding scene/behavior embeddings to item embeddings and using straightforward models like SASRec or BERTRec, often considered strong baselines in practice; (3) multi-behavior sequential recommendation models [6-11] and multi-behavior recommendation models [12-15].

4. **Lack of context in related work**. This study's application is highly relevant to multi-behavior recommendations, particularly multi-behavior sequential recommendations [16], yet these works are overlooked without any discussion, reflecting a gap in the paper's background and depth.


--------
***References***

[1] Zhou K, Wang H, Zhao W X, et al. S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization[C]//Proceedings of the 29th ACM international conference on information & knowledge management. 2020: 1893-1902.

[2] Chen Y, Liu Z, Li J, et al. Intent contrastive learning for sequential recommendation[C]//Proceedings of the ACM Web Conference 2022. 2022: 2172-2182.

[3] Yang Y, Huang C, Xia L, et al. Debiased contrastive learning for sequential recommendation[C]//Proceedings of the ACM web conference 2023. 2023: 1063-1073.

[4] Wu L, Li S, Hsieh C J, et al. SSE-PT: Sequential recommendation via personalized transformer[C]//Proceedings of the 14th ACM conference on recommender systems. 2020: 328-337.

[5] Rashed A, Elsayed S, Schmidt-Thieme L. Context and attribute-aware sequential recommendation via cross-attention[C]//Proceedings of the 16th ACM Conference on Recommender Systems. 2022: 71-80.

[6] Elsayed S, Rashed A, Schmidt-Thieme L. Multi-Behavioral Sequential Recommendation[C]//Proceedings of the 18th ACM Conference on Recommender Systems. 2024: 902-906.

[7] Su J, Chen C, Lin Z, et al. Personalized behavior-aware transformer for multi-behavior sequential recommendation[C]//Proceedings of the 31st ACM International Conference on Multimedia. 2023: 6321-6331.

[8] Xia L, Huang C, Xu Y, et al. Multi-behavior sequential recommendation with temporal graph transformer[J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 35(6): 6099-6112.

[9] Yang Y, Huang C, Xia L, et al. Multi-behavior hypergraph-enhanced transformer for sequential recommendation[C]//Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. 2022: 2263-2274.

[10] Yuan E, Guo W, He Z, et al. Multi-behavior sequential transformer recommender[C]//Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval. 2022: 1642-1652.

[11] Liu Z, Hou Y, McAuley J. Multi-Behavior Generative Recommendation[C]//Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. 2024: 1575-1585.

[12] Jin B, Gao C, He X, et al. Multi-behavior recommendation with graph convolutional networks[C]//Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval. 2020: 659-668.

[13] Xuan H, Liu Y, Li B, et al. Knowledge enhancement for contrastive multi-behavior recommendation[C]//Proceedings of the sixteenth ACM international conference on web search and data mining. 2023: 195-203.

[14] Xia L, Xu Y, Huang C, et al. Graph meta network for multi-behavior recommendation[C]//Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval. 2021: 757-766.

[15] Wu Y, Xie R, Zhu Y, et al. Multi-view multi-behavior contrastive learning in recommendation[C]//International conference on database systems for advanced applications. Cham: Springer International Publishing, 2022: 166-182.

[16] Chen X, Li Z, Pan W, et al. A Survey on Multi-Behavior Sequential Recommendation[J]. arXiv preprint arXiv:2308.15701, 2023.

questions:
1. In Table 1, why does BERT4Rec encounter 'OOM' while the other models do not?

2. During inference, how are the metrics calculated? Besides considering the accuracy of item prediction, was the accuracy of behavior prediction also considered?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730649144784

thread_type:
official_review

<thread_object>
creator: reviewer_8nu5

<content>
summary:
This paper introduces the Dual Sequence Prediction network (DSPnet) to improve future user action prediction by modeling interdependencies between "scene" features (like “text2product search”) and item sequences. This paper claimed that traditional models often overlook these dynamics, leading to potential information loss. DSPnet addresses this by using parallel networks to capture scene and item dependencies and incorporates Conditional Contrastive Regularization (CCR) loss to handle sequence noise. The approach shows enhanced robustness and effectiveness on both public and industrial datasets.

strengths:
(1) This paper proposes a unique Dual Sequence Prediction network (DSPnet) that explicitly models both scene and item sequences. By focusing on the interdependencies between scenes and items, DSPnet captures a level of contextual nuance that is often overlooked in traditional sequential behavior models. This dual-focus approach is well-aligned with real-world applications where user behaviors are influenced by both the content itself and the surrounding context.
(2) Moreover, the introduction of CCR loss is a good point, as it addresses the noise and randomness inherent in sequential user behaviors. By focusing on the invariance of similar historical sequences, CCR loss enhances the model robustness and improves its ability to generalize to diverse, noisy data. This addition is valuable in real-world applications, where user behavior data can be unpredictable.
(3) The dataset may be valuable for the future research if it will be released as promised.

weaknesses:
**Unclear Definition and Scope of "Scene"**: The paper does not clearly define what constitutes a "scene" in the context of user behavior modeling. While scenes are described as features crafted by app or website designers (e.g., “text2product search” and “recommendation”), the criteria for selecting or categorizing these scenes remain ambiguous. It is also unclear how many types of scenes the model considers and whether these categories are generalizable to various platforms or domain-specific. A more precise definition would help readers understand the breadth of the model's applicability and the nature of the contextual features being leveraged.

**Limited Comparison with Multi-Behavior Models**: If scenes are understood as representations of multi-behavior patterns, DSPnet could benefit from comparison with established multi-behavior modeling approaches, such as the work by Cho et al. on Dynamic Multi-Behavior Sequence Modeling for Next Item Recommendation (AAAI 2023). Multi-behavior models aim to capture diverse user actions within sequences, which aligns with the paper’s goal to model interdependencies between scenes and items. A direct comparison could illustrate DSPnet’s advantages (or limitations) over these approaches in capturing the nuances of dynamic user behavior.

**Lack of Standard Metrics for Evaluation**: In evaluating DSPnet on the OutBrain dataset, the paper does not include Mean Average Precision @12, a widely recognized metric for this dataset. Including this metric would facilitate a more transparent and standardized assessment of DSPnet’s performance, allowing comparisons with other models that have used this benchmark. This would also provide a clearer view of DSPnet's efficacy across different evaluation metrics relevant to the field.

**Omission of Scene Prediction Accuracy**: Although DSPnet is designed to jointly predict both scenes and items, the accuracy of its scene prediction component is not reported. Providing this accuracy would clarify how effectively the model captures the contextual "scene" aspect in addition to item sequence modeling. Without this metric, it is challenging to evaluate the effectiveness of DSPnet’s dual-stream approach in fully capturing the dependencies between scenes and items.

**Limited Citation of Recent Works**: The paper’s references do not include any work from 2024 or later, which may indicate that it has not incorporated the most recent advancements in the field. This omission is notable for an ICLR 2025 submission, as the field of sequential user behavior modeling is rapidly evolving. Including more recent literature would strengthen the theoretical grounding of DSPnet and ensure that it is evaluated within the current landscape of user behavior modeling techniques.

**Commonality of Dual-Stream Architectures**: The paper presents DSPnet’s dual-stream architecture, which separately models scene and item sequences, as a novel approach. However, dual-stream architectures have become relatively common for tasks with multiple related objectives, such as multi-task learning frameworks. The paper would benefit from a deeper exploration of how DSPnet’s dual-stream setup is unique or offers advantages over other dual-stream or multi-task models. Additional insights on why DSPnet’s architecture is particularly suited to user behavior prediction would highlight its contributions more effectively.

questions:
(1) How many types of the scenes does this paper consider?
(2) Please justify the lack of recent works and the comparison with multi-behavior recommendation.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730650731425

thread_type:
official_review

<thread_object>
creator: reviewer_fswh

<content>
summary:
The paper addresses the challenge of predicting future user actions by effectively modeling sequential user behaviors with a focus on integrating contextual information, particularly the scene feature. Scene features, designed by app or website developers, significantly influence user engagement and exhibit distinct usage patterns and product themes. Traditional models often overlook the scene feature or treat it superficially, leading to potential information loss and failing to capture the complex interdependencies between scenes and items.

To tackle these issues, the authors propose a Dual Sequence Prediction network (DSPnet), a novel approach that simultaneously predicts scene and item sequences while capturing their inter-dependencies. DSPnet comprises two parallel networks for scene and item predictions and a sequence feature enhancement module to integrate these dependencies. To improve the model's robustness against the randomness and noise inherent in sequence data, the authors introduce a Conditional Contrastive Regularization (CCR) loss, which helps maintain the invariance of similar historical sequences during training.

Theoretical analysis indicates that DSPnet can effectively learn the joint relationships between scene and item sequences, thereby enhancing the accuracy of future behavior prediction. The effectiveness of DSPnet is validated through extensive experiments on a public benchmark dataset and two proprietary industrial datasets. The authors plan to make the source code and datasets publicly available to facilitate further research.

strengths:
The proposed Dual Sequence Prediction network (DSPnet) method exhibits several strengths:

1. **Innovative Modeling of Inter-Dependencies**: DSPnet captures the inter-dependencies between scene and item sequences, addressing a critical gap in existing sequential behavior modeling methods. By using two parallel networks and a sequence feature enhancement module, it effectively integrates the dynamics of both scenes and items, leading to more accurate and comprehensive behavior predictions.

2. **Theoretical Robustness**: The theoretical analysis demonstrates that training DSPnet is equivalent to maximizing the joint log-likelihood of both scene and item sequences. This ensures that the model can effectively learn and represent the relationships between scenes and items, enhancing its predictive power.

3. **Conditional Contrastive Regularization (CCR)**: The introduction of CCR helps in capturing the invariance of similar historical sequences, which is crucial for dealing with the randomness and noise in user behavior data. CCR uses learned conditional weights to promote similarity among sequences, thereby improving the robustness and generalizability of the model, especially in scenarios with skewed user behaviors.

4. **Rich and Diverse Datasets**: The authors have collected 37 days of sequential user behavior data from their e-commerce app, constructing two industrial datasets. These datasets, along with a public benchmark, provide a rich and diverse set of data for validating the effectiveness of DSPnet. The datasets contain chronological purchase behaviors on nearly thirty million items, addressing the research data gap in this field.

5. **Empirical Validation**: Extensive experiments on three datasets—one public benchmark and two industrial datasets—demonstrate the superior performance of DSPnet compared to state-of-the-art baselines. The results highlight the importance of incorporating scene information in sequential behavior modeling and showcase the practical benefits of the proposed method.

These strengths collectively position DSPnet as a significant advancement in the field of sequential user behavior modeling, offering improved accuracy and robustness in predicting future user actions.

weaknesses:
- The paper does not explicitly discuss the potential for integrating Conditional Contrastive Regularization (CCR) with other recommendation models.

questions:
- Can Conditional Contrastive Regularization (CCR) be integrated with other recommendation models and be effective?

- Please provide an analysis of the model's complexity and efficiency and how to tune the hyperparameters for the training loss?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732422083837

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of Key Questions to Reviewers and ACs

comment:
First, we would like to thank the reviewers for their valuable comments and constructive suggestions on the previous version of our submission. According to the comments, we have made the following changes in the revised version(changes are  marked with blue color in the new submitted pdf):

1. **Considering there is a misunderstanding of our scene feature for reviewers, we have clearly defined our scene features, setting our work apart from studies focused on multi-type behaviors.** Integrating our scene features into sequential behavior modeling presents an important problem derived from real-world industrial applications, which remains largely unexplored due to previously inaccessible data. We plan to make our extensive industrial datasets publicly available to the research community.

2. **We have highlighted the contributions of our work.** The proposed DSPnet is specifically designed to model sequential user behaviors with scenes. Its two main components: dual sequence learning with sequence feature enhancement and CCR loss, are both designed to address practical challenges in modeling sequential user behaviors with scenes.   

3. **We have included a complexity analysis of various models.** This analysis demonstrates that our DSPnet either has significantly lower complexity or does not substantially increase compared to recent state-of-the-art (SOTA) methods. Additionally, DSPnet has been successfully deployed in our system using 16 A100 GPUs.  

4. **We have conducted additional experiments including scene prediction and comparisons with more baselines to demonstrate the superiority of our method. Furthermore, we have analyzed and included more recent works in the reference.**  

5. **We illustrate the problem difference, technical difference and result difference between our work and multi-behavior works.** We demonstrate the misalignment issue between scene and item in data, and why our dual sequence-level correspondence encoder outperforms the one-to-one correspondence encoding in multi-behavior works by additional experiments and comparisons.   

We summarize the comments into the following sub-problems and answer each one separately.

Thank you for your kind consideration!   


==============================================================================    
**The used references in following response are:**     
**Reference:**    
1. Session-based Recommendation with Recurrent Neural Networks, ICLR'2016  
2. BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer, CIKM'2019    
3. Sequential Recommendation with Multiple Contrast Signals, TOIS'2023  
4. CARCA: Context and Attribute-Aware Next-Item Recommendation via Cross-Attention, RecSys'2022   
5. Enhancing Sequential Recommendation via LLM-based Semantic Embedding Learning, WWW'2024   
6. CALRec: Contrastive Alignment of Generative LLMs for Sequential Recommendation, RecSys'2024  
7. Harnessing Large Language Models for Text-Rich Sequential Recommendation, WWW'2024   
8. LLaRA: Large Language-Recommendation Assistant, ACM SIGIR'2024  
9. Plug-In Diffusion Model for Sequential Recommendation, AAAI'2024   
10. Generate what you prefer: Reshaping sequential recommendation via guided diffusion, NeurIPS'2024  
11. Conditional denoising diffusion for sequential recommendation, PAKDD'2024  
12. Linear Recurrent Units for Sequential Recommendation, WSDM'2024   
13. Mamba4rec: Towards efficient sequential recommendation with selective state space models   
14. Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts, KDD'2018   
15. Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations, RecSys'2020   
16. Scene-adaptive knowledge distillation for sequential recommendation via differentiable architecture search   
17. SceneRec: Scene-Based Graph Neural Networks for Recommender Systems   
18. LARR: Large Language Model Aided Real-time Scene Recommendation with Semantic Understanding, RecSys'2024   
19. Scene-wise Adaptive Network for Dynamic Cold-start Scenes Optimization in CTR Prediction, RecSys'2024   

---
</content>
</thread_object>

</thread>
<thread>
creation date:
1732982180443

thread_type:
official_comment

<thread_object>
creator: area_chair_pihv

<content>
title:
The deadline for Author/Reviewer discussion period is in three days!

comment:
Dear Reviewers,

Thanks again for providing your constructive comments and suggestions. The deadline for the Author/Reviewer discussion period is in three days (December 2). Please make sure to read the authors' responses and follow up with them if you have any additional questions or feedback.

Best,

AC
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper presents a Dual Sequence Prediction network (DSPnet), aiming to capture the interplay between scene and item sequences for future behavior prediction. 

**Strengths**
- It is interesting to study the scene feature, which provides critical contextual information to understand user behavior. 
- The Conditional Contrastive Regularization (CCR) loss provides an effective way to capture the invariance of similar historical sequences. 
- The effectiveness of the proposed approach is justified through both theoretical analysis and empirical evaluation. 

**Weaknesses**
- While the paper provides examples on the scene feature, a more precise definition of a "scene" in the context of user behavior modeling is missing. 
- The absence of a clear scene definition creates confusion regarding its distinction from multi-behavior modeling approaches, as noted by several reviewers.

Without a clear scene definition and more thorough distinction from existing methods in multi-behavior modeling, the overall contribution of the paper is not clearly established to meet the acceptance criteria.