Semantic Codebook Learning for Dynamic Recommendation Models

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-03 20:09:56

Abstract: Dynamic Sequential Recommendation (DSR) systems, which adapt to users' changing preferences by adjusting their parameters in real time, have emerged as a significant advancement over traditional static models. Despite their advantages, DSR systems face challenges including large item parameter spaces and heterogeneous user-item interactions, which can destabilize the recommendation process. To address these issues, we introduce the Semantic Codebook Learning for Dynamic Recommendation Models (SOLID) framework. SOLID compresses the parameter generation model's search space and utilizes homogeneity within the recommendation system more effectively. This is achieved by transforming item sequences into semantic sequences and employing a dual parameter model, which combines semantic and item-based cues to tailor recommendation parameters. Our innovative approach also includes the creation of a semantic codebook, which stores disentangled item representations to ensure stability and accuracy in parameter generation. Through extensive testing, SOLID has shown to surpass traditional DSR systems, providing more precise, stable, and dynamically adaptable recommendations.~\footnote{Our source code can be referred to \url{https://anonymous.4open.science/r/SOLID-0324}}.

Main paper: 
 related work 2.1 sequential recommendation: 
 Sequential recommendation systems analyze users' historical interactions to predict future preferences, playing a crucial role in various applications like e-commerce and short video platforms.Early approaches, such as the FPMC model [15] , leveraged non-deep learning techniques and Markov decision processes to understand user behavior sequences. However, to enhance model capabilities, recent advancements [4, 9, 11, 17, 24, 29] have shifted towards deep learning-based sequential recommendation systems. For instance, GRU4Rec [9] employs Gated Recurrent Units (GRUs) to effectively model sequential behavior, demonstrating impressive results. Additionally, models like DIN [29] and SASRec [11] incorporate attention mechanisms and transformers, respectively, to achieve rapid and efficient performance enhancements. BERT4Rec further applies the principles of BERT for superior outcomes. These four models-GRU4Rec, DIN, SASRec, and BERT4Rec-have significantly influenced both academic research and industry practices with their innovative approaches to sequential recommendation. However, these SR Models struggle to achieve optimal performance across every data distribution when dealing with users' real-time changing behaviors and interest preferences. Therefore, we have designed a more robust DSR Model that retains the advantages of existing DSR Models while addressing their shortcomings.Disentangled Representation Learning: The goal of disentangled representation learning is to parse the data into distinct, interpretable components by identifying different underlying latent factors [2, 3] . The Variational Autoencoder (VAE) [6] , a cornerstone in this field, employs variational inference alongside an encoderdecoder structure to differentiate between various latent factors.The 𝛽-VAE model [10] further refines this by adjusting the balance between the model's ability to disentangle and its capacity to represent information. This approach has found notable applications in areas like recommendation systems, where it addresses the diverse purchasing preferences of users. By incorporating multiinterest methods [12, 14] along with disentangled representation learning, several studies [20] [21] [22] [23] 28] have demonstrated significant advancements in recommendation tasks. We draw on the idea of disentangling and apply it to dynamic model parameter generation to reduce the parameter search space and leverage the homogeneous information of user behavior.Research in dynamic neural networks unfolds across two primary branches: HyperNetworks and Dynamic Filter Networks. Hyper-Networks, initially introduced by Ha et al. [8] , represent a novel approach wherein one neural network dynamically generates the parameters for another, offering a significant reduction in the number of parameters required for training, thus achieving model compression. This foundational concept has sparked extensive exploration into various applications and enhancements of HyperNetworks. For instance, Oscar et al. [5] delved into parameter initialization techniques specifically for HyperNetworks. The versatility of HyperNetworks has been demonstrated across a wide array of tasks, including continual learning [18] , graph analysis [27] , meta-learning [25] , and federated learning [16] . Recent advancements have particularly focused on leveraging HyperNetworks to generate unique network parameters based on differing data inputs, with HyperStyle [1] and HyperInverter [7] showcasing the potential for enhancing image reconstruction quality by producing distinct decoder parameters for various images. APG [26] and DUET [13] are the most recently and the SOTA dynamic recommendation models. However, existing DSR models are affected by the heterogeneity of user behavior, the sparsity of user-item interactions, etc., leading to drawbacks such as an overly large parameter search space and inaccurate parameter generation. Our method effectively addresses these shortcomings of the DSR models. 
methodology 3.1 notations and problem formulation: 
 First, we introduce the notation in sequential recommendations. We use E v and E c to represent the item feature set and semantic feature set extracted from 𝑠 𝑣 and 𝑠 𝑐 respectively. Specifically,E v = {𝑒 1 𝑣 , 𝑒 2 𝑣 , ..., 𝑒 𝐿 𝑠 𝑣 }, E c = {𝑒 1 𝑐 , 𝑒 2 𝑐 , ..., 𝑒 𝐿 𝑠 𝑐 }. e v and e c are the sequence features obtained through sequence feature extraction models such as Transformer or GRU, via E v and E c , respectively. The length of an item representation or a semantic representation is set to 𝐿 𝑟 .3.1.4 Formula. Sequential Recommendation Models (SR), Dynamic Sequential Recommendation Models (DSR), and Disentangled Multimodal Dynamic Sequential Recommendation Models (SOLID) can be formalized as follows:SR : M (X ori ; Θ) Recommendation Procedure Gradients ← -------------------- → Output ( Ŷ⇐⇒Y) Loss Calculation.(1) SOLID :DSR : M (X ori ; Θ 𝑠 , Θ 𝑑 = M 𝑣 (X ori )) Recommendation Procedure Gradients ← -------------------- → Output ( Ŷ⇐⇒Y) Loss Calculation . (2)                 X ori , X mm ↦ → 𝑐 = 𝑓 (𝑣, 𝑖, 𝑡) ↦ → X dec , Θ 𝑑 = M 𝑣 (X ori ) ⊕ M 𝑐 (X dec ), M (X ori ; Θ 𝑠 , Θ 𝑑 ) Recommendation Procedure Gradients ← -------------------- → Output ( Ŷ⇐⇒Y)Loss Calculation .(3) In the aforementioned formula, 𝑎 → 𝑏 indicates that the direction of information transfer is from 𝑎 to 𝑏, with the text next to it representing the content of the transfer. 𝑎 ↦ → 𝑏 signifies that 𝑏 is derived from 𝑎. The loss function can set to CE (Cross Entropy) loss and MSE (Mean Squared Error) loss, etc. However, since sequential recommendation often focuses more on CTR (Click-Through Rate) prediction tasks, and this paper is also focused on CTR prediction, the recommendation loss in this paper is CE loss and represented by 𝑙 CE .Models. DSR generate model parameters based on users' real-time user behaviors. Then the updated model is used for current recommendations. The layers whose parameters are updated are referred to as "dynamic layers", while the layers whose parameters remain unchanged are referred to as "static layers". DSR treat the parameters of one of the dynamic layers as a matrix 𝐾 ∈ R 𝑁 𝑖𝑛 ×𝑁 𝑜𝑢𝑡 , where 𝑁 𝑖𝑛 and 𝑁 𝑜𝑢𝑡 represent the number of input neurons and output neurons of a fully connected layer (FCL), respectively. DSR utilize a encoder 𝐸 𝑣 to extract the sequence feature 𝒆 𝑣 from the user's behavior sequence 𝑠 𝑣 to generate the parameters of the model's dynamic layers.EQUATIONAfter parameter generation, the parameters of the model will be reshaped into the shape of 𝐾.During training, all layers of the M 𝑣 are optimized together with the static layers of the M. The loss function L DSR is defined as follows:minEQUATION)Although the Item-based Dynamic Recommendation Model can obtain personalized model parameters based on users' real-time behavior and achieve superior performance, it also faces multiple challenges. 1) The user-item interaction is extremely sparse, leading to inaccurate item representation learning, making the model parameters customized based on item-based features inaccurate.2)The personalized model parameters obtained by this strategy are highly mixed.3) The generated parameters are not subject to any constraints, which poses challenges to the stability of the generated model. So we design the novel methods to address the challenges mentioned above.Our proposed Semantic Codebook Learning for Dynamic Recommendation Models (SOLID), which is shown in the Figure 2 . 3.For data with category labels, since the same item often belongs to multiple categories, we need to select a primary category as semantic it. First, we can define the centroid 𝑚 𝑐 of each category 𝑐, which is the average of embeddings 𝑒 for all items belonging to category 𝑐. Assuming 𝑛 𝑐 is the number of items belonging to category 𝑐, the centroid 𝑚 𝑐 for category 𝑐 can be represented as:EQUATIONwhere 𝑒 𝑣 , 𝑒 𝑖 , 𝑒 𝑡 are the representation of item ID 𝑣, item image 𝑖, item title 𝑡, respectively. Next, we compute its distance to each category center 𝑚 𝑐 . Assuming we use the Euclidean distance, it can be represented as,EQUATIONwhere ∥ • ∥ denotes the norm of the vector, typically the Euclidean norm. Finally, we select the closest category as the semantic for item 𝑣. That is, the semantic 𝑐 𝑝 for item 𝑣 can be represented as:EQUATIONThis way, we can use formulas to represent the process of computing the primary category for an item. Semantic-based Parameter Generation. After converting items into semantics, a semantic-to-parameter model can be trained. The training process is similar to that of the item-to-parameter model. The only differences are that the input for the item-toparameter model is an item sequence, whereas for the semantic-toparameter model, it is a semantic sequence; similarly, the outputs are the target item and target semantic, respectively.EQUATIONIn the above equation, 𝐸 𝑐 represents the semantic encoder, which is similar to the item encoder 𝐸 𝑣 .To obtain a semantic metacode that provides a good initialization for the semantic codebook, our strategy involves using the weights of a semantic encoder, which incorporates substantial homogeneity information for semantic encoding, as the semantic metacode. To acquire the semantic encoder, we need to disentangle the parameter generation into trunk parameters and branch parameters, using semantic sequences and item sequences for generation, respectively. Once training is complete, we can obtain the semantic metacode.Disentangling the parameter generation model into item-toparameter and semantic-to-parameter means separating the model parameters into a user model and a user group model. The advantage of the former lies in its extreme personalization, but the downside is the inaccurate parameter generation due to the high heterogeneity of data. The latter has the advantage of low data heterogeneity, allowing for more stable and accurate parameter generation, but its downside is that the personalization level of the semantic sequence is not as high as that of the item sequence. Therefore, how to complement the advantages of both is a question worth considering.Our solution is to use semantic-to-parameter as the trunk, generating the main part of the model parameters, which can be based on the semantic sequence to obtain the user group model. Then, the parameters generated by item-to-parameter act as branches, limited within a smaller threshold, to ensure the stability of the generated parameters while personalizing from the user group model to the user model. The process can be formulated as follows,EQUATION)where T is a hyperparameter to control the threshold of the generated parameter shift. The training procedure can thus be formulated as the following optimization problem.Learning. However, even if we disentangle the model parameters and deal with the item-to-parameter process, the item-to-parameter mapping still needs to be used to generate model parameters. Therefore, to further improve the accuracy of the item-to-parameter mapping, so we designe a Semantic Codebook (SC). Upon obtaining the semantic metacode, we can initialize the semantic codebook with it. Subsequently, we continue using the trunk and branch method of parameter generation, specifically semantic-to-parameter and item-to-parameter, to derive the parameters for the dynamic layer of the model. In the branch branch, the item representations are replaced with semantic codes from the codebook, which are then used to further predict model parameters. The generated model parameters are used for click prediction on item sequences, just as before, ultimately allowing for the training of the semantic codebook. The specific method for computing the loss is described below. SC is denoted as 𝐷, and 𝐷 ∈ R N 𝑐 ×𝐿 𝑟 . Specifically, we first use the weights of the semantic encoder in the semantic-to-parameter to initialize the item representation, as their dimensions are the same. Then, we encode the user's item representation. For a piece of data, as introduced in the notation description section, its item representation is E 𝑣 = {𝑒 1 𝑣 , 𝑒EQUATIONwhere 𝑙 MSE represents the MSE loss function, and the 𝜆 is a hyperparameter. 
experiments 4.1 experimental setup: 
 4.1.1 Datasets and Preprocessing. We evaluate SOLID and baselines on eight datasets: Amazon Arts (Arts), Amazon Instruments (Instruments), Amazon Office (Office), Amazon Scientific (Scientific), Amazon CDs (CDs), Amazon Electronic (Electronic), Douban Book (Book), and Douban Music (Music). Arts, Instruments, Office, and Scientific are four benchmarks that was recently released but has been widely used in the multimodal recommendation tasks [19] . CDs, Electronic, Book, and Music are four widely used public benchmarks in the recommendation tasks. The details of these datasets and preprocessing methods can be found in the Appendix. We choose the leave-one-out approach to process the dataset, taking the last action of each user as the test and all previous actions as the train. Our task is CTR (Click-through Rate) prediction, so we process these datasets into CTR prediction datasets. These datasets consist of user rating datasets with complete reviews. We treat all user-item interactions in the dataset as positive samples because having a rating implies that the user clicked on the item. Further, to ensure the training process goes smoothly with both positive and negative samples, we sample 4 negative samples for each positive sample in the training set and 99 negative samples for each positive sample in the test set.To evaluate the effectiveness of our method, we selected baselines from the following multiple categories:• Static Recommendation Models. DIN [29] , GRU4Rec [9] , SASRec [11] , and BERT4Rec [17] are all highly prevalent sequential recommendation methods in both academic research and the industry. They each incorporate different techniques, such as Attention, GRU (Gated Recurrent Unit), and Self-Attention, to enhance the recommendation process. • Dynamic Recommendation Models.DUET [13] and APG [26] consists of two parts: a parameter generation model and a main model. The main model refers to the aforementioned models like DIN, GRU4Rec, SASRec, BERT4Rec, etc. After pre-training, the parameter generation model can generate model parameters for the main model during inference based on the sample.In the experiments, we use the widely adopted AUC, UAUC, NDCG, and Recall as the metrics to evaluate model performance. They are defined as follows,EQUATIONEQUATIONEQUATIONRecall@𝐾 = (𝑢,𝑣,𝑠,𝑦EQUATIONIn the equation above, M represents the model, 1(•) is the indicator function. D + Test and D - Test represent the positive testing dataset and negative testing dataset respectively. D 𝑢,+ Test and D 𝑢,- Test represent the each user's positive testing dataset and negative testing dataset respectively, 𝑥, 𝑥 + , 𝑥 -, 𝑥 𝑢,+ , 𝑥 𝑢,-represent samples, positive samples, negative samples, each user's positive samples and negative samples, respectively. sort 𝐾 is an operator that sorts items by their scores and retrieves the top-𝐾 items with the highest scores. U is the user set.As shown in Table 1 , we evaluate the overall performance across four multimodal datasets: Arts, Instruments, Office, and Scientific.For each dataset, we test the performance of four SR Models: DIN, GRU4Rec, SASRec, and BERT4Rec. We measure performance using six metrics: AUC, UAUC, NDCG@10, Recall@10, NDCG@20, and Recall@20. For each SR Model, there are five options for DSR Models: none ("-"), APG, Ours (APG), DUET, and Ours (DUET), where "-" indicates no DSR Model usage, i.e., the inherent performance of the SR Model itself. Since the "-" option consistently performs worse than using a DSR Model, our comparison primarily focuses on the performance of APG vs. Ours (APG) and DUET vs. Ours (DUET) for each SR Model. Across all datasets, all SR Models, and all metrics, our proposed methods significantly outperform both APG and DUET.(a) Amazon CDs (b) Amazon Electronic We conduct ablation studies on each module and each modality to further analyze the impact of them. Here, we choose the Arts dataset, the SASRec model for SR, and the DUET model for DSR. Each row's and respectively indicate with and without the module/modality. The second line represents generating parameters using a semantic sequence. The third line represents the joint generation of parameters using both item sequence and semantic sequence, with joint training. The fourth line represents using semantic codebook learning without using semantic information. The fifth line represents our complete method. The experiments show that the model performs best when all three modules are used. In terms of individual modules, SCL has the greatest impact on performance. show that the fusion of three modalities-ID, Image, and Text-is not necessarily the best option. In terms of the impact on performance for individual modalities, Text > Image > ID. For the fusion of two modalities, in terms of impact on performance, ID + Text > Image + Text > ID + Image. We conduct experimental analysis on the main hyperparameters 𝜆 and T , as well as their grid search.The Impact of 𝜆. As shown in Table 4 , we fix T = 0.01 and vary 𝜆 to observe changes in performance, comparing it across six metrics. 𝜆 is a hyperparameter that influences the fusion weight between the MSE (Mean Squared Error) loss in recommendation task learning and the CE (Cross Entropy) loss in semantic codebook learning. The results indicate that the performance is optimal when 𝜆 = 0.1, suggesting that at this setting, the model achieves the best joint learning effect for the recommendation task and the semantic codebook. 5 , we fix 𝜆 = 0.1 and vary T to observe changes in performance, conducting comparisons across six metrics. T is a hyperparameter that affects the fusion weight between homogeneous information, i.e., semantic sequence to parameter, and heterogeneous information, i.e., item sequence to parameter. The results show that the optimal performance is achieved when T = 0.01, indicating that at this point, the fusion effect between semantic sequence to parameter and item sequence to parameter is the best. 5 , the horizontal axis represents 𝜆, and the vertical axis represents T . The depth of the color and the radius of the circle represent the magnitude of the value; the larger the value, the deeper the color and the larger the circle (i.e., the larger the radius). Blue, green, and orange represent the metrics UAUC, NDCG@10, and Recall@10, respectively. The results show that the best performance is achieved when 𝜆 = 0.1 and T = 0.01.  
conclusion: 
 In this paper, we have presented the Semantic Codebook Learning for Dynamic Recommendation Models (SOLID) as a solution to the limitations faced by existing dynamic sequence recommendation systems (DSR). Our framework integrates multimodal information, including images and text, with user-item interactions to enhance recommendation accuracy and adaptability. By disentangling model parameters into trunk parameters capturing generalized user behavior trends and branch parameters tailored to individual user actions, SOLID offers a more efficient and effective recommendation system. Through extensive experimentation across multiple datasets, we have demonstrated that SOLID significantly outperforms previous DSR models, with an significant improvement on extensive datasets and models. These results underscore the potential of leveraging multimodal information to advance the capabilities of dynamic recommendation systems, paving the way for more personalized and responsive user experiences in the era of digital personalization. 
: 
 dynamic sequential recommendation models (DSR) like DUET [13] and APG [26] have been developed.The DSR paradigm consists of two parts: (1) The primary model. This model has a structure similar to conventional sequential recommendation models like SASRec, but it is divided into a static layer and a dynamic layer. The parameters of the static layer remain unchanged after pre-training, whereas the parameters of the dynamic layer change with the user's behavior. ( 2) The parameter generation model. This is mainly used to sparse user behavior and generate the parameters for the dynamic layer of the primary model based on this behavior. The DSR paradigm enables traditional static sequential recommendation models to quickly adjust their parameters according to the potential shift of interests and intentions reflected in user behaviors, thus dynamically obtaining more interest-aligned models in real time.Despite the promising potential of Dynamic Sequential Recommendation (DSR) systems, they face significant challenges, primarily stemming from the item-to-parameter modeling scheme: (1) A large number of items result in a vast search space for the parameter generation model. Slight variations in user behavior sequences, such as "shirt, tie, suit" versus "tie, shirt, suit, " which suggest similar preferences, can unpredictably alter the item-to-parameter modeling, introducing complexity and potential instability. (2) The interaction between users and items is generally sparse and potentially noisy (e.g., the notorious implicit feedback issue), leading to heterogeneous behavior sequences that complicate the learning of accurate item representations. This results in inaccurate item representation learning, weakening the precision of model parameter customization based on item sequence features, and further exacerbating the inaccuracy of generated parameters.To address these issues, we propose the Semantic Codebook Learning for Dynamic Recommendation Models (SOLID). The core objective of SOLID is to compress the search space of the parameter generation model, promoting homogeneity signals utilization within the recommendation system. We construct a semantic codebook that better utilizes these homogeneity signals. In this codebook, item representations are disentangled into semantics that are learned to be absorbed in the codebook elements, such that the homogeneity between items in the disentangled latent space can be established. The user-item interactions are transformed into density-enriched user-semantic interactions in the latent space. The enriched density reduces the heterogeneity and complexity of user behavior space modeling in the parameter generator. Moreover, SOLID shifts from a traditional item sequence-based parameter model to a dual approach (item sequence + semantic sequence) → model parameter, effectively merging both uniform and diverse information in a structured manner. Uniform information derived from the semantic-to-parameter part is utilized to develop parameters that generalize across certain user behaviors, while diverse information allows for the crafting of specific parameters tailored to individual behavioral nuances. Crucially, by aligning the dimensions of the codebook with those of the semantic encoder, we transform the semantic encoder into a meta-code that serves as an initial state for the codebook, further easing the modeling of parameter generation. Specifically, to reduce the search space of the parameter generation model through the semantic codebook, SOLID involves three main modules. Initially, SOLID employs a pretrained model to extract semantic components from item, image, and text features. This disentanglement transitions the focus from item sequences to semantic sequences, shifting the modeling approach from itembased to semantics-based parameter generation. This design results in trunk parameters that generalize behaviors from the entire user base to specific groups, and branch parameters that cater to individual user behaviors, both derived from semantic and item sequences respectively. Parameters derived from items are tightly controlled (e.g., ±0.01) before their integration into the dynamic layer of the primary model, ensuring a responsive and adaptive system based on real-time user activity. Despite this, branch parameters still adhere to an item-centric approach, necessitating the use of a Semantic Codebook (SC) to maintain personalization and stability in representation. This codebook stores semantic vectors of behavior, progressively aligned with the nearest matches during learning. The weights of the semantic encoder are used to initialize the SC, easing the semantic codebook learning. As demonstrated, SOLID is designed to pursue the precision, stability, and clarity of model parameter generation, trying to promote the dynamic recommendation model's response to sparse, heterogeneous, and potentially noisy user behaviors.Our contributions can thus be summarized as:• We identified the limitations of the existing Dynamic Sequential Recommendation (DSR) paradigm and designed the SOLID framework with a semantic codebook to address these deficiencies. • We first learned to disentangle the parameter generation model, vertically splitting the item-to-parameter scheme into semantic-to-parameter as the trunk and item-to-parameter as the branch. This approach ensures that the generated model parameters contain both homogeneous and heterogeneous information. • To enhance the semantic codebook learning, we transformed the semantic encoder used for trunk parameter generation into a semantic metacode, which is used to initialize the semantic codebook. • We conducted extensive experiments across multiple datasets, where varied analysis demonstrates the rationality and efficacy of SOLID. 
 
 Appendix:  


Decision: Reviewer 96Rr's feedback is too brief, and despite requests for further expansion, the reviewer declined to elaborate, leading to the exclusion of the comments from the decision-making process. Besides, because reviewers qgia and WDiW offer divergent perspectives, I have carefully examined the manuscript and the authors' rebuttal. While the manuscript's motivation is not sufficiently robust, the overall methodology is novel, and the experiments are well-executed. The authors' detailed responses to the critiques of reviewers qgia and WDiW are particularly noteworthy. Based on these considerations, I recommend the acceptance of this paper. It is advised, however, that the authors carefully address the further inquiries posed by Reviewer WDiW in their subsequent revisions.

Reviews: review: paper_summary: paper_summary: This manuscript focuses on the problem of changing users' preferences while generating recommendations. The semantic codebook is used to replace some old parameters with the current encoder parameters.
strength_weakness: strength_weakness: 1. Presentation
2. Experiments including the improvements.
3. The topic is interesting.
limitations: limitations: None.
paper_summary: paper_summary: This article primarily introduces a framework called Semantic Codebook Learning for Dynamic Recommendation Models (SOLID), aimed at addressing challenges faced by Dynamic Sequential Recommendation (DSR) systems. DSR systems, which adjust their parameters in real time to adapt to users' evolving preferences, offer significant advancements over traditional static models.
strength_weakness: strength_weakness: 1.The authors identified limitations within the existing Dynamic Sequential Recommendation (DSR) paradigm, indicating a clear understanding of the challenges in the field.
2.The introduction of the SOLID framework, coupled with a semantic codebook, represents an innovative solution to address the deficiencies observed in the DSR paradigm.
limitations: limitations: Q1: How can we demonstrate that the method proposed in this paper, shifting the modeling approach from item-based to semantics-based parameter generation, addresses the complexity and instability issues mentioned in the motivation?It would be beneficial to provide experimental results regarding the amount of parameters saved and the time-space saved compared to the baseline.

Q2: While slight variations in behavior sequences may alter the modeling of item-to-parameter relationships, what is the magnitude of their impact on the overall performance of the model? 

Q3:The complexity of the latent space has been reduced, but what is the cost of the semantic codebook? Could you provide a comparison of complexity before and after?

Q4:Could you please check the writing of the last paragraph in the introduction to make it easier to understand and structurally stronger?

Q5:Based on the ablation experiments, it is evident that the performance significantly deteriorates when modal information is not utilized. In comparison to traditional methods, the introduced approach incorporates additional modal information.  these pieces of information could primarily contribute to the performance improvement, rather than the two main reasons stated by the authors, especially considering that the semantic codebook's ability to better learn homogeneity signals, etc., is not extensively discussed in the paper. It is suggested that the authors include an experiment comparing the direct addition of modal features to traditional DSR methods without using a semantic codebook to observe the resulting effects.
paper_summary: paper_summary: This paper proposes a method named SOLID to address issues in conventional dynamic sequential recommendation models, such as instability due to minor variations in item sequences and difficulties in learning item representations due to the sparsity of user-item interactions. SOLID generates parameters based on the semantic sequences of each item, demonstrating improved accuracy in dynamic recommendation models APG and DUET when enhanced with SOLID.
strength_weakness: strength_weakness: While I am not a specialist in dynamic sequential recommendation models, the idea of transforming item sequences into semantic sequences to solve problems inherent in traditional models seems reasonable. The publication of the source code is also commendable.
limitations: limitations: Due to insufficient experimental details, I found it difficult to accept this paper:
1. As noted in Section 4.4 and the Appendix, the proposed method involves several hyperparameters. However, how these values were determined for the results in Table 1 is not described. Section 4.1.1 only mentions dividing the dataset into training and test data, without any mention of validation data. To prevent readers from suspecting that hyperparameters were tuned using the test data, it is essential to create and use validation data for hyperparameter tuning.
2. In Figures 3 and 4, it is unclear which methods labeled as "Ours" were used for SR and DSR.
3. The ablation study in Section 4.3.2 concludes that using both ID and Text yields the highest accuracy, yet it is unclear which data (ID, Image, Text) were used in Table 1. The values in Table 2 for SPG+SML+SCL match those in Table 1 for Arts in SASRec+Ours(DUET); however, Table 3 lacks a corresponding row. In addition, why are the NDCG and Recall values in Table 3 significantly lower than those in Table 1?
4. The ablation study reports only on the Arts dataset results for SASRec+Ours(DUET). Were similar results obtained for other datasets, and with other SR and DSR? To avoid suspicion that only favorable results were reported, it should be clarified if similar results were observed.
5. Section 4.3 mentions using SASRec+Ours(DUET) on the Arts dataset for evaluation, but Section 4.4 does not specify the datasets, SR, or DSR used. It concludes that the best accuracy was achieved with $\lambda = 0.1$ and $\tau = 0.01$, but it is unclear whether these values were optimal for all datasets and methods.

Minor Comments:
1. In Figure 1(d), it appears as if "Ours" achieved a UAUC and Recall@10 of 1.0 across all datasets, which could be misleading. If all values are normalized based on the value of “Ours”, then the Arts data set for Recall@10 has a DIN value of about 0.1. However, looking at the results in Table 1, the Recall@10 value of DIN is still inaccurate because it is about 90% of the value of the proposed method.
2. The definition of "Clip" in Equations 12 and 13 is not described.
3. The intention behind the phrase "The translation from Chinese to English for your text is:" in Section 4.3.1 is unclear.