An Active Masked Attention Framework for Many-to-Many Cross-Domain Recommendations

Authors: feng  zhu
xinxing  yang
longfei  li
jun  zhou
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 06:57:22

Abstract: Cross-Domain Recommendation (CDR) has been proposed to improve the recommendation accuracy in the target domain (the sparser dataset) by benefiting from the auxiliary information transferred or the knowledge learned from one or many source domains (the denser datasets). However, most of the existing CDR approaches still suffer from the problem of negative transfer caused by undifferentiated knowledge transfer, and thus the recommendation accuracy in some domains, especially in the sparser domains, is still too low, which is not practical in real application scenarios. To address this problem, we propose a novel Active Masked Attention framework, i.e., AMA-CDR, for many-to-many CDR scenarios. Our AMA-CDR pursues a higher goal for CDR approaches, i.e., \textit{improving the recommendation performance in the target domain to achieve a practically usable level}, which is meaningful and challenging in real CDR systems. Specifically, AMA-CDR adopts an end-to-end graph embedding to reduce the objective distortion between graph embedding and embedding combination. More importantly, we propose an active mask for the embedding combination to ease negative transfer, which leverages both the prior knowledge, i.e., data density, and the posterior knowledge, i.e., sample uncertainty. Extensive experiments conducted on two public datasets demonstrate that our proposed AMA-CDR models significantly outperform the state-of-the-art approaches and achieve the new goal.

Main paper: 
 introduction 1.background: 
 The data sparsity problem is a long-standing problem in many recommender systems, e.g., Amazon, Taobao, Facebook, and Tiktok, which may lead to over-fitting to the sparse training dataset. There are many classical graph-based and factorization-based solutions, e.g., Random Walk [7] , Deep Walk [40] , Probabilistic Matrix Factorization (PMF) [36] , and Bayesian Personalised Ranking (BPR) [41] , attempting to address this intractable problem in a single dataset. However, the training data in a sparse domain is very limited, which means there are still not enough well-trained embeddings of users/items for information propagation in the graph-based models and for neighbourhood integration in the factorization-based models. It is realised that without the help of the auxiliary information from other related domains, the traditional single-domain approaches cannot break the above-mentioned bottleneck. Thus, Cross-Domain Recommendation (CDR) [2] is proposed to leverage the auxiliary information from the dense domain to help improve the recommendation accuracy in the sparse domain.According to transfer paradigms, existing CDR models can be generally classified into four big groups: (1) one-to-one paradigm [4, 48, 54] , (2) many-to-one paradigm [22, 39, 59, 60] , (3) one-to-many paradigm [20, 22] , and (4) many-to-many paradigm [8, 28, 37, 67] . In this paper, we mainly focus on the hardest but meaningful CDR scenario, i.e., many-to-many CDR, which leverages the knowledge from multiple domains (many) to improve the recommendation in multiple domains (many). The first two categories are called single-target CDR and the last two categories are called multi-target CDR as well in the literature [57, 66] .However, for the existing CDR approaches, the performance improvement in the target domain is still limited. Most of the existing approaches only focus on performance improvements compared with some single-domain models and their state-of-the-art CDR models. For example, they can improve the sparse domains from 0.2 to 0.22 (i.e., the improvement is 10%) in terms of the NDCG (Normalised Discounted Cumulative Gain) metric. Although the performance improvement is significant compared with their poor baselines, the recommendation performance (NDCG = 0.2) is still too low in real application scenarios. It should be highlighted that if the performance in the target domains cannot reach an ideal level, the relative improvement of performance is useless in real recommendation scenarios. Therefore, in addition to enhancing the recommendation performance in all domains, in this paper, we also propose a new goal for CDR approaches, i.e., improving the recommendation performance in the sparse domain to achieve a practically usable level, e.g., closing to the recommendation performance in the source domain. (New Goal).To achieve New Goal, we need to enhance the exchangeability of knowledge among multiple domains. In this paper, we construct interaction graphs for different domains and integrate these graphs via a suitable mask strategy, which can significantly enhance the exchangeability among multiple domains (as designed in Figure 2 ). This is because embedding propagation and combination for common users can mine the underlying relations among different graphs (domains). However, a research problem in the literature significantly reduces the effectiveness of the existing CDR approaches.Research Problem: Negative transfer caused by undifferentiated knowledge transfer. With the increase in the number of auxiliary domains, if we cannot design a reasonable mechanism to punish the negative information from auxiliary domains, the problem of negative transfer seems to be inevitable (see the detailed problem definition and analysis in the supplementary material). Although some multi-task learning (MTL) approaches [31] and transfer learning (TL) approaches [6, 42, 53] have attempted to address this problem (see the approach details in Section 4.2), some transfer-related prior knowledge, e.g., data density and sample uncertainty, in the area of CDR are ignored in these approaches.To address the above problem and achieve New Goal, in this paper, we propose a novel Active Masked Attention framework for manyto-many CDR scenarios. The characteristics and contributions of our work are summarized as follows:• In general, we design an end-to-end graph framework to reduce the objective distortion between graph embedding, i.e., our proposed revisiting graph embedding strategy, and transfer strategy, i.e., the attention for embedding combination.• To address the problem of negative transfer in many-to-many CDR scenarios, we propose an active mask for the element-wise attention weights to ease the problem of negative transfer, which leverages both the prior knowledge, i.e., data density, and the posterior knowledge, i.e., sample uncertainty, to actively choose a suitable probability of uncertain samples. Actively selecting doubtful samples and giving them a higher probability to absorb the knowledge learned from other domains can reduce their uncertainty and thus enhance the performance of all sparser and denser domains. • Extensive experiments conducted on two public datasets with seven domains demonstrate that our proposed AMA-CDR models significantly improve the state-of-the-art approaches by an average improvement of 25.34%. With the help of our AMA-CDR, in most experimental tasks, the sparser domain can even outperform the denser domain. 
the proposed model: 
 The research problem in Section 1, i.e., negative transfer, motivates us to design an effective end-to-end framework for many-to-many CDR scenarios. In this section, we introduce the general structure of our AMA-CDR and explain the rationale behind the structure.After that, we present the detailed components of our AMA-CDR, i.e., a novel Active Masked Attention framework for many-to-many CDR scenarios. Due to the space limitations, we explain the main notations and the training samples in the supplementary material.For each domain in many-to-many CDR scenarios, the users and items can form a heterogeneous graph via their interaction relations. Graph embedding has shown its powerful ability in embedding generation for recommendation [11, 30, 49, 65] . Therefore, we tend to use a graph embedding method to respectively generate the embeddings of users and items in each domain, e.g., 𝑈 𝑥 and 𝑉 𝑥 in the domain 𝐷 𝑥 (see the graph embedding of Figure 2 ).To design an effective end-to-end framework, the structure of graph embedding should be lightweight rather than complex. Therefore, in this paper, we only apply a revisited graph neural network to learn the embeddings of users/items. Next, we will introduce the process of linear embedding propagation.2.1.1 Linear embedding propagation. Given ratings of users to items, the user-item bipartite graph can be denoted as 𝐺 = ({U, V}, 𝐸), with 𝐸 is the set of user-item interaction relationships, i.e., observed ratings. Inspired by the revisiting graph-based collaborative filtering in [3] , the updating rule of user/item embeddings can be represented as follows. (user 𝑢 𝑖 or item 𝑣 𝑗 ) in graph 𝐺 𝑥 , and 𝑊 𝑥,𝑞 is the transformation parameters for domain 𝐷 𝑥 .Traditional transfer strategies tend to transfer the knowledge learned from one domain to another domain. However, in many-to-many CDR scenarios, things become more complex as we need to leverage the data of multiple domains (many) to improve the recommendation performance themselves (many). Unlike the traditional transfer strategies, in this paper, we combine the embeddings of common entities generated from multiple domains, i.e., their corresponding user-item graphs. By doing so, the combined embeddings of common entities for each domain can retain all knowledge learned from the multiple domains to different extents. To this end, the finegrained attention mechanism has been widely used in the literature [26, 27, 62, 67] . For a common entity (a user 𝑢 𝑖 or an item 𝑣 𝑖 ), our attention mechanism pays different attention to the user/item embeddings generated from different domains. The biggest advantage of this mechanism is that for an individual common entity, we can integrate their representations from different domains in different proportions. Therefore, it is expected to reduce the effect of inaccurate embeddings learned from sparser domains and emphasize the accurate embeddings learned from denser domains. Also, the attention mechanism is fine-grained that corresponds to the level of elements, and thus remains more informative elements (latent representation) from each set of embedding elements in{𝑈 0 𝑖 , 𝑈 1 𝑖 , ..., 𝑈 𝑎-1 𝑖 } or {𝑉 0 𝑗 , 𝑉 1 𝑗 , ..., 𝑉 𝑎-1 𝑗 }.The combined embedding Ũ 𝑥 𝑖 of a common user 𝑢 𝑖 in the domain 𝐷 𝑥 can be represented as:EQUATIONwhere 𝑊 𝑥,𝑦 ∈ R 𝑘 is the weight vector of domain 𝑥 for domain 𝑦 and ⊙ denotes the element-wise multiplication, and 𝑘 is the embedding dimension of 𝑈 𝑦 𝑖 . In fact, we apply the softmax activation 0.05 0.13 0.03 0.04 0.85 0.56 0.66 0.95 0.10 0.33 0.31 0.01 function to guarantee 𝑎-1 𝑦=0 𝑊 𝑥,𝑦 = 1 𝑘 . Similarly, we can obtain the combined embedding Ṽ 𝑥 𝑗 of a common item 𝑣 𝑗 in the domain 𝐷 𝑥 if there are common items among the multiple domains.𝑊 ! ∈ ℝ "×$ 𝑀 ! ∈ {0,1} "×$ (𝑝 ! = 0.𝑊 %&'( ! ∈ ℝ "×$ 1 1 1 1 0 1 1 1 1 0 1 1 ⨂ 𝑊 !,! 𝑊 !,# 𝑊 !,$As introduced in Section 2.2, the attention mechanism can control the importance of embeddings of users/items learned from multiple domains according to its attention weights for different domains. However, the attention weights are trained via the training samples in different domains, which means that the attention weights for sparser domains are hard to be properly tuned because of limited training samples. This is a major cause of negative transfer in manyto-many CDR scenarios. Accordingly, in this section, we propose another promising solution, i.e., a new active mask on the attention mechanism, to achieve a better recommendation performance in many-to-many CDR scenarios. Especially for sparser domains, with the help of the proposed active mask, we can verify the claim in Section 1, i.e., sparser domains can outperform denser domains.The proposed attention mechanism in Section 2.2 ignores the difference among multiple domains and expects that the attention weights can be properly tuned. To avoid negative transfer, we can obtain some useful prior knowledge about the difference among multiple domains according to the statistics of training datasets. In this paper, we choose the density of training datasets to measure the demand degree of transferred knowledge from other domains. The general idea of this strategy is that if a domain is sparse, then it needs more transferred knowledge from other domains, and vice versa. Specifically, we will mask fewer attention weights for sparser domains (transferring more knowledge from other domains) and mask more attention weights for denser domains (remaining more knowledge learned from themselves). A similar strategy has been used in some existing CDR approaches, e.g., DCDCSR [64] , which leverages the sparsity degrees to combine the user embeddings learned from different domains. However, the density of training datasets is general knowledge to measure the difference among different domains, and we still cannot distinguish the difference among different training samples. In addition to the generally masked probabilities for multiple domains according to their density, inspired by the basic idea of active learning, we also need to give more opportunities for the attention weights of uncertain samples in each training epoch to not be masked. This means that we need to transfer more knowledge from other domains for uncertain samples rather than for certain samples in each training epoch. Note that the uncertainty is a posterior knowledge that needs to be computed in each training batch. Therefore, in this section, we will utilise both the prior knowledge, i.e., density, and the posterior knowledge, i.e., uncertainty, to control the demand degree of transferred knowledge in the process of embedding combination.According to the density, the demand probability 𝑝 𝑥 of transferred knowledge of domain 𝐷 𝑥 can be represented as follows.EQUATIONwhere 𝑑𝑒𝑛 𝑥 is the density of domain 𝐷 𝑥 and 𝑑𝑒𝑛 𝑡 is the total density of all domains. Here, the demand probability 𝑝 𝑥 is a relative probability compared with other domains. For example, as shown in Table 1 of Section 3, in the Douban datasets, we can obtain the demand probabilities of all the three domains 𝑝 0 = 0.7581 (Book domain), 𝑝 1 = 0.7292 (Music domain), and 𝑝 2 = 0.5126 (Movie domain), according to Eq. (3).In this section, to avoid negative transfer, we set a mask on the attention weights of the domain 𝐷 𝑥 according to the demand degree of transferred knowledge, i.e. 𝑝 𝑥 , from other domains. For domain 𝐷 𝑥 , its attention weights for all domains (including itself) can be denoted as 𝑊 𝑥 = {𝑊 𝑥,0 ,𝑊 𝑥,1 , ..., 𝑊 𝑥,𝑎-1 }, 𝑊 𝑥 ∈ R 𝑎×𝑘 . For clarity, we take an example to show the masking process in Figure 3 . Note that for a target domain 𝐷 weights for uncertain samples. This is because uncertain samples lack confidence in their predictions, and thus they are more likely negatively affected by the knowledge transferred from other domains. In this paper, we use the entropy of the prediction to measure the uncertainty. The larger is the entropy, the more uncertain is the prediction. The uncertainty 𝛾 of a sample can be represented as follow.EQUATION)The detailed processes are demonstrated in Algorithms 1 and 2. We can obtain the masked attention weights 𝑊 𝑥 𝑚𝑎𝑠𝑘 for domain 𝐷 𝑥 according to Algorithms 1 and 2. Therefore, the function of the embedding combination (Eq. ( 2)) will be improved as follows. Ũ 𝑥 𝑖 = 𝑎-1 ∑︁ 𝑦=1 𝑊 𝑥,𝑦 𝑚𝑎𝑠𝑘 ⊙ 𝑈 𝑦 𝑖 , 𝑎-1 ∑︁ 𝑦=1 𝑊 𝑥,𝑦 𝑚𝑎𝑠𝑘 = 1 𝑘 =< 1, 1, ..., 1 >∈ R 𝑘 . (5)As introduced in Section 2.3, we can obtain the masked attention weights for the combined embeddings of users and items, i.e., Ũ 𝑥 and Ṽ 𝑥 , for domain 𝐷 𝑥 . As shown in Figure 2 , we employ an MLP to learn the interaction relations between users and items. Therefore, we can obtain the output of the MLP in the domain 𝐷 𝑥 as follows.EQUATIONwhere 𝑓 𝑢 ( * ) and 𝑓 𝑣 ( * ) are the MLP networks with the activation function ReLU for user and item embeddings. Θ 𝑥 𝑀𝐿𝑃,𝑈 and Θ 𝑥 𝑀𝐿𝑃,𝑉 are the corresponding parameters of user MLP and item MLP. Next, we can obtain the rating predictions in the domain 𝐷 𝑥 via a cosine similarity as follows:EQUATIONTo avoid the seesaw phenomenon caused by a joint objective, we train our proposed model in each domain via a separate optimiser. The objective function of our proposed AMA-CDR in domain 𝐷 𝑥 can be represented as follows.minEQUATION 
experiments and analysis: 
 We conduct extensive experiments on two real-world datasets, i.e., the Douban dataset with three domains and the Amazon dataset with four domains, to answer the following key questions:• Q1: How does our AMA-CDR model perform when compared with the state-of-the-art models (see Result 1)? • Q2: Can sparser domains outperform denser domains with the help of our AMA-CDR (see Result 1)? • Q3: How do the end-to-end attention and the active mask contribute to performance improvement (see Result 2)? • Q4: How does our AMA-CDR model perform when the negative transfer occurs (see Result 3)?3.1.1 Experimental Datasets and Tasks. To validate the performance of our AMA-CDR and the state-of-the-art (SOTA) baselines, we choose two public datasets, i.e., Douban 1 [63] with the 1 Douban dataset URL: https://github.com/FengZhu-Joey/GA-DTCDR/tree/main/Data three domains (DoubanBook, DoubanMusic, and DoubanMovie) and Amazon 2 [21] with the four domains (AmazonMusic, Ama-zonElectronics, AmazonMovie, and AmazonVideoGames). For the Douban dataset, we retain the users and items with at least 5 interactions, while for the Amazon dataset, we retain the users and items with at least 10 interactions. The same filtering strategy has been widely used in the existing approaches [56, 67] . Also, to train the CDR models smoothly, we serialise the user IDs of the multiple domains. The common users from different domains have duplicate IDs and distinct users in different domains have different IDs. Therefore, all users from different domains are in the same user space. For example, in the three domains of the Douban dataset, all the users from these three domains are mapped into the same user space. In this way, we can easy to combine the embeddings of common users from different domains. Note that for the Douban dataset and the Amazon dataset, the items are different among different domains, and thus we cannot combine the embeddings of common items. We list the dataset statistics and tasks in Table 1 .To ensure a fair comparison, we optimize the hyper-parameters of our AMA-CDR with those SOTA baseline models according to the parameter settings reported in their original papers. For all experimental models, we set the maximum number of training epochs to 50 and only report the best-performing result of all training epochs. For our proposed AMA-CDR, we randomly choose 7 unobserved interactions for each observed interaction (positive sample) as negative samples, set the depth of the revisiting graph 𝑞 (in Eq. ( 1)) as 2, the dimension 𝑘 of user/item embeddings is 32, the structure of the MLP for user embeddings and item embeddings in Eq. ( 6) is '𝑘', the regularisation coefficient 𝜆 is 0.001, the learning rate is 0.01, the optimiser function is Adam [19] , and the batch size is 1, 024. We list the demand probability of transferred knowledge 𝑝 for each domain in Table 1 .Similar to the majority of SOTA baselines [12, 67] , we also adopt the leave-one-out evaluation. This evaluation strategy will scan all users in a domain, and for each test user, we choose his/her latest interaction as a positive testing sample and randomly select 99 observed interactions for this test user as negative testing samples. The process of evaluation is to rank these 100 testing samples according to the prediction results of recommendation models and observe the positive testing samples in the ranking list. There are two metrics for this evaluation strategy, i.e., Hit Ratio (HR) and Normalised Discounted Cumulative Gain (NDCG) [51] . HR@𝑁 means that the recall rate when the positive testing sample rank in the top-𝑁 list. In contrast, NDCG@𝑁 measures the ranking quality of the positive testing sample in the top-𝑁 list.Considering the comparison fairness, our proposed AMA-CDR can only be compared with single-domain recommendation baselines in each domain, two-to-two CDR baselines in both two domains (a specific case of the many-to-many transfer paradigm), and many-to-many CDR baselines in multiple domains. Thus, we choose seven SOTA or representative baselines in three groups, i.e., (1) single-domain recommendation (SDR), ( 2) two-to-two CDR, and (3) many-to-many CDR. To validate the detailed contribution of the main components of our proposed The experimental results (HR@10 & NDCG@10) for Tasks 1 to 6 (the results of best-performing baselines are marked with '*' while the results of our best-performing models are marked with bold font). Note that the SDR models can be applied in each domain and thus we can obtain their results on seven domains (3 Douban domains and 4 Amazon domains). The two-to-two CDR models can not be applied in many-to-many tasks, i.e., Tasks 5 and 6, but the many-to-many CDR models can be applied in two-to-two tasks, i.e., Tasks 1 to 4 To answer Q1, we compare the recommendation performance of our AMA-CDR with those of the seven baselines listed in Table 2 . We report their experimental results in Table 3 . Table 3 demonstrates the experimental results in terms of HR@10 & NDCG@10 for the six experimental tasks (see the tasks in Table 1 ). As we can see from Table 3 , our proposed AMA-CDR can outperform all the SDR, two-to-two CDR, and many-to-many CDR baselines by an average improvement of 30.60%, including HR@10 & NDCG@10. In particular, our AMA-CDR improves the best-performing baselines (with results marked by * in Table 3 ) by an average of 30.14% for Task 1, an average of 39.42% for Task 2, an average of 22.46% for Task 3, an average of 6.21% for Task 4, an average of 29.49% for Task 5, and an average of 11.47% for Task 6. Also, for the sparser domain, our AMA-CDR can significantly improve the performance of the best-performing baseline, and the performance of the sparser domain can reach and even surpass that of the denser domain (for Q2). Therefore, for Q2, our AMA-CDR can help the sparser domain outperform the denser domain. Note that we mark the detailed results for Q2 in the last column of Table 3 . In addition to Task 4, the performance of the sparser domains in other tasks can outperform those of the denser domains. Comparison analysis. As for Q1, our proposed AMA-CDR can outperform the SOTA or representative baselines due to (1) the endto-end training paradigm that can reduce the objective distortion between the generation of graph embeddings and transfer process, and (2) the active mask mechanism that can leverage both the prior knowledge (data density) and the posterior knowledge (uncertainty of training samples) to avoid the problem of negative transfer. As for Q2, the performance of the sparser domain can reach and even surpass that of the denser domain because a higher demand probability 𝑝 (see Table 1 ) of transferred knowledge from other domains can force the recommendation system in the sparser domain to absorb more knowledge (user/item embeddings) from other domains rather than to retain their own inaccurate knowledge.To answer Q3, we implement two variants of our proposed AMA-CDR, i.e., AMA-CDR-A (only with element-wise attention and without any mask strategies) and AMA-CDR-RM (with random masked attention). To validate the detailed contributions of the two components of our proposed AMA-CDR, i.e., random masked attention and active masked attention, we compare the two variants with our full-version AMA-CDR and report their results for Tasks 5 and 6 in Figure 4 . Firstly, as shown in Figure 4 , even only with revisiting GNN and element-wise attention, our end-to-end AMA-CDR-A can significantly improve the SOTA or representative baselines in terms of HR@10 & NDCG@10. This means that if we can design suitable graph embedding and transfer strategies, our end-to-end models can significantly ease the problem of objective distortion.Secondly, comparing AMA-CDR-RM with AMA-CDR-A, we can see from Figure 4 that with the help of the random mask, i.e., leveraging the prior knowledge -data density, our AMA-CDR-RM can ease the problem of negative transfer to some extent. This means that the prior knowledge, i.e., data density, can guide our CDR model to effectively combine the knowledge, i.e., user/item embeddings, from other domains.Finally, comparing AMA-CDR (the full version) with AMA-CDR-RM, we can see from Figure 4 that with the help of the active mask, i.e., leveraging both the prior knowledge -the data density and the posterior knowledge -the uncertainty of training sample, our AMA-CDR can further address the problem of negative transfer. Actively selecting uncertain samples and giving them a higher probability to absorb the knowledge learned from other domains can reduce their uncertainty and thus enhance the performance of all sparser and denser domains.To validate the performance of our AMA-CDR when the negative transfer occurs, we report the performance of the best-performing baseline, i.e., GA-MTCDR-P, and our AMA-CDR, and compare the changes from Task 1/2 to Task 5 (Douban dataset) and from Task 3/4 to Task 6 (Amazon dataset). For the Douban dataset, from Task 1/2 (Book+Movie or Music+Movie) to Task 5 (Book+Music+Movie), we only add a new domain, while for the Amazon dataset. from Task 3/4 (Music+Electronics or Movie+VideoGames) to Task 6 (Mu-sic+Electronics+Movie+VideoGames), we add two new domains. As we can see from Figure 5 , from Task 1/2 to Task 5 and from Task 3/4 to Task 6, our AMA-CDR can improve the performance in all domains while GA-MTCDR-P occurs the phenomenon of negative transfer, i.e., with more auxiliary domains, the performance of GA-MTCDR-P decreases in some cases. This result demonstrates that our AMA-CDR can avoid negative transfer to a large extent.  
related work 4.1 cross-domain recommendation: 
 According to transfer paradigm, we classify the existing crossdomain recommendation (CDR) approaches into the following four groups. (1) One-to-one paradigm: Most traditional CDR approaches belong to the one-to-one paradigm, which tends to leverage auxiliary information from the source domain (one) to improve the recommendation accuracy in the target domain (one) (also called as single-target CDR in the literature [66] ). These approaches either leverage content information to link and share information across domains [2, 17, 43, 45, 47, 48, 50, 58] or leverage common entities (users/items) as a bridge to transfer their embeddings [5, 13, 15, 18, 23, 29, 34, 35, 52, 61, 64, 68] or rating patterns [10, 56] across domains. (2) Many-to-one paradigm: This category of CDR approaches tends to leverage the auxiliary information from multiple domains (many) to improve the recommendation accuracy in the target domain (one) [22, 60] . (3) One-to-many paradigm: There are few related approaches in this category [20, 22, 46] , which mainly focus on one-dense and many-sparse CDR scenarios. ( 4) Many-to-many paradigm: Recently, some CDR approaches have attempted to leverage the rich information from multiple domains (many) to enhance the recommendation performance in these multiple domains (many) [8, 9, 14, 16, 22, 33, 37, 37, 55, 67] . Compared with the first three paradigms, the many-to-many paradigm is more meaningful because it can almost serve all CDR scenarios.In the literature, some multi-task learning (MTL) approaches [31, 32] and transfer learning (TL) approaches [6, 42, 53] have attempted to address this problem by measuring the distribution gap between two domains or leveraging adversarial learning to re-weight the importance of samples, and thus they can generate a kind of hidden irrelevance among different domains to avoid the negative transfer. However, some transfer-related prior knowledge, i.e., data density, and the posterior knowledge, i.e., sample uncertainty, in the area of CDR are ignored in these approaches.Because we employ attention and active learning for our proposed AMA-CDR, we also review the related literature as follows. (1) Attention: Attention is first introduced in [1] , which provides a more accurate alignment for each position in a machine translation task. In the area of CDR, the attention mechanism is applied to select more informative parts of the auxiliary domains to help the target domain [25, 26, 65] . (2) Active learning: The key idea of active learning is that machine learning algorithms can achieve greater accuracy with fewer training samples if the algorithms can actively choose the samples from which are learned [44] . As for CDR, active learning can actively choose the uncertain samples that urgently need auxiliary information from other domains [38, 60] . 
conclusion and future work: 
 In this paper, we have proposed a novel Active Masked Attention framework, i.e., AMA-CDR, for many-to-many CDR scenarios. Specifically, we have designed an end-to-end graph framework to reduce the objective distortion between graph embedding and transfer strategy. More importantly, we have proposed an active mask to address the problem of negative transfer. Also, we have conducted extensive experiments to demonstrate the superior performance of our AMA-CDR and verify that the recommendation performance in the target domain can be close to or even overhauled in the source domain. In the future, we plan to study more prior knowledge and posterior knowledge to control the demand probability of transferred information and choose suitable uncertain samples. 
 
 Appendix:  


Decision: This paper propose a novel Active Masked Attention framework, i.e., AMA-CDR, for many-to-many CDR scenarios which designs an end-to-end graph framework to reduce the objective distortion between graph embedding and transmission strategy and introduces an active masking approach to tackle the problem of negative transfer. Experiments were conducted on public datasets, demonstrating superior performance compared to baseline models and alleviating the data sparsity issue to some extent.

The paper is well-written and illustrate the proposed method clear. But there are one concern raised by reviewers have not been addressed in the rebuttal materials:
1) The author's final experimental results indicate that using this method can make the recommendation performance on sparse data exceed that of a single rich domain without using cross-domain methods. This result is quite surprising because usually the sparsity of the data largely determines the quality of the results.

Overall, the concerns are well addressed by the authors. The proposed method addressed an important problem in recommendation area and the contribution is sufficient. I would recommend to accept the paper based on the discussions between reviewers. But I am an expert on the recommendation area.

Reviews: review: paper_summary: paper_summary: The paper addresses the common issue of data sparsity in cross-domain recommendation systems and then proposes a model, AMA-CDR, which designs an end-to-end graph framework to reduce the objective distortion between graph embedding and transmission strategy and introduces an active masking approach to tackle the problem of negative transfer. Experiments were conducted on Douban and Amazon datasets, demonstrating superior performance compared to baseline models and alleviating the data sparsity issue to some extent.
strength_weakness: strength_weakness: 1. The questions addressed in this article are very novel and the ones proposed are quite creative.
2. The structure diagram of the method in this article is very clear, making it easy for people to understand the author's intention.
3. The experiment was conducted relatively comprehensively, with a wide range of dimensions for comparison.
limitations: limitations: 1. When describing the shortcomings of other models, the author mentions "is still too low in real application scenarios". Is there any data to support this point? What is the real application scenario referred to?
2. The author's final experimental results indicate that using this method can make the recommendation performance on sparse data exceed that of a single rich domain without using cross-domain methods. This result is quite surprising because usually the sparsity of the data largely determines the quality of the results.
3. The author addressed two issues but only mentioned one in the introduction section.
paper_summary: paper_summary: The paper  presents a novel approach to address the challenges faced in cross-domain recommendation systems (CDR). The authors identify the problem of negative transfer, which occurs when the knowledge transferred from source domains to a target domain results in reduced recommendation accuracy, particularly in sparse domains. To tackle this, they propose the Active Masked Attention framework (AMA-CDR) for many-to-many CDR scenarios.
strength_weakness: strength_weakness: 1.The paper is well-structured, with a clear problem statement, methodology, and evaluation. 
2.The proposed AMA-CDR framework is innovative and addresses a significant challenge in CDR systems. 
3.The results are promising, demonstrating the effectiveness of the model.
limitations: limitations: 1.the paper would benefit from a more detailed discussion on how the active mask is applied in practice and any potential limitations or challenges in implementing the framework in real-world scenarios. 
2.Additionally, future work could explore how the framework scales with an increasing number of domains and how it performs in dynamic environments where domains are continually evolving.
paper_summary: paper_summary: This work focuses on many-to-many cross-domain recommendation scenarios and proposes a novel active masked attention framework, i.e., AMA-CDR. AMA-CDR adopts an end-to-end graph embedding to reduce the objective distortion between graph embedding and embedding combination and leverages an active mask for the embedding combination to ease negative transfer. The experiments show that their proposed model outperforms other baselines and achieves the new goal.
strength_weakness: strength_weakness: 1. This work mainly focuses on the hardest but meaningful CDR scenario, i.e., many-to-many CDR, and improves the recommendation performance in the sparse domain to achieve a practically usable level, with a clear and well-founded motivation.
2. The proposed model and the two designed modules, i.e., an end-to-end graph embedding and an active mask, by the authors are reasonable and highly innovative, providing a complementary approach to existing methods.
3. The authors conducted extensive experiments to demonstrate the superiority of the proposed AMA-CDR framework.
4. This work exhibits a well-organized and logically sound structure, with clear and easily understandable writing.
limitations: limitations: 1. In the introduction section, regarding the research problem (line 139), the description of the objective distortion problem is missing.
2. This work is not open source and is not easy to follow.
3. In the experimental section, for the many-to-many CDR task, the number of domains is at most 4, and more domains need to be validated.
4. In section 3.1.2, the user/item embedding dimension is set to 32. However, in more existing research work, the embedding dimension is set to either 64 or 128. Further clarification from the authors on this parameter setting would be appreciated.
5. In section 3.1.3, the authors state that the experiment conducts the top-N recommendation, but in Table 3 only top-10 results are given. Then $N$ in 3.1.3 should be changed to 10.
paper_summary: paper_summary: This article proposes active masked attention in the problem setting of multi-domain Graph-based CDR to overcome the negative transfer issue, thereby improving the performance of CDR and achieving state-of-the-art performance on several datasets. Overall, this paper selects a BITGCF-like model as the baseline and adds an attention module to improve performance.
strength_weakness: strength_weakness: This paper has the following strengths: 
--- 1. The experiments are solid and achieved significant performance improvements.
--- 2. The methodology section is easy to follow.
limitations: limitations: This paper has the following limitations: 

--- 1. The writing in the first chapter of the article is unclear and difficult to follow. For instance, in line 137, it mentions "there are two main problems in the literature," but which two problems does it refer to? Is one of the problems negative transfer? What is the other problem?
--- 2. The paper introduces a new metric, "New Goal," but it seems to me that the paper does not clearly define what "the new goal" is. My question is: in Table 3, under what circumstances can a recommended task be considered to have met the criteria for "achieving the new goal"?
--- 3. In my opinion, there is no strict logical relationship between "whether the sparser domain can outperform the denser domain" and "practically usable level." It would be better if the author provided a rigorous discussion on these issues.
--- 4. The apperance of the paper needs to be improved.