Identify Then Recommend: Towards Unsupervised Group Recommendation

Authors: yue  liu
shihao  zhu
tianyuan  yang
jian  ma
wenliang  zhong
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 18:10:07

Abstract: Group Recommendation (GR), which aims to recommend items to groups of users, has become a promising and practical direction for recommendation systems. This paper points out two issues of the state-of-the-art GR models. (1) The pre-defined and fixed number of user groups is inadequate for real-time industrial recommendation systems, where the group distribution can shift dynamically. (2) The training schema of existing GR methods is supervised, necessitating expensive user-group and group-item labels, leading to significant annotation costs. To this end, we present a novel unsupervised group recommendation framework named $\underline{\text{I}}$dentify $\underline{\text{T}}$hen $\underline{\text{R}}$ecommend ($\underline{\text{ITR}}$), where it first identifies the user groups in an unsupervised manner even without the pre-defined number of groups, and then two pre-text tasks are designed to conduct self-supervised group recommendation. Concretely, at the group identification stage, we first estimate the adaptive density of each user point, where areas with higher densities are more likely to be recognized as group centers. Then, a heuristic merge-and-split strategy is designed to discover the user groups and decision boundaries. Subsequently, at the self-supervised learning stage, the pull-and-repulsion pre-text task is proposed to optimize the user-group distribution. Besides, the pseudo group recommendation pre-text task is designed to assist the recommendations. Extensive experiments demonstrate the superiority and effectiveness of ITR on both user recommendation (e.g., 22.22\% NDCG@5 $\uparrow$) and group recommendation (e.g., 22.95\% NDCG@5 $\uparrow$). Furthermore, we deploy ITR on the industrial recommender and achieve promising results.

Main paper: 
 introduction: 
 Online platforms' expansion and information overload make finding pertinent content difficult. Recommendations help by supplying users with items that match their tastes and activities. This paper categorizes recommendations into User Recommendation (UR) and Group Recommendation (GR). UR, the conventional method, delivers personalized content for single users. Differently, GR targets user collectives, striving for a consensus to please most group members. 
limitations: 
 Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: See conclusion part.Guidelines:• The answer NA means that the paper has no limitation , while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when the image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not to penalize honesty concerning limitations. Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute wworkerCPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
basic notation & task definition: 
 In this paper, we denote U = {u 1 , u 2 , ..., u i , ..., u n }, T = {t 1 , t 2 , ..., t i , ..., t m }, and G = {g 1 , g 2 , ..., g i , ..., g k }, as user set, item set, and group set, respectively. n = |U|, m = |T |, and k = |G| is the number of users, the number of items, and the number of groups, respectively. A ∈ R n×k denotes the user-group distribution, where a ij = 1 represents that the user u i participates in the group g j , and a ij = 0 represents the opposite case. Besides, P ∈ R n×m is the user-item interaction, where p ij = 1 denotes the user u i has the interaction with the item t j , and a ij = 0 represents the opposite case. Moreover, Q ∈ R g×m is the group-item interaction, where q ij = 1 denote the group g i has the interaction with the item t j , and q ij = 1 denote the opposite case. The basic notations are summarized in Table 5 of Appendix.For the conventional group recommendation, the methods aim to recommend items to the user groups. Mathematically, give the user-group distribution A, and the training set of user-item interaction P, group-item interaction Q, the method aims to train a group recommendation model F. After training, given the test groups, F can provide the potential item recommendation for the groups. Also, F usually has the ability of user recommendation and can provide candidate items for the test users.By analyzing the above task definition, we notice that the existing group recommendation methods rely on the given user-group distribution A. However, we argue that the user-group distribution A is hard to obtain in the real-world recommendation system. First, on the large-scale data of users, we can not know how many groups they will form. Second, it is hard to know which groups the user will join in, and this process should be dynamic. On another aspect, the previous methods are trained based on group-item interaction Q. It is also a scarce resource. First, as mentioned above, the group information is hard to capture on large-scale data. Second, the interaction between groups and items is also dynamic and costs large human annotations. To this end, we raise an essential question for research and industry. Can group recommendations be handled well without the group annotations (i.e., user-group distribution A and group-item interaction Q)?This paper aims to deploy the proposed method in the real-time large-scale industrial recommendation system. On the open benchmarks, we admit that the annotations of users and groups have already existed and in the experiments, we remove them for the unsupervised experimental setting. We also admit that annotating these toy datasets may not be very expensive. However, the group assignments must change dynamically during training, especially at the early stage. However, note that on real-time large-scale data, the annotation costs a lot, and the distribution will shift dynamically. For example, in our scenario, the application contains 130 million page views and 50 million user views per day. The group assignment and annotation will be changed daily since it is a real-time recommendation. In addition, this is a newly launched application. Therefore, the activities of users will shift drastically, e.g., from new users to old users. We believe it will lead to large annotation costs and distribution shifts, and we aim to develop a pure, unsupervised group identification method for the user/group recommendation. In this scenario, performing several runs of clustering methods (the one that needs a pre-defined number of clusters) is not applicable since the search space will become very large, especially since we don't know the cluster number for the daily data. The current methods cannot deal with the data without a given number of clusters. For our proposed method, we just need one pass to determine the cluster number, which can fulfill the requirement of the daily data. The complexity of the multiple trials will be T times than our proposed method, where T denotes the time of trials. The details regarding the application can be found in Section 7.6.From this motivation, we propose a novel unsupervised group recommendation framework termed Identify Then Recommend (ITR), which can automatically identify the user groups and conduct self-supervised group recommendation, even without giving the group number k and the user-group distribution A. It mainly consists of two modules, including the group identification module and the self-supervised group recommendation module as follows.We briefly introduce the core idea and the primary designs of our proposed method before introducing the method part. Concretely, we aim to develop an unsupervised group recommendation method since we find the promising performance of existing state-of-the-art methods relies on the annotations of the groups. The experimental evidence can be found in Figure 1 . However, in the real-world scenario, the annotations regarding the group-item interactions and the user-group assignments are always not available. Labeling them in the real-time recommendation system is expensive and even impossible. To this end, we develop a pure, unsupervised group recommendation method, which can automatically discover the user groups and provide precise recommendations for them. Therefore, the core ideas of our methods are twofold, including the group identification and the group recommendation. For group identification, our initial solution is to adopt some existing clustering or community detection methods that do not require the number of clusters, e.g., DBSCAN, DeepDPM, etc. However, these methods can not automatically discover the user groups since they need other hyper-parameters, such as the radius and calculation of the density. Therefore, we design an adaptive density estimation method, which can automatically estimate the density of the samples. Then, for the group discovery, we propose a heuristic merge-and-split strategy to merge similar users into the group and split different user groups in the large group. Besides, the group embeddings are set as the learnable neural parameters and can be optimized during the learning process. Moreover, for the group recommendation part, the existing method can not deal with it in the unsupervised setting. We propose two pre-text tasks, including the pseudo group recommendation pre-text task and the pull-and-repulsion pre-text task. The pull-and-repulsion pre-text task aims to optimize the group embeddings by separating the different groups and pushing the samples together to the corresponding groups. Besides, for the pseudo group recommendation pre-text task, it generates the pseudo labels for the group recommendation task and guides the network to conduct group recommendation even without the precise annotations. By these designs, our proposed ITR is able to automatically discover the user groups and then provide precise group recommendations for them. Therefore, it can be applied in the real-time large-scale recommendation system.In this section, we propose a group identification module (GIM) to discover the user groups on the user embeddings in an unsupervised manner. Given the user set U = {u 1 , u 2 , ..., u i , ..., u n }, the item set T = {t 1 , t 2 , ..., t i , ...t m }, and their interaction P ∈ R n×m , we first encode the user and item into the latent space and obtain the user embedding U ∈ R n×d and the item embedding I ∈ R m×d , where d denotes the dimensions of latent embeddings.Adaptive Density Estimation. Then, we regard the user embeddings as the initial candidate centers of the user groups. We define the identified user groups and their embeddings asG ′ = {g ′ 1 , g ′ 2 , ..., g ′ i , ..., g ′ k ′ } and G ′ ∈ R k ′ ×d , respectively.Here, k ′ denotes the number of user groups. Note that k ′ is initialized as n and will dynamically change. After initialization, the density of each user group is adaptively estimated. Concretely, take user group g ′ i as an example, we first calculate the minimal distance d (i) min = min j dis(i, j) and maximal distance d (i) max = max j dis(i, j) from it to all other groups, where dis(i, j) denote the distance between the i-th group and the j-th group. Then, based on the quantile valuable q, we calculate the radius proposal as follows.EQUATIONwhere r q denotes the radius proposal with the quantile q. Subsequently, based on r q , we estimate the density of the user group g ′ i as formulated.EQUATIONwhere 1 dis(i,j) is an indicator function, which equals 1 when dis(i, j) ≤ r q , and equals 0 when dis(i, j) > r q . Besides, πr 2 q denotes the area of the user group g ′ i . In equation ( 2), the group with high density µ, i.e., containing more users but with less area, is more likely to be recognized as the group center, and the opposite case is more likely to be recognized as the decision boundary.Heuristic Merge-and-split Strategy. After obtaining the estimated density of each group candidate, this section proposes a heuristic merge-and-split strategy to merge similar user groups dynamically and split the distinct user groups. Specifically, we first sort the group candidates based on the density µ in descending order, and the group with high density will be processed first. This process can help our model skip some repetitive operations, therefore saving time. Then, we process each group in order based on the idea of the explore-and-exploit rule. Concretely, we design a greedy parameter α to control the exploding rate. For each group, we conduct the exploring strategy with the probability of α ∈ (0, 1], while with the probability of 1 -α, we conduct the exploiting strategy. The greedy parameter α is calculated as follows.EQUATIONwhere s explore and s all denotes the explored step and the overall steps. By this setting, the exploring rate will decrease quickly when the explored steps increase, therefore encouraging the model to explore at the beginning steps and exploit after the beginning steps.Next, we detail the exploring and exploiting strategies for our group identification task. For the exploring strategy, we aim to discover the new user group candidates based on the existing information.To achieve this target, we utilize the group embedding at this step s, denoted as g ′(s) ∈ R 1×d , and the group embedding at last step s -1, denoted as g ′(s-1) . Subsequently, the new user group is generated as follows.EQUATIONwhere σ ∈ N (0.5, 0.5) is the balance parameter. N denotes the Gaussian distribution of the 0.5 mean and 0.5 standard deviation. The equation ( 4) implies that the closer to the group center at the current step (with the highest density at the current step) or to the group center at the last step (with the highest density at the last step), with the lower possibility that the area contains the new candidate group center. In this manner, we guide the model to explore the new possible group candidate g ′new . In addition, it will be added to the existing group candidate set:G ′ ← {g ′ 1 , g ′ 2 , ..., g ′ i , ..., g ′ k ′ , g ′new }, k ′ ← k ′ + 1.Also, the adaptive density of g ′new will be estimated through equation ( 1) and (2) .For the exploiting strategy, we aim to conduct a merge or split operation for the current user group. We take the group g ′ i as an example and denote its radius and corresponding density as r i and µ i , respectively. Recall that we have sorted the user groups, so g i is the unprocessed group center with the highest density. Therefore, we first conduct a merge operation for g i as follows.EQUATIONwhere 1 dis(i,j)≤ri indicate all of the groups within the circle area with r i radius. In this manner, similar user groups are merged together into the group center with the highest density at the current step. However, we argue that the merged group may contain several isolated density peaks of the user groups. Therefore, in addition to the merge strategy, we conduct the split strategy as formulated.EQUATIONwhere 1 dis(i,j)≤ri represents that it merges the group candidates within the circle area with r i radius while (1 -1 dis(i,j)>rj ) represents that it splits the group candidates whose circle area with r j radius can not contain the current user group g i . By these designs, the user groups are first merged and then several isolated user groups are split. This merge-and-split process updates the existing group candidate set as follows.EQUATIONIn summary, in the proposed group identification module, we first estimate the adaptive density and the radius of each group candidate. Subsequently, the heuristic merge-and-split strategy dynamically identifies the user groups with the exploring and exploiting. It explores the new groups around the current decision boundary and exploits the current information to merge and split the group candidates. By these designs, our proposed method is empowered to identify groups without the scarce group annotation information. Next, we introduce how to utilize identified groups.In this section, a self-supervised group recommendation module (SGRM) is proposed to utilize the discovered user groups and promote the group recommendation. It consists of two pre-text tasks, including the pull-and-repulsion task and the pseudo-group recommendation task.Pull-and-repulsion Pre-text Task. After obtaining the discovered user groups G ′ and corresponding group center embeddings G ′ ∈ R k ′ ×d , we aim to optimize the group center embeddings along with the user embeddings via the pull-and-repulsion pre-text task as formulated.minEQUATIONwhere ũi = u i /∥u i ∥ 2 denotes the normalized user embeddings and g′ j = g ′ j /∥g ′ j ∥ 2 denotes the normalized group embeddings. In equation (8) , the pull term aims to pull the users to the user groups, while the repulsion term aims to push the distinct groups away. In this manner, the users and the identified groups are optimized in a unified framework during training, achieving better performance for both the user and group recommendations.Pseudo Group Recommendation Pre-text Task. In addition to the pull-and-repulsion pre-text task, we develop a pseudo-group recommendation pre-text task to assist the group recommendation. Concretely, we first calculate the distance D ∈ R n×k ′ between users and groups, and then estimate the user-group distribution A ′ ∈ R n×k ′ as follows.EQUATIONwhere ũi = u i /∥u i ∥ 2 denotes the normalized user embeddings and g′ j = g ′ j /∥g ′ j ∥ 2 denotes the normalized group embeddings. Subsequently, we further estimate the pseudo interactions Q ′ ∈ R k ′ ×m between the discovered user groups and the items as follows.EQUATIONwhere P ∈ R n×m denotes the user-item interaction and A ′ ∈ R n×k ′ denotes the estimated usergroup distribution. Next, we conduct a pseudo group recommendation pre-text task to enhance the performance of the model as follows.EQUATIONwhere I ∈ R m×d is the item embedding and G ′ ∈ R k ′ ×d denotes the discovered group embedding. In summary, in our proposed self-supervised group recommendation module, we conduct two pre-text tasks to refine the discovered user groups and further promote the group recommendation.We integrate the above two modules and provide the overall process of our proposed ITR model. The overall objective L ITR of ITR consists three parts, including the loss of pull-and-repulsion pre-text task L PAR , the loss of pseudo group recommendation task L PGR , and the loss of user recommendation L U2I . L U2I is a binary personalized ranking loss. L ITR is formulated as follows.EQUATIONwhere a and b denote the trade-off parameters. The process of ITR is summarized in Algorithm 1.BPR loss is a commonly used loss function in the recommendation. In our proposed method, we follow ConsRec for the BPR loss. And L U2I is the same as the L user in the ConsRec. It is formulated asL user = -us∈U 1 |Du s | (j,j ′ )∈Du s lnσ(r sj -rsj ′ ), where D us is the user-item training set sample for user u s and the (j, j ′ ) denotes the user u s prefers observed item i j over unobserved item i j ′ . For the sampling of positive and negative sample pairs, we also follow ConsRec, i.e., randomly sampling from missing data as negative instances to pair with each positive instance. For the number of negative samples, ConsRec conducts experiments and analyses in Figure 8 of their original paper. For fairness, we keep the original setting of the ConsRec.Input: user set U; item set V; user-item interaction P; epoch number E; trade-off parameter a, b; range of quantile q. Output: Trained ITR model.1:for epoch = 1, 2, ..., E do 2:Encode user embeddings U and item embeddings I.if At group identification stage then 4:Initialize group center G ′ based on U.Estimate the adaptive density via Eq. ( 1) and ( 2). 6:if Exploring then 7:Discover new user groups via Eq. ( 4). 8:else if Exploiting then 9:Conduct merge-and-split strategy via Eq. ( 6) and ( 7). 10:end if 11:end if 12:Calculate LPAR to conduct pull-and-repulsion task via Eq. ( 8).Calculate LPGR to conduct pseudo group rec. task via Eq. ( 11). 14:Calculate LU2I to conduct user recommendation task.Minimize LITR to train the ITR model via Eq. ( 12). 16: end for 17: Return Well-trained ITR model.We conduct complexity analyses of our proposed ITR method. First of all, we define the number of users, the number of groups, and the average size of groups as n, k ′ , and n/k ′ , respectively. In the process of the adaptive density estimation, the time complexity and space complexity of calculating radius proposal for one group is O(1), O(k ′ 2 ), respectively, and for all groups is O(k ′ ), O(k ′ 2 ), respectively. Then, the time complexity and space complexity of density estimation for all groups are k ′ × n/k ′ , n × k ′ , respectively. Subsequently, at the heuristic merge-and-split strategy stage, for the explore step, it takes O(k ′ ) time complexity and O(1) space complexity, respectively. And for the exploit step, it takes O(k ′ × n/k ′ ) time complexity, and O(nk ′ ) space complexity, respectively. Besides, for the proposed pseudo recommendation pre-text task, the time complexity and space complexity are O(n × k ′ ), and O(n × k ′ ), respectively. In addition, for the proposed pull-and-repulsion pre-text task, the time complexity and space complexity is O(nk ′ + k ′ 2 ), and O(nk ′ ), respectively. Moreover, for the BPR loss, the time complexity and space complexity are O(n) and O(n), respectively. Overall, the time complexity and space complexity of our proposedITR method is O(k ′ + k ′ × n/k ′ + k ′ + k ′ × n/k ′ + n × k ′ + nk ′ + k ′ 2 + n) → O(nk ′ + k ′ 2 )and O(k ′ 2 + n × k ′ + 1 + nk ′ + n × k ′ + nk ′ + n) → O(nk ′ + k ′ 2 ), respectively. Therefore, our proposed method will not bring large memory and time costs since the complexity of our method is linear to the number of users. 
experiment: 
 In this section, we aim to demonstrate the superiority of our proposed ITR model. First, we introduce the experimental setup, including the experimental environment, public benchmarks, and evaluation metrics, and compare baselines and implementation details. Second, we conduct experiments to support our motivation and demonstrate the problems in the existing group recommendation methods. Third, we show the superiority of the ITR model by comparing it with the recent state-of-the-art methods. Subsequently, we conduct ablation studies to demonstrate the effectiveness of the proposed modules in the ITR model. Moreover, we analyze the hyper-parameters and conduct A/B testing in the Appendix. Public Benchmark. We conduct experiments on two real-world public datasets, Mafengwo and CAMRa2011 [7] . Mafengwo is a tourism website where users can record their traveled venues and create or join group travel. CAMRa2011 is a real-world dataset containing individual users' and households' movie rating records.Evaluation Metric. To evaluate the ITR model, we adopt two groups of metrics, including Hit Ratio@x (HR@x) and Normalized Discounted Cumulative Gain@x (NDCG@x), where x ∈ {5, 10}.Compared Baseline. We compare ITR with twelve state-of-the-art GR methods including Pop [18] , NCF [26] , AGREE [7] , MAML, GroupIM [53] , HyperGroup [23] , HCR [32] , HHGR [75] , MeLU, CubeRec [16] , ConsRec [64] , and CoHeat [30] . The introduction refers to the Section 7.1.Implementation Detail. For the baseline methods, we use recorded results or adopt their original code to reproduce results. In our ITR model, we set b as 10. And a is set to 0.01 for Mafengwo and 10 for CAMRa2011, respectively. The range of q is set as {0.1, 0.2, 0.3}. The learning rate is set as 0.001 for CAMRa2011 and 0.0001 for Mafengwo, respectively. All results are obtained from three runs. Recall the analyses in Section 3.2, we point out the issue of the existing group recommendation methods, i.e., the promising performance relies on the extensive group annotations, including the user-group distribution and the group-item interaction. To verify our claim and support our motivation, we conduct experiments on user recommendation tasks and group recommendation tasks on two datasets. Concretely, we test the performance of ConsRec [64] and it without the group annotations. The experimental results are presented in Figure 1 , where the first and second rows denote the group recommendation and user recommendation tasks, respectively. From these results, we have the following conclusions. 1) For the group recommendation task, the performance decreases sharply when we remove the group annotations. For example, on the Mafengwo dataset, regarding to HR@5 metric, the performance drops from 0.8844 to 0.0563, leading to the failure of group recommendation.2) The group annotations have little effect on the user recommendation task. In this multi-task learning process, the group annotations may have a positive effect (since they add information) or a negative effect (since they influence user recommendations we aim to develop a new method that can handle group recommendations even without group annotations.We conduct extensive comparison experiments with twelve baselines and our ITR model on two open datasets regarding two recommendation tasks. The experimental results are presented in Table 1 (for group recommendation) and Table 2 (for user recommendation). Based on these results, we find that 1) Firstly, regarding to group recommendation task, our proposed ITR model achieves the best results compared with the state-of-the-art baselines, even without giving the group annotations. For Table 1 : Group recommendation performance. Bold and underlined values denote the best and the runner-up. • denotes the model relies on group annotations. • denotes the unsupervised model.CAMRa2011 Method HR@5 HR@10 NDCG@5 NDCG@10 Avg. HR@5 HR@10 NDCG@5 NDCG@10 Avg. example, on the CAMRa2011 dataset, our ITR model achieves 0.2040 NDCG@10 improvement compared to the runner-up. The main reason is that the proposed GIM can precisely discover the potential user groups, and the proposed SGRM can take advantage of the group information to assist the group recommendation. 2) Secondly, for the user recommendation task, benefiting from the supplemental information of discovered user groups, our ITR also achieves significant improvement and the best performance. For example, on the Mafengwo dataset, we achieved 0.0474% NDCG@5 improvement compared to the runner-up. 3) Thirdly, on the CAMRa2011 dataset, for the HR@10 metric, our method can not beat the best methods. This could be due to the fact that GroupIM is unable to effectively capture the order of user preferences. While the items of interest to the user appear in the recommended list, these items are not ranked highly. In summary, this section verifies the superiority of our proposed ITR on both user recommendation and group recommendation. For the performance of our proposed method on the CAMRa2011 dataset, we consider it as a corner case since our proposed method can beat all the baselines with different metrics. And we want to give a reasonable explanation here. For the results, we can observe that our method can beat GroupIM with HR@5 but can not beat it with HR@10. And HR@5 is a more precise metric than HR@10 since it requires the model to rank correctly in the top 5 items. Therefore, we suspect that the ranking ability of GroupIM may not be strong and robust since it can achieve very promising performance when ranking in the top 10 items but can not beat our method when ranking in the top 5 items.In this section, we conduct ablation studies to verify the effectiveness of our proposed modules, including GIM and SGRM. The experimental results are shown in Table 3 (for group recommendation) and Table 4 (for user recommendation). Concretely, "Base", "Base+GIM+PAR", "Base+GIM+PGR", and "ITR" denote the baseline with group annotations, the baseline with GIM and pull-and-repulsive pretext task, the baseline with GIM and pseudo group recommendation pre-text task, and our proposed method, respectively. From these results, we have the following observations. 1) "Base+GIM+PAR" achieves better performance compared with "Base", demonstrating the effectiveness of our proposed pull-and-repulsive pre-text task. 2) "Base+GIM+PGR" beats "Base", showing the effectiveness of our proposed pseudo group recommendation pre-text task. 3) Due to the GIM being merely a user group identification module, we can not use it to complete the recommendation task. But by combining observations 1) and 2), we find that GIM has contributed to the down-streaming tasks and brings performance improvement. 4) The combination variant "ITR" achieves the best performance. 
conclusion: 
 This paper finds that the existing group recommendation methods rely on the pre-defined and fixed group number and the expensive user-group and group-item labels. To solve this problem and improve the applicability of group recommendation, we developed a novel unsupervised group recommendation method named ITR. It first identifies the user groups in an unsupervised manner HR@5 HR@10 NDCG@5 NDCG@10 Avg. HR@5 HR@10 NDCG@5 NDCG@10 Avg. and then conducts two pre-text tasks to promote the recommendation. For the group identification, we calculate the adaptive density of groups to determine the group centers and decision boundaries aa and then conduct a heuristic merge-and-split strategy to discover new groups and merge similar user groups. For the self-supervised group recommendation, we design the pull-and-repulsion pre-text task to optimize the users and groups in a unified framework. Besides, a pseudo group recommendation task is designed by estimating user-group distribution and group-item interactions to assist user recommendation and group recommendation. Extensive experiments demonstrate the superiority and effectiveness of our proposed method on both user recommendation tasks and group recommendation tasks. Benefiting the superiority of ITR, it is also deployed on the large-scale industrial recommendation system and achieves promising improvements. However, ITR still relies on the user-item interaction for the user recommendation and group recommendation. In the future, we aim to develop an unsupervised recommendation method or the zero-shot recommendation method to solve the data sparsity problem. 
acknowledgment: 
 We thank all anonymous reviewers for their constructive and helpful reviews. This work was supported by the National Natural Science Foundation of China (No. 62325604 and 62276271).channel hyper-graph convolutional network to capture the member-level and group-level preferences.Similarly, a hierarchical hyper-edge embedding-based group recommender named Hypergroup is proposed by [23] . In addition, [75] proposes HHGR based on a hierarchical hyper-graph convolutional network and double-scale node dropout strategy. Furthermore, [70] proposes MMAN by designing the GNN-based encoder and neural-based aggregating networks. ConsRec [64] builds three distinct views that provide mutually complementary information to enable multi-view learning.Moreover, self-supervised learning [79, 66, 62, 72] becomes a valuable training paradigm to improve the representation learning capability of group recommendation methods via pre-text tasks [33, 36] . Concretely, crossCBR [46] designs the cross-view contrastive learning method by encouraging the alignment of the two separately learned views. GroupIM [53] aims to overcome data sparsity by maximizing mutual information between groups and group members. Besides, [47] utilizes a strategy that first fuses the multi-view representations before performing self-supervised learning. In addition, a dual-supervised contrastive learning method named DSCBR [61] is proposed to integrate supervised learning and self-supervised learning. Furthermore, [16] replaces the point representations with hyper-cubes to improve flexibility and capacity to account for the multi-faceted user preferences. Moreover, [30, 28, 38, 3] are proposed to solve the cold-start problem in the group recommendation.[74] solve group buying problems via multi-task learning.Although verified effective, the above methods rely on given group information in advance. Therefore, researchers aim to empower GR models with the group discovery capability by utilizing existing group annotations. Specifically, [68] CFAG is proposed to conduct group identification based on the tripartite graph convolution layers and the propagation augmentation layer. In addition, DiRec [63] is proposed by exploiting the social intent and interest-intent of users, better motivating them to join groups. Moreover, in GTGS [69] , researchers design transitional hyper-graph convolution layer and cross-view self-supervised learning to improve the performance.However, two problems with existing GR methods limit their performance and applicability. (1) The number of user groups is predefined and fixed. (2) The training scheme requires expensive group annotations, including the user-group distribution and the group-item interaction. Therefore, we develop an unsupervised group recommendation framework ITR to solve them. 
unsupervised clustering: 
 Unsupervised clustering [? ] is a fundamental and challenging task that aims to group samples into distinct clusters without supervision. By exploiting the power of unlabelled data, clustering algorithms have found applications in various domains, including computer vision [12] , natural language processing [2] , graph learning [44] , recommendation systems [45] , and community detection [55] .In the early stages, several traditional clustering methods were proposed. For example, classical kmeans clustering [25] iteratively updates cluster centers and assignments to group samples. Spectral clustering [59] constructs a similarity graph and uses eigenvalues and eigenvectors to perform clustering. In addition, GMM [50] assumes that the data distribution is a mixture of Gaussians and estimates parameters by maximum likelihood. In addition, repulsive clustering methods [34, 19, 1] cluster the data using the repulsive terms. In contrast, density-based methods [21, 51, 17] overcome the need to specify the number of clusters as a hyperparameter. In recent years, the impressive performance of deep learning has led to a growing interest in deep clustering [52, 48, 35] . For example, Xie et al. propose DEC [65] , a deep learning-based approach to clustering. They initialize cluster centers using k-means clustering and optimize the clustering distribution using a Kullback-Leibler divergence clustering loss [65, 24] . JULE [67] and DeepCluster [9] both take an iterative approach, updating the deep network based on learned data embeddings and clustering assignments. SwAV [10] , an online method, focuses on clustering data and maintaining consistency between cluster assignments from different views of the same image. DINO [11] introduces a momentum encoder to address representation collapse. Moreover, DCRN and IDCRN models introduced a dual correlation reduction strategy to tackle the issue of representation collapse [40, 41] . Additionally, the HSAN framework employed a dynamic weighting strategy to effectively mine challenging sample pairs [? ]. The SCGC model focused on simplifying graph augmentation by utilizing parameterunshared Siamese encoders along with embedding disturbance [? ]. Meanwhile, the TGC framework presented a versatile approach for deep node clustering specifically designed for temporal graphs [? ]. Despite their contributions, many of these methods struggle to handle large graphs containing millions of nodes. To address this scalability issue, approaches like S 3 GC [? ] and Dink-Net [43] were developed. Furthermore, the RGC algorithm was introduced to address the challenge of unknown cluster numbers through reinforcement learning [42] . In our research, we propose to develop an unsupervised group recommendation framework that clusters users into distinct groups and subsequently delivers personalized recommendations tailored to these user groups.We list the basic notations in Table 5 . The group identification module (GIM) aims to discover the user groups based on the embeddings in the pure, unsupervised setting. In the pure unsupervised setting, the GIM is a must-used module to discover the user groups. And the effectiveness can be verified indirectly by the performance improvement of downstream tasks. To directly demonstrate the effectiveness of the proposed group identification module, we follow your suggestion and verify the quality of the discovered user group. Concretely, on the open benchmarks, we adopt the unsupervised clustering metric Silhouette Coefficient (SC) to evaluate the quality of the discovered user groups. SC is an important indicator for evaluating clustering quality, which can help evaluate the rationality of clustering results. It combines the cohesion and separation of each data point to quantify the clustering effect, and its value range is from -1 to 1. Specifically, the larger the SC, the better the clustering effect of the data points. On the average of the four datasets used in our paper, the discovered user groups achieve 0.879 SC, demonstrating the promising clustering performance of our proposed GIM. On the industrial data, due to the labels are not available and the data size being large, we conducted case studies to demonstrate the effectiveness of GIM. Specifically, we select some hot livestream rooms for the case studies, e.g., the gold-selling livestream room, movie-ticket-selling livestream room, and the face-tissue-selling livestream room. Then, we check the user group distribution and find the major group. Subsequently, we use some tags of the user interests to provide the labels of the major groups. In this manner, we can determine the interest of the major groups in the studies livestream room. We discuss the results as follows. For the gold-selling livestream room, the interests of the major groups include jewelry & gold and financial management. Besides, for the movie-ticket-selling lives interests of the major groups include movies and entertainment. Differently, for the face-tissue-selling livestream room, the interests of the major groups include food and hygiene products. These case studies can demonstrate that our proposed GIM method can effectively group the similar users to one group and separate the different groups. Also, the captured information of the major group can also be used for the refined recommendation.For convergence, we checked the loss and performance of our method on all datasets and found that they converge well, i.e., the loss decreases and gradually converges to a low value, while the performance increases and gradually converges to a high value.We apply the propose ITR method in the livestreaming recommendation scenario. In a liveslive streaming, the users naturally form several user groups based on their interests. For instance, in a sports goods life streaming, the users with basketball interests will form a user group, while the users with badminton interests will form another user group. These user groups are important information to describe the profile of the live streaming. For example, when this live streaming begins to sell basketball-related goods, the basketball user group will be activated and be willing to click, comment, or buy merchandise. Therefore, we should guide more users with basketball interests into this live streaming. Following this principle, we aim to group the users in the live streaming live streaming different groups and utilize this group information to assist the recommendation. However, the number of user groups is always unknown and dynamic. Besides, the different live streaming rooms have different group numbers, e.g., the gold live streaming room has fewer group numbers than the groceries live streaming room since user interests in the gold live streaming room are more concentrated. In addition, the group annotation information, such as user-group distribution and group-item interactions, is also missing or needs extensive human annotation costs in this scenario. To solve these problems, we present the ITR model in this paper. The proposed GIM can identify the user groups in the live streamings, and the proposed SGRM can assist and promote the recommendation.Furthermore, upon acquiring the identified group embeddings, we can also utilize them to enhance the traffic mechanism within real-time livestreaming recommendations. In practical terms, we initially designate the primary user groups within live streaming as the room's profile. Additionally, we devise and calculate the matching score between the primary user groups and potential users, subsequently leveraging these matching scores to dynamically bolster the ranking score of relevant users, therefore improving the recommendation performance and the interactive streaming live streaming room.For the click metrics, there are for MCS, including uv (user view), uvctr (user view click-through), PV (page view), and pvctrand (page view click-through). And for the trade metrics, there are two metrics, including iv (user view) and pPV(page view). For the data scale, the application contains about 130 million page views and 50 million users per day. And the baseline model is the MMOE model for recommendation. We merely compared the baseline model with our strategy and the baseline model without our strategy to demonstrate the effectiveness of our proposed method.The baseline model is the conventional MMOE-based recommendation model. We aim to adopt our proposed method at the re-rank stage, i.e., modify the predicted score of the recommendation model by considering the user groups. Next, we assume you have carefully read the background of this application and introduce the details of adopting our method into the re-rank stage. For different live streamings, the number of user groups will be different. For example, the gold live streaming room has fewer group numbers than the groceries live streaming room since user interests in the gold live streaming room are more concentrated. Therefore, we need an unsupervised group recommendation method to discover the user groups in different live streamings. Here, we adopt our ITR method to discover the user groups and then produce the group embeddings. Then upon acquiring the identified group embeddings, we can also utilize them to enhance the traffic mechanism within real-time livestreaming recommendations. In practical terms, we initially designate the primary user groups within live streaming as the room's profile. Additionally, we devise and calculate the matching score between the primary user groups and potential users, subsequently leveraging these matching scores to dynamically bolster the ranking score of relevant users, therefore improving the recommendation performance and the interactive environments in the live streaming room. And oOurosed ITR will not be used with the group annotations since if the group annotations are easy to obtain, we don't need to develop an unsupervised learning-based group recommendation method.To further demonstrate the effectiveness of ITR, A/B testing is conducted in this section. The experimental results shown in Table 6 indicate that ITR can improve the baseline model by about 2.45% performance on average, showing the effectiveness of our ITR model. In our view, we think unsupervised learning is a setting that is more challenging compared with supervised learning. Correct me here if you have different opinions. Therefore, the unsupervised user/group recommendation is more challenge compared with the supervised ones (see the experimental results in Figure 1 ). In this paper, we aim to point out that most of the existing deep group recommendation methods are supervised, but in real-world scenarios, the labels are always missing. Therefore, we need to process the user/group recommendation in an unsupervised manner. In addition, with the unsupervised learning setting, we develop a novel group recommendation method by identify-then-recommend schema. Concretely, the designed group identification module aims to discover the user groups on the user embeddings. The proposed pseudo-group recommendation pre-text task and the pull-and-repulsion pre-text task aim to produce high-quality group embeddings and conduct precise recommendations, respectively. Besides, thanks for your suggestion on the experiment. However, the variant of "known group annotation + GIM + PAR + PGR" may not be reasonable since if we already know the group annotation, we don't need to conduct the group identification. Therefore, if the group annotation is missing (the settings of this paper), we must use the GIM and then conduct the ablation studies on the PAR and PGR. The GIM is a must-use module, and the effectiveness of GIM can be verified through the performance of the downstream tasks. The experimental results can be found in Table 3 and Table 4. The clustering methods, which need a pre-defined number of cluster centers, such as k-means and spectral clustering, cannot be applied in this scenario. For other clustering methods, which do not require the pre-defined number of cluster centers, such as DBSCAN, it is hard to deal with our scenario well since 1) the data distribution of changes plication changes dynamically; therefore, it's hard to determine the parameters regarding the density, such the radius, and the threshold. But our proposed group identification module can automatically estimate the density and discover the groups via the heuristic merge-and-split strategy, thus can be easily applied in the real-time largescale unsupervised recommendation system. 2) They can not be integrated into our framework without our proposed pre-text tasks, including the pseudo group recommendation pre-text task and the pull-and-repulsion pre-text task.In this paper, the setting of the user/group recommendation is pure unsupervised and the cluster number is not pre-given in both the open dataset and the industrial data. Therefore, during the training process, the cluster number will dynamically change on the open benchmarks since the status of the user embeddings changes. Our proposed GIM can automatically discover and optimize the group embedding and the group number. After training, the number of clusters and the status of the group embeddings are restored, and then they will be used to conduct the testing of the group recommendation and the user recommendation.Besides, on the real-time large-scale data, we first train the recommendation model using the past data. During this process, the number of clusters and the status of the group embeddings are optimized dynamically. After the training stage, they are restored to the database for serving the downstream tasks. Then, when the training data is updated, the recommendation model will be continued trained, and the number of clusters also change dynamically.In our paper, we claimed that two critical problems limit the applicability of the recent state-of-the-art methods are twofold. Firstly, the promising performance of these methods relies on a pre-defined and fixed number of user groups. Therefore, they can not handle group recommendations without giving the number of user groups, and unfortunately, the number of groups is usually unknown and dynamic in real-time industrial data. Secondly, the supervised training schema of existing GR methods requires extensive human annotations for user-group distribution and group-item interaction, easily leading to significant costs. The previous methods either rely on the pre-defined cluster number or the user-group / group-item annotations. Note that the setting of our experiments is purely unsupervised, and we are not merely solving the occasional group recommendation problem. For MoSAN [58] , in Section 3.1 of its original paper, we find that it still needs the ad-hoc group for the training. Therefore, it is not a pure, unsupervised group recommendation method. And for the COM [73] , it also uses the group assignment information. Besides, for PIT [39] , thanks for your suggestion; we missed this paper in the survey process since it seems old. It solves the group recommendation problem via the personal impact topic modeling. And following your suggestion, we will add it to the related work part in the revised paper. For the additional comparison experiments, we think it is hard to produce the results since all these methods are close-resourced. Besides, they are also not new papers and may not achieve very promising performance, especially in the pure, unsupervised learning setting.Imbalance problem is an essential factor in our scenario. We conduct some case studies in our application. Concretely, we select some hot livestream rooms for the case studies, e.g., the goldselling livestream room and the grocery-selling livestream room. For the gold-selling livestream room, the imbalance problem exists since most users in this room are interested in the gold and will form one large cluster, but the other users in this room are interested in other items and will form several small clusters. Differently, for the grocery-selling livestream room, the imbalance problem is not very serious since there are so many small user groups, and the large user group may not easily form. For the downstream task, in our scenario, the imbalance groups may not be a bad phenomenon since we can use this property to find the major user groups in the livestream room and provide recommendations for them. Guidelines: 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: We check the NeurIPS Code of Ethics and our paper conform in every aspect with them.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We demonstrate the practical application of our proposed method in real-world scenarios that directly impact people's lives.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [Yes] Justification: We protect the model's weights trained on the sensitive data of users. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: We have mentioned and cited their papers. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. 
 
 Appendix: • The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results,and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.This section presents our proposed method. First, we provide the notations and task definitions. Second, we analyze and identify the limitations of existing group recommendations and group discovery. Last, we propose our solutions to address these challenges. 


Decision: Very slightly over the threshold. Reviewers mostly praise the writing, presentation, experiments & study design, though have concerns about
-- Alignment between claimed contribution and experiments
-- Annotation and computational costs
-- Some issues of correctness (though somewhat addressed in rebuttal)
I'll also note that the reviewers don't read all that positively, in spite of marginally positive scores, and are on the short side, though the discussion is quite lengthy and adds more detail than what is in the reviews.

Reviews: review: paper_summary: paper_summary: This study pointed out two issues in group recommendation in the context of industrial applications. First, the group label can be dynamic and may require constant training. Second, the annotation cost for the supervised learning is great. To address these two issues, the study proposed an unsupervised group recommendation framework ITR, which identifies the user group in an unsupervised manner and performs two self-supervised learning tasks. Experimental results show superior improvement.
strength_weakness: strength_weakness: (1) The claimed contribution on the introduction of unsupervised learning does not align closely with the experimental results. More specifically, I think the current study did not show that the improvement indeed came from the unsupervised learning. To show that, the authors can add an additional variant of "known group annotation + GIM + PAR + PGR". 

(2) The main issues that the author pointed out in the current group recommendation problem are that the group assignment can be dynamic and the annotation cost can be high. But the current experimental setting does not seem to support the evaluation for such scenarios. Does the group assignment really change a lot? Is it really expensive to do the annotation? Can we just perform several runs of clustering methods (the ones need a pre-defined number of clusters) to decide the number of clusters? How much more would it cost compared to the current framework?
questions: questions: (1) Can the authors provide more details about the binary personalized ranking loss? What are considered the positive and negative pairs? What is the strategy of finding these pairs?

(2) What is unique about the current clustering method? In the current manuscript, it seems that replacing it with any other clustering methods that do not require a pre-defined number of clusters still works for the whole framework.

(3) What are the evaluation protocols for group and user recommendation?
limitations: limitations: (1) This study should include a comprehensive complexity analysis as it tries to address the annotation cost issues in the context of group recommendation.
paper_summary: paper_summary: This paper tackles the group recommendation problem by proposing an unsupervised group recommendation framework named ITR (Identify Then Recommend). Specifically, the paper first identifies the area and density of each region automatically, then combining with a heuristic strategy to identify groups. Then, performing group recommendation task by constructing the pseudo group-item labels to guide the self-supervised learning of the group recommendation model.
strength_weakness: strength_weakness: Weaknesses:

1.	One of the two issues used in this paper is not correct. Previous related works such as PIT[1], COM [2] or MoSAN [3] already can work well with ad-hoc groups (i.e., not require pre-defined groups). Therefore, it’s not correct to say that “the promising performance of these methods relies on a pre-defined and fixed number of user groups.” (L38-39)
2.	As a result, I believe the authors should compare with those methods above in the experiments
3.	Moreover, [1], [2], [3] provided other group recommendation datasets with a larger group size. If I recall correctly, Mafengwo and CAMRa2011 have very limited number of members in a group. Also, statistics of the datasets should be given.
4.	L298-300 “From these results, … remove the group annotations”, the papers I mentioned also already had similar observations.
5.	In Section 4.3 (L318-320), it’s unclear to me about the explanation of HR@10 for CAMRa2011 dataset. So, GroupIM is unable to effectively capture the order of user preferences, but still beat our proposed method? Does it mean that we should have another metric to evaluate the ‘order of user preferences’ to make sure our method is better than GroupIM? Please correct me here.

[1] Exploring personal impact for group recommendation. CIKM 2012.

[2] COM: a generative model for group recommendation. KDD 2014

[3] Interact and Decide: Medley of Sub-Attention Networks for Effective Group Recommendation. SIGIR 2019.
questions: questions: Please refer to my questions above. I may have more questions during the discussion phase.
limitations: limitations: Yes. According to the authors, "ITR still relies on the user-item interaction for the user recommendation and group recommendation", but they aim to address that in the future to solve the data sparsity problem.
paper_summary: paper_summary: The research topic of this paper is group recommendation, i.e., recommend items to groups of users. The authors point out two issues of existing models including the fixed number of groups and the supervised training schema. To solve these problems, they propose a novel unsupervised group recommendation model named ITR (Identify Then Recommend). Experiments demonstrate the effectiveness on user recommendation and group recommendation. ITR is deployed on industrial recommender.
strength_weakness: strength_weakness: -  The demonstration figures are missing. Additionally, the method is relatively complex. The authors should outline the core ideas and primary designs before delving into the methodology section.

-  Hyper-parameter experiments are missing. Does the proposed ITR utilize any hyper-parameters during the density estimation or the merge-and-split stage?

- What are the advantages of the proposed ITR compared to other clustering methods that do not require specifying the number of clusters, such as DBSCAN?
questions: questions: 1. How scalable is the proposed ITR? Can it be easily applied in industrial recommender systems with million users?

2. Is the number of clusters dynamic, or is it determined at the final iteration? In my view, the estimated number of clusters for open datasets may be fixed after training, but is the cluster number dynamic for industry-specific data?

3. How does the proposed method maintain balance between different groups? Are there cases where some groups contain many users while others have very few? I believe this imbalance could significantly impact the recommendation performance.
limitations: limitations: Yes
paper_summary: paper_summary: This paper addresses the group recommendation task and innovatively proposes an unsupervised approach to automatically infer user-group distributions and make suitable recommendations.

For the group identification stage, a heuristic-based merge-and-split method is developed to facilitate this inference. For group recommendation, two self-supervised pre-training objectives are introduced: group-level pseudo group recommendation and user-group alignment through pull-and-repulsion mechanisms.

Extensive experiments on both open-sourced benchmarks and industrial scenarios demonstrate the effectiveness of the proposed approach.
strength_weakness: strength_weakness: 1. Presentation error in Line 147: the matrix should have the dimension of $k \times m$.

2. Methodology part requires further clarification:

   * Although the proposed GIM is intuitive, why not directly adopt existing community detection algorithms applied to the user network? This approach is applicable when user links are available or can be adaptively constructed based on user-item interactions (e.g., adding a link between two users who consumed the same items). Additionally, what is the **complexity and convergence** of this algorithm?
   * The authors seem to omit crucial technical details. **How does the ITR model make inferences during testing**? When the test data consists of an annotated group and candidate items, how is the embedding of this group generated? What strategy is adopted for recommendation given the computed group embedding and pre-trained item embedding? This process is also unclear when applying ConsRec to the scenario without annotations.

3. Experiments part requires further explanation:

   * In the ablation study, the meaning and implementation of the "base" variant are unclear and need further explanation. I am confused how to extend ITR model to the w. group annotation scenario. 

   * **The effectiveness of GIM should be demonstrated through experiments**. How do the identified groups compare with ground-truth user-group relations? Either qualitative or quantitative evaluations would be acceptable. 

   * Details of the A/B Testing experiments should be provided, such the meaning of evaluation metrics, data scale, performance of different models, etc.

   * (A minor point as I understand time is limited for rebuttal) One research line in group recommendation is the occasional group recommendation, where testing groups are newly formed and do not appear in the training set [1,2,3,4]. I highly recommend the authors conduct supplementary experiments on these datasets (such as Yelp, Weeplaces, etc.) to provide a more comprehensive evaluation, as these datasets align well with the motivation of this study.

----

[1] Sankar et al. GroupIM: A Mutual Information Maximization Framework for Neural Group Recommendation. In SIGIR, 2020.

[2] Chen et al. Thinking Inside The Box: Learning Hypercube Representations for Group Recommendation. In SIGIR, 2022.

[3] Zhang et al. GBERT: Pre-training User Representations for Ephemeral Group Recommendation. In CIKM, 2022.

[4] Li et al. Self-Supervised Group Graph Collaborative Filtering for Group Recommendation. In WSDM, 2023.
questions: questions: Please refer to weaknesses
limitations: limitations: NA

Comments: title: Dataset concern remains
Address some concerns
increase rating from 6 to 7
Comment
Follow Up for Reviwer x7i3
Follow Up for Reviwer dQ8T
Follow Up for Reviewer hAA1
Follow Up for Reviewer GTSo
comment: Thanks for the response.

I did see the Large Group Size section. But I still disagree with the goal is to perform quick experiments on well-known open benchmarks before applying the method to real-time as:

1. [1, 2, 3] and other papers (like CubeRec you pointed out) did test on other group recommendation datasets with various group sizes and statistics reported. I do not think using Mafengo and CAMRa2011 are good enough because the group sizes are really small. CAMRa2011 even has the average group size of around 2. My opinion is we need more datasets with different group sizes to verify the robustness of the group recommendation model.

2. I do not see much details on the industrial datasets, details setup, deployment, pipeline components, etc. What I only see is Appendix 6.4.2 A/B Testing results and "the application contains 130 million page views and 50 million user views per day" in some of the responses. I did not find much details in the original paper (correct me if I'm wrong). Therefore, I cannot check with the 'internal' part and I only can mark the paper based on the 'external' (i.e., Mafengo and CAMRa2011).

I still want to keep my score at this point.
Dear Reviewer x7i3,

Thank you for your prompt response.

Regarding the correspondence between detected groups and ground-truth groups, we intend to address this from two perspectives.

During the training process, we cannot align the groups with ground-truth groups for industrial data because such labels do not exist. Similarly, on open benchmarks, we are operating in an unsupervised learning setting, meaning ground-truth groups cannot be used for alignment.

During the evaluation process, for industrial data, we conduct case studies to demonstrate our method's ability to discover user groups with similar intent within livestreaming rooms. Additionally, we employ the discovered user groups in a traffic control mechanism and achieve better recommendation performance, thereby demonstrating the effectiveness of GIM in downstream tasks. For open benchmarks, we first map the predicted clustering assignments to the ground-truth labels using the Kuhn-Munkres algorithm [1] and then evaluate the discovered user groups, therefore also demonstrating the effectiveness of GIM.

We appreciate your willingness to raise the score and will include more details and experiments in the revised paper based on the rebuttals and discussions. If you have any other questions, feel free to reach out before the discussion deadline.

    [1] Zhu H, Zhou M C, Alkins R. Group role assignment via a Kuhn–Munkres algorithm-based solution[J]. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 2011, 42(3): 739-750.

Kindest regards,

Authors of paper 7269
Thank you to the authors for the clarification.

Just one minor concern: as I have mentioned, "the groups are inferred, how can you ensure the detected group corresponds to the ground-truth group?" I'm curious about how you determine the correspondence between your framework's detected groups and the ground-truth groups, as computing evaluation metrics requires such matching.

I appreciate the authors' efforts and will adjust my score accordingly. Additionally, I highly recommend further clarifying the inference details, the effectiveness of the GIM component, and including more experimental datasets in the revised manuscript.
Dear Reviewer x7i3,

Thank you for your feedback and thoughtful discussion. We truly appreciate engaging with professional experts to enhance the quality of our manuscripts.

Regarding the understanding, we completely agree with you. It is indeed reasonable to observe degraded performance when transitioning from a supervised learning scenario to an unsupervised learning setting. This aligns with our belief that most existing group recommendation methods struggle with the unsupervised group recommendation task. 

Additionally, one of the aims of our motivation experiments was to examine the impact on user recommendation tasks and group recommendation tasks when group annotations are removed. From our experimental results, we observed a significant impact on group recommendation tasks and a minimal impact on user recommendation tasks. 

Furthermore, upon reviewing the original paper, we noticed a mismatch between the figure and its caption. This has been corrected: (b) now denotes GR on CAMRa2011, and (c) denotes UR on Mafengwo.

Regarding the evaluation of the detected groups, we have addressed this in the "Effectiveness of GIM" section. In short, for industrial data, we perform case studies to validate the detected groups since labels are not available. For open benchmarks, we use the SC clustering metric to evaluate the detected groups. 

Following your suggestion, we further tested the clustering accuracy (ACC) of our method by comparing the predicted clustering assignments with the ground-truth labels. Specifically, we first map the predicted clustering assignments to the ground-truth labels using the Kuhn-Munkres algorithm and then calculate the accuracy of these assignments. On average of the open datasets, our method achieves an ACC of 0.764, further verifying the effectiveness of our proposed GIM.

If you still have any additional concerns, please let us know. We are very willing to discuss them further.

Kindest regards,

Authors of paper 7269
Dear Reviewer hAA1,

Thanks for your responses and improving the score. We are glad that our responses can solve part of your concerns now, such as Previous Related Methods, HR@10 for CAMRA2011 Dataset and Similar Observations. 

1. Regarding your question about datasets, we have already addressed this in the Large Group Size section. Our goal is to perform quick experiments on well-known open benchmarks before applying the method to real-time, large-scale data, which is significantly larger than the existing benchmarks. Besides, for the experimental results on Yelp and Weeplaces, we start to run the experiments on them. Once they are ready, we will post the experimental results. 

2. Regarding ad-hoc groups or occasional group recommendation, we agree that comparing with these methods is valid. If you carefully read the original paper, you will find that we have surveyed, discussed, and tested several ad-hoc group methods, including GroupIM [1] and CubeRec [2]. These are more recent and relevant compared to the papers you pointed out, having been published in 2020 and 2022 respectively.


[1] Sankar et al. GroupIM: A Mutual Information Maximization Framework for Neural Group Recommendation. In SIGIR, 2020.

[2] Chen et al. Thinking Inside The Box: Learning Hypercube Representations for Group Recommendation. In SIGIR, 2022.

If you still have any concerns, please let us know. We are very willing to discuss them further.

Best regards,

Authors of Submission 7269
Thank you to the authors for their reply.

I appreciate the contribution in proposing a new setting and an effective framework. 

Regarding my second question, it seems most group recommendation baselines degrade to a vanilla NCF model, but the authors didn't confirm if this understanding is correct. Under such conditions, it's reasonable for their performance to degrade significantly. Additionally, since the groups are inferred, how can you ensure the detected group corresponds to the ground-truth group? Given that SC is an unsupervised metric and ground-truth group labels are available, evaluating the detected groups first could demonstrate the effectiveness of the GIM module.
Thanks for your response. I slightly increased the score due to the efforts of addressing all the comments (from other reviewers as well), but it cannot pass my acceptance threshold due to two reasons:

1. 'Mafengwo and CAMRa2011 have very limited number of members in a group.' And I do not see using only those datasets could really convince the effect of this model. I don't see the authors address this comment as well.
2. As far as I understand, ad-hoc groups [1, 2, 3] are groups that form just for one-off events; and it's still comparable with the proposed method.

Thank you.
Dear Reviewer x7i3,

Thanks for the clarification of your concerns. Its latter, namely, ITR, can automatically discover the user groups without annotations and outperform baselines in the unsupervised scenario. You can get a better understanding of the pipeline of ITR by carefully reading the method part of the original paper. And yes, we have already answered this question in "Effectiveness of GIM" and we demonstrate the effectiveness of GIM via clustering metric SC. We would appreciate you could read it carefully. For the motivation experiment, we aim to test the performance of the baseline in the unsupervised scenario. And you suspect you should be right. The current group recommendation methods may lose the ability to recommend items to groups since they don't have the group discover modules like GIM and the unified training task like pseudo group recommendation pre-text task and the pull-and-repulsion pre-text task. Thanks for your insights here, and we will add it to the revised paper. 

Best regards,

Authors of Submission 7269
Thank you very much for your reply.

I still have some concerns regarding the inference. For the results in Table 1, it seems they are reported on Mafengwo and CAMRa2011's original splits, where group annotations are available during model testing. I'm curious whether your framework leverages these annotations (i.e., known groups) by using a pooling strategy to compute group embeddings for recommendations, or if it can detect these testing groups without annotations and outperform baselines in unsupervised scenarios. If it's the latter, I assume there are metrics like the Jaccard score to measure the accuracy between your detected groups and the ground-truth groups, demonstrating the effectiveness of the GIM module.


Regarding the motivation, the authors conduct experiments on the baseline model ConsRec without group supervision information. In this context, does this baseline degrade to a vanilla Neural Collaborative Filtering model targeted only for user recommendation?
Dear (Senior) Area Chairs,

As the discussion deadline approaches, we kindly request your assistance in reminding the reviewers to provide their post-rebuttal responses. We have made every effort to address their concerns thoroughly.

Currently, Reviewer hAA1 has not yet responded to the authors. The reviewer mentioned, "Please refer to my questions above. I may have more questions during the discussion phase," but we have not received any further feedback. We wonder if our responses have adequately addressed hAA1's concerns. Should Reviewer hAA1 have any additional questions or concerns, we are fully committed to addressing them promptly and thoroughly.

Besides, Reviewers GTSo and dQ8T have acknowledged that our responses have addressed their questions well and are willing to raise the scores (4->5 and 6->7) for the manuscript. Reviewer x7i3 points out some remaining concerns, and we actively provide detailed replies and are now awaiting the experimental results and feedback.

Thank you for your assistance!

Best regards,

Authors of Submission 7269
Dear Reviewer GTSo,

Thanks for your professional reviews and valuable suggestions. They improve the quality of our paper significantly. And we are glad that our responses can address your concerns well and you are willing to raise the score. If you have any further questions, we are very willing to discuss them with you.

Warm regards,

Authors of Submission 7269
Dear Reviewer dQ8T,

We sincerely thank you for your meticulous and thoughtful reviews of our submission, which significantly improve the quality and clarity of our work. Thank you once again for your professional assistance and valuable contribution.

Warm regards,

Authors of Submission 7269
Thanks for addressing my concerns. Considering this, I have increased the rating from 6 to 7.
Thank you for preparing the rebuttal which has mostly addressed my concerns. I decide to raise my rating.
Dear Reviewer x7i3,

Thanks for your reminder. We will respond and try to address your remaining concerns in this box. For the inference, see the details in the “Detailed Introduction of the Application” and “Technical Details During Inference.” For the strengths, see “Existing Community Detection”. For the effectiveness of the proposed pre-text tasks, refer to Table 3 in the paper. For the weaknesses of the existing methods, refer to Figure 1 in the paper. Using a pre-defined group number is not applicable in our scenario since the number of groups dynamically changes, referring to “Annotation Costs and Dynamic Change” in response to Reviewer GTSo. Could you please clarify the meaning of “w.”. If “w.” means “with”, we have already answered your question and please carefully check it in “Explanation of“base” & Utilization of ITR”. For experimental evidence of the large-scale data, please refer to the results of the application in Section 6.4.2. For the experimental results on Yelp and Weeplaces, we start to run the experiments on them. Once they are ready, we will post the experimental results. Thanks for your response again. If you have any further concerns, don’t hesitate to tell us and start the discussion.

Best regards,
Authors of Submission 7269
Thank you to the authors for your reply.
I'd like to mention some actions that may not align with the rebuttal guidelines:

* Lengthy responses across **three** 6,000-character responses
* Inclusion of outside links
* Explicit request for reviewers to "raise the score"

It's important to follow the guidelines by providing concise, high-quality responses that effectively address concerns.

--- 

Regarding your response, some issues remain:

* **Inference Mechanism**: It seems you identify groups and use pooling methods for group recommendations. If so, the innovation appears to be in the group identification process, which might be replaced by traditional clustering or community detection methods. You mentioned these cannot be directly adopted, but they might be ***adapted***, such as using a pre-defined group number. Could you elaborate on the strengths of your group identification module and provide any empirical results?
* I’m still unsure how to extend the framework to a scenario *w. group annotation scenario*
* **Extra Datasets**: Yelp and Weeplaces are large-scale datasets. Conducting experiments on these could further demonstrate consistent high performance. Textual explanations cannot replace empirical findings.

I kindly recommend addressing these questions in a single reply within 6,000 characters.
To: AC

Dear AC

Greetings! We highly appreciate your efforts and contributions to NeurIPS 2024 and this submission. The discussion period has begun, but none of the reviewers have started discussing the responses. We are very eager to communicate with reviewers during the discussion period to further enhance the quality of this research, as the NeurIPS is an open, prestigious, and high-caliber conference. Could you help us ask the reviewers to engage in the discussion and whether their concerns are solved or if there are any additional questions? We sincerely thank you for your kind help.

With best wishes,

Authors of Submission 7269
To: Reviewer hAA1

Dear Reviewer hAA1,

Hi Reviewer hAA1! We highly appreciate your valuable and insightful reviews. We hope the above response has addressed your concerns. If you have any other suggestions or questions, feel free to discuss them. We are very willing to discuss them with you in this period. If your concerns have been addressed, would you please consider raising the score? It is very important for us and this research. Thanks again for your professional comments and valuable time!

We sincerely appreciate your constructive reviews and questions. We provide detailed responses regarding Previous Related Methods, Large Group Size, Similar Observations, HR@10 for CAMRa2011 Dataset as above. We hope our responses can effectively address your concerns. If they don't, let's have further discussion now.

Besides, if you have any additional suggestions or questions (as you mentioned, you may have more questions during the discussion phase), please do not hesitate to bring them up. We are more than willing to engage in further discussion to improve the quality of this research.

If you feel that your concerns have been satisfactorily resolved, we kindly ask you to consider revising your score. Your rating is crucial for us and our research. Thank you once again for your professional comments and the time you have invested!

Best wishes,

Authors of Submission 7269
Dear Reviewer x7i3,

We highly appreciate your valuable and insightful reviews. We hope the above response has addressed your concerns. If you have any other suggestions or questions, feel free to discuss them. We are very willing to discuss them with you in this period. If your concerns have been addressed, would you please consider raising the score? It is very important for us and this research. Thanks again for your professional comments and valuable time!

Best wishes,

Authors
Dear Reviewer dQ8T,

We highly appreciate your valuable and insightful reviews. We hope the above response has addressed your concerns. If you have any other suggestions or questions, feel free to discuss them. We are very willing to discuss them with you in this period. If your concerns have been addressed, would you please consider raising the score? It is very important for us and this research. Thanks again for your professional comments and valuable time!

Best wishes,

Authors
Dear Reviewer hAA1,

We highly appreciate your valuable and insightful reviews. We hope the above response has addressed your concerns. If you have any other suggestions or questions, feel free to discuss them. We are very willing to discuss them with you in this period. If your concerns have been addressed, would you please consider raising the score? It is very important for us and this research. Thanks again for your professional comments and valuable time!

Best wishes,

Authors
Dear Reviewer GTSo,

We highly appreciate your valuable and insightful reviews. We hope the above response has addressed your concerns. If you have any other suggestions or questions, feel free to discuss them. We are very willing to discuss them with you in this period. If your concerns have been addressed, would you please consider raising the score? It is very important for us and this research. Thanks again for your professional comments and valuable time!

Best wishes,

Authors
Dear Reviewer x7i3,

Thank you very much for your precious time and valuable comments. We hope our responses have addressed your concerns. Please let us know if you have any further questions. We are happy to discuss them further. Thank you.

In addition, we have attached the revised paper, if necessary, you can check it: https://anonymous.4open.science/r/NeurIPS-7269-ITR-revised-7D83/NeurIPS-7269-ITR-revised.pdf.

Best regards,

Authors
Dear Reviewer dQ8T,

Thank you very much for your precious time and valuable comments. We hope our responses have addressed your concerns. Please let us know if you have any further questions. We are happy to discuss them further. Thank you.

In addition, we have attached the revised paper, if necessary, you can check it: https://anonymous.4open.science/r/NeurIPS-7269-ITR-revised-7D83/NeurIPS-7269-ITR-revised.pdf.

Best regards,

Authors
Dear Reviewer hAA1,

Thank you very much for your precious time and valuable comments. We hope our responses have addressed your concerns. Please let us know if you have any further questions. We are happy to discuss them further. Thank you.

In addition, we have attached the revised paper, if necessary, you can check it: https://anonymous.4open.science/r/NeurIPS-7269-ITR-revised-7D83/NeurIPS-7269-ITR-revised.pdf.

Best regards,

Authors
Dear Reviewer GTSo,

Thank you very much for your precious time and valuable comments. We hope our responses have addressed your concerns. Please let us know if you have any further questions. We are happy to discuss them further. Thank you.

In addition, we have attached the revised paper, if necessary, you can check it: https://anonymous.4open.science/r/NeurIPS-7269-ITR-revised-7D83/NeurIPS-7269-ITR-revised.pdf.

Best regards,

Authors
## **Response to Reviewer x7i3 [3/3]**

### **Effectiveness of GIM**
Thanks for your suggestion. The group identification module (GIM) aims to discover the user groups based the embeddings in the pure unsupervised setting. In the pure unsupervised setting, the GIM is a must-used module to discover the user groups. And the effectiveness can be verified indirectly by the performance improvement of downstream tasks. To directly demonstrate the effectiveness of the proposed group identification module, we follow your suggestion and verify the quality of the discovered user group. Concretely, on the open benchmarks, we adopt the unsupervised clustering metric Silhouette Coefficient (SC) to evaluate the quality of the discovered user groups. SC is an important indicator for evaluating clustering quality, which can help evaluate the rationality of clustering results. It combines the cohesion and separation of each data point to quantify the clustering effect, and its value range is from -1 to 1. Specifically, the larger the SC, the better the clustering effect of the data points. On the average of the four datasets used in our paper, the discovered user groups achieve 0.879 SC, demonstrating the promising clustering performance of our proposed GIM. On the industrial data, due to the labels are not available and the data size being large, we conducted case studies to demonstrate the effectiveness of GIM. Specifically, we select some hot livestream rooms for the case studies, e.g., the gold-selling livestream room, movie-ticket-selling livestream room, and the face-tissue-selling livestream room. Then, we check the user group distribution and find the major group. Subsequently, we use some tags of the user interests to provide the labels of the major groups. In this manner, we can determine the interest of the major groups in the studies livestream room. We discuss the results as follows. For the gold-selling livestream room, the interests of the major groups include jewelry & gold and financial management. Besides, for the movie-ticket-selling livestream, the interests of the major groups include the movie and entertainment. Differently, for the face-tissue-selling livestream room, the interests of the major groups include food and hygiene products. These case studies can demonstrate that our proposed GIM method can effectively group the similar users to one group and separate the different groups. Also, the captured information of the major group can also be used for the refined recommendation.


### **Complexity Analyses & Convergence**
Thanks for your suggestion. Following your suggestion, we conduct complexity analyses of our proposed ITR method. First of all, we define the number of the users, the number of groups, the average size of groups, as $n$, $k’$, $n/k’$, respectively. In the process of the adaptive density estimation, the time complexity and space complexity of calculating radius proposal for one group is $\mathcal{O}(1)$, $\mathcal{O}({k’}^{2})$, respectively, and for all groups is $\mathcal{O}(k’)$, $\mathcal{O}({k’}^{2})$, respectively. Then, the time complexity and space complexity of density estimation for all groups is $k’ \times n/k’$, $n \times k’$, respectively. Subsequently, at the heuristic merge-and-split strategy stage, for the explore step, it takes $\mathcal{O}(k’)$ time complexity, and $\mathcal{O}(1)$ space complexity, respectively. And for the exploit step, it takes $\mathcal{O}(k’ \times n/k’)$ time complexity, and $\mathcal{O}(nk’)$ space complexity, respectively. Besides, for the proposed pseudo recommendation pre-text task, the time complexity and space complexity is $\mathcal{O}(n \times k’)$, and $\mathcal{O}(n \times k’)$, respectively. In addition, for the proposed pull-and-repulsion pre-text task, the time complexity and space complexity is $\mathcal{O}(nk’+{k’}^{2})$, and $\mathcal{O}(nk’)$, respectively. Moreover, for the BPR loss, the time complexity and space complexity is $\mathcal{O}(n)$, and $\mathcal{O}(n)$, respectively. Overall, the time complexity and space complexity of our proposed ITR method is $\mathcal{O}(k’+ k’ \times n/k’+ k’+ k’ \times n/k’+ n \times k’+ nk’+{k’}^{2}+n) \rightarrow \mathcal{O}(nk'+{k’}^{2})$and $\mathcal{O}({k’}^{2}+ n \times k’+1+ nk’+ n \times k’+ nk’+n)\rightarrow  \mathcal{O}(nk'+{k’}^{2})$, respectively. Therefore, our proposed method will not bring large memory and time costs since the complexity of our method is linear to the number of users. 

For convergence, we checked the loss and performance of our method on all datasets and found that they converge well, i.e., the loss decreases and gradually converges to a low value, while the performance increases and gradually converges to a high value.
## **Response to Reviewer GTSo [3/3]**

### **Complexity Analyses**
Thanks for your suggestion. Following your suggestion, we conduct complexity analyses of our proposed ITR method. First of all, we define the number of the users, the number of groups, the average size of groups, as $n$, $k’$, $n/k’$, respectively. In the process of the adaptive density estimation, the time complexity and space complexity of calculating radius proposal for one group is $\mathcal{O}(1)$, $\mathcal{O}({k’}^{2})$, respectively, and for all groups is $\mathcal{O}(k’)$, $\mathcal{O}({k’}^{2})$, respectively. Then, the time complexity and space complexity of density estimation for all groups is $k’ \times n/k’$, $n \times k’$, respectively. Subsequently, at the heuristic merge-and-split strategy stage, for the explore step, it takes $\mathcal{O}(k’)$ time complexity, and $\mathcal{O}(1)$ space complexity, respectively. And for the exploit step, it takes $\mathcal{O}(k’ \times n/k’)$ time complexity, and $\mathcal{O}(nk’)$ space complexity, respectively. Besides, for the proposed pseudo recommendation pre-text task, the time complexity and space complexity is $\mathcal{O}(n \times k’)$, and $\mathcal{O}(n \times k’)$, respectively. In addition, for the proposed pull-and-repulsion pre-text task, the time complexity and space complexity is $\mathcal{O}(nk’+{k’}^{2})$, and $\mathcal{O}(nk’)$, respectively. Moreover, for the BPR loss, the time complexity and space complexity is $\mathcal{O}(n)$, and $\mathcal{O}(n)$, respectively. Overall, the time complexity and space complexity of our proposed ITR method is $\mathcal{O}(k’+ k’ \times n/k’+ k’+ k’ \times n/k’+ n \times k’+ nk’+{k’}^{2}+n) \rightarrow \mathcal{O}(nk'+{k’}^{2})$and $\mathcal{O}({k’}^{2}+ n \times k’+1+ nk’+ n \times k’+ nk’+n)\rightarrow  \mathcal{O}(nk'+{k’}^{2})$, respectively. Therefore, our proposed method will not bring large memory and time costs since the complexity of our method is linear to the number of users.
## **Response to Reviewer x7i3 [2/3]**
### **Technical Details During Inference**
Thanks for your question. For our proposed ITR, the embeddings of the groups are generated as shown in Eq. (6) and are optimized by minimizing Eq. (8) and Eq. (11). Concretely, the adaptive density estimation, and the heuristic merge-and-split method will discover the group assignments, and the group embeddings will be formed with the average of the assigned user embeddings. During the training stage, the group embeddings are also optimized with the pseudo group recommendation pre-text task and the pull-and-repulsion pre-text task. For the inference, during the testing stage, the group embeddings are learned, and user assignments are fixed. Therefore, we can calculate the recommended score between groups and the items (for the group recommendation task) and the recommended score between users and the items (for the user recommendation task). For calculating recommendation scores, we follow ConsRec and adopt the dot product or the neural mapping method. For applying ConsRec to the scenario without annotations, we just remove the interaction labels between the groups and the items, namely removing the group loss in Eq. (5) in the original paper of ConsRec.


### **Explanation of “base” & Utilization of ITR**
Thanks for your question. The baseline model is the convention MMOE-based recommendation model. And we aim to adopt our proposed method at the re-rank stage, i.e., modified the predicted score of the recommendation model by considering the user groups. Next, we assume you have carefully read the background of this application and introduce the details of adopting our method into the re-rank stage. For different livestreaming rooms, the number of user groups will be different. For example, the gold livestreaming room has less group number than the groceries livestreaming room since user interests in the gold livestreaming room are more concentrated. Therefore, we need an unsupervised group recommendation method to discover the user groups in different livestreaming rooms. Here, we adopt our ITR method to discover the user groups and then produce the group embeddings. Then upon acquiring the identified group embeddings, we can also utilize them to enhance the traffic mechanism within real-time livestreaming recommendations. In practical terms, we initially designate the primary user groups within a livestreaming room as the room's profile. Additionally, we devise and calculate the matching score between the primary user groups and potential users, subsequently leveraging these matching scores to dynamically bolster the ranking score of relevant users, therefore improving the recommendation performance and the interactive environments in the livestreaming room. And our proposed ITR will not be used with the group annotations since if the group annotations are easy to obtain, we don’t need to develop an unsupervised learning-based group recommendation method.


### **Experiments for Occasional Group Recommendation**
Thanks for your suggestions. In our paper, the setting of the experiments is purely unsupervised. However, the occasional group recommendation methods still rely on the annotation of the groups. Although they are essentially different, we are still glad to survey, discuss, and test them in the revised paper. Actually, for GroupIM [1] and CubeRec [2], we have already surveyed, discussed, and tested in our original paper. Please carefully check the related work part and the comparison experiment part in the original paper. For GBERT [3] and SGGCF [4], thanks for your suggestion and we will briefly introduce them and compare them with our method. GBERT [3] is proposed to solve the data sparsity and cold-start problems by the pre-training and fine-tuning techniques on BERT. SGGCF [4] aims to solve the data sparsity and high-order interaction problems by the heterogenous graph and the self-supervised learning. For GBERT [3], the code is not available online. Therefore, it’s hard to reproduce their results. And for SGGCF, we have started the experiments. Due to the limited rebuttal time, the experiments are not finished yet, and once they are finished, we will post the experimental results during the discussion period.


    [1] Sankar et al. GroupIM: A Mutual Information Maximization Framework for Neural Group Recommendation. In SIGIR, 2020.

    [2] Chen et al. Thinking Inside The Box: Learning Hypercube Representations for Group Recommendation. In SIGIR, 2022.

    [3] Zhang et al. GBERT: Pre-training User Representations for Ephemeral Group Recommendation. In CIKM, 2022.

    [4] Li et al. Self-Supervised Group Graph Collaborative Filtering for Group Recommendation. In WSDM, 2023.

**to be continue...**
## **Response to Reviewer dQ8T [2/2]**

### **Core Ideas & Primary Designs**
Thanks for your suggestion. Following your suggestion, we briefly introduce the core idea and the primary designs of our proposed method before introducing the method part. Concretely, we aim to develop an unsupervised group recommendation method since we find the promising performance of existing state-of-the-art methods relies on the annotations of the groups. The experimental evidences can be found in Figure 1. However, in the real-world scenario, the annotations regarding the group-item interactions and the user-group assignments are always not available. And labeling them in the real-time recommendation system is expensive and even impossible. To this end, we develop a pure unsupervised group recommendation method, which can automatically discover the user groups and provide the precise recommendation for them. Therefore, the core ideas of our methods are twofold, including the group identification and the group recommendation. For the group identification, our initial solution is to adopt some existing clustering or community detection methods, which do not require the number of clusters, e.g., DBSCAN, DeepDPM, etc. However, these methods can not automatically discover the user groups since they need other hyper-parameters such as the radius and calculation of the density. Therefore, we design an adaptive density estimation method, which can automatically estimate the density of the samples. Then, for the group discovery, we propose a heuristic merge-and-split strategy to merge the similar users into the group and split different user groups in the large group. Besides, the group embeddings are set as the learnable neural parameters and can be optimized during the learning process. Moreover, for the group recommendation part, the existing method can not deal with it in the unsupervised setting. And we propose two pre-text tasks, including the pseudo group recommendation pre-text task and the pull-and-repulsion pre-text task. The pull-and-repulsion pre-text task aims to optimize the group embeddings via separating the different groups and pushing the samples together to the corresponding groups. Besides, for the pseudo group recommendation pre-text task, it generates the pseudo labels for the group recommendation task and guide the network to conduct group recommendation even without the precise annotations. By these designs, our proposed ITR is able to automatically discover the user groups and then provide the precise group recommendation for them. Therefore, it can be applied in the real-time large-scale recommendation system. The details can be found in Section 6.4. We have added these insights at the method Section in the revised paper.


### **Cluster Number**
Thanks for your question. In this paper, the setting of the user/group recommendation is pure unsupervised and the cluster number is not pre-given in both the open dataset and the industrial data. Therefore, during the training process, the cluster number will dynamically change on the open benchmarks since the status of the user embeddings change. Our proposed GIM can automatically discover and optimize the group embedding and the group number. After training, the number of clusters and the status of the group embeddings are restored, and then they will be used to conduct the testing of the group recommendation and the user recommendation.

Besides, on the real-time large-scale data, we first train the recommendation model using the past data. During this process, the number of clusters and the status of the group embeddings are optimized dynamically. After the training stage, they are restored to the database for serving the downstream tasks. Then, when the training data update, the recommendation model will be continue trained and the number of clusters also change dynamically.

### **Imbalance Problem**
Thanks for your question. The imbalance problem does an essential factor in our scenario. We conduct some case studies in our application. Concretely, we select some hot livestream room for the case studies, e.g., the gold-selling livestream room, the grocery-selling livestream room. For the gold-selling livestream room, the imbalance problem exists since the most users in this room are interested in the gold and will form one large cluster, but the other users in this room are interested in other items and form several small clusters. Differently, for the grocery-selling livestream room, the imbalance problem is not very serious, since there are so many small user groups and the large user group may not easily form. For the downstream task, in our scenario, the imbalance groups may not be a bad phenomenon since we can use this property to find the major user group in the livestream room and provide more precise recommendation for them. We have added these claim and discussion at the application Section in the revised paper.
## **Response to Reviewer GTSo [2/3]**

### **Unique about Current Clustering Method**
Thanks for your question. The clustering methods, which need the pre-defined number of cluster centers, such as k-means, spectral clustering, cannot be applied in this scenario. For other clustering methods, which do not require the pre-defined number of cluster centers, such as DBSCAN, is hard to deal with our scenario well since 1) the data distribution of our application change dynamically, therefore it’s hard to determine the parameters regarding to the density, such the radius, and the threshold. But our proposed group identification module can automatically estimate the density and discover the groups via the heuristic merge-and-split strategy, thus can be easily applied in the real-time large-scale unsupervised recommendation system. 2) They can not be integrated into our framework without our proposed pre-text tasks, including the pseudo group recommendation pre-text task and the pull-and-repulsion pre-text task. We have added these claims in the revised paper.

### **Evaluation Protocols**
Thanks for your question. For the evaluation, we follow ConsRec and adopt four metrics to evaluate the ranking capability of the methods, including Hit Rate @ {5, 10} and Normalized Discounted Cumulative Gain @ {5, 10}.

**to be continue...**