Paper: Learning Multi-Faceted Prototypical User Interests (ICLR-2024)

Reviewer comment (Novelty):
The proposed method is somehow incremental and is a combination of existing techniques. This work is based on an existing work that learns prototypical representation of items and users that separate items into different groups. This work further incorporates disentangled representation learning to explore item representations of various aspects of items, which has been also considered in previous works [1]. The main difference appears to be the combination of these various aspects.

Author response:
Regarding the motivation, the proposed method distinguishes itself by two key aspects: discovering multi-faceted item space and modeling the composition of user interests. These motivations have not been considered in [1]. 

In terms of technical aspects, we designed a novel model including a prototype-based multi-faceted item clustering mechanism, a binding block to model user interest compositionality and the dedicated bi-directional binding block. These components have been empirically shown to be important to achieve higher recommendation accuracy. In contrast, these components were not studied in [1]. Moreover, our model is more general and effective than MacridVAE [1]. 

As MacridVAE serves as the base of recently disentangled recommendation models, our idea improving MacridVAE, thus, can be applied to a range of problems.

*[1] Ma et al. Learning Disentangled Representations for Recommendation. NeurIPS 2019.*

Reviewer comment (Presentation):
Figure 2 is confusing because it takes explicit user-item interaction in a rating form. However, the paper is designed based on implicit user-item interaction. This misleads the reader on the output form and training strategy of the proposed method.

Author response:
In Figure 2, the input is binary rating vector, where the numbers are IDs of items. Shaded circles are those that user interacted with while dashed circles are unobserved interaction items. 

We updated Figure 2 and added clearer notations to ease the understanding.

Reviewer comment (Experiments):
Would it be better to include item attribute information in prototype learning as auxiliary information? Without low-level attributes as input, the so-called low-level user interest about a facet might also be a composite feature of some facets. This could result in the difference in meaning between the two levels of interests not being clear.

Author response:
It is an interesting direction to incorporate item attributes into the proposed approach. With item attributes such as item categories, one can formulate a classification task to match the learned item cluster with ground-truth categories as classification labels.  

In this paper, we consider user-item interaction data for collaborative filtering task, which is the most popular method in recommender systems. As future work, once item attributes are provided, one can integrate them into our proposed approach as aforementioned.

Reviewer comment (Evaluation):
More qualitative analysis and case studies about how learned latent representations correspond to different aspects of characteristics may help to better evaluate the interpretability of the model. For example, more details on how the "latent interest" corresponds to explicit interests 1, 2, and 3 in Table 4 would be appreciated.

Author response:
Each user's latent interest, which is denoted by **z**$^u_k$ (Equation 5) in the main text, is expected to capture one aspect of user preferences. It is derived in an unsupervised manner without any prior information. Thus, to understand the meaning of these latent interests, we find the top 3 items that are closest in representation space with those interests by leveraging output of decoder, i.e., top 3 items with highest decoded score. This score is simply the cosine similarity between latent interest representation and item representations stored in decoder (as shown in Equation 7). Then we visualize the item side information of those items in Table 4 to understand each latent interest.

Reviewer comment (Evaluation):
The proposed approach has worse complexity than baseline MacridVAE.

Author response:
The proposed method outperforms MacridVAE on chosen datasets. Despite having slightly higher complexity than MacridVAE in item clustering step, the complexity is a linear function of the number of items. Moreover, thanks to the multi-faceted item grouping step, FacetVAE can handle large and complex datasets, e.g., with 8 facets and 5 clusters per facet, FacetVAE can describe a dataset including up to $5^8$ (~400k) characteristics. Thus, the proposed method can be applied to large scale datasets.

Reviewer comment (Experiments):
For real-world recommendation problems with huge spaces of users and items, the approach seems less applicable.

Author response:
The proposed method outperforms MacridVAE on chosen datasets. Despite having slightly higher complexity than MacridVAE in item clustering step, the complexity is a linear function of the number of items. Moreover, thanks to the multi-faceted item grouping step, FacetVAE can handle large and complex datasets, e.g., with 8 facets and 5 clusters per facet, FacetVAE can describe a dataset including up to $5^8$ (~400k) characteristics. Thus, the proposed method can be applied to large scale datasets.

Reviewer comment (Reproducibility):
For Table 1, what is the number of clusters and the number of parameters for the baseline MacridVAE?

Author response:
In table 1, we set the number of clusters for MacridVAE is 4, which is also the number of user interests. The number of user interests (K) in FacetVAE is also 4 for fair comparison. In supplementary Figure 4, we varied the number of user interests. In these experiments, FacetVAE still achieves higher recommendation accuracy than MacridVAE.

We configure the settings of MacridVAE and FacetVAE almost the same for fair comparison. The main difference between parameters of these two is multi-faceted clustering step. 

FacetVAE uses slightly more parameters than MacridVAE, including $F * J * d^{enc}$ parameters and parameters of $w$ in Equation 3. With small values of $F$ and $J$ (3 or 4) and $d^{enc}$ is not too big (64 or 300), the difference between parameters of MacridVAE 
and FacetVAE is small. Table below shows the relative comparison w.r.t. the number of parameters between MacridVAE and FacetVAE

|             | Citeulike-a |     Yelp   | ML-1M  |
|:---------:|:------------:|:---------:|:------:|
| FacetVAE  |   100.22%   | 100.18% | 101.5% |
| MacridVAE |    100%     |  100%   |  100%  |

Reviewer comment (Evaluation):
Is it possible to report the performance with respect to the number of parameters, and computation time across the baselines and the proposed approach? That seems to be able to give readers a better understanding of accuracy and efficiency.

Author response:
In the answer to **Q1** above, we (relatively) compare the number of parameters of FacetVAE  and the closest baseline MacridVAE. It is shown that FacetVAE just uses a small number of parameters more than MacridVAE.  

In Section 5.2, we already provided the comparison between running time to train single epoch (the reported numbers are averaged over 10 runs) of FacetVAE and MacridVAE. In summary, FacetVAE requires slightly more time than MacridVAE, demonstrating its efficiency.

Reviewer comment (Experiments):
In Table 2, why is the number of prototypes fixed to 12 instead of other numbers? Can the best F and J have different trends with different number of prototypes?

Author response:
Due to limited space, in Table 2, we fix the number of prototypes as 12 for illustration. In supplementary Figure 5, we report the accuracy with various values of number of facets $F$ and number of clusters per facet $J$, which correspond to various values of number of prototypes.

Reviewer comment (Experiments):
In real world recommender systems, different product categories will have different facets, in total, it can reach up to thousands of facets in the recommendation space, especially when we want to do whole website recommendation. It's unclear how this method can handle and scale to large scale product facets.

Author response:
**A1** The proposed method can handle large scale datasets including thousands of facets thanks to the multi-faceted modeling. For example, a dataset contains *m* categories $C_1, C_2 …, C_m$; each includes $F_1, F_2, …, F_m$ facets, respectively. A naïve solution grouping items using single facet would require $F_1 * F_2 * … * F_m$ clusters, which is exponentially large and thus it might be feasible to handle huge datasets. However, the proposed method only requires $F_1 + F_2 + … + F_m$ clusters, which scales linearly. Concretely, if there are 6 categories, each including 4 facets, the total number of required clusters for a naïve solution is $4^6 = 4096$ clusters while the proposed method needs $4 * 6 = 24$ clusters to model item space structure.

Reviewer comment (Novelty):
This work can be viewed as an incremental upon MacridVAE, which limits the technical novelty contribution.

Author response:
**A2** Our work is distinct from MacridVAE in three aspects: discovering multi-faceted item space, binding compositional user representations and bi-directional binding block to model compositional user representations. These ideas were not discussed in MacridVAE. Moreover, these aspects have been shown to achieve higher recommendation accuracy. 

It is worth noting that MacridVAE serves as the base of recently developed recommendation models. Our key ideas for improving MacridVAE, therefore, are applicable to a wide range of problems.

Reviewer comment (Experiments):
The performance on large scale (millions of users and items) datasets is unclear.

Author response:
**A3** Currently, we are unable to conduct experiments on such huge datasets due to limited computational resources, which we leave these experiments for future work. In any case, the methodology we propose would scale linearly with the number of users or items.

Reviewer comment (Experiments):
How to leverage product knowledge/other knowledge source to automatically choose #facets should be discussed to make it applicable in real world applications.

Author response:
**A4** When product knowledge, e.g., item category, is available, one could use this information to supervise the item grouping in our proposed method, i.e., a classification task between the learned item group and the ground-truth item category as label.