Title: FACE: A General Framework for Mapping Collaborative Filtering Embeddings into LLM Tokens
Venue: NeurIPS 2025
Accepted: yes
Keywords: Recommendation System

Large Language Models

Interpretability
Abstract: Recently, large language models (LLMs) have been explored for integration with collaborative filtering (CF)-based recommendation systems, which are crucial for personalizing user experiences. However, a key challenge is that LLMs struggle to interpret the latent, non-semantic embeddings produced by CF approaches, limiting recommendation effectiveness and further applications. To address this, we propose FACE, a general interpretable framework that maps CF embeddings into pre-trained LLM tokens. Specifically, we introduce a disentangled projection module to decompose CF embeddings into concept-specific vectors, followed by a quantized autoencoder to convert continuous embeddings into LLM tokens (descriptors). Then, we design a contrastive alignment objective to ensure that the tokens align with corresponding textual signals. Hence, the model-agnostic FACE framework achieves semantic alignment without fine-tuning LLMs and enhances recommendation performance by leveraging their pre-trained capabilities. Empirical results on three real-world recommendation datasets demonstrate performance improvements in benchmark models, with interpretability studies confirming the interpretability of the descriptors. Code is available in \url{https://github.com/YixinRoll/FACE}.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to bridge the semantic gap between the collaborative filtering models and the large language models. The authors proposed to map the CF ID embeddings into the semantic tokens of LLMs, with the following novelties: a disentangled projection module, a vector quantized variational autoencoder, and a contrastive alignment loss.\n\nSTRENGTHS AND WEAKNESSES:\n- Motivation. The authors aims to bridge the gap between the collaborative filtering and large language models. This is an interesting and critical research problem.\n\n- Presentation. The whole paper is written well and is easy to follow.\n\n- Reproduction. The authors provide the released code on the proposed FACE model for reproduction.\n\nQUESTIONS:\n1. The authors aim to achieve disentanglement when mapping the CF representation to the vocabulary space by employing \"orthogonally initialized\" projection heads. However, I'm not sure whether this (orthogonally initialized heads) is enough. These initially orthogonal heads may not become as orthogonal after the model optimization. I agree that disentangling these heads is beneficial, and I wonder whether it's better to achieve this goal by a more direct approach: employing a disentanglement loss (such as the de-correlation loss [1,2]) upon the outputs of these heads. \n\n2. In the \"Residual quantization\" paragraph, the authors propose to \"shares the same codebook across all quantization levels, with this codebook being composed of pre-trained LLM token embeddings\". However, I'm not sure whether this is the best choice. The codebooks and \"their embeddings\" in RQ-VAE should contain a hierarchical structure of the data, while the LLM token embeddings don't. More experiments or analysis on justifying the claim would be great.\n\n[1] Barlow twins: Self-supervised learning via redundancy reduction. PMLR, 2021.\n\n[2] Enhancing CTR Prediction with De-correlated Expert Networks. 2025.\n\nLIMITATIONS:\n1. Performance Evaluation. There are many existing works to inject LLM into recommendations(here I list several works for sequential recommendation:  [1,2,3], there should be many more for the general collaborative filtering/recommendation domain). In the performance evaluation, the authors should prove the effectiveness of the proposed approach by comparing it with these \"LLM4Rec\" methods, instead of just showing that FACE can enhance the performance of SOTA collaborative filtering models, because it's already well known that involving LLMs can boost the performance of CF. The authors' finding that \"the gains achieved by the PLMRec model integrated with the FACE framework are minimal\" precisely reflects my above-mentioned concern: existing work can already combine the LLM well, if so, then what's the advantage of FACE itself?\n\n2. In-depth Analysis. Besides the performance evaluation, the author should provide more in-depth analysis on the advantages of the proposed approach over existing works. For example, does FACE align the two spaces better than existing works[1]?\n\n3. Justification of the modules. The authors proposed several novel modules in Sec. 3.2 and 3.3. It would be great to justify them by: 1. What's the challenge the authors would like to tackle? 2. To what extent do existing approaches or naive methods resolve it? 3. Can the proposed module do better? How to measure or quantify it?\n\n[1] CTRL: ConnectCollab- orative and Language Model for CTR Prediction. 2023.\n\n[2] Enhancing Taobao Display Advertising with Multimodal Representations: Challenges. 2024.\n\n[3] DisCo: Towards Harmonious Disentanglement and Collaboration between Tabular and Semantic Space for Recommendation. KDD 2024.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 1, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes FACE, a general framework for mapping collaborative filtering (CF) embeddings into interpretable LLM tokens to bridge the gap between non-semantic CF representations and LLM understanding. The motivation stems from the challenge that LLMs cannot directly interpret latent CF embeddings, limiting their effectiveness in recommendation systems. The methodology involves two main components: (i) a disentangled projection module that decomposes CF embeddings into concept-specific vectors, and (ii) a quantized autoencoder that converts continuous embeddings into discrete LLM tokens (descriptors) using residual quantization with a frozen LLM vocabulary as codebook. A contrastive alignment objective ensures semantic consistency between mapped tokens and textual signals. Experiments on three datasets demonstrate consistent performance improvements across various CF models, with interpretability studies confirming that the generated descriptors can be understood by LLMs.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- The paper targets a useful and interesting research task in LLM-enhanced recommendation. How can we translate hidden embeddings given by collaborative filtering models into understandable LLM tokens? The paper decomposes this task into multiple steps: i) disentangling CF embeddings into multiple embeddings related to specific semantic aspects, and ii) quantizing the embeddings into LLM tokens. The framework design is elegant and intuitive, addressing a fundamental challenge in bridging CF and LLM modalities. This design not only greatly improves the explainability of recommendation models, but also provides a better alignment with textual features to improve recommendation accuracy.\n\n- The novelty of the proposed method is notable, particularly in the quantization approach that uses frozen LLM vocabulary as codebook combined with residual quantization for hierarchical semantic representation. The disentanglement method through orthogonally initialized multi-projectors followed by transformer encoding is also innovative. The integration of these components with contrastive alignment learning represents a significant technical contribution that enables end-to-end learning of semantic mappings.\n\n- The proposed method provides consistent performance improvements for recommendation models of different categories, demonstrating its model-agnostic nature and general applicability across various CF architectures from matrix factorization to graph neural networks.\n\n- The real user study effectively demonstrates the interpretability of the proposed method, with both item retrieval and generation tasks showing that LLMs can understand and utilize the generated descriptors, and human evaluation confirming the quality of interaction explanations.\n\n- The paper is well-written with clear motivation, comprehensive methodology description, and thorough experimental evaluation. The technical details are presented clearly, and the experimental setup is comprehensive with proper ablation studies and hyperparameter analysis.\n\n**Weaknesses:**\n- The proposed method is mainly focused on improving the explainability of recommendation models, but the experiments are mainly designed to evaluate recommendation accuracy. There is a gap in the former part of the paper to encounter the connection between these two objectives.\n\n- The proposed method only involves one LLM-enhanced recommendation model (i.e. RLMRec) in the experiments, limiting the comparison with other LLM-based recommendation approaches and making it difficult to assess the relative advantages of FACE over alternative LLM integration strategies.\n\n- The proposed method seems reliant on the training dataset (textual features and the results of CF training on this dataset). It is not clear how the trained mapping functions can be applied to other datasets or domains, raising questions about the transferability and generalizability of the learned mappings.\n\nQUESTIONS:\n1. Could the authors provide more theoretical or empirical analysis on how the semantic alignment objective (contrastive learning) directly contributes to recommendation accuracy improvements, beyond just enabling interpretability? The connection between these dual objectives needs clearer justification.\n\n2. How does FACE compare with other recent LLM-enhanced recommendation methods beyond RLMRec? Additionally, could the authors discuss the computational overhead and scalability of the three-stage training process compared to simpler alternatives?\n\n3. Can the trained FACE framework be transferred across different domains or datasets without retraining? What are the limitations regarding domain adaptation, and how sensitive is the method to the quality and availability of textual features in new datasets?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-30"}

comments: SUMMARY:
This paper presents **FACE**, a novel framework designed to bridge collaborative filtering (CF) models with large language models (LLMs) by mapping CF embeddings into LLM tokens. It introduces a disentangled projection module and a quantized autoencoder to convert CF embeddings into discrete LLM tokens. Additionally, a contrastive alignment objective is proposed to ensure semantic consistency throughout the process.

STRENGTHS AND WEAKNESSES:
**Strengths**

1. The methodology is well-structured and technically sound. The combination of disentangled projection, residual quantization, and contrastive learning reflects a thoughtful approach to addressing the complexities of mapping between different embedding spaces.

2. The authors provide anonymized codes, which enhances the reproducibility of the work.

3. Comprehensive experiments are conducted to effectively demonstrate the proposed method's efficacy.

**Weaknesses**

1. In Section 3.3.1, the description of the Descriptor Embedding is somewhat difficult to follow. It would be beneficial to provide a clearer explanation of "descriptor embeddings," including the distinction between descriptor and summary embeddings. Additionally, clarification on the pseudo-inverse matrix and how it is obtained would be helpful.

2. A more detailed time complexity analysis of the proposed framework would be advantageous.

QUESTIONS:
See Weaknesses.

LIMITATIONS:
I believe this work does not pose potential negative societal impacts.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes FACE, a model-agnostic recommendation framework that combines a RQ-VAE and contrastive learning to map collaborative filtering embeddings into a set of pretrained LLM tokens. It aims to leverage the power of LLMs to improve both performance and interpretability.\n\nSTRENGTHS AND WEAKNESSES:\nThe core idea of enabling direct interpretation of collaborative filtering representations by large language models addresses a fundamental challenge in the field. This approach establishes a bridge between traditional CF methods and modern LLMs, which could potentially unlock new possibilities for advanced recommendation tasks such as controllable generation, cross-domain transfer, and multi-modal recommendation scenarios.\n\nThe paper has the following strengths:\n\nS1: The proposed strategy of combining a quantized autoencoder for CF-LLM mapping while maintaining information integrity and contrastive learning for semantic alignment demonstrates solid technical grounding. The design choices appear well-motivated and the three-stage training strategy shows practical consideration for optimization stability.\n\nS2: The framework's model-agnostic nature represents a significant practical advantage. Rather than requiring specific architectural modifications, FACE can be applied to various existing CF models (GMF, LightGCN, SimGCL, etc.), which enhances its potential for widespread adoption and real-world deployment.\n\nS3: The direct mapping of embeddings into existing semantic tokens as descriptors offers an elegant solution to the interpretability challenge. This approach not only provides human-readable explanations but also enables LLMs to perform complex reasoning tasks based on these semantic representations, opening doors for sophisticated recommendation applications beyond simple ranking.\n\nS4: The experimental validation is comprehensive, demonstrating consistent improvements across multiple datasets and base models. The interpretability studies, particularly the real user evaluation comparing FACE descriptors with RLMRec profiles, provide convincing evidence that the mapped tokens capture meaningful semantic information while being more concise (16 tokens vs. paragraph-length profiles).\n\nThe paper may have the following weaknesses:\n\nW1: While the overall methodology is well-structured, the distinction between the latent variables $z_e$, $z_d$, and $z_q$ could be clarified further. For instance, by emphasizing the role of $z_d$ or providing a brief background on RQ-VAE to aid reader understanding.\n\nw2: Some text in Figure 1 is a bit small, which could be adjusted to improve readability for readers.\n\nOverall, I think this paper presents very interesting and beneficial work for recommendation systems. Based on the above analysis, I recommend acceptance of this paper.\n\nQUESTIONS:\n+ Just an extended question: can the proposed autoencoder-based bidirectional mapping strategy be extended to facilitate alignment across other modalities?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a tokenization framework FACE, that bridges the gap between embedding-based representation derived from collaborating filtering methods and the discrete representations used in LLM embeddings. The method maps the CF embeddings into discrete LLM tokens referred to as descriptors. To capture multiple semantically meaningful concepts, the method leverages disentangled projections of the CF embedding and then learns a RQ-VAE tokenizer with frozen LLM embeddings as the codebook. In addition, a contrastive alignment objective is introduced for learning semantically meaningful user/item representations, allowing LLMs to understand items/users without additional LLM fine-tuning. Experiments across three different recommendation datasets depict the efficacy of the method proposed.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The proposal for doing end-to-end tokenization with LLM finetuning for recommendation tasks seems novel. \n- The paper is overall clear and easy to understand. \n\n**Weaknesses**\n\n*Lack of discussions around scalability*\n- The key desirable property for models in recommendation system is the ability to scale and to be able to operate in cold-start settings. However, the paper doesn't discuss this. How will the chosen LLM codebook scale to millions/billions of items for a real-world recommendation system corpus? Can the alignment of CF embeddings and LLM  embeddings help with generalization in cold-start settings?\n\n*Missing crucial baselines and citations*\n- Generative Retrieval for RecSys (Rajput et al. 2023): This paper first introduced learning Semantic ID for recommendation items using RQ-VAE for the generative retrieval task. This paper should be among the main baselines for the proposed item tokenization approach.\n- Traditional recommender systems (S3-Rec (Zhou et al. (2020)), SASRec (Kang et al. 2018)) are not considered as baselines.\n- SPAE tokenization SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs. This work also uses frozen LLM embeddings to learn hierarchical descriptors when learning multi-modal tokens. Please consider adding it to related work.\n\n*Few implementation details are unclear*\n- In Section 3.3.1, it is stated that we have summaries for both users and items. Are we doing contrastive loss for both items and users separately? Please clarify in the text? \n- For joint training, what is recommendation objective? Is it retrieval/ranking task? Do we optimize all the parameters of the LLM for this task?\n- Why is Residual Quantization used if we only consider the first code as the descriptor of the encoded disentangled CF embedding?\n\n*Unclear where the improvement is coming from*\n- It is not clear from the paper what is causing the improvement shown in Table 1. Is it because of the proposed tokenization framework or  because the LLMs already have enough knowledge about these chosen datasets? Perhaps LLMs already contain the collaborative knowledge of these open-sourced recommendation benchmarks?\n\nQUESTIONS:\n- In Section 3.3.1, it is stated that we have summaries for both users and items. Are we doing contrastive loss for both items and users separately? Please clarify in the text? \n- For joint training, what is recommendation objective? Is it retrieval/ranking task? Do we optimize all the parameters of the LLM for this task?\n- Why is Residual Quantization used if we only consider the first code as the descriptor of the encoded disentangled CF embedding?\n- How does the method generalize in the cold-start setting for users and items?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-04"}