Breaking the Hourglass Phenomenon of Residual Quantization: Enhancing the Upper Bound of Generative Retrieval (2024)

Abstract: Generative retrieval (GR) has emerged as a transformative paradigm in search and recommender systems, leveraging numeric-based identifier representations to enhance efficiency and generalization. Notably, methods like TIGER, which employ Residual Quantization-based Semantic Identifiers (RQ-SID), have shown significant promise in e-commerce scenarios by effectively managing item IDs. However, a critical issue termed the " Hourglass " phenomenon, occurs in RQ-SID, where intermediate codebook tokens become overly concentrated, hindering the full utilization of generative retrieval methods. This paper analyses and addresses this problem by identifying data sparsity and long-tailed distribution as the primary causes. Through comprehensive experiments and detailed ablation studies, we analyze the impact of these factors on codebook utilization and data distribution. Our findings reveal that the “Hourglass” phenomenon substantially impacts the performance of RQ-SID in generative retrieval. We propose effective solutions to mitigate this issue, thereby significantly enhancing the effectiveness of generative retrieval in real-world E-commerce applications.

## introduction in recent years, gr has surfaced as a groundbreaking retrieval paradigm, marking significant advancements in search and recommendation environments including recommender systems @xcite @xcite , search question answering @xcite , and e-commerce retrieval @xcite @xcite . in this paradigm, target items are initially represented as identifiers (e.g., numbers, subwords, n-grams, token ids, urls, semantic codes). subsequently, leveraging input information such as queries and user details, large models are employed to output the final items in an end-to-end manner. this approach not only enhances retrieval efficiency but also improves the model's generalization capability. in generative retrieval, numeric-based identifier representation methods are widely adopted in the industry due to their simplicity, efficiency, and strong generalization, especially in long behavior sequence recommendations. these methods significantly reduce sequence lengths and accelerate the inference process. notable methods include dsi @xcite , nci @xcite , tiger @xcite , gdr @xcite , and genret @xcite . among these, the tiger method generates semantic identifiers (sid) through residual quantization (rq) @xcite , effectively capturing both semantic information and hierarchical structures. this approach is particularly advantageous in item-dominated e-commerce scenarios, where it accurately reflects the complex hierarchical relationships and semantic features inherent in e-commerce data, thereby significantly enhancing recommendation performance. it is important to highlight that the performance upper bound of rq-based methods critically depends on the generation of sid. however, we have identified a significant "hourglass" phenomenon in sid produced via rq, as illustrated in figure 1 . specifically, the codebook tokens in the intermediate layers are excessively concentrated, leading to a one-to-many and many-to-one mapping structure. this concentration results in path sparsity, where the matching paths for the item constitute a minimal fraction of the total path space and a long-tail distribution of intermediate layer tokens with a majority of sid concentrated in a few head tokens. this hourglass effect is particularly exacerbated in datasets with long-tail characteristics, which substantially constrains the representational capacity of gr methods. the underlying cause of this issue stems from the intrinsic nature of progressively quantizing high-dimensional vector residuals. furthermore, we analyzed the process of generating sid from residuals, demonstrating that sparsity and long-tail distributions are inevitable. to assess the general impact of sid on downstream gr tasks, we trained models of different scales (such as 0.8b, 7b) and types (qwen1.5 @xcite , baichuan2 @xcite , llama2 @xcite ) based on rq-sid. through a series of experiments, including altering the distribution of semantic ids by interacting with the first and second layers and swapping tokens between the first and second layers, we not only confirmed the existence of the hourglass effect but also detailed its specific impact on model performance. this analysis provides a robust foundation for future model optimization. to alleviate the hourglass effect, we propose two straightforward yet effective methods: the heuristic approach and the adaptive variable-length token strategy. the heuristic method involves directly removing the second layer, while curtailing the long-tail impact, it may lead to insufficient spatial capacity. the second method implements an adaptive token distribution adjustment to remove the top tokens from the second layer, thereby transforming the semantic id into a variable-length structure. this strategy ensures that the overall distribution remains consistent while effectively mitigating the hourglass effect by selectively token removal. extensive experimental results reveal that although both methods are straightforward, they successfully alleviate the impact of the hourglass effect to varying extents. notably, the adaptive variablelength token strategy method emerges as the most effective. the contributions of this paper can be summarized as follows: ## methods and experiments to alleviate the hourglass effect, we propose two simple yet effective methods.