D i AL : Diversity Aware Listwise Ranking for Query Auto-Complete (2024)

Abstract: Query Auto-Complete (QAC) is an essential search feature that suggests users with a list of potential search keyword completions as they type, enabling them to complete their queries faster. While the QAC systems in eCommerce stores generally use the Learning to Rank (LTR) approach optimized based on customer feedback, it struggles to provide diverse suggestions, leading to repetitive queries and limited navigational suggestions related to product categories, attributes, and brands. This paper proposes a novel DiAL framework that explicitly optimizes for diversity alongside customer feedback signals. It achieves this by leveraging a smooth approximation of the diversity-based metric ( \alpha NDCG) as a listwise loss function and modifying it to balance relevance and diversity. The proposed approach yielded an improvement of 8.5% in mean reciprocal rank (MRR) and 22.8% in \alpha NDCG compared to the pairwise ranking approach on an eCommerce dataset, while meeting the ultra-low latency constraints of QAC systems. In an online experiment, the diversity-aware listwise QAC model resulted in a 0.48% lift in revenue. Furthermore, we replicated the proposed approach on a publicly available search log, demonstrating improvements in both diversity and relevance of the suggested queries.

## introduction query auto-complete is a valuable tool in ecommerce that helps customers articulate their query by suggesting relevant completions saving time as well as improving overall search relevance. the qac problem is usually formulated as a two-step process of matching and ranking. matching entails retrieving the list of most popular completions (mpc) @xcite based on the characters entered by the user in the search box (or prefix). this is followed by re-ranking of the retrieved keywords by using ltr to finally select figure 1 : qac inference flow for the prefix 'mobile' employing two ltr models: 'before' showing results when trained solely on relevance, and 'after' presenting results after implementing diversification in india market. learning to rank (ltr) is a widely adopted approach for modeling qac recommendations, typically implemented through pairwise ranking techniques @xcite @xcite . in these methods, given a pair of suggestions, the model learns to assign a higher score to the more frequently clicked suggestion (query) compared to the less clicked one. addressing diversity in qac suggestions is commonly handled as a post-processing step. first, suggestions relevant to the prefix (as reflected by the ml model score) are selected, and then post-processed to obtain diverse suggestions concerning the navigational topic @xcite @xcite . however, this approach is disadvantageous as the trade-off between relevance and diversity is determined by heuristics, involving approximations to resolve ties. furthermore, these greedy selection techniques involving one-by-one comparisons often fail to meet real-time diversity requirements for ranking. to address existing limitations, we propose dial, a listwise ranking method with a tailored scoring function to simultaneously optimize relevance and diversity. notably, we employ a smooth version of the diversity-based metric (αndcg) as the loss function, where rank is approximated using scores of queries in the list. we modify this loss to suit qac constraints, balancing relevance and diversity. uniquely, we propose a novel diversifying strategy for qac by mining navigational entities and further utilizing these entities with hierarchical intents in the loss. the overall score-and-sort strategy with a diversity-aware loss, deployable under real-time qac constraints, has not been studied before. therefore, we list the main contributions of our work below: ## related work the study of diversification in qac has not been extensively explored. @xcite conducted seminal research on diversifying qac through a greedy query selection approach, suggesting the next query based on query popularity, aspects of the query already in the list, and previous search sessions. subtopics or aspects are extracted from clicked document urls for a given query. @xcite proposed improving quality in qac using multi-objective ranking by boosting navigational queries using pairwise ranking. while their approach improves the ranking of navigational queries over low-quality or non-navigational queries, it does not explicitly diversify topics or subtopics within navigational queries. our work draws inspiration from the related area of diversifying web search results, exploring two paths based on whether subtopics are already known (explicit diversification) or not (implicit diversification). for explicit diversification, studies like @xcite @xcite have been conducted. for implicit diversification, researchers such as @xcite @xcite @xcite have made contributions. @xcite employed distributed embeddings to uncover latent subtopics and used an approximate diversification metric as a loss to enhance search diversity. jointly training all queries in the list is critical for training a diversity-aware loss. as such, we adapt the listwise ltr framework @xcite to score the queries and incorporate a query interaction layer similar to the document interaction network (din) @xcite to produce higher-order features for queries in the list. @xcite provided essential benchmarks and investigated various architectures and loss functions for ltr. we show how listwise ltr with din is superior to pairwise ltr with feed-forward layers @xcite for modeling click-based relevance in qac. following @xcite , who suggested using the optimization metric as a loss function for similar or better results over standard ltr loss functions, we train a diversification-aware loss that performs direct metric optimization using a smooth variation of αndcg. ## diversified auto-complete we present the diversity-aware listwise ranking for auto-complete (dial) framework that models diversity alongside relevance in qac. dial applies listwise ranking with a diversity-aware loss, detailed in this section. ## experiment and results ## conclusion in this paper, we introduce diversified listwise ltr for the qac task using a score-and-sort strategy. unlike previous greedy approaches employed in qac, the score-and-sort approach is quicker and more efficient in diversifying qac suggestions. we demonstrate the importance of capturing listwise context in qac ranking with query interaction network and show improved click based relevance and relevance w.r.t different product attributes. further, we diversify qac on different dimensions (or intents) and modify the approx αndcg loss to penalize longer queries and assign different weights to the intents based on their relative importance. this technique jointly boosted relevance and diversity with speedy inference. jective (same as equation 10) and the correlation term denotes the qq objective. to replicate this model for aol search logs, we assign a navigational score proportional to the likelihood of topic presence in the query.