Search Query Embeddings via User-behavior-driven Contrastive Learning (2025)

Abstract: Universal query embeddings that accurately capture the semantic meaning of search queries are crucial for supporting a range of query understanding (QU) tasks within enterprises.However, current embedding approaches often struggle to effectively represent queries due to the shortness of search queries and their tendency for surface-level variations.We propose a user-behavior-driven contrastive learning approach which directly aligns embeddings according to user intent.This approach uses intent-aligned query pairs as positive examples, derived from two types of real-world user interactions: (1) clickthrough data, in which queries leading to clicks on the same URLs are assumed to share the same intent, and (2) session data, in which queries within the same user session are considered to share intent.By incorporating these query pairs into a robust contrastive learning framework, we can construct query embedding models that align with user intent while minimizing reliance on surface-level lexical similarities.Evaluations on real-world QU tasks demonstrated that these models substantially outperformed state-of-the-art text embedding models such as mE5 and SimCSE.Our models have been deployed in our search engine to support QU technologies.

## introduction query understanding (qu) tasks, such as query classification and suggestion, play a crucial role in improving user search experiences by interpreting users' search intents and supporting search behavior @xcite . embedding-based approaches have gained prominence in addressing these tasks due to their robustness to lexical variations @xcite . building tailored embeddings for every qu task is costly, making universal query embeddings essential. such universal embeddings enable accurate representation of search intent and provide a versatile solution applicable across qu tasks, which is highly valuable for enterprises. despite their importance, developing query embeddings that well reflect users' intent presents unique challenges. since search queries are typically short, they lack rich contextual information, making it difficult to precisely capture users' search intent @xcite . this shortness also means that minor wording changes in queries, e.g., replacing even a couple of words with their synonyms, can noticeably alter their appearances. for example, "buy car" and "purchase an automobile" express the same intent but differ substantially in wording. these challenges highlight the need to consider suitable learning embedding approaches for search queries. a widely recognized approach for learning robust text embeddings is contrastive learning, which has demonstrated notable success in this field. state-of-the-art (sota) contrastive learning approaches typically use large-scale weak supervision from web sources, such as question-answer pairs from qa forums or title-passage pairs from encyclopedic articles @xcite . however, as these datasets primarily consist of longer, contextually detailed sentences, models trained on them struggle to handle short, context-poor queries. an alternative approach is to use unsupervised contrastive learning models, such as unsup. sim-cse @xcite , directly on a large corpus of search queries. unsup. simcse generates pseudopositive examples by encoding the same sentence twice with different dropout noise. a model trained on such positive examples tends to overemphasize lexical overlap as a cue for semantic equivalence; we observed that unsup. simcse struggles to capture semantic similarities between queries with different appearances but the same intent, such as "buy car" and "purchase an automobile" ( §5.1). overall, current contrastive learning approaches are suboptimal when creating effective positive examples for search query embeddings: typical weakly supervised approaches struggle to generalize to context-poor queries, while the representative unsupervised approach results in models that are overly sensitive to surface-level variations. to address these problems, we propose user behavior-driven contrastive learning with intent alignment for search query embeddings (ubique). ubique directly aligns embeddings according to user intent, using intent-aligned query pairs derived from real-world user interactions as positive examples. as shown in figure 1 , we explore two types of user interactions. (1) clickthrough data are records of users' clicking on web pages after submitting search queries. queries are considered to have the same intent if they lead to clicks on the same url, as users tend to click on results that satisfy similar information needs. (2) session data are sequences of queries a single user takes on a search engine within a given time frame. queries within the same user session are assumed to share the same search intent. by using a robust contrastive learning framework @xcite on these intent-aligned query pairs, ubique constructs models that precisely capture the inherent intent of context-poor queries. this approach also minimizes reliance on appearances, as these intentaligned query pairs are constructed independently of surface-level similarities. for our experiments, we built four practical qu datasets using real-world search queries to evaluate ubique from multiple perspectives ( §4). the results indicate that our click-based model (ubique click ) and session-based model (ubique session ) substantially outperformed baselines such as me5 large and unsup. simcse. specifically, compared to me5 large , ubique click achieved an average improvement of 8.7 points in task-performance metrics across all tasks, while ubique session showed strengths in a querysuggestion task, achieving an improvement of 5.3 points in ndcg@10 score. our analysis also confirmed their robustness to lexical variations, effectively capturing semantic similarities where unsupervised models fail ( §5.1). these findings highlight the effectiveness of leveraging user behavior data in learning universal query embeddings. more recently, pre-trained language models have advanced qu. @xcite mitigated context absence in queries via extended token classification, while @xcite proposed a pre-training framework using a query-url bipartite graph. we fine-tuned pre-trained language models using user interactions to construct fixed-size text embeddings for general qu tasks. our approach can be combined with these pre-training techniques. closely related is the study by @xcite , who proposed a bi-gru-based gen encoder to compute intent similarity using clickthrough data and task-specific human annotations. unlike their method, ubique constructs generalpurpose search query embedding models that rely solely on automatically collected user interactions. contrastive learning contrastive learning has proven effective for learning text embeddings by pulling similar pairs closer and pushing dissimilar pairs apart @xcite . prior research typically focused on constructing positive examples. early studies relied on annotated datasets, such as the nli dataset @xcite , while more recent studies used largescale weak supervision from web resources, achieving sota results @xcite . while these datasets consist of longer texts, we focused on handling short-text queries using weak supervision from user interactions. to reduce reliance on annotated data, unsupervised approaches have also been explored. a prominent example is unsup. simcse, which uses dropout as minimal noise to generate positive pairs @xcite . while @xcite addressed the length biases inherent in unsup. simcse, we examined its ineffectiveness with search queries, particularly its sensitivity to surface-level variations. ## ubique this section introduces ubique for constructing universal query-embedding models. ## experiment we evaluated ubique on four real-world qu tasks using japanese search query logs. ## analysis to understand the effectiveness of ubique models, we conducted comparative analyses with representative baseline models. ## conclusion and future work we proposed ubique, a simple yet effective approach to address the challenges of learning universal search query embeddings by harnessing user behavior data through contrastive learning. ubique constructs positive query pairs from clickthrough and session data, enabling the model to align embeddings based on user intent rather than surfacelevel similarities. the empirical results on four practical qu tasks demonstrated that ubique models outperformed strong baselines, particularly in their robustness to lexical variations in search queries. while our study focused on a japanese search system, we recognize that search styles can vary across languages @xcite . since ubique is theoretically applicable to other languages, evaluating its effectiveness in diverse linguistic contexts is an exciting future direction. although we constructed our models separately using clickthrough and session data, combining these data sources may lead to further performance improvements. incorporating additional information from search results, such as titles and documents, could further enhance ubique, provided the potential increase in inference latency is acceptable.