Enhancing Sequence Representation for Personalized Search (2024)

Abstract: “The critical process of personalized search is to reorder candidate documents of the current querybased on the user’s historical behavior sequence. There are many types of information containedin user historical information sequence, such as queries, documents, and clicks. Most existingpersonalized search approaches concatenate these types of information to get an overall userrepresentation, but they ignore the associations among them. We believe the associations ofdifferent information mentioned above are significant to personalized search. Based on a hierar-chical transformer as base architecture, we design three auxiliary tasks to capture the associationsof different information in user behavior sequence. Under the guidance of mutual information,we adjust the training loss, enabling our PSMIM model to better enhance the information rep-resentation in personalized search. Experimental results demonstrate that our proposed methodoutperforms some personalized search methods.”

## introduction search engines have emerged as pivotal instruments for users to access information. however, search engines often yield identical results in response to identical queries, regardless of different information needs from different users, e.g., some users use the word "mac" to refer to "mac computer", in contrast, some other users use it to denote "mac lipstick". to tackle this problem, personalized search has emerged as a highly effective strategy. it models a user's preferences by user's historical behavior, and returns customized ranking results. personalized search has been proved to provide a better re-ranking document list and improve user satisfaction @xcite @xcite @xcite @xcite . prevalent models integrate semantic context from user histories with current queries and documents. deep learning models like hierarchical recurrent network with query-aware attention and graphaugmented approach have advanced the field but typically neglect the interconnectedness of historical informational units @xcite . we believe that capturing contextual relationships between historical queries/documents and their behavioral contexts is vital for better personalization. hence, we introduce an innovative methodology rooted in mutual information maximization (mim, elaborated upon in appendix a). as a form of selfsupervised learning, mim exploits the inherent correlations present in user historical sequences to blend different types of data into a cohesive representation. this learning paradigm @xcite enables models to learn from the underlying structure of raw data, creating self-derived training signals and initializing model parameters through specifically designed optimization objectives. we present the psmim (personalized search mutual information maximization) model, incorporating auxiliary tasks to enhance user representations for personalized search. by exploiting self-supervised information from user historical sequences, we design three auxiliary tasks to capture the following correlations: (1) the correlation between historical query and historical sequence information, (2) the correlation between historical document and historical sequence information, and (3) the correlation between the current query and historical sequence information. these objectives are unified under the framework of maximizing mutual information, enabling the model to uncover associations among different information types in user sequences and adapt to new data or association patterns. by effectively fusing various user information types through auxiliary tasks, we obtain a richer user historical context representation, which is then integrated with candidate document representations. we employ transformers to capture interactions among the historical behavior sequence, current query, and candidate documents to compute personalized ranking scores. our model is trained with pair-wise ranking loss. through experiments, we validate the effectiveness of our approach, demonstrating that our model, psmim, outperforms several previous personalized search models. our contributions include proposing a personalized search model psmim which systematically addresses inter-associations among user sequence information through auxiliary objectives, and empirically validating the practical effectiveness and robustness of psmim. ## problem definition first, let us define the personalized search problem. consider a user's historical information sequence @xmath0, p 2 , . . . }, with each page p i contains an issued query, a set of documents, and a clicked document with its timestamp. acknowledging that issued queries and their corresponding clicked documents are more informational, so our study focuses primarily on modeling these two critical types of historical c c l 2 0 2 4 data within the sequence. thus, we can assume a user's history as @xmath1, d 1 ), . . . , (q n , d n )}, with n representing the total count of queries in the user's history. given a current query q c and a set of candidate documents d @xmath2, d c 2 , . . . , d cm }, the personalized search task is to score each document in d c according to the current query q c and the user's historical information sequence h. ## our method our method maximizes mutual information across user historical data, including historical queries, historical documents, alongside the current query. auxiliary tasks help to enhance the model's capacity to represent diverse historical information types. leveraging word-level and sequence-level transformers, our method effectively encodes the contextual and sequential nuances within user historical data.