Exploring the Value of Multi-View Learning for Session-Aware Query Representation (2022)

Abstract: Recent years have witnessed a growing interest towards learning distributed query representations that are able to capture search intent semantics. Most existing approaches learn query embeddings using relevance supervision making them suited only to document ranking tasks. Besides, they generally consider either user’s query reformulations or system’s rankings whereas previous findings show that user’s query behavior and knowledge change depending on the system’s results, intertwine and affect each other during the completion of a search task. In this paper, we explore the value of multi-view learning for generic and unsupervised session-aware query representation learning. First, single-view query embeddings are obtained in separate spaces from query reformulations and document ranking representations using transformers. Then, we investigate the use of linear (CCA) and non linear (UMAP) multi-view learning methods, to align those spaces with the aim of revealing similarity traits in the multi-view shared space. Experimental evaluation is carried out in a query classification and session-based retrieval downstream tasks using respectively the KDD and TREC session datasets. The results show that multi-view learning is an effective and controllable approach for unsupervised learning of generic query representations and can reflect search behavior patterns.

## introduction understanding user's search intent is central in information retrieval (ir). modeling user's intent inevitably requires to capture search context. search history is arguably the most salient facet of context that has been widely captured and used in previous work @xcite @xcite . it mainly includes the following: (1) the previous user's queries, generally recorded into physical sessions (also called time-based sessions @xcite ) or task-based sessions (also called missions @xcite ); (2) the retrieved documents that the user subsequently selects (e.g., based on clicks), among those retrieved by the ir system in response to her queries. mining user's search intent from search history is challenging because of phenomena such as vocabulary mismatch between the query and documents, ambiguity issues since two queries even with slight lexical variations may underline different intents @xcite , and topic change in user's search behavior which is particularly prominent while completing complex search tasks (e.g., exploratory and multi-step tasks @xcite . to address these challenges, recent years have witnessed a growing interest in learning query representations to capture hidden syntactic and semantic relationships @xcite @xcite @xcite . however, learning context-aware query embeddings faces two main issues: (1) user's query formulations included in the search sessions bring word contexts that do not extensively occur at the training phase in web search data @xcite ; (2) queries do not exhibit a clear structure as sentences. in most of previous work, query embeddings are learned based on search session contexts modeled from relevant or pseudorelevant documents returned by the system @xcite @xcite . these methods are suited to supervised relevance ranking tasks with sufficient training data. other methods learn distributed query representations based on user's query reformulations in the search session @xcite @xcite . these methods are rather unsupervised and applicable to a wide range of downstream language processing tasks making them generic. in this work, we explore the unsupervised problem of learning generic distributed query representa-tions, able to support a wide range of downstream search tasks. as outlined recently, unsupervised representation learning for ir has not received much attention yet @xcite . this paper attempts to fill this gap by following a query oriented fashion. specifically, we argue that by considering only one facet of the search session (i.e., documents vs. query reformulations) as done in @xcite ; @xcite ; zamani and croft (2016); zhang et al. (2019); zhou et al. (2020) , or by considering them both but without relating the semantics underlying between the user's search intent and the system's document results @xcite , we lose valuable mutual information about the interactive intentions (xie, 2002) that could act as a soft supervision during the search task. based on previous findings @xcite showing how user's query behavior and knowledge change from system's results during the search session, we propose a framework for session-aware query representation learning based on multi-view learning (saquevil). saquevil is a two-step architecture that consists of two single-view query encoders, namely user-view and system-view query encoders, and a multi-view query encoder. each single-view query encoder is based on a bidirectional transformer @xcite at the session level. by investigating the use of unsupervised multi-view based learning algorithms, namely cross-modal factor analysis (cfa) and uniform manifold approximation and projection (umap), the multi-view encoder takes as input the two single-view query embeddings related to the same query and provides a multi-view query representation. the underlying objective functions aim to maximize the alignment of features between both views which leans to reveal the underlying manifold. in the multi-view embedding space, similar queries formulated in the context of similar tasks have spatially close representations. our key contributions are: 1) we model generic session-aware query representation as an unsupervised multi-view learning task using a two-step framework architecture, saquevil; 2) we experimentally show the effectiveness of multi-view based representations in query classification and session-retrieval as downstream tasks; 3) we conduct quantitative and qualitative analyses showing the potential of saquevil in understanding user's search behavior. ## experimental setting we address the following research questions: rq1) how does the saquevil framework perform in query classification and session-based retrieval as downstream tasks? ## conclusion the paper presented saquevil, a framework that learns query representations that reflect users' intents within a session-based search. by relying on the key finding that system's results affects user's query behavior and knowledge, we advocate the use of unsupervised multi-view learning to capture manifolds in a shared distributed representation space. through experimental evaluation in two downstream tasks, we show the effectiveness of saquevil over supervised and unsupervised pretrained encoders, though improvements are limited in session-based retrieval that inherently requires relevance supervision. a series of experiments and qualitative analyses also show the potential of saquevil to control the representation space through key parameters that directly influence performance of downstream tasks and additionally, to