C onv T rans: Transforming Web Search Sessions for Conversational Dense Retrieval (2022)

Abstract: Conversational search provides users with a natural and convenient new search experience. Recently, conversational dense retrieval has shown to be a promising technique for realizing conversational search. However, as conversational search systems have not been widely deployed, it is hard to get large-scale real conversational search sessions and relevance labels to support the training of conversational dense retrieval. To tackle this data scarcity problem, previous methods focus on developing better few-shot learning approaches or generating pseudo relevance labels, but the data they use for training still heavily rely on manual generation.In this paper, we present ConvTrans, a data augmentation method that can automatically transform easily-accessible web search sessions into conversational search sessions to fundamentally alleviate the data scarcity problem for conversational dense retrieval. ConvTrans eliminates the gaps between these two types of sessions in terms of session quality and query form to achieve effective session transformation. Extensive evaluations on two widely used conversational search benchmarks, i.e., CAsT-19 and CAsT-20, demonstrate that the same model trained on the data generated by ConvTrans can achieve comparable retrieval performance as it trained on high-quality but expensive artificial conversational search data.

## introduction conversational search systems support multi-turn interaction with users through conversations to meet various information needs. such a new search paradigm is more natural, convenient, and userfriendly, which is expected to hugely improve users' search experience in the future @xcite . different from traditional ad-hoc search systems, conversational search systems have to understand the user's real search intent of the current ## methodology formally, for a raw web search session s @xmath0, p k )} m @xmath1, where q k is the k-th ad-hoc query and p k is the user's clicked passage for q k , our con-vtrans is to generate new pseudo conversational search sessions s conv based on s web , and the relevance labels of s conv are directly inherited from s web . we show an overview of convtrans in fig. 2 and two concrete examples in appendix c. on the whole, convtrans consists of three key steps, including session graph construction, query transformation, and conversational session generation. we elaborate on them in the following. ## main results ## impact of training data size to study the impact of the amount of training data, we test the performance of the baseline methods and convtrans with 20% to 100% of training sessions on cast-20. results are shown in fig. 3 . we find that with the increase in the amount of training data, the retrieval performance shows an upward trend, but this trend gradually becomes slower. this indicates that although increasing the amount of training data can basically improve the cast-19 cast-20 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.453 0.312 0.260 0.161 0.316 0.185 0.339 0.224 0.274 0.155 dataset ndcg@3 convtrans direct direct + sg direct + qt ance figure 4: results of ablation studies of convtrans. besides, to remove the influence of the training data size for comparisons, we randomly select the same number of training sessions (i.e., 5,644) for all these methods. as shown in fig. 3 , our convtrans still significantly outperforms the two autorewriter methods and only performs slightly worse than cqe (0.283 v.s 0.289 w.r.t. ndcg@3) whose sessions are human-written, further demonstrating the effectiveness of the training sessions generated by convtrans. ## ablation studies in this section, we investigate the effects of the session graph construction and the query transformation of convtrans. specifically, we build three ablations: (1) direct, which directly uses the raw web search sessions as the training data. (2) direct + sg, which builds session graph to refine the session quality but does not perform query transformation. ## investigation of using different session generation strategies as introduced in section 4.3, we will sample both topic-shared and response-induced conversational queries for each central query (node). in this section, we investigate the influence of topic-shared and response-induced query relations on the effectiveness of the generated conversational search sessions. specifically, we build two variants: (1) convtrans-ts, which only samples topic-shared queries. in particular, convtrans-ts achieves similar performance to convtrans on cast-19, but it still performs worse than convtrans on cast-20. this is because the cast-20 dataset has response-induced queries while the cast-19 dataset does not. 10 conclusion in this work, we propose convtrans, a data augmentation method which can transform large-scale web search sessions into effective conversational search sessions to support the training of conversational dense retrieval. to mitigate the gaps in terms of the session quality and the query form between the two types of search sessions, we design a graph-based re-organization method and a two-step conversational query rewriter. the pseudo conversational search sessions are generated by performing a tailored random walk on the constructed session graph. we run extensive experiments on two cast datasets. results demonstrate that con-vtrans outperforms state-of-the-art data augmentation baselines for conversational search and can support the model to achieve comparable retrieval performance as using the expensive artificial conversational search data. limitations and future work. our work demonstrates the feasibility of transforming web search sessions for helping the training of conversational dense retrieval. however, we only consider three coarse-grained query relation types (i.e., responseinduced, topic-shared, and topic-changed) in this work. in practice, the real conversational search sessions can have more diverse query shifts, such as returning to previous main search topics and referring to more earlier response passages. studying better ways of session graph construction to handle more complex conversational search behaviors would be a good improvement to this work. another limitation of this work is that, since we added some queries and adjusted the order of some queries for the raw web search sessions, their relevance labels may also change to some extent because of changed search session context. but the relevance labels of the generated conversational search sessions are still directly inherited from the raw web search sessions in this work. therefore, it would be better to develop a method to further check and improve the relevance labels. finally, as pointed out in section 7, convtrans still has a large improvement space for generating more powerful sessions that can be continuously absorbed by the model to finally achieve better retrieval performance.