Q uery E xplorer: An Interactive Query Generation Assistant for Search and Exploration (2024)

Abstract: Formulating effective search queries remains a challenging task, particularly when users lack expertise in a specific domain or are not proficient in the language of the content. Providing example documents of interest might be easier for a user. However, such query-by-example scenarios are prone to concept drift, and the retrieval effectiveness is highly sensitive to the query generation method, without a clear way to incorporate user feedback. To enable exploration and to support Human-In-The-Loop experiments we propose QueryExplorer– an interactive query generation, reformulation, and retrieval interface with support for Hug-gingFace generation models and PyTerrier’sretrieval pipelines and datasets, and extensivelogging of human feedback. To allow users to create and modify effective queries, our demo supports complementary approaches of using LLMs interactively, assisting the user with edits and feedback at multiple stages of the query formulation process. With support for recording fine-grained interactions and user annotations, QueryExplorer can serve as a valuable experimental and research platform for annotation, qualitative evaluation, and conducting Human-in-the-Loop (HITL) experiments for complex search tasks where users struggle to formulate queries.

## introduction being able to retrieve documents in multiple languages is becoming critical as the internet increasingly provides access to information across a wide range of languages and domains. however, creating effective search queries for cross-language and multi-language retrieval can be a daunting task for users. first, users may be unfamiliar with the language of the documents with the information they need, or may even be unaware of this information, making it hard to craft effective queries. while a good qbe interface could be valuable, a tool that can facilitate generating queries with a human-in-the-loop (hitl) setting can result in an even more effective search. extensive prior work has shown that automatically generated queries can be improved with the searcher's inputs @xcite , via providing in-domain and worldknowledge @xcite , as well as from relevance or pseudo-relevance feedback @xcite @xcite through search results and this process can iterate several times till the searcher is satisfied with the presented search results. researchers studying searcher behavior and gathering associated annotated data may need iterating over different query generators, multiple prompting and training strategies, several retrieval pipelines, recording various ways to incorporate implicit feedback, and a rigorous number of hyperparameters. for instance, a search engine analyst might be interested in how searchers edit queries. to investigate these much-needed capabilities, we propose a configurable, interactive search interface tool, queryexplorer, which supports automatic and interactive query generation and reformulation for mono-lingual and multi-lingual interactive search. queryexplorer can both facilitate and record the query generation process and interactions of a searcher -from query formulation in a pure qbe setting to tracking the resulting query's impact on the retrieved search results and the user's exploration. to our knowledge, queryexplorer is the first such interactive query exploration interface to be shared with the research community. specifically, queryexplorer demonstrates the following novel capabilities: in summary, we believe queryexplorer could provide valuable tools for i) performing qualitative analysis over information retrieval experiments and datasets, ii) investigating interactive retrieval feedback and performing human-in-the-loop (hitl) studies, and iii) gathering user annotations and understanding searcher behavior. our system was built initially @xcite for the bet-ter search tasks 3 (mckinnon and rubino, 2022; soboroff, 2023) and was later generalized and expanded to support end-to-end query generation and reformulation experiments. we share the python code, a google colab notebook as well as the video demonstration here 4 . next, we provide an overview of related retrieval tools and the importance of query generation in section 2 to place our contribution in context. we then review the different components and capabilities of queryexplorer in section 3. ## related work there have been many ranking and retrieval tools with annotation support released previously @xcite @xcite @xcite , but all of them have focused on the ad-hoc search setting by assuming a readily available query without the need for generation, reformulation or feedback. spacerini @xcite leveraged the pyserini @xcite toolkit and the hugging face library to facilitate the creation and hosting of search systems for adhoc search. simplyretrieve @xcite , fas-trag @xcite and ralle @xcite focus on retrieval augmented generation. recent advancements in transformer models and their availability via open-source ecosystems like huggingface @xcite and langchain (chase, 2022) have facilitated the seamless integration of multiple models @xcite . however, despite the accessibility of these tools for researchers and annotators, the integration of the query generator pipeline into search engines remains underdeveloped. furthermore, the expansion of these tools into multilingual search capabilities has been limited. besides, success in few-shot prompting @xcite @xcite has led large language models to play a key role in reducing the information burden on users by especially assisting them with writing tasks namely essay writing, summarisation, transcript and dialog generation, etc. this success has also been transferred to tasks related to query generation @xcite . while large language model applications are prevalent and numerous studies have been conducted for search interfaces @xcite @xcite , there has been little impetus to combine search interfaces with large language model-based query generation. queryexplorer distinguishes itself by offering a more comprehensive integration of various search frameworks, including query generators, reformulators, and multilingual models. unlike previous approaches, our tool addresses query generation by assuming the 'query-by-example' setting, which operates without an explicit query. the query generator component overcomes this challenge by generating a suggested query and refining it through iterative human interaction and feedback. we now briefly describe the different components of queryexplorer. ## queryexplorer the queryexplorer interface is made up of 2 tabs -the query generation tab and the settings tab. both of them are described below. the query generation tab is displayed to end users or searchers and annotators and the settings tab is reserved for researchers foot_2 looking to gather data for query generation and ir studies by allowing them to investigate different settings. the complete interface is built using huggingface's gradio platform. gradio @xcite is an open-source python package to quickly create easy-to-use, configurable ui components and has been popularly used for machine learning models. ## conclusion interactive query generation and reformulation is of significant interest for many search and exploration tasks like (document-query) pairs augmentation @xcite , document expansion @xcite and keyword expansion @xcite @xcite . query-explorer acts as a resource to permit qualitative evaluation of query generation and retrieval in conjunction. such a combined interface is crucial as it permits immediate retrieval feedback from the user to be incorporated into the search process. this paper demonstrates the novel capabilities of queryexplorer to assist researchers with investigating the construction, feedback, and evaluation of the interactive query generation process. furthermore, queryexplorer provides extensive finegrained instrumentation to record the end-to-end generation process from query formulation to retrieval feedback to enable the construction of search interaction and feedback datasets. researchers can also quickly perform qualitative analysis by loading up queryexplorer's lightweight search interface through colab and gather data quickly. queryexplorer's interface also provides an avenue to perform human-in-the-loop (hitl) studies. apart from qualitative studies, we believe queryexplorer could be effective in performing more sophisticated information retrieval experiments as well as serve as a tool to incorporate retrieval feedback and conduct human-in-the-loop studies.