Dynamic Facet Selection by Maximizing Graded Relevance (2021)

Abstract: Dynamic faceted search (DFS), an interactive query refinement technique, is a form of Human–computer information retrieval (HCIR) approach. It allows users to narrow down search results through facets, where the facets-documents mapping is determined at runtime based on the context of user query instead of pre-indexing the facets statically. In this paper, we propose a new unsupervised approach for dynamic facet generation, namely optimistic facets, which attempts to generate the best possible subset of facets, hence maximizing expected Discounted Cumulative Gain (DCG), a measure of ranking quality that uses a graded relevance scale. We also release code to generate a new evaluation dataset. Through empirical results on two datasets, we show that the proposed DFS approach considerably improves the document ranking in the search results.

## introduction human-computer information retrieval (hcir) is the study of techniques that takes advantage of human intelligence into the search process. through a multi-step search process, it facilitates opportunities for human feedback by taking into account the query context. examples of hcir approaches include -faceted search, relevance feedback, automatic query reformulation, illustration by tag clouds, etc. faceted search (fs) @xcite , a form of hcir, is a prevalent technique in e-commerce where document retrieval systems are augmented with faceted navigation. facets are terms that present an overview on the variety of data available given the user query, thereby hinting at the most relevant refinement operations for zooming in on the target information need @xcite . ## related work a closely related research task of facet generation is to generate alternative queries, also known as query suggestion @xcite . other related tasks are query substitution @xcite ) and query refinement @xcite . the main difference between these tasks and facet generation is that facets are not alternative/substitute/refined queries but rather a way to organize the search results obtained using the original query. another related task is query expansion @xcite where the goal is adding related words to a query in order to increase the number of returned documents and improve recall accordingly. in contrast, selection of facets allow to narrow down search results. there is a considerable amount of work on faceted search @xcite . for brevity, here we focus on dfs only. dfs can be divided into two categories. first, dfs on databases @xcite @xcite . databases have a rich meta-data in the form of tables, attributes, dimensions, etc. dfs on databases focuses on the best possible attributes from the meta-data, to be presented as facets. our contributions are in the second category -dfs on textual data. an early approach was proposed by @xcite , where the generated dynamic facets are constrained by the ability to sum pre-defined boolean expressions. @xcite proposed an approach, given a keyword as query, to dynamically select a small set of "interesting" attributes and present their aggregation to a user. their work is focused on evaluating the execution time rather than result re-ranking. @xcite proposed an approach using external resources, namely wordnet and wikipedia, to generate facets given a query. our proposed dfs approach on text generates dynamic facets that are terms (which are not restricted), not just aggregated values, and does not rely on any external resource. input queries can be natural language texts, not restricted to keywords. in a recent relevant work, @xcite proposed an unsupervised dfs approach that exploits different types of word embedding models to extract so called flat and typed facets. the typed facets are organized in hierarchies while the flat facets are simply a list of facets without hierarchy. they show empirically both set of facets yield similar results. ## proposed dynamic facet generation given a ranked set of search results from a traditional search engine, our proposed approach, namely optimistic facet set selection, tracks document ranking changes produced by selecting each candidate facet, and uses this information to select a subset of best possible facets. we use the following notations in this section: results for the initial query, q 0 returned by initial traditional ir component/search engine. ## results we implemented the flat facets proposed by @xcite to compare with our results on both datasets. we use bm25 @xcite as ir baseline for the stackoverflow benchmark. for the techqa dataset, we use the state-of-the-art ir approach of @xcite built using roberta @xcite as baseline. @xcite generously shared with us their system's output for the techqa-dr (i.e. document retrieval) task mentioned in their paper. we feed this output as input in our system as well as our implementation of @xcite to extract facets from corresponding search results. for a given query, we consider maximum 50 search results retrieved by the ir baseline. then, the oracle accepts only up to 5 facets generated from a dfs approach, and chose only one facet (i.e. a single interaction with the dfs system) as a filter. if a corresponding search result does not containing this facet, it is discarded which changes ranks of some of the remaining search results.