N eural QA : A Usable Library for Question Answering (Contextual Query Expansion + BERT ) on Large Datasets (2020)

Abstract: Existing tools for Question Answering (QA) have challenges that limit their use in practice. They can be complex to set up or integrate with existing infrastructure, do not offer configurable interactive interfaces, and do not cover the full set of subtasks that frequently comprise the QA pipeline (query expansion, retrieval, reading, and explanation/sensemaking). To help address these issues, we introduce NeuralQA - a usable library for QA on large datasets. NeuralQA integrates well with existing infrastructure (e.g., ElasticSearch instances and reader models trained with the HuggingFace Transformers API) and offers helpful defaults for QA subtasks. It introduces and implements contextual query expansion (CQE) using a masked language model (MLM) as well as relevant snippets ( RelSnip ) - a method for condensing large documents into smaller passages that can be speedily processed by a document reader model. Finally, it offers a flexible user interface to support workflows for research explorations (e.g., visualization of gradient-based explanations to support qualitative inspection of model behaviour) and large scale search deployment. Code and documentation for NeuralQA is available as open source on Github.

## introduction the capability of providing exact answers to queries framed as natural language questions can significantly improve the user experience in many real world applications. rather than sifting through lists of retrieved documents, automatic qa (also known as reading comprehension) systems can surface an exact answer to a query, thus reducing the cognitive burden associated with the standard search task. this capability is applicable in extending conventional information retrieval systems (search engines) and also for emergent use cases, how many rose species are found in the montreal botanical garden? such as open domain conversational ai systems @xcite . for enterprises, qa systems that are both fast and precise can help unlock knowledge value in large unstructured document collections. conventional methods for open domain qa @xcite follow a twostage implementation -(i) a retriever that returns a subset of relevant documents. retrieval is typically based on sparse vector space models such as bm25 @xcite and tf-idf @xcite ; (ii) a machine reading comprehension model (reader) that identifies spans from each document which contain the answer. while sparse representations are fast to compute, they rely on exact keyword match, and suffer from the vocabulary mismatch problem -scenarios where the vocabulary used to express a query is different from the vocabulary used to express the same concepts within the documents. to address these issues, recent studies have proposed neural ranking @xcite and retrieval methods @xcite @xcite , which rely on dense representations. however, while dense representations show significantly improved results, they introduce additional complexity and latency, which limits their practical application. for example, @xcite require a specialized mlm pretraining regime, as well as a supervised fine-tuning step, to obtain representations used in a retriever. similarly @xcite use dual encoders in learning a dense representation for queries and all documents in the target corpus. each of these methods require additional infrastructure to compute dense representation vectors for all documents in the target corpus as well as implement efficient similarity search at run time. in addition, transformerbased architectures @xcite used for dense representations are unable to process long sequences due to their self-attention operations which scale quadratically with sequence length. as a result, these models require that documents are indexed/stored in small paragraphs. for many use cases, meeting these requirements (rebuilding retriever indexes, training models to learn corpus specific representations, precomputing representations for all indexed documents) can be cost-intensive. these costs are hard to justify, given that simpler methods can yield comparable results @xcite . furthermore, as reader models are applied to domain-specific documents, they fail in counter-intuitive ways. it is thus valuable to offer visual interfaces that support debugging or sensemaking of results (e.g., explanations for why a set of documents were retrieved or why an answer span was selected from a document). while several libraries exist to explain nlp models, they do not integrate interfaces that help users make sense of both the query expansion, retriever and the reader tasks. collectively, these challenges can hamper experimentation with qa systems and the integration of qa models into practitioner workflows. in this work, we introduce neuralqa to help address these limitations. our contributions are summarized as follows: overall, neuralqa complements a line of endto-end applications that improve qa system @xcite and provide visual interfaces for understanding machine learning models @xcite @xcite . ## the question answering pipeline there are several subtasks that frequently comprise the qa pipeline and are implemented in neuralqa. ## neuralqa system architecture in this section, we review the architecture for neu-ralqa, as well as design decisions and supported workflows. the core modules for neuralqa (fig. 2 ) include a user interface, retriever, expander, and reader. each of these modules are implemented as extensible python classes (to facilitate code reuse and incremental development), and are exposed as rest api endpoints that can be either consumed by 3rd party applications or interacted with via the neuralqa user interface. ## conclusion in this paper, we presented neuralqa -a usable library for question answering on large datasets. neuralqa is useful for developers interested in qualitatively exploring qa models for their custom datasets, as well as for enterprise teams seeking a flexible qa interface/api for their customers. neu-ralqa is under active development, and roadmap features include support for a solr retriever, additional model explanation methods and additional query expansion methods such as rm3 @xcite . future work will also explore empirical evaluation of our cqe and relsnip implementation to better understand their strengths and limitations.