Schema Aware Semantic Reasoning for Interpreting Natural Language Queries in Enterprise Settings (2020)

Abstract: Natural Language Query interfaces allow the end-users to access the desired information without the need to know any specialized query language, data storage, or schema details. Even with the recent advances in NLP research space, the state-of-the-art QA systems fall short of understanding implicit intents of real-world Business Intelligence (BI) queries in enterprise systems, since Natural Language Understanding still remains an AI-hard problem. We posit that deploying ontology reasoning over domain semantics can help in achieving better natural language understanding for QA systems. In this paper, we specifically focus on building a Schema Aware Semantic Reasoning Framework that translates natural language interpretation as a sequence of solvable tasks by an ontology reasoner. We apply our framework on top of an ontology based, state-of-the-art natural language question-answering system ATHENA, and experiment with 4 benchmarks focused on BI queries. Our experimental numbers empirically show that the Schema Aware Semantic Reasoning indeed helps in achieving significantly better results for handling BI queries with an average accuracy improvement of ~30%

## introduction natural language query (nlq) interfaces for information access (e.g., alexa, google assistant, and siri) have gained popularity in recent times as they allow the end-users to specify their information needs in natural language. in enterprise settings, systems with nlq interfaces @xcite @xcite are especially desirable as they enable the business analysts to access domain-specific knowledge without having to learn specialized query languages such as sql, sparql, or other customized query languages. nlq interfaces also allow the end-user to explore and analyze the data without the need to learn about the storage mechanism and schema-specific details of the underlying system. thus such systems can help boost analyst productivity and minimize training costs and time. ## system in this section, we detail the proposed reasoning framework for natural language query interpretation. we start with the overall system architecture and then discuss each of the components in detail. the proposed framework, as illustrated in figure 2 , can be divided into (i) an offline part and (ii) an online part. the offline part consists of components that are curated one time for a target domain and are used as resources later to facilitate reasoning for query interpretation. the online part consists of components that work on each input natural language query and help interpret the query in the context of the target domain by using the offline components for domain reasoning. ## experiments in this section, we present a comprehensive evaluation of our proposed system. we describe in detail the experimental setup, benchmarks on which our prototype has been tested, followed by the set of evaluation metrics we have used, and finally the results with in-depth analysis. as described in section 3, we used our reasoning approach on top of existing state-of-the-art athena. we have reused the existing language processors from athena, whereas the algorithms with reasoning components e.g. consistency checker, interpreter, etc. are implemented using jfact reasoner @xcite to create our prototype reasonedathena. the modular architecture of our reasoning framework allows it to be built on top of any other state-of-the-art qa systems as well which rely on an intermediate level of annotations as athena. this allows our proposed framework to be adaptable to different state-of-the-art implementations too. ## summary we proposed a novel schema-aware semantic reasoning framework where we have used ontology reasoning as a tool to help natural language query interpretation, specifically to interpret complex bi queries across domains. ontology reasoning, being a well studied problem and with a theoretically sound solution framework, provides us a sound foundation to develop a robust query interpretation framework. for a quantitative evaluation, we have applied our reasoning framework over a state-of-the-art ontology-based question answering system athena and observed significant improvements in accuracy and coverage of query interpretation across 4 benchmarks. deep domain understanding and domain reasoning are some of the hard challenges faced by qa systems today, limiting their applicability in enterprise settings. our proposed reasoning framework can provide the necessary bridge to use qa systems for bi queries in enterprise settings.