P hoton: A Robust Cross-Domain Text-to- SQL System (2020)

Abstract: Natural language interfaces to databases(NLIDB) democratize end user access to relational data. Due to fundamental differences between natural language communication and programming, it is common for end users to issue questions that are ambiguous to the system or fall outside the semantic scope of its underlying query language. We present PHOTON, a robust, modular, cross-domain NLIDB that can flag natural language input to which a SQL mapping cannot be immediately determined. PHOTON consists of a strong neural semantic parser (63.2% structure accuracy on the Spider dev benchmark), a human-in-the-loop question corrector, a SQL executor and a response generator. The question corrector isa discriminative neural sequence editor which detects confusion span(s) in the input question and suggests rephrasing until a translatable input is given by the user or a maximum number of iterations are conducted. Experiments on simulated data show that the proposed method effectively improves the robustness of text-to-SQL system against untranslatable user input. The live demo of our system is available at http://www.naturalsql.com

## introduction natural language interfaces to databases @xcite democratize end user access to relational data and have attracted significant research attention for decades @xcite @xcite @xcite @xcite . most existing nlidbs adopt a modular architecture consisting of rule-based natural language parsing, ambiguity detection and pragmatics modeling @xcite execution on db select count(*), courses.course_name from courses join student_course_registratio ns where courses.@xmath0 ## system design in this section, we will elaborate on the system design of photon. 1 we are continuously improving the performance of the neural semantic parser. currently the semantic parser only accepts standalone question as input. we plan to also model the interaction context in future work. ## evaluation in this section, we empirically evaluate the robustness and effectiveness of photon. in particular, we examine two key modules of photon: the confusion detection module and the neural semantic parser. the former aims to detect the untranslatable questions and predicts the confusing spans; if the question is translatable, it then applies the proposed neural semantic parser to perform the text-to-sql parsing. since photon is designed as a stagewise system, we can evaluate the performance of each module separately. ## conclusion and future work we present photon, a robust modular crossdomain text-to-sql system, consisting of semantic parser, untranslatable question detector, human-inthe-loop question corrector, and natural language response generator. photon has the potential to scale up to hundreds of different domains. it is the first cross-domain text-to-sql system designed towards industrial applications with rich features, and bridges the demand of sophisticated database analysis and people without any sql background knowledge. the current photon system is still a prototype, with very limited user interactions and functions. we will continue to add more features to photon, such as voice input, spelling checking, and visualizing the output when appropriate to inspect the translation process. we also plan to improve the performance of core models in photon, such as semantic parsing (text-to-sql), response generation (table-to-text) and context-aware user interaction (text-to-text). a comprehensive evaluation will also be conducted among the users of our system.