Bridging Textual and Tabular Data for Cross-Domain Text-to- SQL Semantic Parsing (2020)

Abstract: We present BRIDGE, a powerful sequential architecture for modeling dependencies between natural language questions and relational databases in cross-DB semantic parsing. BRIDGE represents the question and DB schema in a tagged sequence where a subset of the fields are augmented with cell values mentioned in the question. The hybrid sequence is encoded by BERT with minimal subsequent layers and the text-DB contextualization is realized via the fine-tuned deep attention in BERT. Combined with a pointer-generator decoder with schema-consistency driven search space pruning, BRIDGE attained state-of-the-art performance on the well-studied Spider benchmark (65.5% dev, 59.2% test), despite being much simpler than most recently proposed models for this task. Our analysis shows that BRIDGE effectively captures the desired cross-modal dependencies and has the potential to generalize to more text-DB related tasks. Our model implementation is available at https://github.com/salesforce/TabularSemanticParsing .

## introduction text-to-sql semantic parsing addresses the problem of mapping natural language utterances to executable relational db queries. early work in this area focus on training and testing the semantic parser on a single db @xcite @xcite @xcite . however, dbs are widely used in many domains and developing a semantic parser for each individual db is unlikely to scale in practice. more recently, large-scale datasets consisting of hundreds of dbs and the corresponding question-sql pairs have been released @xcite @xcite to encourage the development of semantic parsers that can work well across different dbs @xcite @xcite @xcite . the setup is challenging as it requires the model to interpret a question conditioned on a relational db unseen during training and accurately express the question intent via sql logic. consider the two examples shown in figure 1 , both questions have the intent to count, but the corresponding sql queries are drastically different due to differences in the target db schema. as a result, cross-db text-to-sql semantic parsers cannot trivially memorize seen sql patterns, but instead has to accurately model the natural language question, the target db structure, and the contextualization of both. state-of-the-art cross-db text-to-sql semantic parsers adopt the following design principles to address the aforementioned challenges. first, the question and schema representation should be contextualized with each other @xcite @xcite . second, large-scale pre-trained language models (lms) such as bert @xcite and roberta @xcite can significantly boost parsing accuracy by providing better representations of text and capturing long-term dependencies. third, under data privacy constraints, leveraging available db content can resolve ambiguities in the db schema @xcite @xcite ). consider the second example in figure 1 , knowing "plvdb" is a value of the field journal.name helps the model to generate the where condition. we present bridge, a powerful sequential text-db encoding framework assembling the three design principles mentioned above. bridge represents the relational db schema as a tagged sequence concatenated to the question. different from previous work which proposed specialpurpose layers for modeling the db schema @xcite @xcite and cross text-db linking @xcite , bridge encodes the tagged hybrid sequence with bert and lightweight subsequent layers -two single-layer bi-directional lstms @xcite . each schema component (table or field) is simply represented using the hidden state of its special token in the hybrid sequence. to better align the schema components with the question, bridge augments the hybrid sequence with anchor texts, which are automatically extracted db cell values mentioned in the question. anchor texts are appended to their corresponding fields in the hybrid sequence (figure 2 ). the text-db alignment is then implicitly achieved via fine-tuned bert attention between overlapped lexical tokens. combined with a pointer-generator decoder @xcite and schema-consistency driven search space pruning, bridge performs competitively on the well studied spider benchmark (structure acc: 65.6% dev, 59.2% test, top-4 rank; execution acc: 59.9% test, top-1 rank), outperforming most of recently proposed models with more sophisticated neural architectures. our analysis shows that when applied to spider, the bert-encoded hybrid representation can effectively capture useful cross-modal dependencies and the anchor text augmentation resulted in significant performance improvement. ## model in this section, we present the bridge model that combines a bert-based encoder with a sequential pointer-generator to perform end-to-end cross-db text-to-sql semantic parsing. ## discussion anchor selection bridge adopts simple string matching for anchor text selection. in our experiments, improving anchor text selection accuracy significantly improves the end-to-end accuracy. extending anchor text matching to cases beyond simple string match (e.g. "la"→"los angeles") is a future direction. furthermore, this step can be learned either independently or jointly with the textto-sql objective. currently bridge ignores number mentions. we may introduce features which indicate a specific number in the question falls within the value range of a specific column. input size as bridge serializes all inputs into a sequence with special tags, a fair concern is that the input would be too long for large relational dbs. we believe this can be addressed with recent architecture advancements in transformers @xcite , which have scaled up the attention mechanism to model very long sequences. relation encoding bridge fuses db schema meta data features to each individual table field representations. this mechanism is not as strong as directly modeling the original graph structure. it works well in spider, where the foreign key pairs often have exactly the same names. we consider regularizing specific attention heads to capture db connections @xcite a promising way to model the graph structure of relational dbs within the bridge framework without introducing (a lot of) additional parameters. ## conclusion we present bridge, a powerful sequential architecture for modeling dependencies between natural language question and relational dbs in cross-db semantic parsing. bridge serializes the question and db schema into a tagged sequence and maximally utilizes pre-trained lms such as bert to capture the linking between text mentions and the db schema components. it uses anchor texts to further improve the alignment between the two crossmodal inputs. combined with a simple sequential pointer-generator decoder with schema-consistency driven search space pruning, bridge attained state-of-the-art performance on spider. in the future, we plan to study the application of bridge and its extensions to other text-table related tasks such as fact checking and weakly supervised semantic parsing. ure a1 and figure a2 , we find strong connection between the anchor text and their corresponding question mention in layer 2, 4, 5, 10 and 11. we further notice that the layers effectively captures the relational db structure. as shown in figure a3 and figure a4 , we found attention patterns in layer 5 that connect tables with their primary keys and foreign key pairs. we notice that all interpretable attention connections are between lexical items in the input sequence, not including the special tokens ([t], [c], @xcite ). this is somewhat counter-intuitive as the subsequent layers of bridge use the special tokens to represent each schema component. we hence examined attention over the special tokens (figure a5 ) and found that they function as bindings of tokens in the table names and field names. the pattern is especially visible in layer 1. as shown in figure a5 , each token in the table name "poker player" has high attention to the corresponding @xcite . similarly, each token in the field name "poker player id" has high attention to the corresponding @xcite . we hypothesize that this way the special tokens function similarly as the cell pooling layers proposed in tabert @xcite . (a) table poker_player (b) table people people has high attention weights to poker_player.people_id, a foreign key referring to its primary key people.people_id.