An Unsupervised Joint System for Text Generation from Knowledge Graphs and Semantic Parsing (2020)

Abstract: Knowledge graphs (KGs) can vary greatly from one domain to another. Therefore supervised approaches to both graph-to-text generation and text-to-graph knowledge extraction (semantic parsing) will always suffer from a shortage of domain-specific parallel graph-text data; at the same time, adapting a model trained on a different domain is often impossible due to little or no overlap in entities and relations. This situation calls for an approach that (1) does not need large amounts of annotated data and thus (2) does not need to rely on domain adaptation techniques to work well on different domains. To this end, we present the first approach to unsupervised text generation from KGs and show simultaneously how it can be used for unsupervised semantic parsing. We evaluate our approach on WebNLG v2.1 and a new benchmark leveraging scene graphs from Visual Genome. Our system outperforms strong baselines for both text<->graph conversion tasks without any manual adaptation from one dataset to the other. In additional experiments, we investigate the impact of using different unsupervised objectives.

## introduction knowledge graphs (kgs) are a general-purpose approach for storing information in a structured, machine-accessible way @xcite . they are used in various fields and domains to model knowledge about topics as different as lexical semantics @xcite , common sense @xcite , biomedical research @xcite and visual relations in images @xcite . this ubiquity of kgs necessitates interpretability because diverse users -both experts and nonexperts -work with them. even though, in prin-ciple, a kg is human-interpretable, non-experts may have difficulty making sense of it. thus, there is a need for methods, such as automatic natural language generation ("@xmath0, that support them. semantic parsing, i.e., the conversion of a text to a formal meaning representation, such as a kg, ("@xmath1, thus assisting knowledge base engineers in kg creation and completion. as kgs are so flexible in expressing various kinds of knowledge, separately created kgs vary a lot. this unavoidably leads to a shortage of training data for both graph↔text tasks. we therefore propose an unsupervised model that (1) easily adapts to new kg domains and (2) only requires unlabeled (i.e., non-parallel) texts and graphs from the target domain, together with a few fact extraction heuristics, but no manual annotation. to show the effectiveness of our approach, we conduct experiments on the latest release (v2.1) of the webnlg corpus @xcite and on a new benchmark we derive from visual genome @xcite . while both of these datasets contain enough annotations to train supervised models, we evaluate our unsupervised approach by ignoring these annotations. the datasets are particularly well-suited for our evaluation as both graphs and texts are completely humangenerated. thus for both our tasks, models are evaluated with natural, i.e., human-generated targets. concretely, we make the following contributions: (1) we present the first unsupervised non-template approach to text generation from kgs (@xmath2. (2) we jointly develop a new unsupervised approach to semantic parsing that automatically adjusts to a target kg schema (@xmath3. there is a large body of literature about supervised text generation from structured data, notably about the creation of sports game summaries from statistical records @xcite . recent efforts make use of neural encoderdecoder mechanisms @xcite . although text creation from relational databases is related and our unsupervised method is, in principle, also applicable to this domain, in our work we specifically address text creation from graph-like structures such as kgs. one recent work on supervised text creation from kgs is @xcite . they generate a short description of an entity, i.e., a single kg node, based on a set of facts about the entity. we, however, generate a description of the whole kg, which involves multiple entities and their relations. @xcite generate texts from whole kgs. they, however, do not evaluate on human-generated kgs but automatically generated ones from the scientific information extraction tool sciie @xcite . their supervised model is based on message passing through the topology of the incidence graph of the kg input. such graph neural networks @xcite have been widely adopted in supervised graph-to-text tasks @xcite @xcite . even though @xcite report that graph neural networks can make better use of graph input than rnns for supervised learning, for our unsuper-vised approach we follow the line of research that uses rnn-based sequence-to-sequence models @xcite operating on serialized triple sets @xcite @xcite @xcite . we make this choice because learning a common semantic space for both texts and graphs by means of a shared encoder and decoder is a central component of our model. it is a nontrivial, separate research question whether and how encoder-decoder parameters can effectively be shared for models working on both sequential and non-sequential data. we thus leave the adaptation of our approach to graph neural networks for future work. our @xmath4, but we convert statements into kg facts whereas semantic parsing typically converts a question into a kg or database query. @xcite proposed the first unsupervised approach. they, however, still need an additional kg alignment step, i.e., are not able to directly adjust to the target kg. other approaches overcome this limitation but only in exchange for the inflexibility of manually created domain-specific lexicons @xcite . poon (2013)'s approach is more flexible but still relies on preprocessing by a dependency parser, which generally means that language-specific annotations to train such a parser are needed. our approach is endto-end, i.e., does not need any language-specific preprocessing during inference and only depends on a pos tagger used in the rule-based @xmath5. unsupervised sequence generation. our unsu-pervised training regime for both text↔graph tasks is inspired by @xcite . they used self-supervised pretraining and backtranslation for unsupervised translation from one language to another. we adapt these principles and their noise model to our tasks, and introduce two new noise functions specific to text↔graph conversion. ## noise ablation study our unsupervised objectives are defined by different types of noise models. hence, we examine their impact in a noise ablation study. for both datasets and tasks, introducing variation via noise functions is crucial for the success of unsupervised learning. the model without noise (i.e., c(x) = x) fails completely as do all models lacking rule as type of noise, the only exception being the only-drop system on webnlg. even though drop seems to work equally well in this one case, the simple translations delivered by our rulebased systems clearly provide the most useful information for the unsupervised models -notably in combination with the other noise functions: removing rule and keeping all other types of noise (cf. "sample all but rule" and "comp. all but rule") performs much worse than leaving out drop. we hypothesize that our two rule systems provide two important pieces of information: we moreover observe that our unsupervised models always improve on the rule-based systems even when rule is the only type of noise: @xmath7.2/18.3 to 19.5/37.4 on vg ball /webnlg and @xmath8.4/0.0 to 18.5/31.0. finally, our ablation study makes clear that there is no best noise model for all datasets and tasks. we therefore recommend experimenting with both different sets of noise functions and noise injection regimes (sampled vs. composed) for new data. ## conclusion we presented the first fully unsupervised approach to text generation from kgs and a novel approach to unsupervised semantic parsing that automatically adapts to a target kg. we showed the effectiveness of our approach on two datasets, webnlg v2.1 and a new text↔graph benchmark in the visual domain, derived from visual genome. we quantitatively and qualitatively analyzed our method on text↔graph conversion. we explored the impact of different unsupervised objectives in an ablation study and found that our newly introduced unsupervised objective using rule-based translations is essential for the success of unsupervised learning.