  Generating text from structured inputs, such as meaning representations or
RDF triples, has often involved the use of specialized graph-encoding neural
networks. However, recent applications of pretrained transformers to
linearizations of graph inputs have yielded state-of-the-art generation results
on graph-to-text tasks. Here, we explore the ability of these linearized models
to encode local graph structures, in particular their invariance to the graph
linearization strategy and their ability to reconstruct corrupted inputs. Our
findings motivate solutions to enrich the quality of models' implicit graph
encodings via scaffolding. Namely, we use graph-denoising objectives
implemented in a multi-task text-to-text framework. We find that these
denoising scaffolds lead to substantial improvements in downstream generation
in low-resource settings.
