Graph-to-text generation , a subtask of data-to-text generation, aims to generate fluent texts from graph-based data. Many graph-to-text models have shown strong performance in this task employing specialized graph encoders. However, recent approaches employ large pretrained language models (PLMs) achieving state-of-the-art results in data-to-text generation. In this paper, we aim to investigate the impact of large PLMs in graph-to-text generation. We present a study across three graph domains: meaning representations, Wikipedia knowledge graphs (KGs) and scientific KGs. Our analysis shows that PLMs such as BART and T5 achieve  state-of-the-art results in graph-to-text benchmarks without explicitly encoding the graph structure. We also demonstrate that task-adaptive pretraining strategies are beneficial to the target task, improving even furtherthe state of the art in two benchmarks for graph-to-text generation. In a final analysis, we investigate possible reasons for the PLMs' success on graph-to-text tasks. We find evidence that their knowledge about the world gives them a big advantage, especially when generating texts from KGs .
Graph-to-text generation aims to generate fluent texts from graph-based data. In this paper, we investigate two recently proposed pretrained language models (PLMs) and analyze the impact of different task-adaptive pretraining strategies for PLMs in graph-to-text generation. We present a study across three graph domains: meaning representations, Wikipedia knowledge graphs (KGs) and scientific KGs. We show that the PLMs BART and T5 achieve new state-of-the-art results and that task-adaptive pretraining strategies improve their performance even further. In particular, we report new state-of-the-art BLEU scores of 49.72 on LDC2017T10, 59.70 on WebNLG, and 25.66 on AGENDA datasets - a relative improvement of 31.8\%, 4.5\%, and 42.4\%, respectively. In an extensive analysis, we identify possible reasons for the PLMs' success on graph-to-text tasks. We find evidence that their knowledge about true facts helps them perform well even when the input graph representation is reduced to a simple bag of node and edge labels .
D
, a subtask of data-to-text generation,
R
Many graph-to-text models have shown strong performance in this task employing specialized graph encoders. However, recent approaches employ large
In this paper, we investigate two recently proposed
R
achieving state-of-the-art results in data-to-text generation. In this paper, we aim to investigate
and analyze
R
large
different task-adaptive pretraining strategies for
R
Our analysis shows that PLMs such as
We show that the PLMs
A
new
R
in graph-to-text benchmarks without explicitly encoding the graph structure. We also demonstrate
and
R
are beneficial to the target task, improving even furtherthe state of the art in two benchmarks for graph-to-text generation. In a final
improve their performance even further. In particular, we report new state-of-the-art BLEU scores of 49.72 on LDC2017T10, 59.70 on WebNLG, and 25.66 on AGENDA datasets - a relative improvement of 31.8\%, 4.5\%, and 42.4\%, respectively. In an extensive
R
investigate
identify
R
the world gives them a big advantage, especially when generating texts from KGs
true facts helps them perform well even when the input graph representation is reduced to a simple bag of node and edge labels