Generating natural sentences from Knowledge Graph (KG) triples, known as Data-To-Text Generation, is a task with many datasets for which numerous complex systems have been developed. However, no prior work has attempted to perform this generation at scale by converting an entire KG into natural text  . In this paper,  we verbalize the entire  Wikidata KG, and create a KG-Text aligned corpus in the training process. We discuss the challenges in verbalizing an entire KG versus verbalizing smaller datasets . We further show that verbalizing an entire KG can be used to integrate structured  and natural language data . In contrast to the many architectures that have been developed to integrate the structural differences between these two sources, our approach converts the KG into the same format as natural text allowing it to be seamlessly plugged into existing natural language systems. We evaluate this approach by augmenting the retrieval corpus in REALM and showing improvements, both on the LAMA knowledge probe and open domain QA  .
Prior work on Data-To-Text Generation, the task of converting knowledge graph (KG) triples into natural text , focused on domain-specific benchmark datasets . In this paper, however, we verbalize the entire English Wikidata KG, and discuss the unique challenges associated with a broad, open-domain, large-scale verbalization . We further show that verbalizing a comprehensive, encyclopedic KG like Wikidata can be used to integrate structured KGs and natural language corpora . In contrast to the many architectures that have been developed to integrate these two sources, our approach converts the KG into natural text, allowing it to be seamlessly integrated into existing language models. It carries the further advantages of improved factual accuracy and reduced toxicity in the resulting language model. We evaluate this approach by augmenting the retrieval corpus in a retrieval language model and showing significant improvements on the knowledge intensive tasks of open domain QA and the LAMA knowledge probe .
R
Generating natural sentences from Knowledge Graph (KG) triples, known as
Prior work on
R
is a task with many datasets for which numerous complex systems have been developed. However, no prior work has attempted to perform this generation at scale by converting an entire KG
the task of converting knowledge graph (KG) triples
A
, focused on domain-specific benchmark datasets
A
however,
A
English
R
create a KG-Text aligned corpus in the training process. We discuss the challenges in verbalizing an entire KG versus verbalizing smaller datasets
discuss the unique challenges associated with a broad, open-domain, large-scale verbalization
R
an entire KG
a comprehensive, encyclopedic KG like Wikidata
A
KGs
R
data
corpora
D
the structural differences between
R
the same format as natural text
natural text,
R
plugged into existing natural language systems.
integrated into existing language models. It carries the further advantages of improved factual accuracy and reduced toxicity in the resulting language model.
R
REALM and showing improvements, both on the LAMA knowledge probe and
a retrieval language model and showing significant improvements on the knowledge intensive tasks of
A
and the LAMA knowledge probe