KGAST : From Knowledge Graphs to Annotated Synthetic Texts (2024)

Abstract: In recent years, the use of synthetic data, either as a complement or a substitute for original data, has emerged as a solution to challenges such as data scarcity and security risks. This paper is an initial attempt to automatically generate such data for Information Extraction tasks. We accomplished this by developing a novel synthetic data generation framework called KGAST, which leverages Knowledge Graphs and Large Language Models. In our preliminary study, we conducted simple experiments to generate synthetic versions of two datasets—a French security defense dataset and an English general domain dataset, after which we evaluated them both intrinsically and extrinsically. The results indicated that synthetic data can effectively complement original data, improving the performance of models on classes with limited training samples. This highlights KGAST’s potential as a tool for generating synthetic data for Information Extraction tasks.

## introduction information extraction (ie) models serve as crucial components across various domains, enabling us to make informed decisions based on complex data. however, the effectiveness of these models is dependent on the availability and quality of training data. in this context, we encounter two critical challenges: 1) data scarcity: frequently, despite having complex modeling techniques, researchers deal with datasets that are either insufficient in size or entirely unavailable. without a sufficient number of labeled examples, ie models struggle to generalize effectively, compromising their predictive capabilities. 2) privacy and compliance concerns: in an era of heightened data privacy regulations, organizations must navigate the balance between model performance and safeguarding sensitive information. certain datasets whether due to privacy risks or legal constraints cannot be openly shared, which further complicates training effective ie models. to address these issues, researchers often resort to manual data augmentation. this labor-intensive process involves collecting additional data points and meticulously annotating them. while effective, it is time-consuming and resource-intensive, making it less feasible for small organizations operating on limited budgets. numerous studies have explored the expansion of training data by introducing additional synthetic data @xcite @xcite . these studies presented straightforward strategies, such as substituting certain words with their equivalent terms. these equivalents can be retrieved from external sources like wordnet @xcite , dbnary @xcite , or they can be calculated using word embedding models such as word2vec @xcite , fasttext @xcite , and glove @xcite . although these techniques can indeed augment the initial training dataset, they fail to generate adequate diversity for the models to generalize effectively in subsequent tasks, owing to the minimal semantic variations from the original data. back-translation is another recognized technique for augmenting the initial training data. with machine translation models @xcite , paraphrases of each sentence can be obtained through the back-translation process. while back-translation effectively amplifies the dataset size twofold, it introduces a notable challenge in terms of annotation. the text derived from back-translation diverges from the original annotation. thus, either careful manual annotation or sophisticated annotation algorithms are required to update the annotations in alignment with the back-translated text, ensuring the precision of the dataset. to address these issues of low data diversity and misalignment of the original annotation, we propose a novel synthetic data generation framework called kgast. this framework leverages knowledge graph to automatically generate annotated synthetic texts that can be used for training ie models. in this study, we sought to answer questions similar to the ones raised in this paper @xcite : ## limitation one of the limitations of this study is that we only generated new data based on the original data's knowledge graphs, which led to low diversity in the dataset. future work could involve experimenting with modified knowledge graphs to enhance diversity. we acknowledge that the annotations produced by our framework are far from perfect and require further enhancements. one potential im-provement could be the use of a dependency tree to identify co-references and annotate them. it could also be used to extract relations between entities. another path for improvement could be the use of attention weights from the generated texts. this could help identify the evidence of relations by pinpointing where the head and tail entities most attentively interact within the texts.