Semantic Accuracy in Natural Language Generation: A Thesis Proposal (2023)

Abstract: With the fast-growing popularity of current large pre-trained language models (LLMs), it is necessary to dedicate efforts to making them more reliable. In this thesis proposal, we aim to improve the reliability of natural language generation systems (NLG) by researching the semantic accuracy of their outputs. We look at this problem from the outside (evaluation) and from the inside (interpretability). We propose a novel method for evaluating semantic accuracy and discuss the importance of working towards a unified and objective benchmark for NLG metrics. We also review interpretability approaches which could help us pinpoint the sources of inaccuracies within the models and explore potential mitigation strategies.

## introduction the introduction of the transformer architecture @xcite irreversibly changed the research landscape in natural language processing. moreover, in the past year, large pre-trained language models (llms) have managed to permeate into the hands and minds of millions of users worldwide @xcite @xcite . with a growing public interest in natural language generation (nlg) and dialogue systems, it is essential to thoroughly research their reliability. if a human does not know the answer to a question, the socially acceptable behavior is to say 'i do not know' instead of making up a plausibly sounding lie. this is how many users expect intelligent systems to behave, and failing to fulfill this expectation can lead to distrust, or in a worse scenario, even to the spread of misinformation. we believe it is worth trying to propose evaluation schemes that could incentivize institutions and companies to optimize their models for reliability rather than just fluency and impressiveness. the proposed thesis aims to take a step in this direction by investigating semantic accuracy in a data-to-text generation setting. we consider a text semantically accurate if it faithfully represents the underlying input data. despite the fact that inaccurate does not always mean wrong @xcite , i.e. conflicting with our current understanding of the world, we argue that an nlg system should produce semantically accurate texts to be considered reliable. we still consider it important to research nlg through the lens of semantic accuracy, without the intent of explicitly fact-checking @xcite , for the following reasons: ## evaluating semantic accuracy many aspects of nlg system outputs can be evaluated: fluency, grammatical correctness, acceptability with respect to a context, or similarity to a given reference text, etc @xcite . in this thesis, we focus solely on the aspect of semantic accuracy which is far from being solved. we aspire to evaluate how accurately a target text represents given source data either in a set of semantic triples (subject-predicate-object), a table, or a different structured form. our proposed output is not only the numeric result of the metric which can be used in a development or research setting, but primarily a set of alignments between the text and the data @xcite this will allow for an intuitive visualization for a user in a fact-checking setting. we consider three major types of semantic inaccuracy, following @xcite the first is extrinsic hallucination -a phenomenon where the text includes additional information that is not directly inferrable from the input data, such as introducing new entities. the second and more subtle way of introducing semantic inaccuracy is intrinsic hallucination -creating new relations between entities that are not described in the input data. finally, we consider omission -omitting some information from the source data in the target text. ## mitigating inaccuracies with interpretability in the second part of the thesis, we will use various techniques to uncover the sources of semantic inaccuracies within networks. we will then use the gained knowledge to improve the semantic accuracy of the generated text. in the first subsection, we discuss the methods we intend to explore. in the second subsection, we name the research questions we seek to answer. ## conclusion this thesis proposal has outlined the importance of investigating semantic accuracy in natural language generation. by focusing on this important aspect, we aim to address the challenge of ensuring that nlg systems generate text that represents the underlying data more faithfully. we proposed a unified benchmark for nlg metrics focusing on semantic accuracy, which will enable researchers to compare them in an objective and standardized manner. additionally, we introduced a novel semantic accuracy evaluation method, which measures how accurately the generated text represents the underlying data while also providing data-text alignments. furthermore, we discussed ways to investigate where inaccuracies appear inside nlg models, with the aim of identifying potential areas for improvement. our proposed approach includes attention visualization and probing, which provide insights into the decision-making process of the models and enhance their interpretability. the mitigation strategies we aim to use with this knowledge are attention head pruning, fine-tuning, and updating the weights using estimated uncertainty. we also aim to explore how prompt engineering can contribute to more semantically accurate texts. we hope our research will lead to improved communication between humans and machines, enhanced user experiences, and more trust from the public. challenges there is a possibility that certain llms may have already encountered the development and testing portions of the datasets that we plan to use for evaluation during their training process. we will be very mindful of this while conducting all evaluations and aim to use training data extraction techniques @xcite to verify whether this is the case for a particular set of data and a given llm. however, searching for new unseen data will be challenging and is definitely something that should be addressed by a wider scientific community.