Text Fact Transfer (2023)

Abstract: Text style transfer is a prominent task that aims to control the style of text without inherently changing its factual content. To cover more text modification applications, such as adapting past news for current events and repurposing educational materials, we propose the task of text fact transfer, which seeks to transfer the factual content of a source text between topics without modifying its style. We find that existing language models struggle with text fact transfer, due to their inability to preserve the specificity and phrasing of the source text, and tendency to hallucinate errors. To address these issues, we design ModQGA, a framework that minimally modifies a source text with a novel combination of end-to-end question generation and specificity-aware question answering. Through experiments on four existing datasets adapted for text fact transfer, we show that ModQGA can accurately transfer factual content without sacrificing the style of the source text.

## introduction text style transfer aims to control the stylistic attributes of text, such as sentiment or formality, without affecting its factual content @xcite . this task has several applications, including personalizing dialogue agents @xcite , increasing persuasiveness in marketing or news @xcite , or simplifying educational resources @xcite . while text style transfer models can adeptly alter stylistic elements, they do not address all text modification needs, especially those centered on factual modifications. specifically, there exist several applications that require the transfer of factual content between topics without altering style, such as adapting past news articles for current events ( @xcite and repurposing educational materials for new subjects @xcite , which are outside the scope of text style transfer. further, studying methods to transfer facts while preserving style could be useful for augmenting datasets, i.e., expanding training sets with new, factual training examples in a similar style @xcite , or evaluating the factual accuracy of text generation models @xcite . to address these needs, we propose the task of text fact transfer, which aims to modify the factual content of a source text while preserving its style. we define factual content as topic-specific entities that convey knowledge and style as how the factual content is phrased and organized, as well as its level of specificity foot_1 . as shown in figure 1 (top), given as inputs a source text, source topic, target topic, and corpus of facts for the target topic, we seek to generate a target text that matches the style of the source text and contains factual content specific to the target topic. thus, while text style transfer aims to modify subjective, stylistic aspects of text, text fact transfer controls the objective, factual content. one approach for text fact transfer (on parallel corpora) is to train/prompt seq2seq or large lms @xcite . however, there are two inherent challenges to text fact transfer that cannot be overcome by directly applying these models. first, the generated text must not deviate from the wording of the source text, but lms may not always succeed in this regard @xcite . for example in figure 1 , gpt-3.5 states that nelson mandela "was a member of" the anc, which is inconsistent with the phrasing of "belongs to" present in the source text. second, along with being accurate, the factual content must align with the specificity of the source text to best maintain its style. however, lms have been shown to hallucinate @xcite and struggle to control the specificity of their outputs @xcite . for example, as seen in figure 1 , the seq2seq lm states that "nelson mandela belongs to rhodesia." although the leader has some links to rhodesia, it is inaccurate to state that he belongs there. further, the source text contains the political party of joseph stalin (i.e., communist party), so the target text should contain a political party (i.e., anc) rather than a country, which is less specific. further, these challenges become more complex if supervised text fact transfer is infeasible. for example, when adapting past news for current events or augmenting datasets, it could take ample time and effort to construct parallel corpora and train a supervised model. in such cases, 0-shot models, while harder to develop, are preferred, as they can adapt to domains without extra training. hence, we must study 0-shot and supervised text fact transfer models to ensure adaptability in downstream tasks. to address these challenges of text fact transfer, we extend the concept of minimal alterations for text style transfer proposed by @xcite and seek to execute the two-step process of: (1) locating factual entities in the source text; and (2) solely transferring these entities between topics. to perform step one, we note that factual entities are inherently question-driven, and thus any entity in the source text that must be transferred can answer a question. for example in figure 1 , the factual entity "communist party" answers the question "what is joseph stalin's party?". to perform step two, we find that transferring entities between topics is challenging, but transferring questions that can retrieve said entities is simple. for example, transferring "communist party" to "anc" directly is difficult, but we can easily transfer "what is joseph stalin's party?" to "what is nelson mandela's party?" by replacing the source topic (joseph stalin) with the target topic (nelson mandela), returning a question that can be used to retrieve the entity "anc." exploiting these findings, we design modqga, a model that minimally modifies the source text with a combination of question generation (qg) and answering (qa). as shown in figure 2 , mod-qga first uses end-to-end qg to jointly produce entities from the source text and questions that can be answered by said entities. next, these questions are transferred to pertain to the target topic. mod-qga then uses specificity-aware qa to retrieve an answer from the corpus for each transferred question, while matching the specificity of the source text entities. finally, these answers are filled into the source text. solely modifying factual entities allows for the preservation of the phrasing of the source text, while the focused approach of transferring entities with specificity-aware qa promotes factuality and matched specificity, as shown in figure 1 . further, we can train the qg and qa models of modqga on external qa datasets @xcite , resulting in a 0-shot model that can be applied to diverse domains without extra training. we showcase the strength of modqga for text fact transfer by creating four parallel corpora from existing datasets, spanning expository text generation @xcite and relationship triples @xcite . hence, our initial study of text fact transfer focuses on the adaptation of expository texts and relationship triples, leaving applications such as repurposing news articles and dataset augmentation for future research. using these datasets, we design a 0-shot and supervised version of modqga and in our experiments, find that both models outperform their respective baselines in style preservation and factuality on a majority of datasets. our contributions can be summarized as follows: ## methodology given a source text d s , source topic t s , and target topic t t , text fact transfer aims to produce a target text d t that matches the style of d s and modifies the entities related to the source topic t s with entities related to the target topic t t . to serve as ground truth information for t t , we also provide a corpus of factual sentences c related to the target topic t t . as illustrated in figure 2 , the backbone of mod-qga consists of two key modules: (i) an end-toend question generator p(q s |d s , t s ) that produces question/entity pairs (q, e) ∈ q s , where each q can be answered by e using the source text d s ; and (ii) a specificity-aware question answering model p(⟨a i , a j ⟩|c, q, e) that extracts an answer span ⟨a i , a j ⟩ from the context c (where c ⊆ c), which answers question q and matches the specificity of the entity e. after training these models, modqga performs text fact transfer via: 1) endto-end question generation with p(q s |d s , t s ); 2) target text: stanford has a suburban location and its campus size is 3,180 acres. ## experimental setup we provide a detailed setup in appendix a. ## conclusion we propose the task of text fact transfer and develop modqga to overcome the difficulty of lms to perform our task. modqga leverages a novel combination of end-to-end question generation and specificity-aware question answering to perform text fact transfer. through experiments on four datasets, including human evaluation, we find that 0-shot and supervised modqga excel in style preservation and factuality on a majority of datasets. we conduct an ablation study to reveal the strengths of our design choices of modqga. finally, we perform a qualitative analysis of our specificity-aware question answering model, which shows at least some ability to control the specificity of its answers.