Venue: NAACL 2021

# ENTRUST: Argument Reframing with

Language Models and Entailment

Tuhin Chakrabarty1, Christopher Hidey 3 Smaranda Muresan1,2

Department of Computer Science, Columbia University 1

Data Science Institute, Columbia University 2

Google 3

{tuhin.chakr, smara}@cs.columbia.edu

 chris.hidey@gmail.com

Footnote 1: The work is not affiliated to Google and was conducted independently outside of the organization

###### Abstract

Framing involves the positive or negative presentation of an argument or issue depending on the audience and goal of the speaker (Entman, 1983). Differences in lexical framing, the focus of our work, can have large effects on peoples' opinions and beliefs. To make progress towards reframing arguments for positive effects, we create a dataset and method for this task. We use a lexical resource for connotations to create a parallel corpus and propose a method for argument re-framing that combines controllable text generation (positive connotation) with a postdecoding entailment component (same denotation). Our results show that our method is effective compared to strong baselines along the dimensions of fluency, meaning, and trustworthiness/reduction of fear.

## 1 Introduction

Public opinion has been shown to be significantly influenced by _framing effects._ Framing refers to the presentation of an issue, where even small changes may have outsized effects on beliefs Chong Druckman (2007). For example, when asked about "welfare," the American public is largely against increasing spending (with only 20% in favor), but when asked about "assistance to the poor," 65% believe that the government is not spending enough Rasinski (1989).

While other research has focused on _syntactic_ framing Greene and Resnik (2009) or _issue_ framing Hartmann (2019), we focus specifically on _lexical_ framing, distinguishing sentences by their _monotative_ meaning even where they have the same _denotative_ meaning. According to Frege (1892), two sentences with the same truth conditions may refer to the same entities or state of affairs (_"reference,"_ also known as denotation) but be presented differently (_"sense"_ or connotation). For example, "undocumented workers" and "illegal aliens" have the same denotation but different connotations Webson et al. (2020).

The examples in Table 1 are instances of _lexical_ framing, where word choice determines the difference in presentation McCombs and Ghanem (2001). For example, Arg1 and Arg2 contain collocations (in red) that have a high partisan skew Webson et al. (2020), while Arg 3 and Arg 4 are examples of appeal to fear or prejudice argument fallicies from propagandist news articles Da San Martino et al. (2019). The goal is to reframe such arguments to be more trustworthy (e.g., less partisan, no appeal to fear fallacy).

Connotations may be distinguished along the

\begin{table}
\begin{tabular}{|p{14.2pt}|l|} \hline  & Alabama’s Supreme Court Chief Justice was suspended... for ordering state probate judges not to grant _marriage licenses_ to gay couples.. \\ \hline  & Alabama’s Supreme Court Chief Justice was suspended.... for ordering state probate judges not to grant _legal marriage equality_ to gay couples... \\ \hline \multirow{3}{*}{Arg1} & Every nation with _retriorial claims_ in the article is a member of NATO, except Russia. \\ \cline{2

dimensions of politeness, sentiment, or tangibility, among others (Allaway and McKeown, 2020), but in our work we consider emotional association such as fear and trust. Appeal to _fear_ is considered an argumentative fallacy (Walton, 2006; Thierer, 2012) and appears prominently in manipulative text such as propaganda (Da San Martino et al., 2019). On the other hand, arguments with _trusted_ language align with the Aristotelian modes of persuasion, specifically ethos (Aristotle and Bartlett, 2019).

In our work, we leverage such a lexical resource for connotations (Allaway and McKeown, 2020) to reframe arguments to be more trustworthy (e.g., less partisan, no appeal to fear fallacy), while maintaining the same denotative meaning. While retrieve-and-replace methods perform well on other attribute transfer tasks such as sentiment (Li et al., 2018; Sudhakar et al., 2019), our task is more dependent on broader context within a sentence even though we are performing localized replacement. Thus, there are two main challenges we need to address: 1) the lack of a parallel dataset of negatively and positively framed arguments (naturally-occurring); and 2) a generation approach that can not only change the connotative meaning but also keep the same denotative meaning of the input argument.

We introduce our approach called **ENTRUST** : Argum**ENT** Reframing with langUage modelS and enTailment, with the following contributions: 1) A Connotation-guided Masked Language Model approach to generate a parallel dataset of naturally occuring arguments and their refrarmings (Section 2); 2) A method for argument reframing that combines controllable text generation (connotative meaning associated with trust) and entailment (same denotative meaning) (Section 3); 3) An evaluation on two different tasks -- reframing partisan arguments and appeal to fear/prejudice fallacies -- showing that our method is preferred over a strong retrieval-based baseline (Sudhakar et al., 2019) and state-of-the-art pretrained language model (Lewis et al., 2019), and it is close to human performance on several evaluation criteria such as fluency, meaning, trustworthiness/reduction in fear. Code, data, and models available at [https://github.com/tuhinjubcse/ArgReframingNAACL2021](https://github.com/tuhinjubcse/ArgReframingNAACL2021)

## 2 Automatic Parallel Data Creation

To facilitate the referring of arguments, we require a large-scale parallel corpora of sentences with the same denotation but different connotative meaning.

Selection of naturally-occurring argumentsSince our goal is to re-write arguments, it is essential to identify an abundant source of naturally-occurring arguments. The _Change My View_ subreddit, an argumentative discussion forum intended for persuasion on diverse topics, has been used extensively in computational argumentation research (Tan et al., 2016; Wei et al., 2016; Musi et al., 2018; Chakrabarty et al., 2019, 2019). We collect sentences from the same source and classify them as _claim_, _premise_, or _non-argument_ using the fine-tuned BERT model released by Chakrabarty et al. (2019). This results in 301,166 arguments labeled as premises. We consider only premises to create our parallel data because argumentative appeals occur within justifications (premises) for or against the speaker's claim.

Connotation-guided Masked Language ModelAllaway and McKeown (2020) provide a resource with words labeled for lexical connotations, using the aspects of _Social Value, Politeness, Impact, Factuality, Sentiment, Emotional Association_. For our work we only consider _Emotional Association_, although in future work our methods could be applied for other aspects. To create a parallel corpus, we use this lexical resource and the 301,166 automatically identified premises from _Change My View_ to obtain candidate words within those premises for replacement. We match words from the premises to those that have entries in the dictionary with _emotional_ connotations such as fear, trust, anticipation, and joy. To generate replacements for these words, we need to find substitutions that maintain denotative meaning while changing connotative meaning. We use the connotation dictionary to address the latter. However, to address the former, we need to provide only paraphrases that consider the context in which these words occur. We thus use a masked language model (MLM).

Masked language modeling approaches like BERT (Devlin et al., 2018) and RoBERTa (Liu et al., 2019) can be considered _cloze_ or "fill-in-the-blank" tasks, where the model uses the context surrounding a masked-out token to try to predict what the masked word should be. We borrow this framework (RoBERTa-large, in particular) to mask the candidate words we identified via the connotation lexicon. However, the rank of a predicted token from an MLM is based on the language model 

probability - it provides no information about lexical connotations. A premise re-written from MLM replacements may thus have the same connotative meaning. To avoid this scenario, we restrict the MLM replacements to be words with different connotations than the original masked word (i.e., different Emotional Association).

Our data creation process is depicted in Figure 1. In example 2, the word _"resources"_ has the connotations _joy:trust_ in our dictionary. The MLM generates the replacement _"tools,"_ which we verify has a different connotation (_emotionally neutral_). For example 1, the words _"prepare,"__real,"_ and _"defense"_ have the emotional connotations _anticipation_, _trust_, and _anticipation;anger:fear_ respectively. These words are replaced with _plan_, _your_, and _safety_, using our MLM.

We treat the original premises as the "target" and the connotation-guided MLM generated premises as the "source" for our method of argument reframing detailed in the next section (Figure 1). While this process provides us with a parallel dataset for reframing, we enhance the source-side of the data to provide additional control during generation. Motivated by the work of Schiller et al. (2020), which used aspect as a "control code" Keskar et al. (2019) for argument generation, we also prepend the emotional associations of the replaced words. Using the connotations from the lexical resource, we add all listed emotions as control codes by separating them with a special token ("[DELIM]") (the top right block of Figure 1). During inference, we thus have more control over the emotion of the words we are generating (in our case we specifically use trust as the control code). For additional control, we also insert demarcator tokens ("[SEP]") at the boundary of the words we aim to replace to provide our generative model with a better signal on what to replace or rewrite. While the downside is that we need to identify spans for replacement at test/inference time, our experiments will show that using collocations or fear words makes it unnecessary.

By using the lexical connotation resource we do not have to rely on a separate module/tagger based approach like that of Pryzant et al. (2020) to find biased or problematic words that may introduce additional noise during training. Our parallel data has 271,022 pairs for training and 30,114 for validation on which perplexity is evaluated.

## 3 Method for Argument Reframing

As our goal is to change _connotation_ while maintaining _denotation_, we divide our approach to rewriting arguments into two primary tasks: 1) generating the appropriate lexical substitutions while being pertinent to the context; 2) ensuring that re-written arguments reflect the desired emotional association while maintaining the same denotative meaning as the input.

Figure 1: A schematic illustration of our system ENTRUST, where the top block shows our **training** process where we use MLM along with connotation resource to transform an original argument to an argument bearing different emotional connotation and use them to fine-tune BART. The block below shows the **inference** step (test time) where we use fine-tuned BART to reframe the argument containing _partisan collocation_ along with a NLI component to ensure the same denotation with the input argument.

 



### Controllable Text Generation

BART Lewis et al. (2019) is a pre-trained model combining bidirectional and auto-regressive transformers that achieves state-of-the-art results in several text generation tasks. It is implemented as a sequence-to-sequence model with a bidirectional encoder over corrupted text and a left-to-right auto-regressive decoder. In principle, the pre-training procedure has two stages: (1) text is corrupted with an arbitrary noising function, and (2) a transformer-to-transformer model is learned to reconstruct the original text. Because BART has an auto-regressive decoder, it can be directly fine-tuned for most sequence generation tasks. Here, the encoder input is a sequence of words, and the decoder generates outputs auto-regressively. We refer the reader to Lewis et al. (2019) for further details.

For our task, we fine-tune BART on our parallel data, where the reframed argument using MLM & connotation dictionary is the encoder source and the original argument is the decoder target (Figure 1). The emotional connotations added to the source via the special token DELIM (see Section 2) act as a "control code" for generation. Moreover, for lexical framing, subtle differences in word choices matter the most. By explicitly using special tokens ([SEP]) in our parallel data during fine-tuning, the BART model learns what to edit, instead of editing random words in the sentence, a common issue often found in attribute transfer models Li et al. (2018a); Sudhakar et al. (2019). At test time, therefore, we can ensure the model reframes a desired content span. All hyper-parameters are mentioned in the **Appendix A**.

Post fine-tuning at the decoding step, we use a top-k sampling strategy Fan et al. (2018) to reframe arguments conditioned on a input argument and a target emotion.

### Post-decoding NLI

Our task is challenging in comparison to traditional text attribute transfer tasks as we need to maintain the same denotative meaning as the input. While in most cases BART is able to generate content which is semantically similar to the input, it sometimes contradicts the input. For example, Table 2 shows that BART changes _soft power_ to _military strength_. Here the denotative meaning changes. To control for this, we introduce an additional post-processing step.

We generate multiple outputs by varying the value of \(k\) (between 5 and 50) while conducting top-k sampling. We then calculate the entailment scores of these outputs with the input argument respectively using a RoBERTa Liu et al. (2019) model fine-tuned on the Multi-NLI dataset Williams et al. (2018) and then select the output having the best entailment score. We also experimented with other methods for incorporating entailment during decoding based on prior work (Section 8), but found these techniques to be less effective than our method. As pre-trained sequence-to-sequence language models are good at copying input and generating naturalsounding output, we hypothesize that our approach will better allow us to change connotative meaning without affecting fluency and denotation. In contrast, approaches such as "vocab boosting" Ghosh et al. (2017) increase the logits of key connatotive words, which would necessarily decrease the probabilities of functional words and words necessary for maintaining denotative meaning. Other approaches such as reinforcement learning Pasunuru and Bansal (2017) may further decrease these desired qualities, while trying to maximize another objective.

## 4 Evaluation Tasks and Test Data

To evaluate our methods for argument reframing we need to look beyond our automatically labeled data. We consider two tasks: 1) reframing an argument that contains partisan language to a less partisan argument; and 2) reframing an appeal to fear or prejudice fallacy to an argument without this fallacy.

Recently Webson et al. (2020), proposed resources and methods to disentangle denotation and connotation in vector spaces. They evaluate their methods on a sample of around 300 collocations from vocabulary of Congressional records Gentzkow et al. (2019) and Hyper-partisan News

\begin{table}
\begin{tabular}{c|l} \hline  & trust <V>I suppose we could argue that they’re much better at _soft power_ than Nazi Germany or the USSR, but come mono \\ \hline BART & I suppose we could argue that they’re much better at _military strength_ than ......... \\ \hline BART & I suppose we could argue that they’re much better at _diplomatic communication_ than ....... \\ \hline \end{tabular}
\end{table}
Table 2: Generation from fine-tuned BART without control for entailment can sometime contradict the input thereby failing to maintain the same denotative meaning

(Kiesel et al., 2019) that occur at least 100 times and have high partisan skew. We use these words to filter arguments from the subreddits _ChangeMyView_ and _Politics_. Some of these collocations include phrases such as _abortion providers, investment vehicles, broken system, soft power,_ and _territorial claims_. We randomly sample 100 such arguments to benchmark the performance of our model and further use towards human evaluation.

In addition, we test our models on propaganda techniques employed in news articles with an _Appeal to Fear or Prejudice_(Da San Martino et al., 2019). There are a total of 182 sentence-level text fragments labeled as Appeal to Fear or Prejudice in the dataset released by Da San Martino et al. (2019). We classify these 182 fragments as claims/premises/non-argument and randomly sample 50 premises. Our goal is to reduce the fallacious nature of the argument without changing the denotative meaning.

As our training distribution is different from these two datasets, these tasks and test sets allow us to better test the generalization capabilities of our models. Furthermore, almost none of the collocations introduced by Webson et al. (2020) appear in the connotation dictionary of Allaway and McKeown (2020), which helps us avoid the risk of mimicking replacements from our training data.

For both of these tasks, we ask humans to generate reframings based on our input test data for comparison and benchmark. We recruit two experts with argumentation and journalism background (not authors of this paper) to reframe arguments. For Appeal to Fear the instructions given were to make it less fallacious by reducing the fear and rephrasing the argument (HUM1 in Table 3), while for arguments with partisan collocation the human was instructed to change the collocation so as to make it trustworthy (HUM2 in Table 3).

## 5 Experimental Setup

To compare the quality of the reframed arguments, we benchmark our ENTRUST model against human performance and four baseline systems described below. For the data containing collocations from (Webson et al., 2020), because we know they represent partisan language the ideal goal is to reframe them. For Appeal to Fear or Prejudice data we reframe words which portray an emotion of fear based on the popular NRC Emotion Lexicon Mohammad and Turney (2013).

### Baseline Systems

As argument reframing is a new task, we adapt several baselines that have been used for other generation tasks and also compare with human-generated reframings.

**Bart wIthout Demarcator and ENtailment (BART\({}^{w/o}_{D+EN}\)):** This is the pre-trained BART model fine-tuned on our parallel data without explicitly adding signals on what to edit or reframe and without post-processing based on entailment scores. This experiment helps us understand if BART learns to adapt to the emotional connotations and can automatically edit partisan collocations or words inducing fear without control.

**Bart without EnTAilment (BART\({}^{w/o}_{EN}\)):** This is the pre-trained BART model fine-tuned on our parallel data with explicit signals ([SEP] token) but without the NLI component as a post-processing tool. This experiment helps us understand how well BART learns to adapt to the emotional connotations without altering the denotative meaning once guided with what to reframe.

**Lexical Replacement (LEXREP):** We use a similar method employed for our parallel data creation. We rely on Masked Language Models for lexical substitutions. Because our goal is to reframe arguments to be trustworthy we prefer substitutions which have a connotation of trust in the resource by Allaway and McKeown (2020). In case we cannot find the substitution in the connotation dictionary we honor default MLM predicted infilling.

Generative Style Transformer (GST):We use the state of art for text style transfer by Sudhakar et al. (2019a), which is a part of a larger "_Delete Retrieve Generate_" framework (Li et al., 2018a). To maintain parity with other baselines, instead of letting the model delete attribute keywords we delete

\begin{table}
\begin{tabular}{|l|l|} \hline INP1 & It would be _dangerous_ , _s suicidal folly_ for \\  & infields to pretend that ramadians not the month of jihad \\ \hline HUM1 & It would be _counterproductive and_ \\  & _unreasonable_ for infields to ......... jjhad \\ \hline INP2 & Trump backs away from further \\  & _military confrontation_ with Iran \\ \hline HUM2 & Trump backs away from further \\  & _military engagement_ with Iran \\ \hline \end{tabular}
\end{table}
Table 3: INP1 and INP2 are test data instances where INP1 is a Appeal to Fear example while INP2 is an argument containing partisan collocation 

the partisan collocations or fear related words from the arguments as the first step, followed by the usual retrieve and generate steps. Our training data for this method includes only arguments labeled with their attribute (e.g., positive or negative). Arguments containing lexical connotations catering to trust are positive, while those not catering to trust are negative.

### Evaluation Criteria

Automatic evaluation.One important criterion is to measure if the reframed arguments are faithful to the input. Even though we are changing the argument for connotations, it should still maintain the same denotative meaning as the input. To this end we calculate **Semantic Similarity** with our input using SENTENCE BERT(SBERT) Reimers and Gurevych (2019).

Human evaluation.We use Amazon Mechanical Turk to evaluate on a total of 900 utterances, 750 generated from 5 systems and 150 utterances generated by humans. We proposed a set of 3 criteria to evaluate the generated output: (1) **Fluency (F)** ("How fluent and grammatical are the utterances?"), (2) **Meaning Preservation (M)** ("How well does the reframed argument capture the same denotative meaning as the input argument?"), (3) **Trustworthiness/Presence of Fear(T/PF)**. For the 100 input arguments reflecting partisan view we ask Turkers to rate reframed arguments based on trustworthiness with respect to the input. For the 50 Appeal to Fear or Prejudice fallacies we ask Turkers to rate reframed arguments based on presence of fear (the intention behind this being that we want to rank systems which portray the least amount of fear). In both of these ratings we still ask Turkers to keep into account the denotative meaning (i.e., making it trustworthy or less fallacious at the expense of meaning alterations should be scored lower). We hired 40, 25, 39 (23 and 16) Turkers for the three separate tasks respectively. The computed IAA using Krippendorff's alpha for Fluency, Meaning Preservation , Trustworthiness and Presence of Fear is 0.62, 0.65, 0.51, 0.46, respectively.

## 6 Results

Automatic Evaluation.As can be seen in Table 4 our model ENTRUST maintains the denotative meaning with the input better than other systems (\(p<.001\) using approximate randomization tests) and only marginally behind humans when it comes to arguments with partisan collocations. For Appeal to Fear or Prejudice our system maintains better denotative meaning than all systems except LEXREP (\(p<.001\)). The automatic metric somewhat penalizes humans for changing more content than just targeted words; this unreliability is a known issue with automated metrics Novikova et al. (2017) and strongly implies a need for human evaluation.

Human Evaluation.Table 5 shows the results of our human-based evaluations. For fluency, meaning preservation, trustworthiness, and reduction of fear the ENTRUST model is better than all the baselines (\(p<.001\) using approximate randomization tests). It is further encouraging to see that the entailment step helps us maintain better denotative meaning (See Table 5 Col3: Row 4 vs Row 7). For Presence of Fear, Turkers often rate our ENTRUST model to be the least fearful, including slightly when compared to reframings of an expert. We hypothesize this is because the human judges found it difficult to completely remove fear while keeping the denotative meaning (indeed the

\begin{table}
\begin{tabular}{|l|c|l|} \hline System & Pariusan Task & Appeal to Fear Task \\ \hline BART\({}^{w/o}_{D+EN}\) & 64.1 & 38.5 \\ \hline BART\({}^{w/o}_{EN}\) & 91.9 & 43.1 \\ \hline GST & 86.4 & 38.3 \\ \hline LEXREP & 92.4 & 44.3 \\ \hline ENTRUST & 92.9 & **44.5** \\ \hline \hline HUMAN & **93.9** & 41.6 * \\ \hline \end{tabular}
\end{table}
Table 4: Semantic Similarity of reframed arguments with input arguments. (*) Here human did not restrict themselves to just lexical framing, so automated metrics might penalize them for more reframing.

\begin{table}
\begin{tabular}{|l|l|l|l|l|} \hline System & Fluency & Meaning & Trust \(\uparrow\) & Fear\(\downarrow\) \\ \hline INPUT & - & - & 3.24 & 3.36 \\ \hline BART\({}^{w/o}_{D+EN}\) & 2.78 & 2.56 & 2.60 & 3.01 \\ \hline BARIC\({}^{w}_{CN}\) & 3.39 & 3.00 & 3.13 & 2.58 \\ \hline LEXREP & 3.38 & 3.00 & 3.08 & 2.54 \\ \hline GST & 2.14 & 1.81 & 2.01 & 2.44 \\ \hline ENTRUST & 3.51 & 3.30 & 3.52 & **2.39** \\ \hline \hline HUMAN & **3.72** & **3.63** & **3.71** & 2.59 \\ \hline \end{tabular}
\end{table}
Table 5: Fluency and Meaning Preservation scores given by human judges on a scale of (1-5) for reframed arguments with respect to input arguments. Fluency and Meaning Preservation ratings are for all arguments in test set, while Trust ratings are for arguments with Partisan collocation (higher scores better), and Fear ratings for Appeal To Fear or Prejudice ones only (lower scores better).

 

humans scores slightly better on meaning comparing with our system ENTRUST). Sometimes, an ungrammatical generation or a reframing which change the meaning will contain less fear (rating 1 meaning no fear at all). However, to avoid this we explicitly asked Turkers to rate those samples as moderate so as to not bias the overall results.

## 7 Analysis and Discussion

As can be seen in Table 6, the ENTRUST model accurately captures _diplomatic credibility_ as an alternate for soft power which is encouraging as soft power is measured through _culture, diplomacy, education, business/innovation, and government.1_ The BART\({}^{w/o}_{D+EN}\) model often fails to reframe anything, which shows the importance of adding [SEP] tokens as explicit supervision so that the model knows what to edit. The GST model fails at both grammaticality and meaning preservation, which makes it harder to judge its trustworthiness and ability to ameliorate fearful appeal. Finally, ENTRUST reformings are not static. Table 8 shows that for the same collocation of _targeted killing_, the reframings are different, contingent on the context. This goes on to prove that our model not only

\begin{table}
\begin{tabular}{|p{52.0pt}|p{34.1pt}|p{29.9pt}|c|c|c|} \hline Original Argument & System & reframed Argument & F & M & T\(\uparrow\)/PF \\ \hline \multirow{4}{*}{It is difficult to think of any single act that would do more to restore America’s _soft power_ than the election of Obama to the preslency} & \multirow{4}{*}{BART\({}^{w/o}_{D+EN}\)} & It is difficult to think of any single act that would do more to restore America’s _soft power_ than the election of Obama to the preslency} & \multirow{4}{*}{3.7} & \multirow{4}{*}{2.3} & \multirow{4}{*}{3.3} \\  & & It is difficult to think of any single act that would do more to restore America’s _soft power_ than the election & & & & \\ \cline{2-5}  & \multirow{4}{*}{EXREP} & It is difficult to think of any single act that would do more to restore America’s _modal standing_ than the election of Obama to the preslency} & \multirow{4}{*}{3.7} & \multirow{4}{*}{3.3} & \multirow{4}{*}{2.7} \\ \cline{2-5}  & & It is difficult to think of any single act that would do more to restore America’s _economy_ than the election of Obama to the preslency} & & & & \\ \cline{2-5}  & \multirow{4}{*}{ENTRUST} & It is difficult to think of any single act that would do more to restore America’s _eholomorphic credibility_ than the election of Obama to the preslency} & **4.3** & **3.7** & **5.0** \\ \cline{2-5}  & \multirow{4}{*}{HUMAN} & It is difficult to think of any single act that would do more to restore America’s _poolitical and economic influence_ than the election of Obama to the preslency} & & & & \\ \cline{2-5}  & \multirow{4}{*}{BERT\({}^{w/o}_{D+EN}\)} & Or he can _seize_ this opportunity and use his _government's_ & diplomatic _influence_ to _defend_ the life of an australian citizen 

generalizes to unseen test data, but can produce novel, grammatical and meaningful edits based on context.

## 8 Related Work

The effects of lexical framing have been studied for social and political issues, although our work is the first to use lexical framing in generation for positive framing effects (less partisan, no appeal to fear fallacy). Demszky et al. (2019) and Tyagi et al. (2020) study political polarization and how this manifests in differences in word choice among different groups; KhudaBukhsh et al. (2020) provide an interpretable framework using machine translation between groups to generate differences. While these works encourage computational approaches to reframe arguments for better lexical choice, these approaches do not control for denotation or connotation and thus may cause differences in word choice to result in a change in meaning. The most similar work to ours is that of Pryzant et al. (2020), who use a corpus of Wikipedia edits to train a model for debiasing, which includes framing. However, in their work communicative intent is left implicit; the corpus is only labeled for types of debiasing, which includes framing at a high level and not the connotations involved. Thus, their model only learns lexical differences, whereas our model is controllable.

While our focus is on lexical framing, other work has investigated the identification of other types of frames and their effects. Greene and Resnik (2009) studied _syntactic_ framing, finding a link between implicit sentiment and syntactic packagings. Previous studies have also involved _emphasis_ framing - Ding and Pan (2016) find that emphasizing aspects of products given personal information is more effective for content selection in advertisements. Other research has involved _issue_ framing - Ajour et al. (2019) and Hartmann et al. (2019) study how arguments are framed in debates (e.g., in terms of economics or safety). Nguyen (2013) and Field et al. (2018) study "agenda-setting" for news and congressional debates and August et al. (2018) for study recruitment. Cano-Basave and He (2016) and Musi and Aakhus (2019) leverage _semantic_ frames for distant labeling and analysis of arguments in political debates, respectively, and find, for example, that evidence and reasoning are among the most common. However, these approaches have focused on identification rather than generation.

Finally, our work is also related to style transfer and controllable generation. Much of the work in "style transfer" has referred to changing the sentiment of a statement, which changes the truth condition and thus the denotative meaning. Sentiment is often explicitly marked and thus approaches such as deleting and replacing lexical markers are effective Li et al. (2018b); Sudhakar et al. (2019), although our experiments showed the difficulty of applying these techniques to our task. To control text generation by limiting contradictions, Pasunuru and Bansal (2017) use an entailment score as a reward in Reinforcement Learning, ensuring that a generated text is logically implied by the ground-truth text. Holtzman et al. (2018) utilize a discriminative model trained on SNLI Bowman et al. (2015) to complement an RNN generator and guide the decoding process to improve contradictions in generation. Although we experimented with both of these approaches, including the approach of Holtzman et al. (2018) with MNLI to account for entailment in text generation, none of them yielded better results than our method. Other approaches have explored "vocab boosting" Ghosh et al. (2017) for tasks such as de-biasing Ma et al. (2020), which involves increasing the values of certain words; however, as these values are on the simplex, the softmax function necessarily decreases the values of other logits which are key to fluency such as function words.

## 9 Conclusion

Our experiments showed that our approach is effective in reframing partisan arguments and appeals to fear for increased trustworthiness. We provided a method for creating a dataset using a lexical resource for connotations and masked lan

\begin{table}
\begin{tabular}{l} An Iranian government official seemed to suggest that President Trump’s properties could be potential targets in retaliation for the US _targeted killing_ of Iranian \\ general Qassem Soheimani. \\ \hline An Iranian government official seemed to .... suggest that President Trump’s properties ..... in retaliation for the US _improvement in the execution_ of Iranian ...... \\ \end{tabular} \begin{tabular}{l} An Iéderal appeals court ordered the U.S. Department of Justice to turn over key portions of a memorandum justifying the government’s _targeted killing_ of people \\ linked to terrorism, including Americans \\ \end{tabular} 
\begin{tabular}{l} A federal appeals court ordered the U.S. Department of Justice to turn .......... justifying the government’s _extrapidicial execution_ of people ... terrorism, .......... \\ \end{tabular}
\end{table}
Table 7: Different reframed arguments by ENTRUST model based on same collocation 

guage modeling. We used this dataset to fine-tune a controllable text generation model for the task of changing connotative meaning and used a model trained for natural language inference to maintain the denotative meaning. Our evaluations found that our approach generalized to two different tasks and data sets. In future work, we plan to directly incorporate the role of stance in framing (for arguments and counter-arguments). We also plan to expand our work to generating concessions Musi et al. (2018), where the goal is for the speaker to portray some point of agreement in a positive light before disagreeing.

## 10 Ethics

Our data is collected from Reddit and we understand and respect user privacy. Our models are fine-tuned on sentence level data obtained from user posts. These do not contain any explicit detail which leaks information about a users name, health, negative financial status, racial or ethnic origin, religious or philosophical affiliation or beliefs, sexual orientation, trade union membership, alleged or actual commission of crime.

Second, although we use language models trained on data collected from the Web, which have been shown to have issues with bias and abusive language Sheng et al. (2019); Wallace et al. (2019), the inductive bias of our models should limit indifferent negative impacts. Unlike model variants such as GPT, BART is a conditional language model, which provides more control of the generated output. We have two levels of control on our generation approach: lexical replacements via connotations associated with trust and an entailment method that aims to keep the same denotation of the original argument. While dual-use concerns are certainly possible here, we think that open-sourcing this technology will help to generate arguments with more balanced and trusted language that are less targeted towards partisanship or appeals to fear.

Finally, while there may be concerns about building generative models for persuasion, social scientists distinguish persuasion from manipulation based on two aspects: dissimulation and constraint Nettel and Roque (2012). Dissimulation involves concealing intention, which requires hiding information, whereas constraint involves removing options from the audience and forcing them to accept the conclusion. Our work on reframing arguments does not aim to hide information about a topic or present it as the only choice, but aims to provide the same argument using more balanced and trusted language. We achieve this by two key components of our technology: controllable text generation (connotation associated with trust) and entailment model to ensure same denotation.

The technology should be used responsibly, particularly making sure the generation is controllable for trust and positive emotion and that the entailment component is used for ensuring the same denotation with the original argument.

Finally we pay the Turkers at a rate of \(15\$/\text{hour}\), complying with minimum wage standards in most places.

## References

* Ajjour et al. (2019) Yamen Ajjour, Milad Alshomary, Henning Wachsmuth, and Benno Stein. 2019. Modeling frames in argumentation. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 2922-2932, Hong Kong, China. Association for Computational Linguistics.
* Allaway and McKeown (2020) Emily Allaway and Kathleen McKeown. 2020. A unified feature representation for lexical connotations.
* Aristotle and Bartlett (2019) Aristotle and R. Bartlett. 2019. Aristotle's "art of rhetoric".
* August et al. (2018) Tal August, Nigini Oliveira, Chenhao Tan, Noah A. Smith, and Katharina Reinecke. 2018. Framing effects: Choice of slogans used to advertise online experiments can boost recruitment and lead to sample biases. _PACMHCI_, 2(CSCW):22:1-22:19.
* Bowman et al. (2015) Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015. A large annotated corpus for learning natural language inference. _arXiv preprint arXiv:1508.05326_.
* Cano-Basave and He (2016) Amparo Elizabeth Cano-Basave and Yulan He. 2016. A study of the impact of persuasive argumentation in political debates. In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 1405-1413, San Diego, California. Association for Computational Linguistics.
* Chakrabarty et al. (2019a) Tuhin Chakrabarty, Christopher Hidey, and Kathleen McKeown. 2019a. Imho fine-tuning improves claim detection. _arXiv preprint arXiv:1905.07000_.
* Chakrabarty et al. (2019b) Tuhin Chakrabarty, Cristopher Hidey, Smaranda Muresan, Kathleen McKeown, and Alyssa Hwang. 2019b. Ampersand: Argument mining for persuasive online discussions. In _Proceedings of the 

2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 2926-2936. Cited by: SS4.
* D. Lee Chong and J. N. Druckman (2007)Framing theory. External Links: Link Cited by: SS5.
* G. Da San Martino, S. Yu, A. Barron-Cedeno, R. Petrov, and P. Nakov (2019)Fine-grained analysis of propaganda in news article. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 5640-5650. Cited by: SS1.
* D. Demszky, N. Garg, R. Voigt, J. Zou, M. Gentzkow, J. Shapiro, and D. Jurafsky (2019)Analyzing polarization in social media: method and application to tweets on 21 mass shootings. arXiv preprint arXiv:1904.01596. Cited by: SS1.
* J. Devlin, M. Chang, K. Lee, and K. Toutanova (2018)Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. Cited by: SS1.
* T. Ding and S. Pan (2016)Personalized emphasis framing for persuasive message generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, pp. 1432-1441. Cited by: SS1.
* R. Entman (1983)Framing: toward clarification of a fractured paradigm. Journal of Communications. External Links: Document Cited by: SS1.
* A. Fiedl, D. Kliger, S. Wintner, J. Pan, D. Jurafsky, and Y. Tsvetkov (2018)Framing and agenda-setting in russian news: a computational analysis of intricate political strategies. arXiv preprint arXiv:1808.09386. Cited by: SS1.
* G. Frege (1892)Uber sinn und bedeutung.. Zeitschrift fur Philosophie und philosophische Krtik_, 100:25-50. Cited by: SS1.
* Matthew Gentzkow, J. M. Shapiro, and M. Taddy (2019)Measuring group differences in high-dimensional choices: method and application to congressional speech. Econometrica87 (4), pp. 1307-1340. Cited by: SS1.
* S. Ghosh, M. Chollet, E. Laksana, L. Moreny, and S. Scherer (2017)Affect-LM: a neural language model for customizable affective text generation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, Canada, pp. 634-642. External Links: ISBN 978-3-319-1475-5, Link, Document Cited by: SS1.
* S. Greene and P. Resnik (2009)More than words: syntactic packaging and implicit sentiment. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Boulder, Colorado, pp. 503-511. External Links: ISBN 978-3-319-1475-5, Link, Document Cited by: SS1.
* F. Hartmann (2019)Predicting historical phonetic features using deep neural networks: a case study of the phonetic system of Proto-Indo-European. In Proceedings of the 1st International Workshop on Computational Approaches to Historical Language Change, Florence, Italy, pp. 98-108. External Links: ISBN 978-3-319-1475-5, Link, Document Cited by: SS1.
* M. Hartmann, T. Jansen, I. Augenstein, and A. Sogaard (2019)Issue framing in online discussion fora. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 1401-1407. External Links: ISBN 978-3-319-1475-5, Link, Document Cited by: SS1.
* C. Hidey, E. Musi, A. Hwang, S. Muresan, and K. McKeown (2017)Analyzing the semantic types of claims and premises in an online persuasive forum. In Proceedings of the 4th Workshop on Argument Mining, pp. 11-21. Cited by: SS1.
* A. Holtzman, J. Buys, M. Forbes, A. Bosselut, D. Golub, and Y. Choi (2018)Learning to write with cooperative discriminators. arXiv preprint arXiv:1805.06087. Cited by: SS1.
* N. S. Keskar, B. McCann, L. R. Varshney, C. Xiong, and R. Socher (2019)Crtl: a conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858. Cited by: SS1.
* A. R. KhudaBukhsh, R. Sarkar, M. S. Kamlet, and T. M. Mitchell (2020)We don't speak the same language: interpreting polarization through machine translation. arXiv preprint arXiv:2010.02339. Cited by: SS1.
* J. Kiesel, M. Mestre, R. Shukla, E. Vincent, P. Adineh, D. Corney, B. Stein, and M. Potthast (2019)SemEval-2019 task 4: hyperpartisan news detection. In Proceedings of the 13th International Workshop on Semantic Evaluation, Minneapolis, USA, pp. 829-839. External Links: ISBN 978-3-319-1475-5, Link, Document Cited by: SS1.
* M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer (2019)BAT: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461. Cited by: SS1.
* J. Li, R. Jia, H. He, and P. Liang (2018)Delete, retrieve, generate: a simple approach 

to sentiment and style transfer. _arXiv preprint arXiv:1804.06437_.
* Li et al. (2018b) Juncen Li, Robin Jia, He He, and Percy Liang. 2018b. Delete, retrieve, generate: a simple approach to sentiment and style transfer. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long Papers)_, pages 1865-1874, New Orleans, Louisiana. Association for Computational Linguistics.
* Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. _arXiv preprint arXiv:1907.11692_.
* Ma et al. (2020) Xinyao Ma, Maarten Sap, Hannah Rashkin, and Yejin Choi. 2020. Powertransformer: Unsupervised controllable revision for biased language correction. _arXiv preprint arXiv:2010.13816_.
* Mccombs and Ghanem (2001) M. Mccombs and Salma I. Ghanem. 2001. The convergence of agenda setting and framing.
* Mohammad and Turney (2013) Saif M Mohammad and Peter D Turney. 2013. Crowdsourcing a word-emotion association lexicon. _Computational Intelligence_, 29(3):436-465.

* Musi and Aakhus (2019) Elena Musi and Mark Aakhus. 2019. Framing fracking: Semantic frames as meta-argumentative indicators for knowledge-driven argument mining of controversies.
* Musi et al. (2018) Elena Musi, Debanjan Ghosh, and Smaranda Muresan. 2018. Changymview through concessions: Do concessions increase persuasion? _arXiv preprint arXiv:1806.03223_.
* Nettel and Roque (2012) Ana Laura Nettel and Georges Roque. 2012. Persuasive argumentation versus manipulation. _Argumentation_, 26(1):55-69.
* Nguyen (2013) Viet-An Nguyen. 2013. Lexical and hierarchical topic regression. In _NIPS 2013_.
* Novikova et al. (2017) Jekaterina Novikova, Ondrej Dusek, Amanda Cercas Curry, and Verena Rieser. 2017. Why we need new evaluation metrics for NLG. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_, pages 2241-2252, Copenhagen, Denmark. Association for Computational Linguistics.
* Ott et al. (2019) Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. _arXiv preprint arXiv:1904.01038_.
* Pasunuru and Bansal (2017) Ramakanth Pasunuru and Mohit Bansal. 2017. Reinforced video captioning with entailment rewards. _arXiv preprint arXiv:1708.02300_.
* Pryzant et al. (2020) Reid Pryzant, Richard Diehl Martinez, Nathan Dass, Sadao Kurohashi, Dan Jurafsky, and Diyi Yang. 2020. Automatically neutralizing subjective bias in text. In _Proceedings of the AAAI Conference on Artificial Intelligence_, volume 34, pages 480-489.
* Rasinski (1989) K. Rasinski. 1989. The effect of question wording on public support for government spending. _Public Opinion Quarterly_, 53:388-394.
* Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019. Sentencebert: Sentence embeddings using siamese bertnetworks. _arXiv preprint arXiv:1908.10084_.
* Schiller et al. (2020) Benjamin Schiller, Johannes Daxenberger, and Iryna Gurevych. 2020. Aspect-controlled neural argument generation. _arXiv preprint arXiv:2005.00084_.
* Sheng et al. (2019) Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019. The woman worked as a babysitter: On biases in language generation. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 3407-3412, Hong Kong, China. Association for Computational Linguistics.
* Sudhakar et al. (2019a) Akhilesh Sudhakar, Bhargav Upadhyay, and Arjun Mashewaran. 2019a. Transforming delete, retriere, generate approach for controlled text style transfer. _arXiv preprint arXiv:1908.09368_.
* Sudhakar et al. (2019b) Akhilesh Sudhakar, Bhargav Upadhyay, and Arjun Mashewaran. 2019b. "transforming" delete, retrieve, generate approach for controlled text style transfer. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 3269-3279, Hong Kong, China. Association for Computational Linguistics.
* Tan et al. (2016) Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2016. Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions. In _Proceedings of the 25th international conference on world wide web_, pages 613-624.
* Thierer (2012) Adam Thierer. 2012. Technopanics, threat inflation, and the danger of an information technology precautionary principle. _IO: Productivity_.
* Tyagi et al. (2020)Aman Tyagi, Anjalie Field, Priyamle Lathwal, Yulia Tsvetkov, and Kathleen M Carley. 2020. A computational analysis of polarization on indian and pakistani social media. _arXiv preprint arXiv:2005.09803_ 

Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal adversarial triggers for attacking and analyzing NLP. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 2153-2162, Hong Kong, China. Association for Computational Linguistics.
* Walton [2006] D. Walton. 2006. Fundamentals of critical argumentation. In _Critical reasoning and argumentation_.
* Webson et al. [2020] Albert Webson, Zhizhong Chen, Carsten Eickhoff, and Ellie Pavlick. 2020. Do "Undocumented Workers" == "Illegal Aliens"? Differentiating Denotation and Connotation in Vector Spaces. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 4090-4105, Online. Association for Computational Linguistics.
* Wei et al. [2016] Zhongyu Wei, Yang Liu, and Yi Li. 2016. Is this post persuasive? ranking argumentative comments in online forum. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, pages 195-200.
* Williams et al. [2018] Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_, pages 1112-1122. Association for Computational Linguistics.

## Appendix A Hyper-Parameters and Other Experimental Settings

1. **No of Parameters:** For Connotation guided MLM and Entailment we use RoBERTa large model (355M). For generation we use the BART large checkpoint (400M parameters) and use the implementation by FAIRSEQ Ott et al. [2019]2. Footnote 2: [https://github.com/pytorch/fairseq/tree/master/examples/bart](https://github.com/pytorch/fairseq/tree/master/examples/bart)
2. **No of Epochs:** We fine-tune pre-trained BART for 20 epochs for ENTRUST model and save best model based on validation perplexity.
3. **Training Time:** Our training time is 80 minutes for BART.
4. **Hardware Configuration:** We use 4 RTX 2080 GPU
5. **Training Hyper parameters:** We use the same parameters mentioned in the github repo where BART was fine-tuned for CNN-DM summarization task with the exception of MAX-TOKENS (size of each mini-batch, in terms of the number of tokens.) being 1024 for us. For entailment we rely on AllenNLP roberta checkpoint that is finetuned on MNLI 3 Footnote 3: url[https://demo.allennlp.org/textual-entailment](https://demo.allennlp.org/textual-entailment)
6. **Decoding Strategy & Hyper Parameters:**For decoding we reframe from our models using a top-k random sampling scheme Fan et al. [2018]. At each timestep, the model generates the probability of each word in the vocabulary being the likely next word. We randomly sample from the k = 5 to k=50 for most likely candidates from this distribution and re-rank them based on entailment scores.

### Role of Context in re-framing

As mentioned ENTRUST FFarnings are not static. Table 8 shows that for multiple same collocations the refinramings are different, contingent on the context. This supports our claim that our model not only generalizes on unseen test data, but can produce novel, grammatical and meaningful edits based on context. We notice that for the first example, ENTRUST believes scientific proof leads to be better credibility instead of using the broad umbrella term experts. For the same collocation of _leading expert_ we see that given the premise talks about a neurological basis related to birth , ENTRUST FFrames it to reflect to _leading medical fact_ as the term "expert" can be ambiguous, while people are generally more likely to believe medical facts.

 



\begin{table}
\begin{tabular}{|p{284.5pt}|} \hline Statistically based-wise though, the number of violent crime is more prevalent in warm areas,_leading experts_ to believe a correlation between warm and people being more aggressive. \\ \hline Statistically based-wise though, the number of violent crime is more prevalent in warm areas,_leading scientific proof_ to believe a correlation between warm and people being more aggressive. \\ \hline \hline The _leading experts_ all agree that there’s a neurological basis \& there is significant evidence showing it’s caused during fetal neurological development \\ \hline The _leading medical facts_ all agree that there’s a neurological basis \& there is significant evidence showing it’s caused during fetal neurological development \\ \hline We need leaders who recognize our _moral obligation_ for health equality \\ \hline We need leaders who recognize our _moral accountability_ for health equality \\ \hline We all have a _moral obligation_ to the next generation to leave America’s natural resources in better condition than when we inherited them \\ \hline We all have a _personal oath_ to the next generation to leave America’s natural resources in better condition than when we inherited them \\ \hline \end{tabular}
\end{table}
Table 8: Different reframed arguments by ENTRUST model based on same collocation



\begin{table}
\begin{tabular}{|p{108.4pt}|l|c|c|c|} \hline Original Argument & System & reframed Argument & F & M & T\(\uparrow\)/PF\(\downarrow\) \\ \cline{2-5} AIPAC called on the Obama administration to take steps to dofuse tension with Israel, while Israel’s ambassador to the defuse tension with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel, while Israel’s ambassador to the dwelling station with Israel and the dwelling station with Israel, while Israel’s amb