Venue: NAACL 2022

# Explaining Dialogue Evaluation Metrics using Adversarial Behavioral Analysis

Baber Khalid

Rutgers University

110 Frelinghuysen Rd

Piscataway, NJ 08854

baber.khalid@rutgers.edu Sungjin Lee

Amazon Alexa

300 Pine Street

Seattle, WA 98103

sungjinl@amazon.com

###### Abstract

There is an increasing trend in using neural methods for dialogue model evaluation. Lack of a framework to investigate these metrics can cause dialogue models to reflect their biases and cause unforeseen problems during interactions. In this work, we propose an adversarial test-suite which generates problematic variations of various dialogue aspects, _e.g._ logical entailment, using automatic heuristics. We show that dialogue metrics for both open-domain and task-oriented settings are biased in their assessments of different conversation behaviors and fail to properly penalize problematic conversations, by analyzing their assessments of these problematic examples. We conclude that variability in training methodologies and data-induced biases are some of the main causes of these problems. We also conduct an investigation into the metric behaviors using a black-box interpretability model which corroborates our findings and provides evidence that metrics pay attention to the problematic conversational constructs signaling a misunderstanding of different conversation semantics.

+
Footnote †: _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 5871 - 5883


technique, a modification of Kernel SHAP from Lundberg and Lee (2017), to find important language features for the metrics to provide additional insights into the behavior of evaluation metrics. This test-suite is meant to provide a novel benchmark for community which can be used to analyze the performance of proposed metrics and can be improved with further research.

## 2 Related Work

### Language Model Evaluation Test-Suites

There have been several test-suites which compare the performance of the language models on contrasting examples. Both Gauthier et al. (2020); Warstadt et al. (2020) compute _surprisal_ for sentence pairs, where one of the pair has a syntactical mistake, to test if a language model finds the syntactically wrong sentence more _surprising_. Beyer et al. (2021a); Pishdad et al. (2020) both rely on this concept of calculating surprisal but use pairs of coherent and in-coherent language uses. However, they don't release heuristics for automatic generation of adversarial cases and just focus on coherence-based manipulations while we go one step further to see the effect of other manipulations on various core aspects of dialogue in an automated way. Ribeiro et al. (2020) presents a tool which evaluates language models with their performance on pre-determined tests and their outcomes.

### Adversarial Evaluation Techniques

Previous works have tried to use adversarial evaluations to judge the performance of dialogue models. Cheng et al. (2019) successfully trains a RL agent using adversarial rewards against a negotiation dialogue agent and reduces its effectiveness. Jia and Liang (2017) proposes an adversarial attack where adding extra sentences in the comprehension text reduces the performance of comprehension models significantly. The closest work to ours is Sai et al. (2019) which evaluates a neural metric Lowe et al. (2017). It shows effectiveness of simple syntactical manipulations, like reversing a sentence, in fooling the metric. We, however, rely on attacking more complex semantics using simple heuristics, like breaking co-reference chains, to pinpoint metric performance fluctuations.

### Interpretability Techniques

There have been several works proposing algorithms to explain the predictions of neural networks. A measure of co-operative game theory used to measure marginal contribution of players in a game is called Shapley value Shapley (1952). It has been a focus of attention for ML community to explain predictions of neural models. LIME Ribeiro et al. (2016) and trumbelj and Kononenko (2013) present methods which rely on feature perturbations to generate explanations. Bach et al. (2015) presents a method which provides explanation in form of pixel-based heatmaps. Lundberg and Lee (2017) modifies the methods mentioned earlier to approximate Shapley values. Li et al. (2017) presents the variations in model outputs by erasing several input features as a measure of importance. We use a modification of Kernel SHAP from Li et al. (2017) to approximate feature importance in this work.

## 3 Conversation Properties of Interest

Dialogue systems- specially those based on neural architectures, show poor performance in generating consistent responses by keeping track of information in a dialogue context and are prone to generating repetitive, unnatural and bland responses Yang et al. (2021); Khandelwal et al. (2018); Gao et al. (2018). These problems also guide the research in the area of dialogue system evaluation with many metrics focusing on evaluating coherence and consistency of dialogue models Lai and Tetreault (2018); Beyer et al. (2021b); Gao et al. (2020); Pang et al. (2020); Sun et al. (2021). These findings help us choose relevant dialogue attributes to manipulate so the insights from the performance of different metrics can be used to address different problems in dialogue modeling.

To generate adversarial examples we focus on the following aspects of conversations: 1) _coherence_ 2) _naturalness_ 3) _interestingness_. We manipulate each of these attributes by targeting specific aspects which contribute to these: **Coherence:** i) _entailment_ ii) _pronoun use_ iii) _named-entity use_ iv) _co-reference_ v) _contradictions_ vi) _speaker sensitiveness_ **Naturalness:** vii) _unnatural repetitions_ viii) _vocabulary diversity_ ix) _unnatural paraphrasing_ x) _natural paraphrasing_ xi) _entrainment_ **Interestingness:**xii) _dullness_. One example for manipulation of each of the major attributes is shown in the table 2 and other examples are presented in the appendix 



## 4 Adversarial Techniques

We specifically focus on four conversational datasets _Daily Dialogue (DDial)_Li et al. (2017), _Persona Chat (PerCh)_Zhang et al. (2018), _Reddit_Gao et al. (2020) and _MultiWOz_Budzinowski et al. (2018) to test our heuristics. We test an implementation of our heuristics using these datasets and release it with this work1.

Footnote 1: [https://github.com/baber-soos/Explaining-Dialogue-Evaluation-Metrics-using-Adversarial-Behavioral-Analysis](https://github.com/baber-soos/Explaining-Dialogue-Evaluation-Metrics-using-Adversarial-Behavioral-Analysis)

### Problem Definition

In accordance with the given datasets, we consider a conversation \(D\) as a series of utterances between two alternating speakers \((x_{1},y_{1},x_{2},y_{2},...,x_{n},y_{n})\). We formulate the adversarial conversation generation problem as follows: Given a conversation snippet \(C_{i}=(x_{1},y_{1},...,x_{i},y_{i},x_{i+1})\), from a randomly sampled conversation \(D=(x_{1},y_{1},...,x_{n},y_{n})\), we generate an adversarial conversation snippet \(C^{\prime}_{i}=(x_{1},y_{1},...,x_{i},y_{i},x^{\prime}_{i+1})\). This helps us learn the impact of different responses to the same context on neural conversational metrics. \(x^{\prime}_{i+1}\) is generated using adversarial heuristics which are explained as follows:

### Coherence

The purpose of these attacks is to disturb the logical flow of human conversations and investigate how different evaluation metrics react to these disturbances.

#### 4.2.1 Entailment

In this case, the adversarial response \(x^{\prime}_{i+1}\) is either a randomly sampled utterance from a random conversation in the dataset or a random response from the following: \((y_{i+1},x_{i+2},...,x_{n},y_{n})\) from \(D\). (See Appendix A for example.)

#### 4.2.2 Pronoun Use

Given \(x_{i+1}=(w_{1},w_{2},...,w_{k})\), where \(w_{j}\) is an utterance token, we manipulate \(w_{j}\subset P\), where \(P\) is the set of pronouns, in the following manner: 1) We convert gender specific pronouns to object pronouns e.g. he/she to it/they and vice versa. 2) We convert singular pronouns _it/this/I_ to the plural ones and vice versa. 3) We convert 1st person pronouns like _Iwe/our_ to a random 2nd/3rd person pronoun or vice versa. (See Appendix A for example.)

#### 4.2.3 Named Entities

We replace named entities in \(x_{i+1}\) with 1) another random entity of same type _e.g._ person with person 2) a named entity of different type _e.g._ person with company to generate \(x^{\prime}_{i+1}\). (See Appendix A for example.)

#### 4.2.4 Co-Reference

Human conversations have extensive use of references to different entities being discussed. We use a pre-trained reference resolution model to detect co-reference clusters in the conversation snippet and replace references in \(x_{i+1}\) to generate \(x^{\prime}_{i+1}\). This results in adversarial \(x^{\prime}_{i+1}\) which means the same thing as \(x_{i+1}\) and should be assigned a similar score by an ideal metric. (See Appendix A for example.)

#### 4.2.5 Speaker Sensitiveness

Given a conversation snippet \(C\) which ends at a response \(x_{i+1}\) to a question-answer pair \((x_{i},y_{i})\), we augment the response \(x_{i+1}\) by concatenating it to the answer \(y_{i}\) to the question \(x_{i}\) to generate the adversarial \(x^{\prime}_{i+1}=[y_{i},x_{i+1}]\). This creates situations which contain unnatural redundancies because a speaker does not acknowledge the answer

\begin{table}
\begin{tabular}{|p{96.7pt}|p{142.3pt}|p{142.3pt}|} \hline Coherence & Naturalness & Interestingness \\ \hline Contraction & Unnatural Paraphrasing & Duliness \\ \hline A: Could you explain what you saw? **B**: I was in the bank at the time of the robbercy. **A**: What did you see? **B**: I saw a man come in with a gun. **A**: Did you see his face? **Actual**: No . He had a mask on. **Actual**: Let me buy you lunch today to pay you back. & **A: What are you doing tonight? **B**: I hav to run to the grocery store. **A**: Don’t you haate fighting the crowds on the weekends? **Actual**: Yes, but I am out of food and milk. \\ \hline
**Adversarial**: 1 did not see a man come in with a gun. & **Adversarial**: I don’t have a good answer to that. \\  & luncheon today to give you back. & **Adversarial**: allow me purchase you luncheon today to give you back. \\ \hline \end{tabular}
\end{table}
Table 2: This table showcases one adversarial example for each dialogue attribute we target: coherence, naturalness and interestingness.



to a given question. A simple and effective heuristic to detect question answer pairs is to find an utterance with the question-mark ?. (See Appendix A for example.)

#### 4.2.6 Contradictions

Here we focus on generating adversarial \(x^{\prime}_{i+1}\) which contradicts the contextual information in a given conversation \(C\). We generate \(x^{\prime}_{i+1}\) by augmenting \(x_{i+1}\) in the following manner: 1) by negating the verbs in \(x_{i+1}\) using simple heuristics, 2) by replacing named entities in \(x_{i+1}\) occurring at least twice in \(C\) with a random entity of same type or 3) by replacing references in \(x_{i+1}\) to different named entities with the random entities of same type. For the verb negation, we either add a _not_ after an auxiliary or add _did not_ and _does not_ depending on plurality of the subject and tense of the sentence while changing the verb form if needed. (See the table 2 for example.)

### Naturalness

Humans prefer some vocabulary or style choices over others and a good evaluation metric should respect this behavior. We disturb these natural choices in the following manner:

#### 4.3.1 Unnatural Repetitions

We augment \(x_{i+1}\) to generate \(x^{\prime}_{i+1}\) by randomly sampling and repeating \(k\) times 1) non-stop English words in \(x_{i+1}\) or 2) multi-word noun phrases to generate unnatural repetitions in the adversarial response. (See Appendix A for example.)

#### 4.3.2 Vocabulary Diversity

We replace randomly sampled words \(w_{j}\), which have a synonym in the conversation \(C\), from \(x_{i+1}\) with one of their synonyms in \(C\). This results in \(x^{\prime}_{i+1}\) with vocabulary choices which may not be as natural. An ideal metric would assign these examples a score no better than their human generated counter-parts. As a simple heuristic, we consider two words \((w_{j},w_{k})\) as synonyms if they are in synset of each other as specified in the Wordnet[50]. (See Appendix A for example.)

#### 4.3.3 Unnatural Paraphrasing

We generate unnatural paraphrase \(x^{\prime}_{i+1}\) by replacing randomly sampled non-stop English words from \(x_{i+1}\) with their synonyms from Wordnet. For synonym sampling, we rank the Wordnet synonyms according to word2vec embedding [79] similarity and sample from the top \(k\) synonyms. To make paraphrasing unnatural, \(1/4^{th}\) of words are replaced with synonyms which are least likely to be used by humans while keeping it grammatically correct. (See the table 2 for example.)

#### 4.3.4 Natural Paraphrasing

We generate natural paraphrases using a T5 model fine-tuned on the _PAWS_[88] dataset. T5 model makes minor structural changes to \(x_{i+1}\) like removing punctuation or re-organizing utterances while using same vocabulary. An ideal metric should assign both \(x_{i+1}\) and \(x^{\prime}_{i+1}\) similar scores. (See Appendix A for example.)

#### 4.3.5 Entrainment

To disturb entrainment, we replace non-stop English words \((w_{j}\in x_{i+1})\) used by both parties involved in the conversation with a synonym sampled from the top \(k\) synonyms in the Wordnet. (See Appendix A for example.)

### Interestingness

Interestingness is a subjective measure but humans prefer interesting conversations over the dull ones. We consider those responses _interesting_ which progress the conversation in a natural way than those which do not add anything meaningful to it.

#### 4.4.1 Dullness

To generate a dull \(x^{\prime}_{i+1}\) we either 1) replace \(x_{i+1}\) with an utterance from a set of generic responses, 2) replace an answer in a QA pair with a generic answer or 3) replace \(x_{i+1}\) with one of following: \((x_{1},...,x_{i})\) which results in the repetition of a speaker contribution. (See the table 2 for example.)

## 5 Experiment Results and Analysis

We use three different evaluation metrics _DialogRPT_[10], _Pang-Evaluation_[11] and _User Satisfaction_[12] to test our adversarial test-suite. The first two metrics we test are for open-domain conversations while the last one is trained to judge task-oriented conversations. We focus on both open-domain dialogue and task-oriented dialogue metrics to highlight the diversity of our test-suite and show how neural metrics in both of these settings contain fundamental problems. Both DialogRPT and User Satisfaction metrics use human feedback 

during the training while components of Pang-Evaluation rely on pre-trained neural backbones. We postulate that this variability in training methods might help provide specific insights for different methods.

From our analysis of metric behavior on adversarial conversations, we have the following key takeaways: _1)_ neural metrics are not suited to judge overall quality of the conversations; _2)_ they are unable to correctly understand conversation semantics; and 3) they are prone to data and training-induced biases.

### Noise in Adversarial Heuristics

Proposed heuristics are expected to fail in some cases while generating adversarial conversations. To estimate this failure, we manually analyze the 250 generated conversations across all attacks and find that heuristics work successfully _84%_ of the time. This signifies that heuristics are succeeding most of the time but requires further investigation for a more detailed estimate of their noise.

### Analysis Methodology

An ideal metric would grade adversarial conversations worse than the original conversations in most cases, as most of the attacks are focused on generating bad behaviors. However, natural paraphrasing and attacks on reference use are expected to be graded similarly to the original conversations as they generate similar conversations as the original ones. Similarly, conversations generated by vocabulary diversity and entertainment attacks are expected to be scored _at most_ as well as the original conversations as both of these manipulate vocabulary choices and sometimes these manipulations result in natural outcomes. To capture this variability, we define _error rate_ as the proportion of times a conversation metric does not conform to the expected behavior.

We sample 100 adversarial conversations for each attribute per dataset and analyze the metric performance on those. We compute the proportion of times the metrics rate original conversations higher, equal and lower than the adversarial ones and use these statistics to compute the error rates. To make sure, we don't categorize insignificant

\begin{table}
\begin{tabular}{c c c c c c c} \multirow{2}{*}{**Metrics**} & \multirow{2}{*}{**Datasets**} & \multicolumn{1}{l}{**Unnatural**} & \multicolumn{1}{l}{**Vocabulary**} & \multicolumn{1}{l}{**Unnatural**} & \multicolumn{1}{l}{**Natural**} & \multicolumn{1}{l}{**Entrainment**} \\  & & **Repetitions** & **Diversity** & **Paraphrases** & **Paraphrases** & & \\ \hline \multirow{3}{*}{**DiaglogRPT**} & Dbial & 0.86 & 0.14 & 0.42 & 0.44 & 0.62 \\  & Reddit & 0.90 & 0.08 & 0.42 & 0.35 & 0.59 \\  & PerCh & 0.91 & 0.08 & 0.53 & 0.41 & 0.67 \\ \hline \multirow{3}{*}{**Pang-C**} & Dbial & 0.02 & 0.00 & 0.00 & 0.87 & 0.46 \\  & Reddit & 0.05 & 0.02 & 0.15 & 0.33 & 0.59 \\  & PerCh & 0.01 & 0.06 & 0.00 & 0.89 & 0.23 \\ \hline \multirow{3}{*}{**Pang-F**} & Dbial & 0.99 & 0.88 & 1.00 & 0.97 & 1.00 \\  & Reddit & 0.80 & 0.64 & 0.88 & 0.44 & 0.93 \\ \cline{1-1}  & PerCh & 0.87 & 0.83 & 0.98 & 0.87 & 0.94 \\ \hline
**UWBERT** & & 1 & 0.01 & 0.98 & 0.04 & 1 \\
**WHiGRU** & MultiWOz & 0.96 & 0.02 & 0.98 & 0.1 & 0.95 \\
**WBERT** & & 0.99 & 0 & 0.98 & 0.03 & 1 \\ \end{tabular}
\end{table}
Table 4: Error rates of different evaluation metrics on different datasets for naturalness attacks.

\begin{table}
\begin{tabular}{c l l r r r r r r} \multirow{2}{*}{**Metrics**} & \multirow{2}{*}{**Datasets**} & \multirow{2}{*}{**Entailment Pronoun**} & \multirow{2}{*}{**NER**} & \multicolumn{1}{l}{**Co-reference**} & \multicolumn{1}{l}{**Speaker**} & \multicolumn{1}{l}{**Contradic-**} & \multicolumn{1}{l}{**Dullness**} \\  & & & & & & **ness** & & \\ \hline \multirow{3}{*}{**DiaglogRPT**} & DDial & 0.3 & 0.48 & 0.6 & 0.56 & 0.98 & 0.78 & 0.48 \\  & Reddit & 0.13 & 0.66 & 0.64 & 0.32 & 0.91 & 0.72 & 0.53 \\  & PerCh & 0.4 & 0.55 & 0.74 & 0.45 & 0.89 & 0.87 & 0.47 \\ \hline \multirow{3}{*}{**Pang-C**} & DDial & 0.46 & 0.08 & 0.49 & 0.03 & 0.01 & 0.26 & 0.78 \\  & Reddit & 0.5 & 0.49 & 0.41 & 0.58 & 0.04 & 0.47 & 0.9 \\  & PerCh & 0.47 & 0.2 & 0.53 & 0.74 & 0.00 & 0.51 & 0.86 \\ \hline \multirow{3}{*}{**Pang-L**} & DDial & 0.45 & 0.75 & 0.72 & 0.23 & 0.74 & 0.32 & 0.67 \\  & Reddit & 0.49 & 0.63 & 0.57 & 0.35 & 0.59 & 0.29 & 0.81 \\  & PerCh & 0.54 & 0.76 & 0.73 & 0.35 & 0.69 & 0.33 & 0.81 \\ \hline
**UWBERT** & & 0.98 & 1.00 & 1.00 & 0.01 & 1.00 & 0.97 & 0.98 \\
**WHIGRU** & MultiWOz & 0.93 & 0.98 & 0.94 & 0.07 & 0.96 & 0.96 & 0.95 \\
**WBERT** & & 0.98 & 0.97 & 0.97 & 0.01 & 0.96 & 0.98 & 0.94 \\ \end{tabular}
\end{table}
Table 3: Error rates of different evaluation metrics on different datasets for coherence and interestingness attacks.

 

changes as greater or lesser we compute a minimum score threshold by computing the minimum change from the mean of metrics scores assigned to human conversations required to get a p-value less than _0.05_\((p<0.05)\) in a t-test. We use pre-trained NER model in the space python package Honnibal and Montani (2017) for named-entity detection and reference resolution models in Clark and Manning (2016, 2018) to detect co-reference clusters.

### DialogRPT

This is an evaluation metric which was trained on Reddit threads using human feedback in the form of up/down votes, number of replies, and the depth of a conversation thread. It also has a component focused on separating human from random utterances. DialogRPT requires significant improvements when dealing with the most adversarial cases as shown by error rates calculated in tables 3 and 4. It performs the best on the entailment task on Reddit data because it was trained to pick relative human responses from random ones. Additionally, it performs better on Reddit data on average than other datasets which provides evidence for better performance on in-distribution data.

It is evident that DialogRPT favors repetitions when we analyze adversarial cases which are generated using repetitions e.g. speaker sensitive cases in table-3 and unnatural repetitions cases in table-4. DialogRPT has a 93% error rate out of which 17% of the adversarial conversations are rated similar to the original for the speaker sensitivity. This proportion of similar ratings increases to \(57\%\) in case of unnatural repetitions. The fact that conversations in speaker sensitive case are rated higher more often provides evidence for a bias towards rating conversations with similar responses to context.

The performance of DialogRPT on attacks other than entailment is not good with error rates greater than \(40\%\), which provides evidence that DialogRPT does not have a correct understanding of different dialogue aspects especially contradictions and individual speaker contributions.

### Pang-Evaluation Metric

Pang-Evaluation presents four evaluation metrics which try to evaluate specific properties of dialogues. 3 of these depend on a pre-trained neural backbone: i) context-coherence (Pang-C) ii) fluency (Pang-F) iii) logical consistency (Pang-L). Pang-C and Pang-F rely on a GPT2 Radford et al. (2018) backbone fine-tuned on Daily Dialogue dataset while the Pang-L relies on a Roberta model Liu et al. (2019) fine-tuned on MNLI task Williams et al. (2018) to detect contradictions.

We test Pang-C on all of the adversarial cases while we test the Pang-L metric on coherence and interestingness attacks and Pang-F metric on the naturalness because Pang-C is compared with other overall evaluation metrics in the paper while Pang-F and Pang-L are presented to judge fluency and logical consistency of a dialogue response. Pang-C seems to be robust to attacks which induce unnatural sentence structure. This is evident by looking at the results for 1) pronoun, speaker sensitiveness and co-reference attacks in the table 3; and 2) unnatural repetitions, paraphrasing attacks and vocabulary diversity attacks in the table 4. This highlights again that metrics perform best on the tasks they are designed for. Since GPT2 predicts the likelihood of next token given history, the metric is sensitive to unnatural manipulations. However, its performance varies across other attacks e.g. the metric fails to reliably penalize contradictions or entailments. Similar to DialogRPT, Pang-C also performs worse on out-of-distribution data as visible from the performance drop on datasets other than Dialy Dialogue. GPT2 could be fine-tuned on datasets to retain performance, but not every dataset has a large number of dialogues and fine-tuning large neural models is not a trivial task. This highlights that pre-trained models like GPT2 may retain performance for attacks which generate unnatural examples but still show a significant room for improvement when dealing with complex conversation semantics or out-of-distribution data.

Pang-C metric performs the best out of the three metrics presented by the Pang-Evaluation. Their Pang-F metric, results shown in the table 4, differs from Pang-C metric because it does not take context into account while assigning scores to the utterance under consideration. However, this causes it to under-perform in comparison to Pang-C metric while judging the adversarial cases. This also highlights a weakness in the training methodology itself. Since the metric has only seen full conversations during training, it fails to reliably penalize adversarial vocabulary choices when it evaluates individual utterances.

Pang-L metric is proposed to score the logical consistency of the conversations. The metric performs better on average than Pang-C at detecting contradictions (the task it was trained for), compa 

rable while detecting broken entailments but worse in other cases as visible from the table 3. Its performance is also consistent across datasets most-likely because it was trained on the MNLI task and not on any of the conversation datasets it was being evaluated on. This again highlights the limited applicability of the metric in-line with the other metric we have examined.

### User Satisfaction Metric

User satisfaction simulation metric is trained using human-feedback ratings to predict a score from 1-5, given a conversation. The authors pose this task as a classification problem and train a classifier to predict a score. One of the major problems with this metric, lies in the data it was trained on. Since the data they use has an highly skewed rating distribution, the metric performance reflects this. We test the classifier version of the metric using fine-tuned BERT model (UWBERT). This version assigned the same score to almost all of the adversarial and original conversations. We also computed the score as the weighted average of ratings, using both their hierarchical GRU (WHiGRU) and BERT (WBERT) models, to test if minor variation in assigned scores could provide some insights but the results remained the same. As shown in the tables 3 and 4, this bias in the data results in high error rates closer to \(1.0\). When error rate relies on scores being similar e.g. in case of co-reference attack, it drops to \(~{}0.0\) because the model assessment never changes. This highlights a case of bias in model assessments because of the problems in training data.

### Performance Highlights and Conclusion

All metrics that we test perform better on the data they were trained on with the best performance on the trained down-stream task(s). However, none of them are suitable to judge the overall quality of conversations. To be more specific the results show that variability in training methods can cause metrics to assess conversation behaviors differently e.g. DialogRPT vs Pang-C metric performance, metrics are dependent on the training data and reflect the biases in the data and training methodologies e.g. performance of User Satisfaction metric. These insights help us draw two conclusions: 1) a series of metrics specific to individual dialogue behaviors might judge overall quality better than a single evaluation metric and 2) failure of metrics to reliably judge complex behaviors like contradictions indicate that either metrics should be trained using examples of these behaviors or should be exposed to more information about conversation flow than just surface textual features.

## 6 Interpretability Analysis

### Method

In addition to the behavioral analysis described above, we use a black-box model interpretability technique to approximate the feature importance for different adversarial cases for the DialogRPT metric. This shows the relative importance of adversarial features as the metrics paying more importance to adversarial features to assign higher rating to adversarial conversations would show further evidence for misunderstanding of different dialogue semantics.

More specifically, we use Kernel SHAP algorithm presented by Lundberg and Lee (2017) which is a modification of LIME Ribeiro et al. (2016). To determine the Shapley values, the algorithm masks the features (utterance tokens in this case), and replaces them with features from a pre-specified set. The dialogue metric scores are then measured using the augmented instances. These scores along with the score assigned to the original utterance are used to approximate a linear model over the utterance features. The weights of this linear model are the approximate Shapley values.

Instead of replacing the masked tokens with the vocabulary from other utterances we replace them with the masking token specific to the model under consideration. This helps measure the effect on the metric when a certain feature is missing. This was inspired by the _Partition SHAP_ algorithm in SHAP python package released by the authors and erasure of feature representations in Li et al. (2017a).

Figure 1: The graph shows relative importance of adversarial conversation features. All adversarial cases having relative importance \(>=1.0\) shows that DialogRPT is paying more importance to adversarial features.

 



### Overall Analysis

We conduct an interpretability analysis of DialogRPT metric performance on adversarial conversations from the Daily Dialogue dataset. We compare the aggregate Shapley values of the human and adversarial features to judge how the DialogRPT metric pays importance to different tokens in the adversarial and human utterances. The aggregation of Shapley values is done using an addition operation. We normalize the Shapley values of features using the minimum and maximum Shapley values for all the features. This helps in conducting a fair comparison of contribution each feature makes to the result. We compare the aggregate Shapley values : 1) between features that are different in human and adversarial responses for cases generated by mutating some of the human features 2) of all features for cases in which the whole human utterance is replaced with an adversarial response. The average relative feature importance of adversarial features (_average adversarial feature importance divided by original feature importance_) is presented in the figure 1. The relative importance of \(>=1.0\) for almost all the adversarial cases shows that DialogRPT considers adversarial features more important than the original ones and points towards a misunderstanding of the role that language features play in the conversation flow.

### Feature Importance Analysis

We present an analysis of feature importance for three adversarial cases below. These use the same conversations presented earlier in the table 2.

#### 6.3.1 Contradictions

Figure 2(a) shows the importance assigned by the DialogRPT to the adversarial features in the contradiction example, shown in the table 2. Tokens such as _"did not see"_ directly contradict the context and are given relatively higher importance in comparison to the other features in the utterance. This proves that DialogRPT fails to understand the role of verb negations in contradicting the context.

#### 6.3.2 Dullness

Figure 2(b) shows the importance of human and adversarial features side by side for the dullness example in the table 2. DialogRPT metric pays more importance to the features which do not progress the conversation in comparison to the human response which contributes meaningfully to the conversation by answering the question being asked. This shows that DialogRPT does not understand which features progress the conversation meaningfully.

#### 6.3.3 Unnatural Paraphrasing

As shown in the figure 2(c), for the unnatural paraphrasing example in the table 2, it is clear that the DialogRPT metric pays more or relatively same importance to the features which are either not used in the same sense, _give_ instead of _pay_, or not used as frequently in the human conversations, like the use of _luncheon_ instead of _lunch_. This shows that DialogRPT does not prefer human-like vocabulary choices.

### Discussion

Our analysis on the DialogRPT suggests that it does not know the correct semantics of conversations. We hypothesize it could be because it is not exposed to bad dialogue behaviors and that leads to the metric paying high attention to erroneous constructions. To correct these, it would be a good idea to either augment the training data with adversarial dialogue behaviors, use adversarial learning to make the metrics more robust or use semantic features in addition to the surface language features during training.

## 7 Choice of Evaluation Metrics

The metrics evaluated in this work are some of the recently proposed with varied training methodolo

Figure 2: Shapley values of adversarial features for each case mentioned above. Each adversarial example shows that DialogRPT metric pays high importance to the adversarial features.



gies e.g., training using human judgments versus some measure of text similarity. Also, our choice of these metrics is rooted in the fact that these metrics are primarily judged by a singular measure of improvement which in most cases is the correlation with human judgments. Our analysis highlights that such singular measures of improvement are not enough to capture the variability of performance exhibited by the evaluation metrics in judging complex conversation behaviors like contradictions. Our goal is not to provide an exhaustive accounting of the performance of all available neural metrics but empirically highlight the problems and performance variability which arise because of different training methodologies. Given our results, we hypothesize that other metrics like Mehri and Eskenazi (2020); Lowe et al. (2017); Tao et al. (2018) trained similarly have deeper problems which need to be highlighted using a fine-grained methodology similar to ours.

## 8 Conclusion and Future Work

Our test-suite helps us draw various insights about performance of different metrics, and shows that all the metrics we test have room for improvement. It points at several flaws in the metrics: 1) lack of generalization ability to unseen data, 2) inability to correctly understand different conversation semantics and 3) prone to training and data-induced biases. Furthermore, our interpretability analysis further corroborates these shortcomings and helps us conclude that metrics need to be exposed to more information about conversation behaviors to make them more robust.

Adversarial behaviors from our test-suite help point to many shortcomings in the metrics we test, but many behaviors are very simplistic _e.g._ speaker sensitivity attacks. Further research can help make these attacks more human-like which may help reveal more information about the evaluation metrics. Our test-suite can also be used directly to make the metrics more robust _e.g._ by augmenting their training data with adversarial examples or by using it as a reward signal in a RL setup for the training of evaluation metrics. This shows several use-cases of our test-suite and directions for future research.

## References

* Bach et al. (2015) Sebastian Bach, Alexander Binder, Gregoire Montavon, Frederick Klauschen, Klaus-Robert Muller, and Wojciech Samek. 2015. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. _PLOS ONE_, 10(7):1-46.
* Banerjee and Lavie (2005) Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In _Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization_, pages 65-72, Ann Arbor, Michigan. Association for Computational Linguistics.
* Boyer et al. (2021a) Anne Boyer, Sharid Loiciga, and David Schlangen. 2021a. Is incoherence surprising? targeted evaluation of coherence prediction from language models. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 4164-4173, Online. Association for Computational Linguistics.
* Beyer et al. (2021b) Anne Beyer, Sharid Loiciga, and David Schlangen. 2021b. Is incoherence surprising? targeted evaluation of coherence prediction from language models. _CoRR_, abs/2105.03495.
* a large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue modelling. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 5016-5026, Brussels, Belgium. Association for Computational Linguistics.
* Cheng et al. (2019) Minhao Cheng, Wei Wei, and Cho-Jui Hsieh. 2019. Evaluating and enhancing the robustness of dialogue systems: A case study on a negotiation agent. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 3325-3335, Minneapolis, Minnesota. Association for Computational Linguistics.
* Clark and Manning (2016a) Kevin Clark and Christopher D. Manning. 2016a. Deep reinforcement learning for mention-ranking coreference models. In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_, pages 2256-2262, Austin, Texas. Association for Computational Linguistics.
* Clark and Manning (2016b) Kevin Clark and Christopher D. Manning. 2016b. Improving coreference resolution by learning entity-level distributed representations. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 643-653, Berlin, Germany. Association for Computational Linguistics.
* Cuayahuitl et al. (2010) Heriberto Cuayahuitl, Steve Renals, Oliver Lemon, and Hiroshi Shimodaira. 2010. Evaluation of a hierarchical reinforcement learning spoken dialogue system. _Computer Speech & Language_, 24(2):395-429 

W. Eckert, E. Levin, and R. Pieraccini (1997)User modeling for spoken dialogue system evaluation. In _1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings_, pages 80-87. Cited by: SS2.
* J. Gao, M. Galley, and L. Li (2018)Neural approaches to conversational ai. CoRRabs/1809.08267. External Links: Link, Document Cited by: SS4.3.
* X. Gao, Y. Zhang, M. Galley, C. Brockett, and B. Dolan (2020)Dialogue response ranking training with large-scale human feedback data. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online, pp. 386-395. External Links: Link, Document Cited by: SS2.
* J. Gauthier, J. Hu, E. Wilcox, P. Qian, and R. Levy (2020)SyntaxGym: an online platform for targeted evaluation of language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, Online, pp. 70-76. External Links: Link, Document Cited by: SS2.
* M. Honnibal and I. Montani (2017)spaCy 2: natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing. To appear. Cited by: SS2.
* Y. Jang, J. Lee, and K. Kim (2020)Bayes-adaptive monte-carlo planning and learning for goal-oriented dialogues. Proceedings of the AAAI Conference on Artificial Intelligence34 (05), pp. 7994-8001. External Links: Link, Document Cited by: SS2.
* Robin Jia and Percy Liang (2017)Adversarial examples for evaluating reading comprehension systems. External Links: Link Cited by: SS2.
* S. Jung, C. Lee, K. Kim, M. Jeong, and G. Geunbae Lee (2009)Data-driven user simulation for automated evaluation of spoken dialog systems. Computer Speech & Language23 (4), pp. 479-509. External Links: Link, Document Cited by: SS2.
* M. Kadhuee, J. Nam, S. Ahuja, J. Won, and S. Lee (2022)Scalable and safe self-learning for skill routing in large-scale conversational ai systems. In NAACL, Cited by: SS2.
* M. Kadhuee, H. Yuan, Y. Kim, and S. Lee (2021)Self-supervised contrastive learning for efficient user satisfaction prediction in conversational agents. In NAACL, Cited by: SS2.
* B. Khalid, M. Alikhani, M. Fellner, B. McMahan, and M. Stone (2020)Discourse coherence, reference grounding and goal oriented dialogue. External Links: Link Cited by: SS2.
* B. Khalid, M. Alkhani, and M. Stone (2020)Combining cognitive modeling and reinforcement learning for clarification in dialogue. In Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain, pp. 4417-4428. External Links: Link, Document Cited by: SS2.
* U. Khandelwal, He, P. Qi, and D. Jurafsky (2018)Sharp nearby, fuzzy far away: how neural language models use context. In Proceedings of the 56th Annual Meeting of the Associtior for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia, pp. 284-294. External Links: Link, Document Cited by: SS2.
* F. Kreyssig, I. Casanueva, P. Budzianowski, and M. Gasic (2018)Neural user simulation for corpus-based policy optimisation of spoken dialogue systems. In Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue, Melbourne, Australia, pp. 60-69. External Links: Link, Document Cited by: SS2.
* A. Lai and J. Tetreault (2018)Discourse coherence in the wild: a dataset, evaluation and methods. In Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue, Melbourne, Australia, pp. 214-223. External Links: Link, Document Cited by: SS2.
* Jiwei Li, W. Monroe, and D. Jurafsky (2017)Understanding neural networks through representation erasure. External Links: Link Cited by: SS2.
* Y. Li, H. Su, X. Shen, W. Li, Z. Cao, and S. Niu (2017)DailyDialog: a manually labelled multi-turn dialogue dataset. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Taipei, Taiwan, Asian Federation of Natural Language Processing, External Links: Link Cited by: SS2.
* Chin-Yew Lin (2004)ROUGE: a package for automatic evaluation of summaries. In Text Summarization Branches Out, Barcelona, Spain, pp. 74-81. External Links: Link, Document Cited by: SS2.
* Yinhan Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov (2019)Roberta: a robustly optimized bert pretraining approach. External Links: Link Cited by: SS2.
* R. Lowe, M. Noseworthy, I. Vlad Serban, N. Angelard-Gontier, Y. Bengio, and J. Pineau (2017)Towards an automatic Turing test: learning to evaluate dialogue responses. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, Canada, pp. 1116-1126. External Links: Link, Document Cited by: SS2.
* Scott Lundberg and Su-In Lee (2017)A unified approach to interpreting model predictions. External Links: Link Cited by: SS2.
* R. Meena, G. Skantze, and J. Gustafson (2014)Data-driven models for timing feedback responses in a map task dialogue system. Computer Speech & Language28 (4), pp. 903-922. External Links: Link, Document Cited by: SS2.
* Sh. Mehri and M. Eskenazi (2020)Unsupervised evaluation of interactive dialog with DialoGPT. In Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 

pages 225-235, 1st virtual meeting. Association for Computational Linguistics.
* Mehri and Eskenazi (2020)Shikib Mehri and Maxine Eskenazi. 2020b. USR: An unsupervised and reference free evaluation metric for dialog generation. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 681-707, Online. Association for Computational Linguistics.
* Mikolov et al. (2013) Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In _Advances in Neural Information Processing Systems_, volume 26. Curran Associates, Inc.
* Miller (1995) George A. Miller. 1995. Wordnet: A lexical database for english. _Commun. ACM_, 38(11):39-41.
* Panfili et al. (2021) Laura Panfili, Steve Duman, Andrew Nave, Katherine Phelps Ridgeway, Nathan Eversole, and Ruhi Sarikawa. 2021. Human-ai interactions through A gricean lens. _CoRR_, abs/2106.09140.
* Pang et al. (2020) Bo Pang, Erik Nijkamp, Wenjuan Han, Linqi Zhou, Yixian Liu, and Kewei Tu. 2020. Towards holistic and automatic evaluation of open-domain dialogue generation. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 3619-3629, Online. Association for Computational Linguistics.
* Papineni et al. (2002)Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics_, pages 311-318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
* Park et al. (2021) Sunghyun Park, Han Li, Ameen Patel, Sidharth Mudgal, Sunjin Lee, Young-Bum Kim, Spyros Matsoukas, and Ruhi Sarikaya. 2021. A scalable framework for learning from implicit user feedback to improve natural language understanding in large-scale conversational AI systems. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 6054-6063, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Pshhad et al. (2020) Leila Pishhad, Federico Fancellu, Ran Zhang, and Afsaneh Fazly. 2020. How coherent are neural models of coherence? In _Proceedings of the 28th International Conference on Computational Linguistics_, pages 6126-6138, Barcelona, Spain (Online). International Committee on Computational Linguistics.
* Radford et al. (2018)Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2018. Language models are unsupervised multitask learners.
* Ribeiro et al. (2016) Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "why should i trust you?": Explaining the predictions of any classifier. In _Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_, RDD '16, page 1135-1144, New York, NY, USA. Association for Computing Machinery.
* Ribeiro et al. (2020) Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020. Beyond accuracy: Behavioral testing of NLP models with CheckList. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 4902-4912, Online. Association for Computational Linguistics.
* Saii et al. (2019)Ananya B. Saii, Mithun Das Gupta, Mitesh M. Khapra, and Mukundhan Srinivasan. 2019. Re-evaluating adem: A deeper look at scoring dialogue responses.
* Shapley (1952) Lloyd S. Shapley. 1952. _A Value for N-Person Games_. RAND Corporation, Santa Monica, CA.
* Shim et al. (2021) Heereen Shim, Dietwig Lowet, Stijn Luca, and Bart Vanrumste. 2021. Building blocks of a task-oriented dialogue system in the healthcare domain. In _Proceedings of the Second Workshop on Natural Language Processing for Medical Conversations_, pages 47-57, Online. Association for Computational Linguistics.
* Sun et al. (2021a) Weiwei Sun, Shuo Zhang, Krisztian Balog, Zhaochun Ren, Pengjie Ren, Zhumin Chen, and Maarten de Rijke. 2021a. Simulating user satisfaction for the evaluation of task-oriented dialogue systems. In _Proceedings of the 44rd International ACM SIGIR Conference on Research and Development in Information Retrieval_, SIGIR '21. ACM.
* Sun et al. (2021b) Weiwei Sun, Shuo Zhang, Krisztian Balog, Zhaochun Ren, Pengjie Ren, Zhumin Chen, and Maarten de Rijke. 2021b. _Simulating User Satisfaction for the Evaluation of Task-Oriented Dialogue Systems_, page 2499-2506. Association for Computing Machinery, New York, NY, USA.
* Tao et al. (2018) Chongyang Tao, Lili Mou, Dongyan Zhao, and Rui Yan. 2018. RUBER: an unsupervised method for automatic evaluation of open-domain dialog systems. In _Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018_, pages 722-729. AAAI Press.
* trumbelj and Kononenko (2013) E. trumbelj and I. Kononenko. 2013. Explaining prediction models and individual predictions with feature contributions. _Knowledge and Information Systems_, 41:647-665.
* experts vs. users. In _Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 569-578, Atlanta, Georgia. Association for Computational Linguistics
 

Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohannaney, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. 2020. BLiMP: The benchmark of linguistic minimal pairs for English. _Transactions of the Association for Computational Linguistics_, 8:377-392.
* Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_, pages 1112-1122, New Orleans, Louisiana. Association for Computational Linguistics.
* Yang et al. (2021) Jie Yang, Hirofumi Kikuchi, Takatsugu Uegaki, and Hideaki Kikuchi. 2021. The effect of the repetitive utterances complexity on user's perceived empathy and desire to continue dialogue by a chat-oriented dialogue system. In _Proceedings of the 9th International Conference on Human-Agent Interaction_, HAI '21, page 241-244, New York, NY, USA. Association for Computing Machinery.
* Zhang et al. (2018) Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 2204-2213, Melbourne, Australia. Association for Computational Linguistics.
* Zhang et al. (2019) Yuan Zhang, Jason Baldridge, and Luheng He. 2019. PAWS: paraphrase adversaries from word scrambling. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 1298-1308, Minneapolis, Minnesota. Association for Computational Linguistics.

## Appendix A Adversarial Examples

We present one adversarial example for all of the dialogue attributes (not present in the table 2) in the tables 5 and 6. These examples are generated in an automatic way using our test-suite and testify to its effectiveness in generating different adversarial behaviors 



\begin{table}
\begin{tabular}{|l|p{142.3pt}|p{142.3pt}|} \hline  & \multicolumn{2}{c|}{Naturalness} \\ \hline Entrainment & Vocabulary Diversity & Natural Paraphrasing \\ \hline
**A**: what music do you like? & **A**: i for sure read an speak english **B**: it don really prefer any kind of music & **A**: my hobbies are fashion an clothes! **B**: that is helpful. i do as well and just graduated college. **A**: i love pork, especially bacon. **Actual**: awesome! what is second life? never heard of it \\ \hline
**Adversarial**: well, does your mom play any euphony at her restaurant? & **Adversarial**: bacon is well. i do not eat much much meat though. do you have pets? & **Adversarial**: What is Second Life? Never heard of it! \\ \hline \end{tabular}
\end{table}
Table 6: Examples of different adversarial responses for naturalness cases generated by our heuristics.

\begin{table}
\begin{tabular}{|l|p{142.3pt}|p{142.3pt}|} \hline  & \multicolumn{2}{c|}{Coherence} \\ \hline Entailment & Pronoun & Named Entity Attacks \\ \hline
**A**: It’s a Taiwanese puppet doll. & **A**: Harry, come here immediately! **B**: It’s huge! & **A**: Can you tell me what bus to catch from Altadena to downtown LA? **B**: You can catch the 486. **Actual**: That bus goes all the way to LLA? \\ \cline{2-3}
**A**: Teah. They’re usually this big. & **Actual**: Don’t take that tone with saw you hit your brother. & **Abraisal**: That bus goes all the way to LA? \\ \cline{2-3}
**Letustrial**: I’m Rose Teller. I think I’ve seen you somewhere before? & **Adversarial**: Don’t take that tone with you! we saw me hit your brother. & **Adversarial**: That bus goes all the way to Chilin? \\ \hline \hline Co-reference Attacks & Speaker Sensitiveness & Naturalness \\ \hline
**A**: i am only five feet and five inches, so i am short too. are you married? & **A**: i hat families. i prefer to be alone. **A**: i just ate a mango and now i need to go to the hospital. **B**: i am short at 5 ft tall. how about you? & **A**: i just ate a mango and now i need to go to the hospital. **Actual**: are you allergic? dogs give me bad allergies. \\ \cline{2-3}
**A**: i do not like dogs. i was attacked when i was a little girl. & **A**: i am 5ft and 6in tall.i weigh 220 pounds and its all muscle. & **Adversarial**: are you allergic? dogs give me bad allergies bad allergies. \\ \cline{2-3}
**Actual**: i am so sorry to hear that. i bet you would like pedro be is sweet & **Adversarial**: i am 5ft and 6in tall. i weigh 220 pounds and its all muscle. & **Adversarial**: are you allergic? dogs give me bad allergies bad allergies. \\ \hline \end{tabular}
\end{table}
Table 5: Examples for the manipulation of coherence based attacks.