What Was Your Name Again? Interrogating Generative Conversational Models For Factual Consistency Evaluation (2022)

Abstract: Generative conversational agents are known to suffer from problems like inconsistency and hallucination, and a big challenge in studying these issues remains evaluation: they are not properly reflected in common text generation metrics like perplexity or BLEU, and alternative implicit methods like semantic similarity or NLI labels can be misguided when few specific tokens are decisive. In this work we propose ConsisTest; a factual consistency benchmark including both WH and Y/N questions based on PersonaChat, along with a hybrid evaluation pipeline which aims to get the best of symbolic and sub-symbolic methods. Using these and focusing on pretrained generative models like BART, we provide detailed statistics and analysis on how the model’s consistency is affected by variations in question and context.

## introduction although capable of producing fluent responses, generative conversational models have proved to be prone to problems like hallucination and selfcontradiction, which do not go away with increasing model size @xcite @xcite . in knowledge-grounded conversation, the prominent issue is hallucination -i.e. generating plausible looking statements that are factually incorrectand can be partly mitigated by improving the knowledge selection or retrieval module @xcite . in open-domain chitchat, where the grounding "knowledge" is mainly personal attributes and facts (preset before or said during the conversation), this manifests itself as inconsistency or forgetfulness ( agent persona my name is paul. i live with my two sons in chicago. my favorite color is green. i am a professional golf player. utterances @xmath0. how are you today? @xmath1. my name is luna. -nice to meet you luna. what do you do for work? @xmath2. are you married ? -no i am not married. do you have any kids? @xmath3. how about you? -i have two sons. do you have any hobbies? @xmath4. what was your name again?. -my name is luna. what is your favorite color? on a general level, open-domain conversational consistency can be defined with respect to persona, style, and context @xcite . when limited to persona and context (which is often the case in literature), due to the inherent logical aspect, consistency becomes a relatively objective and quantifiable attribute compared to most other conversational traits: while qualities like humanness, interestingness or engagingness @xcite usually need human evaluation for a reliable assessment, consistency can be fairly estimated using simplifying assumptions. in particular, identifying inconsistent utterances can be reduced to a classic nli (natural language inference) problem by assuming that contradictions are contained in a sentence pair @xcite . following the nli paradigm, many studies have tried to provide better and bigger datasets or models to train for detecting contradictions. these efforts how-ever mainly focused on the overall assessment at inference time -which allows for improving consistency by re-ranking response candidates-, and have not explored enough the fine-grained dependencies of consistency, subject to parameters like data and training. in this work we try to get new insights into conversational consistency, via simplifying assumptions that allow to reduce the problem one step further, into a pseudo-qa case. to this end, we create an evaluation dataset following an interrogative approach; i.e. posing factual questions about the facts that are already mentioned in the conversation history or persona. this allows us to develop and use a hybrid evaluation method for precise performance assessment. our contribution is threefold:(1) we present con-sistest: an interrogative conversational qa dataset with both wh and y/n questions to assess factual consistency in open-domain conversational agents. ## related work consistency and factuality of responses has always been one of the main qualities in the assessment of conversational agents but framing it as an nli problem by @xcite opened the way for reliable automatic evaluations using models trained on labelled data: they introduced the dnli dataset, comprising of premise-hypothesis pairs semi-automatically generated from the per-sonachat persona statements and showed that using the nli model to re-rank generated responses, improves persona consistency in dialogue. @xcite created inferconvai, another dataset based on personachat personas, and applied it for dialogue topic coherence evaluation. @xcite employed such an evaluator for unlikelihood training and showed its effectiveness for improving logical consistency, while @xcite used it as a reward function in reinforcement learning with positive impact on the factual consistency between response and persona facts. to address the limitations of dnli, @xcite introduced decode, a human-written fully-conversational dataset based on multiple datasets and covering logical and context-related reasoning beyond personal facts, which proved to result in significantly more robust consistency evaluation. another approach (besides the nli-based methods) for automatic evaluation of consistency is asking and answering questions, which is more applicable to knowledge-grounded conversation. originally proposed in abstractive summarization @xcite , it assumes that factually equivalent or consistent texts should be interchangeably usable to generate factual questions and to answer them. @xcite adapted it to introduce q 2 , to assess factual consistency in knowledge-grounded dialogues with significantly higher correlation with human judgement. more related to our work, @xcite applied a similar interrogative approach (but limited to wh questions on history) combined with nli-based assessment, to provide a framework for evaluating consistency in open-domain conversational agents, and used it to compare chatbots in interactive setups. finally, rashkin et al. (2021) explored adding control code features (via special tokens) to inform a pretrained model about the groundedness of responses in knowledge-grounded conversations, and showed that by using these codes during the inference, the model can be effectively persuaded to generate more grounded responses. ## consistest the consistest benchmark is based on 'interrogative factual questioning': since a consistent dialogue agent should commit to its personal narrative, to assess consistency, we ask the agent factual questions about previously stated or uttered facts, and demand accordance with them. to create the benchmark, we apply this approach on the popular personachat dataset @xcite which contains crowd-sourced conversations grounded in predefined "personas" (i.e. a set of 4-6 simple personal statements), and therefore allows us to study both the persona and history consistency. figure 1 demonstrates the overall process in two steps: first, a personachat persona+conversation pair (a) is studied to produce simple factual questions in both wh and y/n formats (b). then these questions are appended to conversation segments to create a benchmark sample (c). next we discuss the process in more detail. ## evaluation method evaluating generated text is a well-known challenge in nlp, situated between the inadequacies of automatic metrics and difficulties of human evaluation (van der @xcite . when limited to answering questions, the task becomes more manageable and well-defined since there are often logically limited sets of correct or 'gold' answers that can act as reference. in particular, factual questions provide the possibility to formulate the problem as span extraction (e.g. squad dataset @xcite ) which then can be evaluated with more confidence using token-level comparison methods like the f1-score. but the conversational aspect of our problem prevents us from directly and exclusively relying on token-level evaluation, since the overall semantic agreement between the response and reference fact is not guaranteed. a common alternative, especially when dealing with consistency, is using nli models @xcite which classify the relationship between a pair of phrases (corresponding to the reference fact and model response in our case) as one of entailment, neutral or contradiction. this is quite helpful in identifying sharp inconsistencies but is less sensitive to token-level nuances that might be of interest in factual qas. regarding our benchmark, there are cases where the pure nli method often falls short of a valid assessment, most notably: fine-tuning the classic nli model on conversational data (e.g. the dnli dataset @xcite ) leads to partial improvement but to achieve more accurate results we decide to develop a hybrid pipeline which tries to get the best of the symbolic and sub-symbolic methods. ## experiments having a reliable evaluation method, we can now delve deeper to see how/if the consistency of a conversational model is affected by question properties (e.g. source and type) and training setups (e.g. input encoding scheme). as the standard model, we pick the base version of bart @xcite which is a pretrained encoder-decoder transformer with a total number of 12 layers (6+6) and 140m parameters. to encode the inputs, we follow the standard practice of identifying the speakers @xmath5. figure 3 shows the result of applying such an encoding to an arbitrary sample of consistest or personachat dataset. we choose 3 full turns -or 6 utterances-for the memory size (maximum number of previous utterances kept in the context) and finetune the model on personachat train set for 6 epochs (early-stopping) with an effective batch size of 128. we then use the finetuned model to do inference on consistest-02; i.e. generating responses to the provided con-text+question pairs. to ease reproducibility, we implement the training and inference using the trainer and generate methods from the huggingface transformers library @xcite . ## conclusion and future work in this work we tried to obtain new insights into one of the prominent issues in open-domain conversational modeling, i.e. consistency. taking a factual questioning approach, we built a benchmark dataset (consistest) based on personachat, and developed a hybrid evaluation pipeline that takes advantage of both symbolic and sub-symbolic methods to achieve high correlation with human evaluation of factual consistency. then, focusing on pretrained generative transformers (i.e. bart), we studied how the consistency score varies in different subsets of our benchmark: we confirmed the intuition that in a persona-history setting, remaining consistent with respect to conversation history is significantly more challenging than commitment to persona, and we showed that this gap is mainly rooted in structural and positional advantages of the latter. we also observed that in the case of y/n questions, agreement between the short and long answer is not a given with these models, and accounts for the majority of y/n inconsistencies. many more aspects and dependencies of conversational consistency remain to be explored, including the difference between the history-generated and history-extracted questions, effect of insertion distance (d), importance of the base model (e.g. decoder-only models like gpt2 vs. encoderdecoders like bart), and the detailed dynamics behind the apparent positional bias observed in 5.1. these, along with the refinement and expansion of the dataset, provide interesting options for the future work.