Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach (2024)

Abstract: Humans work together to solve common problems by having discussions, explaining, and agreeing or disagreeing with each other.Similarly, if a system can have discussions with human partners when solving tasks, it has the potential to improve the system’s performance and reliability.In previous research on explainability, it has only been possible for systems to make predictions and for humans to ask questions about them, rather than having a mutual exchange of opinions.This research aims to create a dataset and computational framework for systems that discuss and refine their predictions through dialogue. Through experiments, we show that the proposed system can have beneficial discussions with humans, improving the accuracy by up to 25 points on a natural language inference task.

## introduction today's deep learning systems are performant but opaque, leading to a wide variety of explainability techniques that attempt to take in a system prediction and output an explanation justifying the prediction @xcite @xcite @xcite . many such explainability techniques require significant expertise in deep learning to use effectively, requiring consumers of the explanations to analyze the data, internal states, and output trends of the system of interest @xcite @xcite . however, many potential system users lack this expertise, such as medical or legal professionals who want to use machine learning models and need to confirm the veracity of the generated results or rectify any mistaken predictions. to address this issue, researchers are working to find ways to both explain system predictions in nat1 our dataset is publicly available at: https://github. com/kanekomasahiro/discussion_nlp ural language @xcite @xcite @xcite and give instructions and feedback to systems through natural language @xcite @xcite @xcite . chain-of-thought (cot) prompting has shown that natural language contributes to performance improvements in complex multistep inference @xcite @xcite . step-by-step reasoning in cot relies solely on the system to make predictions without human involvement. there is also work that allows users to ask questions about the system's predictions and tasks @xcite in a conversational format. compared to the more standard learning and explanation paradigms, this approach allows humans to understand and teach the system intuitively. however, in these works, the communication tends to be one-sided, from human-tosystem or system-to-human, which still falls short of the full interactive problem solving process experienced by human interlocutors @xcite . in this study, we take the first steps towards es-tablishing a framework for human-system collaboration on prediction problems through discussion (illustration in figure 1 ). if such a system is realized, it will allow both humans and the system to engage in explanations of predictions, ask questions about unclear points, refine their thoughts, and solve problems. first, we create a dataset of human-human discussions regarding a prediction task (section 2). in particular, we use the task of natural language inference (nli): prediction of the relationship between a "premise" sentence and a "hypothesis" sentence is entailment, contradiction, or neutral @xcite . we specifically choose relatively difficult or ambiguous cases to spur discussion between the participants. second, we train and evaluate a system that is capable of discussing an nli problem with a human (sections 3, 4). it is achieved by constructing prompts with manually created discussion examples so the system can learn from humans how to discuss, accept, or object to the provided opinions about the topic. the results of both quantitative and human evaluation demonstrate that a system could perform more informative discussions by training to have a discussion with few-shot learning (section 5). we also found that providing the system with information about the discussion topic improved its performance in many cases compared to the system that did not have access to such information. on the other hand, the discussion revealed that the system tends to be too compliant with human opinions. therefore, addressing the risk of transmitting incorrect knowledge or maliciously altering the system's knowledge of humans is necessary. we also show that few-shot usage of discussion data can enable the system to counter human arguments correctly (section 6). finally, we demonstrate that using discussion data generated by the system @xcite can achieve equivalent results to those of the system that used manually created discussion data in few-shot learning or finetuning cases. ## discussion dataset creation the nli task aims to determine the logical relationship between a hypothesis sentence and a premise sentence @xcite . the task involves classifying whether the hypothesis sentence is entailment, contradiction, or neutral. for example, given the premise "the cat is sitting on the mat" and the hypothesis "the mat is empty", the task would involve classifying the relationship as a contradiction. nli tasks require deep assimilation of fine nuances of common sense knowledge, and much work has been done to explain this with natural language as a prediction reason @xcite . therefore, we also target the nli task and build a system that predicts entailment, contradiction, or neutrality through discussion. to train a system that can engage in a discussion, we create a dataset of human annotators discussing nli problems. we use the stanford nli (snli) dataset @xcite , a common benchmark dataset in nlp, to create the discussion data. collecting high-quality discussion data among humans is costly, as it requires knowledgeable annotators about the task and multiple dialogue turns for each problem. fourteen annotators with knowledge of nlp were asked to annotate the data. 2 first, the annotators were presented with premise and hypothesis sentences and asked to predict labels such as entailment, contradiction, or neutral. we randomly paired two annotators to have them assign labels for the same premise and hypothesis. then, they discussed the labels that they had assigned differently and decided on the final labels based on those discussions. the premise and hypothesis sentences were sampled from 300 problems from the development data and 750 problems from the evaluation data of snli. these were used as development and evaluation data in the discussion data, respectively. each annotator pair is asked to predict the labels of 150 problems. snli development data originally consists of problems with labels from five crowd workers, and the majority vote of these labels determines the golden label. to find relatively hard cases that might spur more discussion, we sampled problems for annotation from those in which three of the five had the same label. our annotators were then paired with each other and discussed the questions for which they had given different labels. they discussed in a freeform manner until they agreed on a final decision. 3 preliminary experimental results showed that the number of discussion turns tended to be higher for oral rather than text-based discussions. therefore, we created discussion data by transcribing oral discussions among the annotators, using whisper (medium.en) @xcite 4 for transcription. the text transcribed by whisper was manually corrected for transcription errors and manually separated into speech segments. then, for each utterance, we assigned the evidential utterances for the final label and the labels of "supportive", "unsupportive", or "irrelevant" to each utterance. for example, for figure 1 , "both have a person sitting in the chair, but they are neutral because no gender is specified." is labeled as supportive, "it is entailment because the person sits in a chair." is unsupportive, and "yes." is labeled as irrelevant. these labels are not used in the fewshot learning process but are used to evaluate the discussion ability of the system automatically. in this annotation work, discussion data were collected for 102 problems. of these, 10 problems were used as prompts for few-shot learning, 27 for validation data, and 65 for evaluation data. the average number of utterances for each problem in the prompt, validation, and evaluation data is 4.4, 6.3, and 5.1 respectively. for validation and evaluation data, the number of supportive/unsupportive utterances are 85/23 and 133/72 respectively. ## discussion system we use three types of systems in the experiments: zero-shot, few-shot, and few-shot-discussion. in the zero-shot system, only the task description is given as a prompt. in the few-shot system, the examples' task description and premise, hypothesis, and gold labels are given as prompts. in the few-shot-discussion system, in addition to the task description and examples, human discussion examples about the labels of the examples are given as prompts. these prompts are concatenated with the problem to be solved and given as input to the system to perform inference. examples of each prompt are shown in figure 2 . the discussion example distinguishes human utterances between "human1:" and "human2:". the examples used in the prompts are the same for both the few-shot and the few-shot-discussion systems. we use the same examples for all problems. all methods do not update the parameters of the systems. we use gpt-3.5 foot_3 @xcite and chatgpt foot_4 (openai, 2023) for the zero-shot, few-shot, and few-shot-discussion systems. ## evaluation method we evaluate a system's discussion ability from the following three perspectives: (1) can the system generate utterance content that contributes to the final label? (2) can the system agree with statements that support the correct label and refute statements that support the incorrect label? (3) does discussion with humans improve task performance? to examine these discussion abilities, we compare each system by performing automatic and manual evaluations. we investigate utterances generated from the systems to determine if they contribute to the automatic evaluation's final label. for that, we use the utterances generated by the system for the given problems and evaluate how well they match the reference utterances between humans from discussion evaluation data. each utterance in our discussion evaluation data is annotated as either supportive or unsupportive of the gold label. if a system is more likely to generate a supportive utterance than an unsupportive utterance for the gold label, the system can be considered capable of making correct discussions that lead to the correct answers. for example, "i think it is also better to consider the general cases." is the supportive utterance, and "is the phone in the hypothesis necessarily a cellphone? it could be a landline phone." is the unsupportive utterance in figure 2 . therefore, we also investigate whether the system is better at generating supportive utterances over unsupportive ones. specifically, we evaluate the similarity between the system-generated utterances and the actual human utterances for supportive and unsupportive utterances, respectively. we concatenate the input problem and the discussion utterance up to the target utterance and generate the next target utterance. for example, if the second human's utterance in the discussion is the target utterance, then the prompt is "premise: a nun is taking a picture outside. hypothesis: a nun is taking a selfie. label: entailment or neutral discussion: human1: i think it is entailment, because the nun is taking a picture, so it might be a selfie. human2:", and the system should generate an utterance that would be evaluated against the following utterance made by a human "since it is outside, it is conceivable that the nun is taking some scenery.". at this point, the problem has two opposing labels in the prompt because we want it to discuss two different labels. we use actual human utterances as references and compute the bertscore @xcite of the system's outputs for evaluation. bertscore leverages the pre-trained language model such as bert @xcite and roberta @xcite and matches words in candidate and reference sentences by cosine similarity. bertscore computes precision, recall, and f1 measures. therefore, bertscore can be used to compare the system's content and human utterances with each other. we use roberta-large 7 for the 7 https://huggingface.co/roberta-large pre-trained language model for bertscore. we conduct a significance test using t-test (@xmath0.01). we set the temperature parameter of gpt-3.5 and chatgpt to 0.7 and generate ten outputs for each input. we calculate bertscore for each of the ten outputs and test for significance among the calculated ten scores. next, we use human evaluation to examine whether the system can agree with supportive human utterances and refute unsupportive human utterances. the human participants and the system predict different labels for the same problem. then, they engage in a discussion, and the final label result is demonstrated to be in agreement with the labels assigned in the snli data through the consistency of the agreement rate. in this process, we evaluate the ability of the system to accept a human's opinion when the system's label is incorrect, and when the human's label is correct, and the ability of the system to object to a human's opinion when the human's label is incorrect, and the system's label is correct. similarly to above, we selected those data with the same label 3 times (e.g., entailment, entailment, neutral, entailment, neutral). as a result, we sampled 140 problems that differ from the problems collected in section 2. during this process, if the system's label was correct, humans engaged in adversarial discussions to change the system's label. if the system's label was incorrect, humans engaged in discussions to guide the system toward the correct label. here, the discussion was text-based rather than verbal, as the system takes textual input. to conduct a discussion with the system, we input the prompt and problem shown in figure 2 to the system and then inputted additional human utterance examples related to the discussion after each system predicted the label. in the additional input, the beginning of human utterance is prefixed with "human:" and the end is prefixed with "system:" to indicate that the next is a system's utterance. specifically, the first prompt for discussion is "human: let's discuss it more. i think neutral, because there may be a kitchen in the barn. system:". the system predicts the final label when the discussion is finished. we investigate how discussion with humans improves nli task performance. the system predicts the label, then the human and the system discuss and decide on the final label. we compare the performance of each label before and after the dis ## analysis 6.1 pseudo-discussion data one drawback of using discussion data is that it can be costly to create compared to datasets that only have gold labels. using pre-trained models to annotate unlabeled data and use this data for training has been shown to improve performance (wang snli r1 r2 r3 w/ dis. mpt 85.2 67.4 † 55.2 † 55.0 † mpt-inst. 87.7 † 68.2 † 56.1 † 55.3 † falcon 86.2 † 67.6 55.5 † 54.9 falcon-inst. 90.3 † 71.7 † 58.4 † 57.6 † w/o dis. mpt 85.4 65.2 53.9 52.4 mpt-inst. 85.1 64.0 51.1 50.7 falcon 84.6 67.9 54.7 54.2 falcon-inst. 85.3 66.2 53.1 53.0 w/ dis. mpt 86.7 † 68.3 † 55.2 † 55.0 † mpt-inst. 86.9 68.8 † 56.1 † 55.3 † falcon 88.1 68.1 55.5 54.9 falcon-inst. 90.7 † 71.9 † 58.4 † 57.6 † w/o dis. mpt 85.4 65.2 53.9 52.4 mpt-inst. 86.0 64.0 51.1 50.7 falcon 88.5 67.9 54.7 54.2 falcon-inst. 89.7 67.8 55.5 56.4 in generating human discussions, the system is given prompts in the form of the premise, hypothesis, gold label, and the labels from each human. the human labels are randomly chosen to be the gold label or the other incorrect label. for example, given the premise "a nun is taking a picture outside." and hypothesis "a nun is taking a selfie." with the gold label of neutral, the prompt would be "reproduce a multi-turn interactive discussion in which the following premise and hypothesis are entailment, contradiction, or neutral, with the humans agreeing with each other on the final label. human1's label is neutral, and human2's label is a contradiction. in the end, they agree on the label of neutral. premise: a nun is taking a picture outside. hypothesis: a nun is taking a selfie.". the gpt-3.5 and chatgpt generate human discussions for 10 problems used in the few-shot and 2,000 problems used in the fine-tuning, respectively. the average number of utterances in humancreated discussions was 4.4, and the average number of utterances in system-generated discussions was 4.7. regarding the number of utterances, human and system arguments are almost the same. we used instruction tuned and non-instruction tuned models for mpt foot_7 (team, 2023) and falcon foot_8 @xcite as pre-trained models for fine-tuning. we used hyperparameters from existing studies @xcite as a reference and fine-tuned the batch size to 128, the learning rate to 2e-5, and the epoch to 3. we used five nodes, each containing eight nvidia a100 gpus. the system is given both the labels and discussions as golds during training, and we evaluate using only labels during inference. we train models without pseudo-discussion data as a baseline. the baseline models are trained with only the labels. these results indicate that the system is capable snli r1 r2 r3 random dis. ## related work in this study, systems and humans discuss a problem through dialogue. dialogue systems can be broadly classified into two types: task-oriented systems that perform specific tasks, and non-taskoriented systems that do not have the goal of task completion, such as casual conversation. this study aims to conduct appropriate predictions in nlp tasks through discussions between humans and the system and is classified as a task-oriented system. many existing dialogue systems target daily life tasks such as hotel reservations and transportation inquiries @xcite . pre-trained models such as bert @xcite and gpt-2 @xcite are also utilized in dialogue systems for daily life tasks. recently, chatgpt (openai, 2023) has been proposed for more generic interaction based on a pre-trained model. we similarly use a pretrained model for our system. as far as we know, few studies use discussion for nlp tasks similar to ours. @xcite proposed the talktomodel, which explains through dialogue three tasks of loan, diabetes, and recidivism prediction. the user can talk to the talkto-model in five categories: prediction explanation, data modification, error analysis, dialogue history reference, and experimental setting explanation. data for learning and evaluating the talktomodel are generated by instructing the annotator to converse about these categories. however, the categories were not determined based on interviews or data but were defined subjectively by the authors. therefore, it is possible that the categories do not reflect actual conversations that humans need. on the other hand, our study was conducted in an open-ended dialogue to generate data. additionally, our study aims for mutual understanding through a bidirectional dialogue where both humans and the system express opinions and questions, unlike the systems that only respond to human questions in a unidirectional dialogue. there is research on generating explanatory text for predictions as a way to transfer information from systems to humans through natural language. for example, research regarding natural science tests @xcite , image recognition and image question answering @xcite , mathematics tests @xcite have been studied. additionally, systems for generating explanations using pretrained models such as t5 @xcite and gpt-3.5 @xcite have also been proposed @xcite . however, as these generated explanations cannot be used to seek additional explanations or specific explanations, the interpretability is not sufficient in practice as pointed out by @xcite . instead of directly predicting answers, cot uses natural language to derive answers step-bystep @xcite . this leads to complex multistep inferences. by adding the phrase "let's think step by step" before each answer, @xcite demonstrate that language models are competent zero-shot cot. on the other hand, @xcite shows that cot can achieve competitive performance even with invalid reasoning steps in the prompt. cot's step-by-step approach is based on the system only, whereas our proposed method incorporates human involvement in the system to facilitate collaboration between humans and the system. additionally, our approach utilizes discussions for a step-by-step thinking process. research is also being conducted on the use of natural language by humans to provide instructions and feedback to the system. @xcite has developed multi-modal grounded language agents that perform reinforcement learning on human dialogue-based instructions. @xcite proposed a method to integrate humanprovided feedback in natural language to update a robot's planning cost applied to situations when the planner fails. @xcite proposed a method to modify a model by natural language patches and achieved performance improvement in sentiment analysis and relationship extraction tasks. campos and shern (2022) proposed a method for training a model to behave in line with human preferences, by learning from natural language feedback, in text summarization. on the other hand, these studies cannot be explained or questioned by the system to humans. ## conclusion while deep learning systems have been highly effective in various tasks, their lack of interpretability poses a challenge to their use in real-world applications. to address this, we proposed a system that engages in a dialogue with humans in the form of discussing predictions, which allows both humans and the system to engage in explanations, ask questions, refine their thoughts, and solve problems. our experimental results showed that the system trained with few-shot learning for discussion could perform more useful discussions than the system that was not trained for discussion and provided insights on the challenges and opportunities of this approach. this research provides a new avenue for developing more interactive deep-learning systems.