Explanation Graph Generation via Pre-trained Language Models: An Empirical Study with Contrastive Learning (2022)

Abstract: Pre-trained sequence-to-sequence language models have led to widespread success in many natural language generation tasks. However, there has been relatively less work on analyzing their ability to generate structured outputs such as graphs. Unlike natural language, graphs have distinct structural and semantic properties in the context of a downstream NLP task, e.g., generating a graph that is connected and acyclic can be attributed to its structural constraints, while the semantics of a graph can refer to how meaningfully an edge represents the relation between two node concepts. In this work, we study pre-trained language models that generate explanation graphs in an end-to-end manner and analyze their ability to learn the structural constraints and semantics of such graphs. We first show that with limited supervision, pre-trained language models often generate graphs that either violate these constraints or are semantically incoherent. Since curating large amount of human-annotated graphs is expensive and tedious, we propose simple yet effective ways of graph perturbations via node and edge edit operations that lead to structurally and semantically positive and negative graphs. Next, we leverage these graphs in different contrastive learning models with Max-Margin and InfoNCE losses. Our methods lead to significant improvements in both structural and semantic accuracy of explanation graphs and also generalize to other similar graph generation tasks. Lastly, we show that human errors are the best negatives for contrastive learning and also that automatically generating more such human-like negative graphs can lead to further improvements.

## introduction pre-trained sequence-to-sequence language models (plms) like bart @xcite ) and 1 our code and models are publicly available at https: //github.com/swarnahub/explagraphgen . belief: collectivism is terrible for society. argument: collectivism increases empathy. stance: counter increases empathy collectivism capable of causes improve human relationship terrible for society is not a increases empathy collectivism capable of terrible for society is not a empathy society not part of gold graph t5-generated graph structurally and semantically incorrect belief: since fast foods are greasy and fattening, banning them would control obesity. argument: mcdonalds has salads. stance: counter mcdonalds fast food part of not has context greasy and fattening gold graph t5-generated graph semantically incorrect salads part of fast food greasy and fattening salads banning them control obesity capable of has context part of causes figure 1 : two representative examples from expla-graphs @xcite showing the belief, argument, stance, gold explanation graph, and t5-generated explanation graph. the dashed nodes represent commonsense nodes and the dashed edges are incorrect edges. the first generated graph is structurally incorrect and the second graph is semantically incorrect. t5 @xcite have led to significant advances in many natural language generation tasks like text summarization and machine translation. the models are pre-trained on massive amounts of text data with self-supervision, thus enabling them to construct coherent natural language sentences for downstream tasks. this then raises the question whether pre-trained language models, trained on free-form natural language data, can also adapt themselves to generate structured outputs like graphs. graphs are common in nlp tasks that involve representing structured knowledge in the form of knowledge bases @xcite , constructing event chains from documents @xcite , or more recent work on encoding reasoning chains, explanations, or deductive proofs @xcite @xcite . graphs differ from free-form natural language. in the context of nlp, natural language graphs (consisting of textual nodes and edges) can have distinct structural and semantic properties. for example, consider a recently proposed commonsense explanation graph generation task shown in fig. 1 @xcite . each example shows a belief, an argument and an explanation graph explaining how the argument supports or refutes the belief. these explanation graphs encode structured knowledge (augmented with commonsense) and consist of concepts as nodes and relations from conceptnet @xcite ) as edges. for example, the second graph encodes the knowledge that "both salads and fast food are part of mcdonalds and hence mcdonalds is not greasy and fattening", thus explicitly refuting the belief. from prior work, the structural constraints enforce the graphs to be connected directed acyclic and the nodes to contain at least two concepts from the belief and two from the argument. the semantic aspect deals with commonsense and evaluates whether each edge expresses coherent relational knowledge and if the whole graph explains the stance. following @xcite , we represent graphs as strings composed of concatenated edges and fine-tune t5 to generate graphs in an autoregressive manner. we observe that while moderate amount of supervision enables the model to learn valid graph encodings, the graphs frequently violate task-specific structural constraints (like connectivity). for instance, the first example in fig. 1 shows a graph generated by t5 that is disconnected and hence structurally incorrect. moreover, for the fraction of graphs that are structurally correct, the model also makes commonsense mistakes, a type of semantic error, by inferring wrong or incoherent relations between concepts. both t5-generated graphs shown in fig. 1 contain incoherent or noncommonsensical edges (marked by dashed arrows) like "fast food; has context; salads". based on these observations, we study plms that generate explanation graphs in an end-to-end manner and analyze their ability to learn the structural constraints as well as the semantics of such graphs. while a general recipe towards improving the structural and semantic aspects of graph generation can be via large-scale training with more humanannotated graphs, it is prohibitive under most practical scenarios because of the cognitive load associated with a complex data creation task like graph annotation @xcite . hence, we propose simple yet effective methods of graph perturbations that perform various kinds of node and edge addition, deletion, and replacement operations to construct structurally and semantically positive (correct) and negative (incorrect) graphs. overall, we leverage three types of negative graphs (synthetic structural, synthetic semantic, and human-created semantic) and develop multiple contrastive learning models @xcite @xcite for effectively distinguishing between correct and incorrect graphs. our first method is a generate-and-refine model that first generates an initial graph and further refines it using another t5 model. next, we propose two improved modelsone that uses the negative graphs in a max-margin formulation and another that uses both positive and negative graphs with a infonce (van den oord et al., 2018) contrastive loss. on two real-world tasks of explanation graph generation and temporal graph generation, with varied node and edge semantics, we observe that our proposed methods and graph perturbation techniques generalize well and lead to improvements in both structural and semantic accuracy of graphs. further analysis of different types of negative graphs reveal that the human-error graphs are the hardest, most diverse, and hence the best type of negatives to learn from in contrastive learning. hence, we also develop methods to automatically generate more such human-like semantic negative graphs, which leads to further improvements. we summarize our contributions as follows. ## motivation and background our primary task of interest is a recently proposed commonsense explanation graph generation task called explagraphs @xcite . in sec. 6.4, we also experiment with another related task of temporal graph generation @xcite . in both these tasks, the structural aspect deals with satisfying certain task-specific constraints on the graph (like connectivity) and the semantic aspect deals with the construction of meaningful edges (that adhere to commonsense). below we discuss explagraphs briefly and analyze pre-trained language models for their ability to generate explanation graphs. explagraphs @xcite . in this task, given a belief and an argument, an agent has to perform two sub-tasks -predict the stance (support/counter) and also generate an explanation graph explaining the stance. explanation graphs are structured explanations that capture explicit reasoning chains between the belief and the argument, thereby making models more interpretable. formally, an explanation graph is a connected dag with nodes as concepts and edges as commonsense relations between two concepts (see fig. 1 ). the concepts are either part of the belief or the argument (represented with solid boxes) or any external commonsense phrase (represented with dashed boxes). each edge in the graph forms a coherent sentence and the graph, when read as a whole, forms reasoning structures explaining why the argument supports or refutes the belief. saha et al. (2021b) evaluate explanation graphs by defining two accuracy metrics -(1) structural correctness accuracy (stca): fraction of graphs that satisfy all structural constraints, and (2) semantic correctness accuracy (seca): fraction of graphs that are both structurally and semantically correct. a graph is considered structurally correct if it satisfies the following constraints: (1) it is connected, ( 2 ) it is a dag, (3) the edge relations belong to a pre-defined list, (4) there are at least two concepts from the belief and two from the argument. if all these constraints are satisfied, the graph is next evaluated for semantic correctness by a model-based metric @xcite . it works on the principle that an explanation graph is semantically correct if the stance inferred from the belief and the graph matches the gold stance. refer to appendix a for a detailed description of all evaluation metrics. baseline t5 model. following prior work @xcite , we generate explanation graphs as post-hoc explanations by conditioning on the belief, argument and the predicted stance. 2 the stance prediction model is a fine-tuned roberta model @xcite which we keep unaltered from prior work and focus on the graph generation sub-task. we generate graphs as linearized strings in an endto-end manner by leveraging an encoder-decoder pre-trained language model, t5 @xcite . the input to the model is the concatenated belief, argument and the stance along with a prefix "generate an explanation graph for". the graphs are encoded as concatenated bracketed edges, in which the edges are ordered according to the depth first search (dfs) order of the nodes. while we choose t5 because of its superior performance @xcite , we do not make any model-specific assumptions and graphs can be generated via any encoder-decoder style pre-trained language model (e.g., see appendix e for results with bart). ## graph perturbations most prior works that collect human-annotated graphs for a downstream nlp task have found such collection processes to be quite expensive and tedious @xcite @xcite . for instance, @xcite obtained high-quality data only after multiple rounds of refinement and @xcite employ trained expert annotators for entailment tree construction. the corresponding datasets are also relatively small in size (2-3k), thus limiting the prospect of large-scale training. hence, our approach towards improving explanation graph generation is through data augmentation techniques that perturb human-curated graphs to construct positive and negative graphs. as noted earlier, we wish to construct graphs that enable better learning of structural graph constraints and their semantics. ## augmentation with perturbed graphs next we propose different methods of leveraging these positive and negative graphs for explanation graph generation. our models either use only positive graphs as simple data augmentation, only negative graphs in a max-margin model, or both in a generate & refine model and a contrastive model. ## experiments 6.1 impact of different models on graph structural and semantic accuracy in ## conclusion we presented an empirical study of graph structure and semantics for end-to-end explanation graph generation from pre-trained language models and showed that the generated graphs often violate structural constraints or are semantically incorrect. we significantly improve both the structural and semantic accuracy of graph generation by proposing contrastive learning models that leverage simple yet efficient methods of graph perturbations and also generalize to similar graph generation tasks.