Conditional Natural Language Inference (2023)

Abstract: To properly explain sentence pairs that provide contradictory (different) information for different conditions, we introduce the task of conditional natural language inference (Cond-NLI) and focus on automatically extracting contradictory aspects and their conditions from a sentence pair. Cond-NLI can help to provide a full spectrum of information, such as when there are multiple answers to a question each addressing a specific condition, or reviews with different opinions for different conditions. We show that widely-used feature-attribution explanation models are not suitable for finding conditions, especially when sentences are long and are written independently. We propose a simple yet effective model for the original NLI task that can successfully extract conditions while not requiring token-level annotations. Our model enhances the interpretability of the NLI task while maintaining comparable accuracy. To evaluate models for the Cond-NLI, we build and release a token-level annotated dataset BioClaim which contains potentially contradictory claims from the biomedical domain. Our experiments show that our proposed model outperforms the full cross-encoder and other baselines in extracting conditions. It also performs on-par with GPT-3 which has an order of magnitude more parameters and trained on a huge amount of data.

## introduction we introduce the task of conditional natural language inference (cond-nli) that extends the traditional natural language inference (nli) to be more suitable for finding a full spectrum of information. the original nli task involves inferring an entailment or contradiction relationship between a pair of sentences: a premise and a hypothesis. this is typically modeled as a three-way classification, predicting if the premise entails, contradicts, or remains neutral to the hypothesis. interpretation losartan prevents more cardiovascular morbidity and death than atenolol for a similar reduction in blood pressure and is better tolerated. we develop a modeling framework to capture the relationship between a pair of sentences that provides different answers under diverse conditions. such sentence pairs are henceforth referred to as conditionally-compatible, since none of the entailment, contradiction, or neutral classes of nli precisely describes their relationship. cond-nli includes two token-level tasks -one is to identify contradictory tokens that embody contradictory aspects and the second is to identify neutral tokens that indicate conditions that are not en-tailed by the other sentence. the focus of this study is to determine different conditions in a pair of conditionally-compatible sentences. for the example pair in automatic identification of different conditions in conditionally-compatible sentence pairs allows us to summarize and provide a full spectrum of answers in a form where users are not overloaded with excessive information. this is of particular practical importance as it has shown that there are usually multiple answers to a user's question in different domains, such as biomedical @xcite , and factoid question-answering @xcite , where the difference between answers/opinions is their provided conditions. we propose partial-attention model, pat, a simple yet effective model for natural language inference that can address the cond-nli task. pat predicts an nli label for a sentence pair from the intermediate labels for their partitions. the intermediate labels for partitions of sentences can be subsequently used to attribute these labels into the token-level. the nli token-level attributions align closely with the objective of cond-nli . different conditions in a claim pair would cause an nli model to predict the pair to be neutral. thus, identifying the tokens responsible for triggering neutral labels could serve as a technique to detect different condition tokens in cond-nli. similarly, contradictory tokens of cond-nli can be attained from attributing contradiction label in nli. finally, pat effectively solves cond-nli through training with sentence-level nli data, without requiring taskspecific token-level annotations. to evaluate different models for cond-nli, we build (and make publicly available) the bioclaim dataset, an extension of an existing corpus initially built to assist systematic reviews @xcite . the bioclaim dataset provides a challenging benchmark for the nli models. in contrast to the sci-entsbank dataset @xcite , which lacks contradictory sentence pairs, bioclaim includes conditionally-compatible sentence pairs. such pairs require the identification of neutral tokens in the presence of contradictory tokens. compared to other token-level explanation datasets such as e-snli and mnlitag @xcite , which are built on nli corpora @xcite , bioclaim has longer hypothesis sentences. this characteristic introduces additional complexity in the selection of non-entailed tokens. perturbation-based methods @xcite have shown to be effective in identifying tokens that contribute to contradiction or neutral labels when evaluated on e-snli @xcite or mnlitag @xcite . however, we show that these perturbation-based explanation models face challenges in accurately identifying condition tokens when hypothesis sentences are long and contain a large number of nonentailed tokens (conditions in cond-nli). extensive experiments on the bioclaim and scientsbank @xcite datasets show that our pat significantly outperforms strong and state-of-the-art baseline models. against intruct-gpt @xcite and chatgpt (ope-nai, 2022), pat shows better performance on the scientsbank dataset and comparable performance on the bioclaim dataset, while pat has a significantly smaller number of parameters. while our pat model slightly underperforms the crossencoder bert model on the original nli task, its enhanced interpretability enables effective finegrained token-level inference required for cond-nli. ## partial-attention nli model the typical effective approach for text-pair classification, such as the nli task, using transformerbased language models such as bert @xcite , is by concatenating the text pair as input, which we refer to as full cross-encoder bert. specifically, cross-encoder bert takes the concatenation of premise p and hypothesis h, denoted by p • h, taking the [cls] token vector as sentence representation, and outputs classification probability y as: output y in the nli task is a 3-dimensional vector representing the probabilities of the entailment, neutral, and contradiction classes. gross national saving has increased over the past 5 years. ## conclusion we proposed pat, a partial attention model, capable of attributing the model decision into the parts of input. using pat, we address the cond-nli task, a token-level prediction task that explains conditionally-compatible claims. we built the bioclaim dataset for cond-nli . the proposed method shows the accuracy up to 8% higher than the best nli-based baseline method in predicting condition tokens.