Zero-Shot Classification by Logical Reasoning on Natural Language Explanations (2023)

Abstract: Humans can classify data of an unseen category by reasoning on its language explanations. This ability is owing to the compositional nature of language: we can combine previously seen attributes to describe the new category. For example, we might describe a sage thrasher as “it has a slim straight relatively short bill, yellow eyes and a long tail”, so that others can use their knowledge of attributes “slim straight relatively short bill”, “yellow eyes” and “long tail” to recognize a sage thrasher. Inspired by this observation, in this work we tackle zero-shot classification task by logically parsing and reasoning on natural language explanations. To this end, we propose the framework CLORE (Classification by LOgical Reasoning on Explanations). While previous methods usually regard textual information as implicit features, CLORE parses explanations into logical structures and then explicitly reasons along this structure on the input to produce a classification score. Experimental results on explanation-based zero-shot classification benchmarks demonstrate that CLORE is superior to baselines, which we show is mainly due to higher scores on tasks requiring more logical reasoning. We also demonstrate that our framework can be extended to zero-shot classification on visual modality. Alongside classification decisions, CLORE can provide the logical parsing and reasoning process as a clear form of rationale. Through empirical analysis we demonstrate that CLORE is also less affected by linguistic biases than baselines.

## introduction humans are capable of understanding new categories by reasoning on natural language explanations @xcite . for example, in figure 1 , we can describe sage thrashers as "having a slim straight relatively short bill, 1 code and data will be made publicly available upon publication "this is a short-billed bird with yellow eyes and a long tail." ## logical parsing and reasoning explanation-based classification is, in essence, a bilateral matching problem between inputs and explanations. instead of simply using similarity or entailment scores, in this work we aim at better utilizing the logical structure of natural language explanations. a detailed illustration of our proposed model, clore, is shown in figure 2 . at the core of the approach is a 2-stage logical matching process: logical parsing of the explanation (figure 2(d) ) and logical reasoning on explanation and inputs to obtain the classification scores (figure 2(e) ). rather than using sentence embeddings, our approach focuses more on the logical structure of language "the sage thrasher has a slim straight relatively short bill, yellow eyes and a long tail." ## experiments on zero-shot classification in this section we conduct in-depth analysis of our proposed approach towards zero-shot classification with explanations. we start with a latest benchmark, clues @xcite , which evaluates the performance of classifier learning with natural language explanations. clues focuses on the modality of structured data, where input data is a table of features describing an item. this data format is flexible enough for computers on a wide range of applications, and also benefits quantitative analysis in the rest part of this section. ## conclusions and future work in this work, we propose a multi-modal zero-shot classification framework by logical parsing and reasoning on natural language explanations. our method consistently outperforms baselines across modalities. we also demonstrate that, besides being interpretable, clore also benefits more from tasks that require more compositional reasoning, and is more robust against linguistic biases. there are several future directions to be explored. the most intriguing one is how to utilize pre-trained generative language models for explicit logical reasoning . another direction is to incorporate semantic reasoning ability in our approach, such as reasoning on entity relations or event roles.