Testing the limits of logical reasoning in neural and hybrid models (2024)

Abstract: We study the ability of neural and hybrid models to generalize logical reasoning patterns. We created a series of tests for analyzing various aspects of generalization in the context of language and reasoning, focusing on compositionality and recursiveness. We used them to study the syllogistic logic in hybrid models, where the network assists in premise selection. We analyzed feed-forward, recurrent, convolutional, and transformer architectures. Our experiments demonstrate that even though the models can capture elementary aspects of the meaning of logical terms, they learn to generalize logical reasoning only to a limited degree.

## introduction despite the enormous successes of models based on deep learning, we still need to know more about how and what these models learn. the question of fundamental importance is to what extent they can 'grasp' the rules (or -more generally -the structure) governing involved data and tasks. it can be phrased as the problem of generalization, i.e., the ability to perform on data unseen during training. language structure is well understood from several perspectives: grammar, semantics, or rules of reasoning have been extensively studied and successfully formalized. however, even in this area, despite the available theoretical background, the methodology for studying generalization is still not well developed. the need for a systematic approach to this problem is indicated by a recent survey @xcite of generalization research in nlp. so far, the study of neural models for tasks related to logic and reasoning is rather limited. an early attempt is @xcite , where networks learn logical relations, such as entailment, between pairs of sentences in a simple artificial language. more recent work @xcite involves models that determine whether a given inference can be proved from a given set of premises by providing the list of inference rules as an output. in @xcite , models learn to reason with prescribed rules, while in @xcite , the authors consider models deciding whether a given set of sentences is consistent. it is worth mentioning that investigating reasoning has a sound linguistic motivation. to take a straightforward example, it is hard to argue that a model grasps the meaning of quantifier "all" if it is not able to perform reasonings of the form: "all a are b" and "all b are c" implies "all a are c." in this paper, we focus on logical reasoning in the syllogistic fragment of the natural language. the syllogistic logic has nice properties, e.g., soundness and completeness. notably, the logic is non-trivial but still sufficiently elementary to play the role of a benchmark for models of reasoning. we investigate the generalization of inference patterns in the training data and the following task. the network, presented with a knowledge base kb (i.e., a set of premises) and a hypothesis h selects the premises required to construct a proof of h from kb (if it exists); see figure 1 for an illustrative example. thus, one can think of our models as hybrid models: by selecting premises, the network assists the prover that is supposed to construct a proof. the paper can also be described as a study of reasoning in the presence of multiple premises, a research line rarely explored in deep learning. we are mainly interested in two aspects of generalization: recursiveness (elements can be iteratively combined) and compositionality (structures are determined by their constituents). it is worth emphasizing that they are frequently conflated even though conceptually different. there are fully recursive systems that are not compositional, the bestknown example being tarski's interpretation of first-order logic; see @xcite for a detailed discussion and more examples. there are also fully compositional structures with limited recursiveness, e.g., boolean operations on a finite family of sets are compositional but can be combined only in a finite number of ways. in the context of reasoning, we will say that a model processes inferences in a recursive manner if it is capable of applying inference patterns learned during training to more complex instances. going back to the previous example, if the model knows that "all a are b" and "all b are c" implies "all a are c," it should also be able to conclude from the extra piece of information "all c are d" that "all a are d." compositionality means the converse situation: provided that the model knows how to apply an inference pattern to complex instances, it should be able to do so for simpler ones. in other words, the derivation of "all a are d" from "all a are b," "all b are c," and "all c are d," should be accompanied by the derivation of "all a are c." in the study, we employed different types of architectures, multilayer perceptron, recurrent neural networks, convolutional neural networks, and transformers, to compare their performance and capabilities for generalization on artificially generated syllogistic corpora. on the surface of things, the models manage to learn the assigned task almost perfectly (see last but not least, one of our primary goals is to contribute to developing a methodology for investigating generalization in the context of recursiveness and compositionality. the approach proposed in this paper can be exploited in other settings, either directly related to reasoning, e.g., other fragments of language and inference systems, or not, e.g., sequence-to-sequence models studied in @xcite and lake and baroni (2023). ## syllogistic logic pratt-hartmann pratt-hartmann (2004) defines a fragment of a natural language as a subset of that language with an uncontroversial translation into a formal language that reconstructs logical entailment. the syllogistic fragment, first introduced and studied by aristotle, is the simplest non-trivial language fragment. aristotle considered only syllogisms consisting of two premises and a conclusion. a well-known example is "if all men are mortal and all greeks are men, then all greeks are mortal." however, classical syllogistic can be easily extended to inferences involving more than two premises, see, e.g., @xcite ; @xcite . in our setting, only general names with nonempty denotations are allowed. thus, "socrates is a man" is not a syllogistic formula for us, while "every unicorn is an animal" implies "some unicorn is an animal". ## synthetic data and neural models in order to avoid problems related to the choice of premises needed to infer a given hypothesis, we used only non-redundant knowledge bases, i.e., knowledge bases such that for every valid hypothesis, there is a unique minimal set of premises that proves it. we represent a knowledge base as a graph where vertices are constants and edges denote quantifiers (see figure 1a ). all a-formulas are a set of m disjoint trees t 1 , . . . , t m (or a forest). each tree is a directed graph t @xmath0, e) such that there is at most one path between any two vertices. we created synthetic consistent non-redundant knowledge bases kb for training and testing the neural models using the following general algorithm: ## experiments and results the data and scripts to run these experiments are available online foot_0 . we randomly generated 5 consistent knowledge bases. each of them consists of 4 trees, 66 constants, and 78 formulas. we made sure that no valid hypothesis gave rise to the same label in two different knowledge bases. in the following experiments, we trained 4 different architectures of neural models: multilayer perceptron (mlp), recurrent neural network (rnn), convolutional neural network (cnn), and transformers (tra) (as a matter of fact, we also considered some variants of these architectures, e.g., lstm or gru, but the results were very similar). we employed gridsearch techniques to optimize the configurations for overall performance. the detailed description of optimization procedures and final specifications can be found in a.3. we trained each knowledge base for 3 runs. being part of a hybrid model, networks are supposed to provide premises for the prover. therefore, beside the standard measure of accuracy (correct label), we consider another one: an output is correct if it involves all the necessary premises, i.e., it is a correct but not necessarily minimal (nnm) inference. ## discussion the paper's main contributions are two-fold: methodological and experimental. studies of logical reasoning in neural networks usually consider much simpler toy logic examples, often not even fully recursive, than the experimental setup offered in this paper, cf. @xcite . on the other hand, articles focusing on various aspects of generalizations, like compositionality, systematicity, or recursiveness, often adopt empirical frameworks less straightforwardly linked to reasoning and semantics, cf. @xcite or @xcite . moreover, many papers do not distinguish between recursiveness and compositionality. for example, in lake and baroni (2023), a sequence-to-sequence model's performance on unseen combinations of functions is tested (see fig. 2 in the paper); however, it is not verified whether the model can correctly process corresponding sub-combinations, which is a necessary condition for compositionality. similarly, in @xcite , the authors investigate the generalization of certain rule-based reasonings to patterns longer than those seen in training (see the current paper proposes solving these problems by a systematic study of reasoning in a natural language fragment @xcite . our experiments show that even though the neural net-work models can grasp some elementary aspects of syllogistic reasoning, they cannot learn the logic's fully recursive and compositional nature. they manifest various aspects of the meaning of involved terms, e.g., the principle of contradiction, non-emptiness of denotations, or the symmetry of quantifiers. they also exhibit some ability to combine inferences into more complex ones, which agrees with findings, e.g., from lake and baroni (2023). at the same time, they do not assimilate the recursive structure of inferences, so high performance on shorter inferences of a given type does not translate to high performance on longer ones (see @xcite for similar results for the task of recognizing consistency of a set of formulas). moreover, the networks do not pass the compositionality test: they appear to apprehend complex inferences without apprehending the constituent subinferences. from the semantical perspective, this shows that the models do not understand the meanings of syllogistic formulas because, ultimately, it is their meanings that determine the structure of syllogistic inferences.