Blackbird language matrices ( BLM ), a new task for rule-like generalization in neural networks: Can Large Language Models pass the test? (2023)

Abstract: How do we evaluate Large Language Models (LLMs) and determine the aspects and limits of their intelligent behaviour? It is currently conjectured that shortcomings of LLMs in multi-linguality and reasoning are due to a lack of ability to generalize. It has been argued that, instead, humans are better at generalization because they have a tendency at extracting rules from complex data. We propose a method to evaluate LLMs ability to rule-based generalization. When exposed to tests of analytic intelligence, for example the visual RAVEN IQ test, human problem-solvers identify the relevant objects in the picture and their relevant attributes and reason based on rules applied to them. Based on the induced rules, they are able to provide a generalisation and a solution to the test. An analogous language task has recently been proposed (called BLM) for LLM. In this paper, we argue that we can use this task to investigate what linguistic reasoning LLM develop, by asking them to solve some simple variants of the BLM task. We find that current state-of-the-art generative models, such as ChatGPT, can handle the task in the sense that they easily understand the instructions and can provide step-by-step reasoning that shows that it can solve two of the main cognitive hurdles: correspondence finding (object and attribute identification) and item novelty. However, overall they cannot find the correct answer, even with considerable help. In particular, they never identify the structure of the problem, exhibiting, we hypothesize, a lack of goal and subgoal management abilities, an ability that has been argued to measure differential abilities in humans. We argue that this finding supports the usefulness of the task as a method to test the limits and specific properties of generalisation ability in Large Language Models, providing an intrinsic evaluation method inspired by tests of human intelligence.

## prologue how do we evaluate llms, determine the aspects and limits of their intelligent behaviour and compare them to human intelligence? a new trend of large-scale multi-task llm and neural network evaluation has emerged that aims to evaluate llm in realistic settings. in nlp we see big-bench initiatives, to better chart llms capabilities @xcite . in neuroscience, a similar trend, dubbed integrative bench-marking, looks at large language models as mechanistic hypotheses on how the brain encodes intelligence, models that can be tested on behavioural and neural data @xcite . finally, another point of view has been put forth recently that, to further understand the components of human intelligence, one needs to study how this intelligence evolved, decomposing it into large-scale studies of primitive skills that humans share with other animals @xcite . large scale, highly integrative, comparative approaches are necessary: they give a bird's eye view essential to map unknown territories, define boundaries, guide long-term trends. but we also think that curated data and in-depth qualitative analyses of the intelligent behaviour of these models are needed @xcite . we will concentrate on one such skill, one of the core intelligent human abilities, the ability for analytic rule-based generalisation. we will illustrate what tests exists for this skill and evaluate qualitatively how gpts rate on that, based on carefully curated data. ## rule-based generalisation recent approaches to generalisation have shifted attention from the traditional narrow definitionan extension from a set of data points to new data points of exactly the same nature (i.i.d. assumption) (schölkopf, 2019)-to out-of-distribution generalisation, be it across languages, across domains, or new unseen combinatorial tokens and structures @xcite . humans are good generalizers. a large body of literature of experimental work has demonstrated that the human mind is predisposed to extract regularities and infer the generative rules from data, in a way that is distinct from the patterns of activation of neural networks @xcite @xcite . one possible approach to develop more robust methods, then, is to pay more attention to the decomposition of complex observations and to the causal chains in the generative process that gives rise to the data. to discover the underlying problem structure, the notion of disentanglement comes to the rescue. a disentangled representation can be defined as one where single latent units are sensitive to changes in single generative factors, while being relatively invariant to changes in other factors @xcite . let's look at an illustrative example of complex linguistic relations that might require discovering the underlying disentangled factors and causal relations: the causative alternation exhibited by english verbs, shown in (1). agent theme the door opened. theme this alternation applies to change of state verbs, such as open, break, melt, burn among many others, verbs that describe a change that affects the state of the undergoing participant (the door changes from a state of being closed to a state of being open). they occur in two subcategorisation frames that are related to each other in a regular way: the object of the transitive frame is the subject of the intransitive frame. this way, in terms of semantic roles, the subject of the transitive is an agent, but the subject of the intransitive is a theme. to learn the structure of such a complex alternation automatically, despite the simplicity of the sentences, a neural network must be able to identify and solve a set of subproblems and compose them into an organised solution: (i) identify the elements manipulated by the alternation (teacher, door); (ii) identify the relevant attributes of these elements (open, closed state of door; volition of teacher); (iii) the object of the transitive sentence is the same event participant as the subject of intransitive, that is the rules of change of the attributes must be identified across the two sentences. to study what factors and models lead to learning more disentangled linguistic representations, representations that reflect the underlying linguistic rules of grammar, a possible approach is to develop curated data on a large scale, building models to learn these data and investigating the models' behaviour. to this end, a new linguistic task has been developed, (similar to the iq test called rpm, @xcite progressive matrices for vision), called blackbird language matrices (blms) @xcite . blms define prediction tasks to learn complex linguistic patterns and paradigms. because this task relies on a mixture of language and abstract rule learning abilities, we conjecture that, used in an evaluation setting, it is appropriate to investigate the limits and properties of linguistic intelligence and abilities in current state-of-the-art llm, and compare them to humans. ## how do humans solve rpms? raven's progressive matrices are a visual test of analytic intelligence @xcite . to establish how to develop human-like rule-based inference, it pays to survey how humans solve this kind of intelligence test. ## blackbird language matrices like rpms, a blm instance consists of a context set and an answer set. the context is a sequence of sentences that encode a linguistic rulee.g., subject and verb number should agree in their number independently of how many noun phrases intervene between them. blms are presented (to an artificial language model or to a human participant) as linguistic puzzles requiring the selection of the missing sentence. in order to examine the representations underlying the response, the answer sets include negative examples following corrupted generating rules. an example template is illustrated in figure 2 . for example, the blm problem for subject-verb agreement @xcite consists of a context set of seven sentences that share the subject-verb agreement phenomenon, but differ in other aspects -e.g. number of intervening attractors between the subject and the verb, different grammatical numbers for these attractors, and different clause structures. the answer set comprises contrastive sentences that violate some of the generative rules. actual examples can be found in section 5. ## quantitative analysis the interaction illustrated in the previous section are a limited sample. we also developed a controlled sample of interactions, and analyse it quantitatively. we use chatgpt-4 (between the 28th and the 30th of september, 2023). ## further related work traditionally, progressive matrices as intelligence tests are designed by hand, but recent research in vision that has used this task to train neural networks has typically employed some structured generative model to create a large number of questions. wang and su (2015) generate rpms using firstorder logic, which have further inspired @xcite al. (2018a) the controlled construction of the context data and of the contrastive answer set is crucial. in this way, the learning process basically discovers how to induce the model and it makes it possible to identify clear dimensions of successful and unsuccessful generalisation. for example, matrices for vision have shown that the best models can apply known abstract relationships in novel combinations, but fail in applying known abstract relationships to unfamiliar entities @xcite . ## a roadmap for the future the investigation reported here aims to prove a concept: curated data and in-depth qualitative analyses inspired by human intelligence tests can complement large-scale benchmarking in an interesting way. in particular, large-scale benchmarking based on correlations of humans and machines does not enlighten us on the causal sources of behaviours, and instead hypothesis testing experiments need to be performed. it is also interesting to collect data about human and machine overt knowledge, rather than phychophysical measures (eye-tracking, brain activations) or internal ann layer representations. a larger-scale investigation applying the curated data and qualitative method presented here is however also possible. we need to develop multiple datasets and run several interactive experiments that need then to be measured and assessed by proper statistical methods. ## ethics statement to the best of our knowledge, there are no ethics concerns with this paper. ## limitations our preliminary conclusions about human-machine correspondence sketch the avenue for future developments. the work presented here is meant as a programmatic call. the main limitations concern limitations in the blm development, in the range of examined architectures and the level of internal analysis, and in the range of cognitive concepts investigated. blms are being developed with larger-scale multi-linguality and greater variety of linguistic phenomena. work is also planned to study how they affect internal layers of llms.