Can In-context Learners Learn a Reasoning Concept from Demonstrations? (2023)

Abstract: Large language models show an emergent ability to learn a new task from a small number of input-output demonstrations. However, recent work shows that in-context learners largely rely on their pre-trained knowledge, such as the sentiment of the labels, instead of finding new associations in the input. However, the commonly-used few-shot evaluation settings using a random selection of in-context demonstrations can not disentangle models’ ability to learn a new skill from demonstrations, as most of the randomly-selected demonstrations do not present relations informative for prediction beyond exposing the new task distribution. To disentangle models’ in-context learning ability independent of models’ memory, we introduce a Conceptual few-shot learning method selecting the demonstrations sharing a possibly-informative concept with the predicted sample. We extract a set of such concepts from annotated explanations and measure how much can models benefit from presenting these concepts in few-shot demonstrations. We find that smaller models are more sensitive to the presented concepts. While some of the models are able to benefit from concept-presenting demonstrations for each assessed concept, we find that none of the assessed in-context learners can benefit from all presented reasoning concepts consistently, leaving the in-context concept learning an open challenge.

## introduction in-context learning (icl) is the alternative to the conventional training of large language models (llms) for specific task(s), where models are expected to learn a new task solely from the input text. in few-shot in-context learning that we focus on, the input text contains a set of demonstrations, i.e. the input-output examples of the task to be learned @xcite ). an ability to learn unseen tasks from natural instructions has practical and theoretical implications, figure 1 : in this work, we assess in-context learners' ability to improve when presented with demonstrations using a reasoning concept applicable in the prediction ( §2). we extract these concepts from human explanations ( §3.2) and assess models' ability to learn to use these concepts, as reflected in improving their prediction quality. recent llms trained on vast mixtures of tasks @xcite @xcite show a certain level of new-task icl and gradually bring more attention and expectations in this direction. however, counterintuitively to the overall evaluations, in-context learners (icls) also expose surprising behavioural artefacts; @xcite show icls' sensitivity to the ordering of in-context demonstrations. similarly, @xcite find surprising sensitivity of icls to the specific wording of the prompts. @xcite show that most of the model performance is persisted even when the contents of the demonstrations are randomly swapped. contrary to the ability to learn from input, @xcite propose to attribute this to the over-reliance of incontext learners on semantics of the label tokens, especially in smaller models. we find that the discrepancy in the perceived abilities of icls might be attributed to their limited evaluation, commonly performed with a random set of task demonstrations. however, for many open-ended tasks, such as question answering, or translation, randomly-chosen demonstrations rarely present a reasoning pattern which can help with the prediction of new input (figure 1 ; right). we argue that the evaluation with mostly non-informative contexts also can not reflect on the ability of learning, as observed in humans foot_0 , as the gain of extrapolating associations presented in non-informative demonstrations can only bring little benefit to the practice. we note that in the absolute numbers, the random-demonstrations evaluation also favours very large llms with a capacity to remember a wider variety of input distributions from pretraining; conditioned by the capacity, very large llms can better modulate the behaviour based on demonstrations' distribution, instead of learning new association(s) from the context. hence, in section 2, we propose to evaluate models' in-context learning ability primed with the demonstrations that exhibit a reasoning analogical to the one required for a robust prediction of the predicted sample (fig. 1 ). we measure how well can the recent few-shot learners utilize identified concepts for more accurate predictions ( §3) and find large discrepancies among the models and concepts. our main contributions are following: (i) we introduce a task of conceptual few-shot learning, disentangling models' ability to learn a new reasoning concept from other aspects of prediction quality. we show how such reasoning concepts can be extracted from human explanations. (ii) for a wide variety of recent in-context learners, we measure the ability to benefit from presented reasoning concepts. we show that while some models are better at learning concepts on average, this ability can not be attributed to the models' size or training strategy. problem definition given a dataset d : {(x 1 → y 1 ), .., (x @xmath0, an in-context few-shot learner θ(x) → y aims to predict a correct label y k+1 = y k+1 given a sequence of k input-output demonstrations, and the predicted input x k+1 : we expect in-context few-shot learner θ to model the relation of x i and y i by (i) identifying and (ii) extrapolating the relations of input and output presented in demonstrations. each such relation is modelled by one or more latent concepts c: we broadly define a concept c as any function c(x, y) → {0, 1}, constraining a space of valid outputs y to the ones where c(x, y) = 1. thus, if θ learns a concept c, it will never predict for x such y that c(x, y) = 0. in a composition {c} = {c 1 , .., c j }, all c @xmath1. given that modelling of each c valid for the task of d restrain a set of possible predictions of θ exclusively from incorrect predictions, extending a set of concepts learned in-context with complementary one(s) should never decrease the performance of the model θ on d. ## conceptual few-shot learning we reformulate in-context few-shot learning (1) to a conceptual few-shot learning, evaluating the ability of a few-shot learner θ to identify and apply a user-chosen reasoning concept c shown in demonstrations. first, we classify evaluation samples such that the samples of the same category x i require the concept c i to map x to y . subsequently, in conceptual few-shot learning, we let the learner to infer a prediction for input x k+1 by presenting it with demonstrations (x @xmath2..@xmath3, thus sharing the reasoning concept c i with the predicted input x k+1 : we note that θ can rely on other features than c i , and such reliance is not easy to disentangle. therefore, we propose to contextualize the results of conceptual few-shot learning on a concept c i a difference to the performance obtained in a random selection of demonstrations. additionally, to make the predictions based on two different sets of demonstrations mutually comparable without systematic bias (e.g. in samples' complexity), we perform both random and conceptsharing evaluations with the same predicted samples x k+1 , and only change the demonstrations foot_2 . informative concepts extraction constructing a scaled evaluation with annotated reasoning concepts c is challenging since the annotations of such concepts in associated with the datasets are rare. however, we find such reasoning inherently captured in human explanations of some datasets, where annotators are asked to collect answers to a question "why is [input] assigned [output]?" @xcite . the form of these explanations ranges from free-text explanations, including annotator-specific slang and stylistics, to semi-structured and structured explanations, cast to a pre-defined format, often consisting of a set of relations in a form "[sub-ject1] [relation] [subject2]" that transitively maps [input] to [output] @xcite . we focus on extracting the concepts from the subset of the semi-structured and structured explanations where the format consistency and non-ambiguity of the operands are reassured. ## evaluations this section introduces few-shot learners that we evaluate for conceptual few-shot learning and the datasets allowing us to extract reasoning concepts. ## results and discussion figure 2 shows the change of models' error between a random selection of demonstrations and conceptual few-shot learning, i.e. with demonstrations sharing a selected concept ( §2), ordered by models' size. for each of the assessed concepts, we observe statistically significant improvement for at least one of the models, which confirms our initial assumption on the informativeness of the extracted concepts in prediction. however, we can see that the selection of demonstrations makes a large difference in many cases, and the difference also largely depends on the inspected concept. following the results of specific models, we see many cases where the model is able to utilise one concept but fails to utilise, or even worsen the prediction then exposed to the other. the variance is larger for instruction-tuned tk-instruct models, excelling in utilising shared reasoning logic of glue, but to the contrary, degrading when being exposed to demonstrations supported by the shared facts in worldtree. contrary to these results is the case of instructgpt that is agnostic to concepts except for glue. figure 3 shows the average of changes of conceptual few-shot evaluation over the inspected four concepts. the aggregation uncovers that the gain from providing informative demonstrations largely varies among models, with t0 and smaller models (≤3b) benefiting from the presented concepts slightly more often; this could be caused by larger models' increasing reliance on their memo-rized knowledge. however, within the model-type groups, we also note that this trend is disputed by t0 and flan models. ## conclusion this work introduces a task of conceptual few-shot learning that reflects on in-context learners' ability to learn to apply a specific reasoning concept that can be informative for prediction. we assess a set of recent in-context learners for this ability over a set of concepts extracted from human explanations. we find that none of the learners can benefit consistently from all concepts, even though at least one of the other models proves the concept to bear an informative value. despite that, we still observe some interesting trends, such as the models of t0 are able to benefit from the concepts more often than others or that the concept-learning ability does not appear to relate to the model size. we believe the future work can inspire in identifying possibly complex reasoning concepts in the explanations of human annotators and will scale the conceptual evaluation to a wider variety of concepts. we trust that an evaluation with a comprehensive selection of the concepts will allow us to more realistically assess the abilities of the newlydesigned language models in the fast-progressing development of new in-context learners.