Experimental Contexts Can Facilitate Robust Semantic Property Inference in Language Models, but Inconsistently (2024)

Abstract: Recent zero-shot evaluations have highlighted important limitations in the abilities of language models (LMs) to perform meaning extraction. However, it is now well known that LMs can demonstrate radical improvements in the presence of experimental contexts such as in-context examples and instructions. How well does this translate to previously studied meaning-sensitive tasks? We present a case-study on the extent to which experimental contexts can improve LMs’ robustness in performing property inheritance—predicting semantic properties of novel concepts, a task that they have been previously shown to fail on. Upon carefully controlling the nature of the in-context examples and the instructions, our work reveals that they can indeed lead to non-trivial property inheritance behavior in LMs. However, this ability is inconsistent: with a minimal reformulation of the task, some LMs were found to pick up on shallow, non-semantic heuristics from their inputs, suggesting that the computational principles of semantic property inference are yet to be mastered by LMs.

## introduction carefully controlled behavioral analyses on meaning-sensitive tasks have revealed holes in the ability of language models (lms) to demonstrate robust meaning extraction and use @xcite @xcite @xcite . however, since a large subset of these investigations uses zero-shot evaluation as the primary methodology, there are growing concerns that they do not paint a complete picture of lms' abilities @xcite . conclusions that lms lack a particular ability may be overhasty if it turns out the ability is easily accessed through in-context learning, different question formulations, or particular instructions @xcite . figure 1 : lms are prompted with in-context examples that are compatible with both, robust property inheritance, as well as position-based heuristics. at test time, we evaluate on cases where the heuristics support desirable behavior and on cases where they do not. we use stimuli from comps and its reformulation as a qa task. our focus 1 in this paper is a particularly challenging dataset for meaning-sensitive behavior: comps @xcite , which contains minimal pair sentences that test the ability of lms on property knowledge of everyday concepts (a beaver/gorilla has a flat tail) and their inheritance for novel concepts (a wug is a beaver/gorilla. therefore, a wug has a flat tail). contemporary lms failed miserably on the hardest subset of the comps stimuli, the examples of which contain two novel concepts (wug vs. dax), where only one of them inherits the target property (has a flat tail): given the success of lms on a wide variety of complicated tasks, their utter failure on this seemingly straightforward task remains puzzling. here, we systematically explore comps on 12 lms ranging from 1.5-13b parameters, varying (a) whether they are evaluated zero-shot or with multiple examples, and (b) whether or not instructions are present. unlike other minimal-pair datasets, using comps in an in-context learning setting is nontrivial (and thus potentially informative). this is because the task can be solved using a positionbased heuristic. for example, in one subset of comps, the target property is always attached to first novel concept-like in (1). importantly, lms' failures on comps were shown to be in part a result of models' tendencies towards heuristic behavior: the performance of autoregressive lms was found to be particularly bad when the distractor (a dax is a gorilla) is recent-i.e., they showed a recency bias in attributing properties to novel concepts. in that sense, comps follows a rich body of work in which tasks are set up in a manner that two types of generalization mechanisms can lead to the same prediction, but only one of which is desirable @xcite @xcite @xcite . we find that experimental contexts, as operationalized using in-context examples and instructions, can in fact demonstrate robust improvements in lms' property inheritance behavior as measured by comps. however, this improvement comes with a caveat: with a minimal reformulation of comps into a qa task, where there is a direct link between the lms' output space and the features of the input that control the heuristic, many lms showed a strong preference towards the heuristic, and were therefore at chance. interestingly, lms that receive explicit supervision to follow instructions do show some resistance to positional heuristics, but not always-occasionally, certain instruction-tuned lms also tend to show a preference for one particular type of heuristic. these discrepancies in lms' performance underscore their difficulty of mastering the reasoning ability to robustly demonstrate semantic property inheritance. ## methodology dataset we use the most difficult subset of the comps dataset @xcite )-comps-stimuli re-design we take a number of steps to minimize noise from other (likely uninterpretable) heuristics beyond the ones we have set out to target. first, we enforce that the concepts and properties that appear in the in-context examples are disjoint from ones that are used in tests. to this end, we sample 50 concepts and their relevant properties and reserve it for our in-context examples, leaving the rest to be sampled for our test set. we also enforce this constraint for our novel conceptsi.e., all in-context examples contain different nonce words, and the collection of nonce words for the in-context examples and the test set is disjoint. furthermore, we counterbalance the nonce words in the test set in a manner that having a bias towards one of them would lead to chance performance. we additionally also use multiple different sets of incontext examples, to add variability and to ensure that the results are not only due to one particular choice of in-context examples. in total, we use 10 different in-context learning example sets, each containing 6 different comps stimuli. for our test set, we use a constant set of 256 unique pairs sampled from our pool of stimuli containing unused concepts and properties. heuristics our most important design decision is to consider two distinct sets of stimuli-each separately making available the two types of heuristics that the lms could rely upon: first-correct and recent-correct, where the property is inherited by the first and the most recent novel concept, respectively. that is, for the same set of in-context examples, we have a version where the first concept is correct like in (1), and one where the most recent concept is correct: for each type of in-context stimuli, we similarly have two versions of test stimuli: one that is consistent with the target heuristic, and one that is not. that is, a test example that is consistent with the first-correct heuristic will also have its first concept be the one that inherits the property in question, while one which is inconsistent will have the most recent concept be the inheritor of the property. therefore, a model that shows a preference for a given heuristic will succeed only on one test set and succumb on the other, while a model that is robust to the heuristics will succeed on both. reformulation into qa the original comps stimuli test for property inheritance using declarative statements, where models are tested for the log-probability they asign to the property (has a flat tail) given either of the two concepts (wug vs. dax). here we additionally consider an alternate formulation of comps as a question answering task (comps-qa), where we make the property explicit in the prompt to the model and instead ask which of the two concepts possesses it: since the shallow heuristics we consider are controlled by the relative ordering of the novel concepts, this formulation of the task directly allows us to link the models' output space (the novel concepts) to the heuristics (positions). testing setup for the original comps setting we follow @xcite and compare the logprobability of the property phrase given the correct vs. the incorrect prefix. for comps-qa however, since we have a constant prefix (same premises and question), we evaluate the relative log-probability of the two novel concepts, only one of which is the correct answer. accuracy in both cases is the proportion of cases the correct surface form was assigned relatively higher log-probability. since we use pairwise comparisons throughout, chance performance is 50%. instructions we consider four different kinds of instruction templates, with varying levels of detail (see appendix b) per formulation (comps and comps-qa). in our experiments we report results on the instruction that gives the best average performance for a given model. ## analyses and results we evaluate on comps and comps-qa, with and without instructions. in each case, we progressively supply 0 through 6 in-context examples, allowing us to track the dynamics of the models' performance with an increasing amount of demonstrations. together with our separate types of test sets and heuristics encoded in the in-context examples, along with five different instruction settings (four with and one without) we run 2420 experiments per lm. we hypothesize that lms would be more sensitive to the positional heuristics in comps-qa because of the clear link between their output space and the relative position of the novel concepts-the feature that controls our target heuristics. figure 2 shows accuracies of the tested lms on our four different comps settings as a function of the number of in-context examples provided to them, for both: cases where the heuristics are consistent with success on the test set, and cases where they are not. we also show an additional curve denoting the average performance across the two types of test sets to paint an overall picture of the models' performance. in this figure, the extent to which a model relies on heuristic is indicated by the gap between the dotted ( ) and the dashed (▲) lines. a model that is robust to the heuristics will have curves of both colors rise above chance, with no gap between the two, while one that is prone to using heuristics will have its dotted ( ) curve be substantially greater than its dashed (▲) curve. following the same format, figure 3 shows results on the four instruction-tuned models evaluated. in all these plots, we also visualize the lms' 'base' property knowledge, where we compare (in a zeroshot manner) the lms' preference for attributing the properties in the test set to the correct 'known' concepts (i.e., beaver vs. gorilla in our examples). experimental context can improve attribution of properties to concepts... on comps, models unsurprisingly start off at chance performance on average, corroborating the previous findings of @xcite . however, in the presence of in-context examples and instructions, they are able to improve monotonically as the number of in-context examples increases, reaching similar levels of performance on property inheritance as that of the models' base knowledge. it is worth noting llama-2-13b and olmo-7b do occasionally show a preference for heuristics in the absence of instructions. an intermediate conclusion that we draw here is that lms can indeed demonstrate non-trivial property inheritance on observing a few examples that reflect that behavior. ...but not the attribution of concepts to properties while experimental context seems to aid models in attributing properties to the right concept in context, the same does not hold on comps-qa. similar to comps, models start off at chance performance on average with a zero-shot set up, however, unlike in the case of comps, many lms seem to consistently prefer the heuristics available in the prompt, showing, at times, worse than chance performance on cases where the test set does not follow the heuristic. this is most apparent for gpt-2 xl, opt6.7b, llama-2-7b, olmo-7b-here there is a substantial gap between the accuracy for cases where heuristics support performance on the test set and the accuracy for cases where they do not. other models, like mistral-7b and gemma-2-9b occasionally show behavior compatible with the heuristics, but this does not hurt their overall performance, which is consistently above chance, and close to their base knowledge. our results suggest that lms are more likely to show behavior that is compatible with the use of positional heuristics when their output space (choice between the two novel concepts) has a clear connection with positional artifacts in their input (relative ordering of the novel concepts). this is consistent with our hypothesis in about 11 out of 16 cases (where for 9 of the cases, heuristics end up drastically hurting performance). when this link is not clear and models must instead predict likely properties given a novel concept (i.e., in comps), instructions and in-context examples do seem to lead to robust performance. it is important to note that instructions alone do not always account for the observed improvement-lms' performance on zero-shot settings are consistently still at chance in all cases, suggesting that it is their interaction with in-context examples that critically alters models' output distribution to support desirable property inference behavior. ...and its occasional pitfalls while figure 3 shows that lms benefit from instruction-tuning in the context of robust property inheritance, a detailed breakdown of the results offers a few caveats. in figure 4 we show olmo-7b-instruct's accuracies broken-down by heuristic on comps-qa, the setting where we hypothesized lms to be most likely relying on heuristics. for the first-correct heuristic, the model shows a preference for the heuristic (greater performance when it supports success on the test set), while for recent-correct, we see the opposite to be true. here, we observe that the lm does better when the heuristic does not support success on the test set, suggesting that the lm has a bias for preferring entities mentioned first-a bias so strong that it persists even when there is ample evidence in the context ## conclusion we investigated the extent to which in-context examples and instructions-key components that drive impressive performance in contemporary lms-can overcome important limitations of lms at tests that have poked holes in their ability to extract conceptual meaning from text. as a case study, we analyzed how well such experimental contexts can improve lm abilities to perform property inheritance @xcite in context: binding of novel concepts to existing concepts, and endowing them with valid property inferences as a result. our findings suggest that mastery of this ability has yet to be robustly achieved, and that many lms trained on the next-word prediction objective are still prone to using shallower patterns in their context rather than systematically extracting conceptual meaning. there are several exceptions to this, however, for example, modern lms such as mistral-7b and gemma-2-9b did show heuristic behavior, but were also well above chance, especially in the presence of instructions. additionally, instruction tuned lms were found to be substantially more robust than their non-instruction tuned counterparts, suggesting advantages to explicit instruction tuning. however, they too sometimes showed counterintuitive behavior, where their strong bias for a consistent position overcomes the heuristic patterns available in the input. we leave the nuanced exploration of this behavior induced by instruction-tuning for future work.