Characterizing the Role of Similarity in the Property Inferences of Language Models (2025)

Abstract: Property inheritance—a phenomenon where novel properties are projected from higher level categories (e.g., birds) to lower level ones (e.g., sparrows)—provides a unique window into how humans organize and deploy conceptual knowledge. It is debated whether this ability arises due to explicitly stored taxonomic knowledge vs. simple computations of similarity between mental representations. How are these mechanistic hypotheses manifested in contemporary language models? In this work, we investigate how LMs perform property inheritance with behavioral and causal representational analysis experiments. We find that taxonomy and categorical similarities are not mutually exclusive in LMs’ property inheritance behavior. That is, LMs are more likely to project novel properties from one category to the other when they are taxonomically related and at the same time, highly similar. Our findings provide insight into the conceptual structure of language models and may suggest new psycholinguistic experiments for human subjects.

## introduction categories are fundamental to human semantic cognition. our knowledge of categories allows us to draw everyday inferences; a prominent example of which is property inheritance, where properties are projected from a category to its members. for example, if we learn that dogs have the t9 hormone, we can reasonably assume that corgis also have the t9 hormone. in cognitive psychology, an obvious and previously popular explanation for property inheritance is that it is the natural consequence of our minds organizing categories hierarchically into taxonomies @xcite @xcite . figure 1 : property inheritance involves projecting properties from a category to its members. for example, if dogs are daxable, are corgis daxable? here, daxable is a nonce word used to study property inheritance without any confounding effects from parametric knowledge. language models may rely on taxonomic (left) and/or similarity (right) relations to perform property inheritance. we investigate the interplay between these two effects in lms' property inheritance judgments using both behavioral and mechanistic analyses. at the same time, empirical evidence has called this assumption into question. @xcite showed that humans were highly sensitive to the similarity of the categories when performing property inheritance; for example, they were more likely to project novel properties from birds to robins (a more typical bird) than from birds to penguins (a less typical bird), despite them agreeing that both were members of the bird category. sloman concluded that instead of explicitly using stored taxonomies, humans might simply be computing categorical similarities to demonstrate inheritancecompatible behavior. in such a case, a property is more likely to be shared between categories insofar as the similarity between them is high enough. debates about the mechanisms that underlie human property inheritance offer an exciting avenue to analyze conceptual organization and use in language models (lms). first, due to the close connec-tion of a word to its conceptual meaning @xcite @xcite , property inheritance provides an opportunity to diagnose meaning-sensitivities in lms @xcite . next, similarity effects like those found by @xcite are related to the notion of content effects shown by humans and lms on logical reasoning tasks @xcite @xcite ; the presence of similarity effects goes against the idea that humans rely on abstract taxonomic principles (where every category member would be equally likely to inherit the property). finally, the principles of taxonomy vs. similarity offer two clear hypotheses about what mechanisms could govern property inheritance behavior in a system. this allows us to go beyond prior work investigating property inheritance in lms @xcite , which has largely focused on their behavior, leaving open questions about the underlying mechanisms. in this study, we empirically investigate the roles of taxonomic relations and categorical similarities on lms' property inheritance behavior, using both behavioral and causal interpretability methods on four lms. first, we behaviorally analyze lms' sensitivity to taxonomic relations when performing property inheritance ( §3.4). we demonstrate that noun similarity, known to correlate with human property inheritance behavior @xcite , also correlates strongly with property inheritance judgments in language models, and explains many of the false positives and false negatives. then, we causally localize property inheritance to specific activation subspaces ( §4) using distributed alignment search @xcite . by training and evaluating das on controlled subsets of the data, we show that the subspaces responsible for property inheritance reflect both taxonomic and similarity features. together, these results indicate that lms are susceptible to non-trivial content effects rooted in noun similarities, and moreover that taxonomy and similarity are fundamentally entangled in model representations. ## conceptual organization in lms to what extent is the similarity vs. taxonomy debate relevant to lms? by definition, lms learn representations from distributional statistics of tokens in context, so any reasoning they demonstrate is a result of distributional similarity. while technically true, this viewpoint may be an oversimplification: two words can be distributionally similar if they participate in antonymy (hot vs. cold), metonymy (car, wheel), hypernymy (robin, bird), or even if they share thematic relations (dog, bone). for property inheritance, however, it is only the hypernymy relation that plays a critical role @xcite . is it possibile that something structural like a taxonomy might arise through distributional statistics? it has been postulated that symbolic behavior can emerge as a result of sub-symbolic processes internal to a neural network @xcite @xcite . empirically, this has been shown in a number of recent works, e.g., @xcite on lms and modular arithmetic, and @xcite on lms learning to simulate dynamic programming-both of which require operating over symbolic representations. the above discussion raises multiple interrelated questions: when performing property inheritance, do lms show sensitivity to taxonomic relations? if so, is their behavior graded, where properties are less likely to be projected to concepts that are taxonomically related to the higher-level category but low in similarity? or, is it more binary, where there is no significant difference between how likely a property is projected from the higher-level category to its high-and low-similarity members? if their behavior is graded, then does it differ between non-category members that might share similarities with the higher level category (e.g., bird and giraffe) vs. those that do not (e.g., bird and chair)? related work by focusing on the mechanisms that underlie lms' property inheritance behavior, our work contributes to a rich body of work that investigates conceptual representations in language models @xcite @xcite @xcite @xcite . complementing these, our focus in this work is to understand how lms use their conceptual structure to project properties from one category to another-an important task in the broader space of inductive problems solved by humans @xcite . in this vein, recent works have investigated lms' property inheritance behavior-either by purely behavioral analyses @xcite , or by manipulating lms' internal representations to edit their taxonomic knowledge @xcite , and measuring the implications of the edit. importantly, these works assume that changes to taxonomic membership must be directly reflected in the lms' behavior, without necessarily considering the impact of categorical similarity. grounding our investigation in empirical observations about humans @xcite , we abstract away from this assumption, and explicitly consider the interplay between similarity and taxonomic relations by characterizing it using both behavioral and causal analysis methods. ## causal investigation of taxonomy and similarity given that lms perform property inheritance with high taxonomic sensitivity, we now use causal interpretability tools @xcite to localize this behavior. we then investigate (i) whether the subspaces that are causally responsible for inheritance perform inductive generalization in a manner that is better explained by the similarity of the noun concepts, rather than their taxonomic relationship; and (ii) whether taxonomic information and noun similarity are fundamentally entangled in the model representations. we investigate both questions using distributed alignment search (das; @xcite , a causal interpretability method which has been used to localize and mechanistically explain syntax-sensitive behaviors @xcite and arithmetic abilities @xcite in lms. to avoid needing to search over all possible subspaces of an activation vector manually, we employ the boundless das variant @xcite implemented in pyvene @xcite to learn the appropriate subspace. we use das rather than other causal interpretability techniques 7 because it allows us to investigate to what extent a specific hypothesized causal model is being implemented by the neural network. because das isolates a specific subspace corresponding to a given hypothesized causal variable, the subspace can be evaluated in novel contexts post hoc to better characterize its functional role. we leverage this to evaluate whether subspaces that are sensitive to taxonomic relationships are also sensitive to similarity-based but non-taxonomic relationships. this type of experimental design involving ambiguous training sets with separate test and generalization sets has been used to characterize the inductive biases of lms @xcite @xcite , though this has typically been evaluated behaviorally rather than at the level of internal mechanisms. there is a limited but growing literature validating how well discovered mechanisms (or explanations thereof) generalize to novel types of contexts @xcite @xcite . given an lm m , a hypothesized causal graph l, and counterfactual input (source, base) pairs (s, b) that target a specific intermediate variable v in the causal graph, das learns a rotation r for the activations of a given submodule, and then performs counterfactual interventions in this rotated space. this is done as follows: l v (s), the value of variable v in l when applied to source s, is patched into the corresponding variable of the base b, given that birds are daxable, is it true that orchids are daxable? given that birds are daxable, is it true that robins are daxable? b a p true true yes property conclusion concept premise concept b a p false true property conclusion concept premise concept true b is a kind of a a has property p b has property p b is a kind of a a has property p b has property p no yes s o u r c e b a s e figure 3 : we hypothesize that language models rely on this causal graph to perform the categorical inference task. we are interested specifically in the node responsible for taxonomic judgments, so we use das ( §4) to isolate the subspace encoding this causal variable. we causally verify its sensitivity to taxonomy by setting its value to what it would have been on counterfactual source inputs, and observing whether model behavior changes appropriately. here, activations at the isolated subspace for the base are replaced with activations from the source. iia measures to what extent this intervention results in the expected behavior given the hypothesized causal graph across inputs. and then the final output of l is computed. translating this to operations on the low-level computation graph, a rotation r is learned for activations in the network m from a position i to position j. we patch r(m (s after learning the rotation, we compute the interchange intervention accuracy (iia) on a held-out test set. this scores how frequently the learned subspace performs as expected given the hypothesized causal graph. intuitively, iia is defined as the proportion of (s, b) pairs where patching in the rotated source activation into the base model m causes m 's output to match the causal model l's output after the intervention. it thus measures the degree of alignment between the neural network and the causal graph. the intuition for learning a rotation is that the features of interest may not necessarily be aligned to the bases of the original activation space @xcite @xcite , rendering neuron-based approaches insufficient for capturing the target variable. we refer readers to @xcite for details. in our experiments, we hypothesize that a given subspace will be sensitive to taxonomic relationships-if this is true, then passing an activation into this subspace from taxonomically unrelated inputs should yield a variable output of false. however, if we intervene on this subspace passing in activations from an alternate input where the nouns are taxonomically related (while holding all other activations in the model constant), then the variable's output should flip to true. we illustrate the intuition and the causal graph that we hypothesize lms implement in figure 3 . by only changing the premise and conclusion nouns across inputs (e.g., birds, robins, orchids), and keeping the rest of input constant, we effectively target the node "b is a kind of a" in figure 3 . 8 we train das on a subset of 3,000 stimuli and evaluate on the rest. implementation and training details are given in appendix d. next, we describe variations of the training and test sets used in our experiments. balanced for each base stimulus in the set of 3,000 stimuli, we pick a source counterfactual stimulus by sampling from this set of stimuli without replacement. this ensures a balanced coverage of counterfactual stimuli in terms of similarity and whether or not the premise and conclusion are taxonomically related. this experimental setting is used to localize property inheritance in the network. control das requires training, raising concerns that its expressivity may lead to discovered causal effects where none exist @xcite , similar to concerns raised in the probing literature (hewitt and liang, 2019). to mitigate this, we follow @xcite and map the labels yes and no to the semantically irrelevant words chart and view, respectively. this setting evaluates the extent to which das memorizes an arbitrary token mapping versus learning task-specific structure. by only training on taxonomic examples with high similarity and negative examples with low similarity, we learn an intervention which is ambiguous with respect to what features it is using. to disambiguate these, we evaluate on two different test sets: an in-domain (ambiguous-test) test set composed in the same way as the train set (taxonomic examples with high similarity and negative examples with low similarity), and an out-of-domain generalization test set (ambiguous-gen), composed of taxonomic examples with low similarity and nontaxonomic examples with high similarity. a drop in iia values between the test and generalization settings would show that learned interventions rely more on similarity than taxonomic features. unambiguous finally, we test whether the subspace sensitive to taxonomic relationships is also sensitive to similarity-based relationships-i.e., whether taxonomic information and noun similarity are entangled in the model. to do this, we take the intervention trained under the balanced setting and evaluate it on (ambiguous-gen), defined above. here the training signal for das is unambigious-it is only compatible with capturing taxonomic information. a drop in scores would then indicate that taxonomic and similarity features are entangled. we use p rel (yes) rather than the top predicted token as done in @xcite when com-puting iia across experiments, in order to have a reasonable comparison against the control setting. foot_8 ## discussion and conclusions our goal in this paper was to characterize categorybased inferences in lms. we derived inspiration from psychological experiments that have sought to understand how humans transfer properties from a higher level category like mammals to lower level ones such dogs and cats @xcite @xcite -i.e., show property inheritance. findings from these experiments show that instead of relying purely on abstract category-membership principles (where all members of a category are equally likely to inherit properties), humans show graded behavior proportional to the similarity between the categories @xcite . through behavioral and representational analyses methods across four lms, we found that language models' property inheritance behavior is sensitive to both taxonomic relations and similarity. behaviorally, lms were more likely to extend the property from the premise to the conclusion when they were taxonomically related, and at the same time, showed positive correlations with the similarity between them. this finding was further reinforced in our representational investigations, where we found taxonomy and similarity to be largely entangled in the lm subspaces causally responsible for property inheritance. our results cast doubt on previous works' assumption of pure taxonomic principles with respect to property inheritance in lms, without considering similarity @xcite @xcite . in the broader context of lms' reasoning behaviors, our results showcase yet another instance where lms were susceptible to human-like content effects @xcite . our findings also suggest interesting hypotheses for future human analyses. importantly, sloman (1998)'s experiments never tested humans on cases where the categories in a property inheritance setting did not share taxonomic relations (e.g., birds and bats). this makes it unclear if the principles of taxonomy and similarity are also not mutually exclusive for humans @xcite -e.g., it could be possible that both mechanisms are at play, where taxonomic relations demarcate the inference (i.e., whether the property will be inherited) and similarity modulates its strength. future experiments could test this hypothesis and shed further light on the extent to which lms and humans show similar sensitivities and biases during reasoning. overall, our work shows how tools from modern interpretability research, primarily used to isolate sub-structures in neural networks, may also be used to perform hypothesis testing of the abstractions encoded within them.