Toward Compositional Behavior in Neural Models: A Survey of Current Views (2024)

Abstract: Compositionality is a core property of natural language, and compositional behavior (CB) is a crucial goal for modern NLP systems. The research literature, however, includes conflicting perspectives on how CB should be defined, evaluated, and achieved. We propose a conceptual framework to address these questions and survey researchers active in this area.We find consensus on several key points. Researchers broadly accept our proposed definition of CB, agree that it is not solved by current models, and doubt that scale alone will achieve the target behavior. In other areas, we find the field is split on how to move forward, identifying diverse opportunities for future research.

## introduction compositionality -the ability to correctly process wholes given the ability to correctly process their parts -is a core property of language @xcite , enabling unbounded expressivity through the "infinite use of finite means" (von humboldt 1836, as quoted by @xcite . in the past decade, artificial neural network models of natural language have made impressive progress toward human-like language use; however, it is not clear whether their language use consistently demonstrates human-like compositional behavior, especially during generalization @xcite @xcite . this question has been the subject of considerable debate in the field of natural language processing (nlp), as researchers have proposed diverse methods to model and assess compositionality @xcite . we contribute a conceptual organization of current issues surrounding compositionality in artificial neural network models, and use this framework to survey researchers active in this area. we find consensus (roughly 75%+ concordance) on several crucial points. researchers broadly agree with our proposed definition of compositional behavior (cb, §2.1). they also agree that cb is not a solved problem: current models do not achieve compositional behavior, and scale alone is unlikely to get us there -a perspective consistent with findings from the recent nlp metasurvey @xcite . in other areas, we find the field is split on how to move forward. in terms of evaluation, researchers disagree on whether current behavioral methods can assess a model's capability for compositional behavior, and remain divided on how best to pursue implementation. we believe there is value for the research community in identifying points of shared understanding and dispute, particularly on a topic foundational to the study of language. ## framing compositional behavior we conceptually frame our compositionality survey around three key themes, expressed in a series of statements, s0-s11. respondents provide a graded level of dis/agreement, from strongly agree to strongly disagree. we first define compositional behavior (cb; s0) and ask participants whether they agree with our definition. given this definition, we then ask which methods are necessary and sufficient to evaluate models' capacity for cb (s1-s6). finally, we ask whether current neural models achieve cb (s7), and if not, which interventions are needed (s8-s11). here, we briefly review the relevant literature informing each of these sections, and present the corresponding statements in the form that they appear on the survey. further methodological details of the survey are presented in §3. ## survey methodology this framework ( §2) structures the survey which we circulated to active researchers working in the general area of compositionality, with irb approval from the university of edinburgh (rt 541309). the anonymized dataset is available by request for research purposes. distribution our target respondent pool for the survey comprised all researchers currently publishing on the topic of compositionality in machine learning. we compiled a list of relevant research publications from three sources: ## survey results aggregate responses are shown in figure 2 (see also figure 1 for presentation in survey order). to our surprise, we found much more agreement across the community than expected, with researchers expressing a consensus opinion for 7 of the 12 statements listed on the survey. we define "consensus" as survey statements for which roughly 75% or more of respondents converge on agreement (i.e. strongly agree, agree, or somewhat agree) or disagreement. 81% of respondents agree with the statement s0, our proposed definition of compositional behavior (cb). we also find near-consensus agreement for statement s10: 73% of respondents agree that model-internal interventions are likely necessary to achieve cb. otherwise, we found consensus on disagreement. on the topic of interpretable representations, 82% of respondents judge that current methods are not sufficient to evaluate cb (s2), but 74% also judge that interpretable representations are not nec13 for the five participants who updated their responses, figure 3 plots their new position and cluster assignment. the scale result mirrors findings from the larger nlp metasurvey @xcite ): 83% of their 327 respondents disagree with the view that scaling up would solve "practically any important problem in nlp," and 71% believe that nlp research is excessively focused on scale. we interpret these convergent findings as evidence that skepticism about scale is not restricted to researchers focused on compositionality, but characteristic of the broader nlp community. ## discussion beyond the quantitative overview in §4, many survey respondents provided highly thoughtful written comments. we regret our inability to thoroughly engage all of the excellent points raised. here, we discuss three key statements -our proposed definition of cb (s0), the adequacy of behavioral evaluation (s1), and the need for interpretable representations (s4) -in light of the nuanced perspectives found in the comments. we focus on the role of model interpretability and the adequacy of current evaluation because these concepts roughly correspond to the main axes of variation identified in our principal components analysis (figure 3 ). defining cb as discussed in §2.1, we propose defining compositional behavior (cb) with respect to an informal human-like conception of wholes and parts. though a few commenters consider such human-level perceptibility an irrelevant constraint, most agree with the criterion; many of those who agree, however, nonetheless find cb too vague, or insufficiently formal for useful research. several highlight the difficulty of finding consensus in human judgments. for instance, andrew lampinen cites @xcite 's finding that educational level affects semantic composition in compound words, and lake and baroni (2023) observe considerable variability in the composition rules used by human participants in a highly constrained experimental setting. we recognize the diverse nature of human judgment, and the challenge for scientific evaluation. we also thank respondents for highlighting an overlooked ambiguity in our proposed cb definition: while we intend our appeal to human judgment to apply to both a) the decomposition of an input i into parts and b) the correctness of the respective model output, the definition as written only states (a) explicitly. dieuwke hupkes, james l. mcclelland, and andrew lampinen each propose amended cb definitions which directly incorporate (b). many other comments raise related points: that correct decomposition of the input does not entail correct composition of the output, that decomposition and composition are contextually variable in natural language, that partial composition is possible, and that the meaning of composed expressions in natural language often rely upon factors beyond the contents of input parts. we find these observations insightful, and consider them at least partly addressed by deferring to human judgment of compositional outcomes, despite the challenges outlined above. a final key point raised by several commenters is the central importance of generalization. ellie pavlick, jake russin, dieuwke hupkes, and emmanuel dupoux, inter alia, note that a model which achieves compositional behavior on a given dataset through memorization would not be interesting from a research perspective, as we would not expect it to extend cb to other datasets and domains. this contention highlights the central challenge of cb evaluation for machine learning: how can we be sure that compositional behavior arises for the right reasons? behavioral evaluation survey respondents are almost perfectly divided on the adequacy of current methods for behavioral evaluation. 53% agree that current behavioral methods are sufficient to establish cb -though as noted by raphaël millière and others, this requires proper experimental design: careful control of training data, such that the model is not exposed to the generalizations necessary to succeed on the test set. behavioral evaluation also permits greater ecological validity, as we can often directly compare human performance on the same behavioral task. the other 47% of respondents are more skeptical. marco baroni and andrew lampinen characterize current behavioral methods as "necessary, but not sufficient;" many other commenters note that behavioral evaluation on a limited phenomenon or domain cannot establish fully general cb, and raise concerns about synthetic tasks which may not reflect performance in more realistic settings. we note that many respondents who agree with s1 nonetheless raise similar concerns in their comments. researchers have a shared view of the limitations of current behavioral evaluation, but differ on whether these limitations prevent these methods from sufficiently demonstrating cb. interpretable representations we were particularly interested in how researchers view the connection between interpretable representations and evaluating compositional behavior (cb). the results reveal a strong consensus that no such connection is necessary. of those disagreeing with statement s4 ( ## conclusion compositionality, a foundational aspect of natural language, has taken on new significance in light of modern neural models and uncertainty about their capacities. this paper offers a framework for defining, evaluating, and achieving compositional behavior in neural models, and surveys the views of researchers active in this area. we identify key points of consensus and division, providing a snapshot of the field to inform future research.