Context Effects on the Interpretation of Complement Coercion: A Comparative Study with Language Models in N orwegian (2025)

Abstract: In complement coercion sentences, like *John began the book*, a covert event (e.g., reading) may be recovered based on lexical meanings, world knowledge, and context. We investigate how context influences coercion interpretation performance for 17 language models (LMs) in Norwegian, a low-resource language. Our new dataset contained isolated coercion sentences (context-neutral), plus the same sentences with a subject NP that suggests a particular covert event and sentences that have a similar effect but that precede or follow the coercion sentence. LMs generally benefit from contextual enrichment, but performance varies depending on the model. Models that struggled in context-neutral sentences showed greater improvements from contextual enrichment. Subject NPs and pre-coercion sentences had the largest effect in facilitating coercion interpretation.

## introduction coercion results from a semantic type mismatch between a predicate and its argument @xcite @xcite . in john began the book, the aspectual verb begin requires an eventdenoting complement, but is instead combined with an entity-denoting np (the book). the covert event can be recovered by exploiting the meaning of lexical constituents, world knowledge, and context @xcite @xcite . hence, speakers can interpret the sentence above as meaning, for example, that john began reading the book. because the resulting interpretation is not a strict function of constituent meanings and syntax, coercion has been argued to violate strong versions of the principle of compositionality @xcite @xcite . experiments found longer reading times @xcite and on-line processing costs @xcite @xcite for coercion sentences compared to controls in which the relevant event is expressed by a non-aspectual verb (e.g., john read the book) or an event-denoting complement. transformer-based language models (lms) @xcite have become popular in nlp owing to their success in a range of tasks. however, few studies addressed how lms process complement coercion. previous research focused mainly on coercion as a challenge for sentence interpretation and framed it as a task where lms have to predict the best covert event given an aspectual verb-complement combination @xcite @xcite @xcite . @xcite demonstrate that lms have difficulty retrieving covert events for coercion sentences in norwegian in the absence of context. the present study extends that work by investigating the role of context. transformers' self-attention mechanism captures local contextual information by assigning greater relevance to some tokens compared to others within a sequence @xcite @xcite . the result is the generation of dynamic linguistic representations that vary according to the surrounding context. we expect that contextual information will improve the performance of transformer-based lms in covert event interpretation of coercion sentences. ## theories of coercion in context one hypothesis assumes that a coercion interpretation is the result of enriched composition: lexicosemantic properties of words are leveraged to enrich the meaning of the sentence, resulting in an eventive reading @xcite @xcite . each lexical item is associated with a qualia structure that includes, among others, a specification of telic (the purpose of an object) and agentive (how an object is created) roles of the relevant entity. for coercion sentences, a type mismatch between the aspectual verb and its complement leads to the recovery of the covert event by exploiting the qualia roles of the entity book: the telic role implies that reading is the covert activity, while the agentive role implies writing. contextual information can motivate different interpretations than those suggested by default qualia roles @xcite @xcite . in the author began the book, the subject np can facilitate the recovery of the agentive quale write @xcite . in the climber enjoyed the rock, instead, where no specific telic role is provided by rock, the complement is enriched through co-composition of the subject np climber, suggesting the interpretation that the climber enjoyed climbing the rock @xcite . the contextual enrichment of coercion sentences is also motivated by empirical studies. @xcite , for instance, acknowledge that the "properties of the subject np appear to determine the default interpretation in an otherwise neutral context." in eye-tracking experiments, traxler et al. @xcite concluded that contextual information does not necessarily attenuate processing costs in coercion sentences, but can be exploited as an 'extended lexicon', licensing an eventive interpretation that could be otherwise costly to generate. the pragmatic hypothesis proposes a different account of complement coercion compared to the more constrained approach of the lexical analysis, which claims that coercion sentences are enriched solely via default qualia-based lexical information @xcite . building on relevance theory @xcite , the proposal by de @xcite and de @xcite claims that lexical entries only specify an expression's denotation or type @xcite . the interpretation of coercion sentences is therefore not lexically-driven but guided by postlexical pragmatic inferences, world knowledge, and context. this leads to more flexible interpretations and a wider variety of readings compared to those afforded by qualia roles @xcite @xcite . @xcite and @xcite provides instead support for generalized event knowledge (gek) @xcite in coercion interpretation, an alternative to both lexical qualia-based and pragmatic hypotheses. the words-as-cues hypothesis @xcite claims that speakers store event knowledge in memory: words serve as cues that allow access to such knowledge, modulating expectations about upcoming words. in a self-paced reading study, @xcite found that if coercion interpretations align with more typical events, sentences are read faster. ## lms and complement coercion the first study evaluating lms on complement coercion was @xcite , who analyzed the performance of pretrained transformers of the bert and gpt families. they used datasets from different behavioral studies @xcite @xcite . the results revealed that transformer-based models behaved differently from each other depending on the model's framework. roberta, for example, emerged as the most robust lm, performing better than other models on the lapata-lascarides dataset @xcite , with 80% accuracy in binary classification and 73% in a correlation task. in contrast, gpt-2 appeared to be more unstable, with a better score in the binary classification task (87%) but poorer performance in the correlation task (43%). vanilla bert, on the other hand, showed a marginal improvement over the baseline, suggesting a limited ability for contextualized embeddings in capturing eventive information from context. finally, the authors report that distributional and non-transformer frameworks, such as the structured distributional model (sdm), performed similarly to roberta despite being pretrained on smaller datasets. gietz and beekhuizen (2022) consider coercion as a case of flexible semantic enrichment based on context, rather than as obligatory semantic completion. they analyzed a vanilla bert model using a dataset with naturally-occurring coercion sentences from the coca corpus, successively annotated by humans. they argue that traditional 'hand-crafted' coercion sentences from previous studies always allow clear event interpretations, while naturallyoccurring sentences usually include additional contextual information. bert performed well in cases where consensus between annotators on a covert event was high, but struggled with sentences with less consensus. the model benefited from contextual information, improving event prediction. @xcite used a dataset of naturallyoccurring coercion sentences extracted from the c4 corpus @xcite . the authors argue that the process of coercion interpretation is analogous to paraphrasing: the coercion sentence is rephrased in a way that ambiguity is eliminated and the covert event is revealed. they found that pretrained bert has difficulty with coercion interpretation, while a model fine-tuned with explicitly paraphrased sentences leads to better performance. @xcite investigate whether lms can leverage syntactic structure and lexical meaning toward recovering covert events. they conduct a large-scale evaluation of lms in norwegian, a low-resource language with variable grammatical realization of coercion, which partly depends on the aspectual verb used. initiation verbs usually combine with entity-denoting nps in pps introduced either by på or by med (john begynte på/med boken; 'john began on/with the book'). with continuation and cessation verbs, complements are mainly introduced by med-pps or directly an np (john avluttet med/ø boken, 'john finished (with) the book'). @xcite released a new dataset of sentence pairs, each containing a contextneutral coercion sentence and an event resolution prompt. the dataset included 90 distinct entities from 6 different categories, and the syntactic realization of coercion was varied systematically by the aspectual verb and pp/np. the study tested 17 norwegian lms, spanning bert-like autoencoders and autoregressive models. in general, lms struggled to recover implicit events. surprisal estimates for whole sentences indicate that most lms tested are unable to leverage the syntactic structure of the vp to interpret coercion items, showing no significant performance changes across syntactic constructions. for more details, see section 5.1. ## task proposal here, we explore the role of context in coercion interpretation, extending @xcite 's work on norwegian context-neutral sentences. we study how different types of context influence the prediction distributions for lms in a covert event interpretation task. we used the same evaluation strategy as @xcite : instead of assessing models' performance only on a pre-defined set of top-1 ranked predictions as gold standard, we considered the ranked prediction distribution for each model; for each coercion sentence, a model must output a set of top-5 ranked predictions @xmath0, ..., o 5 . the distribution is then evaluated by calculating the mean average precision metric, which captures the consistency of lms in predicting appropriate events (see below) in the top ranking. we consider a model 'sensitive' to coercion, if it can provide a prediction distribution that is relevant to event interpretations: given a coercion sentence, expressed as a triplet 〈subject, aspectual verb, entity〉, we expect a redistribution of output predictions in a way that eventive interpretations are at the top of the ranking. the addition of contextual information should lead to further redistribution of the outputs, possibly with a shift towards the event interpretations suggested by the context. the output predictions for each sentence will be evaluated by considering any event (verb) as correct as long as it satisfies the semantic constraints required by coercion and by the context. following piñango and deo ( 2016) and spalek and saebø (2019) , a covert event is a plausible candidate for coercion when its combination with subject and complement expresses telicity, implying a "natural endpoint or goal state" that is coherent with the overall meaning of the sentence. the class of accomplishments is our ground truth for event classification, as it specifies durative, dynamic, and telic situation types or aktionsart @xcite . all predicted events that are accomplishments are compositionally appropriate candidates, including those that may be weakly associated in coercion contexts. for example, the triplet 〈goat, begin, book〉 can suggest the covert event eat @xcite . some events must however be discarded: although they belong to the accomplishment class, their combination with the given subject and object results in a semantic anomaly. for example, a verb like klatre ('climb') could be plausible when predicted with objects that afford movement (e.g., mur; 'wall') but not with food items (e.g., salat; 'salad'). ## general discussion and conclusion our results indicate that contextual enrichment of coercion sentences in norwegian generally leads to better prediction distributions of covert events in almost all tested lms. additional context in specific sentence regions, such as in subject position, or the inclusion of sentences preceding the coercion construction, leads to most benefits in performance. in this study, we found that performance varies also according to lm framework: bert-like autoencoders appear to benefit most from contextual enrichment as compared to autoregressive models. this is consistent with the conclusion of @xcite , where lms were tested on coercion sentences without context. the advantage for autoencoders may be their bidirectional selfattention mechanism, which may be better able to capture semantic relations between constituents. however, models such as mbert, nb-bert-base, and norbert3-xs, for example, still showed only marginal improvements when exposed to context. better performance for such models may be related to the interplay between their size and the amount of pretraining data: the multilingual model was one of the worst performing probably due to its small training data in norwegian. conversely, results from norbert3-xs, suggest that, despite the large pretraining data, a smaller model still has limitations. performance increases when the model's size increases, as shown for the larger nor-bert3 models. other factors could also play a role. the norbert family showed more robust performance compared to nb-bert models and mbert, probably because the model was trained entirely from scratch on norwegian and employed a custom wordpiece vocabulary. in contrast, nb-bert starts from the mbert framework and is trained on additional data in norwegian without further changes @xcite . moreover, the third norbert generation, also introduces optimized training methods by excluding the next sentence prediction task and improving the masked language modeling objective task, increasing the span-based masking rather than masking single tokens @xcite . from the analysis of lm scores, we also found a consistent pattern linking their performance on context-neutral sentences and their improvement when context is introduced. specifically, models that previously obtained poor results appear to benefit the most from context. models like norbert2, nb-bert-base, and norbert3-xs obtained a significant boost in performance compared to others. such an improvement is however relative to their poor performance in context-neutral sentences. their capacity to exploit contextual signals appears to compensate for such limitations. it is particularly noteworthy that the lms that obtained relatively high scores with context-neutral items are those that also showed more limited improvement when context is provided. this claim requires further research, but we hypothesize that such behavior may reflect a form of 'encoding saturation' by transformer-based models, manifested in a limited capacity to integrate additional semantic information once a certain level of encoding complexity in a model's embedding-based representations has been reached. this behavior can also be observed when comparing models with almost identical architectures: norbert3-xs and norbert3-large differ only in their parameter sizes, but they showed different improvement trends. we hypothesize that contextual information can compensate for gaps in world knowledge as required by coercion resolution. consequently, context may not generally boost performance but rather benefits most the weaker models: stronger models show little change in their performance as they may have already reached a performance plateau, which cannot be improved by the integration of additional contextual information. this hypothesis is partially confirmed by the general improvement trend in results observed in although contextual information generally led to better performance, lms still show difficulties in interpreting complement coercion sentences. this aligns with the conclusions of earlier studies, such as @xcite and @xcite in english. it has been often observed that lms lack a capacity for common sense reasoning based on plausible world models. this would also apply to natural language interpretation, in that current lms have limited linguistic common sense: they lack the capacity to retrieve and exploit the kind of linguistic and world knowledge that would allow them to reliably make sense of complex, underspecified inputs @xcite @xcite . a closer look at our results sheds light on how and to what extent the behavior of transformer models aligns with expectations based on different theoretical accounts on complement coercion. at first glance, the improvements seen for most models appear compatible with the pragmatic hypothesis: context and world knowledge can modulate or restrict coercion interpretations according to information that is not necessarily available from constituent meanings. however, such improvements were only seen for those models that were shown to be weaker in context-neutral scenarios, presumably due to a limited encoding of semantics in the learned embeddings. more semantically robust lms were less influenced by context, suggesting that at least some relevant event information is encoded in the embeddings: this is more consistent with the lexical and generalized event knowledge (gek) hypotheses than with pragmatic accounts. on the other hand, our results cannot confirm the lexical hypothesis either, as context still has an effect in changing prediction distributions. moreover, if models had learned and used lexically-bound representations such as qualia, we would not expect to see as outputs events that belong to incorrect aktionsart, as in the example above. in addition, high performing models like norllama were even negatively influenced, suggesting a complex role of context in this task.