Gathering Compositionality Ratings of Ambiguous Noun-Adjective Multiword Expressions in G alician (2025)

Abstract: Multiword expressions pose numerous challenges to most NLP tasks, and so do their compositionality and semantic ambiguity. The need for resources that make it possible to explore such phenomena is rather pressing, even more so in the case of low-resource languages. In this paper, we present a dataset of noun-adjective compounds in Galician with compositionality scores at token level. These MWEs are ambiguous due to being potentially idiomatic expressions, as well as due to the ambiguity and productivity of their constituents. The dataset comprises 240 MWEs that amount to 322 senses, which are contextualized in two sets of sentences, manually created, and extracted from corpora, totaling 1,858 examples. For this dataset, we gathered human judgments on compositionality levels for compounds, heads, and modifiers. Furthermore, we obtained frequency, ambiguity, and productivity data for compounds and their constituents, and we explored potential correlations between mean compositionality scores and these three properties in terms of compounds, heads, and modifiers. This valuable resource helps evaluate language models on (non-)compositionality and ambiguity, key challenges in NLP, and is especially relevant for Galician, a low-resource variety lacking annotated datasets for such linguistic phenomena.

## introduction multiword expressions (mwe) are idiosyncratic word combinations that constitute both major challenges and interests in natural language processing @xcite . the reasons lie in their intricate nature, as mwes can fall within a wide range of semantic compositionality, and both the expressions and their constituents may present different degrees of semantic ambiguity, among other challenging properties that complicate most nlp tasks @xcite . an example of the former is dark horse, which can be interpreted literally as a horse that is of a dark color or idiomatically as an unexpected winner, depending on the context. an example of the latter is common sense, which may be understood as the most frequent meaning of a word, expressions, etc., but can also be used to refer to a person's reasonable or good judgment, depending on the context. in the last two decades, numerous datasets have been put forward to address the issues mwes pose @xcite . among them, we can find the dataset of english noun-compounds with compositionality ratings @xcite , as well as its extensions for french and portuguese @xcite . for german, there exists a noun-noun compound dataset featuring compositionality ratings (schulte im @xcite . similarly, (schulte im walde, 2024) put forward a collection that comprises german compounds with compositionality ratings, where compound and constituent properties are also taken into account. related datasets contain binary or three-way classification (literal/idiom/other) of mwes and potentially idiomatic expressions (pie), such as the vnc-tokens dataset @xcite , comprising about 3,000 verb-noun combinations in english, or magpie @xcite , featuring around 56k english pies in corpora-extracted sentences, also featuring the literal/idiomatic/other classification. likewise, the semeval-2022 task 2 introduced binary classification datasets in english, @xcite . these datasets present highly valuable resources for nlp tasks. however, most of them are annotated at a type level @xcite . on the other hand, those that operate at a token level @xcite tend to comprise mwes that convey the same meaning in all sentences compiled in the dataset. therefore, such resources may not account for the wide variety of senses these idiosyncratic expressions can have. this situation is only more dire in the case of language varieties with few annotated resources, like galician, for which few such works exist despite being essential to explore if language models can adequately process mwes @xcite @xcite . we address such shortcomings by presenting a collection of noun-adjective compounds in galician. their ambiguity is two-fold, since the dataset contains 1) potentially idiomatic mwes with different degrees of compositionality, and 2) mwes whose constituents present different degrees of ambiguity and productivity. these expressions are disambiguated, and their senses are contextualized and preliminarily classified in terms of compositionality. overall, the dataset comprises 240 nounadjective mwes, and 322 senses. each sense is contextualized in two manually-written and four corpora-extracted sentences, which account for a total of 1,858 contextualizing sentences. 1 as a key contribution of this paper, this resource provides a set of human ratings on semantic compositionality levels for the 322 senses, along with additional linguistic information. in this regard, we enrich the dataset with frequency, ambiguity, and productivity data extracted from corpora and lexical resources for compounds and their constituents, used to explore potential correlations between linguistic features of the dataset. this publicly available dataset constitutes a valuable resource for evaluating language models on compositionality prediction and sense disambiguation tasks, among others. 2 2 creation of the dataset the goal was to create a dataset of potentially idiomatic, ambiguous mwes in galician, contextualized in validated sentences used to rate the expressions' senses in terms of compositionality. ## frequency, ambiguity, and productivity following previous works on datasets of similar nature, frequency, ambiguity, and productivity data were obtained for compounds, heads, and modifiers of all senses, aimed at studying the relationships between these properties regarding their compositionality degrees (schulte im @xcite . for frequency and productivity, the original corpus of mwe extraction was used, while ambiguity data was extracted from galnet @xcite ). foot_2 ## conclusions and further work in this work, we have introduced a dataset comprised of 240 noun-adjective mwes in galician that account for 322 senses, which present varying degrees of compositionality as well as semantic ambiguity. we have put forward human judgments on compositionality scores, which served to ascertain where mwes fall within the spectrum of idiomaticity, and also provided frequency, ambiguity and productivity data. using this information, we only found very weak and non-significant correlations with compositionality scores. the dataset, which comprises manually created sentences and examples extracted from corpora, fills a gap in annotated resources for galician. the dataset will be freely released, except for the manually annotated sentences, which will be kept for evaluation purposes only to prevent data contamination in language models. for future work, we aim to collect additional human ratings to strengthen the annotation of the dataset presented in this paper, ensuring greater reliability and consistency. furthermore, we plan to apply the same methodology to construct similar datasets for other types of linguistic expressions, such as verb-object combinations. additionally, it would be valuable to explore other linguistic properties and contextual cues that may influence human perception of semantic compositionality, providing deeper insights into the factors that shape meaning construction.