Let's keep an eye on Russian: testing sensitivity to the change in the grammatical number in somatic idioms with ruBERT
Authors: Elena Spaziani, Eliana Di Palma (Sapienza Università di Roma, Università Roma Tre, 2023)
Topic: Lexical and Semantic Resources and Analysis

Abstract
Recently, linguistic research and computational linguistics have increasingly focused on the morpho-semantic properties of grammatical number in nouns. In this context, our study aimed to use a language model to examine number variation in selected Russian somatic idioms that include both locally and generally marked nouns. Overall, the model’s sensitivity confirmed the significance of number in idioms, particularly in cases where the semantics of number is especially relevant.

Introduction
Somatic idioms are phraseological units containing lexemes that refer to human or animal body parts [1, 2]. This class of idioms appears in many languages, due to the universal nature of somatic lexicons. Somatisms are often studied from a semantic perspective, particularly through contrastive analyses, as their meanings are considered culturally specific conceptualizations. However, to our knowledge, theoretical and computational linguistic research has not yet explored the morpho-semantic features of grammatical number [3] within idiomatic meanings. Moreover, our focus on number is driven by the observation that pluralization often affects the lexical properties of somatic terms. Specifically, nouns representing bipartite or complex body parts, which naturally imply plurality, tend to occur more frequently in the plural form across languages. This phenomenon, known as local or semantic markedness [4, 5], occurs when the singular form becomes the locally marked member of the pair, altering the morpho-semantic interaction between singular and plural forms. Our study aims to investigate how grammatical number changes impact the structure of somatic idioms. In particular, we posed two questions: Does variation in grammatical number significantly impact the idiomatic structure of Russian somatic idioms? Does the type of markedness influence the likelihood of singular or plural usage? We hypothesized that changes in number would affect the structure of somatic idioms, especially in cases of local markedness. Specifically, we predicted that switching the number in locally marked somatic terms would have a more pronounced effect on idioms than it would for generally marked terms. The following sections outline our approach to these questions, beginning with a review of related work. We describe the creation of a new dataset of Russian somatic idioms to test our hypotheses1. Next, we detail our experiment conducted with ruBERT, a Russian-trained BERT model, employing methodologies similar to Salazar et al. [6] for acceptability and Pedinotti et al. [7] for semantic plausibility. The results and their implications are discussed in the final sections. We conclude by addressing our research questions and highlighting the significance of our findings and potential directions for future research.

Conclusion
Based on the experiment presented, we can conclude that our initial assumptions are partially justified. By comparing the original sentences to the manipulated and control sentences, we found that, overall, number significantly affects the probability of the constituents within the idiom. Specifically, the variation of idioms appears to be influenced by the morpho-semantic properties of the nominal constituent in the idioms considered. Our data showed that the result varies depending on the type of markedness. For generally marked constituents, a change in number is more probable. This may confirm the general tendency of number as a flexible dimension in idioms, which does not alter the idiomatic structure. On the other hand, locally marked terms show a significant difference in probability between singular and plural forms. This difference suggests that number variation is less probable in these cases, which may indicate a less flexible idiomatic structure. However, some terms exhibit a peculiar condition whose interpretation may require future hypotheses. Despite the correlation between human judgment and the PLL measure shown in Pedinotti et al. [7], the inclusion of human evaluations could still be necessary due to the figurative meaning of idioms. This comparison could improve the robustness of the analysis while clarifying the results obtained by the model. Additionally, given the unexpected outcomes for some nouns, a larger dataset that includes more idioms per noun and a wider variety of nouns could be designed to enhance the study.