A Cognitive Linguistics analysis of Phrasal Verbs representation in Distributional Semantics
Authors: Melissa Donati, Carlo Strapparava (Università di Bologna, Fondazione Bruno Kessler, 2023)
Topic: Distributional Semantics

Abstract
Phrasal Verbs (PVs) constitute a peculiar feature of the English language and represent a challenge for both language learners and computational models because of their complex and idiomatic nature, which has made them appear unsystematic and unpredictable. Recently, Cognitive Linguistics has offered a more systematic explanation of the semantics of PVs by relating their non-compositional meanings to the metaphorical extensions of the particle’s meaning. In order to assess the computational suitability of this approach using Distributional Semantics, we analyzed three different semantic spaces to understand how PVs and particles are represented and whether any of the embeddings capture the significance of particles in the semantics of the entire construction. The results indicate that phrase embeddings are effective in representing the meanings of PV constructions, while word embeddings excel at capturing particle meanings and additionally support the Cognitive Linguistics hypothesis. Since improving the semantic representation of PVs can benefit various NLP applications, further research is necessary to validate these findings.

Introduction
Phrasal Verbs (PVs) represent a distinctive peculiarity of the English language and are defined as a lexicon unit composed of a verb (e.g., look) and a particle (e.g., out), whose meaning is often non-compositional (e.g., look out means "to beware") [1, 2]. PVs comprise a significant portion of the verb vocabulary [3] and are commonly used in everyday language, particularly in spoken and informal contexts [4, 5]. In addition, they are highly productive, with new ones continually being coined to reflect societal changes (e.g., google up) [6]. They are also characterized by their polysemy, with each phrasal verb having multiple meanings on average [5]. To further enhance their complexity, for a long time, linguists and grammarians have claimed that the selection of verb and particle in the PV construction is totally unsystematic and unpredictable [6, 7, 8], therefore the traditional pedagogical approach to PVs has always been based on memorization of the verb-particle combination and the corresponding meaning causing a general discouragement of learners around PVs. Recent research in Cognitive Linguistics, however, has proposed a more systematic explanation of the association between these verb-particle combinations and their apparently randomly assigned idiomatic meaning, by suggesting that it is the particle (in particular its metaphorically extended meaning [9]) that plays a crucial role in shaping the overall meaning of the PV [2]. Given that this approach has shown promising results in language learning [10], and that computational models of language face difficulties that are similar to English as a Second Language (ESL) learners in understanding the semantic complexity of PVs, we wanted to examine whether such a Cognitive Linguistics account holds also from a Distributional Semantics perspective, where the representation of words’ meanings and semantic relationships have repeatedly been proved to be similar to the way they are represented in the human cognitive system [11]. With this aim, we analyzed three different semantic spaces – word embeddings, phrase embeddings, and POS-tagged embeddings – to determine the most accurate way of representing PVs and particles and whether the Cognitive Linguistics hypothesis was accounted for in any of them. The importance of the particle in shaping the meaning of PVs was confirmed, but the results appeared to vary across the different semantic spaces, suggesting the need for further and more detailed research.

Conclusion
The aim of this work was to analyze the distributional representation of PVs from a Cognitive Linguistics perspective. More specifically, we wanted to examine three different semantic spaces (word embeddings, phrase embeddings, and POS-tagged embeddings) using simple vector combination (sum) and mathematical computations (cosine similarity) to evaluate whether: 1) the meaning of the PV construction is properly represented; 2) the particles’ embeddings truthfully capture their meaning; 3) the greater role of the particle in shaping the semantics of the PV, as posited by the Cognitive Linguistics approach, is accounted for. The current results showed that, as expected, phrase embeddings performed best in capturing the complex semantics of PVs, supporting the idea of treating PVs as single tokens when training the embeddings so as to capture the true context of occurrence and obtain more accurate meaning representation that accounts also for the less compositional meanings. As far as particles are concerned, word embeddings outperformed both phrase and POS-tagged embeddings. These results were unexpected because we anticipated better performance from POS-tagged embeddings, which were designed to isolate particle occurrences. We identified different possible explanations for these results, including limitations in integrating POS-tag information (that reduces the number of occurrences) compared to including occurrences with different syntactic functions, the issues related to the general difficulty of representing function words, and also the possibility that words used as particles carry a unique core meaning regardless of their syntactic functions. Further research is needed, along these lines, to disentangle these factors and understand what could be the most effective way to represent particles and function words in Distributional Semantics. Finally, the results of the third and last type of analysis, that was designed precisely to test the Cognitive Linguistics claim on the role of the particle, showed that when separate vector representations are built for particles, i.e., distinguishing the occurrences of the same words with other syntactic functions, as was the case with POS-tagged embeddings, particles do appear to play a greater role than the verbs proper in the semantics of the PVs. Conversely, when the vector representation of particles is less accurate, i.e., includes occurrences of the same words with other syntactic functions, as it was the case with word embeddings, it is the verb proper that appears to be more crucial in the semantics of the PV in most cases. Overall our findings align with the literature, in that they support the idea that vectors for PVs should be treated as single tokens rather than splitting them into individual words [38] and that representing the meaning of particles is challenging, justifying their removal in many NLP applications [46]. However, we believe that understanding how to build appropriate semantic representations for particles is crucial for analyzing their contribution to larger constructions, such as PVs. In order to do so, future studies can explore different types of embeddings and test whether refining POS-tagged embeddings (for example, by weighting each POS-tag feature according to the task or alternatively using Neural Networks for combining these features into a unique meaningful hidden representation) could improve representation accuracy and thus lead to better performance of the models in specific semantic tasks. Last but not least, our results provide initial evidence supporting the Cognitive Linguistics account of PV semantics from a Distributional Semantic perspective, although further confirmation is needed. Adopting the Cognitive Linguistics approach to PVs in education and leveraging NLP applications in this direction can facilitate the acquisition of this complex English structure for ESL learners. Additionally, capturing and representing PV meanings and the semantic roles of their components can benefit NLP tasks involving semantic and morphosyntactic relations (such as machine translation, question-answering, summarization, automatic synonym detection, etc.). For these reasons, we hope this work stimulates further advanced research in this area, leveraging the insights from Cognitive Linguistics and Computational Linguistics.