Multi-Relational Hyperbolic Word Embeddings from Natural Language Definitions (2024)

Abstract: Natural language definitions possess a recursive, self-explanatory semantic structure that can support representation learning methods able to preserve explicit conceptual relations and constraints in the latent space. This paper presents a multi-relational model that explicitly leverages such a structure to derive word embeddings from definitions. By automatically extracting the relations linking defined and defining terms from dictionaries, we demonstrate how the problem of learning word embeddings can be formalised via a translational framework in Hyperbolic space and used as a proxy to capture the global semantic structure of definitions. An extensive empirical analysis demonstrates that the framework can help imposing the desired structural constraints while preserving the semantic mapping required for controllable and interpretable traversal. Moreover, the experiments reveal the superiority of the Hyperbolic word embeddings over the Euclidean counterparts and demonstrate that the multi-relational approach can obtain competitive results when compared to state-of-the-art neural models, with the advantage of being intrinsically more efficient and interpretable

## introduction a natural language definition is a statement whose core function is to describe the essential meaning of a word or a concept. as such, extensive collections of definitions @xcite , such as the ones found in dictionaries or technical discourse, are often regarded as rich and reliable sources of information from which to derive textual embeddings @xcite @xcite @xcite . a fundamental characteristic of natural language definitions is that they are widely abundant, pos-1 code and data available at: https://github. com/neuro-symbolic-ai/multi_relational_ hyperbolic_word_embeddings ## methodology we present a multi-relational model to learn word embeddings exclusively from natural language definitions that can leverage and preserve the semantic relations linking definiendum and definiens. the methodology consists of two main phases: (1) building a specialised semantic role labeller ## qualitative analysis in addition to the qualitative evaluation, we perform a qualitative analysis of the embeddings. this is performed in two different ways: traversal of the latent space and relation-adjusted transformations. ## related work considering the basic characteristics of natural language definitions here discussed, efforts to leverage dictionary definitions for distributional models were proposed as a more efficient alternative to the large unlabeled corpora, following the rising popularity of the latter @xcite @xcite . simultaneously, efforts to improve compositionality @xcite and interpretability (de @xcite of word representations led to different approaches towards the incorporation of definition resources to language modelling, with the idea of modelling definitions becoming an established task @xcite . more recently, research focus has shifted towards the fine-tuning of large language models and contextual embeddings for definition generation and classification @xcite @xcite , with interest in the structural proper-ties of definitions also gaining attention @xcite . finally, research on hyperbolic representation spaces has provided evidence of improvements in capturing hierarchical linguistic features, over traditional (euclidean) ones @xcite @xcite . this work builds upon the aforementioned developments, and proposes a novel approach to the incorporation of structural information extracted from natural language definitions by means of a translational objective guided by explicit semantic roles @xcite , combined with a hyperbolic representation able to embed multi-relational structures. ## conclusion this paper explored the semantic structure of definitions as a means to support novel learning paradigms able to preserve semantic interpretability and control. we proposed a multi-relational framework that can explicitly map terms and their corresponding semantic relations into a vector space. by automatically extracting the relations from external dictionaries, and specialising the framework in hyperbolic space, we demonstrated that it is possible to capture the hierarchical and multi-relational structure induced by dictionary definitions while preserving, at the same time, the explicit mapping required for controllable semantic navigation.