Definition Frames: Using Definitions for Hybrid Concept Representations (2020)

Abstract: Advances in word representations have shown tremendous improvements in downstream NLP tasks, but lack semantic interpretability. In this paper, we introduce Definition Frames (DF), a matrix distributed representation extracted from definitions, where each dimension is semantically interpretable. DF dimensions correspond to the Qualia structure relations: a set of relations that uniquely define a term. Our results show that DFs have competitive performance with other distributional semantic approaches on word similarity tasks.

## introduction ontologies have been widely used in lexical semantics to organize and represent knowledge. carefully built by experts, they contain semantically meaningful information in the form of relations between concepts. however, being manually constructed, they struggle to assimilate new information. compared to ontologies, distributed representations are fully automated and can be fine-tuned for new tasks. despite their exceptional performance, most distributional methods do not have an explicit semantic interpretation. the resulting representations encode a tremendous amount of information, but afford no way to interpret what this information is and how it relates to the concept. thus, one cannot choose which type of information is useful for a specific task, unless one has a lot of data and resources to fine-tune. although a few approaches have tried to bridge the gap between semantics and distributed representations @xcite , (1) they only encode information from ontologies, which are not extensible, and (2) the final representations are still not semantically meaningful. motivated by these problems, we introduce a novel hybrid representation called definition frames (df), which encode semantic information extracted from definitions. dfs are matrix representations, where each row corresponds to a particular relation. the set of the relations used is based on the qualia structure suggested in @xcite , and they are extracted automatically from definitions via a domain-adaptation approach. to the best of our knowledge, df is the first hybrid representation, combining an explicit structure through semantically meaningful rows, while still being decomposed into distributional vectors. ## prior work prior research on lexical semantics has established a set of relations that are sufficient to uniquely define a concept. such work includes the qualia structure @xcite ) and the generative lexicon theory @xcite . other related work includes ontological approaches @xcite @xcite and more fine-grained definition-based frames like semagrams @xcite . in distributional semantics, approaches including glove @xcite , word2vec @xcite , and fasttext @xcite obtain generic word embeddings by pre-training on large corpora. recent work focused on context-sensitive embeddings like elmo @xcite and bert @xcite , which achieve significant improvements in downstream nlp tasks. earlier work on definitions extracted the type of a concept (genus) and the relations distinguishing it from other members of the same type (differentia) via syntax and string matching heuristics @xcite @xcite . recent approaches directly encoded definitions to distributed representations. @xcite obtained embeddings via a skip-gram model trained on definitions, while @xcite used an auto-encoder. other work includes definition generation @xcite , binary classification of sentences on whether they are definitional (anke and schockaert, 2018), reverse dictionary look-up @xcite , and extraction of hypernymy relations from definitions using syntactic patterns @xcite . ## approach our framework consists of two parts: the relation retriever and the definition frame (df) encoder. the wordnet definition for any given term is used by the relation retriever model to extract the qualia structure relations. the set of extracted terms pertaining to these relations form the definition frame. the df encoder encodes this output to a distributed matrix representation, which can be used in downstream nlp tasks. qualia structure the qualia structure (formal, constitutive, telic, and origin) is defined as the complete modes of explanation associated with an entity @xcite . these relations suffice to uniquely and completely define a concept. in fact, several relation extraction tasks @xcite contain relations similar to qualia describing the type (isa), structure (madeof, partof, hasa), function (usedfor), or provenance (createdby) of a concept. ## experiments word-similarity task we perform experiments on benchmark word-similarity datasets provided by @xcite : simlex999 @xcite , mc30 @xcite , rg65 @xcite , ws353 @xcite and men @xcite . following agirre (2009), we split them into word-similarity (ws-sim, simlex999, mc30, rg65) and word-relatedness (ws-rel, men) datasets, as they evaluate different semantic affinities. we only consider nominal terms that exist in wordnet and report spearman's correlation ρ. we perform experiments with three types of embeddings used as basis: glove @xcite , dict2vec trained on wikipedia @xcite , and retrofit embeddings @xcite based on glove. since the task comprises of pairs of words without any context, we do not compare against context-based representations. ## conclusion we propose definition frames, a hybrid semantically interpretable representation that is grounded in both lexical semantics and distributed representations. by disentangling the qualia structure relations, dfs can capture different types of similarity (relatedness and similarity) and achieve improved performance on word similarity tasks. finally, we demonstrate the explainability of definition frames via a human study showing that they provide valid insights on how terms are related. dfs are independent of the distributed representation used as basis. future work could explore the use of contextual embeddings basis and the benefits of definition frames in downstream tasks.