W ordnet-oriented recognition of derivational relations (2023)

Abstract: Derivational relations are an important element in defining meanings, as they help to explore word-formation schemes and predict senses of derivates (derived words). In this work, we analyse different methods of representing derivational forms obtained from WordNet – from quantitative vectors to contextual learned embedding methods – and compare ways of classifying the derivational relations occurring between them. Our research focuses on the explainability of the obtained representations and results. The data source for our research is plWordNet, which is the wordnet of the Polish language and includes a rich set of derivation examples.

## introduction word formation processes can be observed in many, if not all natural languages: derivatives are formed from derivational bases by means of language specific derivational mechanisms, e.g. a teacher from to teach, a duchess from a duke or, from the polish language, domeczek ≈'a nice, little house' from dom 'a house', białość ≈'a state of being white' from biały ≈'white'. in some natural languages, especially in the case of inflectional ones, e.g. slavic languages, such mechanisms constitute a very productive system. that is why native speakers can recognise a new derived word forms (derivatives) as a language unit and identify their derivational bases with high precision. what is more, derivational relations, in contrast to morpho-syntactic word formation processes (e.g. different forms of nouns related to the grammatical cases or verb forms representing persons), signal a meaning change between a basis and the derivative. such lexical meaning transformations are also predictive to a very large extent, e.g. palarnia ≈ 'a place for smoking' derived from palić 'to smoke'. due to this property, such a class of derivational relations, described in lexico-semantic networks, is called morphosemantic relations @xcite . it is worth to notice that morphosemantic relations combine two transformations: one between word forms and, the second, in parallel, between lexical meanings, that are tightly coupled: different types of word form transformations are characteristic for some types of semantic derivations, e.g. kierowniczka ≈ 'a female head or manager' derived from kierownik 'a head or manager' primarily by the suffix -ka. derivation rules can be described to some extent by a combination of suffixes, prefixes and inside stem alternations. however such word form level rules are semantically, ambiguous with respect to the meaning derivation. e.g., the suffix -ka mostly signals: a transformation from +@xmath0, but it appears in tool name derivation, too: wiercić 'to drill' → wiertarka 'a driller', and can be also misleading: pierwiastka 'a woman giving birth for the first time' is not a female form of pierwiastek 'root', in spite of 'ka'. thus proper recognition and interpretation of derivational requires taking into account both types of transformations: morphological and semantic. the general objective of our work is to developed a mechanism for recognition and interpretation of derivatives in a way combining morphological and lexico-semantic level. for a given word, a potential derivative, we want to recognise not only a set of words with which it is in a certain lexicosemantic relation, and also a word from which it has been morphologically derived -its derivational basis. we study machine learning means taking into account both levels: word form and semantic. the unique feature of our approach is a combination of transformer-based neural architecture for modelling derivational patterns tightly coupled with recognition of lexico-semantic relations based on non-contextual word embeddings as semantic representation. we focus on the polish language for which a large and rich model of morphosemantic relations is included in plwordnet @xcite . contrary to many other wordnets and deriva-tional dictionaries, the plwordnet morphosemantic relations link particular senses of two words, not the word forms. in addition, these relations are always directed according to the derivational processes in polish: from a derivational basis to the derivative. derivational relations are often described in morphological dictionaries as links between lemmas foot_0 e.g. @xcite , @xcite or a very large morphological and derivational network derinet @xcite , only later automatically classified to 5 very coarse-grained semantic classes @xcite . in @xcite the training data were pairs of words (not senses) and classification was based on morphological features of word forms. semantic annotation of word pairs was adopted for wordnets (lexico-semantic networks), e.g. rowordnet @xcite , bulnet @xcite or crown @xcite . however, in wordnets, links between lemmas are additionally labelled with semantic relations, i.e. mapped onto morphosemantic relations. plwordnet @xcite showed that such an approach is simplification and prone to errors, as different morphosemantic relations may be valid only for selected senses of lemmas. thus, we focus on morphosemantic relations as linking senses, but signalled by derivational associations. in @xcite two character-level transducers were built on the basis from training data (with post-pruning generalisation) and combined with internal stem alternations. relations suggested by transducers were next filtered by grammatical patterns, corpus frequency and semantic classifiers for word pairs. trained a combination of features describing word distributions in a large corpus. the best results were reported for the set of 9 most populated relations: 36.84 (the young being relation) up to 97.19 (femininity) of f1. however, it should be emphasised that in this case wordnet-internal knowledge about assignment of lemmas to wordnet domains @xcite was utilised. we do not use such knowledge in our approach. in a similar approach @xcite , but much more supported by hand-crafted knowledge @xmath1.682 was achieved for verb and noun synset pairs in bulnet. a sequential pattern mining technique based on regular expressions as features for ml was proposed in @xcite and tested on polish and spanish. it was trained on "1500 pairs of base words with their derivatives". however, the annotation guidelines are unknown, semantics of the links was not taken into account, as well as the direction of derivation. finally, the accuracy of 82.33% was achieved with "53.5 thousand links in the network". word embeddings (word2vec and neural language models) were investigated in @xcite for the czech coarse-grained derivational relations. neural character encoder-decoder was applied to predict a derivative from a derivational base in @xcite . it used occurrence context too, but was limited to deverbal nouns. ## embedding methods in our experiments, we wanted to compare different methods for representing words (in fact lemmas) by vector spaces for the needs of recognition of semantic relations linking them, where all relations of interest are associated also with some relation between the word forms. first of all we used word embedding vectors, i.e. representation of words in dense spaces of real number vectors. word embeddings were often used in recognition of lexical semantic relations. we conducted experiments with both context-free methods and those that use word context information (acquired during the learning process). we also tried to model words using vectors representing their character structure. concerning the latter, we call such a representation bag of characters (henceforth boc). the vector for a word is simply constructed by counting the occurrences of different characters from the dictionary -i.e. simply letters of the polish alphabet. such a representation is an analogue of bag of words model used in information retrieval. it is relatively simple, but looses a lot of information related to object structures: documents and words in our case. it is known to be inferior in comparison to representations based on embeddings, so we expected it to be a kind of informative baseline. a bag of characters vector of is easily interpretable in terms of its values, but unfortunately it is insensitive to the order of occurrence of the elements, i.e. character sequences that are very important in expressing derivational changes and morphems. nevertheless, we wanted to check to what extent such an simplified representation is sufficient in representing derivational relations, which are characterised by relatively regular exchanges of characters in words. an example of such a vector is presented in figure 1 . ## classification experiments in order to compare the effectiveness of using different vector representations for the task of classifying derivational relations, we first used all vector versions to train a multi-class classifier based on an mlp neural network, as a classification model that seem to be in good balance between expressiveness and requirements for the size of a data set that is limited in our case (e.g. especially coverage for different relation types). we used the package default settings during learning the classifier, because our main focus was on different vector representations of examples. since bag of characters vectors are discrete in nature and their singular values are interpretable, we also decided to train classifiers using directly this representation, i.e. decision trees, both a single tree method and a random forest approach. in our experiments, we followed a multi-class classifier scheme. each example in the training and test data subsets is an instance of a derivational relation (i.e. a pair of lemmas: a derivational basis and a derivative) so in the experiments we did not assume the possibility of labelling a pair with the label 'no relation'. we examined each prepared vector representation in the following configurations: we called this vector a 3-way vector. this is shown in figure 2 . the 3-way representation was shown to be effective in recognition of wordnet relations, especially in combination with fasttext representation, e.g. @xcite . it is meant to represent semantic characteristic of both elements, but also to emphasise differences between them, together with the directions of the differences. the directions are potentially important for plwordnet morphosemantic relations, as they are all defined and edited in the direction from a derivational basis to the derivate (the derived word). for the final experiment, we also analysed combination of the two different representations. whole words were embedded using fasttext vectors and concatenated together with a difference vector obtained using the bag-of-characters technique. the aim of this experiment was to test whether combining a semantic representation based on word vectors and a discrete representation associated with an orthographic form would result in an improvement in the classification task. we implemented the classifier models for all experiments using the scikit-learn library @xcite . ## conclusions our research has shown that the limit of contextfree classification of derivational relations lies not in the representation of examples, but in the absence of any other source of information for the classifier. in the final version of the system for context-free classification of derivational relations, we decided to stay with bag of characters vectors, due to their simple human interpretability. we want to direct our further research to the study of derivation in the context of -both the preparation of datasets (such as a corpus) and methods for detecting and classifying relations.