Recognition of Hyponymy and Meronymy Relations in Word Embeddings for P olish (2018)

Abstract: Word embeddings were used for the extraction of hyponymy relation in several approaches, but also it was recently shown that they should not work, in fact. In our work we verified both claims using a very large wordnet of Polish as a gold standard for lexico-semantic relations and word embeddings extracted from a very large corpus of Polish. We showed that a hyponymy extraction method based on linear regression classifiers trained on clusters of vectors can be successfully applied on large scale. We presented also a possible explanation for contradictory findings in the literature. Moreover, in order to show the feasibility of the method we extended it to the recognition of meronymy.

## introduction a very large wordnet, e.g. plwordnet @xcite describes many lexico-semantic relations, linking lexical units foot_0 (or word senses) by thousands of relation instances. however, even in a very large wordnet some relation instances can be omitted and typically wordnets are very biased towards only a few relations, e.g. hypernymy/hyponymy for nouns, with much smaller coverage for the other. measures of semantic relatedness constructed on the basis of word embeddings @xcite are known to express many different lexico-semantic relations, e.g. on a list of the k most related words to a word x we can typically find words associated with x by different relations. however, word embeddings are very heterogeneous with respect to the types of semantic associations expressed, also including syntactic and pragmatic relations. what is worse, word embeddings have problems with representing different senses of a word (typically only a few most frequent ones can be spotted on the lists) and with proper representation of less frequent words (even words with frequency 100-200 per 1g can be erroneously described, not mentioning @xmath0. so, the question is whether we can successfully recognise among the associations suggested by word embeddings those that correspond to lexico-semantic relations, i.e. whether we can interpret word embeddings in a meaningful way for humans. the works presented for english are contradictory even in the case of hypernymy -the relatively simplest relation: from successful extraction @xcite till denial of the feasibility of such a method @xcite . we want to re-approach this intriguing issues, first checking the contradictory points of the view on large corpora and comprehensive wordnet for polish, second by expanding this research with one more relation, a more difficult one, namely meronymy. this is a part of a broader work on the automated extraction of lexico-semantic relations that are under-represented in wordnets, e.g. in order to improve wordnet-based wsd. however, there are also many other works that report on successful extraction of hypernymy from contextual features, e.g. @xcite . ## search for relations in word embeddings the method proposed by @xcite intuitively seems to be correct: elements of the word embeddings are derived from the occurrence con-texts and correspond to the semantic features of words, while the similarity of features of two words correspond to the amount of overlapping in the values. this inspired us to revisit the method of @xcite in a new setting and confront it once again with the objections of @xcite . ## results for pairs of experiments performed using two different ways of the selection of negative examples, as well as for two different sizes of the vectors we checked the statistical significance of the differences. first we tested if the results obtained in different folds come from the normal distribution by applying shapiro-wilk test, e.g. for hypo-mono we obtained p value of 0.8082 for the random selection results and 0.8648 for the lexical split series, so with the confidence level of 0.05 we cannot reject the null hypothesis that the results come from a normal distribution. having normality confirmed, we applied t-student test to the differences between results, e.g. in the case of hypo-mono p value is 0.4583 and with the confidence level 0.05 we cannot reject the null hypothesis of the lack of a difference between both series of results. in the case of hypo-poly the fold results do not come from a normal distributions according to shapiro-wilk test, so we applied mann-whitney u test and the obtained p value of 0.008931 shows the lack of statistical significance of the differences. in a similar way we checked that the differences between results for different vector sizes are significant, namely hypo-mono vs hypo-mono300 and hypo-poly vs hypo-poly300. we did not analysed differences between the results for monosemous and polysemous words, but these differences are very visible. ## conclusions the claim of @xcite that supervised classifiers trained on combinations of word embeddings vectors are learning in fact that one of the words is a prototypical hypernym, instead recognising the pair as an instance of the hyponymy relation seemed to be well motivated. however, it was contradictory with intuition and many results reported in literature. one of them, namely @xcite , presented good results, but tested on a limited scale of only 412 words. in our work we verified both claims using a very large wordnet of polish (developed in a more linguistically oriented way and closer to the language data in corpora) as a gold standard for lexico-semantic relations and word embeddings extracted from a very large, automatically pre-processed corpus of polish. we showed that the method proposed by @xcite can be successfully applied to the extraction of the hyponymy relations. in series of carefully conducted and evaluated experiments we verified negatively the objections of @xcite . this contradiction can be due to different languages and datasets used, but also to the fact that they concentrated their attention on the use of svm classifiers only, while we showed that in some settings svm classifier can produce much worse results for this particular task. in addition we applied successfully the method of @xcite also to the recognition of meronymy achieving very good results tested on a very large data sample prepared manually. we plan to expand this approach to other relations, e.g. lexico-semantic relations manifested derivationally that are quite numerous in polish. we aim at building a semi-automated system for improving the density of relations in a wordnet. it will be also very valuable to continue the research on types of classifiers and experimental settings that make extraction methods of this types successful.