Hypernymy Detection for Low-Resource Languages via Meta Learning (2020)

Abstract: Hypernymy detection, a.k.a, lexical entailment, is a fundamental sub-task of many natural language understanding tasks. Previous explorations mostly focus on monolingual hypernymy detection on high-resource languages, e.g., English, but few investigate the low-resource scenarios. This paper addresses the problem of low-resource hypernymy detection by combining high-resource languages. We extensively compare three joint training paradigms and for the first time propose applying meta learning to relieve the low-resource issue. Experiments demonstrate the superiority of our method among the three settings, which substantially improves the performance of extremely low-resource languages by preventing over-fitting on small datasets.

## introduction hypernymy is a fundamental asymmetric lexicosemantic relation. it expresses is-a relationship between concepts and is widely used to build taxonomies @xcite or large-scale knowledge bases @xcite . lexicosemantic patterns (e.g., x such as y) are generally employed to harvest benchmark datasets or resources from large english corpus due to their high precision @xcite . however, hearstlike patterns of english can not be easily transferred to other languages such as chinese. creating high-quality hypernymy benchmarks for other languages requires much more human-annotation efforts and hypernymy detection in those languages remains low-resource tasks @xcite . in this paper, we focus on the question: how could we make full use of hypernymy pairs of high-resource languages such as english for other low-resource languages, e.g., japanese and thai? to investigate this question, we firstly assume a strong feasibility of semantic relation transfer across languages, which is in line with existing findings on human cognition. @xcite uncovered the universal conceptual structure of human lexical semantics among cross-lingual dictionaries and revealed the language-independent attribute for semantic similarity of concepts. @xcite studied cross-lingual training by simply merging high-resource language pairs and lowresource language ones, which is prone to overfitting to low-resource ones. based on the above interesting findings and the datasets in @xcite , we study three training paradigms of combining training data from multiple different languages, i.e., cross-lingual training, multilingual training, as well as meta learning. to the best of our knowledge, meta learning algorithms have not been previously applied to hypernymy detection. we propose applying meta learning algorithms in low-resource hypernymy detection and perform extensive comparisons with multilingual training. meta-learning algorithms aim at learning language-independent models and then fine-tuning on multiple languages with minimal training instances. in our experiments, we further explore the two following questions: the results for question 1 are surprising. obvious improvement is observed from neither bilingual cross-training nor multilingual training. the perfor-mance even drops on extremely low-resource languages as the models easily over-fit low-resource language datasets. meta learning algorithms, on the other hand, significantly relieve these cases by learning good model initialization for all languages. in the end, meta learning achieves the best performance among three training paradigms, which answers the main questions of this work. ## training settings in this section, we first introduce the base supervised model for hypernymy detection, and then illustrate three joint training paradigms. ## conclusion transferring lexical knowledge across languages are important especially for low-resource cases. in this paper, we investigate three joint train-ing paradigms for detecting hypernymy in lowresource languages. we show that simple multilingual training is not helpful for all tasks and we significantly improve the performance using meta learning. our study demonstrates the feasibility and effectiveness to combine high-and lowresource data to jointly train hypernymy detection models.