Lifelong Word Embedding via Meta-Learning
Learning high-quality word embeddings is of significant importance in achieving better performance in many down-stream learning tasks.On one hand, traditional word embeddings are trained on a large scale corpus for general-purpose tasks, which are often sub-optimal for many domain-specific tasks.On the other hand, many domain-specific tasks do not have a large enough domain corpus to obtain high-quality embeddings.We observe that domains are not isolated and a small domain corpus can leverage the learned knowledge from many past domains to augment that corpus in order to generate high-quality embeddings.In this paper, we formulate the learning of word embeddings as a lifelong learning process.Given knowledge learned from many previous domains and a small new domain corpus, the proposed method can effectively generate new domain embeddings by leveraging a simple but effective algorithm and a meta-learner, where the meta-learner is able to provide word context similarity information at the domain-level.Experimental results demonstrate that the proposed method can effectively learn new domain embeddings from a small corpus and past domain knowledgesfootnote.Wealso demonstrate that general-purpose embeddings trained from a large scale corpus are sub-optimal in domain-specific tasks.
learning better domain embeddings via lifelong learning and meta-learningPresents a lifelong learning method for learning word embeddings.This paper proposes an approach to learn embeddings in new domains and significantly beats the baseline on an aspect extraction task. 