While one of the first steps in many NLP systems is selecting what pre-trained word embeddings to use, we argue that such a step is better left for neural networks to figure out by themselves. To that end, we introduce dynamic meta-embeddings, a simple yet effective method for the supervised learning of embedding ensembles, which leads to stateof-the-art performance within the same model class on a variety of tasks. We subsequently show how the technique can be used to shed new light on the usage of word embeddings in NLP systems.
Kiela et al. REF introduced dynamic metaembeddings, which was an effective method for the supervised learning of embedding ensembles.
Dynamic Meta-Embeddings for Improved Sentence Representations
EMNLP
Computer Science