Word Embeddings, which encode semantic and syntactic features, have achieved success in many natural language processing tasks recently . However, the lexical semantics captured by these embeddings are difficult to interpret due to the dense vector representations. In order to improve the interpretability of word vectors, we explore definition modeling task and propose a novel framework (Semantics-Generator) to generate more reasonable and understandable context-dependent definitions. Moreover, we introduce usage modeling and study whether it is possible to utilize distributed representations to generate example sentences of words. These ways of semantics generation are a more direct and explicit expression of embedding's semantics . Two multi-task learning methods are used to combine usage modeling and definition modeling . To verify our approach, we construct Oxford-2019 dataset, where each entry contains word, context, example sentence and corresponding definition. Experimental results show that Semantics-Generator achieves the state-of-the-art result in definition modeling and the multi-task learning methods are helpful for two tasks to improve the performance.
Word embeddings are substantially successful in capturing semantic relations among words . However, these lexical semantics are difficult to be interpreted. Definition modeling provides a more intuitive way to evaluate embeddings by utilizing them to generate natural language definitions of corresponding words. This task is of great significance for practical application and in-depth understanding of word representations. We propose a novel framework for definition modeling, which can generate reasonable and understandable context-dependent definitions. Moreover, we introduce usage modeling and study whether it is possible to utilize embeddings to generate example sentences of words. These ways are a more direct and explicit expression of embedding's semantics for better interpretability. We extend the single task model to multi-task setting and investigate several joint multi-task models to combine usage modeling and definition modeling together. Experimental results on existing Oxford dataset and a new collected Oxford-2019 dataset show that our single-task model achieves the state-of-the-art result in definition modeling and the multi-task learning methods are helpful for two tasks to improve the performance.
R
Embeddings, which encode semantic and syntactic features, have achieved success in many natural language processing tasks recently
embeddings are substantially successful in capturing semantic relations among words
R
the lexical semantics captured by these embeddings
these lexical semantics
R
interpret due to the dense vector representations. In order to improve the interpretability of word vectors, we explore definition modeling task and
be interpreted. Definition modeling provides a more intuitive way to evaluate embeddings by utilizing them to generate natural language definitions of corresponding words. This task is of great significance for practical application and in-depth understanding of word representations. We
R
(Semantics-Generator) to generate more
for definition modeling, which can generate
R
distributed representations
embeddings
D
of semantics generation
R
. Two
for better interpretability. We extend the single task model to
R
learning methods are used
setting and investigate several joint multi-task models
R
. To verify our approach, we construct
together. Experimental results on existing Oxford dataset and a new collected
R
dataset, where each entry contains word, context, example sentence and corresponding definition. Experimental results show that Semantics-Generator
dataset show that our single-task model