Sequence labeling architectures use word embeddings for capturing similarity, but suffer when handling previously unseen or rare words. We investigate character-level extensions to such models and propose a novel architecture for combining alternative word representations. By using an attention mechanism, the model is able to dynamically decide how much information to use from a word-or character-level component. We evaluated different architectures on a range of sequence labeling datasets, and character-level extensions were found to improve performance on every benchmark. In addition, the proposed attention-based architecture delivered the best results even with a smaller number of trainable parameters.
Rei et al. REF presented an architecture that amalgamates character-based word embedding by using an attention mechanism, surpassing the architecture based on concatenating the word-and character-level representations.
Attending to Characters in Neural Sequence Labeling Models
ArXiv
ArXiv
Computer Science