Recursive neural models, which use syntactic parse trees to recursively generate representations bottom-up from parse children, are a popular new architecture, promising to capture structural properties like long-distance semantic dependencies. But understanding exactly which tasks this parse-based method is appropriate for remains an open question. In this paper we benchmark recursive neural models against sequential recurrent neural models, which are structured solely on word sequences. We investigate 4 tasks: (1) sentence-level sentiment classification; (2) matching questions to answer-phrases; (3) discourse parsing; (4) computing semantic relations (e.g., component-whole between nouns). We implement apply basic, general versions of recursive and recurrent models and apply to each task. Our analysis suggests that syntactic tree-based recursive models are very helpful for tasks that require representing long-distance relations between words (e.g., semantic relations between nominals), but may not be helpful in other situations, where sequence based recurrent models can produce equal performance. Our results offer insights on the design of neural architectures for representation learning.
Li et al. REF benchmark recursive neural models against sequential recurrent neural models on for nature language processing tasks including sentiment classification at the sentence level and phrase level.
When Are Tree Structures Necessary for Deep Learning of Representations?
EMNLP
Computer Science