  We study a family of data augmentation methods, substructure substitution
(SUB2), for natural language processing (NLP) tasks. SUB2 generates new
examples by substituting substructures (e.g., subtrees or subsequences) with
ones with the same label, which can be applied to many structured NLP tasks
such as part-of-speech tagging and parsing. For more general tasks (e.g., text
classification) which do not have explicitly annotated substructures, we
present variations of SUB2 based on constituency parse trees, introducing
structure-aware data augmentation methods to general NLP tasks. For most cases,
training with the augmented dataset by SUB2 achieves better performance than
training with the original training set. Further experiments show that SUB2 has
more consistent performance than other investigated augmentation methods,
across different tasks and sizes of the seed dataset.
