Before revision: Adversarial training is effective in improving the robustness of neural networks . In NLP, languages are discrete in nature, separate tokens possess discrete semantics. Therefore, to incorporate adversarial trainingin sequence-level tasks, we introduce a novel training strategy: Text Adversarial Training with token-level perturbation.
After revision: Adversarial training is effective in improving the robustness of neural networks , while it cannot be easily adapted to natural language processing tasks since the embedding space is discrete. In natural language processing fields, virtual adversarial training is introduced since texts are discrete and cannot be perturbed by gradients directly. Alternatively, virtual adversarial training, which generates perturbations on the embedding space, is introduced in NLP tasks. Despite its success, existing virtual adversarial training methods generate perturbations roughly constrained by Frobenius normalization balls. To craft fine-grained perturbations, we propose a Token-Aware Virtual Adversarial Training method. We introduce a novel training strategy: Text Adversarial Training with token-level perturbation.