Before revision:  We fist craft perturbations that are initialized using a fine-grained token-level accumulated perturbations. Then we constrain these perturbations considering that inputs are separate tokens, rather than constraining them under a naive normalization ball . We validate the effectiveness of such normalization method using large-scale Transformer-based language models. Experiments on GLUE benchmark and NER task show that our adversarial training strategy improves the performances on various tasks including text classification and sequence labeling .
After revision:  We fist craft perturbations that are initialized using a fine-grained token-level accumulated perturbation vocabulary to initialize the perturbations better and use a token-level normalization ball to constrain these perturbations pertinently. Experiments show that our adversarial training strategy improves the performances on various tasks including text classification and sequence labeling .