  We propose an effective consistency training framework that enforces a
training model's predictions given original and perturbed inputs to be similar
by adding a discrete noise that would incur the highest divergence between
predictions. This virtual adversarial discrete noise obtained by replacing a
small portion of tokens while keeping original semantics as much as possible
efficiently pushes a training model's decision boundary. Moreover, we perform
an iterative refinement process to alleviate the degraded fluency of the
perturbed sentence due to the conditional independence assumption. Experimental
results show that our proposed method outperforms other consistency training
baselines with text editing, paraphrasing, or a continuous noise on
semi-supervised text classification tasks and a robustness benchmark.
