  For both human readers and pre-trained language models (PrLMs), lexical
diversity may lead to confusion and inaccuracy when understanding the
underlying semantic meanings of given sentences. By substituting complex words
with simple alternatives, lexical simplification (LS) is a recognized method to
reduce such lexical diversity, and therefore to improve the understandability
of sentences. In this paper, we leverage LS and propose a novel approach which
can effectively improve the performance of PrLMs in text classification. A
rule-based simplification process is applied to a given sentence. PrLMs are
encouraged to predict the real label of the given sentence with auxiliary
inputs from the simplified version. Using strong PrLMs (BERT and ELECTRA) as
baselines, our approach can still further improve the performance in various
text classification tasks.
