Before revision:  In recent years, self-attention based models like Transformers and BERT have achieved state-of-the-art performance in several natural language understanding tasks. These kinds of models, however, are still restricted to short text sequences like sentences due to the quadratic computational time and space complexity of self-attention with respect to the input sequence length.
After revision:  In recent years, self-attention based models like Transformers and BERT have achieved state-of-the-art performance in the task of text matching. These models, however, are still restricted to short text sequences like sentences due to the quadratic computational time and space complexity of self-attention with respect to the input sequence length.