Before revision: How to explicitly encode positional information into neural networks is an important problem in natural language processing. In the Transformer model , the positional information is simply encoded as embedding vectors, which are used in the input layer, or encoded as a bias term in the self-attention module.
After revision: How to explicitly encode positional information into neural networks is important in learning the representation of natural languages, such as BERT. Based on the Transformer architecture , the positional information is simply encoded as embedding vectors, which are used in the input layer, or encoded as a bias term in the self-attention module.