Before revision:  Most of them are based on the Transformer architecture , which distinguishes tokens with the token position index of the input sequence. However, sentence index and paragraph index are also important to indicate the token position in a document.
After revision:  Most of them are based on the Transformer architecture , which distinguishes tokens with the token position index .