Before revision:  To verify this, we propose a segment-aware BERT , by replacing the token position embedding of Transformer with a combination of paragraphindex, sentenceindex ,  and token index embeddings.
After revision:  To verify this, we propose a segment-aware BERT , by replacing the token position embedding of Transformer with a combination of paragraphindex, sentenceindex , sentence, and token index embeddings.