Before revision:  To verify this, we propose a segment-aware BERT , by replacing the token position embedding of Transformer with a combination of paragraphindex, sentenceindex ,  and token index embeddings.
After revision:  To verify this, we propose a segment-aware Transformer (Segatron) , by replacing the token position embedding of Transformer with a combination of paragraphindex, sentenceindex , and token index embeddings.