Before revision:  We hypothesize that better contextual representations can be generated from the text encoder with richer positional information.
After revision:  We hypothesize that better contextual representations can be generated from the Transformer with richer positional information.