Before revision:  Our analysis also provides insights on the role of self-attention mechanism in modeling certain behavior and the influence of positional encoding schemes on the learning and generalization ability of the model.
After revision:  Our analysis also provides insights on the role of self-attention mechanism in modeling certain behaviors and the influence of positional encoding schemes on the learning and generalization ability of the model.