Before revision:  In the self-attention module, the word  correlation and positional correlation are computed separately with different parameterizations and then added together.
After revision:  In the self-attention module, the word contextual correlation and positional correlation are computed separately with different parameterizations and then added together.