Before revision:  In the self-attention module,  the word contextual correlation and positional correlation are computed separately with different parameterizations and then added together.
After revision:  In the self-attention module, the word contextual correlation and positional correlation separately with different parameterizations and then added together.