Before revision:  In the self-attention module,  the word contextual correlation and positional correlation are computed separately with different parameterizations and then added together.
After revision:  In the self-attention module, TUPE computes the word contextual correlation and positional correlation are computed separately with different parameterizations and then added together.