Before revision:  However, we observed in trained models, the diagonal attention matrices indicating the redundancy of the corresponding attention heads.
After revision:  However, for trained models, the diagonal attention matrices indicating the redundancy of the corresponding attention heads.