add the missing keymods to the pattern side as well