More generic way to decide on the permutation