Simplify permute() avoiding moving data when it stays the same.