Allow all integer types in permutation vector

This was a problem on 64 bit machines with 32 bit integer BLAS.