Get rid of permutation objects
Some dubious optimizations