Fix performance blunder, a.k.a prevent a lot of unneeded allocations