Move old knife config out of the way