Fix performance regression getting format class for Unit

Discovered by asv.