Move always_miss check up front to get it over with.