Enable max/min fast path for all non-iter cases