runner: allow max_in_flight to be overridden