Don't require "extra_runners" to be set in config