Avoid creating new instance of command runner for each query

cc @calonso as he worked on this originally
Stop failing fast for rspec