Fix a couple of bugs with retry query runner after testing it locally