fix flaky ReferenceInfosITest

forgot the rare case that all shards are on 1 node