PERF: Don't query solr again if we know there will be no results

Fixes #644