Make sure we do a keys_only query when it makes sense