Indeed, it happens because all shards fail. I guess you want to verify that a normal query would fail but since we parse it only for the right indices we don't get back the failure using indices filter/query. I would then try and catch the exception that's thrown.