elasticsearch/elasticsearch — pull request #6303 by EricMCornelius (2014-05-23 19:30:05+00:00)
Title: Add short_circuit option to limit filters, triggering a shard search failure on overly broad searches

Proposing this to get a discussion started on how to identify when a limit filter is dropping potentially valid matching documents.

The case I'm attempting to tackle occurs when a user makes a broad search with high computational complexity, say a script filter. What I need is a mechanism to be able to terminate searches on shards after exceeding a certain number of filtered documents. A better timeout mechanism would also help to alleviate this problem.

Limit filters are currently the best means to restrict expensive searches, but unfortunately there is no context returned to identify when they are tripped. This makes it impossible to know when to inform a user that their query was too broad vs. trust that there are in fact no matches.

I see throwing an unchecked exception as a massive kludge, but I don't see a much better option to handle this sort of situation. Any feedback/recommendations welcome.