elasticsearch/elasticsearch — pull request #4586 by markharwood (2014-01-02 14:27:41+00:00)
Title: A new generic timeout handler and changes to existing search classes to use it

A more effective approach to time-limiting activities such as search requests. Special runtime exceptions can now short-cut the execution of long-running calls to Lucene classes and are caught and reported back, not as a fatal error but using the existing “timedOut” flag in results.

Phases like the FetchPhase can now exit early and so also have a timed-out status. The SearchPhaseController does its best to assemble whatever hits, aggregations and facets have been produced within the provided time limits rather than returning nothing and throwing an error.

ActivityTimeMonitor is the new central class for efficiently monitoring all forms of thread overrun in a JVM.
The SearchContext setup is modified to register the start and end of query tasks with ActivityTimeMonitor.
Store.java is modified to add timeout checks (via calls to ATM) in the low-level file access routines by using a delegating wrapper for Lucene's IndexInput and IndexInputSlicer.
ContextIndexSearcher is modified to catch and unwrap ActivityTimedOutExceptions that can now come out of the Lucene calls and report them as timeouts along with any partial results.
FetchPhase is similarly modified to deal with the possibility of timeout errors.