[Test] Let SuggestStatsTests.testSimpleStats do more work

The test verifies that stats are measure by checking timeInMillis>0. On fast machines the suggestions are done in < 1 millis time. The tests now index documents (to power suggestions) and does multiple suggestions per iterations to slow things down.
A new ClusterStateStatus to indicate cluster state life cycles

When the ClusterService applies a new cluster state, it is first assigned as the new active one and then all listeners are called. Some of ES's features sample the current state and try to take action on it (for example index a document). If that fails, they will wait for change in the cluster state and try again (for example, wait for a shard to start and try indexing again).

If you're unlucky you sample the state after it has been assigned as the "active" state but before all listeners has done the work. In this cases the action take (i.e., indexing a doc) will still fail (as the shard is not yet started) but waiting for a new state may take a long time or fail.

 This commit adds a new ClusterStateStatus that allows to better track the stages a cluster state goes through (currently `RECEIVED`, `BEING_APPLIED` & `APPLIED`). This allows detecting that a cluster state is not yet fully applied and retry without waiting for a new state to arrive.

 This commits also adds a utility class , ClusterStateObserver, to make this pattern slightly simpler and avoid common pit falls.
Feedback round + some cleanup
Added ObservingContext to unify access to a couple of observing related variables.
minor tweak to make the code more readable
Bundeled lastObservedState and lastObservedStatus into a single class
Sampled wrong cluster state when evaluating change (though shouldn't be a problem). Add logging.
Missing return at ClusterStateObserver.ValidationPredicate.apply

Also added more logging
Added more logging to SimpleFacetTests
Removed trace logs and added progress logs.
extra fix after rebasing