[TEST] Wait for yellow before verifying - sometimes the shard is not even started
[CORE] Remove explicit .cleanUp() on cache clear

Calling cache.cleanUp() is kind of like calling System.gc(), meaning
that we should never have (non-test) things that rely on this
functionality.

For the field data and filter cache, we already have a periodic process
that runs this .cleanUp(), so there is no need to block index
closing/clearing on it. Instead, we can clean the field data cache in
InternalTestCluster before we check the circuit breaker.

This can help tests that time out because cleaning the cache is taking
too long
[TEST] get trace logs for search packages
[TEST] wait for yellow before searching
[TEST] use ensureYellow()
Core: use compound file by default for merged segments < 10% of index size

Change Elasticsearch to use Lucene's defaults, to reduce file descriptor count.

Closes #8934

Closes #8919
Test: reduce join timeout in testFullRollingRestart

 Occasionally a the join thread successfully connected to a just closed node and which causes the subsequent join request to time out. It's default timeout 60s throws the test off when it waits for a cluster to form.
Test: missing {} from log command in indexRandom
[SEARCH] close active contexts on SearchService#close()

When we close a node all pending / active search requests need to be
cleared otherwise a node will wait up to 30 sec for shutdown sicne there
could be open scroll requests. This behavior was introduces in 1.5 such that
versions <= 1.4.x are not affected.

Closes #8940
Default _cat APIs to verbose

`?v=false` can be used if the headers are not desired.

Resolves #8922

Fixes #8927
Discovery: concurrent node failures can cause unneeded cluster state publishing

When a node fails (or closes), the master processes the network disconnect event and removes the node from the cluster state. If multiple nodes fail (or shut down) in rapid succession, we process the events and remove the nodes one by one. During this process, the intermediate cluster states may cause the node fault detection to signal the failure of nodes that are not yet removed from the cluster state. While this is fine, it currently causes unneeded reroutes and cluster state publishing, which can be cumbersome in big clusters.

Closes #8804
Closes #8933
Docs: Adding REST ACL plugin

Closes #8925
Update percolate.asciidoc

Updating the `associated` spelling

Closes #8907
Docs: fix a typo in cluster stats documentation example

Closes #8898
[CORE] Remove IndexEngine

IndexEngine was an abstraction where we had index-level engines (instead
of shard-level) that could store meta information about the index. It
was never actually used by Elasticsearch, and only there for plugins.

This removes it, because it is a confusing abstraction and not needed,
no plugins should be implementing their own IndexEngines.
Remove Gateway abstraction

We only have a single gatweway since es 1.3. There is no need to keep all
these abstractsion and nested packages. We can fold most of it into simpler
structures.
Remove unused code