Internal: add indices setter to IndicesRequest interface

We currently expose generic getters for indices and indicesOptions on the IndicesRequest interface. This commit adds a generic setter as well, which can be used to set the indices to a request. The setter impl throws `UnsupportedOperationException` if called on internal requests. Also throws exception if called on single index operations, since it accepts an array as argument.

Closes #7734
[TEST] Reimplement AckTests#testDeleteWarmerNoAcknowledgement

This test was not testing what it was supposed to test. This commit
implements the test as an actual delete warmer test without ack
returned.
[CORE] Add AbstractRunnable support to ThreadPool to simplify async operation on bounded threadpools

today we have to catch rejected operation exceptions in various places
and notify an ActionListener. This pattern is error prone and adds a lot
of boilerplait code. It's also easy to miss catching this exception
which only is relevant if nodes are under high load. This commit adds
infrastructure that makes ActionListener first class citizen on async
actions.

Closes #7765
[DOC] Add GET Alias API note to breaking changes

Note explains that GET Alias API now supports IndicesOptions and will error if a index is missing
[TEST] close REST test execution context only if not null

The context can be null when REST tests are disabled via sysprop.
Aggregations: Removes isSingleUserCriteria check

This change removes the backwards compatibility workaround that checks that a compoundOrder originated from a single user defined criteria for the purposes of serialising to older versioned nodes.
[TEST] allow to fully disable REST tests included parsing via -Dtests.rest=false

We currently look for REST tests on file system although they are disabled. We should not do that and move the check earlier on. This way third parties using our test infra, which don't have REST tests on file system, can effectively disable the REST tests, otherwise they would get initialization error despite having disabled them.  The downside is that the number of tests visualized is going to be zero instead of the real number of parsed REST tests, but there is nothing we can do about this. Tests get ignored anyways.
[ENGINE] try increment store before searcher is acquired

InternalEngine#refreshNeeded must increment the ref count on the
store used before it's checking if the searcher is current since
internally a searcher ref is acquired and if that happens concurrently
to a engine close it might violate the assumption that all files
are closed when the store is closed.

This commit also converts some try / finally into try / with.
[TEST] move REST tests to their own test group

Closes #7795
Bulk API: Do not fail whole request on closed index

The bulk API request was marked as completely failed,
in case a request with a closed index was referred in
any of the requests inside of a bulk one.

Implementation Note: Currently the implementation is a bit more verbose in order to prevent an instanceof check and another cast - if that is fast enough, we could execute that logic only once at the beginning of the loop (thinking this might be a bit overoptimization here).

Closes #6410
Bulk Request : Add Document Request

This file was missing.
Revert "Bulk Request : Add Document Request"

This reverts commit 86f575dcea31e2e1486a17ed248708084905aeea.
Revert "Bulk API: Do not fail whole request on closed index"

This reverts commit 405e5816b827db5823e6c2ed9c1bbe9fa4629355.
Bulk API: Do not fail whole request on closed index

The bulk API request was marked as completely failed,
in case a request with a closed index was referred in
any of the requests inside of a bulk one.

Implementation Note: Currently the implementation is a bit more verbose in order to prevent an instanceof check and another cast - if that is fast enough, we could execute that logic only once at the
beginning of the loop (thinking this might be a bit overoptimization here).

Closes #6410
Add option to take currently relocating shards' sizes into account

When using the DiskThresholdDecider, it's possible that shards could
already be marked as relocating to the node being evaluated. This commit
adds a new setting `cluster.routing.allocation.disk.include_relocations`
which adds the size of the shards currently being relocated to this node
to the node's used disk space.

This new option defaults to `true`, however it's possible to
over-estimate the usage for a node if the relocation is already
partially complete, for instance:

A node with a 10gb shard that's 45% of the way through a relocation
would add 10gb + (.45 * 10) = 14.5gb to the node's disk usage before
examining the watermarks to see if a new shard can be allocated.

Fixes #7753
Relates to #6168
Indexed Scripts/Templates :  Cleanup

This contains several cleanups to the indexed scripts.
Remove the unused FetchSourceContext from the Get request..
Add lang,_version,_id to the REST GET API.
Removes the routing from GetIndexedScriptRequest since the script index is a single shard that is replicated across all nodes.
Fix backward compatible template file reference
Before 1.3.0 on disk scripts could be referenced by requesting
````
_search/template

{
  "template" : "ondiskscript"
}
````
This was broken in 1.3.0 by requiring
````
{
  "template" :
  {
    "file" : "ondiskscript"
  }
}
````
This commit restores the previous behavior.
Remove support for preference, realtime and refresh
These parameters don't make sense anymore for indexed scripts as we always force the preference to _local and
always refresh after a Put to the indexed scripts index.

Closes #7568
Closes #7559
Closes #7647
Closes #7567
Internal: remove ForceSyncDirectory

Historical code, not used anymore.

Closes #7804
Core: Check if from + size don't cause overflow and fail with a better error.

Closes #7778
Discovery: Give a unique id to each ping response

During discovery a node gossips with other nodes to discover the current state of the cluster - what nodes are out there, what version they use and most importantly whether there is an active master out there. During this ping process we may end up in a situation where old information is mixed with new. This is comment if a couple of master election happen in rapid succession.

This commit adds a monotonically increasing id to each ping response. This makes it easy to always select the last ping from every node.

Closes #7769
Test: RecoveryWhileUnderLoadTests.recoverWhileRelocating should report cluster state when failing to reach green