[TEST] wait unitl all machines joined the cluster
Snapshot/Restore: switch to write once mode for snapshot metadata files

This commit removes creation of in-progress snapshot file and makes creation of the final snapshot file atomic.

Fixes #8696
Clarify InternalIndexShard callbacks

This commit changes internal callback to be clear
about when they are called and also provide the
exception that was potentially thrown as a callback argument.

Closes #5945
Docs: add randomizedtesting-runner to testing-framework.asciidoc

Close #8450
[ALLOCATION] Speed-up disk-threshold decider

Instead of iterating all shards of all indices to get all relocating
shards for a given node we can just use the RoutingNode#shardsWithState
method and fetch all INITIALIZING / RELOCATING shards and check if they
are relocating. This operation is much faster and uses pre-build
data-structures.

Relates to #6372
Tests: move RecoverAfterNodesTests to org.elasticsearch.gateway.local and increase BLOCK_WAIT_TIMEOUT to 10s

The tests were still in org.elasticsearch.gateway.none but the none gateway was removed.
Packaging: Check if proc file exists before calling sysctl

The packaged init scripts could return an error, if the file
/proc/sys/vm/max_map_count was not existing and we still called
sysctl.

This is primarly to prevent confusing error messages when elasticsearch
is started under virtualized environments without a proc file system.

Closes #4978
Scripting: Add explicit error message when script_score script returns NaN

When a scoring script returns not a number, the current message is confusing (IllegalArgumentException[docID must be >= 0 and < maxDoc=3 (got docID=2147483647)]). This commit adds the error message ScriptException[script score function returns a wrong score: NaN].

Closes #2426
Add index.data_path and index.data_template settings

This allows specifying the path an index will be as well as the layout
of the folder for each individual shard.

`index.data_path` and `index.data_template` are both specified in the
settings when creating an index, and can not be dynamically changed.

`index.data_path` is the path to the base directory for the index.

`index.data_template` is a mustache template that will be used when
resolving the path for an index or translog. It currently supports three
injected variables:

{{index_name}} - the name of the index
{{node_id}} - numeric id of the node
{{shard_num}} - shard number

An example request would look like:

POST /myindex
{
  "settings": {
    "number_of_shards": 2,
    "data_path": "/tmp/myindex",
    "data_template": "i_{{index_name}}/{{node_id}}/shard_{{shard_num}}"
  }
}

And would put data in /tmp/myindex/i_myindex/0/shard_0 and
/tmp/myindex/i_myindex/0/shard_1