[TEST] Ensure all benchmarks are aborted on failure and latches are counted down
Fix bug for BENCH thread pool size == 1

On small hardware, the BENCH thread pool can be set to size 1. This is
problematic as it means that while a benchmark is active, there are no
threads available to service administrative tasks such as listing and
aborting. This change fixes that by executing list and abort operations
on the GENERIC thread pool.

Closes #6174
Fix debug logging message for put template action
[TEST] Randomize number of available processors

We configure the threadpools according to the number of processors which is
different on every machine. Yet, we had some test failures related to this
and #6174 that only happened reproducibly on a node with 1 available processor.
This commit does:
  * sometimes randomize the number of available processors
  * if we don't randomize we should set the actual number of available processors
    in the settings on the test node
  * always print out the num of processors when a test fails to make sure we can
    reproduce the thread pool settings with the reproduce info line

Closes #6176
Added some debug logs to the recovery process
Added meta data and routing version to cluster state's pretty print
When sending shard start/failed message due to a cluster state change, use the master indicated in the new state rather than current

This commit also adds extra protection in other cases against a master node being de-elected and thus being null.

Closes #6189
Unified MetaData#concreteIndices methods into a single method that accepts indices (or aliases) and indices options

Added new internal flag to IndicesOptions that tells whether aliases can be resolved to multiple indices or not.

Cut over to new metaData#concreteIndices(IndicesOptions, String...) for all the api previously using MetaData#concreteIndices(String[], IndicesOptions) and removed old method, deprecation is not needed as it doesn't break client code.

Introduced constants for flags in IndicesOptions for more readability

Renamed MetaData#concreteIndex to concreteSingleIndex, left method as a shortcut although it calls the common concreteIndices that accepts IndicesOptions and multipleIndices
Shade mustache into org.elasticsearch.common package

Previously we shared the jar but never rewrote the packages such
that the shading had no effect.

Closes #6192
Randomize CMS settings in index template

This commit adds randomization for:
 * `index.merge.scheduler.max_thread_count`
 * `index.merge.scheduler.max_merge_count`

This commit also moves to use
EsExecutors#boundedNumberOfProcessors(Settings) to default
configure the default `max_thread_count` for better reproducibility

Closes #6194
Check that a plugin is Lucene compatible with the current running node using `lucene` property in `es-plugin.properties` file.

* If plugin does not provide `lucene` property, we consider that the plugin is compatible.
* If plugin provides `lucene` property, we try to load related Enum org.apache.lucene.util.Version. If this fails, it means that the node is too "old" comparing to the Lucene version the plugin was built for.
* We compare then two first digits of current node lucene version against two first digits of plugin Lucene version. If not equal, it means that the plugin is too "old" for the current node.

Plugin developers who wants to launch plugin check only have to add a `lucene` property in `es-plugin.properties` file. If you are using maven to build your plugin, you can do it like this:

In `pom.xml`:

```xml
    <properties>
        <lucene.version>4.6.0</lucene.version>
    </properties>

    <build>
        <resources>
            <resource>
                <directory>src/main/resources</directory>
                <filtering>true</filtering>
            </resource>
        </resources>
    </build>
```

In `es-plugin.properties`, add:

```properties
lucene=${lucene.version}
```

BTW, if you don't already have it, you can add the plugin version as well:

```properties
version=${project.version}
```

You can disable that check using `plugins.check_lucene: false`.
Add simple example to XContentParser how to obtain an instance of it
RecoveryID should not be a per JVM but per Node

Today the RecovyerID is taken from a static atomic long which
is essentially a per JVM ID. We run the tests within the same
JVM and that means we don't really simulate what happens in
production environments. Instead we should use a per node generated
ID.
Remove dead code
[TEST] Do not filter gz files
added install instruction with apt

Closes #6206
Use UTF-8 as string encoding
Add some finals where appropriate
Use EnumSet rather than static mutable arrays

ClusterBlockLevel uses arrays but should use EnumSets instead
Remove SoftReferences from StreamInput/StreamOutput

We try to reuse character arrays and UTF8 writers with softreferences.
SoftReferences have negative impact on GC and should be avoided in
general. Yet in this case it can simply replaced with a per-stream
Bytes/CharsRef that is thread local and has the same lifetime as the
stream.