Update mapping on master in async manner
Today, when a new mapping is introduced, the mapping is rebuilt (refreshSource) on the thread that performs the indexing request. This can become heavier and heavier if new mappings keeps on being introduced, we can move this process to another thread that will be responsible to refresh the source and then send the update mapping to the master (note, this doesn't change the semantics of new mapping introduction, since they are async anyhow).
When doing so, the thread can also try and batch as much updates as possible, this is handy especially when multiple shards for the same index exists on the same node. An internal setting that can control the time to wait for batches is also added (defaults to 0).

Testing wise, a new support method on ElasticsearchIntegrationTest#waitForConcreteMappingsOnAll to allow to wait for the concrete manifestation of mappings on all relevant nodes is added. Some tests mistakenly rely on the fact that there are no more pending tasks to mean mappings have been updated, so if we see, timing related, failures down later (all tests pass), then those will need to be fixed to wither awaitBusy on the master for the new mapping, or in the rare case, wait for the concrete mapping on all the nodes using the new method.
closes #6648

allow to change the additional time window dynamically

better sorting on mappers when refreshing source
also, no need to call nodes info in test, we already have the node names

clean calls to mapping update to provide doc mapper and UUID always
also use the internal cluster support method to get the list of nodes an index is on

reverse the order to pick the latest change first

remove unused field

and fix constructor param

move to start/stop on mapping update action

randomize INDICES_MAPPING_ADDITIONAL_MAPPING_CHANGE_TIME
[TEST] properly wait for mappings when needed
[TEST] properly wait for mapping on master node
add helper method to do so, by not assuming that the mapping will exists right away by waiting for green or refreshing...
Recovery from local gateway should re-introduce new mappings
The delayed mapping intro tests exposed a bug where if a new mapping is introduced, yet not updated on the master, and a full restart occurs, reply of the transaction log will not cause the new mapping to be re-introduced.
closes #6659

add comment on the method
Add ability to restore partial snapshots

Closes #5742
Improve deletion of corrupted snapshots

Makes it possible to delete snapshots that are missing some of the metadata files. This can happen if snapshot creation failed because repository drive ran out of disk space.

Closes #6383
[TEST] Turn off double write check for restore
Docs: Removed Sense mention
Update histogram-facet.asciidoc

Spotted a typo, which I've fixed.
[TEST] Fix ExistsMissingTests failures.

They were due to a combination of mappings propagation delays and the behavior
of MapperService.smartName(String) so mappings are now configured up-front.
start mapping service earlier to be available for recovery
Fixed JSON in fielddata docs
Core: The ignore_unavailable=true setting also ignores indices that are closed.

Closes #6471
Closes #6475
[TEST] Improve reproducibility of mappings propagation delays related issues.
Fixed link to native Java client

Closes #6590
Core: Prevent non segment readers from entering the filter cache and the field data caches.
Percolator: Never cache filters and field data in percolator for the percolator query parsing part.

Closes #6553
[TEST] Fix possible race condition in checksum name generator

When three threads are trying to write checksums at the same time, it's possible for all three threads to obtain the same checksum file name A. Then the first thread enters the synchronized section, creates the file with name A and exits. The second thread enters the synchronized section, checks that A exists, creates file A+1 and exits the critical section. Then it proceeds to clean up  and deletes all checksum files including A. If it happens before the third thread enters the synchronized section, it's possible for the third thread to check for A and since it no longer exists create the checksum file A the second time, which triggers "file _checksums-XXXXXXXXXXXXX was already written to" exception in MockDirectoryWrapper and fails recovery.
[INDEX] Ensure `index.version.created` is consistent

Today `index.version.created` depends on the version of the master
node in the cluster. This is potentially causing new features to be
expected on shards that didn't exist when the index was created.
There is no notion of `where was the shard allocated first` such that
`index.version.created` can't be reliably used as a feature flag.

With this change the `index.version.created` can be reliably used to
determin the smallest nodes version at the point in time when the index
was created. This means we can safely use certain features that would
for instance require reindeing and / or would not work if not the
entire index (all shards and segments) have been created with a certain
version or newer.

Closes #6660
[TEST] Prevent usage of System Properties in the InternalTestCluster

All settings should be passes as settings and the enviroment should not
influence the test cluster settings. The settings we care about ie.
`es.node.mode` and `es.logger.level` should be passed via settings.
This allows tests to override these settings if they for instance need
`network` transport to operate at all.

Closes #6663
Core: cancel entire recovery if shard closes on target node during the recovery operations.

Closes #6645