Add local node to cluster state
Today, the tribe node needs the local node so it adds it when it starts, but other APIs would benefit from adding the local node, also, adding the local node should be done in a cleaner manner, where it belongs, which is right after the discovery service starts in the cluster service
closes #6811
[STORE]: Make use of Lucene build-in checksums

Since Lucene version 4.8 each file has a checksum written as it's
footer. We used to calculate the checksums for all files transparently
on the filesystem layer (Directory / Store) which is now not necessary
anymore. This commit makes use of the new checksums in a backwards
compatible way such that files written with the old checksum mechanism
are still compared against the corresponding Alder32 checksum while
newer files are compared against the Lucene build in CRC32 checksum.

Since now every written file is checksummed by default this commit
also verifies the checksum for files during recovery and restore if
applicable.

Closes #5924

This commit also has a fix for #6808 since the added tests in
`CorruptedFileTest.java` exposed the issue.

Closes #6808
[TEST] Subclass ElasticsearchTestCase in LoggingConfigurationTests
[TEST] Close TransportClient after it's used in BulkProcessorTests otherwise it will leave threads behind
[TEST] Do RollingUpgrade in BWC tests
[TEST] added ability to provide settings for external nodes in backwards compatibility tests

Closes #6809
[TEST] Wait for longer on slow nodes until replicating has kicked in
[STORE] Treat reading past EOF as an corrupted index when we fail to read segment infos
Aggregations: DateHistogramBuilder accepts String preOffset and postOffset

This is what DateHistogramParser expects so will enable the builder to build valid requests using these variables.
Also added tests for preOffset and postOffset since these tests did not exist

Closes #5586
[TEST] Fold SuggestActionTest into SuggestSearchTests

Instead of running the tests twice this commit just randomizes the API
that we use to return the suggestions.
Improve handling of failed primary replica handling
Out of #6808, we improved the handling of a primary failing to make sure replicas that are initializing are properly failed as well. After double checking it, it has 2 problems, the first, if the same shard routing is failed again, there is no protection that we don't apply the failure (which we do in failed shard cases), and the other was that we already tried to handle it (wrongly) in the elect primary method.
This change fixes the handling to work correctly in the elect primary method, and adds unit tests to verify the behavior
closes #6816
Do not ignore ConnectTransportException for shard replication operations

A ConnectTransportException should fail the replica shard

Closes #6183
Revert "Improve handling of failed primary replica handling"

This reverts commit 75ed24f6b6a96ac268f7b085550020252d94d908.
[Test] testCorruptionOnNetworkLayer had a typo in test name.
[TEST] Temporarily don't corrupt .del files since they are generational and we might pick the wrong one
[TEST] Fix CorruptedFileTest to always corrupt the latest delete generation if a .del file is picked
Improve handling of failed primary replica handling
Out of #6808, we improved the handling of a primary failing to make sure replicas that are initializing are properly failed as well. After double checking it, it has 2 problems, the first, if the same shard routing is failed again, there is no protection that we don't apply the failure (which we do in failed shard cases), and the other was that we already tried to handle it (wrongly) in the elect primary method.
This change fixes the handling to work correctly in the elect primary method, and adds unit tests to verify the behavior
The change also expose a problem in our handling of replica shards that stay initializing during primary failure and electing another replica shard as primary, where we need to cancel its ongoing recovery to make sure it re-starts from the new elected primary
closes #6825
More Like This: ensures selection of best terms is indeed O(n)

Previously the size of the priority queue was wrongly set to the total number
of terms. Instead, it should be set to 'maxQueryTerms'. This makes the
selection of best terms O(n), instead of O(n*log(n)).

Jira patch: https://issues.apache.org/jira/browse/LUCENE-5795

Closes #6657
[Gateway] set a default of 5m to `recover_after_time` when any to the `expected*Nodes` is set

The `recovery_after_time` tells the gateway to wait before starting recovery from disk. The goal here is to allow for more nodes to join the cluster and thus not start potentially unneeded replications. The `expectedNodes` setting (and friends) tells the gateway when it can start recovering even if the `recover_after_time` has not yet elapsed. However, `expectedNodes` is useless if one doesn't set `recovery_after_time`. This commit changes that by setting a sensible default of 5m for `recover_after_time` *if* a `expectedNodes` setting is present.

Closes #6742
Only use IndexShard instance to lookup recovery status
make sure we use the instance itself to look it up, and not the shard id, as we might get another instance
leftover from #6825