[RECOVERY] Fix potential NPE status reference must be not null to fail recovery
Update Sonatype release link in build_release.py
[TEST] fixed scriptedMetricTests

The tests were failing because there was a shard which didn't get any documents and the tests assumed all shards had documents. This commit fixes this assumption
[docs] fix typo in getting-started

Closes #8354
Discovery: a more lenient wait joinThread when stopping

When a node stops, we cancel any ongoing join process. With #8327, we improved this logic and wait for it to complete before shutting down the node. In our tests we typically shutdown an entire cluster at once, which makes it very likely for nodes to be joining while shutting down. This introduces a race condition where the joinThread.interrupt can happen before the thread starts waiting on pings which causes shutdown logic to be slow. This commits improves by repeatedly trying to stop the thread in smaller waits.

Another side effect of the change is that we are now more likely to ping ourselves while shutting down, we results in an ugly warn level log. We now log all remote exception during pings at a debug level.

Closes #8359
Revert "Discovery: a more lenient wait joinThread when stopping"

This reverts commit 83d9dab79882700fb21cb496282bafdb1d28a5ca.
[CORE] Cut over to Path API for file deletion

Today we use the File API for file deletion as well as recursive
directory deletions. This API returns a boolean if operations
are successful while hiding the actual reason why they failed.
The Path API throws and actual exception that might provide better
insights and debug information.

Closes #8366
[TRANSLOG] Translog leaks filehandles if it's corrupted or truncated

If the translog file is corrupted or truncated the stream is never closed
and the filehandle leaks. This commit closes the stream in the case of an
exception.
[STORE] increment store in NoneIndexShardGateway and either check if dir is null or different but not both
Refactor shard recovery from anonymous class to ShardRecoveryHandler

Previously the bulk of our shard recovery code was in a 300-line
anonymous class in `RecoverySource`. This made it difficult to find and
more difficult to read.

This factors out that code into a `ShardRecoveryHandler` class, adding
javadocs for each function and phase of the recovery, as well as
comments explaining some of the more esoteric functions performed during
recovery.

It's hoped that this will help more people understand Elasticsearch's
recovery procedure.

No *major* functionality has changed, only typo corrections, some minor
allocation improvements and logging clarification changes.
Eclipse: change import settings to have the same order as IntelliJ IDEA.
Mark ShardRecoveryShandler class as final
Discovery: don't wait joinThread when stopping

When a node stops, we cancel any ongoing join process. With #8327, we improved this logic and wait for it to complete before shutting down the node. However, the joining thread is part of a thread pool and will not stop until the thread pool is shutdown.

Another issue raised by the unneeded wait is that when we shutdown, we may ping ourselves - which results in an ugly warn level log. We now log all remote exception during pings at a debug level.

Closes #8359
[STATE] Observe cluster state on health request

Today we use busy waiting and sampling when we execute HealthReqeusts
on the master. This is tricky sicne we might sample a not yet fully applied
cluster state and make a decsions base on the partial cluster state. This can
lead to ugly problems since requests might be routed to nodes where shards are
already marked as relocated but on the actual cluster state they are still started.
Yet, this window is very small usually it can lead to ugly test failures.

This commit moves the health request over to a listener pattern that gets the actual
applied cluster state.

Closes #8350