Recovery: refactor RecoveryTarget state management

The PR rewrites the state controls in the RecoveryTarget family classes to make it easier to guarantee that:
- recovery resources are only cleared once there are no ongoing requests
- recovery is automatically canceled when the target shard is closed/removed
- canceled recoveries do not leave temp files behind when canceled. 

Highlights of the change:
1) All temporary files are cleared upon failure/cancel (see #7315 )
2) All newly created files are always temporary 
3) Doesn't list local files on the cluster state update thread (which throw unwanted exception)
4) Better resiliency to errors while trying to extract checksums (like missing files / corrupted shards)
5) Recoveries are canceled by a listener to IndicesLifecycle.beforeIndexShardClosed, so we don't need to explicitly call it.
6) Simplifies RecoveryListener to only notify when a recovery is done or failed. Removed subtleties like ignore and retry (they are dealt with internally)
I actually think we should use AbstractRunnable here

```Java

     threadPool.generic().execute(new AbstractRunnable() {
            @Override
            public void onFailure(Throwable t) {
              // call listener
            }

            @Override
            protected void doRun() throws Exception {
 RecoveriesCollection.StatusRef statusRef = onGoingRecoveries.getStatus(recoveryId);
                if (statusRef == null) {
                    return;
                }
                try {
                    doRecovery(statusRef.status());
                } finally {
                    // make sure we never interrupt the thread after we have released it back to the pool
                    statusRef.status().clearWaitingRecoveryThread(Thread.currentThread());
                    statusRef.close();
                }
            }
```
WDYT?