Recovery: refactor RecoveryTarget state management

The PR rewrites the state controls in the RecoveryTarget family classes to make it easier to guarantee that:
- recovery resources are only cleared once there are no ongoing requests
- recovery is automatically canceled when the target shard is closed/removed
- canceled recoveries do not leave temp files behind when canceled. 

Highlights of the change:
1) All temporary files are cleared upon failure/cancel (see #7315 )
2) All newly created files are always temporary 
3) Doesn't list local files on the cluster state update thread (which throw unwanted exception)
4) Better resiliency to errors while trying to extract checksums (like missing files / corrupted shards)
5) Recoveries are canceled by a listener to IndicesLifecycle.beforeIndexShardClosed, so we don't need to explicitly call it.
6) Simplifies RecoveryListener to only notify when a recovery is done or failed. Removed subtleties like ignore and retry (they are dealt with internally)
can we return and assert that the CAS op was successful?