Backport Recovery / Snapshot file identity improvements to 1.3

We saw lots of issues with hash collisions which have been fixed in 1.4 but back then it seemed like an improvement rather than a bugfix. I think in the meanwhile we should really declare it as a bugfix and port it into `1.3` This backport PR includes fixes for #7434 & #7351 as well as related fixes. 

I ran BWC tests against 1.3.2 and 1.2.4 as well as the entire test suite multiple times but I think this need a deep review.
We should remember to somehow back-back-port it to 1.4, which has comparison to `org.elasticsearch.Version.V_1_4_0_Beta1` here. Same in `writeTo()` a few lines below.