Retrying in a loop doesn't guarantee success, but rather only increases the likelihood of success.
But you're right - this covers only the case where a single node uses multiple concurrently started caches (e.g. in Wildfly), where the cache stores of each cache share a parent directory.
The multi-VM issue isn't nearly as easy to reproduce - and can be mitigated by staggering VM startup.