Snapshot/Restore: switch to write once mode for snapshot metadata files

This commit removes creation of in-progress snapshot file and makes creation of the final snapshot file atomic.

Fixes #8696
Clarify InternalIndexShard callbacks

This commit changes internal callback to be clear
about when they are called and also provide the
exception that was potentially thrown as a callback argument.

Closes #5945