Gateway: Prefer recovering the state file that uses the latest format.

Currently MetaDataStateFormat loads the first available state file that has
the latest version. In case several files are available and some of them use
the new format while other ones use the legacy format, it should also prefer
the new format. This is typically useful when we upgrade the metadata when
recovering from the gateway: we might write the upgraded state with the new
format while the previous state used the legacy format, so we end up with
two files having the same version but using different formats.
The reason why it is updated no matter what the version is is to detect pathological cases, eg. when you have two state files for version 1 and 2, yet 1 has been written with the new format and 2 with the legacy format (which should not happen), I added a comment about it below and a test case.

Regarding the file list, I think it can be useful to have the full list of files that have been considered in the exception message?