Repository: minio/minio (language: go)

Issue:
ILM restore inconsistencies 
Scenario:

1) Start two `minio` servers (hot and warm), create and configure some bucket (`tb2` in our case) for 0-day transition from hot to warm:

```
$ mc mb warm/tb2
$ mc mb hot/tb2
$ mc admin tier add s3 hot WARM --access-key=*** --secret-key=*** --bucket tb2 --endpoint <endpoint-to-warm-minio-server>
$ mc ilm add --transition-days 0 --storage-class WARM hot/tb2
```

2) Write some object to the bucket and try to restore it:

```
$ mc cp minio/README.md hot/tb2
$ mc ls hot/tb2
[2022-02-16 13:21:24 GMT]  19KiB STANDARD README.md
$ mc ilm restore hot/tb2/README.md
Sent restore requests to 1 object(s)...
1/1 object(s) successfully restored..
$ 
$ mc ls hot/tb2  # try to list the restored object
$ s3cmd --no-ssl ls s3://tb2
$ 
```

The object is not listed. It takes about 5 or more mins until the object appears in the listing.

Note: after it appears and restore cmd is fired again - it stays. But if object is put/updated and restored again after it - the same issue appears again.

Although the object cannot be listed, it still can be copied fine:

```
$ mc cp hot/tb2/README.md ./bb
...000/tb2/README.md:  19.39 KiB / 19.39 KiB  ▓▓▓▓▓▓▓  1.62 MiB/s 0s
$ mc ls hot/tb2
$ cmp bb minio/README.md 
```

UPD: the object can be seen with `--version` option:

```
$ mc ls --versions hot/tb2
[2022-02-16 20:34:15 GMT]  19KiB STANDARD null v1 PUT README.md
$ mc ls hot/tb2
$ 
```

Tested on commit 27d94c64e (`RELEASE.2022-02-16T00-35-27Z`).

_Originally posted by @andriytk in https://github.com/minio/minio/issues/14278#issuecomment-1041471653_


Discussion / hints:
Metadata when it's not listing:

```
$ xl-meta /Volumes/minio/data1/tb2/README.md/xl.meta 
{
  "Versions": [
    {
      "Header": {
        "Flags": 1,
        "ModTime": "2022-02-16T20:34:15.617383Z",
        "Signature": "6d2156dc",
        "Type": 2,
        "VersionID": "3886a34bbbae40f18ec9b1e60274a968"
      },
      "Idx": 0,
      "Metadata": {
        "DelObj": {
          "ID": "OIajS7uuQPGOybHmAnSpaA==",
          "MTime": 1645043655617383000,
          "MetaSys": {
            "x-minio-internal-free-version": "",
            "x-minio-internal-transition-tier": "V0FSTQ==",
            "x-minio-internal-transitioned-object": "NDU2YjdmZDMtOWZiMS00NjJmLThiODktMjNjMTAyZmM2ZWRhL3RiMi9lYy80NC9lYzQ0M2NhZC0wY2E4LTQ4MWUtOGY3NC1mYmM2YjQ2YTMwMGQ=",
            "x-minio-internal-transitioned-versionID": ""
          }
        },
        "Type": 2
      }
    },
    {
      "Header": {
        "Flags": 6,
        "ModTime": "2022-02-16T20:34:15.617383Z",
        "Signature": "ddb5009a",
        "Type": 1,
        "VersionID": "00000000000000000000000000000000"
      },
      "Idx": 1,
      "Metadata": {
        "Type": 1,
        "V2Obj": {
          "CSumAlgo": 1,
          "DDir": "571c+SxqS2Cp1sR5Tq7YPw==",
          "EcAlgo": 1,
          "EcBSize": 1048576,
          "EcDist": [
            1,
            2,
            3,
            4
          ],
          "EcIndex": 1,
          "EcM": 2,
          "EcN": 2,
          "ID": "AAAAAAAAAAAAAAAAAAAAAA==",
          "MTime": 1645043655617383000,
          "MetaSys": {
            "x-minio-internal-inline-data": "dHJ1ZQ==",
            "x-minio-internal-transition-status": "Y29tcGxldGU=",
            "x-minio-internal-transition-tier": "V0FSTQ==",
            "x-minio-internal-transitioned-object": "NDU2YjdmZDMtOWZiMS00NjJmLThiODktMjNjMTAyZmM2ZWRhL3RiMi9lYy80NC9lYzQ0M2NhZC0wY2E4LTQ4MWUtOGY3NC1mYmM2YjQ2YTMwMGQ=",
            "x-minio-internal-transitioned-versionID": ""
          },
          "MetaUsr": {
            "X-Amz-Restore-Expiry-Days": "1",
            "X-Amz-Restore-Request-Date": "Wed, 16 Feb 2022 20:35:11 GMT",
            "content-type": "text/markdown",
            "etag": "33898cf34cc08ef283923505d3f55938",
            "x-amz-restore": "ongoing-request=\"false\", expiry-date=\"Fri, 18 Feb 2022 00:00:00 GMT\""
          },
          "PartASizes": [
            19859
          ],
          "PartETags": null,
          "PartNums": [
            1
          ],
          "PartSizes": [
            19859
          ],
          "Size": 19859
        }
      }
    },
    {
      "Header": {
        "Flags": 1,
        "ModTime": "2022-02-16T19:53:56.56019Z",
        "Signature": "79cbb710",
        "Type": 2,
        "VersionID": "63b3dcd162d2446f85fc8f220a12cbd9"
      },
      "Idx": 2,
      "Metadata": {
        "DelObj": {
          "ID": "Y7Pc0WLSRG+F/I8iChLL2Q==",
          "MTime": 1645041236560190000,
          "MetaSys": {
            "x-minio-internal-free-version": "",
            "x-minio-internal-transition-tier": "V0FSTQ==",
            "x-minio-internal-transitioned-object": "NDU2YjdmZDMtOWZiMS00NjJmLThiODktMjNjMTAyZmM2ZWRhL3RiMi8xZC83MS8xZDcxOTUyNi01NmI1LTQyMGEtODdhMi1lZDE3MzFlOWIwMzI=",
            "x-minio-internal-transitioned-versionID": ""
          }
        },
        "Type": 2
      }
    }
  ]
}
```
Metadata when it's listing again:

```
$ xl-meta /Volumes/minio/data1/tb2/README.md/xl.meta 
{
  "Versions": [
    {
      "Header": {
        "Flags": 6,
        "ModTime": "2022-02-16T20:34:15.617383Z",
        "Signature": "ddb5009a",
        "Type": 1,
        "VersionID": "00000000000000000000000000000000"
      },
      "Idx": 0,
      "Metadata": {
        "Type": 1,
        "V2Obj": {
          "CSumAlgo": 1,
          "DDir": "571c+SxqS2Cp1sR5Tq7YPw==",
          "EcAlgo": 1,
          "EcBSize": 1048576,
          "EcDist": [
            1,
            2,
            3,
            4
          ],
          "EcIndex": 1,
          "EcM": 2,
          "EcN": 2,
          "ID": "AAAAAAAAAAAAAAAAAAAAAA==",
          "MTime": 1645043655617383000,
          "MetaSys": {
            "x-minio-internal-inline-data": "dHJ1ZQ==",
            "x-minio-internal-transition-status": "Y29tcGxldGU=",
            "x-minio-internal-transition-tier": "V0FSTQ==",
            "x-minio-internal-transitioned-object": "NDU2YjdmZDMtOWZiMS00NjJmLThiODktMjNjMTAyZmM2ZWRhL3RiMi9lYy80NC9lYzQ0M2NhZC0wY2E4LTQ4MWUtOGY3NC1mYmM2YjQ2YTMwMGQ=",
            "x-minio-internal-transitioned-versionID": ""
          },
          "MetaUsr": {
            "X-Amz-Restore-Expiry-Days": "1",
            "X-Amz-Restore-Request-Date": "Wed, 16 Feb 2022 20:35:11 GMT",
            "content-type": "text/markdown",
            "etag": "33898cf34cc08ef283923505d3f55938",
            "x-amz-restore": "ongoing-request=\"false\", expiry-date=\"Fri, 18 Feb 2022 00:00:00 GMT\""
          },
          "PartASizes": [
            19859
          ],
          "PartETags": null,
          "PartNums": [
            1
          ],
          "PartSizes": [
            19859
          ],
          "Size": 19859
        }
      }
    }
  ]
}
```
From the [code](https://github.com/minio/minio/blob/b88c57ba934e2142f3f38ca2c0d620d821e45450/cmd/xl-storage-format-v2.go#L1933) of `xlMetaBuf::ListVersions()` function, looks like the 1st version in the list is the latest one. So it looks like the restored object version is not placed into the 1st position in the list at `xl.meta`.

Fix patch:
diff --git a/cmd/erasure-metadata.go b/cmd/erasure-metadata.go
index 365f438920c13..85e07b75aae8a 100644
--- a/cmd/erasure-metadata.go
+++ b/cmd/erasure-metadata.go
@@ -477,6 +477,18 @@ func (fi *FileInfo) TierFreeVersion() bool {
 	return ok
 }
 
+// IsRestoreObjReq returns true if fi corresponds to a RestoreObject request.
+func (fi *FileInfo) IsRestoreObjReq() bool {
+	if restoreHdr, ok := fi.Metadata[xhttp.AmzRestore]; ok {
+		if restoreStatus, err := parseRestoreObjStatus(restoreHdr); err == nil {
+			if !restoreStatus.Ongoing() {
+				return true
+			}
+		}
+	}
+	return false
+}
+
 // VersionPurgeStatus returns overall version purge status for this object version across targets
 func (fi *FileInfo) VersionPurgeStatus() VersionPurgeStatusType {
 	return fi.ReplicationState.CompositeVersionPurgeStatus()
diff --git a/cmd/xl-storage.go b/cmd/xl-storage.go
index 5cf6e751745a7..204b24bb3f56f 100644
--- a/cmd/xl-storage.go
+++ b/cmd/xl-storage.go
@@ -2241,7 +2241,13 @@ func (s *xlStorage) RenameData(ctx context.Context, srcVolume, srcPath string, f
 		// suspended or disabled on this bucket. RenameData will replace
 		// the 'null' version. We add a free-version to track its tiered
 		// content for asynchronous deletion.
-		xlMeta.AddFreeVersion(fi)
+		if !fi.IsRestoreObjReq() {
+			// Note: Restore object request reuses PutObject/Multipart
+			// upload to copy back its data from the remote tier. This
+			// doesn't replace the existing version, so we don't need to add
+			// a free-version.
+			xlMeta.AddFreeVersion(fi)
+		}
 	}
 
 	// indicates if RenameData() is called by healing.


Test patch: