Repository: minio/minio (language: go)

Issue:
500 Internal Server Error for "mc ls --zip --recursive"
In the article https://blog.min.io/small-file-archives / describes how to view the contents of zip archives. In one of the minio installations, we use this feature (Version: 2022-01-08T03:11:54 I, 4-node cluster). And everything works fine.

We recently deployed another installation (Version: 2022-01-08T03:11:54Z, 1-node). When trying to view the contents of the archive, minio writes an error:

`mcli admin trace -a -v l1`

```
API: ListObjectsV2(bucket=test1)
Time: 19:34:44 UTC 06/20/2022
DeploymentID: e9c3c0f2-859b-42f9-80ef-251303f58029
RequestID: 16FA6B34114EFC08
RemoteHost: 172.17.0.1
Host: 127.0.0.1:9000
UserAgent: MinIO (darwin; amd64) minio-go/v7.0.24 mcli/RELEASE.2022-04-16T21-11-21Z
Error: unknown version (*errors.errorString)
       4: cmd/api-errors.go:2103:cmd.toAPIErrorCode()
       3: cmd/api-errors.go:2128:cmd.toAPIError()
       2: cmd/bucket-listobjects-handlers.go:252:cmd.objectAPIHandlers.ListObjectsV2Handler()
       1: net/http/server.go:2047:http.HandlerFunc.ServeHTTP()

...
REPEATED 8 TIMES
...

API: ListObjectsV2(bucket=test1)
Time: 19:34:48 UTC 06/20/2022
DeploymentID: e9c3c0f2-859b-42f9-80ef-251303f58029
RequestID: 16FA6B3508AA8C4C
RemoteHost: 172.17.0.1
Host: 127.0.0.1:9000
UserAgent: MinIO (darwin; amd64) minio-go/v7.0.24 mcli/RELEASE.2022-04-16T21-11-21Z
Error: unknown version (*errors.errorString)
       4: cmd/api-errors.go:2103:cmd.toAPIErrorCode()
       3: cmd/api-errors.go:2128:cmd.toAPIError()
       2: cmd/bucket-listobjects-handlers.go:252:cmd.objectAPIHandlers.ListObjectsV2Handler()
       1: net/http/server.go:2047:http.HandlerFunc.ServeHTTP()
```


The client outputs an error
```
mcli: <ERROR> Unable to list folder. We encountered an internal error, please try again.: cause(unknown version)
```




## Expected Behavior
Returning the list of files in the archive. Moreover, if you fill in the archive again, then you can again list the files inside it. But only 1 time

## Current Behavior
Error, after the first output of the file list.


## Steps to Reproduce (for bugs)
```
$ docker pull minio/minio:RELEASE.2022-06-17T02-00-35Z
# OR
docker pull minio/minio:RELEASE.2022-01-08T03-11-54Z
$ mkdir -p minio-data
$ docker run -it --rm --name m1 -p 9000:9000 -p 9001:9001 -v $(pwd)/minio-data:/data -e MINIO_ROOT_USER=minio -e $ MINIO_ROOT_PASSWORD=miniominio minio/minio:RELEASE.2022-01-08T03-11-54Z server /data --console-address ":9001"
$ mcli alias set l1 http://127.0.0.1:9000 minio miniominio
$ mcli mb l1/test1
# create zip-file archive.zip with several files
$ FILE="err.zip"
$ mcli cp $FILE l1/test1/$FILE
$ mcli ls --zip -r l1/test1/$FILE
[2022-06-20 21:35:45 MSK]  26KiB STANDARD err.zip/err.txt
[2022-06-20 21:35:45 MSK]  16KiB STANDARD err.zip/err1.txt
$ mcli ls --zip -r l1/test1/$FILE
mcli: <ERROR> Unable to list folder. We encountered an internal error, please try again.: cause(unknown version)
```


## Context
Storing a list of files for 1 day (for archived data) not as a set of individual files, but as a zip archive. If necessary, a list of files is obtained and the user is provided with a link to a specific file from the archive

## Regression
We assumed that such an error occurs only in new versions, so we deployed exactly the same version as on an already running cluster. But it didn't help. The next difference is that on a 4-node instance, minio is running without a container, and on a 1-node instance in a container. We tried to run a 1-node without a container, but it also didn't work.
Perhaps the reason is the number of nodes?

## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used (`minio --version`):
  * 2022-01-08T03:11:54Z
  * 2022-06-17T02:00:35Z
* Server setup and configuration:
* Operating System and version (`uname -a`): 
  * Linux test-polygon 5.4.0-88-generic #99-Ubuntu SMP Thu Sep 23 17:29:00 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux (Docker version 20.10.8, build 3967b7d)
  * Darwin iMac-Serg.local 20.5.0 Darwin Kernel Version 20.5.0: Sat May  8 05:10:33 PDT 2021; root:xnu-7195.121.3~9/RELEASE_X86_64 x86_64 (Docker version 20.10.16, build aa7e414)



Discussion / hints:
> docker pull minio/minio:RELEASE.2022-01-08T03-11-54Z

are you not using latest release? @ssi444 
> are you not using latest release? @ssi444

@harshavardhana 
We have tried the following releases:
 - 2022-01-08T03:11:54Z
 - 2022-06-17T02:00:35Z
 
The behavior of minio is the same in both cases.
would you be able to share the zip file? @ssi444 
> would you be able to share the zip file? @ssi444

It can be any zip.  We created using regular macos and linux tools @harshavardhana 

Fix patch:
diff --git a/cmd/object-api-datatypes.go b/cmd/object-api-datatypes.go
index 6bf4bfd450ded..54a4fa73a14bd 100644
--- a/cmd/object-api-datatypes.go
+++ b/cmd/object-api-datatypes.go
@@ -18,6 +18,7 @@
 package cmd
 
 import (
+	"encoding/base64"
 	"io"
 	"math"
 	"time"
@@ -178,6 +179,26 @@ type ObjectInfo struct {
 	SuccessorModTime time.Time
 }
 
+// ArchiveInfo returns any saved zip archive meta information
+func (o ObjectInfo) ArchiveInfo() []byte {
+	if len(o.UserDefined) == 0 {
+		return nil
+	}
+	z, ok := o.UserDefined[archiveInfoMetadataKey]
+	if !ok {
+		return nil
+	}
+	if len(z) > 0 && z[0] >= 32 {
+		// FS/gateway mode does base64 encoding on roundtrip.
+		// zipindex has version as first byte, which is below any base64 value.
+		zipInfo, _ := base64.StdEncoding.DecodeString(z)
+		if len(zipInfo) != 0 {
+			return zipInfo
+		}
+	}
+	return []byte(z)
+}
+
 // Clone - Returns a cloned copy of current objectInfo
 func (o ObjectInfo) Clone() (cinfo ObjectInfo) {
 	cinfo = ObjectInfo{
diff --git a/cmd/s3-zip-handlers.go b/cmd/s3-zip-handlers.go
index c82656039e571..8b5dbfc9cad76 100644
--- a/cmd/s3-zip-handlers.go
+++ b/cmd/s3-zip-handlers.go
@@ -140,21 +140,10 @@ func (api objectAPIHandlers) getObjectInArchiveFileHandler(ctx context.Context,
 		return
 	}
 
-	var zipInfo []byte
-
-	if z, ok := zipObjInfo.UserDefined[archiveInfoMetadataKey]; ok {
-		if globalIsErasure {
-			zipInfo = []byte(z)
-		} else {
-			zipInfo, err = base64.StdEncoding.DecodeString(z)
-			logger.LogIf(ctx, err)
-			// Will attempt to re-read...
-		}
-	}
+	zipInfo := zipObjInfo.ArchiveInfo()
 	if len(zipInfo) == 0 {
 		zipInfo, err = updateObjectMetadataWithZipInfo(ctx, objectAPI, bucket, zipPath, opts)
 	}
-
 	if err != nil {
 		writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
 		return
@@ -247,15 +236,11 @@ func listObjectsV2InArchive(ctx context.Context, objectAPI ObjectLayer, bucket,
 		return ListObjectsV2Info{}, nil
 	}
 
-	var zipInfo []byte
-
-	if z, ok := zipObjInfo.UserDefined[archiveInfoMetadataKey]; ok {
-		zipInfo = []byte(z)
-	} else {
+	zipInfo := zipObjInfo.ArchiveInfo()
+	if len(zipInfo) == 0 {
 		// Always update the latest version
 		zipInfo, err = updateObjectMetadataWithZipInfo(ctx, objectAPI, bucket, zipPath, ObjectOptions{})
 	}
-
 	if err != nil {
 		return ListObjectsV2Info{}, err
 	}
@@ -332,11 +317,10 @@ func getFilesListFromZIPObject(ctx context.Context, objectAPI ObjectLayer, bucke
 			return nil, ObjectInfo{}, err
 		}
 		b, err := ioutil.ReadAll(gr)
+		gr.Close()
 		if err != nil {
-			gr.Close()
 			return nil, ObjectInfo{}, err
 		}
-		gr.Close()
 		if size > len(b) {
 			size = len(b)
 		}
@@ -439,11 +423,8 @@ func (api objectAPIHandlers) headObjectInArchiveFileHandler(ctx context.Context,
 		return
 	}
 
-	var zipInfo []byte
-
-	if z, ok := zipObjInfo.UserDefined[archiveInfoMetadataKey]; ok {
-		zipInfo = []byte(z)
-	} else {
+	zipInfo := zipObjInfo.ArchiveInfo()
+	if len(zipInfo) == 0 {
 		zipInfo, err = updateObjectMetadataWithZipInfo(ctx, objectAPI, bucket, zipPath, opts)
 	}
 	if err != nil {
@@ -498,20 +479,34 @@ func updateObjectMetadataWithZipInfo(ctx context.Context, objectAPI ObjectLayer,
 	}
 
 	srcInfo.UserDefined[archiveTypeMetadataKey] = archiveType
-	if globalIsErasure {
-		srcInfo.UserDefined[archiveInfoMetadataKey] = string(zipInfo)
+	var zipInfoStr string
+	if globalIsGateway {
+		zipInfoStr = base64.StdEncoding.EncodeToString(zipInfo)
 	} else {
-		srcInfo.UserDefined[archiveInfoMetadataKey] = base64.StdEncoding.EncodeToString(zipInfo)
+		zipInfoStr = string(zipInfo)
 	}
-	srcInfo.metadataOnly = true
 
-	// Always update the same version id & modtime
+	if globalIsGateway {
+		srcInfo.UserDefined[archiveInfoMetadataKey] = zipInfoStr
 
-	// Passing opts twice as source & destination options will update the metadata
-	// of the same object version to avoid creating a new version.
-	_, err = objectAPI.CopyObject(ctx, bucket, object, bucket, object, srcInfo, opts, opts)
-	if err != nil {
-		return nil, err
+		// Use CopyObject API only for Gateway mode.
+		if _, err = objectAPI.CopyObject(ctx, bucket, object, bucket, object, srcInfo, opts, opts); err != nil {
+			return nil, err
+		}
+	} else {
+		popts := ObjectOptions{
+			MTime:     srcInfo.ModTime,
+			VersionID: srcInfo.VersionID,
+			EvalMetadataFn: func(oi ObjectInfo) error {
+				oi.UserDefined[archiveTypeMetadataKey] = archiveType
+				oi.UserDefined[archiveInfoMetadataKey] = zipInfoStr
+				return nil
+			},
+		}
+		// For all other modes use in-place update to update metadata on a specific version.
+		if _, err = objectAPI.PutObjectMetadata(ctx, bucket, object, popts); err != nil {
+			return nil, err
+		}
 	}
 
 	return zipInfo, nil


Test patch: