Repository: minio/minio (language: go)

Issue:
Single drive FS mode - Truncated Response without Continuation Token (RELEASE.2022-01-25T19-56-04Z)
Hello,

Sorry to reopen this issue  #14132 , but it seems that the problem persists with the latest version.
This issue was closed because you said it was resolved by issue #14143 .
But I updated the minio yesterday and we still have the same problem.

I left attached the trace in verbose mode as you requested (check the line 170229) and the profile.

minio --version
mini version RELEASE.2022-01-25T19-56-04Z
[trace.minio-2.txt.zip](https://github.com/minio/minio/files/7944188/trace.minio-2.txt.zip)
[profile.zip](https://github.com/minio/minio/files/7944190/profile.zip)




Discussion / hints:
I honestly cannot see how this can happen with the current code.

Example, `X-Amz-Request-Id: 16CDDEB55761E83A`
@klauspost, do you need any additional info from me?? 
We can reproduce this situation easily.
@mesobreira please share the code if you can. 
Hello @harshavardhana  and @klauspost , 

Please find attached the script we made to reproduce the situation.
It's a simple Python script (using boto3). We can execute it in your environment.
It seems that the problem occurs when we have a concurrency between ListObjectV2 and Delete operations.
When we launch a single instance of the python script the problem does not happen.
But when we have 2 or more instances, we receive truncated reponse without continuation token.
 
[s3-scripts.zip](https://github.com/minio/minio/files/7971440/s3-scripts.zip)


Ah, so when a queued response no longer exists.

Rather tricky situation. I will see if that can be solved without huge complications. Maybe just re-calling the function when that situation occurs could be done.

Fix patch:
diff --git a/cmd/fs-v1.go b/cmd/fs-v1.go
index 5c158183598b5..c26fa8e25d523 100644
--- a/cmd/fs-v1.go
+++ b/cmd/fs-v1.go
@@ -1260,9 +1260,23 @@ func (fs *FSObjects) ListObjectVersions(ctx context.Context, bucket, prefix, mar
 
 // ListObjects - list all objects at prefix upto maxKeys., optionally delimited by '/'. Maintains the list pool
 // state for future re-entrant list requests.
-func (fs *FSObjects) ListObjects(ctx context.Context, bucket, prefix, marker, delimiter string, maxKeys int) (loi ListObjectsInfo, e error) {
-	return listObjects(ctx, fs, bucket, prefix, marker, delimiter, maxKeys, fs.listPool,
-		fs.listDirFactory(), fs.isLeaf, fs.isLeafDir, fs.getObjectInfoNoFSLock, fs.getObjectInfoNoFSLock)
+func (fs *FSObjects) ListObjects(ctx context.Context, bucket, prefix, marker, delimiter string, maxKeys int) (loi ListObjectsInfo, err error) {
+	// listObjects may in rare cases not be able to find any valid results.
+	// Therefore, it cannot set a NextMarker.
+	// In that case we retry the operation, but we add a
+	// max limit, so we never end up in an infinite loop.
+	tries := 50
+	for {
+		loi, err = listObjects(ctx, fs, bucket, prefix, marker, delimiter, maxKeys, fs.listPool,
+			fs.listDirFactory(), fs.isLeaf, fs.isLeafDir, fs.getObjectInfoNoFSLock, fs.getObjectInfoNoFSLock)
+		if err != nil {
+			return loi, err
+		}
+		if !loi.IsTruncated || loi.NextMarker != "" || tries == 0 {
+			return loi, nil
+		}
+		tries--
+	}
 }
 
 // GetObjectTags - get object tags from an existing object


Test patch: