Repository: minio/mc

Issue #3067: mc ls fails on DigitalOcean Spaces with "Unable to list folder. Truncated response should have continuation token set"
Created: 2020-02-04T02:39:57Z
Expected behavior
mc ls lists a prefix containing many files and exits 0.
Actual behavior
Failure with the error: "Unable to list folder. Truncated response should have continuation token set"
Steps to reproduce the behavior
mc ls digitaloceanspacesbucket/username/prefix-with-lots-of-files
mc --version

(paste output of mc --version)
mc version RELEASE.2020-01-25T03-02-19Z

System information
darwin/amd64
example debug output:
mc: <DEBUG> GET /genevera/?location= HTTP/1.1
Host: nyc3.digitaloceanspaces.com
User-Agent: MinIO (darwin; amd64) minio-go/v6.0.45 mc/2020-01-25T03:02:19Z
Authorization: AWS4-HMAC-SHA256 Credential=**REDACTED**/20200204/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
X-Amz-Date: 20200204T022623Z
Accept-Encoding: gzip

mc: <DEBUG> HTTP/1.1 200 OK
Content-Length: 131
Content-Type: application/xml
Strict-Transport-Security: max-age=15552000; includeSubDomains; preload

mc: <DEBUG> Response Time:  1.078912981s

mc: <DEBUG> GET /genevera/?delimiter=%2F&encoding-type=url&fetch-owner=true&list-type=2&prefix=PREFIX HTTP/1.1
Host: nyc3.digitaloceanspaces.com
User-Agent: MinIO (darwin; amd64) minio-go/v6.0.45 mc/2020-01-25T03:02:19Z
Authorization: AWS4-HMAC-SHA256 Credential=**REDACTED**/20200204/nyc3/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20200204T022624Z
Accept-Encoding: gzip

mc: <DEBUG> HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Tue, 04 Feb 2020 02:26:24 GMT
Strict-Transport-Security: max-age=15552000; includeSubDomains; preload
X-Amz-Request-Id: tx000000000000018433970-005e38d650-33d0876-nyc3a

mc: <DEBUG> Response Time:  340.814825ms

mc: <DEBUG> GET /genevera/?delimiter=%2F&encoding-type=url&fetch-owner=true&list-type=2&prefix=PREFIX HTTP/1.1
Host: nyc3.digitaloceanspaces.com
User-Agent: MinIO (darwin; amd64) minio-go/v6.0.45 mc/2020-01-25T03:02:19Z
Authorization: AWS4-HMAC-SHA256 Credential=**REDACTED**/20200204/nyc3/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20200204T022624Z
Accept-Encoding: gzip

mc: <DEBUG> HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Tue, 04 Feb 2020 02:26:25 GMT
Strict-Transport-Security: max-age=15552000; includeSubDomains; preload
X-Amz-Request-Id: tx000000000000018f97f13-005e38d651-33d083a-nyc3a

mc: <DEBUG> Response Time:  569.465558ms

mc: <DEBUG> GET /genevera/?delimiter=%2F&encoding-type=url&fetch-owner=true&list-type=2&prefix=PREFIX%2F HTTP/1.1
Host: nyc3.digitaloceanspaces.com
User-Agent: MinIO (darwin; amd64) minio-go/v6.0.45 mc/2020-01-25T03:02:19Z
Authorization: AWS4-HMAC-SHA256 Credential=**REDACTED**/20200204/nyc3/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20200204T022625Z
Accept-Encoding: gzip

mc: <DEBUG> HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Tue, 04 Feb 2020 02:26:25 GMT
Strict-Transport-Security: max-age=15552000; includeSubDomains; preload
X-Amz-Request-Id: tx0000000000000187d7e3a-005e38d651-33c84fd-nyc3a

mc: <DEBUG> Response Time:  540.980968ms

mc: <ERROR> Unable to list folder. Truncated response should have continuation token set
 (1) ls.go:132 cmd.doList(..) Tags: [https://nyc3.digitaloceanspaces.com/genevera/PREFIX/]
 (0) client-s3.go:1906 cmd.(*s3Client).listInRoutine(..)
 Release-Tag:RELEASE.2020-01-25T03-02-19Z | Commit:b70a79e39bbd | Host:hostname.SOMEDOMA.IN | OS:darwin | Arch:amd64 | Lang:go1.13.6 | Mem:2.7 MB/72 MB | Heap:2.7 MB/66 MB

Duplicate issue #3093: error listing bucket 
Created: 2020-02-25T21:46:03Z
Expected behavior
mc command should list the objects in bucket
Actual behavior
getting mc:  Unable to list folder. Truncated response should have continuation token set error
Steps to reproduce the behavior
mc ls prd-rtp/rtp-registry/
mc:  Unable to list folder. Truncated response should have continuation token set
-> Using Ceph bucket
mc --version

(paste output of mc --version)

mc --version
mc version RELEASE.2020-02-25T18-10-03Z
System information
cat /etc/release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.6 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.6"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.6 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.6:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.6
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.6"
Red Hat Enterprise Linux Server release 7.6 (Maipo)
Red Hat Enterprise Linux Server release 7.6 (Maipo)
cpe:/o:redhat:enterprise_linux:7.6:ga:server

Closing comment (2020-02-26T01:16:10Z): Closing as duplicate of #3067