Repository: minio/minio (language: go)

Issue:
ChecksumAlgorithm.SHA256 PutObjectRequest support
**Is your feature request related to a problem? Please describe.**
Is it possible for minio to support `ChecksumAlgorithm` in the same way that S3 supports it? For the S3 proper scenario: when performing a `PutObjectRequest` against S3 proper and where ChecksumAlgorithm is set to SHA256 in the request, this results in S3 calculating a SHA256 on the S3 side and the SHA256 hash is returned in the response.

**Describe the solution you'd like**
I'd like for setting ChecksumAlgorithm to SHA256 in a `PutObjectRequest` to result in minio calculating a SHA256 hash for the put object and the minio side calculated hash be returned in the `PutObjectResponse`

## Current Behavior
Setting ChecksumAlgorithm to SHA256 in a `PutObjectRequest` with minio currently results in an `The provided 'x-amz-content-sha256' header does not match what was computed.` API exception. minio appears to want the caller to always pass a pre-calculated SHA256 hash when ChecksumAlgorithm is set to SHA256 in a `PutObjectRequest`.

**Describe alternatives you've considered**
If staying with minio I'm not sure that there is an alternative, I've not yet tested with localstack to see if that's an option where this works as expected.

**Additional context**
* Version used (`minio --version`): RELEASE.2023-04-13T03-08-07Z (commit-id=a42650c065fe22c6a6d3ce526d80c5354d4bceac)
* Server setup and configuration: minio is being used in a docker compose stack with a few other services
* Operating System and version (`uname -a`): running in docker in WSL2 (Ubuntu) on Windows 10

Many thanks! :)


Discussion / hints:
Sure. Could you provide a full HTTP dump of the roundtrip to AWS S3, with the request headers and response headers + body?

If you provide the same for multipart (all three calls) , I can also look into that.
Thank you, I'm struggling to capture a HTTP dump of a working AWS S3 request tbh. I've tried getting fiddler proxy inline to capture the request but it looks like the AWS SDK doesn't like the fiddler man in the middle cert. 

Do you have any advice how I might capture an AWS S3 request/response. I'm using the .NET AWS SDK if that's relevant.

As an aside I have tried a more recent version of minio (`RELEASE.2023-07-07T07-13-57Z (commit-id=e20aab25ec26ef3b1e44eac1f6a01f15f043ab42)`) because I realized that the version I was using was quite old. 

It still doesn't work with `RELEASE.2023-07-07T07-13-57Z (commit-id=e20aab25ec26ef3b1e44eac1f6a01f15f043ab42)` but the exception message has changed to: `The request signature we calculated does not match the signature you provided. Check your key and signing method.`
I can't really help you with the .NET AWS SDK. But it would give me a head start to see the actual headers sent and received to S3.

The latter message more indicates wrong credentials. Again seeing the actual request headers would help.
I managed to grab the following by switching `AWSConfigs.LoggingConfig.LogMetrics = true` and by adding a custom `AWSConfigs.LoggingConfig.LogMetricsCustomFormatter`

It's not the greatest to read but it looks like the `CanonicalRequest` shows the headers sent to S3. 

I'm not sure if this is enough for you but I hope that it helps.

```
{
  "properties": {
    "AsyncCall": "True",
    "CanonicalRequest": "PUT\n/content/blocking-policies2/blocking-policies2/general/1.0/source_file1.txt\n\ncontent-length:342\ncontent-type:text/plain\nhost:<REDACTED-BUCKET-NAME>.s3.eu-west-1.amazonaws.com\nuser-agent:aws-sdk-dotnet-coreclr/3.7.106.0 aws-sdk-dotnet-core/3.7.107.7 .NET_Core/6.0.20 OS/Linux_5.15.90.1-microsoft-standard-WSL2_#1_SMP_Fri_Jan_27_02:56:13_UTC_2023 ClientAsync\nx-amz-content-sha256:STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER\nx-amz-date:20230715T115140Z\nx-amz-decoded-content-length:12\nx-amz-sdk-checksum-algorithm:SHA256\nx-amz-trailer:x-amz-checksum-sha256\n\ncontent-length;content-type;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length;x-amz-sdk-checksum-algorithm;x-amz-trailer\nSTREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER",
    "StringToSign": "AWS4-HMAC-SHA256\n20230715T115140Z\n20230715/eu-west-1/s3/aws4_request\nf49c3daed69805fd25f0efb8c16cc40a846e2fefecab366889228d97b2c819e1",
    "ServiceName": "AmazonS3",
    "ServiceEndpoint": "https://<REDACTED-BUCKET-NAME>.s3.eu-west-1.amazonaws.com/",
    "MethodName": "PutObjectRequest",
    "AmzId2": "iPVJYEupxBgeM7aTqNUFj6VlaoXZl3EUwQFwo0l/3PKHJqAJv45zW/weHOweLv6jIonU1nTRMVQ=",
    "StatusCode": "OK",
    "BytesProcessed": "0",
    "AWSRequestID": "0PBWGCMY7P1TPYKV"
  },
  "timings": {
    "CredentialsRequestTime": 0.0014,
    "RequestSigningTime": 0.3039,
    "HttpRequestTime": 210.4437,
    "ResponseUnmarshallTime": 0.0267,
    "ResponseProcessingTime": 0.1369,
    "ClientExecuteTime": 211.1398
  },
  "counters": {}
}
```
I've added a little C# demo application here (for anyone following along in .NET stack): 

https://github.com/bebo-dot-dev/S3-Tests
> I'm not sure if this is enough for you but I hope that it helps.

A bit, but not much. It doesn't print the actual headers sent. In particular "x-amz-trailer". I think it *may* be actually adding the trailing checksum, but I can't tell. If you figure out a way to dump the real headers, that would help a lot.
Ah ok no problem. I did try switching on S3 server access logging yesterday but that didn't help either, it fills a bucket with S3 request traces but there's no useful data for a request/response in a logged item. 

 I'll keep trying, will hopefully get there. 
Hi @klauspost, using the little https://github.com/bebo-dot-dev/S3-Tests test project, I switched the test to http (rather than https) and forced the request through a local proxy to enable the raw S3 http request/response to be captured.

Sensitive parts in the authorization header have been redacted in the request below but hopefully this will be good enough to give you a complete picture of what gets sent and returned:

Request:
```
PUT http://test-bucket-unique-unique.s3.eu-west-1.amazonaws.com/S3.Tests.resources.test-file.txt HTTP/1.1
Expect: 100-continue
x-amz-sdk-checksum-algorithm: SHA256
User-Agent: aws-sdk-dotnet-coreclr/3.7.108.0 aws-sdk-dotnet-core/3.7.108.2 .NET_Core/6.0.20 OS/Linux_5.15.0-76-generic_#83-Ubuntu_SMP_Thu_Jun_15_19:16:32_UTC_2023 ClientAsync
amz-sdk-invocation-id: 31b1eb96-4a42-438e-ad72-057df319792c
amz-sdk-request: attempt=1; max=5
Host: test-bucket-unique-unique.s3.eu-west-1.amazonaws.com
X-Amz-Date: 20230729T143532Z
X-Amz-Trailer: x-amz-checksum-sha256
X-Amz-Decoded-Content-Length: 4
X-Amz-Content-SHA256: STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER
Authorization: AWS4-HMAC-SHA256 Credential=[redacted]/[redacted]/eu-west-1/s3/aws4_request, SignedHeaders=content-length;content-type;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length;x-amz-sdk-checksum-algorithm;x-amz-trailer, Signature=[redacted]
Content-Length: 334
Content-Type: text/plain

4;chunk-signature=4d77cb531cd1d5dcec950de4be7323de208bc32143812753f1f1e3caa818cf58
test
0;chunk-signature=2c36a072d0e02244a6975917d79355f752949d044aa3918cd39aa27eee0110ab
x-amz-checksum-sha256:n4bQgYhMfWWaL+qgxVrQFaO/TxsrC4Is0V1sFbDwCgg=
x-amz-trailer-signature:9344d95a10fa5b36103b11211d872e776b35ec96c943d88d62b6b08565771215
```

Response:
```
HTTP/1.1 200 OK
x-amz-id-2: b5i26ZOACNbUPJYZfYDLQgcifRu9Noh4nKDPMPuaN35wot9Nn5fibGDJnck6Oju3FTYM7D985R5G272RmiGMvg==
x-amz-request-id: PXEE6AMKV47B35TC
Date: Sat, 29 Jul 2023 14:35:35 GMT
x-amz-server-side-encryption: AES256
ETag: "098f6bcd4621d373cade4e832627b4f6"
x-amz-checksum-sha256: n4bQgYhMfWWaL+qgxVrQFaO/TxsrC4Is0V1sFbDwCgg=
Server: AmazonS3
Content-Length: 0
```

We can now see that an `x-amz-checksum-sha256` pre-calculated SHA-256 hash value is present and sent in the body of the request (this must be pre-calculated by the AWS SDK). Then I imagine what happens is that S3 calculates it's own SHA-256 hash for the received object data, performs a comparison against the received hash and it returns the hash when all is well.
@bebo-dot-dev OK. That is also what I (silently) suspected. It is using a trailing header, so it can calculate the SHA256 while it is uploading the data and send it after the object data. We should also support that.

You are saying you are getting an error when running this against the latest MinIO?

Would it be possible to post the roundtrip against MinIO? There must be a slight difference between the request sent and the implementation.

I will see if I can get your sample up and running to test directly.
@bebo-dot-dev I got your sample compiling, but I can't figure out how to modify it to use a custom URL (`127.0.0.1:9000` with custom credentials `minio`/`minio123`, without TLS and with path-style requests).

If it would be possible for you to create a `using var client = new AmazonS3Client(`... I could drop in, that would help a lot.
Hi @klauspost, 

> You are saying you are getting an error when running this against the latest MinIO?

Yes that is what I'm saying and depending on which version of MinIO is used, a different `AmazonS3Exception` error message is returned from MinIO. Regardless of MinIO version and error message returned, it never works as expected.

> Would it be possible to post the roundtrip against MinIO? There must be a slight difference between the request sent and the implementation + If it would be possible for you to create a using var client = new AmazonS3Client(... I could drop in, that would help a lot.

Sure, I've added a couple of MinIO specific test cases to https://github.com/bebo-dot-dev/S3-Tests/blob/main/S3.Tests/MinIOTests.cs, the first test uses the latest available version of MinIO at the time of writing and the second test uses an older version. Both tests assert the failure, they fail as expected and they receive different error messages.

As requested the tests are setup to use plain old HTTP (no TLS) and S3 legacy path style addressing to work against MinIO.
The tests lean on TestContainers with MinIO so it should be reasonably easy for anyone to run these tests so long as docker is available on the test runner box.

The raw request/response played against lastest MinIO is here: https://github.com/bebo-dot-dev/S3-Tests/tree/main#raw-minio-http-requestresponse-http-routed-through-a-local-proxy

Fix patch:
diff --git a/cmd/streaming-signature-v4.go b/cmd/streaming-signature-v4.go
index bbf662af2c80e..7f673ae97e093 100644
--- a/cmd/streaming-signature-v4.go
+++ b/cmd/streaming-signature-v4.go
@@ -443,6 +443,9 @@ func (cr *s3ChunkedReader) Read(buf []byte) (n int, err error) {
 
 // readTrailers will read all trailers and populate cr.trailers with actual values.
 func (cr *s3ChunkedReader) readTrailers() error {
+	if cr.debug {
+		fmt.Printf("pre trailer sig: %s\n", cr.seedSignature)
+	}
 	var valueBuffer bytes.Buffer
 	// Read value
 	for {
@@ -507,6 +510,16 @@ func (cr *s3ChunkedReader) readTrailers() error {
 		}
 		return errMalformedEncoding
 	}
+
+	// TODO: It seems like we may have to be prepared to rewrite and sort trailing headers:
+	// https://docs.aws.amazon.com/IAM/latest/UserGuide/create-signed-request.html
+
+	// Any value must end with a newline.
+	// Not all clients send that.
+	trailerRaw := valueBuffer.Bytes()
+	if len(trailerRaw) > 0 && trailerRaw[len(trailerRaw)-1] != '\n' {
+		valueBuffer.Write([]byte{'\n'})
+	}
 	sig = sig[len("x-amz-trailer-signature:"):]
 	sig = bytes.TrimSpace(sig)
 	cr.chunkSHA256Writer.Write(valueBuffer.Bytes())


Test patch: