aws/aws-cli — pull request #1026 by kyleknap (2014-11-24 22:52:47+00:00)
Title: Do not check md5 for downloading kms objects

Fixes https://github.com/aws/aws-cli/issues/1014

The issue was that we rely on ETAG to perform MD5 checksums. However, you cannot reliably do that if it is encrypted by KMS.

I did some testing on all of the s3 encryption testing to see how MD5's checkout with the ETAG returned here is the summary
```
# Note comparator.py is the object that I am getting for all three cases.
$ md5 comparator.py
MD5 (comparator.py) = 7d62dba263afdd3f901128435ed146b7

# KMS encrypted
# Same md5 as unencrypted

$ aws s3api get-object --bucket mybucketfoo --key kmsencrypted temp.txt
{
"AcceptRanges": "bytes",
"ContentType": "binary/octet-stream",
"LastModified": "Mon, 24 Nov 2014 20:27:51 GMT",
"ContentLength": 6194,
"ETag": "\"d12b33cb96e431763224615d904dc600\"",
"ServerSideEncryption": "aws:kms",
"SSEKMSKeyId": "arn:aws:kms:us-east-1:934212987125:key/c32756f3-bc40-4b64-951f-4785e2423b7d",
"Metadata": {}
}

# SSEC
# Different md5 as unencrypted

$ aws s3api get-object --bucket mybucketfoo --key ssecencrypted temp.txt --sse-customer-key fileb://sse_c.key --sse-customer-algorithm AES256
{
"AcceptRanges": "bytes",
"ContentType": "binary/octet-stream",
"LastModified": "Mon, 24 Nov 2014 20:33:42 GMT",
"ContentLength": 6194,
"SSECustomerAlgorithm": "AES256",
"ETag": "\"3eb52a5967bba8ba6ab7035f94e59b14\"",
"SSECustomerKeyMD5": "F13HSk8VjVCCWFhcFJzuSg==",
"Metadata": {}
}

# SSE
# Same md5 as unencrypted

{
"AcceptRanges": "bytes",
"ContentType": "application/x-python",
"LastModified": "Mon, 24 Nov 2014 20:39:05 GMT",
"ContentLength": 6194,
"ETag": "\"7d62dba263afdd3f901128435ed146b7\"",
"ServerSideEncryption": "AES256",
"Metadata": {}
}
```

Based on the results, the only encryption technique you can use the ETAG for is SSE. All other ones return ETAG's that you cannot use to compare to the MD5. Note that this object was not multipart uploaded/downloaded.

cc @jamesls @danielgtaylor