Repository: googleapis/python-storage

Issue:
Storage: Capture relevant headers to blob properties during download
Residual from googleapis/google-cloud-python#9003.

@william-silversmith notes that even with `raw_download` enabled, he is unable to detect the `content_type` of a downloaded blob without performing an additional `reload` request, which is prohibitive for his usecase at scale.  E.g.:

```python
blob = bucket.blob( key )
binary = blob.download_as_string(raw_download=True)
if blob.content_encoding == 'gzip':
    return gunzip(binary)
elif blob.content_encoding == 'br':
    return brotli.decompress(binary)
else:
    return binary
```
Potentially even...

```python
if blob.content_type == 'application/json':
    return json.loads(binary.decode('utf8'))
```

Discussion:
william-silversmith: @tseaver would these unit tests be sufficient? I can get more involved if necessary. I had trouble invoking _do_download, it was interacting strangely with "MagicMock" and dying on closing a file stream.
tseaver: @william-silversmith Thanks, that new test was sufficient.
william-silversmith: I'm a little confused because 806-808 have to be evaluated for the crc32c and md5_hash tests to pass.
google-cla[bot]: We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors.  If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA ([login here](https://cla.developers.google.com/) to double check)?  If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment `@googlebot I fixed it.`. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ **Googlers: [Go here](https://goto.google.com/prinfo/https%3A%2F%2Fgithub.com%2Fgoogleapis%2Fpython-storage%2Fpull%2F204) for more info**.

<!-- need_author_cla -->
tseaver: @googlebot I consent
google-cla[bot]: We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors.  If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA ([login here](https://cla.developers.google.com/) to double check)?  If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment `@googlebot I fixed it.`. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ **Googlers: [Go here](https://goto.google.com/prinfo/https%3A%2F%2Fgithub.com%2Fgoogleapis%2Fpython-storage%2Fpull%2F204) for more info**.

<!-- need_author_cla -->
william-silversmith: @googlebot I fixed it.
google-cla[bot]: We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors.  If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA ([login here](https://cla.developers.google.com/) to double check)?  If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment `@googlebot I fixed it.`. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ **Googlers: [Go here](https://goto.google.com/prinfo/https%3A%2F%2Fgithub.com%2Fgoogleapis%2Fpython-storage%2Fpull%2F204) for more info**.

<!-- need_author_cla -->
william-silversmith: I could try force pushing a double quote update that removes you as an author if that would be okay with you?
tseaver: @william-silversmith Sure, WFM.

Patch:
diff --git a/google/cloud/storage/blob.py b/google/cloud/storage/blob.py
index efad9ae..07a1786 100644
--- a/google/cloud/storage/blob.py
+++ b/google/cloud/storage/blob.py
@@ -31,6 +31,7 @@ import hashlib
 from io import BytesIO
 import mimetypes
 import os
+import re
 import warnings
 import six
 
@@ -783,6 +784,34 @@ class Blob(_PropertyMixin):
         )
         return _add_query_parameters(base_url, name_value_pairs)
 
+    def _extract_headers_from_download(self, response):
+        """Extract headers from a non-chunked request's http object.
+
+        This avoids the need to make a second request for commonly used
+        headers.
+
+        :type response:
+            :class requests.models.Response
+        :param response: The server response from downloading a non-chunked file
+        """
+        self.content_encoding = response.headers.get("Content-Encoding", None)
+        self.content_type = response.headers.get("Content-Type", None)
+        self.cache_control = response.headers.get("Cache-Control", None)
+        self.storage_class = response.headers.get("X-Goog-Storage-Class", None)
+        self.content_language = response.headers.get("Content-Language", None)
+        #  'X-Goog-Hash': 'crc32c=4gcgLQ==,md5=CS9tHYTtyFntzj7B9nkkJQ==',
+        x_goog_hash = response.headers.get("X-Goog-Hash", "")
+
+        digests = {}
+        for encoded_digest in x_goog_hash.split(","):
+            match = re.match(r"(crc32c|md5)=([\w\d]+)==", encoded_digest)
+            if match:
+                method, digest = match.groups()
+                digests[method] = digest
+
+        self.crc32c = digests.get("crc32c", None)
+        self.md5_hash = digests.get("md5", None)
+
     def _do_download(
         self,
         transport,
@@ -840,8 +869,8 @@ class Blob(_PropertyMixin):
             download = klass(
                 download_url, stream=file_obj, headers=headers, start=start, end=end
             )
-            download.consume(transport, timeout=timeout)
-
+            response = download.consume(transport, timeout=timeout)
+            self._extract_headers_from_download(response)
         else:
 
             if raw_download:


Test patch:
diff --git a/tests/unit/test_blob.py b/tests/unit/test_blob.py
index 4635b05..54aeae6 100644
--- a/tests/unit/test_blob.py
+++ b/tests/unit/test_blob.py
@@ -1451,6 +1451,37 @@ class Test_Blob(unittest.TestCase):
         stream = blob._do_download.mock_calls[0].args[1]
         self.assertIsInstance(stream, io.BytesIO)
 
+    def test_download_as_string_w_response_headers(self):
+        blob_name = "blob-name"
+        client = mock.Mock(spec=["_http"])
+        bucket = _Bucket(client)
+        media_link = "http://example.com/media/"
+        properties = {"mediaLink": media_link}
+        blob = self._make_one(blob_name, bucket=bucket, properties=properties)
+
+        response = self._mock_requests_response(
+            http_client.OK,
+            headers={
+                "Content-Type": "application/json",
+                "Content-Language": "ko-kr",
+                "Cache-Control": "max-age=1337;public",
+                "Content-Encoding": "gzip",
+                "X-Goog-Storage-Class": "STANDARD",
+                "X-Goog-Hash": "crc32c=4gcgLQ==,md5=CS9tHYTtyFntzj7B9nkkJQ==",
+            },
+            # { "x": 5 } gzipped
+            content=b"\x1f\x8b\x08\x00\xcfo\x17_\x02\xff\xabVP\xaaP\xb2R0U\xa8\x05\x00\xa1\xcaQ\x93\n\x00\x00\x00",
+        )
+        blob._extract_headers_from_download(response)
+
+        self.assertEqual(blob.content_type, "application/json")
+        self.assertEqual(blob.content_language, "ko-kr")
+        self.assertEqual(blob.content_encoding, "gzip")
+        self.assertEqual(blob.cache_control, "max-age=1337;public")
+        self.assertEqual(blob.storage_class, "STANDARD")
+        self.assertEqual(blob.md5_hash, "CS9tHYTtyFntzj7B9nkkJQ")
+        self.assertEqual(blob.crc32c, "4gcgLQ")
+
     def test_download_as_string_w_generation_match(self):
         GENERATION_NUMBER = 6
         MEDIA_LINK = "http://example.com/media/"