Repository: googleapis/google-api-python-client

Issue:
json.decoder.JSONDecodeError on csv (res) content
#### Environment details

  - OS type and version: macOS 10.15.7 (19H1419)
  - Python version: 3.8.5
  - pipenv version: 2020.8.13
  - pip version: 20.2.3
  - `google-api-python-client` version: 2.26.1

#### Steps to reproduce

  Trying to use `generateCSV` api from `adsense.accounts.reports.saved.generateCsv`, but it might be a more general issue with non json format content.
  
#### Code example

```python
...
saved_report_id = 'accounts/pub-xxx/reports/yyy_zzz'

def main(argv):
  # Authenticate and construct service.
  credentials = adsense_util.get_adsense_credentials()
  with discovery.build('adsense', 'v2', credentials = credentials) as service:
    try:
      # Let the user pick account if more than one.
      account_id = adsense_util.get_account_id(service)

      # Retrieve report.
      # if saved_report_id:
      result = service.accounts().reports().saved().generateCsv(name=saved_report_id, dateRange='LAST_30_DAYS') .execute()

      print(result)
...
```

#### Stack trace
```
Traceback (most recent call last):
  File "src/generate_report.py", line 88, in <module>
    main(sys.argv)
  File "src/generate_report.py", line 63, in main
    print(result.execute())
  File "[...]/lib/python3.8/site-packages/googleapiclient/_helpers.py", line 131, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "[...]/lib/python3.8/site-packages/googleapiclient/http.py", line 939, in execute
    return self.postproc(resp, content)
  File "[...]/lib/python3.8/site-packages/googleapiclient/model.py", line 218, in response
    return self.deserialize(content)
  File "[...]/lib/python3.8/site-packages/googleapiclient/model.py", line 281, in deserialize
    body = json.loads(content)
  File "[...]/.pyenv/versions/3.8.5/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "[...]/.pyenv/versions/3.8.5/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/[...]/.pyenv/versions/3.8.5/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
```

#### Possible cause

The code involved is in `googleapiclient/model.py` and the class is `JsonModel(BaseModel)` specifically this part of code (275-283):
```python
    def deserialize(self, content):
        try:
            content = content.decode("utf-8")
        except AttributeError:
            pass
        body = json.loads(content)
        if self._data_wrapper and isinstance(body, dict) and "data" in body:
            body = body["data"]
        return body
```

The problem appears when the content of response is not in json format (with generateCsv api the content is csv text) so it produces the JSONDecodeError.


#### Possible patch
One of the following could work:
- In case the request asks for csv generation, the response should be routed to a correct new callable postprocessor method that deals with csv format
- change the above lines of code in this way:
```python
    def deserialize(self, content):
        try:
            content = content.decode("utf-8")
        except AttributeError:
            pass
        try:
            body = json.loads(content)
        except json.decoder.JSONDecodeError:
            body = content
        if self._data_wrapper and isinstance(body, dict) and "data" in body:
            body = body["data"]
        return body
```




Discussion:
Hi @gparonitti ,

Thank you for reporting this bug and proposing a fix. PRs are welcome! (Please include a test as well to avoid future regressions.)
@parthea np.  
I was in doubt whether to open a pull request because I don't know all the details of the code flow, and I'm not sure if this patch will work other than as a mere work around for my use case. It seemed to me semantically wrong to return a type that is not in json(-ish) format from a class called `JsonModel`. Anyway, I'll do the pr and the test asap.

Resolving patch:
diff --git a/googleapiclient/model.py b/googleapiclient/model.py
--- a/googleapiclient/model.py
+++ b/googleapiclient/model.py
@@ -277,9 +277,13 @@ def deserialize(self, content):
             content = content.decode("utf-8")
         except AttributeError:
             pass
-        body = json.loads(content)
-        if self._data_wrapper and isinstance(body, dict) and "data" in body:
-            body = body["data"]
+        try:
+            body = json.loads(content)
+        except json.decoder.JSONDecodeError:
+            body = content
+        else:
+            if self._data_wrapper and "data" in body:
+                body = body["data"]
         return body
 
     @property


Test patch:
diff --git a/tests/test_json_model.py b/tests/test_json_model.py
--- a/tests/test_json_model.py
+++ b/tests/test_json_model.py
@@ -22,6 +22,7 @@
 
 __author__ = "jcgregorio@google.com (Joe Gregorio)"
 
+import io
 import httplib2
 import json
 import pkg_resources
@@ -31,11 +32,11 @@
 
 import googleapiclient.model
 
-
 from googleapiclient.errors import HttpError
 from googleapiclient.model import JsonModel
 
 _LIBRARY_VERSION = pkg_resources.get_distribution("google-api-python-client").version
+CSV_TEXT_MOCK = 'column1,column2,column3\nstring1,1.2,string2'
 
 
 class Model(unittest.TestCase):
@@ -290,6 +291,24 @@ def test_no_data_wrapper_deserialize(self):
         content = model.response(resp, content)
         self.assertEqual(content, {"data": "is good"})
 
+    def test_no_data_wrapper_deserialize_text_format(self):
+        model = JsonModel(data_wrapper=False)
+        resp = httplib2.Response({"status": "200"})
+        resp.reason = "OK"
+        content = CSV_TEXT_MOCK
+        content = model.response(resp, content)
+        self.assertEqual(content, CSV_TEXT_MOCK)
+
+    def test_no_data_wrapper_deserialize_raise_type_error(self):
+        buffer = io.StringIO()
+        buffer.write('String buffer')
+        model = JsonModel(data_wrapper=False)
+        resp = httplib2.Response({"status": "500"})
+        resp.reason = "The JSON object must be str, bytes or bytearray, not StringIO"
+        content = buffer
+        with self.assertRaises(TypeError):
+            model.response(resp, content)
+
     def test_data_wrapper_deserialize(self):
         model = JsonModel(data_wrapper=True)
         resp = httplib2.Response({"status": "200"})