Repository: dbt-labs/dbt-core

Issue:
[Bug] dbt deps should keep retrying if registry response is None
### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

Running dbt deps in a nightly job failed after retrying downloading the packages. The Max retries is listed as 5 and was currently at 0.

### Expected Behavior

The download is retried until the max retries is reached or the package is downloaded successfully. 

### Steps To Reproduce

This cannot be reproduced on demand.  It is an intermittent failure case for `dbt deps`

### Relevant log output

```shell
Console:
2021-10-28 09:23:31.847397 (MainThread): Making package registry request: GET https://hub.getdbt.com/api/v1/dbt-labs/logging.json
2021-10-28 09:23:31.861303 (MainThread): Retrying external call. Attempt: 0 Max attempts: 5
2021-10-28 09:23:32.862660 (MainThread): Making package registry request: GET https://hub.getdbt.com/api/v1/dbt-labs/logging.json
2021-10-28 09:23:32.879183 (MainThread): Response from registry: GET https://hub.getdbt.com/api/v1/dbt-labs/logging.json 200
2021-10-28 09:23:32.881613 (MainThread): Flushing usage events
2021-10-28 09:23:32.905059 (MainThread): argument of type 'NoneType' is not iterable


Datadog: 
Traceback (most recent call last):
  File "/usr/src/app/sinter/clients/dbt.py", line 1164, in call
    dbt_main.handle(command + extra_args)
  File "/usr/local/lib/python3.8/dist-packages/dbt/main.py", line 159, in handle
    res, success = handle_and_check(args)
  File "/usr/local/lib/python3.8/dist-packages/dbt/main.py", line 205, in handle_and_check
    task, res = run_from_args(parsed)
  File "/usr/local/lib/python3.8/dist-packages/dbt/main.py", line 258, in run_from_args
    results = task.run()
  File "/usr/local/lib/python3.8/dist-packages/dbt/task/deps.py", line 53, in run
    final_deps = resolve_packages(packages, self.config)
  File "/usr/local/lib/python3.8/dist-packages/dbt/deps/resolver.py", line 137, in resolve_packages
    target = final[package].resolved().fetch_metadata(config, renderer)
  File "/usr/local/lib/python3.8/dist-packages/dbt/deps/registry.py", line 129, in resolved
    available = registry.get_available_versions(self.package)
  File "/usr/local/lib/python3.8/dist-packages/dbt/clients/registry.py", line 78, in get_available_versions
    response = package(name)
  File "/usr/local/lib/python3.8/dist-packages/dbt/clients/registry.py", line 55, in package
    if ('redirectnamespace' in response) or ('redirectname' in response):
TypeError: argument of type 'NoneType' is not iterable
```


### Environment

```markdown
- dbt: 0.21.0
```


### What database are you using dbt with?

snowflake

### Additional Context

_No response_


Discussion:
It does look like we get a response here:  
```
2021-10-28 09:23:32.879183 (MainThread): Response from registry: GET https://hub.getdbt.com/api/v1/dbt-labs/logging.json 200
```
but that it's potentially empty or malformed, leading to response = None.  Do we need to check for an empty response and retry in that case as well?
@crystalro0 Thanks for the report!

@barberscott Your read here seems right on. It's reasonable to expect that dbt should continue retrying until it returns a response that isn't `None`, not just any successful response.

I think the change for that should happen somewhere in the weeds of [`_get`](https://github.com/dbt-labs/dbt-core/blob/40ae6b6bc860a30fa383756b7cdb63709ce829a8/core/dbt/clients/registry.py#L26), [`_get_with_retries`](https://github.com/dbt-labs/dbt-core/blob/40ae6b6bc860a30fa383756b7cdb63709ce829a8/core/dbt/clients/registry.py#L21), or [`connection_exception_retry`](https://github.com/dbt-labs/dbt-core/blob/40ae6b6bc860a30fa383756b7cdb63709ce829a8/core/dbt/utils.py#L612). While that code is a bit thorny, it's fairly well self-contained. The fix could be a one-liner. I'm going to call this a `good first issue`, and I'd welcome a PR for it.
Hi! I'd be happy to work on this one unless @barberscott wants to create a PR himself.

Resolving patch:
diff --git a/core/dbt/clients/registry.py b/core/dbt/clients/registry.py
--- a/core/dbt/clients/registry.py
+++ b/core/dbt/clients/registry.py
@@ -30,6 +30,10 @@ def _get(path, registry_base_url=None):
     logger.debug('Response from registry: GET {} {}'.format(url,
                                                             resp.status_code))
     resp.raise_for_status()
+    if resp is None:
+        raise requests.exceptions.ContentDecodingError(
+            'Request error: The response is None', response=resp
+        )
     return resp.json()
 
 
diff --git a/core/dbt/utils.py b/core/dbt/utils.py
--- a/core/dbt/utils.py
+++ b/core/dbt/utils.py
@@ -615,7 +615,11 @@ def _connection_exception_retry(fn, max_attempts: int, attempt: int = 0):
     """
     try:
         return fn()
-    except (requests.exceptions.ConnectionError, requests.exceptions.Timeout) as exc:
+    except (
+        requests.exceptions.ConnectionError,
+        requests.exceptions.Timeout,
+        requests.exceptions.ContentDecodingError,
+    ) as exc:
         if attempt <= max_attempts - 1:
             logger.debug('Retrying external call. Attempt: ' +
                          f'{attempt} Max attempts: {max_attempts}')


Test patch:
diff --git a/test/unit/test_core_dbt_utils.py b/test/unit/test_core_dbt_utils.py
--- a/test/unit/test_core_dbt_utils.py
+++ b/test/unit/test_core_dbt_utils.py
@@ -22,6 +22,11 @@ def test_connection_exception_retry_success(self):
         connection_exception_retry(lambda: Counter._add_with_limited_exception(), 5)
         self.assertEqual(2, counter) # 2 = original attempt plus 1 retry
 
+    def test_connection_exception_retry_success_none_response(self):
+        Counter._reset()
+        connection_exception_retry(lambda: Counter._add_with_none_exception(), 5)
+        self.assertEqual(2, counter) # 2 = original attempt returned None, plus 1 retry
+
 
 counter:int = 0 
 class Counter():
@@ -37,6 +42,11 @@ def _add_with_limited_exception():
         counter+=1
         if counter < 2:
             raise requests.exceptions.ConnectionError
+    def _add_with_none_exception():
+        global counter
+        counter+=1
+        if counter < 2:
+            raise requests.exceptions.ContentDecodingError
     def _reset():
         global counter
         counter = 0