Repository: dbt-labs/dbt-core

Issue:
dbt docs generate errors when there's a "null" schema
## Issue

### Issue description
Fivetran inadvertently generated a null schema within our warehouse for google analytics data - this was subsequently fixed but due to laziness, I was yet to remove it.

When running dbt docs generate however this below error was produced.

### Results
What happened? What did you expect to happen?

### System information
The output of `dbt --version`:
```
    from svv_table_info
DEBUG:dbt:SQL status: SELECT in 2.90 seconds
DEBUG:dbt:On get_catalog_data: ROLLBACK
DEBUG:dbt:Flushing usage events
Encountered an error:
INFO:dbt:Encountered an error:
'NoneType' object has no attribute 'lower'
INFO:dbt:'NoneType' object has no attribute 'lower'
DEBUG:dbt:Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/dbt/main.py", line 72, in main
    results, succeeded = handle_and_check(args)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/dbt/main.py", line 117, in handle_and_check
    task, res = run_from_args(parsed)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/dbt/main.py", line 172, in run_from_args
    results = run_from_task(task, proj, parsed)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/dbt/main.py", line 180, in run_from_task
    result = task.run()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/dbt/task/generate.py", line 217, in run
    results = adapter.get_catalog(profile, self.project.cfg, manifest)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/dbt/adapters/default/impl.py", line 838, in get_catalog
    results = table.where(lambda r: r['table_schema'].lower() in schemas)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/agate/table/where.py", line 25, in where
    if test(row):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/dbt/adapters/default/impl.py", line 838, in <lambda>
    results = table.where(lambda r: r['table_schema'].lower() in schemas)
AttributeError: 'NoneType' object has no attribute 'lower'
```

The operating system you're running on:

The python version you're using (probably the output of `python --version`)

### Steps to reproduce
In as much detail as possible, please provide steps to reproduce the issue. Sample data that triggers the issue, example models, etc are all very helpful here.





## Feature

### Feature description
Making dbt docs generate more robust

### Who will this benefit?
people who are strange enough to create null schemas or created for them in error by third parties.



Discussion:

Resolving patch:
diff --git a/dbt/adapters/default/impl.py b/dbt/adapters/default/impl.py
--- a/dbt/adapters/default/impl.py
+++ b/dbt/adapters/default/impl.py
@@ -24,6 +24,24 @@
 connections_available = []
 
 
+def _filter_schemas(manifest):
+    """Return a function that takes a row and decides if the row should be
+    included in the catalog output.
+    """
+    schemas = frozenset({
+        node.schema.lower()
+        for node in manifest.nodes.values()
+    })
+
+    def test(row):
+        # the schema may be present but None
+        table_schema = row.get('table_schema')
+        if table_schema is None:
+            return False
+        return table_schema.lower() in schemas
+    return test
+
+
 class DefaultAdapter(object):
     DEFAULT_QUOTE = True
 
@@ -830,10 +848,5 @@ def get_catalog(cls, profile, project_cfg, manifest):
         finally:
             cls.release_connection(profile, GET_CATALOG_OPERATION_NAME)
 
-        schemas = list({
-            node.schema.lower()
-            for node in manifest.nodes.values()
-        })
-
-        results = table.where(lambda r: r['table_schema'].lower() in schemas)
+        results = table.where(_filter_schemas(manifest))
         return results


Test patch:
diff --git a/test/unit/test_postgres_adapter.py b/test/unit/test_postgres_adapter.py
--- a/test/unit/test_postgres_adapter.py
+++ b/test/unit/test_postgres_adapter.py
@@ -7,6 +7,7 @@
 from dbt.adapters.postgres import PostgresAdapter
 from dbt.exceptions import ValidationException
 from dbt.logger import GLOBAL_LOGGER as logger  # noqa
+import agate
 
 
 class TestPostgresAdapter(unittest.TestCase):
@@ -79,3 +80,31 @@ def test_set_zero_keepalive(self, psycopg2):
             port=5432,
             connect_timeout=10)
 
+    @mock.patch.object(PostgresAdapter, 'run_operation')
+    def test_get_catalog_various_schemas(self, mock_run):
+        column_names = ['table_schema', 'table_name']
+        rows = [
+            ('foo', 'bar'),
+            ('FOO', 'baz'),
+            (None, 'bar'),
+            ('quux', 'bar'),
+            ('skip', 'bar')
+        ]
+        mock_run.return_value = agate.Table(rows=rows,
+                                            column_names=column_names)
+
+        # we should accept the lowercase matching 'foo's only.
+        mock_nodes = [
+            mock.MagicMock(spec_set=['schema'], schema='foo')
+            for k in range(2)
+        ]
+        mock_nodes.append(mock.MagicMock(spec_set=['schema'], schema='quux'))
+        nodes = {str(idx): n for idx, n in enumerate(mock_nodes)}
+        # give manifest the dict it wants
+        mock_manifest = mock.MagicMock(spec_set=['nodes'], nodes=nodes)
+
+        catalog = PostgresAdapter.get_catalog({}, {}, mock_manifest)
+        self.assertEqual(
+            set(map(tuple, catalog)),
+            {('foo', 'bar'), ('FOO', 'baz'), ('quux', 'bar')}
+        )