Repository: dbt-labs/dbt-core

Issue:
BigQuery KMS should not be used on VIEWS
### Describe the bug
If I add one kms key for the entire project that should be used for all models, like below. It will blow up, because you can't add KMS encryption to views. ( Unless I'm missing something here )

```yml
models:
  test:
    +kms_key_name: 'projects/<project>/locations/<location>/keyRings/key-ring/cryptoKeys/key'
```

### Steps To Reproduce
Add kms_key_name to the entire project or specific folder that contains views.

### Expected behavior
It should skip views, and only add the encryption key to the tables.

### System information
**Which database are you using dbt with?**
- [ ] postgres
- [ ] redshift
- [x] bigquery
- [ ] snowflake
- [ ] other (specify: ____________)


**The output of `dbt --version`:**
```
dbt --version
installed version: 0.20.0
   latest version: 0.20.0

Up to date!

Plugins:
  - bigquery: 0.20.0
  - snowflake: 0.20.0
  - redshift: 0.20.0
  - postgres: 0.20.0
```

**The operating system you're using:**
MAC

### Additional context
Should probably be in its own issue, but I would like to be able to add default encryption key to the entire dataset that DBT creates. Maybe this is possible, but I didn't find anything about it :)

Cheers!



Discussion:
Thanks for the report @AndreasTA-AW! I agree, `dbt-bigquery` should be smarter here about which configs can apply to which materialization types.

We have an adapter method called `get_table_options`:

https://github.com/dbt-labs/dbt/blob/159e79ee6ba86c852f4bf067dd0f1b6356135383/plugins/bigquery/dbt/adapters/bigquery/impl.py#L771-L805

That's called in a Jinja macro, `bigquery_table_options`, which templates out the appropriate string:

https://github.com/dbt-labs/dbt/blob/159e79ee6ba86c852f4bf067dd0f1b6356135383/plugins/bigquery/dbt/include/bigquery/macros/adapters.sql#L31-L40

Then that stringified version is included in the `create view as` statement:
https://github.com/dbt-labs/dbt/blob/159e79ee6ba86c852f4bf067dd0f1b6356135383/plugins/bigquery/dbt/include/bigquery/macros/adapters.sql#L61-L70

I can think of two possible resolutions here:
1. Add conditional logic within the `get_table_options` method that checks `config.get('materialization')` for properties like `kms_key_name` that won't work on views.
2. Create a new method, `get_view_options`, that serves the same purpose as `get_table_options` but excludes any configs that are unsupported on views. Then the `bigquery__create_view_as` macro would call `bigquery_view_options` --> `adapter.get_view_options`.

I'm leaning toward the second option. The first obfuscates differences, and it won't account for custom materializations. By contrast, mapping in the second (`create view as` --> `get_view_options`, `create table as` --> `get_table_options`) feels right. The only challenge will be implementing `get_table_options` + `get_view_options` in a way that doesn't duplicate a bunch of logic.

This feels like a straightforward and self-contained change, so I'm going to mark it a good first issue. Would you be interested in contributing the fix for it? :)
Thanks for the quick response! :)
I looked at the code before I submitted the PR and it looks like it worked before a major refactor of the code. Before, I think it only triggered for tables, but now when it’s in python, I think i triggers for both tables and views. I do agree with the second solution though, to keep them isolated.

I will check if I can put some time to submit a PR, I’m a consultant and a father and all that :P but I can try to find some time.

Also, wanna let u know that DBT is a great tool and it makes life a lot easier. Ofc it got some flaws, like everything, but it’s getting there!

cheers!


Resolving patch:
diff --git a/plugins/bigquery/dbt/adapters/bigquery/impl.py b/plugins/bigquery/dbt/adapters/bigquery/impl.py
--- a/plugins/bigquery/dbt/adapters/bigquery/impl.py
+++ b/plugins/bigquery/dbt/adapters/bigquery/impl.py
@@ -769,13 +769,10 @@ def _get_catalog_schemas(self, manifest: Manifest) -> SchemaSearchMap:
         return result
 
     @available.parse(lambda *a, **k: {})
-    def get_table_options(
-        self, config: Dict[str, Any], node: Dict[str, Any], temporary: bool
+    def get_common_options(
+        self, config: Dict[str, Any], node: Dict[str, Any], temporary: bool = False
     ) -> Dict[str, Any]:
         opts = {}
-        if temporary:
-            expiration = 'TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL 12 hour)'
-            opts['expiration_timestamp'] = expiration
 
         if (config.get('hours_to_expiration') is not None) and (not temporary):
             expiration = (
@@ -787,13 +784,25 @@ def get_table_options(
             description = sql_escape(node['description'])
             opts['description'] = '"""{}"""'.format(description)
 
-        if config.get('kms_key_name') is not None:
-            opts['kms_key_name'] = "'{}'".format(config.get('kms_key_name'))
-
         if config.get('labels'):
             labels = config.get('labels', {})
             opts['labels'] = list(labels.items())
 
+        return opts
+
+    @available.parse(lambda *a, **k: {})
+    def get_table_options(
+        self, config: Dict[str, Any], node: Dict[str, Any], temporary: bool
+    ) -> Dict[str, Any]:
+        opts = self.get_common_options(config, node, temporary)
+
+        if temporary:
+            expiration = 'TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL 12 hour)'
+            opts['expiration_timestamp'] = expiration
+
+        if config.get('kms_key_name') is not None:
+            opts['kms_key_name'] = "'{}'".format(config.get('kms_key_name'))
+
         if config.get('require_partition_filter'):
             opts['require_partition_filter'] = config.get(
                 'require_partition_filter')
@@ -804,6 +813,13 @@ def get_table_options(
 
         return opts
 
+    @available.parse(lambda *a, **k: {})
+    def get_view_options(
+        self, config: Dict[str, Any], node: Dict[str, Any]
+    ) -> Dict[str, Any]:
+        opts = self.get_common_options(config, node)
+        return opts
+
     @available.parse_none
     def grant_access_to(self, entity, entity_type, role, grant_target_dict):
         """


Test patch:
diff --git a/test/unit/test_bigquery_adapter.py b/test/unit/test_bigquery_adapter.py
--- a/test/unit/test_bigquery_adapter.py
+++ b/test/unit/test_bigquery_adapter.py
@@ -868,6 +868,31 @@ def test_hours_to_expiration_temporary(self):
         actual = adapter.get_table_options(mock_config, node={}, temporary=True)
         self.assertEqual(expected, actual)
 
+    def test_table_kms_key_name(self):
+        adapter = self.get_adapter('oauth')
+        mock_config = create_autospec(
+            RuntimeConfigObject)
+        config={'kms_key_name': 'some_key'}
+        mock_config.get.side_effect = lambda name: config.get(name)
+
+        expected = {
+            'kms_key_name': "'some_key'"
+        }
+        actual = adapter.get_table_options(mock_config, node={}, temporary=False)
+        self.assertEqual(expected, actual)
+
+        
+    def test_view_kms_key_name(self):
+        adapter = self.get_adapter('oauth')
+        mock_config = create_autospec(
+            RuntimeConfigObject)
+        config={'kms_key_name': 'some_key'}
+        mock_config.get.side_effect = lambda name: config.get(name)
+
+        expected = {}
+        actual = adapter.get_view_options(mock_config, node={})
+        self.assertEqual(expected, actual)
+
 
 
 class TestBigQueryFilterCatalog(unittest.TestCase):