Repository: dbt-labs/dbt-core

Issue:
BigQuery project reversion in 0.19.1
### Describe the bug

0.19.1 reverts the behavior around whether a `database` / `project` is required in `profiles.yml`.

For example:

```yaml
foo:
  target: user
  outputs:
    user:
      type: bigquery
      method: oauth
      dataset: max
      timeout_seconds: 3600
      threads: 24
      # project: x
```

...fails with 

```
Encountered an error:
Field "database" of type typing.Union[str, NoneType] is missing in dbt.adapters.bigquery.connections.BigQueryCredentials instance
```

Un-commenting `project` above works.

### Expected behavior

As above, which worked in 0.19.0

### System information
**Which database are you using dbt with?**
- [ ] postgres
- [ ] redshift
- [x] bigquery
- [ ] snowflake
- [ ] other (specify: ____________)


**The output of `dbt --version`:**
```
installed version: 0.19.1
   latest version: 0.19.1

Up to date!

Plugins:
  - bigquery: 0.19.1
  - snowflake: 0.19.1
  - redshift: 0.19.1
  - postgres: 0.19.1
```

**The operating system you're using:**
MacOS
**The output of `python --version`:**
3.8
### Additional context
Add any other context about the problem here.



Discussion:
@max-sixty You're right—I'm sorry I didn't catch this during the v0.19.1 testing process.

Let's figure out a fix for this, and a way to add an automated test for this—even if we can't fully integration-test it (since the integration tests run via service account)—to ensure that this doesn't happen again in a future version.
So this still works:
```yml
my-bigquery:
  outputs:
    oauth:
      type: bigquery
      method: oauth
      project: "{{ 'None' | as_native }}"
      dataset: dbt_jcohen
  target: oauth
```
The `project` field is indeed allowed to be `None`, and when it is, dbt will use the `gcloud`-configured default project (per the work in https://github.com/fishtown-analytics/dbt/pull/2908). The issue is when the `database`/`project` key is missing entirely, it raises this error:
```
Field "database" of type typing.Union[str, NoneType] is missing in dbt.adapters.bigquery.connections.BigQueryCredentials instance
```

@gshank @kwigley Any chance there's an immediate spark of recognition for either of you in this error message?

Resolving patch:
diff --git a/plugins/bigquery/dbt/adapters/bigquery/connections.py b/plugins/bigquery/dbt/adapters/bigquery/connections.py
--- a/plugins/bigquery/dbt/adapters/bigquery/connections.py
+++ b/plugins/bigquery/dbt/adapters/bigquery/connections.py
@@ -112,15 +112,17 @@ def _connection_keys(self):
         return ('method', 'database', 'schema', 'location', 'priority',
                 'timeout_seconds', 'maximum_bytes_billed')
 
-    def __post_init__(self):
+    @classmethod
+    def __pre_deserialize__(cls, d: Dict[Any, Any]) -> Dict[Any, Any]:
         # We need to inject the correct value of the database (aka project) at
         # this stage, ref
         # https://github.com/fishtown-analytics/dbt/pull/2908#discussion_r532927436.
 
         # `database` is an alias of `project` in BigQuery
-        if self.database is None:
+        if 'database' not in d:
             _, database = get_bigquery_defaults()
-            self.database = database
+            d['database'] = database
+        return d
 
 
 class BigQueryConnectionManager(BaseConnectionManager):


Test patch:
diff --git a/test/unit/test_bigquery_adapter.py b/test/unit/test_bigquery_adapter.py
--- a/test/unit/test_bigquery_adapter.py
+++ b/test/unit/test_bigquery_adapter.py
@@ -97,7 +97,7 @@ def setUp(self):
                     'priority': 'batch',
                     'maximum_bytes_billed': 0,
                 },
-                'oauth--no-project': {
+                'oauth-no-project': {
                     'type': 'bigquery',
                     'method': 'oauth',
                     'schema': 'dummy_schema',
@@ -144,6 +144,25 @@ def get_adapter(self, target):
 
 
 class TestBigQueryAdapterAcquire(BaseTestBigQueryAdapter):
+    @patch('dbt.adapters.bigquery.connections.get_bigquery_defaults', return_value=('credentials', 'project_id'))
+    @patch('dbt.adapters.bigquery.BigQueryConnectionManager.open', return_value=_bq_conn())
+    def test_acquire_connection_oauth_no_project_validations(self, mock_open_connection, mock_get_bigquery_defaults):
+        adapter = self.get_adapter('oauth-no-project')
+        mock_get_bigquery_defaults.assert_called_once()
+        try:
+            connection = adapter.acquire_connection('dummy')
+            self.assertEqual(connection.type, 'bigquery')
+
+        except dbt.exceptions.ValidationException as e:
+            self.fail('got ValidationException: {}'.format(str(e)))
+
+        except BaseException as e:
+            raise
+
+        mock_open_connection.assert_not_called()
+        connection.handle
+        mock_open_connection.assert_called_once()
+
     @patch('dbt.adapters.bigquery.BigQueryConnectionManager.open', return_value=_bq_conn())
     def test_acquire_connection_oauth_validations(self, mock_open_connection):
         adapter = self.get_adapter('oauth')