internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: dbt-labs/dbt-core

Issue:
BigQuery project alias is ignored in config
### Describe the bug
A user reported that a config like:
```
{{ config(project='myproject') }}

...
```

has regressed in dbt v0.17.0. While this config worked in a BQ project in dbt v0.16.1, they reported that they needed to change `project` to `database` to apply the configuration in dbt v0.17.0.

This issue needs to be reproduced - there may be other factors that impact the incidence of this regression.

### Steps To Reproduce
```
-- models/my_model.sql

{{ config(project='custom_project') }}

select 1 as id
```

```
dbt run
```

Confirm that the model was _not_ build into the custom project override

### Expected behavior
The model should be built into the project defined in the `project` config. Database-specific aliases should applied to config names.

### System information
**Which database are you using dbt with?**
- [x] bigquery


**The output of `dbt --version`:**
```
0.17.0
```

**The operating system you're using:** Windows
**The output of `python --version`:** Unknown


Discussion:
I can reproduce this in a test.

Resolving patch:
diff --git a/core/dbt/context/context_config.py b/core/dbt/context/context_config.py
--- a/core/dbt/context/context_config.py
+++ b/core/dbt/context/context_config.py
@@ -131,8 +131,9 @@ def active_project_configs(
     def _update_from_config(
         self, result: T, partial: Dict[str, Any], validate: bool = False
     ) -> T:
+        translated = self.active_project.credentials.translate_aliases(partial)
         return result.update_from(
-            partial,
+            translated,
             self.active_project.credentials.type,
             validate=validate
         )


Test patch:
diff --git a/test/integration/040_override_database_test/models/view_2.sql b/test/integration/040_override_database_test/models/view_2.sql
--- a/test/integration/040_override_database_test/models/view_2.sql
+++ b/test/integration/040_override_database_test/models/view_2.sql
@@ -1,4 +1,6 @@
-{{
-    config(database=var('alternate_db'))
-}}
+{%- if target.type == 'bigquery' -%}
+  {{ config(project=var('alternate_db')) }}
+{%- else -%}
+  {{ config(database=var('alternate_db')) }}
+{%- endif -%}
 select * from {{ ref('seed') }}
diff --git a/test/integration/base.py b/test/integration/base.py
--- a/test/integration/base.py
+++ b/test/integration/base.py
@@ -1021,6 +1021,8 @@ def assertManyRelationsEqual(self, relations, default_schema=None, default_datab
         first_columns = None
         for relation in specs:
             key = (relation.database, relation.schema, relation.identifier)
+            # get a good error here instead of a hard-to-diagnose KeyError
+            self.assertIn(key, column_specs, f'No columns found for {key}')
             columns = column_specs[key]
             if first_columns is None:
                 first_columns = columns
Repository: dbt-labs/dbt-core

Issue:
Enable snowflake-connector-python debug logging with environment variables
### Describe the feature
dbt should support an environment variable to enable debug logging for the `snowflake-connector-python` library. This will help dbt users track down issues that arise like:
```
Apache Arrow format is not supported
```

Snowflake support requires these logs to diagnose and resolve issues that occur using their platform.

### Describe alternatives you've considered
We could enable these logs by default. `snowflake-connector-python` debug logs are pretty verbose though, so it might be wise to restrict this logging behind an env var.

### Additional context
Relevant to intermittent errors like `The result set in Apache Arrow format is not supported for the platform.`

### Who will this benefit?
Snowflake users; snowflake support



Discussion:

Resolving patch:
diff --git a/core/dbt/logger.py b/core/dbt/logger.py
--- a/core/dbt/logger.py
+++ b/core/dbt/logger.py
@@ -597,15 +597,28 @@ def emit(self, record: logbook.LogRecord):
         self.records.append(as_dict)
 
 
-# we still need to use logging to suppress these or pytest captures them
-logging.getLogger('botocore').setLevel(logging.ERROR)
-logging.getLogger('requests').setLevel(logging.ERROR)
-logging.getLogger('urllib3').setLevel(logging.ERROR)
-logging.getLogger('google').setLevel(logging.ERROR)
-logging.getLogger('snowflake.connector').setLevel(logging.ERROR)
+def _env_log_level(var_name: str) -> int:
+    # convert debugging environment variable name to a log level
+    if dbt.flags.env_set_truthy(var_name):
+        return logging.DEBUG
+    else:
+        return logging.ERROR
+
+
+LOG_LEVEL_GOOGLE = _env_log_level('DBT_GOOGLE_DEBUG_LOGGING')
+LOG_LEVEL_SNOWFLAKE = _env_log_level('DBT_SNOWFLAKE_CONNECTOR_DEBUG_LOGGING')
+LOG_LEVEL_BOTOCORE = _env_log_level('DBT_BOTOCORE_DEBUG_LOGGING')
+LOG_LEVEL_HTTP = _env_log_level('DBT_HTTP_DEBUG_LOGGING')
+LOG_LEVEL_WERKZEUG = _env_log_level('DBT_WERKZEUG_DEBUG_LOGGING')
+
+logging.getLogger('botocore').setLevel(LOG_LEVEL_BOTOCORE)
+logging.getLogger('requests').setLevel(LOG_LEVEL_HTTP)
+logging.getLogger('urllib3').setLevel(LOG_LEVEL_HTTP)
+logging.getLogger('google').setLevel(LOG_LEVEL_GOOGLE)
+logging.getLogger('snowflake.connector').setLevel(LOG_LEVEL_SNOWFLAKE)
+
 logging.getLogger('parsedatetime').setLevel(logging.ERROR)
-# want to see werkzeug logs about errors
-logging.getLogger('werkzeug').setLevel(logging.ERROR)
+logging.getLogger('werkzeug').setLevel(LOG_LEVEL_WERKZEUG)
 
 
 def list_handler(


Test patch:
Repository: dbt-labs/dbt-core

Issue:
dbt-snowflake should explicitly require snowflake-connector-python[keyring]
### Describe the feature
- Add the optional `keyring` dependency of `snowflake-connector-python` to `dbt-snowflake` installation requirements
- Help users avoid running into the issue outlined in #2689

### Who will this benefit?
- Users of Snowflake + external browser auth


Discussion:

Resolving patch:
diff --git a/plugins/snowflake/setup.py b/plugins/snowflake/setup.py
--- a/plugins/snowflake/setup.py
+++ b/plugins/snowflake/setup.py
@@ -47,7 +47,7 @@
     },
     install_requires=[
         'dbt-core=={}'.format(package_version),
-        'snowflake-connector-python==2.2.10',
+        'snowflake-connector-python[secure-local-storage]==2.2.10',
         'azure-common<2.0.0',
         'azure-storage-blob>=12.0.0,<13.0.0',
         'urllib3>=1.20,<1.26.0',


Test patch:
Repository: dbt-labs/dbt-core

Issue:
At the end of a dbt run, write the manifest
### Describe the feature
When a dbt run completes, when writing the run_results.json file, also write the manifest.json to disk. TBD if this should be done a second time or if we should skip the current write that occurs after parsing finishes.

This will let us remove the `node` field from `run_results.json`.

Split off from #2671 


Discussion:

Resolving patch:
diff --git a/core/dbt/task/runnable.py b/core/dbt/task/runnable.py
--- a/core/dbt/task/runnable.py
+++ b/core/dbt/task/runnable.py
@@ -425,6 +425,7 @@ def run(self):
         result = self.execute_with_hooks(selected_uids)
 
         if flags.WRITE_JSON:
+            self.write_manifest()
             self.write_result(result)
 
         self.task_end_messages(result.results)


Test patch:
Repository: dbt-labs/dbt-core

Issue:
Update debug task to make the debug query an adapter method
The existing code is here:
https://github.com/fishtown-analytics/dbt/blob/fa8a4f2020655c60bdb66270616f5654e168a729/core/dbt/task/debug.py#L337

But that won't work on some third party adapters. One example is Oracle, which requires a `from` clause in any `select` statement.

Instead, this should be an adapter method on the base adapter, named `debug_query()` that takes no arguments. Then adapter plugins will be able to override the method with an appropriate debug method. For example, on Oracle, it might be `select 1 from dual`.


Discussion:
Hey @PedroMiguelFigueiredo, could you say a bit more here?
@jtcohen6 it looks like on oracle, selects require a `FROM` clause and their solution to the obvious problems that causes is a special identifier named `dual` that's globally accessible.

If that is the case, we probably should have an `adapter.debug_query()` method that defaults to `adapter.execute("select 1 as id")`. Then the author of the oracle plugin (or other databases with similar restrictions) can add their oracle-only special case behavior.
Ahh that makes a lot of sense, and sounds good as an approach
Hi all !

Yes, we had the same idea here in the dbt-oracle repo : 
[Error on dbt debug](https://github.com/techindicium/dbt-oracle/issues/1)

This would be great, and maybe this could be circonveined just in SQLAdapter and OracleAdapter.

By chance, the dremio adapter will not need it ;-), as dremio is [calcite](https://calcite.apache.org/) based : 

[dbt adapter for dremio](https://github.com/fabrice-etanchaud/dbt-dremio)

dbt's community is simply great !

Resolving patch:
diff --git a/core/dbt/adapters/base/impl.py b/core/dbt/adapters/base/impl.py
--- a/core/dbt/adapters/base/impl.py
+++ b/core/dbt/adapters/base/impl.py
@@ -180,6 +180,9 @@ def clear_transaction(self) -> None:
     def commit_if_has_connection(self) -> None:
         self.connections.commit_if_has_connection()
 
+    def debug_query(self) -> None:
+        self.execute('select 1 as id')
+
     def nice_connection_name(self) -> str:
         conn = self.connections.get_if_exists()
         if conn is None or conn.name is None:
diff --git a/core/dbt/task/debug.py b/core/dbt/task/debug.py
--- a/core/dbt/task/debug.py
+++ b/core/dbt/task/debug.py
@@ -334,7 +334,7 @@ def attempt_connection(profile):
         adapter = get_adapter(profile)
         try:
             with adapter.connection_named('debug'):
-                adapter.execute('select 1 as id')
+                adapter.debug_query()
         except Exception as exc:
             return COULD_NOT_CONNECT_MESSAGE.format(
                 err=str(exc),


Test patch:
Repository: dbt-labs/dbt-core

Issue:
set colorama upper bound to <0.4.4
colorama v0.4.4 (released in the last 24 hours) is missing an sdist, which trips up the homebrew packaging step of our [dbt release flow](https://github.com/fishtown-analytics/dbt-release/runs/1249693542). Let's set the [upper bound](https://github.com/fishtown-analytics/dbt/blob/dev/kiyoshi-kuromiya/core/setup.py#L67) to <0.4.4 instead of <0.5 for now.


Discussion:

Resolving patch:
diff --git a/core/setup.py b/core/setup.py
--- a/core/setup.py
+++ b/core/setup.py
@@ -64,7 +64,7 @@ def read(fname):
         'sqlparse>=0.2.3,<0.4',
         'networkx>=2.3,<3',
         'minimal-snowplow-tracker==0.0.2',
-        'colorama>=0.3.9,<0.5',
+        'colorama>=0.3.9,<0.4.4',
         'agate>=1.6,<2',
         'isodate>=0.6,<0.7',
         'json-rpc>=1.12,<2',


Test patch:
Repository: dbt-labs/dbt-core

Issue:
dbt-docs changes for v0.19.0-b1
Reflect changes from:
- https://github.com/fishtown-analytics/dbt-docs/pull/138
- https://github.com/fishtown-analytics/dbt-docs/pull/139
- https://github.com/fishtown-analytics/dbt-docs/issues/148


Discussion:

Resolving patch:
diff --git a/core/dbt/contracts/graph/manifest.py b/core/dbt/contracts/graph/manifest.py
--- a/core/dbt/contracts/graph/manifest.py
+++ b/core/dbt/contracts/graph/manifest.py
@@ -1,7 +1,6 @@
 import abc
 import enum
 from dataclasses import dataclass, field
-from datetime import datetime
 from itertools import chain, islice
 from multiprocessing.synchronize import Lock
 from typing import (
diff --git a/core/dbt/parser/manifest.py b/core/dbt/parser/manifest.py
--- a/core/dbt/parser/manifest.py
+++ b/core/dbt/parser/manifest.py
@@ -1,6 +1,5 @@
 import os
 import pickle
-from datetime import datetime
 from typing import (
     Dict, Optional, Mapping, Callable, Any, List, Type, Union, MutableMapping
 )


Test patch:
Repository: dbt-labs/dbt-core

Issue:
[BigQuery] Allow more recent versions of google-api-core?
### Describe the feature

Currently dbt-bigquery has [an upper limit of 1.16 on `google-api-core`](https://github.com/fishtown-analytics/dbt/blob/v0.18.1b3/plugins/bigquery/setup.py#L53). That release is from Jan of this year.

Would it be possible to loosen that?

While it's clearly not dbt's responsibility for us to be able to install arbitrary packages, here's an example where we can't instally `google-cloud-bigquery-datatransfer` because of this restriction:

```
[SolverProblemError]
Because no versions of google-cloud-bigquery-datatransfer match >2.0.0,<3.0.0
 and google-cloud-bigquery-datatransfer (2.0.0) depends on google-api-core (>=1.22.2,<2.0.0dev), google-cloud-bigquery-datatransfer (>=2.0.0,<3.0.0) requires google-api-core (>=1.22.2,<2.0.0dev).
And because dbt-bigquery (0.18.0) depends on google-api-core (>=1.16.0,<1.17.0), google-cloud-bigquery-datatransfer (>=2.0.0,<3.0.0) is incompatible with dbt-bigquery (0.18.0).
And because dbt (0.18.0) depends on dbt-bigquery (0.18.0)
 and no versions of dbt match >0.18.0,<0.19.0, google-cloud-bigquery-datatransfer (>=2.0.0,<3.0.0) is incompatible with dbt (>=0.18.0,<0.19.0).
So, because {repo} depends on both dbt (^0.18.0) and google-cloud-bigquery-datatransfer (^2.0.0), version solving failed.
```

Thanks as ever for the awesome product!


Discussion:
hey @max-sixty - sure thing - we'd be super happy to expand this version range as long as there are no breaking changes we'd need to account for. 

I scanned through [the changelog](https://googleapis.dev/python/google-api-core/latest/changelog.html) and i don't see anything too controversial in there. I'd be supportive of changing the upper bound to `1.21.0`.

We probably _do not_ want to set the upper bound to `<2.0.0` because breaking changes in libraries like this one can (and do) break older versions of dbt down the line. You buy it?
Superb! That would be perfect. Thank you as ever.

Resolving patch:
diff --git a/plugins/bigquery/setup.py b/plugins/bigquery/setup.py
--- a/plugins/bigquery/setup.py
+++ b/plugins/bigquery/setup.py
@@ -47,11 +47,13 @@
     },
     install_requires=[
         'dbt-core=={}'.format(package_version),
-        'protobuf>=3.6.0,<3.12',
-        'google-cloud-core>=1.3.0,<1.4',
-        'google-cloud-bigquery>=1.25.0,<1.26.0',
-        'google-api-core>=1.16.0,<1.17.0',
-        'googleapis-common-protos>=1.6.0,<1.7.0',
+        'protobuf>=3.13.0,<4',
+        # These are more tightly pinned, as they have a track record of
+        # breaking changes in minor releases.
+        'google-cloud-core>=1.3.0,<1.5',
+        'google-cloud-bigquery>=1.25.0,<2.4',
+        'google-api-core>=1.16.0,<1.24',
+        'googleapis-common-protos>=1.6.0,<1.53',
         'six>=1.14.0',
     ],
     zip_safe=False,


Test patch:
Repository: dbt-labs/dbt-core

Issue:
dbt-redshift pins an old version of boto3/botocore
### Describe the bug
With the new dependency resolver in pip 20.3, mismatched dependencies are raised as an error. I spent many hours last night fighting dbt-redshift's [pin of an old version of boto3/botocore](https://github.com/fishtown-analytics/dbt/commit/c6ab6459fb69a6746982b2416f3f26da78f41fec) - it conflicted with the boto3 requirements of other packages we use. 

It looks like this was done to [match the boto3/botocore versions used by snowflake-connector-python](https://github.com/fishtown-analytics/dbt/commit/c6ab6459fb69a6746982b2416f3f26da78f41fec). That project now has a larger range (snowflake-connector-python==2.3.6 depends on boto3<1.16,>=1.4.4), so I suspect it would be safe to bump up the range supported by dbt-redshift as well. 

### Steps To Reproduce
^

### Expected behavior
^

### Screenshots and log output
pip fails to install packages with conflicting boto3/botocore dependencies.

### System information
**Which database are you using dbt with?**
- [ ] postgres
- [X] redshift
- [ ] bigquery
- [ ] snowflake
- [ ] other (specify: ____________)


**The output of `dbt --version`:**
```
installed version: 0.18.1
   latest version: 0.18.1
```

**The operating system you're using:**
OSX 11.0.1

**The output of `python --version`:**
Python 3.8.2

### Additional context
Add any other context about the problem here.



Discussion:
Thanks for this @pacificsky! Agree, let's bump boto3 + botocore to coordinate with the version ranges supported by `snowflake-connector-python==2.3.6`, as bumped by #2922.

Is this a PR you'd be interested in submitting?
Yeah, happy to do it. Will send out something later tonight.
Also ran into this dependency issue. I'm happy to submit a PR so this doesn't get lost in the shuffle.
yes please!

Resolving patch:
diff --git a/plugins/redshift/setup.py b/plugins/redshift/setup.py
--- a/plugins/redshift/setup.py
+++ b/plugins/redshift/setup.py
@@ -49,7 +49,7 @@
         'dbt-core=={}'.format(package_version),
         'dbt-postgres=={}'.format(package_version),
         # match snowflake-connector-python supported ranges
-        'boto3>=1.4.4,<1.12',
+        'boto3>=1.4.4,<1.16',
         'botocore>=1.5.0,<1.15',
     ],
     zip_safe=False,


Test patch:
Repository: dbt-labs/dbt-core

Issue:
Update comment re: private method rename: _add_ctes
### Describe the bug
In #2834, it looks like we switched the name of the internal method from `_insert_ctes` to `_add_ctes`. That's fine, we should just also update the comment that serves as a hint for plugin authors:

https://github.com/fishtown-analytics/dbt/blob/3af02020ff697cbc34b7bf7d538e99ccd1979afd/core/dbt/compilation.py#L353-L358

### Context
Adapter plugin authors for databases that don't support nested CTEs should have a clear point of intervention to alter dbt's behavior when constructing data tests and ephemeral models:
- [dbt-synapse#25](https://github.com/dbt-msft/dbt-synapse/issues/25)
- [dbt-sqlserver#26](https://github.com/mikaelene/dbt-sqlserver/issues/26)

If we have a little extra time, I'd like to prove it's actually possible to override this behavior for data tests and ephemeral models by overriding `_add_ctes` in an adapter plugin.


Discussion:

Resolving patch:
diff --git a/core/dbt/compilation.py b/core/dbt/compilation.py
--- a/core/dbt/compilation.py
+++ b/core/dbt/compilation.py
@@ -348,13 +348,13 @@ def _add_ctes(
         manifest: Manifest,
         extra_context: Dict[str, Any],
     ) -> NonSourceCompiledNode:
-        """Insert the CTEs for the model."""
+        """Wrap the data test SQL in a CTE."""
 
         # for data tests, we need to insert a special CTE at the end of the
         # list containing the test query, and then have the "real" query be a
         # select count(*) from that model.
-        # the benefit of doing it this way is that _insert_ctes() can be
-        # rewritten for different adapters to handle databses that don't
+        # the benefit of doing it this way is that _add_ctes() can be
+        # rewritten for different adapters to handle databases that don't
         # support CTEs, or at least don't have full support.
         if isinstance(compiled_node, CompiledDataTestNode):
             # the last prepend (so last in order) should be the data test body.


Test patch:
Repository: dbt-labs/dbt-core

Issue:
Remove ellipses logged during parsing
Remove `....` "progress bar" that appears while dbt is parsing. It's fine to leave it in for the `dbt parse` command.

```
$ dbt run -m +fct_sprints
Running with dbt=0.19.0-b1
....
Found 191 models, 249 tests, 3 snapshots, 1 analysis, 325 macros, 1 operation, 10 seed files, 51 sources

13:22:19 | Concurrency: 8 threads (target='dev')
```

**The output of `dbt --version`:**
Current `dev/kiyoshi-kuromiya`


Discussion:

Resolving patch:
diff --git a/core/dbt/parser/manifest.py b/core/dbt/parser/manifest.py
--- a/core/dbt/parser/manifest.py
+++ b/core/dbt/parser/manifest.py
@@ -231,8 +231,6 @@ def parse_project(
             for path in parser.search():
                 self.parse_with_cache(path, parser, old_results)
                 parser_path_count = parser_path_count + 1
-                if parser_path_count % 100 == 0:
-                    print("..", end='', flush=True)
 
             if parser_path_count > 0:
                 project_parser_info.append(ParserInfo(
@@ -241,8 +239,6 @@ def parse_project(
                     elapsed=time.perf_counter() - parser_start_timer
                 ))
                 total_path_count = total_path_count + parser_path_count
-        if total_path_count > 100:
-            print("..")
 
         elapsed = time.perf_counter() - start_timer
         project_info = ProjectLoaderInfo(


Test patch:
Repository: dbt-labs/dbt-core

Issue:
Add exposures count to printed compile stats
### Describe the feature
At the start of an invocation, dbt prints:
```
Found 1 model, 0 tests, 0 snapshots, 0 analyses, 160 macros, 0 operations, 1 seed file, 0 sources
```
We should add `exposures` to this list :)


Discussion:

Resolving patch:
diff --git a/core/dbt/compilation.py b/core/dbt/compilation.py
--- a/core/dbt/compilation.py
+++ b/core/dbt/compilation.py
@@ -52,6 +52,7 @@ def print_compile_stats(stats):
         NodeType.Operation: 'operation',
         NodeType.Seed: 'seed file',
         NodeType.Source: 'source',
+        NodeType.Exposure: 'exposure',
     }
 
     results = {k: 0 for k in names.keys()}
@@ -81,6 +82,8 @@ def _generate_stats(manifest: Manifest):
 
     for source in manifest.sources.values():
         stats[source.resource_type] += 1
+    for exposure in manifest.exposures.values():
+        stats[exposure.resource_type] += 1
     for macro in manifest.macros.values():
         stats[macro.resource_type] += 1
     return stats


Test patch:
Repository: dbt-labs/dbt-core

Issue:
Update to use new version of Hologram
As an extension of https://github.com/fishtown-analytics/hologram/issues/40 -- support Mashumaro in Hologram -- makes changes to pull in version 0.0.13 of Hologram. 


Discussion:

Resolving patch:
diff --git a/core/setup.py b/core/setup.py
--- a/core/setup.py
+++ b/core/setup.py
@@ -70,7 +70,7 @@ def read(fname):
         'json-rpc>=1.12,<2',
         'werkzeug>=0.15,<2.0',
         'dataclasses==0.6;python_version<"3.7"',
-        # 'hologram==0.0.12', # must be updated prior to release
+        'hologram==0.0.13',
         'logbook>=1.5,<1.6',
         'typing-extensions>=3.7.4,<3.8',
         # the following are all to match snowflake-connector-python


Test patch: