internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: dbt-labs/dbt-core

Issue:
[LMA regression] Runs randomly fail with "SchemaError"s
### Describe the bug
If you run dbt commands enough times, you will eventually get a weird jsonschema error:

```
  None is not of type 'object', 'boolean'

  Failed validating 'type' in metaschema['properties']['definitions']['additionalProperties']:
      {'$id': 'http://json-schema.org/draft-07/schema#',
  <snipped a lot of json here>
  On schema['definitions']['Policy']:
      None
```

That last line is one of a few possible values. It only seems to happen with multiple threads. I assume it involves some aspect of the stuff hologram does for schema generation/caching.

dbt should either wrap `hologram` and retry, or we should fix `hologram` and rely on the fixed version.

### Steps To Reproduce
1) Clone jaffle shop, or any other working dbt project
2) echo ''; while [[ $? -eq 0 ]]; do dbt docs generate --threads=8; done
3) wait a bit

There's possibly more to this, I think I've also triggered it with `dbt run`, but that seems much more difficult.


### Expected behavior
dbt should pass/should not raise big errors about metaschema junk

### Screenshots and log output
See above

### System information
**Which database are you using dbt with?**
- [X] postgres

**The output of `dbt --version`:**
```
installed version: 0.15.0-b2
   latest version: 0.14.3

Your version of dbt is ahead of the latest release!
```

**The operating system you're using:**
Macos Mojave, also seen on docker

**The output of `python --version`:**
Python 3.6.5

### Additional context



Discussion:

Resolving patch:
diff --git a/core/setup.py b/core/setup.py
--- a/core/setup.py
+++ b/core/setup.py
@@ -56,7 +56,7 @@ def read(fname):
         'json-rpc>=1.12,<2',
         'werkzeug>=0.15,<0.17',
         'dataclasses==0.6;python_version<"3.7"',
-        'hologram==0.0.4',
+        'hologram==0.0.5',
         'logbook>=1.5,<1.6',
         'pytest-logbook>=1.2.0,<1.3',
         'typing-extensions>=3.7.4,<3.8',


Test patch:
Repository: dbt-labs/dbt-core

Issue:
[LMA Regression] redshift get_catalog requires exactly one database
### Describe the bug
Users are seeing the `redshift get_catalog requires exactly one database` error on dbt v0.15.0-rc1 even though they only have a single database specified in their project.

### Steps To Reproduce
Exact reproduction case tbd


Discussion:

Resolving patch:
diff --git a/core/dbt/adapters/base/relation.py b/core/dbt/adapters/base/relation.py
--- a/core/dbt/adapters/base/relation.py
+++ b/core/dbt/adapters/base/relation.py
@@ -246,7 +246,10 @@ def information_schema(self, view_name=None) -> 'InformationSchema':
         if not isinstance(view_name, str):
             view_name = None
 
-        return InformationSchema.from_relation(self, view_name)
+        # Kick the user-supplied schema out of the information schema relation
+        # Instead address this as <database>.information_schema by default
+        info_schema = InformationSchema.from_relation(self, view_name)
+        return info_schema.incorporate(path={"schema": None})
 
     def information_schema_only(self) -> 'InformationSchema':
         return self.information_schema()


Test patch:
Repository: dbt-labs/dbt-core

Issue:
Fix thread names in dbt server logs
### Describe the bug
The dbt server logging output shows "QueueFeederThread" as the source for all log output. This isn't correct - the logs are actually being emitted by threads named things like `master` or `Thread-3`. Let's update the log handling code to print the correct thread name in the structured logs.

This change will help us better understand:
1. parallelized execution in dbt
2. per-thread connections and query pooling

### Expected behavior
The `thread_name` in structured log output in the dbt server responses (like from `poll`) should render out as `master` or `Thread-N` or similar, not `QueueFeederThread`.

### Screenshots and log output
<img width="531" alt="Screen Shot 2019-11-08 at 12 10 21 PM" src="https://user-images.githubusercontent.com/1541139/68496605-c37a4280-0220-11ea-8902-e17a32fdc5b5.png">

**The output of `dbt --version`:**
```
0.15.0-b3
```

**The operating system you're using:**
macOS

**The output of `python --version`:**
Py3.7.5


Discussion:

Resolving patch:
diff --git a/core/dbt/rpc/logger.py b/core/dbt/rpc/logger.py
--- a/core/dbt/rpc/logger.py
+++ b/core/dbt/rpc/logger.py
@@ -89,6 +89,8 @@ def create(cls):
 
 class QueueLogHandler(logbook.queues.MultiProcessingHandler):
     def emit(self, record: logbook.LogRecord):
+        # trigger the cached proeprties here
+        record.pull_information()
         self.queue.put_nowait(QueueLogMessage.from_record(record))
 
     def emit_error(self, error: JSONRPCError):


Test patch:
Repository: dbt-labs/dbt-core

Issue:
Show a more helpful error message when plugins cannot be imported
### Describe the bug
If a dbt plugin cannot be imported, dbt will show a message about invalid credentials, eg:
```
Running with dbt=0.14.3
Error importing adapter: cannot import name 'collections_abc'
Encountered an error while reading profiles:
  ERROR Runtime Error
  Credentials in profile "<profile name>", target "dev" invalid: Runtime Error
    Could not find adapter type bigquery!
Defined profiles:
 - ....
```

If this is an error that occurs at import-time, dbt should not indicate that the credentials are invalid. dbt should also not show a list of defined profiles in this error message.

Desired output:
```
Running with dbt=0.14.3
The dbt plugin 'bigquery' could not be imported: cannot import name 'collections_abc'
```

dbt version: v0.14.3. I need to test against 0.15.0 to determine if this behavior has recently changed.


Discussion:

Resolving patch:
diff --git a/core/dbt/adapters/factory.py b/core/dbt/adapters/factory.py
--- a/core/dbt/adapters/factory.py
+++ b/core/dbt/adapters/factory.py
@@ -46,16 +46,17 @@ def load_plugin(self, name: str) -> Type[Credentials]:
         # singletons
         try:
             mod = import_module('.' + name, 'dbt.adapters')
-        except ImportError as e:
-            logger.info("Error importing adapter: {}".format(e))
-            raise RuntimeException(
-                "Could not find adapter type {}!".format(name)
-            )
-        if not hasattr(mod, 'Plugin'):
-            raise RuntimeException(
-                f'Could not find plugin in {name} plugin module'
-            )
-        plugin: AdapterPlugin = mod.Plugin  # type: ignore
+        except ModuleNotFoundError as exc:
+            # if we failed to import the target module in particular, inform
+            # the user about it via a runtiem error
+            logger.info(f'Error importing adapter: {exc}')
+            if exc.name == 'dbt.adapters.' + name:
+                raise RuntimeException(f'Could not find adapter type {name}!')
+            # otherwise, the error had to have come from some underlying
+            # library. Log the stack trace.
+            logger.debug('', exc_info=True)
+            raise
+        plugin = mod.Plugin  # type: AdapterPlugin
         plugin_type = plugin.adapter.type()
 
         if plugin_type != name:


Test patch:
Repository: dbt-labs/dbt-core

Issue:
Ephemeral models are shown as cancelled on KeyboardInterrupt
### Describe the bug
This is a minor UI bug - ephemeral models are shown as "cancelled" when a running dbt process is ctrl-c'd. I didn't dig so far into the implementation that I know exactly where this is coming from, but my instinct is that it has something to do with dbt "selecting" all ephemeral models for a run by default, a recent-ish performance improvement.

### Steps To Reproduce
1. Create an ephemeral model
```
{{ config(materialized='ephemeral') }}

select 1 as id
```

2. Create a slow table model
```
{{ config(materialized='table') }}

select 1 as id from pg_sleep(10)
```

3. `dbt run`
4. `ctrl-c` the running process.

Results:
```
14:50:59 | Concurrency: 8 threads (target='dev')
14:50:59 |
14:50:59 | 1 of 1 START table model test_schema.my_first_model............................ [RUN]
^C14:51:05 | CANCEL query model.debug.eph............................................. [CANCEL]
********* ^ this ephemeral should not be shown as cancelled ^ **********
14:51:05 | CANCEL query model.debug.my_first_model.................................. [CANCEL]
14:51:05 | 1 of 1 ERROR creating table model test_schema.my_first_model................... [ERROR in 5.30s]
```

**The output of `dbt --version`:**
```
v0.15.0
```


Discussion:

Resolving patch:
diff --git a/core/dbt/task/runnable.py b/core/dbt/task/runnable.py
--- a/core/dbt/task/runnable.py
+++ b/core/dbt/task/runnable.py
@@ -242,6 +242,11 @@ def execute_nodes(self):
                 raise
 
             for conn_name in adapter.cancel_open_connections():
+                if self.manifest is not None:
+                    node = self.manifest.nodes.get(conn_name)
+                    if node is not None and node.is_ephemeral_model:
+                        continue
+                # if we don't have a manifest/don't have a node, print anyway.
                 dbt.ui.printer.print_cancel_line(conn_name)
 
             pool.join()


Test patch:
Repository: dbt-labs/dbt-core

Issue:
Create -t flag as alias for dbt run --target
I love the `-m` flag as an alias for `--models` (in #1161) but now it's completely messed up my muscle memory for `--target`! I'm now repeatedly typing `-target` instead of `--target` when I want to run dbt on production.


Discussion:

Resolving patch:
diff --git a/core/dbt/main.py b/core/dbt/main.py
--- a/core/dbt/main.py
+++ b/core/dbt/main.py
@@ -244,6 +244,7 @@ def _build_base_subparser():
     )
 
     base_subparser.add_argument(
+        '-t',
         '--target',
         default=None,
         type=str,


Test patch:
Repository: dbt-labs/dbt-core

Issue:
Unhelpful error message with malformed packages.yml
### Describe the bug
```
MacBook-Pro:analytics tristan$ cat packages.yml
packages:
  - package: fishtown-analytics/stripe
    revision: 0.3.1
```

I had recently switched from a git reference to this package to a hub reference, and forgot that I needed to change from `revision` to `version`. This is the error message I received, which made it significantly more challenging to debug:

```
MacBook-Pro:analytics tristan$ dbt clean && dbt deps
Running with dbt=0.15.0
Encountered an error while reading the project:
  ERROR: Runtime Error
  Invalid package config: at path []: 'local' is a required property
```

### Expected behavior
I would expect an error message like:
```
Required parameter version not supplied.
```
...or something similar.

**The output of `dbt --version`:**
```
MacBook-Pro:analytics tristan$ dbt --version
installed version: 0.15.0
   latest version: 0.15.0

Up to date!
```


Discussion:
well `local` is actually also valid error, since it could be the case if package is stored locally. I can suggest error msg like `packages.yml is malformed, please make sure that it matches https://docs.getdbt.com/docs/package-management#section-how-do-i-add-a-package-to-my-project` 
good idea @sonac - are you interested in submitting a PR for that change? I think it should be a quick fix!

Resolving patch:
diff --git a/core/dbt/config/project.py b/core/dbt/config/project.py
--- a/core/dbt/config/project.py
+++ b/core/dbt/config/project.py
@@ -59,6 +59,17 @@
 --no-version-check
 """
 
+MALFORMED_PACKAGE_ERROR = """\
+The packages.yml file in this project is malformed. Please double check
+the contents of this file and fix any errors before retrying.
+
+You can find more information on the syntax for this file here:
+https://docs.getdbt.com/docs/package-management
+
+Validator Error:
+{error}
+"""
+
 
 def _list_if_none(value):
     if value is None:
@@ -130,7 +141,7 @@ def package_config_from_data(packages_data):
         packages = PackageConfig.from_dict(packages_data)
     except ValidationError as e:
         raise DbtProjectError(
-            'Invalid package config: {}'.format(validator_error_message(e))
+            MALFORMED_PACKAGE_ERROR.format(error=str(e.message))
         ) from e
     return packages
 


Test patch:
Repository: dbt-labs/dbt-core

Issue:
Fix docstring for dbt docs generate
### Describe the bug
The docstring for dbt docs seems to indicate that if you want to use a different target, you would put the `--target` parameter before generate:
```
# dbt docs -h
usage: dbt docs [-h] [--project-dir PROJECT_DIR] [--profiles-dir PROFILES_DIR] [--profile PROFILE] [--target TARGET] [--vars VARS] [--bypass-cache] {generate,serve} ...
```
However, when I try that, it doesn't honor the parameter
```
# dbt docs --target prod generate
Running with dbt=0.14.3
Found 125 models, 207 tests, 15 snapshots, 0 analyses, 254 macros, 2 operations, 3 seed files, 101 sources
19:31:38 | Concurrency: 4 threads (target='dev')
```

However, if I put the flag at the end like this, it works fine:
```
# dbt docs generate --target prod
```

### Steps To Reproduce
See above

### Expected behavior
The docs should be in alignment with the actual commands.

### Screenshots and log output
See above

### System information
**Which database are you using dbt with?**
- [ ] postgres
- [x ] redshift
- [ ] bigquery
- [ ] snowflake
- [ ] other (specify: ____________)


**The output of `dbt --version`:**
```
installed version: 0.14.3
```

**The operating system you're using:** Debian GNU/Linux 10 (buster)

**The output of `python --version`:** 3.6.9

### Additional context




Discussion:
Thanks for the report @gnilrets! I think this should be a quick fix - I just queued this up for our 0.16.0 release. Let me know if it's something you would be interested in contributing a fix for!

Resolving patch:
diff --git a/core/dbt/main.py b/core/dbt/main.py
--- a/core/dbt/main.py
+++ b/core/dbt/main.py
@@ -287,7 +287,6 @@ def _build_base_subparser():
 def _build_docs_subparser(subparsers, base_subparser):
     docs_sub = subparsers.add_parser(
         'docs',
-        parents=[base_subparser],
         help='''
         Generate or serve the documentation website for your project.
         '''
@@ -298,7 +297,6 @@ def _build_docs_subparser(subparsers, base_subparser):
 def _build_source_subparser(subparsers, base_subparser):
     source_sub = subparsers.add_parser(
         'source',
-        parents=[base_subparser],
         help='''
         Manage your project's sources
         ''',
@@ -884,7 +882,9 @@ def parse_args(args, cls=DBTArgumentParser):
         sys.exit(1)
 
     parsed = p.parse_args(args)
-    parsed.profiles_dir = os.path.expanduser(parsed.profiles_dir)
+
+    if hasattr(parsed, 'profiles_dir'):
+        parsed.profiles_dir = os.path.expanduser(parsed.profiles_dir)
 
     if not hasattr(parsed, 'which'):
         # the user did not provide a valid subcommand. trigger the help message


Test patch:
Repository: dbt-labs/dbt-core

Issue:
Minor rendering bug for clickable `docs serve` output
### Describe the bug

When running `dbt serve` the printout of the url where the docs are served is rendering with a period ('.') immediately adjacent to the link. This is a minor issue but it prevents link detection from working correctly by terminals (VS Code in my case) which automatically detect URLs and render them as hyperlinks. In this case, the output is correctly parsed as a URL but because the URL includes a '.' at the end, it reaches a dead end (equivalent to pasting `http://localhost:8081.` in chrome address bar).

```
To access from your browser, navigate to http://localhost:8080.
```

### Steps To Reproduce

1. Run `dbt docs serve`

### Expected behavior

Ideally, the printout would be rendered as:

```
To access from your browser, navigate to: http://localhost:8080
```

### Screenshots and log output

```
Running with dbt=0.13.0
Serving docs at 0.0.0.0:8080
To access from your browser, navigate to http://localhost:8080.
Press Ctrl+C to exit.
```

### System information
**Which database are you using dbt with?**
- [ ] postgres
- [ ] redshift
- [ ] bigquery
- [ ] snowflake
- [ ] other (specify: ____________)


**The output of `dbt --version`:**
```
Running with dbt=0.13.0
```

**The operating system you're using:**

**The output of `python --version`:**

### Additional context
Add any other context about the problem here.



Discussion:
PR created here: https://github.com/aaronsteers/dbt/pull/1
Thanks @aaronsteers - I buy this!

Can you open the PR you made against this repo? If you set the base branch to `dev/barbara-gittings`, then I'd be happy to merge it post-haste :) 
Happy to! I'll rebase against that branch (`dev/barbara-gittings`), reopen the PR in this repo, and send this over.
hey @aaronsteers - are you still able to open up that PR against this repo? Would be good to get this fix merged for v0.16.0!
@drewbanin - Yes - sorry to delay this. I'll try to get this done today against `dev/barbara-gittings` - unless I should use a different target (just lmk).

Resolving patch:
diff --git a/core/dbt/task/serve.py b/core/dbt/task/serve.py
--- a/core/dbt/task/serve.py
+++ b/core/dbt/task/serve.py
@@ -20,7 +20,7 @@ def run(self):
 
         logger.info("Serving docs at 0.0.0.0:{}".format(port))
         logger.info(
-            "To access from your browser, navigate to http://localhost:{}."
+            "To access from your browser, navigate to:  http://localhost:{}"
             .format(port)
         )
         logger.info("Press Ctrl+C to exit.\n\n")


Test patch:
Repository: dbt-labs/dbt-core

Issue:
Set upper bound on jinja2 dependency
### Describe the feature
At the time of this writing, jinja2 v3.0.0a1 is now available [on PyPi](https://pypi.org/project/Jinja2/#history). This version is not compatible with dbt.

We should either:
 - set the upper bound on the jinja2 dependency to be `<3.0.0`, or
 - make dbt compatible with version 3.0.0 when it is released (likely as a part of a future issue)

The proximate error here looks like:
```
No module named 'jinja2._compat'
```

as far as I can tell.


Discussion:
I am pretty sure that since we dropped python 2.x support, we don't need that call anymore.

Resolving patch:
diff --git a/core/dbt/clients/jinja.py b/core/dbt/clients/jinja.py
--- a/core/dbt/clients/jinja.py
+++ b/core/dbt/clients/jinja.py
@@ -9,7 +9,6 @@
 )
 
 import jinja2
-import jinja2._compat
 import jinja2.ext
 import jinja2.nodes
 import jinja2.parser
@@ -47,8 +46,6 @@ def _linecache_inject(source, write):
         rnd = codecs.encode(os.urandom(12), 'hex')  # type: ignore
         filename = rnd.decode('ascii')
 
-    # encode, though I don't think this matters
-    filename = jinja2._compat.encode_filename(filename)
     # put ourselves in the cache
     cache_entry = (
         len(source),
@@ -79,10 +76,7 @@ def parse_macro(self):
 
 class MacroFuzzEnvironment(jinja2.sandbox.SandboxedEnvironment):
     def _parse(self, source, name, filename):
-        return MacroFuzzParser(
-            self, source, name,
-            jinja2._compat.encode_filename(filename)
-        ).parse()
+        return MacroFuzzParser(self, source, name, filename).parse()
 
     def _compile(self, source, filename):
         """Override jinja's compilation to stash the rendered source inside
diff --git a/core/setup.py b/core/setup.py
--- a/core/setup.py
+++ b/core/setup.py
@@ -53,12 +53,14 @@ def read(fname):
         'scripts/dbt',
     ],
     install_requires=[
-        'Jinja2>=2.10',
+        'Jinja2>=2.10,<3',
         'PyYAML>=3.11',
         'sqlparse>=0.2.3,<0.4',
         'networkx>=2.3,<3',
         'minimal-snowplow-tracker==0.0.2',
-        'requests>=2.18.0,<3',
+        # match snowflake-connector-python
+        'requests>=2.18.0,<2.23.0',
+        'idna<2.9',
         'colorama>=0.3.9,<0.5',
         'agate>=1.6,<2',
         'isodate>=0.6,<0.7',
diff --git a/plugins/snowflake/setup.py b/plugins/snowflake/setup.py
--- a/plugins/snowflake/setup.py
+++ b/plugins/snowflake/setup.py
@@ -40,7 +40,7 @@
     },
     install_requires=[
         'dbt-core=={}'.format(package_version),
-        'snowflake-connector-python==2.2.0',
+        'snowflake-connector-python==2.2.1',
         'azure-common<2.0.0',
         'azure-storage-blob<12.0.0',
         'urllib3>=1.20,<1.26.0',


Test patch:
Repository: dbt-labs/dbt-core

Issue:
snowflake + cffi = more sadness
### Describe the bug
We can't build homebrew packages, because the tool we use chokes on version conflicts (reasonable!) and snowflake requires `cffi<1.14`, but other dependencies fire first and install `cffi==1.14`.

### Steps To Reproduce
```
pip install dbt==0.16.0rc1 homebrew-pypi-poet
poet --resources dbt
```

Get an error about version conflicts and cffi.

### Expected behavior
I should get some homebrew package stanzas out, not an error.

### System information
**Which database are you using dbt with?**
Any


**The output of `dbt --version`:**
N/A

**The operating system you're using:**
macos

**The output of `python --version`:**
Python 3.8.0

### Additional context
In what's becoming a real tradition, we need to pin yet another snowflake dependency in the root package, in this case `cffi<1.14`.


Discussion:

Resolving patch:
diff --git a/core/setup.py b/core/setup.py
--- a/core/setup.py
+++ b/core/setup.py
@@ -58,9 +58,6 @@ def read(fname):
         'sqlparse>=0.2.3,<0.4',
         'networkx>=2.3,<3',
         'minimal-snowplow-tracker==0.0.2',
-        # match snowflake-connector-python
-        'requests>=2.18.0,<2.23.0',
-        'idna<2.9',
         'colorama>=0.3.9,<0.5',
         'agate>=1.6,<2',
         'isodate>=0.6,<0.7',
@@ -71,6 +68,10 @@ def read(fname):
         'logbook>=1.5,<1.6',
         'pytest-logbook>=1.2.0,<1.3',
         'typing-extensions>=3.7.4,<3.8',
+        # the following are all to match snowflake-connector-python
+        'requests>=2.18.0,<2.23.0',
+        'idna<2.9',
+        'cffi>=1.9,<1.14',
     ],
     zip_safe=False,
     classifiers=[


Test patch:
Repository: dbt-labs/dbt-core

Issue:
[Barbara Gittings] Raise minimum google-cloud-bigquery version to 1.22.0
### Describe the feature
dbt v0.16.0 uses `table.range_partitioning`, which was added in [google-cloud-bigquery v1.22.0](https://googleapis.dev/python/bigquery/latest/changelog.html#id10). The current range for this dependency is [>= 1.15.0, < 1.25.0](https://github.com/fishtown-analytics/dbt/blob/dev/octavius-catto/plugins/bigquery/setup.py#L44), so some folks could have a successful dbt installation but would be unable to use range partitioning in their incremental models!

Let's bump up the required version to 1.22.0

### Describe alternatives you've considered
If we _really_ wanted to, we could catch this error and show a warning that the installed version of `google-cloud-bigquery` does not support range partitioning.

### Additional context
BigQuery only

### Who will this benefit?
Range partitioners


Discussion:

Resolving patch:
diff --git a/plugins/bigquery/setup.py b/plugins/bigquery/setup.py
--- a/plugins/bigquery/setup.py
+++ b/plugins/bigquery/setup.py
@@ -41,7 +41,7 @@
     install_requires=[
         'dbt-core=={}'.format(package_version),
         'google-cloud-core>=1,<=1.3.0',
-        'google-cloud-bigquery>=1.15.0,<1.25.0',
+        'google-cloud-bigquery>=1.22.0,<1.25.0',
         # hidden secret dependency: bq 1.23.0 requires this but only documents
         # 1.10.0 through its dependency chain.
         # see https://github.com/googleapis/google-cloud-python/issues/9965


Test patch: