internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: dbt-labs/dbt-core

Issue:
Ensure that all dbt invocations have an invocation_id
This is a sub-issue of #4260. 

In order to make sure we can distinguish logs from different runs we should always have an invocation_id. If tracking is off, it isn't created today so we would need to create it. (to add it to logging, add it as a class property on the superclass) (previously considered: preserve process id like it is today, global run_id reported at start, or allow some sort of user markers).


Discussion:

Resolving patch:
diff --git a/core/dbt/events/functions.py b/core/dbt/events/functions.py
--- a/core/dbt/events/functions.py
+++ b/core/dbt/events/functions.py
@@ -324,3 +324,10 @@ def get_invocation_id() -> str:
     if invocation_id is None:
         invocation_id = str(uuid.uuid4())
     return invocation_id
+
+
+def set_invocation_id() -> None:
+    # This is primarily for setting the invocation_id for separate
+    # commands in the dbt servers. It shouldn't be necessary for the CLI.
+    global invocation_id
+    invocation_id = str(uuid.uuid4())


Test patch:
Repository: dbt-labs/dbt-core

Issue:
Ensure that all dbt invocations have an invocation_id
This is a sub-issue of #4260. 

In order to make sure we can distinguish logs from different runs we should always have an invocation_id. If tracking is off, it isn't created today so we would need to create it. (to add it to logging, add it as a class property on the superclass) (previously considered: preserve process id like it is today, global run_id reported at start, or allow some sort of user markers).


Discussion:

Resolving patch:
diff --git a/core/dbt/lib.py b/core/dbt/lib.py
--- a/core/dbt/lib.py
+++ b/core/dbt/lib.py
@@ -12,6 +12,7 @@
 def get_dbt_config(project_dir, single_threaded=False):
     from dbt.config.runtime import RuntimeConfig
     import dbt.adapters.factory
+    import dbt.events.functions
 
     if os.getenv('DBT_PROFILES_DIR'):
         profiles_dir = os.getenv('DBT_PROFILES_DIR')
@@ -28,6 +29,8 @@ def get_dbt_config(project_dir, single_threaded=False):
     dbt.adapters.factory.reset_adapters()
     # Load the relevant adapter
     dbt.adapters.factory.register_adapter(config)
+    # Set invocation id
+    dbt.events.functions.set_invocation_id()
 
     return config
 


Test patch:
Repository: dbt-labs/dbt-core

Issue:
Make sure stdout logger goes to stdout.
The console logs were going to stderr. Change to make sure they go to stdout.


Discussion:

Resolving patch:
diff --git a/core/dbt/events/functions.py b/core/dbt/events/functions.py
--- a/core/dbt/events/functions.py
+++ b/core/dbt/events/functions.py
@@ -13,6 +13,7 @@
 import logbook
 import logging
 from logging import Logger
+import sys
 from logging.handlers import RotatingFileHandler
 import os
 import uuid
@@ -38,7 +39,7 @@
 global STDOUT_LOG
 STDOUT_LOG = logging.getLogger('default_stdout')
 STDOUT_LOG.setLevel(logging.INFO)
-stdout_handler = logging.StreamHandler()
+stdout_handler = logging.StreamHandler(sys.stdout)
 stdout_handler.setLevel(logging.INFO)
 STDOUT_LOG.addHandler(stdout_handler)
 
@@ -64,7 +65,7 @@ def setup_event_logger(log_path, level_override=None):
     FORMAT = "%(message)s"
     stdout_passthrough_formatter = logging.Formatter(fmt=FORMAT)
 
-    stdout_handler = logging.StreamHandler()
+    stdout_handler = logging.StreamHandler(sys.stdout)
     stdout_handler.setFormatter(stdout_passthrough_formatter)
     stdout_handler.setLevel(level)
     # clear existing stdout TextIOWrapper stream handlers


Test patch:
Repository: dbt-labs/dbt-core

Issue:
log_format in UserConfig should be str, not bool
https://github.com/dbt-labs/dbt-core/blob/eeb490ed154a71e42564f5ec8d6b1bc03216a378/core/dbt/contracts/project.py#L234

I'd expect to be able to set this as:
```yml
# profiles.yml
config:
  log_format: json
```
Which I think just looks like replacing it with:
```python
  log_format: Optional[str] = None 
```


Discussion:
@jtcohen6 After I make this change and set my config in `profiles.yml` to have `log_format: json` would you expect logs to output as json if I use `dbt --log-format default run`?
The `--log-format` flag should take precedence, so I'd expect it to be text

The order of precedence for global configs is:
1. CLI flags
2. Env vars (`DBT_LOG_FORMAT`)
3. User config in `profiles.yml`
@jtcohen6 Mostly out of curiosity - Why is `default` an option if you can never change the default?

Resolving patch:
diff --git a/core/dbt/contracts/project.py b/core/dbt/contracts/project.py
--- a/core/dbt/contracts/project.py
+++ b/core/dbt/contracts/project.py
@@ -231,7 +231,7 @@ class UserConfig(ExtensibleDbtClassMixin, Replaceable, UserConfigContract):
     printer_width: Optional[int] = None
     write_json: Optional[bool] = None
     warn_error: Optional[bool] = None
-    log_format: Optional[bool] = None
+    log_format: Optional[str] = None
     debug: Optional[bool] = None
     version_check: Optional[bool] = None
     fail_fast: Optional[bool] = None


Test patch:
Repository: dbt-labs/dbt-core

Issue:
`dbt.log` should rollover at 10 mb, as before
https://docs.python.org/3/library/logging.handlers.html#rotatingfilehandler

https://github.com/dbt-labs/dbt-core/blob/74fbaa18cd7522161b8e0db97393c73d869a0498/core/dbt/logger.py#L386-L387


Discussion:

Resolving patch:
diff --git a/core/dbt/events/functions.py b/core/dbt/events/functions.py
--- a/core/dbt/events/functions.py
+++ b/core/dbt/events/functions.py
@@ -82,7 +82,12 @@ def setup_event_logger(log_path, level_override=None):
 
     file_passthrough_formatter = logging.Formatter(fmt=FORMAT)
 
-    file_handler = RotatingFileHandler(filename=log_dest, encoding='utf8')
+    file_handler = RotatingFileHandler(
+        filename=log_dest,
+        encoding='utf8',
+        maxBytes=10 * 1024 * 1024,  # 10 mb
+        backupCount=5
+    )
     file_handler.setFormatter(file_passthrough_formatter)
     file_handler.setLevel(logging.DEBUG)  # always debug regardless of user input
     this.FILE_LOG.handlers.clear()


Test patch:
Repository: dbt-labs/dbt-core

Issue:
[Bug] dbt init overwrite just appends to profile
### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

If you have a profile setup and then you try to setup a new one, it will ask if you want to overwrite existing entry. It just adds another entry and does not overwrite it. 

1) dbt init a new project and verify the connection in profile.yml 
2) inside the existing project run `dbt init` again and when it asks you to overwrite the connection choose yes. 
There are now 2 entries in profile.yml for it

### Expected Behavior

_No response_

### Steps To Reproduce

_No response_

### Relevant log output

_No response_

### Environment

```markdown
- OS:
- Python:
- dbt:
```


### What database are you using dbt with?

_No response_

### Additional Context

_No response_


Discussion:
@leahwicz This isn't something I've managed to reproduce locally. I imagine the fix for it is super straightforward, though
@jtcohen6 yes I was able to do it again so that's why I filed it. I would rate this low on priorities but I would assume its an easy fix as well

Resolving patch:
diff --git a/core/dbt/task/init.py b/core/dbt/task/init.py
--- a/core/dbt/task/init.py
+++ b/core/dbt/task/init.py
@@ -297,6 +297,8 @@ def run(self):
             # just setup the user's profile.
             fire_event(SettingUpProfile())
             profile_name = self.get_profile_name_from_current_project()
+            if not self.check_if_can_write_profile(profile_name=profile_name):
+                return
             # If a profile_template.yml exists in the project root, that effectively
             # overrides the profile_template.yml for the given target.
             profile_template_path = Path("profile_template.yml")
@@ -308,8 +310,6 @@ def run(self):
                     return
                 except Exception:
                     fire_event(InvalidProfileTemplateYAML())
-            if not self.check_if_can_write_profile(profile_name=profile_name):
-                return
             adapter = self.ask_for_adapter_choice()
             self.create_profile_from_target(
                 adapter, profile_name=profile_name


Test patch:
Repository: dbt-labs/dbt-core

Issue:
[CORE-15] `SQLQueryStatus` event expects `get_response` to return `AdapterResponse` object only
See https://github.com/dbt-labs/dbt-spark/pull/265 for context

My proposal would be:

- **v1.0.1:** Check the type of the value returned by `get_response`, then either use its `_message` attribute (if `AdapterResponse`) or itself (if `str`)
- **v1.1.0:** Enforce that the `get_response` abstract method must return an `AdapterResponse` — not a bad change, and it's _effectively_ true, but we should wait for a minor version release, with clearer communication ahead of time


Discussion:

Resolving patch:
diff --git a/core/dbt/adapters/sql/connections.py b/core/dbt/adapters/sql/connections.py
--- a/core/dbt/adapters/sql/connections.py
+++ b/core/dbt/adapters/sql/connections.py
@@ -75,7 +75,7 @@ def add_query(
 
             fire_event(
                 SQLQueryStatus(
-                    status=self.get_response(cursor)._message,
+                    status=str(self.get_response(cursor)),
                     elapsed=round((time.time() - pre), 2)
                 )
             )


Test patch:
Repository: dbt-labs/dbt-core

Issue:
Cancelling Runs Using CTRL-C In DBT 1.0.0 Triggers Verbose Error Message
### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

When I start running a valid model using my valid _dbt_ configuration and then cancel it using Ctrl-C in bash, the error message that appears is:

```The <bound method BaseAdapter.type of <class 'dbt.adapters.bigquery.impl.BigQueryAdapter'>> adapter does not support query cancellation. Some queries may still be running!```

### Expected Behavior

This seems like a regression over the earlier error message in dbt 0.19.1:
```The big query adapter does not support query cancellation. Some queries may still be running!```

### Steps To Reproduce

1.  From a valid BigQuery configuration profile and project with at least one model, enter _dbt run_ in bash.
2. Before the command finishes running, use Ctrl-C to interrupt execution.

### Relevant log output

_No response_

### Environment

```markdown
- OS: Mac OS X 11.4 (20F71)
- Python: Python 3.8.10
- dbt: 1.0.0 (along with dbt-bigquery 1.0.0)
```


### What database are you using dbt with?

bigquery

### Additional Context

I believe that [this line](https://github.com/dbt-labs/dbt-core/blob/b3039fdc7629b5027096e92ed9b7612419cf3fea/core/dbt/task/runnable.py#L362) is at the root of the issue. 

If you change the code from
 ```fire_event(QueryCancelationUnsupported(type=adapter.type))``` 
to 
```fire_event(QueryCancelationUnsupported(type=adapter.type()))```

 the error message returns to the 0.19.1 version.  It seems like the existing code assigns the actual method to QueryCancelationUnsupported.type, which is then passed into the error message, while adding parentheses is required to actually call the function and fetch the string name of the adapter so it can be passed to the error message.



Discussion:
@kdw2126 Right you are! Thank you so much for opening the issue, and finding the specific spot in the code causing the unexpected output.

I think we should change that line exactly as you recommend, switching `type=adapter.type` for `type=adapter.type()`. After all, the event expects `type` to be a string, not an object method:

https://github.com/dbt-labs/dbt-core/blob/b3039fdc7629b5027096e92ed9b7612419cf3fea/core/dbt/events/types.py#L2148-L2149

Let's include this as a fix in v1.0.1. If you'd like to contribute the fix, and get there before we do, you're welcome to it :)

Resolving patch:
diff --git a/core/dbt/task/runnable.py b/core/dbt/task/runnable.py
--- a/core/dbt/task/runnable.py
+++ b/core/dbt/task/runnable.py
@@ -359,7 +359,7 @@ def _cancel_connections(self, pool):
         adapter = get_adapter(self.config)
 
         if not adapter.is_cancelable():
-            fire_event(QueryCancelationUnsupported(type=adapter.type))
+            fire_event(QueryCancelationUnsupported(type=adapter.type()))
         else:
             with adapter.connection_named('master'):
                 for conn_name in adapter.cancel_open_connections():


Test patch:
Repository: dbt-labs/dbt-core

Issue:
[CT-56] Remove support for Python 3.7.0, 3.7.1
As raised in #4564, running dbt v1.0 with Python 3.7.0 or 3.7.1 results in a Python exception being thrown. We should disallow installing dbt with these versions of Python. This issue is resolved with Python 3.7.2.



Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -80,5 +80,5 @@
         'Programming Language :: Python :: 3.8',
         'Programming Language :: Python :: 3.9',
     ],
-    python_requires=">=3.7",
+    python_requires=">=3.7.2",
 )


Test patch:
Repository: dbt-labs/dbt-core

Issue:
[CT-57] [Feature] Clarify what to do when getting a version incompatibility error
### Is there an existing feature request for this?

- [X] I have searched the existing issues

### Describe the Feature

When trying to run dbt with a package that's incompatible due to require-dbt-version, the error message says "Check the requirements for the 'dbt_utils' package, or run dbt again with--no-version-check". 

I think "Check the requirements" is too ambiguous - we could direct them to check for an updated version of the package. 



### Describe alternatives you've considered

Extra for experts _could_ be to go and check the hub automatically (if it's a hosted package), but that would slow down the response due to the network overhead and potentially couple some things too close together 🤔 

### Who will this benefit?

People who are confused when upgrading dbt versions. Post 1.0 this should be less of an issue, but there's a lot of users who are still to update to 1.0 for the first time so there's still time for this to be relevant. 

### Are you interested in contributing this feature?

Definitely if it's just a phrasing change; maybe if we start futzing around with the hub. 

### Anything else?

_No response_


Discussion:

Resolving patch:
diff --git a/core/dbt/config/project.py b/core/dbt/config/project.py
--- a/core/dbt/config/project.py
+++ b/core/dbt/config/project.py
@@ -45,7 +45,7 @@
 This version of dbt is not supported with the '{package}' package.
   Installed version of dbt: {installed}
   Required version of dbt for '{package}': {version_spec}
-Check the requirements for the '{package}' package, or run dbt again with \
+Check for a different version of the '{package}' package, or run dbt again with \
 --no-version-check
 """
 
@@ -54,7 +54,7 @@
 The package version requirement can never be satisfied for the '{package}
 package.
   Required versions of dbt for '{package}': {version_spec}
-Check the requirements for the '{package}' package, or run dbt again with \
+Check for a different version of the '{package}' package, or run dbt again with \
 --no-version-check
 """
 


Test patch:
Repository: dbt-labs/dbt-core

Issue:
[CT-78] [Bug] Adapters should be able to override string and numeric type of columns
### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

String and numeric types are hardcoded in the base Column classes.

https://github.com/dbt-labs/dbt-core/blob/a588607ec6546de441812e215fbbf4e9ff20101b/core/dbt/adapters/base/column.py#L42-L44

When running dbt_utils.union_relations with the Trino adapter, following columns are generated based, hoever varying is not supported by Trino and the Trino adapter suppleis an overriding method for string_type.

```
cast("administratieve groep/iov code" as character varying(256)) as "administratieve groep/iov code"
```

![image](https://user-images.githubusercontent.com/9265503/150424063-74992128-10ca-4428-ae7e-22a76693ea3a.png)


### Expected Behavior

Adapters should be able to override the stirng and numeric types according to the specifications of the specific database.

### Steps To Reproduce

_No response_

### Relevant log output

_No response_

### Environment

```markdown
- OS: Macos
- Python: 3.9.10
- dbt: 1.0.0
```


### What database are you using dbt with?

other (mention it in "Additional Context")

### Additional Context

Trino


Discussion:
@mdesmet Thank you for opening up this issue, been catching up on context and looking over your PRs going to go ahead and move this from triage to a good first issue and see if we can get the tests running on your PR.

Resolving patch:
diff --git a/core/dbt/adapters/base/column.py b/core/dbt/adapters/base/column.py
--- a/core/dbt/adapters/base/column.py
+++ b/core/dbt/adapters/base/column.py
@@ -39,9 +39,9 @@ def quoted(self) -> str:
     @property
     def data_type(self) -> str:
         if self.is_string():
-            return Column.string_type(self.string_size())
+            return self.string_type(self.string_size())
         elif self.is_numeric():
-            return Column.numeric_type(self.dtype, self.numeric_precision, self.numeric_scale)
+            return self.numeric_type(self.dtype, self.numeric_precision, self.numeric_scale)
         else:
             return self.dtype
 


Test patch:
Repository: dbt-labs/dbt-core

Issue:
[CT-71] Restore previous behavior when a test depends on a disabled (vs. missing) resource
_Prompted by Slack threads [here](https://getdbt.slack.com/archives/C02HM9AAXL4/p1642605259009400) + [here](https://getdbt.slack.com/archives/C02HM9AAXL4/p1641486649000200)_

Folks who've newly upgraded to v1.0 are seeing a _lot_ of warn-level (stdout) messages about tests defined on disabled models. This is what our internal analytics project looks like, and I don't think it's an uncommon experience:

<img width="1623" alt="Screenshot 2022-01-19 at 19 16 48" src="https://user-images.githubusercontent.com/13897643/150190177-b46dec94-eb92-4452-a191-d778eafa0db0.png">

(There's many more below that)

It is desirable to warn about tests that have "missed" the correct model, such as due to a typo in a `ref` or yaml property (e.g. see https://github.com/dbt-labs/dbt-core/issues/4532). It's also a very common pattern to disable unused/irrelevant models in _packages_, and the result is a quite cluttered terminal. There are some useful warnings in there (missing node for patch), lost as needles in haystack-traces.

What I proposed in one of the threads linked above:

> If a test depends on a disabled model, log this as DEBUG-level (shows up in `logs/dbt.log`)
> If a test depends on a missing model, log this as WARN-level (shows up in standard CLI output)

I looked quickly at the code, and turns out this was actually our previous behavior:
https://github.com/dbt-labs/dbt-core/blob/08b2f16e10eb3420ecffee3b4d18ac2f45e601ea/core/dbt/parser/manifest.py#L790-L803

We've since replaced the first of those with a call to the `InvalidRefInTestNode` event, so this would just look like changing:
https://github.com/dbt-labs/dbt-core/blob/a588607ec6546de441812e215fbbf4e9ff20101b/core/dbt/events/types.py#L1186-L1192

To:
```python
@dataclass
class InvalidRefInTestNode(DebugLevel):
    msg: str
    code: str = "I051"

    def message(self) -> str:
        return self.msg
```

Voila, just the good stuff:

<img width="1216" alt="Screenshot 2022-01-19 at 19 19 35" src="https://user-images.githubusercontent.com/13897643/150190625-6d826f79-7302-4451-887c-98ff69f67691.png">


Discussion:
I think this is a great update -- happy to be a tester if/when you need!

Resolving patch:
diff --git a/core/dbt/events/types.py b/core/dbt/events/types.py
--- a/core/dbt/events/types.py
+++ b/core/dbt/events/types.py
@@ -1124,12 +1124,12 @@ def message(self) -> str:
 
 
 @dataclass
-class InvalidRefInTestNode(WarnLevel):
+class InvalidRefInTestNode(DebugLevel):
     msg: str
     code: str = "I051"
 
     def message(self) -> str:
-        return ui.warning_tag(self.msg)
+        return self.msg
 
 
 @dataclass


Test patch:
Repository: dbt-labs/dbt-core

Issue:
[CT-246] [Feature] Have a space after model name before the alignment periods, for ease of double-click selection
### Is there an existing feature request for this?

- [X] I have searched the existing issues

### Describe the Feature

In VS Code, double-clicking a model name selects it and all of the dots that line up the results:
![image](https://user-images.githubusercontent.com/7335046/154422957-a578c0ff-0b27-4d56-9562-f7728dc231f9.png)
```
07:00:06  1 of 1 START seed file dbt_jlabes.data_star_aggregate_expected.................. [RUN]
07:00:18  1 of 1 OK loaded seed file dbt_jlabes.data_star_aggregate_expected.............. [INSERT 3 in 11.49s]
```

If there was a space after expected and before the first period, it wouldn't select them all

### Describe alternatives you've considered

Using dbt Cloud in Chrome, which has different word boundary calculation logic

### Who will this benefit?

VS Code users, maybe other IDEs? 

### Are you interested in contributing this feature?

Sure

### Anything else?

_No response_


Discussion:
You're so right! This has bugged me in the past.

I bet it's a one-liner in here:

https://github.com/dbt-labs/dbt-core/blob/3ad61d5d810aab8bf7bf7bc38f62d02348839c80/core/dbt/events/format.py#L20-L32

Anyone want to take a crack at it?

Resolving patch:
diff --git a/core/dbt/events/format.py b/core/dbt/events/format.py
--- a/core/dbt/events/format.py
+++ b/core/dbt/events/format.py
@@ -15,7 +15,7 @@ def format_fancy_output_line(
         progress = ""
     else:
         progress = "{} of {} ".format(index, total)
-    prefix = "{progress}{message}".format(progress=progress, message=msg)
+    prefix = "{progress}{message} ".format(progress=progress, message=msg)
 
     truncate_width = ui.printer_width() - 3
     justified = prefix.ljust(ui.printer_width(), ".")


Test patch:
Repository: dbt-labs/dbt-core

Issue:
[CT-371] [Bug] CLI vars in packages.yml and selectors.yml don't work.
When we did the work to separate out rendering of secrets, a bug was introduced where instead of using cli_vars to construct the contexts for packages and selectors, we use the entire yaml context (which is also a dict). Because of this we get errors like: "Object of type method is not JSON serializable" and also vars are not found when rendering.


Discussion:

Resolving patch:
diff --git a/core/dbt/config/renderer.py b/core/dbt/config/renderer.py
--- a/core/dbt/config/renderer.py
+++ b/core/dbt/config/renderer.py
@@ -115,10 +115,10 @@ def name(self):
         "Project config"
 
     def get_package_renderer(self) -> BaseRenderer:
-        return PackageRenderer(self.context)
+        return PackageRenderer(self.ctx_obj.cli_vars)
 
     def get_selector_renderer(self) -> BaseRenderer:
-        return SelectorRenderer(self.context)
+        return SelectorRenderer(self.ctx_obj.cli_vars)
 
     def render_project(
         self,


Test patch: