Repository: dbt-labs/dbt-core
Pull request: Add flags from dbt_project.yml to the Project and RuntimeConfig objects
Resolves #10618

### Problem

We need to pass project flags from `dbt_project.yml` into the `BaseAdapter` instantiation to support behavior flags. Without this, users cannot override a behavior flag.

### Solution

- add `flags` as an attribute on `Project`, which then adds it to `RuntimeConfig`
- pull the flags from `dbt_project.yml::flags`, not the global flags
  - the flags we're currently looking for are not expected to be overwritten by CLI args or environment variables
- ensure existing tests pass
- add tests that ensure `RuntimeConfig` contains arbitrary flags added in `dbt_project.yml::flags`

### Checklist

- [x] I have read [the contributing guide](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md) and understand what's expected of me.
- [x] I have run this code in development, and it appears to resolve the stated issue.
- [x] This PR includes tests, or tests are not required or relevant for this PR.
- [x] This PR has no interface changes (e.g., macros, CLI, logs, JSON artifacts, config files, adapter interface, etc.) or this PR has already received feedback and approval from Product or DX.
- [x] This PR includes [type annotations](https://docs.python.org/3/library/typing.html) for new and modified functions.


Problem statement:
[Behavior Flags] Add project flags to `Project` to be passed into `BaseAdapter`
### Housekeeping

- [X] I am a maintainer of dbt-core

### Short description

`BaseAdapter` needs access to project flags so that users are able to override default settings on behavior flags. `BaseAdapter` cannot get this information from project files; all config gets passed in during instantiation. The current behavior is to explicitly parse these flags in `dbt-core`. However, this behavior prevents adapter maintainers from creating their own behavior flags, and increases the complexity for first party adapters to create behavior flags.

```[tasklist]
### Acceptance criteria
- [ ] The `RuntimeConfig` object that gets passed into the `BaseAdapter` factory method includes all project flags
- [ ] Existing interfaces are not broken; this is an additive change
- [ ] All protocols are updated to reflect this change (some are in `dbt-adapters`, e.g. `AdapterRequiredConfig`, and will be handled in a separate ticket)
- [ ] The new flags attribute does not consider CLI arguments nor environment variables, only what is contained in `dbt_project.yml`
```

```[tasklist]
### Suggested Tests
- [ ] Create a `RuntimeConfig` instance, show that there's an empty dictionary for `.flags`
- [ ] Create a behavior flag and override it in `project_config_update`, verify the value on `RuntimeConfig`
- [ ] Create a behavior flag and attempt to override it via CLI args, verify it's not overridden on `RuntimeConfig`
- [ ] Create a behavior flag and attempt to override it via environment variables, verify it's not overridden on `RuntimeConfig`
```

### Impact to Other Teams

This impacts Adapters, but is also being executed by Adapters.

### Will backports be required?

N/A

Hints from issue discussion:

Commit message: add the new attribute to the expected repr

Patch to review:
diff --git a/.changes/unreleased/Features-20240829-135320.yaml b/.changes/unreleased/Features-20240829-135320.yaml
new file mode 100644
index 00000000000..c7f5cf9d8b4
--- /dev/null
+++ b/.changes/unreleased/Features-20240829-135320.yaml
@@ -0,0 +1,6 @@
+kind: Features
+body: Add support for behavior flags
+time: 2024-08-29T13:53:20.16122-04:00
+custom:
+  Author: mikealfare
+  Issue: "10618"
diff --git a/core/dbt/config/project.py b/core/dbt/config/project.py
index 313cdb0bddf..25b0f343ef2 100644
--- a/core/dbt/config/project.py
+++ b/core/dbt/config/project.py
@@ -480,6 +480,7 @@ def create_project(self, rendered: RenderComponents) -> "Project":
                 rendered.selectors_dict["selectors"]
             )
         dbt_cloud = cfg.dbt_cloud
+        flags: Dict[str, Any] = cfg.flags
 
         project = Project(
             project_name=name,
@@ -524,6 +525,7 @@ def create_project(self, rendered: RenderComponents) -> "Project":
             project_env_vars=project_env_vars,
             restrict_access=cfg.restrict_access,
             dbt_cloud=dbt_cloud,
+            flags=flags,
         )
         # sanity check - this means an internal issue
         project.validate()
@@ -568,11 +570,6 @@ def from_project_root(
         ) = package_and_project_data_from_root(project_root)
         selectors_dict = selector_data_from_root(project_root)
 
-        if "flags" in project_dict:
-            # We don't want to include "flags" in the Project,
-            # it goes in ProjectFlags
-            project_dict.pop("flags")
-
         return cls.from_dicts(
             project_root=project_root,
             project_dict=project_dict,
@@ -645,6 +642,7 @@ class Project:
     project_env_vars: Dict[str, Any]
     restrict_access: bool
     dbt_cloud: Dict[str, Any]
+    flags: Dict[str, Any]
 
     @property
     def all_source_paths(self) -> List[str]:
@@ -724,6 +722,7 @@ def to_project_config(self, with_packages=False):
                 "require-dbt-version": [v.to_version_string() for v in self.dbt_version],
                 "restrict-access": self.restrict_access,
                 "dbt-cloud": self.dbt_cloud,
+                "flags": self.flags,
             }
         )
         if self.query_comment:
diff --git a/core/dbt/config/runtime.py b/core/dbt/config/runtime.py
index cea8f559a55..e1c24cf5f0c 100644
--- a/core/dbt/config/runtime.py
+++ b/core/dbt/config/runtime.py
@@ -193,6 +193,7 @@ def from_parts(
             log_cache_events=log_cache_events,
             dependencies=dependencies,
             dbt_cloud=project.dbt_cloud,
+            flags=project.flags,
         )
 
     # Called by 'load_projects' in this class
diff --git a/core/dbt/contracts/project.py b/core/dbt/contracts/project.py
index b0b7179f333..8e605665056 100644
--- a/core/dbt/contracts/project.py
+++ b/core/dbt/contracts/project.py
@@ -259,6 +259,7 @@ class Project(dbtClassMixin):
     query_comment: Optional[Union[QueryComment, NoValue, str]] = field(default_factory=NoValue)
     restrict_access: bool = False
     dbt_cloud: Optional[Dict[str, Any]] = None
+    flags: Dict[str, Any] = field(default_factory=dict)
 
     class Config(dbtMashConfig):
         # These tell mashumaro to use aliases for jsonschema and for "from_dict"
diff --git a/dev-requirements.txt b/dev-requirements.txt
index 20605e632b8..04236dc3718 100644
--- a/dev-requirements.txt
+++ b/dev-requirements.txt
@@ -1,6 +1,6 @@
-git+https://github.com/dbt-labs/dbt-adapters.git@main
-git+https://github.com/dbt-labs/dbt-adapters.git@main#subdirectory=dbt-tests-adapter
-git+https://github.com/dbt-labs/dbt-common.git@main
+git+https://github.com/dbt-labs/dbt-adapters.git@behavior-flags
+git+https://github.com/dbt-labs/dbt-adapters.git@behavior-flags#subdirectory=dbt-tests-adapter
+git+https://github.com/dbt-labs/dbt-common.git@behavior-flags
 git+https://github.com/dbt-labs/dbt-postgres.git@main
 # black must match what's in .pre-commit-config.yaml to be sure local env matches CI
 black==24.3.0
diff --git a/tests/unit/config/test_project.py b/tests/unit/config/test_project.py
index ae0ae3928dc..ab842c164d7 100644
--- a/tests/unit/config/test_project.py
+++ b/tests/unit/config/test_project.py
@@ -43,7 +43,7 @@ def test_fixture_paths(self, project: Project):
     def test__str__(self, project: Project):
         assert (
             str(project)
-            == "{'name': 'test_project', 'version': 1.0, 'project-root': 'doesnt/actually/exist', 'profile': 'test_profile', 'model-paths': ['models'], 'macro-paths': ['macros'], 'seed-paths': ['seeds'], 'test-paths': ['tests'], 'analysis-paths': ['analyses'], 'docs-paths': ['docs'], 'asset-paths': ['assets'], 'target-path': 'target', 'snapshot-paths': ['snapshots'], 'clean-targets': ['target'], 'log-path': 'path/to/project/logs', 'quoting': {}, 'models': {}, 'on-run-start': [], 'on-run-end': [], 'dispatch': [{'macro_namespace': 'dbt_utils', 'search_order': ['test_project', 'dbt_utils']}], 'seeds': {}, 'snapshots': {}, 'sources': {}, 'data_tests': {}, 'unit_tests': {}, 'metrics': {}, 'semantic-models': {}, 'saved-queries': {}, 'exposures': {}, 'vars': {}, 'require-dbt-version': ['=0.0.0'], 'restrict-access': False, 'dbt-cloud': {}, 'query-comment': {'comment': \"\\n{%- set comment_dict = {} -%}\\n{%- do comment_dict.update(\\n    app='dbt',\\n    dbt_version=dbt_version,\\n    profile_name=target.get('profile_name'),\\n    target_name=target.get('target_name'),\\n) -%}\\n{%- if node is not none -%}\\n  {%- do comment_dict.update(\\n    node_id=node.unique_id,\\n  ) -%}\\n{% else %}\\n  {# in the node context, the connection name is the node_id #}\\n  {%- do comment_dict.update(connection_name=connection_name) -%}\\n{%- endif -%}\\n{{ return(tojson(comment_dict)) }}\\n\", 'append': False, 'job-label': False}, 'packages': []}"
+            == "{'name': 'test_project', 'version': 1.0, 'project-root': 'doesnt/actually/exist', 'profile': 'test_profile', 'model-paths': ['models'], 'macro-paths': ['macros'], 'seed-paths': ['seeds'], 'test-paths': ['tests'], 'analysis-paths': ['analyses'], 'docs-paths': ['docs'], 'asset-paths': ['assets'], 'target-path': 'target', 'snapshot-paths': ['snapshots'], 'clean-targets': ['target'], 'log-path': 'path/to/project/logs', 'quoting': {}, 'models': {}, 'on-run-start': [], 'on-run-end': [], 'dispatch': [{'macro_namespace': 'dbt_utils', 'search_order': ['test_project', 'dbt_utils']}], 'seeds': {}, 'snapshots': {}, 'sources': {}, 'data_tests': {}, 'unit_tests': {}, 'metrics': {}, 'semantic-models': {}, 'saved-queries': {}, 'exposures': {}, 'vars': {}, 'require-dbt-version': ['=0.0.0'], 'restrict-access': False, 'dbt-cloud': {}, 'flags': {}, 'query-comment': {'comment': \"\\n{%- set comment_dict = {} -%}\\n{%- do comment_dict.update(\\n    app='dbt',\\n    dbt_version=dbt_version,\\n    profile_name=target.get('profile_name'),\\n    target_name=target.get('target_name'),\\n) -%}\\n{%- if node is not none -%}\\n  {%- do comment_dict.update(\\n    node_id=node.unique_id,\\n  ) -%}\\n{% else %}\\n  {# in the node context, the connection name is the node_id #}\\n  {%- do comment_dict.update(connection_name=connection_name) -%}\\n{%- endif -%}\\n{{ return(tojson(comment_dict)) }}\\n\", 'append': False, 'job-label': False}, 'packages': []}"
         )
 
     def test_get_selector(self, project: Project):
diff --git a/tests/unit/utils/project.py b/tests/unit/utils/project.py
index 2e374b82fac..c8a22143daf 100644
--- a/tests/unit/utils/project.py
+++ b/tests/unit/utils/project.py
@@ -70,6 +70,7 @@ def project(selector_config: SelectorConfig) -> Project:
         project_env_vars={},
         restrict_access=False,
         dbt_cloud={},
+        flags={},
     )
 
 


Code review comments:
diff_hunk: @@ -1,6 +1,6 @@
-git+https://github.com/dbt-labs/dbt-adapters.git@main
-git+https://github.com/dbt-labs/dbt-adapters.git@main#subdirectory=dbt-tests-adapter
-git+https://github.com/dbt-labs/dbt-common.git@main
+git+https://github.com/dbt-labs/dbt-adapters.git@behavior-flags
path: dev-requirements.txt
text: @author:
Reminder to repoint this before merging